Thinking about Open Peer Reviews

The suggested oath, broken down by principles and items, is quite interesting, with some of the principles having clear value and little overhead, while others that have potential value also add large amounts of possible overhead.  I suggest that these be broken down into two types of open reviews.  Specifically:

Passive review:
Follows all of Principles 1 through 3, and elements x – xii of Principle 4.  Here, the idea of an open review basically is a review that is signed to make sure the process has appropriate integrity.

Active review:
Includes all elements of Passive review, as well as items xiii and xiv of Principle 4:

• xiii) I will check that the data, software code and digital object identifiers are correct, and the models presented are archived, referenced, and accessible
• xiv) I will comment on how well you have achieved transparency, in terms of materials and methodology, data and code access, versioning, algorithms, software parameters and standards, so that your experiments can be repeated independently

The idea of an open review here seems to be that, in addition to ensuring integrity in the review process, there is also extra work being done beyond a standard review, and the person who does such work should be credited for doing so.

I am uncertain about where elements xv and xvi, as written:

• xv) I will encourage deposition with long-term unrestricted access to the data that underpin the published concept, towards transparency and re-use;
• xvi) I will encourage central long-term unrestricted access to any software code and support documentation that underpin the published concept, both for reproducibility of results and software availability

would fit.  These are neither active nor passive, and as written, they don’t match the review function, but are even more active, more a collaboration than a review.  I suggest that they be rephrased as:

• xv) I will check that the data that underpin the published concept are made available in a manner that provides long-term unrestricted access, towards transparency and re-use;
• xvi) I will check that any software code and support documentation that underpin the published concept are made available in a manner that provides long-term unrestricted access, both for reproducibility of results and software availability

so that they could be part of an Active review.

Of course, this specific remedy is just a suggestion, but I the overall point I want to make is that the added work to be done by the reviewer beyond what is now standard needs to be explicitly considered in both the oath itself as well as the description of the oath.