Will Paying Reviewers Ease the Peer Review Crisis? I am skeptical

A report by Inside Higher Education presents promising trials of journals that paid reviewers for writing a report. Turnaround times dropped and the quality of reports was high. It is certainly good that journals explore payment of reviewers, but I am skeptical that this can ease the “peer review crisis”, meaning to improve low turnaround times and reduce difficulties in recruiting reviewers in the first place.

As the report states, this is plausible because the compensation likely induces a sense of commitment and responsibility that accelerates the review process. I think the key question is: Does a payment model scale? When I get one invitation for a paid review report, I give it priority, but may also put other reports on hold and may decline other invitations for non-paid reports. What would happen when I get paid for all reports and all create the same sense of responsibility? Would one attach priority to all of them, putting research and teaching commitments second? I find this unlikely, so paid reports at scale may leave us in the same place as we are right now. Or researchers become more selective in accepting invitations. This is probably superior to accepting invites and not delivering a report, but it may also mean that fewer manuscript get reviewed at all, which is not desirable, in my opinion.

In relation with this, I don’t follow this statement in the article: “There could be some percentage of papers that never get reviewed because it's not worth a journal's limited resources to peer review, which puts a quality stamp on the papers that do get reviewed," he said. This seems circular. I think the quality stamp derives from the review reports, so how can one say that a manuscript is of low quality if it is not sent to reviewers in the first place? There may be clear cases for desk rejections, but also many grey cases for which low quality is not obvious and papers that seem promising at first and do not stand closer scrutiny.

Another point is the following: I believe that for papers on certain topics or using certain methods it is more difficult to find reviewers. The “field”, generally speaking, does not value them highly for reasons unrelated to quality. Maybe one would have to pay more for getting such papers reviewed, but even this may not help and may create wrong incentives for accepting peer review invitations in the first place.

There may be other arguments for or against paying reviewers. While the goal of easing current issues with the peer review system is laudable, I think payment of reviewers is not clearly an effective instrument for achieving this goal.