Opening up BMJ peer review

The BMJ has until now used a closed system of peer review, where the authors do not know who has reviewed their papers. The reviewers do, however, know the names of the authors. Most medical journals use the same system, but it’s based on custom not evidence. Now we plan to let authors know the identity of reviewers. Soon we are likely to open up the whole system so that anybody interested can see the whole process on the world wide web. The change is based on evidence and an ethical argument.

Peer review is at the heart of the scientific process yet was until recently largely unexamined. Now we begin to have a body of evidence on peer review (www.wame.org), and it illustrates many defects. Peer review is slow, expensive, profligate of academic time, highly subjective, prone to bias, easily abused, poor at detecting gross defects, and almost useless for detecting fraud. Evidence to support all these statements can be found in a book by Stephen Lock, my predecessor as editor of the BMJ,¹ three special issues of JAMA,^2–4 and a forthcoming book.⁵ The benefits of peer review are harder to pin down, but it is probably more useful for improving what is eventually published than for sorting the wheat from the chaff.⁶

Those researching peer review have tried to find better methods, and one of the first randomised controlled trials suggested that blinding reviewers to the identity of authors would lead to better opinions.⁷ Two bigger trials—one that included many journals⁸ and one from the BMJ⁹—both failed, however, to find any benefit.¹⁰ This led to the idea that open peer review might be a better option, and we publish today a randomised controlled trial of open peer review conducted at the BMJ.¹¹ It found that open peer review does not lead to higher quality opinions, but nor does it lead to poorer quality ones, so we are introducing open review—for largely ethical reasons.

The arguments for and against open peer review were explored in depth five years ago in Cardiovascular Research.¹²,¹³ Of six editors asked to contribute commentaries all were for more research, none was against open peer review, and three, including Stephen Lock (my predecessor), declared themselves in favour.¹³ Science is progressively moving away from anonymity. Anonymous editorials in scientific journals were common a decade ago; now they look anachronistic.

The primary argument against closed peer review is that it seems wrong for somebody making an important judgment on the work of others to do so in secret. A court with an unidentified judge makes us think immediately of totalitarian states and the world of Franz Kafka. A related argument is, in the words of Drummond Rennie (deputy editor of JAMA), that identifying the reviewer links “privilege and duty, by reminding the reviewer that with power comes responsibility: that the scientist invested with the mantle of the judge cannot be arbitrary in his or her judgment and must be a constructive critic.” All editors have seen curt, abusive, destructive reviews and assumed that the reviewer would not have written in that way if he or she were identifiable. Openness also links accountability with credit. One important defect of closed review is that reviewers don’t receive academic credit. Finally, openness should eliminate some of the worst abuses of peer review, where reviewers—under the cloak of anonymity—steal ideas or procrastinate.

The main argument against open peer review—a sad one—is that junior reviewers will be reluctant to criticise the work of senior researchers for fear of reprisals. This fear is particularly acute for researchers whose livelihoods depend on winning grants. Junior reviewers, those under 40, have time and again been shown to give the best opinions.¹⁴ By moving to open review we may thus be ruling out the best reviewers. We recognise these arguments, but we don’t think that they outweigh the arguments for open review; in particular, BMJ authors seem broadly in favour of open peer review.¹¹ A few reviewers have said that they don’t want to review if they will be identified, and anyone can decline to review a particular paper. Nevertheless, we hope our small move will contribute to a broader culture change so that junior researchers cease to fear reprisals from senior ones.

From this week, for all new papers that we review, the BMJ will identify to authors the names of those who have reviewed their papers, including the names of our in house editorial and statistical advisers. But we expect to go further, researching as we go. Soon we will probably start to list reviewers at the end of articles. Then we may move to a system where authors and readers can watch the peer review system on the world wide web as it happens and contribute their comments. Peer review will become increasingly a scientific discourse rather than a summary judgment. Through such openness we will hope to show that peer review by journals does add value to the scientific process and that we will thus have a place in an electronic world where authors can potentially go straight to readers.

Papers p 23, Education and debate p 44