Peer review: acknowledging its value and recognising the reviewers

THE CURRENT LANDSCAPE

Peer review — the evaluation of work by people of comparable professional standing and competence — is widely regarded as the scientific seal of approval, denoting quality, validity, and importance. It is a crucial component to publishing and the progression of science. Over the last 20 years or so there has been a recognition of the limitations of peer review, and of the need for more research to ensure that the system is fit for purpose, and is optimised. Much of the focus has been on different peer review systems, ranging from double blind, in which the authors’ and reviewers’ names are unknown to one another, to open peer review, in which the identities of both are known to each other. The BJGP has been using open peer review for many years.

Fundamentally, peer review is concerned with four distinct aspects of quality:¹ the assessment of the validity of the methodology, analysis, and conclusions; the originality, veracity, and significance of the findings; the suitability of the article for the journal to which it has been submitted; and the improvement of the quality of the writing and presentation. Peer review is one mechanism used by journal editors to guard against fraud and plagiarism, and other forms of publication misconduct. Editors, as well as many authors, recognise that high-quality, detailed, and constructive comments from peer reviewers have the potential to transform the quality of submitted manuscripts. Peer reviewers take on this important and time-consuming task with little, if anything, in the way of reward, but little attention has been paid to acknowledging, recognising, and rewarding them. This article acknowledges the importance of the work that peer reviewers do, with a particular focus on reviewers for the BJGP, and explores ways to better acknowledge and reward them.

FORMS OF PEER REVIEW AND THEIR RELATIVE MERITS

No method of peer review has been convincingly shown to be superior to others. For instance, a recent study comparing double versus single blind peer review in Plastic and Reconstructive Surgery found that neither review quality nor the recommendation for publication differed between the two groups.² Blinding was in fact rather difficult, and was only successful 66% of the time, because of authors’ self-citation and peer reviewers’ familiarity with the authors’ research topic. Peer reviewing also requires considerable time and effort.

The evidence base suggests that open peer review offers no clear advantage over other peer review systems. A BMJ randomised controlled trial compared single blind with open review³ and found no significant difference in quality or recommendations between them, or in the time taken to do the reviews (2.25 versus 2.20 hours). Authors preferred the open system and thought that it was fairer, as a result of the increased transparency.

Open peer review is more nuanced than the other system, and can potentially take three forms. The first is the one previously described. The second includes publication of the names of the reviewers when the paper is finally published. In the third, the peer reviews themselves are published online alongside the submitted and final versions of the paper. The BMJ has implemented an open peer review system in which the reviewers’ names are published and the reviews are published online.^3–6 It seems their belief is that open peer review is fairer, more accountable, and provides credit for peer reviewers.⁵ Not everyone is convinced about open peer review, which may have the potential to blunt reviewers’ opinions, decrease the number of reviewers willing to take part, and lead to junior reviewers favourably reviewing seniors in their academic area.⁶

PEER REVIEWERS

Why be a peer reviewer? Most journals provide no training, there are almost no tangible rewards, and little, if any, acknowledgement. It is a time-consuming task, with several sources quoting the average time spent on each review as being as much as 6 hours or more.^1,7 However, this figure may reflect the inclusion of social science papers, which can take longer to review. Many reviewers find themselves having to do this task outside their normal working day. Added to this burden is that peer reviews need to be completed in a timely manner, and most reviewers are not reviewing for just one journal, but often several. Some journals, such as the BJGP, limit the number of review requests per year, but a significant burden of peer review can still exist.

A recent large-scale survey found that most reviewers are also authors,⁸ and that is often how they are recruited. They vary considerably in academic experience and often participate in peer review out of a sense of duty or responsibility. Across the global scientific publishing community there is a shortage of peer reviewers, which threatens current practices,⁸ and the survey of Wiley reviewers asked a number of questions regarding rewards for peer reviewers. Reviewers strongly felt that their work was under-acknowledged. Some common themes in suggesting rewards for reviewing included the idea that peer reviewing could be acknowledged as academic output, some acknowledgement in print, the provision of feedback on review quality, access to journal content, continuing professional development (CPD)/continuing medical education (CME) points, and cash payments or payments in kind.⁸ Feedback was wanted most of all.

BJGP PEER REVIEWERS

The BJGP has long been interested in improving the peer review process, and has made several changes over the last few years. These include:

• publishing an annual list of reviewers in the journal;

• publishing a guide to critical appraisal;

• organising formal peer review training sessions;

• running conference workshops on writing and reviewing for the BJGP;

• conducting research;⁷

• setting up, as a result of that research, a structured feedback system on review quality; and

• providing the opportunity to obtain narrative feedback on reviews from the editor.

We recently conducted an online survey of a random selection of our peer reviewers, asking their opinions about peer review, different forms of peer review, the work required to carry out peer review, and their thoughts around rewards. We invited 91 BJGP reviewers to participate and obtained 57 responses (63% response rate). The sample captured a spectrum of experience regarding number of peer reviews performed and number of peer-reviewed publications authored, from one authored publication and fewer than 10 peer reviews, to 500 papers and more than 30 peer reviews. Peer review is not an exclusive activity of the seasoned academic. Peer review takes time — 98% of reviewers spend over an hour, with 42% spending more than 2 hours on each peer review. Less than a quarter have had formal training in peer review, but over half have had some kind of informal training. Our reviewers participate in peer review for a variety of reasons, but the most common reasons are a sense of duty and feeling it is part of their job. Refreshingly, over half state that they perform peer review because they enjoy it.

If the present open peer review system was extended to include publication of the reviewers’ names, 70% said it would not alter the content of their review, and 56 out of the 57 participants stated they would still participate in this form of review. Most would participate in all peer review systems. The least favoured system is open peer review with reviewers’ names published and the reviews published online, although only one-fifth of respondents said that they would be less likely to participate in this form of review.

Just over 42% said they would spend extra time completing a review if their name was published alongside the paper. Over half (53%) would like more credit for peer reviewing. A variety of free-text suggestions included CPD/CME points, a financial fee, publication fee waiver for the journal, formal recognition in some form (name published, a certificate), and protected time in their schedule to review. Publishing a peer reviewer’s name in a paper was seen by 39% as likely to benefit their career and 35% would favour a payment for their work as a peer reviewer.

DISCUSSION

Our findings are generally consistent with the literature — peer reviewing is seen as an enjoyable part of the job, but it is time consuming and many feel more rewards for their work are needed. We found a lower proportion of reviewers who felt they needed more rewards than indicated in the Wiley survey, which may be due to the feedback system we already operate at the BJGP. Another key theme was receptiveness to open peer review, and the view that open peer review with their name published with the paper would be beneficial to their career. Possible adverse effects of using this system could be the ‘blunting’ of opinions and decreased reviewer participation, although our survey suggests that this would not be a problem for the BJGP. Some reviewers might spend more time reviewing, but it is possible that, even if reviewing time increased, this would only be a temporary effect while reviewers became accustomed to the new system. Open peer review with the names of the reviewers published would lead to increased transparency and increased acknowledgement of peer reviewers. It is also possible to foresee a future where reviewers can include their reviews with their personal bibliographies, adding to their academic body of work. Some of the other suggestions from our peer reviewers were CPD/CME points, other forms of certification, and access to BJGP content. Institutional recognition of the scholarly contribution peer review represents and the provision of training in peer review would also be welcome.

We recognise that our peer reviewers are a crucial part of the BJGP, and that there is more that we could do to help. We will continue offering peer review training and detailed feedback, and are committed to doing further research. We are able to provide evidence for appraisals, and in future will be considering other recognition mechanisms such as CPD/CME credits and access to BJGP content, and will be discussing options for including reviewers’ names in the papers we publish.

Provenance

Freely submitted; externally peer reviewed.

Competing interests

The authors have declared no competing interests.