How to Be an Outstanding Reviewer for the Journal of General Internal Medicine … and Other Journals

Peer reviewers are essential to maintaining the excellence and relevance of the Journal of General Internal Medicine (JGIM). We salute the many volunteers who consistently provide high-quality peer reviews of manuscripts submitted for publication. As advisors to the authors and Editors, your work allows our journal to maintain high standards and provide a well-respected venue for publication of manuscripts essential to the work of the members of the Society of General Internal Medicine. In this editorial, we hope to increase the number of you who feel empowered to participate as peer reviewers. We briefly summarize what is known about peer reviews and reviewers, make transparent the peer review process at the Journal, and highlight the benefits of participating as a reviewer. In this spirit, we provide specific suggestions about how to become an effective reviewer, comments that we hope will assist current and prospective reviewers in preparing their responses. Additional resources are available elsewhere.¹

WHAT DO WE KNOW ABOUT REVIEWS AND REVIEWERS?

While the peer review process is widely used and provides face validity to the publication process, evidence supporting using peer review to improve the quality of manuscripts is limited. We know that reviews appear to improve communication of the precision of results, improve discussion sections, and temper sometimes overreaching conclusions.² Such improvements are most striking for manuscripts judged as low quality initially.² Not surprisingly, technical editing improves manuscripts,³ abstract quality, readability, and accuracy of references.⁴ Despite these advantages, the quality of peer reviews varies widely,⁵ and efforts to improve peer review by blinding the review process^6–8 or training the reviewers^9,10 by brief workshops or giving feedback does not seem to provide a measurable sustained benefit.¹¹

Better quality reviews have been associated with a number of difficult to modify rater characteristics like being younger, of lower academic rank, from strong academic institutions, having previous research training, additional postgraduate degrees, being known to the editors, and blinded to the identity of the manuscript's authors.^5,12 But so much about the peer review process is still to be elucidated. For example, what is the impact of participating in the peer review process to the reviewer? What is the best way to provide feedback to reviewers? What is the best training for reviewers?.

WHAT IS THE REVIEW PROCESS AT JGIM?

Upon submission of a manuscript, 1 of the 2 Editors forwards manuscripts to a Deputy Editor (see Fig. 1). Manuscripts other than Editorials are rarely accepted without peer review. Usually, they are sent for peer review, or rejected without external review. In the latter case, the manuscript is reviewed by one of the Co-Editors or Deputy Editors who provides feedback to the author to help improve the article before it is sent to another journal. Deputy Editors select reviewers using a database of reviewers maintained by the Journal, or based on their knowledge of experts in the field. The reviewer database includes areas of expertise or interest, numbers of reviews made by each reviewer, the date of each reviewer's last review, and the timeliness of prior reviews. If a reviewer has performed a review, their name is blocked from selection for 3 months to assure that we do not overly burden anyone. We attempt to engage 3 reviewers for each manuscript. Importantly, Deputy Editors are asked to rate the quality of each review they receive using a 6-point Likert scale (1=poor, 6=excellent). The quality of past reviews assists Deputy Editors with the decision to use a given reviewer. It is also used along with the number of reviews and timeliness of returning reviews in JGIM's annual recognition of outstanding reviewers. Reviewers who agree to review the manuscript within a defined timeline receive a blinded copy of the manuscript. Once the reviews are returned to the Journal, the Deputy Editor evaluates the manuscript and makes a decision to reject, accept, or ask the author to revise the manuscript. For many other journals, this decision is made either by the Editor or together by the Editor and the Deputy/Associate Editors. But for JGIM, this decision is delegated to the Deputy Editors who can (and often do) seek input from the Co-Editors or other Deputy Editors. The Deputy Editor highlights important areas of the reviews that should be addressed for manuscripts invited for resubmission. Authors receive a blinded copy of the reviews. At the end of the process, reviewers receive a copy of the decision letter and other reviewers' comments. We acknowledge reviewers by publishing their names yearly, and in 2005, the best reviewers received special recognition in an Editorial.¹³

Figure 1.

JGIM manuscript management process. Cond. accept, conditional acceptance.

WHY SHOULD I BECOME A REVIEWER?

Based on our own experiences, we believe that participating as a peer reviewer has many personal benefits. It motivates one to write, is fun and intellectually satisfying, provides an opportunity to be creative, and demystifies the academic publishing process. Also, being a peer reviewer is a discrete and manageable way to contribute to the generation of higher quality scholarship that shapes our profession. Expert review adds to the integrity of the decision to publish submitted manuscripts and therefore is an explicit way to participate in the self-regulation of our profession.

A peer reviewer for the Journal has the opportunity to shape, clarify, and highlight important aspects of the communication of discovery in General Internal Medicine and to benefit personally from this process. First, reviewers learn about their field as they review. Second, reviewers effectively mentor authors in their field by giving them constructive feedback. Third, by shaping how manuscripts finally appear to readers, reviewers contribute to the quality of research, education, practice, and policy making.

Participating in peer review is evidence of scholarly activity and qualitatively indicates regional or national prominence. The perceived value of peer review activities makes them a criterion for promotion and tenure in most academic centers and of advancement in other organizations. Many reviewers have included letters from the JGIM editors among their external reviews for promotion.

Finally, reviewing is also a very efficient tutorial in writing style (“What are they trying to say?”), research methods (“I never thought of that—what a great idea”), and content area (“Hey I could be doing this kind of work also”)—thus sharpening your skills as clinician, researcher, or administrator.

WHAT MAKES A GOOD REVIEW?

A good review can enhance the quality of the manuscript on many levels. Below are tips for constructing a good review for JGIM. In general, a good review discusses the importance of a research question and its originality, identifies the strengths and weaknesses of the methods, provides specific and constructive suggestions, and comments on the interpretation of results.¹⁴ Of course, writing a good review does not guarantee that either the manuscript's author or editor will agree with your review. But a good review should clarify any points of disagreement. The The Appendix (available online) contains annotated samples of reviewers' comments that illustrate some of these tips. Some reviews are deemed good while others are in need of improvement. For more tips, refer to sources that summarize the responsibilities of reviewers,¹⁵ provide additional suggestions^1,16 and guidelines,^17,18 and provide a validated instrument to measure the quality of a review.^10,14

• Cover major areas and be concise: Make your review 1 to 3 pages in length, and answer the questions outlined in Table 1.

• Prioritize and organize your concerns: Prioritizing helps both the author and the editor to know which major issues must be addressed, and which are minor and not as critical to the publication or revision of a paper. Provide brief statements, number and organize the suggestions, and be clear and direct.

• Be specific and give examples from the manuscript: General statements such as “not a significant contribution,”“clinically unimportant,” and “poorly analyzed” are not very useful without more details. Vague reviews frustrate authors because they are forced to guess what parts of their manuscript need to be improved.

• Suggest corrective actions if possible: Specific suggestions for analysis, writing, formatting, or summarizing the information to enhance the manuscript are helpful. Point to fundamental flaws that are unlikely to be overcome.

• Be respectful of the authors: Scholarship is difficult and criticism is not pleasant to receive. Be fair and critical but make your comments in the tone that you would want to receive.

• Act as the expert: Especially for new reviewers, do not be shy. You were chosen because you have some expertise, either as a reader of the literature or a contributor to the field. Do not hold back. One reason for maintaining the anonymity of reviewers is that we believe it encourages frank feedback.

• Use guidelines or checklists as appropriate: Guidelines are available to assist you in your review and in giving feedback to authors. Published guidelines exist for educational interventions,¹⁹ meta-analysis of randomized,²⁰ or observational studies,²¹ economic evaluations,^22,23 diagnostic tests studies,²⁴ randomized-controlled trials,²⁵ and nonrandomized interventions.^26,27

• Follow the instructions: Once you have agreed to review, stay within the timeline. It is not fair to the authors to delay. Provide scoring requested on the reviewer's report form such as appropriateness to the audience, scientific rigor, and clarity of the writing. Also, indicate or suggest whether the manuscript is appropriate for an editorial. Maintain confidentiality and describe any conflicts of interest.

• Be frank when making confidential comments to the editor: This section allows you to make comments that the author does not see. Unlike the comments returned to the authors, these comments may include your opinions about whether the manuscript is ultimately worth publishing. Comments made to the author are not worth repeating in this section.

• Do not convey different messages to the author and editor: It is confusing and sometimes even hurtful to an author to have a reviewer “be kind” or even positive in his or her comments to the author and harsh (rejecting the manuscript) to the editor. It may create a communication problem that the editor will need to fix. Provide unbridled but accurate, constructive criticism to the author.

Table 1.

Areas to Include in Review

Is the paper a useful original contribution?

Is the paper appropriate for the JGIM audience?

Is the literature review current and does it place the study in appropriate context?

Are there any ethical issues that need to be addressed?

Are the methods and analysis valid and clear?

Are the tables and figures clear? Is there good use of space in the tables?

Are the conclusions valid?

Is the discussion insightful?

Are the limitations discussed in enough detail?

What are the relevance and implications of the findings?

Is the writing clear and concise?

CONCLUSION

Like all peer-reviewed journals, JGIM depends on your participation. If you have not volunteered already, register online at the Journal website to be considered for the JGIM reviewer database. Although the literature on this subject is limited, peer review is the best system we have currently for improving manuscript and journal quality. Peer review needs everyone's participation to be valuable and viable. A well-done review is beneficial to the reviewer, Editor, and author. When preparing your review, we encourage you to keep the concepts outlined above in mind, as well as reviewing the annotated samples of good reviews available in the online version of this Editorial.

Supplementary Material

The following supplementary material is available for this editorial online at www.blackwell-synergy.com

Appendix I

Example of three reviews completed on the same manuscript.

Appendix II

Example of two reviews completed on the same manuscript.