JACLP Guide for Manuscript Peer Review: How to Perform a Peer Review and How to Be Responsive to Reviewer Comments

Abstract

Recognizing that very few potential reviewers and authors receive formal training on peer review, we provide guidance on peer reviewing manuscripts and on being responsive to reviewer comments. Peer review provides benefits to all parties involved. Serving as a peer reviewer gives perspective on the editorial process, fosters relationships with journal editors, gives insights on novel research, and provides a means of demonstrating topical expertise. When responding to peer reviewers, authors have the opportunity to strengthen the manuscript, sharpen the message, and address areas of potential misunderstanding.

First, we provide guidance on how to peer review a manuscript. Reviewers should consider the importance of the manuscript, its rigor, and clarity of presentation. Reviewer comments should be as specific as possible. They should also be constructive and respectful in tone. Reviews typically include a list of major comments focused on methodology and interpretation and may also include a list of minor comments that pinpoint specific areas of clarification. Opinions expressed as comments to the editor are confidential.

Second, we provide guidance on being responsive to reviewer comments. Authors are encouraged to approach reviewer comments as a collaboration and to view this exercise as an opportunity to strengthen their work. Response comments should be presented respectfully and systematically. The author’s goal is to signal that they have engaged directly and thoughtfully with each comment. In general, when an author has questions regarding reviewer comments or how to respond, they are invited to contact the editor to review.

Introduction

We as members of the JACLP editorial board have developed a guide on peer reviewing manuscripts (hereafter, simplified as peer review), because we recognize that many invited reviewers have limited experience with the process (e.g., first-time reviewers or early-career psychiatrists) and because several invited reviewers have reached out to us personally asking for guidance. Additionally, prior experience performing peer reviews, often due to the topical expertise of the reviewer, does not imply expertise in peer review. The other side of peer review—responding to reviewer comments—can be equally daunting because authors need to respond to often cryptic comments from anonymous reviewers who influence whether their work is accepted or rejected. Very few receive formal training on either aspect of the peer review process, leaving most to learn through practice (1). Our aims are to offer guidance on peer reviewing manuscripts and on being responsive to reviewer comments, which we hope will be useful for those new to the enterprise and also enrich the skills of experienced reviewers and authors alike.

Peer review is critical to publishing the highest quality scholarship. Reviewers provide an essential service by assessing manuscripts in detail and offering authors suggestions for improving their work. In addition to contributing to the quality of the literature in one’s field, participating in peer review offers benefits to reviewers as well. Serving as a peer reviewer provides greater perspective on the editorial process, facilitates relationships with journal editors, gives insights into novel scholarly work being submitted for publication, and offers a means of demonstrating topical expertise in route to building a national reputation (which is required for academic promotion at many medical schools). These benefits are even more pronounced for those who are contributors to the literature themselves.

Peer reviewing manuscripts

General guidance for peer review

Editors invite peer reviewers based on their published work or demonstrated scholarly interest in a certain topic, or because of their known leadership and perspective in the field. However, a reviewer may be well positioned to provide a thoughtful review based on their background but still feel ill-equipped because they have never had formal training on how to serve as a peer reviewer. This lack of guidance also contributes to a wide diversity of peer reviewing styles across even seasoned reviewers.

We encourage reviewers to approach peer review in the spirit of collaboration (Panel 1). Clear and unambiguous comments are most helpful. The reviewer should identify specific changes that are needed—those that would improve the manuscript either in content or delivery if addressed effectively. Next, even though reviewer identity is anonymous to the authors (i.e., JACLP uses a single-blind peer review process where author identity is known to reviewers, but reviewer identity is unknown to authors), we urge reviewers not to use anonymity as a license for disrespectful comments or indiscretion. Remember that our shared goal is to help patients by advancing the field. Comments should be direct and avoid extraneous commentary such as self-promotion or undue speculation. Rather than simply pointing out deficiencies, reviewers are encouraged to find ways of providing constructive critiques, such as highlighting areas for further development. Reviews may also reference important works the authors may have missed, though reviewers should not unduly recommend one’s own works, which also risks inadvertently disclosing or at least suggesting the reviewer’s identity to authors. Finally, as a courtesy to authors, we request that comments be entered into the webpage textbox (e.g., instead of uploading a PDF with sticky notes), as this makes it easier for authors to respond to each comment. Reviewers may provide notes regarding textual errors if so inclined, but reviewers are not expected or encouraged to provide grammatical proofing except where certain wording or phrasing impairs understanding.

Panel 1: General considerations when performing a peer review.

• Be specific

• Be diplomatic

• Be constructive

• Avoid extraneous commentary

• Reference important works the authors have omitted

• Enter text comments into the webpage textbox

Topics to evaluate as a peer reviewer

When performing a peer review, we recommend considering three categories: importance, rigor, and presentation, as applicable (Panel 2).

Panel 2: Suggested aspects of the manuscript to evaluate as applicable.

1. Importance: Does it contribute something to the field?

   • Is the manuscript sufficiently original or innovative?
   • If a replication, does it add a new perspective?
   • Does the paper fit the standards and scope of the journal?
   • Are the clinical implications discussed?
2. Rigor: Are the methods sound and the results valid?

   • Is the study design appropriate to answer the question?
   • Are the statistical methods appropriate?
   • Is there a relevant EQUATOR network checklist?
   • Was the study conducted ethically?
3. Presentation: Is it written clearly?

   • Is the research question or central theme understandable?
   • Are the methods detailed enough to be replicated?
   • Does the write-up answer the question or make a cogent case?
   • Do the conclusions summarize the results accurately?

The central question for importance is whether the work provides a scholarly contribution to the field. In other words, does it matter? Relevant considerations include whether the work is original or provides a new perspective, whether it appropriately fits within the mission and scope of JACLP, and whether the clinical implications of the work have been addressed.

The second consideration is rigor. Whereas this does not apply to a few article types (e.g., special articles), it does apply to most, including reviews and case reports for which EQUATOR reporting standards are available (2). For primary research articles, the reviewer should consider whether the study design and statistical methods are appropriate to test the hypotheses being evaluated. For all human subject research, ethical study conduct and adherence to reporting standards are mandatory. We also appreciate that reviewers may lack the expertise to comment on every aspect of rigor (e.g., specific statistical methods being used); in such instances, the reviewer is encouraged to provide comments on the manuscript to the extent of their expertise and, under “Comments to Editor,” indicate areas where additional review may be required.

Finally, regarding presentation, it is important to evaluate whether the narrative content is clear, to ensure the style is appropriate (e.g., avoiding casual or nonstandard language), and to consider whether the arguments are being presented logically. Additional considerations for research articles in particular include whether the report is detailed enough to be replicated, whether the primary question or questions have been answered effectively, and whether the conclusions are supported by the results.

The anatomy of peer review comments

We recommend the following structure for peer review comments (Panel 3) and offer a few examples (Supplement). It is common, though not required, to begin peer review comments with orienting remarks regarding the manuscript, including central or important findings. This can be helpful to the authors to demonstrate that the reviewer has understood the work correctly. Such comments may also place the manuscript within the context of current knowledge in the field or highlight salient features that deserve attention.

Panel 3: Structure of a peer review.

• Introductory remarks (optional)

  • Brief remarks summarizing the manuscript and key findings (1 or 2 sentences)
  • Consider any high-level observations for context (a few sentences)
• Major comments

  • May be introduced with the heading “Major comments” or a brief statement indicating the same
  • Focus on methodological or reporting issues
  • Include more important comments earlier
  • Please number comments
• Minor comments

  • May be introduced with the heading “Minor comments” or a brief statement indicating the same
  • These are mainly for clarification
  • Recommend citing page, section, line/paragraph

The primary section of a review includes major comments, concerns, and/or questions about the methods or other fundamental issues that could undermine the work or its interpretation if unaddressed. We recommend that major comments be numbered so that authors can cross-reference their responses when replying to different reviewers (e.g., “Please see response to Reviewer 1, Comment 3.”), with more serious comments listed toward the top of the list to indicate their importance. Whereas we prefer this numbering approach, we recognize that some reviewers prefer to list major comments in the order they occur in the manuscript, divided by section. Although this latter approach is understandable, it can fail to convey to the authors which comments are more important, and it can make succinctly identifying concerns that affect multiple sections of the manuscript more challenging.

Minor comments are included in a subsequent section, because it is important to separate comments of substance that require a re-engagement with methods and reporting from those that are stylistic and readily addressed by minor text changes. Minor comments focus on needed clarification, such as where the phrasing obscures the meaning or improper terminology is being used. Because of their specificity, these comments should direct the authors to the exact line or section where the text in question occurs. Under minor comments, the reviewer may also include suggested changes that they believe may strengthen the paper but are not necessary to be made. Offering helpful suggestions to authors can reflect the reviewer’s investment in the work and signal to authors that their comments are being offered in good faith.

Comments to editor

Reviewers also have the opportunity to provide candid comments to the editor regarding the manuscript under review that will not be seen by the authors. It is not necessary to provide such comments; however, the author may wish to express an opinion regarding the importance of the work or the overall suitability of the manuscript for publication. Typically such comments are brief and written in paragraph form.

Responding to Reviewer Comments

General guidance on being responsive

We provide the following outline to help guide author in responding to reviewer comments (Panel 4). In general, we would encourage authors to approach peer reviewer comments from the perspective of “being responsive” (3) rather than simply offering a reply because not all replies are responsive. Author comments should respond directly to the content of the reviewers’ comments and demonstrate thoughtful engagement with it. When engaging with reviewer comments, the author should keep in mind that the goal of good writing is not to write so that they can be understood but so that they cannot be misunderstood.

Panel 4: Steps to being responsive to reviewer comments.

1. Adjust your mindset

   • Think of peer review as a collaboration
   • Do not take comments personally
   • Comments are generally intended to help with improving the quality of a manuscript
   • Do not argue or be defensive
   • Try to use every comment as an opportunity to strengthen your work

     • If the reviewer misunderstood, readers are also likely to misunderstand
     • If they overlooked something, ask, “Can I do something to make this clearer?”
2. Aim for respectful diplomacy

   • Remember that although their identity is anonymous, yours is not (for JACLP)
   • Be respectful, not ingratiating

     • There is no need to thank the reviewers for every comment
     • Be personable but avoid extraneous commentary
   • Be diplomatic, not evasive
3. Approach responses systematically

   • Start with verbatim comments in order from each reviewer
   • Number major comments (if not already numbered)

     • Split them into sub-items if necessary
     • Cross-reference responses rather than repeating yourself
   • Make clear what changes were made; for example, consider one of the following:

     • Direct to page/line in the tracked-changes version
     • Copy text with changes highlighted, underlined, or in a different color
4. Attempt to make changes to each comment if possible

   • This signals engagement with the reviewer comments
   • There are many ways this can be done, such as:

     • Making a specific change that was requested
     • Conducting an additional analysis
     • Adding a comment regarding unavailability of data
     • Including this as a limitation in the discussion section
     • Highlighting this as a “next step” in research for future study
5. Addressing differences of opinion

   • If you sincerely disagree with the comment,

     • Do so respectfully (see #2 above)
     • Provide a clear rationale
     • Contact the editor to discuss if needed
   • If reviewers offer contrasting perspectives,

     • Consider whether are ways of reconciling these
     • Draw attention to this respectfully (see #2 above)
     • Provide a clear rationale for your perspective
     • Contact the editor to discuss if needed

We recommend organizing reviewer comments in reviewer order (e.g., Reviewer 1, Reviewer 2, and so on), and then numbering each reviewer comment and its associated response. For example, when this method is used, all involved will know that “Reviewer 1, Comment 3” refers to the third comment made by the first reviewer. Responses should also be in a different format (e.g., color, italicized, bold, etc.) so that they clearly stand out from the reviewer comments. The quality of responses to reviewer comments is evaluated by editors and can inform editorial decisions. Remember that it is very likely that reviewers have spent more time reading and considering your manuscript than most readers will. Therefore, if a reviewer finds something unclear, it is very likely that readers will also find the same part of the manuscript unclear or may just pass it over it and miss something the author considered important.

Reflections

We as editors of the JACLP are honored to be a part of such a collaborative and scholarly community, and we are grateful to the peer reviewers who have invested and continue to invest in this journal to ensure the quality of the work we publish. Whereas we recognize that prospective reviewers are not available to accept every invitation to serve as a peer reviewer (nor would we expect anyone to be), we hope that these invitations are understood as our way of acknowledging topical expertise, clinical leadership, and thoughtful engagement. We also hope this guide is useful for those who participate in the peer review process, for this and other journals.