How to be a good reviewer: A step‐by‐step guide for approaching peer review of a scientific manuscript

Abstract

Objectives

The peer review process is critical to maintaining quality, reliability, novelty, and innovation in the scientific literature. However, the teaching of scientific peer review is rarely a component of formal scientific or clinical training, and even the most experienced peer reviewers express interest in continuing education. The objective of this review article is to summarize the collective perspectives of experienced journal editors about how to be a good reviewer in a step‐by‐step guide that can serve as a resource for the performance of peer review of a scientific manuscript.

Methods

This is a narrative review.

Results

A review of the history and an overview of the modern‐day peer review process are provided with attention to the role played by the reviewer, including important reasons for involvement in scientific peer review. The general components of a scientific peer review are described, and a model for how to structure a peer review report is provided. These concepts are also summarized in a reviewer checklist that can be used in real‐time to develop and double‐check one's reviewer report before submitting it.

Conclusions

Peer review is a critically important service for maintaining quality in the scientific literature. Peer review of a scientific manuscript and the associated reviewer's report should assess specific details related to the accuracy, validity, novelty, and interpretation of a study's results. We hope that this article will serve as a resource and guide for reviewers of all levels of experience in the performance of peer review of a scientific manuscript.

Although the peer review process is critical to maintaining quality, reliability, novelty, and innovation in the scientific literature, the teaching of scientific peer review is rarely a component of formal scientific or clinical training, and even the most experienced peer reviewers express interest in continuing education. Herein, we describe the history and provide an overview of the modern‐day peer review process with attention to the role played by the reviewer, including important reasons for involvement in scientific peer review. The general components of a scientific peer review are described, and a model for how to structure a peer review report is provided; these concepts are also summarized in a reviewer checklist that can be used in real‐time to develop and double‐check one's reviewer report before submitting it.

1. INTRODUCTION

The process of peer review is foundational for the publication of high‐quality, reliable, and accurate scientific literature. Although its first signs can be traced back to over 2000 years ago, the origin of peer review for scientific manuscripts can be found during a several hundred‐year period between the 15th and 18th centuries, during which the confluence of the invention of the printing press, the adoption of the scientific method, and the subsequent proliferation of publications by scientific societies are believed to have necessitated closer scrutiny and regulation of the material entering broadly distributed scientific texts. ¹ , ² Although this review process was initially carried out by the journal editor, during the 18th century it grew to include selected individuals with expertise in the manuscript topic. ¹ , ³ Modern‐day peer review is sometimes attributed to William Whewell, who in 1831 proposed a standardized practice that the manuscripts submitted to the journal of the Royal Society of London be reviewed by experts who would summarize their review in written reports that would be published with the manuscripts. ⁴

Today, the role of the reviewer serves a greater significance than ever before. It was estimated that in 2020 alone, over 100 million hours (equivalent to over 15,000 years) were spent on peer review of scientific manuscripts, ⁵ with this time estimated to be worth billions of dollars. ⁵ , ⁶ With the exponential growth in scientific publishing, ⁷ , ⁸ the increasing complexity and sub‐specialization of medicine and science, and the value placed on diverse perspectives in science and peer review, ⁹ , ¹⁰ , ¹¹ one editor (or a small group of hand‐selected experts) cannot possibly review all of the manuscripts that are submitted to the typical journal, nor should that archaic model be the case. In the modern era, reviewers therefore not only provide a valuable service to editors but also are the primary agents for ensuring quality, accuracy, and reliability of the studies that are published in the scientific literature; by extension, reviewers directly inform science—and the clinical care that is informed by that science—in our field.

Despite the critical importance of peer review, there is surprisingly little education in the course of scientific or clinical training that is focused on how to perform peer review of a scientific manuscript. ¹² The majority of reviewers report that they have never received formal peer review training, ¹³ and instead have mostly obtained that training informally from reading journal guidelines or getting advice from mentors and colleagues. ¹⁴ To address this need, increasing numbers of peer reviewer training programs are being developed to teach practical and systematic approaches to scientific peer review. ¹⁵ , ¹⁶ , ¹⁷ However, long‐term continuing education is needed, ¹⁸ and there is a strong desire among reviewers of all levels of experience for continued training in scientific peer view. In one study, 77% of reviewers expressed interest in receiving additional training, which reflected 89% of reviewers with less than 5 years of reviewing experience but also notably 75% of those with 6–10 years of reviewing experience and 64% of those with 11–15 years of experience. ¹⁴

The objective of this article is to summarize the peer review process and provide a step‐by‐step approach for how to perform peer review of a scientific manuscript. Our hope is that this article will serve as an informative, practical real‐time reference, and a training/continuing education resource for reviewers of all levels of experience. Although we will largely focus on clinical research given the prevalence of these studies in the otolaryngology literature, many of the principles discussed can be extrapolated for application to basic science studies as well.

2. WHAT HAPPENS DURING THE PEER REVIEW PROCESS?

The scientific peer review process has been broadly described before, ² although the exact steps are often journal‐specific and the details of which can be gleaned from journal web pages. However, an understanding of the expectations of a reviewer and their function is better framed and appreciated in the context of the overall peer review process (Figure 1). Commonly, when a manuscript is submitted to a scientific journal, an initial check of the manuscript is performed by the editorial staff for fundamental requirements (e.g., formatting); a similarity/duplicity check is frequently used at this stage as well to detect possible plagiarism (using readily available online tools ¹⁹ ) with any identified concerns for plagiarism escalated to the Editor‐in‐Chief (EIC) for further evaluation. After a successful initial check, the manuscript is sent to the journal's EIC who then assigns the manuscript to an Associate Editor (AE), a role which could also be fulfilled by the EIC. In some circumstances, such as for a special issue of the journal, a Guest Editor may be invited on an ad hoc basis to serve in the AE role. At each of these steps, the EIC and AE assess the suitability of the manuscript for the journal. If there are no concerns, then the AE will identify potential reviewers and send them invitations to perform the peer review. Every journal typically has a predetermined number of reviewers—most commonly two— that are sought for each manuscript that undergoes peer review. Once reviewer invitations are sent, the AE and EIC monitor the status of those invitations to ensure that enough are accepted, and if not, then send additional invitations to other potential peer reviewers. Once enough reviewers have accepted the invitation, then the completion of the reviews within the journal's required time frame must be monitored by the editors. If reviews are not returned within the required time frame, then the editors must follow up with the reviewers and/or identify new reviewers. Once all of the reviews are completed, the AE will make a recommendation (Table 1)—based on reviewers' comments and the AE's own read of the manuscript—to the EIC, who will make the ultimate decision that will be sent to the authors. Although multiple individuals are directly involved in the peer review process, the reviewer wields the potential for tremendous impact on the efficiency of this process—both from the standpoint of the temporal length of the review process and the energy spent by the editors—through their response to the review invitation and their timely completion of a thorough review.

FIGURE 1.

Flow diagram for an example peer review process.

TABLE 1.

Common recommendations for a decision on a manuscript.

Recommendation	Definition
Accept	The manuscript should be accepted.
Conditional acceptance	The manuscript is acceptable for publication contingent on the performance of minor revisions first. The manuscript does not need to be rereviewed by the reviewer once these revisions are made; the editors can make the determination if the minor revision requests have been adequately addressed.
Minor revision	The manuscript is viewed very favorably but requires minor revisions. The revised manuscript should be rereviewed by the reviewer.
Major revision	The manuscript is viewed favorably but concerns have been identified that require major revisions. The revised manuscript should be rereviewed by the reviewer.
Reject	The manuscript did not meet criteria for publication in the journal nor is it anticipated that revisions could make the manuscript suitable for publication in the journal.
Reject, transfer to another journal	The manuscript meets “reject” criteria, but it may be suitable for another specified “partner journal” associated with this journal or publisher.
Reject without review (direct reject)	The manuscript is clearly out of the scope of the journal (whether due to topic, methodologic rigor, impact, or novelty), or a similarity/duplicity check (which can be performed by editors or editorial staff using readily available online tools 19 ) is highly suggestive of plagiarism or a duplicate submission.

3. BLINDING IN THE PEER REVIEW PROCESS

There are three popular systems of blinding currently employed in the peer review process: (1) single‐blinded, (2) double‐blinded, and (3) open peer review. ²⁰ In a single‐blinded review, the reviewers are aware of the authors' identities whereas the authors are blinded to the reviewers' identities. In a double‐blinded review, the reviewers and the authors are blinded to each other's identities. In open review, the reviewers and the authors are aware of each other's identities, and the reviewers' scientific review comments may ultimately be published with the final accepted manuscript. ²¹ , ²² Although each of these systems has advantages and disadvantages, each system has the potential for introduction of reviewer bias. ²⁰ In the single‐blinded or open review process, however, is it important for the reviewer to avoid allowing their impression of the authors' or their institutions' reputations or any other personal sentiments about the authors to bias the review.

4. WHY TO SERVE AS A PEER REVIEWER

There are several reasons to become involved in peer review. First, it is our collective duty to ensure—to the best of our abilities and expertise—the quality of the science that is in our field's literature. If we—as the scientists and clinicians who represent the key stakeholders in having an informative and reliable scientific literature—do not, then who will? However, there are multiple personal reasons why a reviewer may choose to become involved in peer review. In one study, the top reasons cited by reviewers for getting involved included that scientific peer review (1) serves as a means for the reviewer to be an active participant in their scientific community, (2) is important for reciprocating the peer review that others have performed for the reviewer's own work, (3) is expected of researchers, and (4) helps to improve the reviewer's own writing skills. ¹⁴ Previous study has in fact confirmed that involvement in peer review may improve writing skills more than simply receiving feedback on one's own writing. ²³

Involvement in scientific peer review not only serves as a means for becoming more involved in the scientific literature, but with many journals, it can also be a practical source of obtaining effective continuing medical education (CME) and CME credits for the reviewer. ²⁴ Peer review may also increase the reviewer's visibility within their field and may help with promotions for those who are in academia. Reviewers for many journals are publicly recognized for their service, whereas sustained service—often quantified by the number of high‐quality reviews performed— is one common pathway for an invitation to join a journal's editorial board or become an editor. For many universities, involvement as a peer reviewer may also contribute toward national/international “service” criteria for academic promotion.

5. RECEIVING AND RESPONDING TO AN INVITATION TO REVIEW

Being a good reviewer begins with the decision to accept or decline a peer review invitation. Peer review invitations should be responded to, whether to accept or decline. Any response—including declining the invitation—is better than not responding to the invitation. Because the review process (Figure 1) is dependent on identifying a predetermined number of reviewers and only that number of invitations is typically sent out, when potential peer reviewers do not respond to invitations—even to turn down the invitation—the process is delayed. For many journals, there is a maximum time period that a potential reviewer's invitation will stay active, after which their non‐response will eventually be interpreted as a declination of the invitation. However, the peer review process will be delayed by at least the length of time spent waiting for a potential reviewer's acceptance or declination of the invitation.

When a potential reviewer is deciding whether to accept a peer review invitation, we recommend that several factors be considered (Table 2); details related to these considerations can largely be gleaned from the invitation email, which may include: the title of the article, an abstract summarizing the work, the authors of the article, and the deadline for the review. First, the potential reviewer must consider whether they have the expertise to perform the peer review. Expertise must be possessed in relation to the topic and/or methodologies that are used in the manuscript. The methods and results described in the abstract will often reveal or reflect for the potential reviewer the technical expertise that is necessary to review the manuscript. Second, the potential reviewer must consider whether they have any conflicts of interest related to the manuscript. A conflict of interest is any factor that could compete with or influence a reviewer to deviate from their responsibility to perform a rigorous, objective, and unbiased review. These conflicts of interest may include financial factors (e.g., the reviewer having a positive or competing financial stake in the subject of the manuscript), scientific conflicts (e.g., the reviewer performs research on the topic of manuscript and has a positive or competing interest in the manuscript being published), and personal (e.g., the reviewer has a relationship with the authors of the manuscript, which may include being from the same organization/institution). Finally, the potential reviewer must consider whether they can perform the review before the deadline. Accepting an invitation without considering these factors could prevent the performance of a timely, fair, and rigorous scientific assessment of the manuscript.

TABLE 2.

Factors to consider in deciding whether to accept a reviewer invitation.

Factors to consider in deciding whether to accept a reviewer invitation
Do I have the expertise to perform this review? Basic science For example, experimental techniques Clinical research For example, clinical knowledge, clinical trial design Computational For example, statistics
Do I have any conflicts of interest? Financial Scientific Personal
Can I complete the review by the deadline? Do I have the time despite other work/obligations I have during the timeframe?

6. PREPARATION FOR WRITING A REVIEW

In preparation to write a review, it is imperative to first carefully read through the manuscript. A reviewer must always remember that the authors of a manuscript will have made a significant investment of time into their study and the writing of their manuscript, and therefore deserve the courtesy of the reviewer's focused attention. In one study of 1500 completed peer reviews, 41% of reviews were found to contain incomplete, inaccurate, or unsubstantiated critiques. ²⁵ From the perspectives of the authors, the journal, as well as ethical considerations, this is simply unacceptable. A thorough reading of the manuscript is absolutely necessary to ensure a rigorous and comprehensive evaluation of the manuscript that does not miss important details or limitations of the study. This prevents erroneously making negative judgments about a manuscript based on concern for problems that do not exist or making a positive assessment of a study without consideration for important limitations. Although the ideal approach for reading a manuscript will be reviewer‐dependent, we highlight that in all cases, a thorough reading of the manuscript for overall content, design/methodology/analysis, and writing is necessary. Especially for those who are new to performing peer review, we recommend one possible step‐wise approach as: (1) read through once to get the overall picture and first impressions, and then take at least a 1‐day break to process the content; (2) re‐read the manuscript to focus on the science/methodology/analysis; (3) re‐read the manuscript to focus on the writing.

A second preparatory step before writing a manuscript review is to gain an understanding of the relevant background, including previous studies on the topic. Presumably, a peer reviewer will have knowledge or expertise in the subject of the manuscript, but if not, it behooves the reviewer to read about the topic so that they understand the significance of the topic as well as any associated caveats/subtleties. Moreover, in order to do a proper review, the reviewer must search the scientific literature for other published works on the topic and ensure the manuscript under review is not a duplicate publication or redundant when compared with previously published works. A sufficient literature review typically does not take very long; the major previously published, related studies can usually be identified within a few minutes by searching a major biomedical bibliographic database (such as using PubMed to search MEDLINE and PubMed Central) with pertinent terms from the manuscript title and specified keywords. In our experience, similar studies will usually be among the first articles discovered in the literature search.

7. WRITING AND ORGANIZING A REVIEW

One of the most cited areas in which reviewers of all levels of experience feel they need more training is how to construct a reviewer report. ¹⁴ Being a good reviewer depends on the ability to clearly and collegially communicate a rigorous review of a manuscript to the authors, EIC, and AE. This depends on how the reviewer report is organized and written. We recommend organizing the review into distinct sections that each serve specific functions toward the overall objective of the peer review (Figure 2).

FIGURE 2.

Example layout for how to write and organize a manuscript review.

7.1. Summary paragraph

A peer review report should begin with a brief descriptive summary of the reviewed manuscript, which is often achievable in as little as three to five sentences. This summary should include the objectives and design of the study, the key results of the article, the authors' conclusions/interpretations, and any positive contributions made by the article. The objective of this summary paragraph is to demonstrate to the authors and the editors that the reviewer has not only read but also understands the manuscript from a “big picture” perspective. This summary paragraph adds validity to—and provides a context for—the reviewer's subsequent comments and may be interpreted as a courtesy to the authors as well. Following the summary paragraph, specific comments or critiques of the manuscript may be organized into sections for “major comments” and “minor comments.”

7.2. Major comments

Major comments reflect the most important concerns and critiques regarding a manuscript that may affect the study's validity or impact (Table 3). Comments should ideally be numbered for ease of reading by the authors and editors as well as ease of responding to them by the authors when a revision is invited. Because even the major comments may vary in terms of their overall importance, we recommend listing the most important first, so the most significant concerns are immediately apparent to the authors and editors. A reviewer can think about types of major comments as those relating to “big picture” concerns, and those relating to each major section of the manuscript (Abstract, Introduction, Methods, Results, and Discussion).

TABLE 3.

“Major” comments in a peer review.

Elements of the major comments
Relevance Is this a topic that matters to readers of this journal (e.g., that they will realistically encounter or need to know about for their work)? Is this a topic that will be of interest to the readers of this journal?
Ethical concerns (if applicable) Has the study undergone a regulatory/IRB review? Was informed consent obtained? Was the study designed in an ethical manner relative to the participants?
Introduction section of the manuscript Does it concisely establish the rationale for the study (1–1.5 double‐spaced pages)? Establishes the significance of the topic Establishes the knowledge gap Describes how that knowledge gap will be addressed
Methods section of the manuscript Are all methods (e.g., data collection, interventions, and statistical analysis) clearly and transparently described? Is the description of the methods sufficient to allow this study to be reproduced? Are there concerns related to internal validity? Improper study design Bias Are there concerns related to external validity? Was the study performed in relation to persons/place/time to make the results generalizable?
Results section of the manuscript Have the necessary analyses been performed to test the authors' hypothesis? Could additional analyses be performed to more comprehensively test the hypothesis? Are there secondary analyses pertinent to the study objectives that could be performed to improve the study overall?
Discussion section of the manuscript Have the data been interpreted fairly and in a balanced manner? Have the results been interpreted in the context of existing literature? Are conclusions drawn logical from the data? Have study limitations been comprehensively discussed?

Tips for the major comments
Organize your thoughts and comments with the most important at the top
Number your comments Ease of reading for authors and editors Ease of responding to comments by authors

“Big picture” concerns that should be elaborated in the major comments section may include concerns about the overall impact of the study, the fit of the manuscript within the scope of the journal as well as ethical concerns. One approach to judging the overall impact of a study is to consider the potential of the study to advance the field or influence the clinical care of patients. Assessing whether the manuscript fits the scope of a particular journal should include consideration for relevance to the journal's mission and overall interest to/relevance to the journal's readership. Relevance of the manuscript to the journal's mission and readership necessitates familiarity with the journal to which the manuscript has been submitted. If the reviewer is not familiar with the journal, this information can be gleaned from the journal's front matter or website, and by browsing the content in prior issues of the journal. Ethical concerns may include, if applicable: omission of institutional review board (IRB)  ‐ or other regulatory ‐ approval, or ascertainment of informed consent from study participants. Although rare, ethical concerns may also arise from elements of the study design.

Other major comments can be derived based on an assessment of each section of the manuscript (Abstract, Introduction, Methods, Results, and Discussion). The Abstract of a manuscript is the section that readers first encounter and it serves to provide a brief overview of the study—why it was performed, its most important findings and the significance of those findings. Inaccuracy in those elements of the Abstract could have detrimental effects on the manuscript as a whole. For example, understatement in the Abstract could discourage potential readers from continuing whereas overstatement could misrepresent the true scientific contribution of the study. The reviewer should assess these elements of the Abstract to ensure that the Abstract accurately communicates the most prominent contributions of the study.

The objective of the Introduction section of a manuscript is to establish the rationale for the study—to establish for the reader why the study was performed and why the reader should continue reading the manuscript. ²⁶ , ²⁷ Establishing the rationale for a study may include providing background, establishing the significance of the topic, identifying a specific knowledge gap and explicitly describing how the study serves to address the knowledge gap. In general, this can be concisely performed in no more than 1–1.5 pages of double‐spaced text. In our own experience as reviewers, a sense of confusion may be felt by the reviewer after reading an Introduction that does not achieve its objective. If the reason for performance and the significance of the study are not clear by the end of the Introduction section, then major comments should point to specific elements of the Introduction that are deficient.

The Methods section of a manuscript should provide enough explicit detail to allow the reader to understand how exactly the study was performed and to make the study reproducible. ²⁶ , ²⁷ , ²⁸ To help the reviewer determine whether the Methods section of a manuscript possesses these qualities, we recommend that the reviewer assess the Methods for several explicitly stated elements: the study design (e.g., case series, cohort, case–control, cross‐sectional, and controlled trial), ²⁹ data—what, when (e.g., at what time points), and how (e.g., retrospectively vs. prospectively)—collected, and what (if any) interventions were provided. All statistical analyses—for example, which methods were used or for multivariable models, which variables were included—should be explicitly described. The Methods section of a manuscript should also be scrutinized for internal and external validity.

Internal validity reflects the appropriateness of the methodology to test the study's hypothesis (i.e., to answer the question that the study asks). A study's internal validity therefore translates to whether its results represent the truth. We recommend that the reviewer especially focus on assessing several elements of internal validity: first, it is important to assess whether the study design was appropriate to test the authors' hypothesis. There are various types of study designs which have overlapping but also unique strengths for investigating particular types of questions ²⁹ ; did the authors employ an appropriate study design to test their hypothesis? Methods employed for data collection also have direct bearing on the internal validity of a study—in particular the “what,” “when,” and “how.” For example, did the authors collect the necessary data (“what”) at the correct time point in relation to the disease course or intervention (“when”), using an appropriate data collection methodology (“how”)? Without a doubt, the particular choice of study design and data collection methodology can bias the results of a study to either prove or disprove the study hypothesis and must therefore be scrutinized by the reviewer.

Assessing the presence of bias in general is an important element of assessing the overall internal validity of a study. There are many types of bias that can affect research, and excellent review articles have been written entirely on sources of bias, to which we would refer readers for a comprehensive discussion of the topic. ³⁰ , ³¹ , ³² , ³³ , ³⁴ However, in Table 4, we have specifically summarized common forms of bias that may be found in clinical research (and may be extrapolated for applicability to basic science studies).

TABLE 4.

Commonly encountered categories of bias. ^a

Category	Definition	Common examples	Examples of ways to evaluate for it
Design bias	Error introduced by choosing a study design that is more likely to confirm (or reject) the hypothesis		Did choice of study design artificially lead to confirmation or rejection of the hypothesis? Do chosen outcome measures artificially increase likelihood of accepting or rejecting the hypothesis?
Information bias	Error introduced during data collection	Misclassification bias: error in application of a classification scheme Observer bias: distortion in classifications, usually to confirm the hypothesis Recall/reporting bias: when participants' remembrances or perceptions of an outcome are artifactually influenced by external factors (e.g., by length of recall period, impact of disease status)	Were standardized classification schemes accurately and equally applied to all participants? When classification could be influenced by subjectivity, were “raters” blinded? When an outcome had to be recalled, was the recall period (length of time over which the outcome had to be remembered) appropriate? Could participants' perceptions/remembrances of outcomes be impacted by their current circumstance or disease state?
Selection bias	Error introduced when the study population does not reflect the target population	Non‐random sampling bias: inclusion is based on a trait that is not representative of the overall target population Spectrum bias: study population does not represent the full heterogeneity of the target population (e.g., due to small numbers)	Do the study participants (e.g., their characteristics) represent the target population indicated in the stated study objectives? Have sufficient numbers of participants been included to reflect the heterogeneity of the target population? Could associations detected (or diagnostic test performance) be due to high prevalence of exposure in artificially homogeneous/unrepresentative “case” populations compared to “controls”?
Confounding bias	Presence of an extraneous factor distorts true effect of an exposure (independent variable) on an outcome (dependent variable)	Confounding by group: an exposure is associated with an outcome but exposure levels are due to patients' populations/groups but within groups, there is no association of the exposure with the outcome Confounding by indication: a treatment is associated with an outcome but the indication for the treatment is the true factor associated with the outcome	Could the association (or lack of association) of an exposure (e.g., a clinical characteristic, environmental exposure, treatment received) with an outcome be due to another unaccounted‐for risk factor associated with the outcome (and—either truly or by chance—associated with the measured exposure) in the study population? Comorbidity/clinical characteristics Population characteristics
Analysis bias	Error introduced by performing analysis in a manner that is more likely to confirm (or reject) the hypothesis	Eliminating unfavorable data (e.g., outliers or subgroups) Multiple testing Performance of subgroup analyses inconsistent with original analytic plan/objectives	Were any data excluded from the analyses? Why? Were many different relationships tested? Independent‐dependent variable combinations Different subgroups Were analyses performed in a manner inconsistent with the study objectives?

^a See Refs. 30, 31, 32, 33, 34 for more comprehensive discussion of these examples.

The external validity of a study refers to the overall generalizability of a study's results beyond the authors' investigation. For example, if the authors performed a clinical study, could the authors' results be applied to patients from other patient populations or geographic locales? Could the authors' study subjects be reflective of the same target population in other parts of the world—in other words, would the broad readership of the study be able to reliably apply these results to their own study subjects? Reviewers should consider that the generalizability of study results (or lack thereof) could be related to the study subjects (e.g., patients' cultural or genetic backgrounds, or disease severity), the setting of the study (e.g. community hospital vs. tertiary care hospital), and timing considerations (e.g., where are the patients in their disease course, or what is the sequence of treatments given). A good reviewer will not only assess whether a study was designed appropriately to test the hypothesis (i.e., has internal validity) but also whether the study results will be applicable to readers and patients outside of the authors' institution or geographic region.

The Results section of the manuscript represents the primary contribution of a study and must therefore be scrutinized by the reviewer, with concerns or critiques stated in the major comments section of the review. First and foremost, a reviewer should evaluate whether the results are reported transparently, which serves to aid in assessing the accuracy of the results. If comparisons are being made between different groups, are the characteristics of those groups explicitly described (e.g., if the groups should be comparable in their demographics, are they)? If there is concern that outliers could be driving associations or correlations (e.g., possibly due to a small sample size or indicated by wide confidence intervals around regression or correlation coefficients), are the data visually illustrated (e.g., in the form of dot plots or box‐and‐whisker plots)? Although the accuracy of study results largely depends on the application of analytic methods to a dataset that only the authors have access to, transparency in the description/illustration of the data may aid the reviewer in judging whether analytical methods have been applied accurately. The good reviewer will therefore scrutinize not only the text of the Results section, but also closely review the tables and figures that are included. As reviewers ourselves, we have identified instances of incorrect analyses when the reported results of statistical analyses (e.g., reported associations or diagnostic accuracy of tests) simply do not appear consistent with the description and graphical illustration of the data. Finally, a reviewer should assess the comprehensiveness of the results presented. Have all necessary analyses been performed to test the hypothesis—that is, should other analyses have been performed? Moreover, given the available data, are there any secondary analyses pertinent to the study objectives that could have been performed to improve the study?

The Discussion section of a manuscript (which includes a Conclusions section that may be incorporated into the Discussion or may stand as a distinct section) plays an important role in interpreting the results of a study. Significant questions or concerns related to the Discussion section of a manuscript are therefore included in the major comments section of a review. The reviewer should approach the Discussion section of the manuscript understanding that the objective of this section is to interpret the authors' results, discuss the significance of those results in the context of other studies and the limitations of their study, and ultimately draw conclusions—especially in relation to the primary objectives of the study—based on those results. ²⁶ , ²⁷ When reading the Discussion section of a manuscript, a reviewer should first assess the authors' assessment of their results. Are the authors' interpretations fair and objectively reflective of their results? Or are the results interpreted in a manner that is biased and unfairly skewed toward one interpretation? Are the results interpreted in a manner that is not directly supported by the results? The resulting conclusions that are drawn by the authors should be similarly assessed. Are the authors' conclusions logical based on their results and has their reasoning been explained with a balanced discussion? The reviewer should also assess whether the authors' have discussed the significance of their results—including in the context of previous studies. Where does this study stand—and what novel insights have been provided or what knowledge has been confirmed— in the context of other studies that have been published on this topic? Have the authors accurately acknowledged similar studies and presented a fair and representative discussion of the existing literature on the topic? Finally, the reviewer should also assess whether an accurate, transparent acknowledgement, and discussion of the study's limitations have been included.

As we have described, organization, transparency, completeness, and clarity in reporting are important pertinent details to assess for all (Abstract, Introduction, Methods, Results, and Discussion) sections of a manuscript. If there is any perceived deficiency in reporting in any of these sections, one excellent resource that the reviewer may recommend to the authors is the Enhancing QUAlity and Transparency Of health Research (EQUATOR) network, an international initiative dedicated to improving the reliability of research reporting. ³⁵ The EQUATOR network offers a wide range of resources for writing research, the most well‐known of which are reporting guidelines. The EQUATOR network has collected well‐established reporting guidelines for different categories of studies, such as randomized trials (CONSORT guidelines ³⁶ ), observational studies (STROBE guidelines ³⁷ ), and systematic reviews (PRISMA guidelines ³⁸ ). Although the use of reporting guidelines is not required and there are different acceptable ways of organizing and reporting each section of a manuscript, the use of reporting guidelines is encouraged by many journals and may certainly be offered as a constructive suggestion for manuscripts in need of improving clarity, transparency, completeness, or organization.

7.3. Minor comments

Minor comments in a scientific review are typically focused on issues that do not significantly impact the content, validity, and the overall scientific contribution of the manuscript. Minor comments may include recommendations about the study's keywords and title, or to point out typographical errors and grammatical issues. We emphasize that although we may consider these to be “minor” comments, these elements of a review are still quite essential. Keywords and the article title are of great importance for the discoverability/searchability of the article in search engines such as PubMed although the title of the article must also serve as an accurate reflection of the study and what it shows.

Although typographical or grammar errors may seem like insignificant issues that could potentially be picked up during typesetting by the publisher after acceptance of the manuscript, these problems speak to the larger issue of the manuscript's clarity. Moreover, typographical and grammar errors as well as poor syntax and use of language—when present in sufficient quantities to impact the ability of the manuscript to communicate its scientific contributions—may be better commented on in the major comments. We also remind reviewers that not all authors are fluent in the language of the journal to which they have submitted their manuscript and reviewers—while reasonably expecting clear and proficient language—should be empathetic and understanding. For this reason, we encourage reviewers to feel free to recommend professional writing and text‐editing services when the text of the manuscript suggests the lack of language writing proficiency by the authors.

8. GENERAL TIPS

We also provide some general considerations for the writing of a scientific review. First and foremost, the reviewer must be collegial and constructive. In one study, 12% of reviews were found to have at least one instance of an unprofessional comment toward the author or their work. ²⁵ Reviewers should not make rude comments and should maintain a professional tone in their written review. Avoid casual comments or making jokes; it must be remembered that the comments from a scientific review directly reflect on large investments of time by the authors and may have an impact on the authors' careers—the language of a scientific review should reflect that gravity. Constructive comments—for example, how to address concerns or improve the manuscript—should be made where possible. Moreover, for any comment that could be interpreted as opinion, justifications should be provided because critiques that sound arbitrary may appear biased. These necessities for collegiality and constructive comments are not only as a general professional courtesy to the authors but also because the reviewer represents the journal for which they are reviewing.

Second, the reviewer's comments should not only be specific in any concerns that are expressed, but also specific in describing how those concerns should/could be addressed. For example, a reviewer comment stating that the previous scientific literature has not been sufficiently discussed is neither specific nor does it give the authors (who presumably believed their discussion to be sufficiently inclusive of the previously published scientific literature) any direction for how to possibly address this comment in a subsequent revision.

Third, reviewer comments to the authors should not indicate a decision such as “major revision” or “reject.” A decision on a manuscript is ultimately the responsibility of the EIC, determined based on the comments and recommendations of the reviewers and AE. Reviewer comments to the authors should only serve to objectively provide an assessment of the manuscript.

9. THE RECOMMENDATION TO THE EDITOR

Based on their assessment of a manuscript, the reviewer will make a recommendation to the EIC regarding the final decision on the manuscript. Although the exact decision recommendations that are available to the reviewer may vary from journal to journal, these decision recommendations typically fall into one of three categories: accept, revise, or reject. In Table 1, we have summarized and defined commonly available recommendations that reviewers can make to the EIC for the decision on the manuscript.

10. COMMENTS TO THE EDITOR

Many journals will give reviewers the opportunity to make confidential comments to editors in a separate comment textbox. We encourage reviewers to use these comments as an opportunity to explain the rationale for their decision recommendation. The comments to the editor are not intended for copy/pasting the comments to the authors. Instead, the reviewer can view these comments as an opportunity to provide a very concise summary of what the manuscript is about and a brief description of the major strengths and/or weaknesses of the manuscript, upon which the reviewer is basing their decision recommendation. For the biomedical literature, we recommend that the reviewer specifically state whether they feel that the study could have sufficient impact to (1) advance the field, (2) motivate future studies, or (3) directly impact clinical care of patients. In particular for studies that the reviewer recommends “revision,” it is important to explain to the editors how the study will have an impact in one of these three ways. The reviewer can communicate their overall assessment of the manuscript's significance to the editors as “priority” for the journal (e.g., a study may be referred to as high, medium or low priority).

The comments to the editor may also be used to raise any unsubstantiated concerns that the reviewer may have but which may not be appropriate for expressing in comments to the authors without direct supporting evidence. These could include simple suggestions for additional statistical/analytical review by a reviewer with specific expertise for studies that are heavily dependent on advanced statistical or analytical methods. These comments could also include more serious concerns that the reviewer may have, such as ethical concerns related to the performance of the study or doubts about the content of the manuscripts or integrity of the reported results.

11. CONCLUSION

Although there are different ways of performing a scientific peer review, we advocate for a systematic approach that methodically assesses each important component of a manuscript, to which end we have created a reviewer checklist that may aid reviewers (Figure 3). The overarching approach by reviewers, as they review a manuscript, should be to think of the field (and the role, impact, and contribution of the study), the authors (in communicating specific, collegial, and constructive comments), the journal (in considering the relevance and fit of the study for journal's readership and in the context of journal's mission), and the editors (in clearly explaining a rationale for their recommendations).

FIGURE 3.

Reviewer checklist.

Although involvement in peer review is an important collective duty to our fields of study or clinical practice, we openly and explicitly acknowledge the tremendous service that reviewers provide to their fields and associated scientific journals and the gratitude that we have for their service. ³⁹ , ⁴⁰ , ⁴¹ We hope that this review will serve to provide a framework that can be used as a reference and aid to scientific peer reviewers, new and experienced.

CONFLICT OF INTEREST STATEMENT

Ahmad R. Sedaghat is Associate Editor for Laryngoscope Investigative Otolaryngology, Rhinology and International Forum of Allergy and Rhinology; Manuel Bernal‐Sprekelsen is Editor‐in‐Chief of European Archives of Otorhinolaryngology, receives speaker honorarium from Sanofi Spain, GSK Spain, and, as scientific advisor, from Bionorica, Germany; Wytske J. Fokkens is editor‐in‐chief of Rhinology and Associate Editor for Allergy; Timothy L. Smith is editor‐in‐chief of International Forum of Allergy and Rhinology; Michael G. Stewart is former editor‐in‐chief of Laryngoscope; Romaine F. Johnson is editor‐in‐chief of Laryngoscope Investigative Otolaryngology.

Sedaghat AR, Bernal‐Sprekelsen M, Fokkens WJ, Smith TL, Stewart MG, Johnson RF. How to be a good reviewer: A step‐by‐step guide for approaching peer review of a scientific manuscript. Laryngoscope Investigative Otolaryngology. 2024;9(3):e1266. doi: 10.1002/lio2.1266