How to be a Good Reviewer for a Scientific Journal

Abstract

In academia, peer-review refers to a fundamental quality control process whereby external experts (reviewers) are invited to provide unbiased critique of a paper (or other submitted material) and advise on suitability for publication. The process must be robust and conducted with honor and integrity and to the highest professional standards. It is not only the responsibility of the authors but also the reviewers to assess the manuscript appropriately and help in improving the quality of the finished article. A good reviewer not only assists the editors and the journal but can also benefit the authors, the wider scientific community and the general readership. In this article, we discuss the salient features of the peer-review process and tips for undertaking peer-review on scientific papers in an effective and professional manner, including opportunities to develop reviewer skills.

Peer-review is a quality control process whereby external experts (reviewers) are invited to objectively critique a paper (or other submitted material) and advise on suitability for publication. Peer-review is fundamental to the integrity of academia and is central to the selection of high-quality papers in academic journals. The success of peer-review hinges on reviewers, many of whom serve in a voluntary capacity to provide expertise and unbiased critique to safeguard the validity and integrity of research. Reviewers are entrusted to provide an overview of submitted work which may have been many years in the making, to comment on the quality and significance of the work and recommend the outcome of the paper. Being a reviewer therefore is a privileged role that should be conducted with honour. In this article, we share our tips for undertaking peer-review on scientific papers in an effective and professional manner, including ways to develop reviewer skills.

The peer review process

To merit publication, a paper must fit the scope of the journal and bring novelty, educational value, or impact on future practice. Most journal submissions will have undergone internal screening by the editorial team to determine suitability for peer-review. As the initiating step, papers are handled by editors who send out invitations for reviews. Responses should be confirmed at the earliest convenience to avoid delays. Reviewers can access the manuscript and are usually asked to (a) provide comments to authors, (b) provide comments to editors, (c) provide an overall recommendation or rating. Once completed, reports are amalgamated by the editorial board to reach a ‘first decision’. If revisions are required, the original reviewers (and occasionally new ones) may be invited back to review the revised manuscript and a ‘response to reviewers’ letter to determine suitability for acceptance.

The key journal metrics influenced by reviewers include:

• (a)Direct: Time to first decision
• (b)
  Indirect:

  • (i)
    Impact factor (or equivalent), i.e., citation potential.
  • (ii)
    Number of downloads.
  • (iii)
    Social media metrics (e.g., Altmetrics/PlumX scores).

Deciding the outcome of a paper

The primary objective of a review is to provide a summative outcome on the manuscript to assist the editorial board with making a decision. Peer-review outcomes may include: (a) accept (rare), (b) minor revisions, (c) major revisions, or (d) reject. This should include a full appraisal of the submitted materials (text, figures and tables, supplementary files, references). The reasons for supporting the decision should be clearly outlined. Comments may be: (a) shared with the authors or (b) confidentially shared with the Editor in Chief. These should be prioritized in order of importance succinctly, e.g., in bullet point form, and courteously.

Deciding factors:

• 1.Novelty—does this paper address knowledge gap or add to the existing body of evidence?
• 2.Is this the right fit for the journal? Although peer-review should be consistent, the summative outcome of peer-review should be personalized according to the journal's standing and impact. Journals with higher impact factors are typically more competitive and incur higher rejection rates.
• 3.Will this inspire or lead to better clinical practice or understanding?
• 4.Fatal limitations—are there critical flaws, e.g. with validity, integrity or impact, that cannot be overcome?

Attributes of a good peer reviewer

The positive qualities of a good peer-reviewer include the following:

• 1.Expertise
• 2.Timeliness
• 3.Good written communication—with authors and editors
• 4.Professionalism
• 5.Empathy and kindness
• 6.Thoroughness
• 7.Intuition and judgement
• 8.Ability to maximize potential of a paper
• 9.Be open to novel and unique ideas

Approach to reviewing a paper

• 1.Responding to the invitation—Consider whether you should take it on. Is the paper within your expertise, and do you have time and the enthusiasm to do this? If you are unfamiliar with the journal, look up the journal and its standing in the field, and scout the quality of similar papers. Avoid reviewing for predatory journals which are often open access and not PubMed indexed. Even if you are unable to review, you should respond promptly to minimize delays and consider recommending alternative reviewers which will assist the editor.
• 2.Preparation – This depends on the type of submission. A full review for original research papers requires approximately 3–4 h on average, whereas case reports or letters will be more straightforward. Reviewing a ‘review article’ requires more attention to the flow of the article, citation of recent articles, and is generally dependent on the presentation of the article and figures/tables. This can be mentally intensive. Allocate time for when you are at your sharpest, ideally with coffee in hand. Reviews can either be printed out (and annotated) or done online. This can be completed all at once or in a staggered manner (go away and think about it). Comments should be typed on to a Word document (with Autosave function), ideally with your review paper side-by-side to make comments as you read. Perform a literature search to ensure you are up-to-date with the latest on the topic of the paper. Look for the duplication of data/papers through a Google/PubMed search.
• 3.Initial impression—Reviewing the high impact areas (Table 1) will usually be sufficient to inform an initial impression/decision. Start with the title and abstract. Is the aim clear? Reflect on how the research question could/should best be answered, and how the study compares. Does the abstract make sense and flow, and are the headline results clearly presented? Do the Figures and Tables present the key findings and do the study justice? Next, focus your attention on priority sections in the full text, including Aims (under Introduction), Methods, Results. Do the study outcomes and methods sufficiently represent the aim, hypothesis or clinical question(s) in an unbiased, valid, and robust manner? Based on the initial impression, is this adequate to inform a formal decision and start writing the report?
• 4.
  Review the full text

  • a.
    Overall—Consider readability: flow (does the story make sense), language (spelling, grammar, and syntax), word count, and overall feel and quality of the manuscript.
  • b.
    Title—Does this capture the essence of the study? Could this be improved to capture keywords, including the study type? This is important for search engine optimization to maximize the paper's visibility on Internet search engines.
  • c.
    Abstract—Has this been optimized to contain accurate facts, headline results? Does it answer the question ‘so what?’. Does the conclusion of the abstract and title match?
  • d.
    Introduction—Does this give a true, up to date and balanced background and set out the need for the study? Are the aims and objectives appropriate?
  • e.
    Methods—Is the study ethical? Are methods (+/− materials) adequately described to enable reproducibility? Is the study design, outcomes, timelines, inclusion and exclusion criteria, and statistical analyses clear to infer validity and generalizability? What are the sources of bias and what steps have been taken to minimize these? Is the study powered to detect a true difference? If there is a registered study protocol, check if this aligns. Have the authors followed standardized reporting guidelines for their type of study? Is an ethics statement included?
  • f.
    Results—Do these flow logically? Are they structured in a readable form? Are they represented in major Figures (and Tables) and do they stand out? Are the statistical tests appropriate? Look in the supplementary files (if available).
  • g.
    Discussion—Do they contain a summary of their key findings? Have they performed an up-to-date literature review and discussed how it adds to the existing literature? Is there any scope for future areas which is appropriately addressed in Discussion? Are the limitations sufficiently presented?
  • h.
    References—Are they recent and relevant? Are there any notable omissions? Are the references presented uniformly?
• 5.Providing feedback—Reflect on the paper, with focus on the high-impact areas. Read and re-read these areas, especially the Title, to ensure this is the best version it can be. Look in the Cover Letter and Supplementary Files in case there are high impact points that have been missed, e.g., important results or figures, what the study adds or how the study changes practice. These are important for dissemination, especially on social media, which can enhance the impact of the work and increase citations.¹ Also consider the flow and ease of comprehension of the article, particularly to non-native English speakers. Consider the merits of the paper and the limitations, in order to deliberate on the paper's outcome. Feedback should be structured below.

Table 1.

High Impact Areas During Peer-Review.

Section	Rationale	Effect
Title	The title unveils the existence of a paper and is important to get right. Embedding keywords into the title helps with article visibility and search engine optimization.	Impact
Abstract	Abstracts provide a summary of the full paper. Ensure that the aims, methods, results, and conclusions are clear. An initial impression can often be made from the abstract alone. The abstract also determines if readers proceed to read the full paper.	Validity Impact
Figures	Figures serve to capture attention and visualize data—these should be relevant, informative, and high-quality. Consider suggestions for enhancing the image content or aesthetics, adding/removing figures, using split figures (dividing one figure into multiple parts), or adding a graphical abstract. These enhance the readability of a paper, social media interest, and citation potential. If needed, figures (and even videos) can also be included in supplementary files.	Validity Impact
Tables	Tables are another form of data visualization. Look for results in text that can be better summarized in table form to minimize word count and improve impact. Tables can be probed to ensure the validity of analyses.	Validity Impact
Introduction	The introduction should include what is already known, what is not known, and why the study exists. The study aim(s) should be clearly laid out.	Validity Impact
Methods	Focus on the study design, outcome(s), and statistical analyses. Are the methods valid for answering the question? Is it appropriately powered? Is there a registered study protocol? What efforts have been made to minimize bias? Is the study generalizable and reproducible? Has the paper followed a reporting guideline? What is the level of evidence generated by this study? The study conduct can be probed to ensure legitimacy.	Validity Integrity
Results	Ensure that results appear credible and are clearly presented. Important results should be in emphasized in figure form.	Validity Impact
Limitations	Avoid being too critical of limitations that have been declared. Look for study weaknesses that could be included.	Integrity

Structure of a good peer review

A good peer-reviewer can give added value to the authors, the editors, the journal, and the general readership (Figure 1). There are 3 components to the review process (Table 2):

• (a)Writing comments to the authors
• (b)Writing confidential comments to the editors
• (c)Overall recommendation

Figure 1.

How to be a good reviewer.

Table 2.

Suggested Template for a Reviewer Report.

To the Editors	To the Authors
Short 1–2 sentence summary (NB editors can see your comments to the authors, so avoid copying and pasting)	Short (one paragraph) summary of study.
Overall impression of the study, what the study adds and how it affects practice.
This paper (has/does not have) novelty and (is/is not) well-written. The study methods, statistical analyses and results (appear/do not appear) valid. The conclusions are/are not supported by the methods and results. I (have/have no) ethical concerns or on the study conduct.	Major concerns/comments. (Focus on high impact areas; prioritize comments; aim to provide added value to enhance the manuscript). Avoid negative/blunt comments. Polite and constructive comments only.)
My recommendation is ________ because_________ (consider deciding factors including: novelty/educational value/impact on future practice/fatal limitations).
(Optional) This manuscript could benefit from: editorial/graphical abstract/professional assistance with data visualization/social media promotion/expert statistical review/plagiarism check.	Minor concerns/comments.

For the comments to authors, consider the following tips:

• •First and foremost, you should write something! There is nothing more useless to editors and to the external peer review process, than a reviewer simply stating that this is an outstanding piece of work and that you have no comments. Nothing is perfect, and we should all strive to make things better by our critique.
• •Always be fair, balanced, polite, and civil. Even if you are going to trash the work, use language that is professional, non-accusatory, does not belittle the authors, and is not sarcastic or cynical. Essentially, be tough but nice. Equally, do not gush too much in your praise of the work even if you think it is the best thing since sliced bread!
• •Treat the paper with respect and review it in a manner that you wish for your paper to be reviewed.
• •Start with the summary of the study and its major findings (without judgement at this stage).
• •Give a comment on the novelty (or lack of) and what it brings to the field. Point out if similar findings have previously been published. You do this to back up your conclusion that the work is perhaps not as novel as the authors claim but express this politely by stating that ‘similar work has recently been reported by X et al, so the novelty of this work is perhaps modest.’
• •Outline your comments as MAJOR and MINOR. These should also be structured and prioritized, so that authors can provide point-by-point responses.
• •MAJOR means either fatal or requiring substantive effort to upgrade to an acceptable scientific level. This includes flawed design, wrong or inadequate controls, wrong statistics leading to wrong interpretation of results, etc.
• •MINOR means things that must be fixed but are not fatal, e.g., confusing charts/tables/figures, language, wrong/old references, data that is missing and could/should be included, organization of sections.
• •If the paper is clearly flawed, you should outline the major flaws and deliver a clear outline of the issues.
• •Do not give away your overall recommendation.
• •Do not spend hours picking out minutiae, such as spelling and grammar mistakes. If these are widespread, simply state that the manuscript would benefit from thorough proofreading or editing.
• •If the paper is clearly suitable for publication, give advice that will improve the impact of the paper. Figures are especially helpful for social media dissemination. Graphical abstracts can also be high-yield and increase the paper's citation potential.²

There is usually a separate section for comments to the editors—this is confidential and should include a brief rationale for your decision or significant concerns. At all costs, please avoid simply copying and pasting your comments to the authors. This is a very important part of the review process, and you must communicate to the editors your confidential views about the work. This may include major concerns, so go for the fatal issues and justify your recommendation. Equally, if you think this work is outstanding, you should explain why. Include any professional concerns with the paper, e.g., ethics, conflicts of interest, plagiarism, etc. Any editorial considerations should be included here, e.g. need for formal statistical review, special issue, value of an accompanying editorial, graphical abstract,² social media promotion,³ controversies that may damage the reputation of the journal, or any unintended consequences in publishing the paper.

According to the editors of one journal, the three factors that determine a high-quality peer-review include⁴:

• •Completeness of the review and the accuracy of assessment of the strengths and limitations
• •Constructiveness of comments
• •Timeliness

Reviewing the revised manuscript

The majority of original articles will either be rejected outright or require revisions. If you are invited to review a revised manuscript, you should accept this opportunity as you will be familiar with the manuscript. Start by studying the point-by-point responses and ensure that these have been addressed satisfactorily. Review the tracked changes to the manuscript to ensure that the reviewers’ comments have been addressed. At this point, avoid subjecting the authors to excessive rounds of revisions as this can not only be frustrating but can also lead to delays with publication, and potential loss of novelty.

Common mistakes in peer-review

Avoid the following mistakes in peer-review:

• •Excessive delays with responses or completion
• •Not being thorough
• •Demanding recommendations that are impossible to remedy
• •Not maintaining confidentiality
• •Duplicating what has been included under limitations
• •Use of discourteous or overly negative language
• •Not providing added value
• •Focus on language versus content
• •Low threshold to accept submissions
• •Failing to justify their decision
• •Lacking professionalism, e.g., self-citations, not declaring conflicts of interest, not respecting intellectual property.
• •Not considering the unintended consequences of a paper
• •Biased by the author's name or institution

Improving your reviewer skills

Good peer-review can be gained through the following:

• (a)Self-reflection: Consider creating a free-to-use Publons (http://www.publons.com) account to maintain an electronic portfolio of reviews over time. These are stored confidentially and can be browsed to aid reflection and development. Users can access their peer-review metrics (e.g. reviews per month; average word count, Altmetrics activity, contributions to different journals) and can also inform you if rejected papers are published in another journal. Reviewer metrics help to quantify your reviewer contributions and can give indicators on your work-life balance. Formal feedback on the quality of your reviews can also be requested from editors enrolled on Publons.
• (b)Other reviewers' feedback: On average, each review is undertaken by 2.2 reviewers.⁵ After submitting your review, you will usually receive the outcome letter containing all reviewers' comments. Compare and contrast your comments to learn from other reviewers. This can be hugely rewarding and eye-opening. Inspiration can also be gained by observing the format, writing style, and tone of others.
• (c)Formal mentorship: For those within their formative phase of being a reviewer, there is ample opportunity to engage in peer review under expert supervision or mentorship. Many reviews are turned down due to lack of time but offer an ideal opportunity for fellows to take part. This can be done locally or through distant mentorship.

Being a reviewer for a scientific journal is an honor and a privilege. The role not only serves the editors and the journal but also benefits the authors, the wider scientific community, and the general readership. In order to be a good reviewer, one must focus on timeliness, completeness, and constructiveness of reviews, whilst maintaining integrity and empathy with their approach. This can be gained with experience, reflective practice by maintaining an electronic portfolio, and with mentorship.

Credit authorship contribution statement

KS, AVK, and EEO made the study concept and design. Compilation and initial drafting by KS. Final editing and critical revision by KS, AVK and EEO. All members approved the final draft.

Conflicts of interest

Dr. Siau, Dr Kulkarni and Prof El-Omar have nothing to disclose.

Funding

None applicable.