Update on the Manuscript Peer Review Process

Quality peer reviewers of manuscripts are the bedrock of any good journal. They play an essential role in assuring the high caliber and accuracy of the science that a journal publishes. A good peer review is used by the journal to help the editorial team make informed decisions and to improve the quality of published manuscripts.

However, good peer reviewers are not always easy to find. Surprisingly, one study has shown that scientific training and increasing practical experience reviewing papers are not reflected in the quality of subsequent reviews performed by a reviewer (Bruce et al., 2016). Other studies have shown that some peer reviewers are not able to appropriately detect errors (Ghimire et al., 2012, Boutron et al., 2010), improve the completeness of reporting (Hopewell et al., 2014), or decrease the distortion of the study results (i.e. “spin”) (Turner et al., 2008, Melander et al., 2003, Lazarus et al., 2015). So, what’s a journal to do?

In 2014, Toxicologic Pathology expanded its editorial board (EB) from 25 to 52. These members are drawn from all the societies who have selected Toxicologic Pathology as their professional journal. The goal was to have a sufficient number of experienced reviewers available to review manuscripts based on the number of papers received in one year. Another goal was to have EB expertise in as many specialty areas as possible. The Associate Editors (AEs), currently 17 in number, are encouraged to determine which specialty reviews a manuscript might need (e.g., statistics, or subject matter expert for a given organ, species, or technology) and invite appropriate authorities in the field. If an AE cannot find a peer reviewer in a specific specialty area, then someone outside of the EB may be invited; in general, society members are considered before external scientists. An effort was also made to clarify the goal of the Toxicologic Pathology peer review process, provide a definition of a good-quality peer review report, and list the outcomes that should be used in making publication decisions. We address these points by providing clear instructions to reviewers (Supplementary Table I), and in 2015 by adding a point-by-point checklist (Table II) for the peer reviewer to use. Once the initial reviews have been completed and evaluated by the AE, the anonymous remarks and AE’s comments (if any) are sent either to the authors with a recommendation for a minor or major review, or to the Editor-in-Chief (EIC) with a recommendation and rationale for rejection. The reviewers receive an anonymous carbon copy of the decision email that is sent to the authors and therefore have the chance to see other appraisals of the same manuscript. An astute peer reviewer will seize this opportunity to discover where they might have done a better job in performing their review. Such learning facilitates better reviewing in the future, especially when it is time to look at the revised version of the same manuscript.

If a minor revision was recommended by the peer reviewers, then the AE will generally assess the revised manuscript to ensure that all recommended edits were appropriately addressed. If issues were addressed effectively, the AE may send a recommendation to the EIC to accept the manuscript without having the revision undergo a second round of formal peer review. The journal has adopted this option as a means of speeding up publication decisions while ensuring that the quality of accepted papers remains high. If a major revision was recommended, however, the reviewers will have an opportunity to review the revised manuscript to ensure that their proposed edits were made or read the authors response indicating why one or more proposed edits were not done. Outcomes for manuscripts that required a major revision may range from another revision (often minor) to an AE recommendation to the EIC for acceptance or rejection.

The journal has incorporated a two-tier screening process for all initial submissions to avoid wasting time in peer reviewing unsuitable manuscripts. The first screen is done by an assigned Associate Editor and is for general readability in terms of the correct use of English grammar and sentence structure. If needed, a professional medical English editing service is recommended before submission. The second screen is done by the EIC and is designed to ensure that 1) the subject matter is appropriate for the journal, 2) the science is generally appropriate, 3) all figures and tables are submitted at the appropriate size and resolution according to the author guidelines, 4) a cover letter has been submitted, 5) the appropriate manuscript format was used, according to author guidelines, 6) the authorship form is completed, and 7) the author has disclosed any conflicts of interest. The journal mission statement was also revised to ensure clarity of acceptable journal content and to decrease the rejections based on item 1. Flagging of items 1 and 2 by the EIC will warrant sending to a journal committee for review, which is composed of 2–3 AEs who serve on the committee for two months each year. Not sending these problematic submissions out initially for review can help to prevent “reviewer fatigue”. Items 3, 4 and 5 are detected by the Managing Editor, who managing the submission and peer-review process for the journal, and will warrant sending the manuscript back to the author prior to undergoing peer review so that the manuscript format and/or figures/tables can be prepared according to journal standards and that the author has disclosed any conflict of interest or lack thereof. In general, once a reviewer receives an invitation from Toxicologic Pathology to review a manuscript, they can feel confident that the topic is suitable, the science is probably appropriate after a superficial screening, and that the figures and tables are of the acceptable size and resolution. However, a detailed review by the assigned reviewers is still necessary, and the figures and tables must still be closely evaluated by the reviewers to determine if the magnification is appropriate to see the described lesion and the annotation is sufficient to illustrate key points.

So, has the quality of peer reviews improved for our journal? Overall, I’d say, “Yes.” A good review may take the better part of a work day to complete, and a reasonable workload for a peer reviewer (especially an EB member) is 3 to 4 manuscripts a year. It may be hard for some to commit this amount of time—even on an occasional basis. But the time spent on the review is reflected in the quality of the review, and the overall scientific quality of our profession’s flagship journal. Most of our peer reviews are very comprehensive. At first glance, an author may feel somewhat offended and feel that the reviewer was being “picky.” No one wants to be told that their baby isn’t beautiful! However, once the edits are done, most authors are quite grateful that someone took the time to read their paper so comprehensively and provided such a thorough and useful critique. Most authors would agree that the review process has improved the quality of their manuscript.

Our anonymous author surveys began in July, 2016, and are still ongoing. They were designed as a checklist that can take as little as 5 minutes to complete, or slightly longer if comments are provided. The peer review process is one of the items that we assess in this survey. One question is, “Were you satisfied with the review process in terms of timeliness and the quality of the reviews you received?” From a total of 46 responses to date, 41 (89%) replied “Yes” and 5 replied “No.” Another question is, “Did the reviewers’ comments and suggestions help to strengthen your manuscript?” Again, out of 46 responses, 41 replied “Yes” and 5 replied “No.” We have a comment section in the survey and look forward to hearing about specific problems that our authors encounter; the names of the authors and specific manuscript titles are not revealed to ensure the anonymity of the authors and reviewers. In general, we receive far more compliments than complaints, which indicates that our review process is working well. However, this does not mean that we don’t always strive to improve the process, so we encourage continued feedback from our authors.

The peer review process seems to be doing quite well, but the AEs are challenged at times in fulfilling their roles. The AEs are required to collect at least two scientific reviews for each manuscript. Sometimes the AEs will receive a review that suggests rejection (and will state the reasons why) while another reviewer will recommend accept with major, minor or no revisions. In situations such as this, the AE either can consult the EIC and/or can make a determination after carefully reviewing the manuscript and the reviewers’ comments. The difficulty comes when one reviewer provides a substandard review. In such a situation, a replacement reviewer must be found; this requirement will cause a delay in the process, and the author must be notified. Since the reviewer instructions and checklist have been incorporated, these situations have become the exception.

So, how are good peer reviewers identified? It starts at the end of each year when it is time to cycle off a subset of reviewers. The typical time for a reviewer to serve on the EB is 2 years. It is the EIC’s responsibility to identify high-quality reviewers who can replace those that are cycling off. One goal is to keep as many subspecialties available for the AEs to choose from, and this can be determined by identifying colleagues who are known experts in the field or by consulting with the societies to find individuals with appropriate expertise. For example, the Society of Toxicologic Pathology (STP) member website possesses a “demographics” search tool for species and/or technical expertise or systemic pathology areas of expertise as well as a “communities” search tool for memberships in special interest groups or working groups. Either search tool can help identify individuals with suitable capabilities for addressing a particular peer review need. Inviting up-and-coming pathologists is another consideration. Perhaps surprisingly, this relatively inexperienced group nonetheless produces some of the very best peer reviews. Regardless of the years of training and experience or level of expertise, a literature search for first-author publications is always done. If you know how to be a good author, then you will most likely understand how to be a good reviewer. But it is not just knowledge and skill of the review process that determines who will be a good reviewer. One must also be able to commit a fair amount of time to reviewing papers. Before inviting a reviewer, the AEs are asked to look up the number of times that person has done a peer review. If someone has completed a review within the past 3 months, or several within the past year, then the AE is advised not to choose that person. One thing we don’t want is reviewer fatigue. Once a review is completed, the AE is asked to rate both the timeliness and quality of review so that other AEs can see this score and decide if that person should be invited to review again.

One request I have for those in academia or training institutions is to provide a formal comprehensive training program that will enable pathology residents and graduate students to develop the fundamental skills of manuscript peer review. This task might be done by having the trainees review the same manuscripts that the advisors are asked to review (with the permission of the EIC), critiquing the trainee’s reviews and comparing with the advisor’s own review, and/or by having trainees provide reviews of their peers submitted manuscripts. Journal clubs with faculty participation are another great way to discuss and review current literature and develop necessary skills for evaluating publications. There are also many blogs and publications with guidance on how to be a good peer reviewer (Allen, 2013, Hill, 2016). Knowing how to be a good reviewer will teach someone how to be a good author, and vice-versa.

An invitation to review is not a demand. One always can choose not to accept the invitation. Keep in mind, however, that the willingness of you and your colleagues to serve as reviewers is critical to the continued vibrancy and relevance of Toxicologic Pathology—and to the timely publication of your own submissions to our profession’s premier scientific journal.

In summary, the peer review process at Toxicologic Pathology has been honed to permit the consistent production of rapid and thoughtful feedback to authors submitting their work to our journal. The careful choice of EB members is based on subspecialty and publishing history and a commitment to review at least 4 manuscripts per year. We try to assist reviewers by providing explicit instructions and a checklist for the review process, and providing an anonymous copy of the other reviews for the same paper to facilitate sharpening of one’s reviewing skills. We believe that this refinement has both allowed our EB members to develop competent review skills and has permitted our journal to provide more timely feedback to authors. The result of this evolving peer review process is reflected in the continued publication of high-quality manuscripts—reflected in the high impact factor for Toxicologic Pathology relative to other journals with similar scientific content—and high author satisfaction.

Table I.

Reviewer Checklist^*

To ensure that Toxicologic Pathology will publish relevant information of the highest quality, I encourage you to be frank in your assessment of the scientific content and significance of this manuscript. Please indicate the strengths and weaknesses of the paper in sufficient detail to help the author make appropriate revisions, or to understand why the paper may be rejected. Thank you for your willingness to review this paper and provide recommendations regarding its suitability for publication. Below is a checklist that may help you with the review process.

Before you begin the review Let the Associate Editor (AE) know ASAP if you determine that you have a conflict of interest. Let the AE know ASAP if you cannot review and verify the statistics, clinical pathology, or any other area that you feel you cannot adequately evaluate. Additional reviewers with expertise in these areas can then be invited to review. In your evaluation, please indicate the areas you did not review. Let the AE know ASAP if the English grammar is so poor that it hinders a critical review. SAGE performs light copyediting for mistakes in grammar, spelling, punctuation, and editorial style so you don’t have to do this.

Title, keywords and abstract The title, keywords and abstract are key to ensuring readers find the article online through online search engines such as Google. Do these adequately reflect the manuscript as a whole? Is the title too long and cumbersome? Have the most salient keywords been used? These should be specific rather than general words.

The introduction Should present an argument for the conduct of the study. Literature review should provide historical perspective and justification.

Materials and methods Review materials and methods section thoroughly to ensure reproducibility of the data. Confirm that animal care is adequately described. * Verify that the choice of statistical analysis is correct.

Results Should be presented in a clear and concise way. Should not draw any conclusions in this section.

Discussion Presents the explanations of study findings. Study limitations should be identified and discussed. Avoids “spin” by not distorting the study results and presents any plausible alternative explanations for the results. Can extend study findings to applications, etc.

Tables and figures Should improve the readability of the manuscript by presenting the findings already discussed in the manuscript. Should be clear, concise, and accurately reflect the findings in the text. Review all figures for scientific quality (Illustrations Editor will review for size and resolution only). Confirm that all figure legends adequately describe the figures. These should be “stand alone” legends and allow the reader to understand the figure without reading the text. Image magnifications are not required, but if included, should read original objective magnification 2X, 4X, lOX, 20X, 40X, 60X, etc.” Determine if any of the tables or figures can be supplemental data as this can save valuable pages.

Final points Provide feedback to the authors about ways to improve the science and communication of that science. An opinion on the strengths and weaknesses of the manuscript is useful. Do not indicate your opinion on the publishability of the manuscript in your comments to the author. Maintain a professional and respectful tone throughout the review. Offer corrective feedback that improves the scientific merit of the manuscript. Manuscripts with uncorrectable fatal flaws will warrant a shorter review (flawed design, inappropriate methods, etc.). If needed, educate the author(s) ab0ut ways to improve their future research. For comments on minor spelling, grammar or publication style errors, please state that “The manuscript contains numerous spelling and grammatical errors” or “References do not follow journal style guidelines”. Please use the text boxes provided in the reviewer score sheet to record your comments for the author. Alternatively, you may submit your comments to the author in a separate document and attach that file to your review. Ensure that the Word document that you send does not contain any information which may identify you as the author. Do not incorporate your comments directly into the manuscript using the track changes or comment tools.

^*These instructions are available online at https://mc.manuscriptcentral.com/toxpath under “Instructions & Forms”.