The importance of peer review skills: Value and necessity of training residents to ensure continued scientific excellence

INTRODUCTION

Peer review, specifically referring to the critical appraisal of scientific scholarship, facilitates the advancement of the medical community and is an essential component of maintaining the quality and integrity of science. With the steadily increasing quantity of manuscripts, the lack of sufficient and timely access to quality peer review has become a concern. Several criticisms of accuracy coupled with predictions of impending shortages of peer reviewers could create a “peer review crisis.” ¹ , ² , ³ , ⁴ , ⁵ , ⁶ , ⁷ Many who participate in peer review feel this is a professional responsibility, but it can require a significant time commitment, reportedly an average of 3–8 hour per review, ⁸ , ⁹ without guarantee of a professional benefit. While this can be frustrating, we believe it has become even more important for more clinicians and scientists to participate in the peer review process as research studies proliferate and the public is deluged with medical misinformation, inaccurate research, and predatory journals.

On the bright side, there has been momentum in recent years to improve the valuation of peer review and even to expand this activity to learners. ¹⁰ For example, the Accreditation Council for Graduate Medical Education (ACGME) acknowledges peer review as a faculty scholarly activity and, based on a request from the Evidence Based Healthcare and Implementation Interest Group of the Society for Academic Emergency Medicine, has recently agreed to separate peer review from other activities for reporting purposes. While this may seem trivial, we feel that this indicates the realization that there are many extra educational and professional benefits to performing peer review and is an important first step in improving its valuation. And these benefits do not stop at the faculty level; residents and fellows (hereafter trainees) can also learn and grow by performing peer review. ⁹ , ¹¹ The ACGME agrees and has recently added peer review as a scholarship option in the program requirements for some specialties (including emergency medicine). ¹² In light of this, this paper will review the proposed methods and potential benefits of teaching peer review to trainees.

PEER REVIEW

Proposed methods for training trainees

Assessing the methodologic quality and scientific merit of a potential publication, a high‐quality peer review can only come from reviewers who have a strong understanding of critical appraisal, which is a cornerstone of evidence‐based health care. These appraisal skills, along with becoming a self‐directed lifelong learner, are now taught in virtually all medical schools and residency training programs, although the quality of this training is highly variable. ¹³ , ¹⁴ , ¹⁵ , ¹⁶ In 2010, the majority of emergency medicine training programs did not have a formalized journal club or evidence‐based medicine curriculum. ¹⁵ While evidence‐based medicine within emergency medicine has evolved since that survey, we feel the first step to teaching peer review to trainees is to have a consistent, standardized approach to teaching evidence‐based medicine concepts and critical appraisal. ¹⁷ Once this is established, the trainees can escalate their learning to involve peer review.

Currently, data on how best to successfully teach peer review skills are lacking, ¹⁸ , ¹⁹ , ²⁰ , ²¹ , ²² , ²³ which is not surprising since most peer reviewers feel they are not adequately trained. ²⁴ Some programs have attempted to engage trainees in appraisal through the publication of letters to the editor based on journal club discussions. ²⁵ This approach has the benefit of measurable scholarship but falls short of a complete peer review. We have several suggestions for how GME programs can implement and address the educational need for teaching and providing quality peer reviews (Box 1).

BOX 1. Potential solutions for integrating peer review into residency training.

Didactic teaching

• Introduction to peer reviewing as a didactic during new resident orientation.

• Participate in peer review workshop.

  • In person available at various national meetings.
  • Virtual available through many journal publishers.

Hands‐on teaching

• Use of publicly available manuscript with embedded errors in a journal club format.

• Mentorship approach.

  • One‐on‐one mentoring relationship.
  • Group mentoring—single mentor with several mentees. ³³
  • Mentees would have a graduated increase in responsibility.

Peer review as component of required scholarship

• Quality and quantity must be measurable.

  1. Incorporating “adequate” reviews as fulfilling part of the resident scholarship requirement:
  1. Some have suggested that for faculty, five adequate reviews could be equivalent to one peer‐reviewed short report article ¹¹ ; a similar approach could be used for residents.

  • b
    Given that editor feedback is not often available, “adequate” reviews could be judged by the mentor using common standards.^a

^aSuperchi C, González JA, Solà I, Cobo E, Hren D, Boutron I. Tools used to assess the quality of peer review reports: a methodological systematic review. BMC Med Res Methodol. 2019;19(1):48.

First, sample manuscripts with standardized errors have been used to study peer review and could potentially serve as the basis for development of peer review training curricula. ¹⁸ Second, many journals and publishing companies offer online training courses and certificates, but the rigor of these courses and validity evidence for their methods are relatively limited. ²⁶ An in‐person course is also an option, such as the Peer Review Workshop at the Society for Academic Emergency Medicine's Annual Meeting or as part of the American College of Emergency Physicians Emergency Medicine Basic Research Skills course, among others. Lastly, the third suggested approach for teaching peer review to trainees is a mentorship model whereby trainees “present” their review to a senior physician. ²⁷ Data showing the efficacy of this method, or any other training program, are also limited. ²² , ²⁶ Similarly, a group mentorship model has also been used, where several mentees work with a mentor to perform a review. Academic Emergency Medicine, AEM Education and Training, and the Journal of Graduate Medical Education have been leaders in writing about mentoring others in peer review, ²⁸ , ²⁹ , ³⁰ , ³¹ but not all journal editorial boards have had similar success in convincing training programs to take up this method. ³²

Ideally, regardless of the specific approach, we should teach trainees the principles of peer review on real manuscripts during the publication process and not only on finished products that are typically reviewed in journal clubs. Journal editors are likely to give permission for this type of mentored peer review assuming the manuscript remains confidential. And, while the mentor ultimately gets the credit, both can benefit.

Benefits to training trainees

Teaching trainees to produce peer reviews can have multiple benefits for the trainee, faculty, and also society. The trainee will improve their critical appraisal skills and will gain understanding of the peer review and academic publishing process. They also have the opportunity to hone their constructive feedback skills; providing actionable feedback in a professional, constructive manner is a skill that physicians will be required to implement regardless of their practice setting. Additionally, trainees interested in academic careers will benefit from exposure to the academic writing styles of other scientists and the latest research in their field. ³³ , ³⁴

In addition to these educational benefits, peer review can also fill regulatory requirements. While the chronic devaluation of peer review is a systemic issue beyond the scope of this piece, residency programs could allow mentored peer review to fulfill a portion (or all) of the scholarly activity requirement for residents. And while faculty may not feel incentivized to teach this to trainees, they can also benefit similarly, as both performing and teaching peer review could potentially be considered in promotion decisions, for example, as professional mentorship. Continuing medical education credit could also be offered. For example, the Royal College of Physicians and Surgeons of Canada awards up to 15 continuing professional development (CPD) credits yearly for peer review ³⁵ and many journals award continuing education credits via the Accreditation Council for Continuing Medical Education (ACCME). ³⁶

By including peer review as a quantifiable and mandatory component of accreditation standards, we may very well encourage residencies (including both faculty and trainees) to take up the activity of peer review rather than solely debating about the quality of journal articles in a journal club format. Junior residents continue to need exposure to journal clubs and other methods where they can learn critical appraisal and knowledge translation, but peer review takes it to the next level; it requires trainees to make concrete decisions about quality and validity of articles and concepts and articulate these decisions, as opposed to the more passive reading process encouraged by other educational formats.

CONCLUSIONS

Peer review is an integral piece of the dissemination of science and the process of improving health outcomes through knowledge translation. Training the next generation of peer reviewers is imperative to ensure that clinicians and scientists have the skills to critically appraise medical scholarship. Peer review training should be implemented throughout the full length of residency and fellowship training and appropriately valued and rewarded as academic scholarship in graduate medical education programs.

CONFLICT OF INTEREST STATEMENT

Dan Mayer reports being an unpaid associate editor of WestJEM and JACEP Open medical journals, and received royalties from sale of the book “Essential Evidence‐Based Medicine Second Edition,” Cambridge University Press, 2014. Teresa M. Chan reports receiving honoraria from the Association of American Medical Colleges (AAMC) for her role as an associate editor at Academic Medicine. She also reports her affiliation with several other journals in a volunteer fashion (Journal of Graduate Medical Education, AEM Education & Training, Perspectives on Medical Education). The other authors declare no conflicts of interest.

Mayer D, Eastin C, Kane B, Lee S, Davis J, Chan TM. The importance of peer review skills: Value and necessity of training residents to ensure continued scientific excellence. AEM Educ Train. 2024;8(Suppl. 1):S76–S79. doi: 10.1002/aet2.10940