A Practical Guide for Group Peer Reviews to Train a New Generation of Reviewers and Scholars

High-quality peer review is a cornerstone of producing accurate, reliable, and actionable scientific literature. With worsening public trust in science and the pervasive spread of misinformation, quality peer review to appropriately vet published literature could not be more important. Fortunately, in pediatric hospital medicine (PHM) we have seen rapid growth of literature to inform evidence-based practice in the past decade, thanks in large part to this journal. Unfortunately, this growth may have outpaced our ability to train and recruit more peer reviewers to meet this escalating demand.

Traditionally, peer review has been a solitary experience of established mid- to senior-career researchers.^{1, 2} For trainees beginning to immerse themselves in the academic world, joining this esteemed cohort can feel intimidating. In this piece, we offer practical guidance for using group peer review (GPR), where a group of reviewers collaborate to systematically critique a manuscript, as an educational tool for fellows to dispel this perception, help fill the reviewer gap, and foster a new generation of scholars.

Peer Review in Fellowship

“Critical appraisal of medical literature” and “peer review” are considered core competencies in PHM fellowship.³ In our fellowship at Children’s Hospital of Philadelphia, we emphasize peer review for a few reasons. First, we believe participating in peer review improves trainees’ abilities to appraise scientific literature and publish their own academic work. Peer review is thought to provide trainees insight into the journal submission process through the reviewer’s lens.⁴ As trainees write their manuscripts, this insight can help them anticipate reviewer feedback to produce stronger first submissions that may be more likely to be accepted.

Second, peer review immerses fellows into the research ecosystem. Fellowships in PHM were established, in part, to expand our field’s scholarly potential, and graduates from PHM fellowships have expressed a desire for more research skills training.⁵ In 2011, a national survey of pediatric hospitalists revealed most participants lacked formal research training and only a quarter had a first-author publication.⁶ A decade later in 2021, a survey of PHM fellowship graduates highlighted that 63% had their fellowship research published, three-quarters continued to conduct research after graduating, and a quarter obtained grant funding.⁶ Peer review can help continue this momentum by providing a wealth of research benefits for trainees, including keeping them updated on the latest developments in their field and facilitating networking among potential future collaborators.⁷ Perhaps most meaningfully, serving as a peer reviewer may help mitigate the impostor syndrome of becoming a researcher and claiming the title of scholar.⁸

Finally, we believe peer review education reduces trainees’ barriers to becoming independent reviewers. Scientific journals are experiencing a shortage of reviewers due to increased volumes of publications and reviewer fatigue.^{1, 2} In our experience, there are some common barriers preventing trainees from becoming peer reviewers to fill this shortage. These include a lack of formal education on peer review throughout medical training, logistical barriers to entering the reviewer network, and the regrettably real sense of self-doubt and lack of self-efficacy in serving as a peer reviewer. Dedicated training in peer review can help reduce these barriers to entry by demystifying the process of peer review, facilitating career development connections to journal editors, and dispelling the myth of who is worthy to peer review.⁹

Of note, providing this education to all fellows, whether or not they are pursuing a career in research, emphasizes that peer review is valuable for all academic clinicians. As opportunities for peer-reviewed scholarship grow across academic tracks, like quality improvement, medical education, and advocacy, training new reviewers in areas outside of traditional clinical and bench research becomes similarly critical.

Group Peer Review

GPR is an approach adopted by experienced peer reviewers as a form of community of practice,¹⁰ and offers promising benefits for trainees. GPR can provide a comfortable learning environment, expose trainees to a diversity of perspectives, and provide an engaging alternative to traditional independent review.^{11, 12} Given the importance of collaboration in medicine, team-based, small-group learning models have gained popularity as effective strategies for medical education. As such, scholars of team-based learning theory have advocated that GPR can improve participants’ reviewing skills and provide journals with higher quality reviews.¹³ Here we outline four steps we use to conduct effective GPRs.

Step 1: Fundamentals

Before trainees can participate in GPR, they should receive formal education on how to conduct a review. Lu et al. published a reference for a systematic process to evaluate manuscripts and write reviews, which we use as an outline for our introductory didactic session.¹⁴ It highlights essential aspects of peer review: assessing the overall purpose of the manuscript, leveraging study-specific reporting guidelines to evaluate quality, and ensuring throughput across sections. For those interested in example synchronous curricula, Way et al. published materials for an interactive workshop to learn the fundamentals of peer review.¹⁵ For asynchronous curricula, Chandran et al. created an interactive online training module on developing peer review skills and reviewer etiquette and a checklist for comprehensive review.¹⁶

Step 2: Journal Selection

Next, we recommend establishing an informal partnership with a journal of interest. We prioritized choosing a journal with content most relevant to our field and that we felt trainees would feel comfortable reviewing for in the future. We also prefer journals with double-blind reviews to reduce the likelihood of conflicts of interest. Most importantly, we recommend contacting the journal editor at least one month in advance to ensure they are amenable to assigning articles for GPRs. This informal partnership has allowed us to share a desired timeline for article assignment and information on trainees’ methodologic expertise. If possible, we prefer manuscripts on core PHM content not aligned with any particular fellow’s scholarly focus to allow for even discussion.

Step 3: Facilitation

Once a partnership is established, we recommend designating a GPR leader. The leader is responsible for coordinating with the journal, facilitating group discussion, and finalizing and submitting the review. This can be a program director, research faculty, or senior fellow with review experience. Leaders should send trainees the article at least one week in advance of meeting to allow for independent review. Leaders can consider assigning each participant a section to focus on, but we opt to have all trainees review the entire article to ensure a comprehensive learning experience. If trainees are unfamiliar with the methodology, consider inviting an expert such as an implementation scientist, biostatistician, or qualitative researcher to participate. If none are available, we recommend the GPR leader communicate their inexperience with the editor for transparency, as the editor may still find value in the trainees’ review or opt to reassign the article.

We allocate one hour for group discussion. Programs could consider having participants send the leader their independent review prior to the GPR to ensure accountability and simplify documentation. During the review, ask the trainees for their overall impression of the manuscript, review it section-by-section as the leader takes notes, and then ask all trainees to provide a decision on the manuscript.

Lastly, have the trainees consider whether the manuscript warrants a commentary and if so, what it should explore. Fontanarosa published guidance on how to write a commentary that effectively highlights an article’s limitations and clinical or policy implications.¹⁷ As all trainees may not be able to co-author an invited commentary based on author number limits, we suggest providing transparency on how co-authors would be chosen in advance. This could be based on seniority, topical experience, or, our preference, contribution to the commentary brainstorm. After the review, the leader should collate the feedback into a final, clean version, share it with the group so they can see what a final product looks like, and submit it to the journal.

Step 4: Promoting Independence

After GPR, provide the fellows with instructions on how they can become independent peer reviewers at their preferred journals. Typically, this might involve emailing the editor to express interest and detail their areas of expertise or having a mentor or program leader email on their behalf. In our fellowship, each fellow follows their own pace in progressing to independent reviews. For those seeking additional experience before establishing independence, we encourage fellows to ask their mentors for opportunities to join them for mentored reviews. When recruiting potential mentors for fellowship, it may be worthwhile to establish an expectation that they conduct at least one mentored review annually with their mentees. This can provide trainees with exposure to manuscripts with more topical relevance to their scholarly interests and allow them to hone their skills with dedicated one-on-one mentorship.

Incorporating GPR into fellowship training is unlikely to fill the reviewer gap alone, and we must continue to explore ways to ensure the publication of high-quality literature. Nevertheless, GPR offers one promising, practical innovation for fellowship by leveraging the known benefits of team-based learning and disrupting the traditional paradigm of peer review. Ultimately, our hope is that opening avenues for our fellows to become independent reviewers helps let the reality set in that they are true emerging leaders in their field so they can reach their full potential as scholars.