An Intramural Peer-Review Concept to Support Development in Academic Medicine

Abstract

Academic medicine professionals spend their careers striving for promotion and standing in their respective institutions and the global scientific community. Publishing in high-impact journals aids in that pursuit; yet, formal coursework and training rarely emphasize scientific writing, making it difficult to gain the skills necessary to succeed. The authors implemented an intramural peer-review service in the medical school of a preeminent university to offer guidance, resources, and hands-on writing assistance at no cost. This program model bridges a gap in scientific writing instruction, boosts academic productivity, and increases opportunities to publish in higher impact journals.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40670-021-01251-9.

Background

Academic careers drive science and medicine forward through accurate and effective written communication. Publishing in high-impact, well-respected sources is the foundation for promotion and successful academic careers. It is a productivity metric for both the individual and the institution that builds reputations in the global scientific community. Yet, scholarly writing in academia is seldom formally taught, leaving novice researchers and clinicians to find their own way, often floundering through a series of rejections.

Role models provide the primary instruction of discipline-specific writing in medical and graduate schools. An ideal mentor supervises and nurtures the academic growth of trainees, ensuring a command of existing knowledge in their area of expertise, while guiding critical thinking, interpretation of evidence, and communication of new findings. However, the ideal mentor–mentee relationship is hard to come by, be it a mismatch of learning and teaching styles or a lack of time or motivation to instruct the finer points of writing [1–3]. Writing workshops are effective [4–7]; however, they are offered infrequently and require full or multiple day commitments, inconvenient to the arduous schedules of students, residents, fellows, and faculty. Overall, the support network available for writing in an academic environment is often insufficient to foster confidence, direction, and growth.

Peer-review is the gold standard to ensure that only high-quality products that meaningfully contribute to a discipline’s existing knowledge are published. Authors should solicit an impartial perspective that can identify flaws that those too close to the project may overlook. Yet, frequently, only authors listed on a manuscript appraise the work before submission to a journal. At the University of South Florida (USF), we developed and implemented a novel concept to act as an intermediary between the completion of a manuscript and submission to a journal, providing writing support for academics in the biomedical and clinical sciences.

Activity

Proposal

Dr. Richard F. Lockey, a senior faculty member at USF Health’s Morsani College of Medicine, proposed an intramural peer-review process and obtained approval and funding to initiate a pilot program in the Department of Internal Medicine. Dr. Lockey serves as an editor-in-chief and a managing editor overseeing the day-to-day operations of IMpact, An Intramural Review to Support Research and Scientific Publication. The model offers an avenue for students, staff, residents, fellows, and faculty to receive assistance at any stage of a scientific writing project at no cost. Services offered are accessible in two ways:

1. Website1: a repository of writing resources designed to provide authors assistance to write various types of manuscripts.

2. Intramural peer-review: hands-on writing support for authors provided by experienced faculty.

Writing Resources

Writing resources that covered all aspects of scientific writing were amassed and organized to allow website visitors easy navigation to those relevant to their needs. The resources comprised topics on writing specific types of scientific manuscripts, as well as general research and writing methods, from both within and outside the University.

Editorial Board

Volunteer faculty serve on IMpact’s editorial board and are chosen based on their track record of scholarly activity and proficiency in science and medicine. The editorial board boasts a broad range of knowledge and experience sufficient to cover topics submitted for review. The editors are provided with instructions for the review process (Appendix A) and asked to return their reviews within three weeks. Active editors with a competent review history (judged by the editor-in-chief) may remain on the board indefinitely.

Intramural Peer-Review

IMpact’s review process, similar to that done in mainstream, peer-reviewed journals, delivers appropriate feedback to improve works of scientific literature, increasing the chance of having a manuscript accepted for publication in a book or journal of the author’s choice. Briefly, authors submit their work via an online form. The form provides the IMpact staff with the necessary information to appropriately identify, channel, and follow the manuscript during and after review.2 The editor-in-chief and managing editor triage the manuscripts to determine the appropriate course of action based on the level of assistance required. If there are fatal flaws or overall poor content, the author is directed to resources that will help improve the manuscript and is invited to resubmit an improved draft. If a submission will benefit from peer-review, 2–3 editors from outside the authors’ home division are invited to review the manuscript. Once accepted, an editor has three weeks to review and return the manuscript. The editors evaluate the manuscript for its content, structure, clarity, and language and provide detailed feedback on how to improve the manuscript. The submitting author then receives the reviews, accompanied by resources to assist with revisions in light of the recommendations. Such resources include referral to statisticians, assistance finding a target journal, use of citation managers, and basic structural requirements for specific types of manuscripts. Reviews are single-blinded (editors are unknown to the authors) except in cases where an editor has offered to discuss revisions directly with the author. Figure 1 illustrates IMpact’s review process.

Fig. 1.

Flow chart of manuscript handling for intramural peer-review

Promotion and Outreach

As a new service available to scholars at all levels of academia in a large university, several avenues to promote IMpact were pursued. These included:

1. Introducing the service to each department.

2. Attending faculty meetings.

3. Maintaining a presence at USF Health events.

4. Circulating brochures and flyers.

5. Marketing via the University email distribution listservs and social media outlets.

6. Distributing a quarterly e-newsletter highlighting tips and tricks for writing in the sciences.

Metrics

Effectiveness and utilization of IMpact were determined by the following metrics:

1. Website analytics collected monthly.

2. Nature of inquiries made via telephone and email.

3. Type of manuscript submitted for review (abstracts, case reports, literature reviews, research articles, book chapters, other).

4. Position of the submitting author (medical or graduate student, staff, resident, fellow, or faculty).

5. Fate of each manuscript submitted for review, including publication status, name of the publisher, and journal impact factor (if applicable).

6. Survey responses. Submitting authors receive an invitation to complete a brief survey of their experience. The survey (1) identifies the author’s university position, (2) defines their purpose for using IMpact, (3) asks authors to grade IMpact using an A–F grading scale, and (4) provides a section for comments and suggestions for improvement.

Results

IMpact launched in February of 2017. In the subsequent three years, it expanded beyond the department of Internal Medicine to include all departments within the Morsani College of Medicine (20 in total), the Taneja College of Pharmacy, College of Nursing, as well as affiliate institutions, James A. Haley Veteran’s Hospital (Tampa, FL), Johns Hopkins All Children’s Hospital (St. Petersburg, FL), Lehigh Valley Health Network (Allentown, PA), and Moffitt Cancer Center (Tampa, FL). Ninety-five faculty members, across the above-listed institutions, volunteer as editors.

From February 1, 2017, to July 31, 2020, 243 submissions were received for peer-review from authors at all levels of the academic hierarchy, with faculty seeking peer-review services more frequently than residents, fellows, post-doctoral students, and medical students (Fig. 2a). Submissions included research articles (34%), case reports (19%), literature reviews (14%), abstracts (15%), and posters (6%), as well as other scientific articles (11%) (Fig. 2b).

Fig. 2.

Use of IMpact by university position and type of scholarly activity. IMpact is available to all students, staff, residents, fellows, and faculty in the USF Health community and its affiliates. b. A wide variety of scholarly activities are submitted to IMpact for review, including research articles, case reports, literature reviews, abstracts and posters, book chapters, and others^§ (brief communications, editorials, guidelines, instructional material, figure and table assistance, etc.)

The first half of fiscal year (FY)20, beginning July 1, 2019, had nearly triple the number of submissions compared with FY19—a growth rate of 286%. Quarter 3 had a modest 11% increase, and Q4, an 83% reduction in the submission rate compared with Q3 and Q4 of the previous year. The reduction was a direct consequence of the impact of COVID-19 on operations, which should be considered in view of the overall growth rate of IMpact.

As of July 2020, 54% (94/174) of the articles bound for peer-reviewed journals were published in journals that had an average impact factor of 4.22 (range 0.31–24.01) (Table 1).

Table 1.

IMpact metrics

	FY17	FY18	FY19	FY20¶
Editorial board members	68	79	87	95
Submitted manuscript (per FY)	33	54	72	84
Submitted manuscripts (cumulative)	33	87	159	243
% increase from previous year		64%	33%	14%
Overall publication rate to date (%)	54%

^¶Operational disruption due to COVID-19

Discussion

Academic medicine relies on scholarly writing to communicate research done within the University, attract highly qualified trainees and faculty, and improve its standings among peer institutions. Successful universities set themselves apart by providing support services at all levels of academic research. Many universities have writing labs or formal courses for scientific writing,3 but to our knowledge, no medical university employs a formal intramural peer-review process. IMpact fills a gap that exists in the academic training of medical students and clinicians at all levels of the University.

IMpact’s intramural review concept continues to expand, and future plans include employment of a full-service copyeditor to complement the content editing performed by the editors. A trainee peer-editing group is also proposed to provide an opportunity for trainees to appraise the work of their peers and make them more aware of their own scientific writing skills.

Several limitations exist to the evaluation of IMpact: 1) Objectively defining and quantifying “success” is challenging. No baseline exists for peer-reviewed journal rejection rates among our population. Therefore, it cannot be asserted that rejection rates were lowered, nor that authors who used IMpact had their papers published in higher impact journals than would otherwise have been achieved. 2) Faculty editors provide content editing, not copy-editing; yet, many IMpact’s authors require the latter. 3) Many authors lack a basic foundation in writing and research methods, a deficiency that IMpact cannot address.

Despite shortcomings, this intramural peer-review model bridges a gap in scientific writing instruction, boosts academic productivity, and increases opportunities to publish in higher impact journals. The design of the intramural review also fosters cooperation and interdisciplinary collaboration in a large health care university.

Supplementary Information

Below is the link to the electronic supplementary material.

Funding

Joy McCann Culverhouse Division of Allergy/Immunology Endowment.

Declarations

Conflict of Interest

The authors declare that they have no conflict of interest.