Medical Student Comfort with Academic Publishing: A Single-Institution Experience with a Supplemental Interactive Lecture

Abstract

Introduction

Since the United States Medical Licensing Exam (USMLE) Step 1 shifted to a Pass/Fail scoring system, increased emphasis has been placed on research output. Medical students choosing to pursue competitive residencies have contributed to research at a higher rate than ever before. Despite this, many medical students may feel uncomfortable leading the effort to widely communicate the results of a project.

Methods

A quantitative survey–based study was performed to investigate the medical student research landscape at the University of Arizona College of Medicine – Tucson (UACOM-T). Our team surveyed the medical student body on their familiarity and comfort with the publishing process, invited the full student body to attend an optional student-led interactive session on the topic, and surveyed attendees of this lecture to assess for any changes in comfort or attitudes.

Results

A total of 71 medical students completed the pre-session survey, 24 medical students attended the lecture, and 23 medical students completed the post-session survey. Most survey respondents were actively involved in research while in medical school, equal to their involvement prior to matriculation into medical school. Most survey respondents were not comfortable with academic publishing or identifying a predatory journal prior to attending the supplemental educational lecture on academic publishing, which increased their comfort on these topics.

Conclusions

In summary, most study participants were actively involved in research and were at baseline not comfortable with the academic publishing process. Attending a supplemental student-led interactive lecture on academic publishing increased medical student attendees’ comfort in being able to identify a fake or predatory journal and increased their overall comfort in the academic publishing process. Similar interactive lectures can help bridge the gap in medical students’ understanding of topics in academic publishing.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40670-024-02278-4.

Introduction

There has been a significant shift in the landscape UME-GME transition since the USMLE Step 1 has shifted to a Pass/Fail scoring system. As US senior medical students and international medical graduates prepare to apply for residency programs, the traditional factors determining their competitiveness or strength of their applications have now changed. Traditionally, USMLE Step 1 score, USMLE Step 2 score, clinical grades, MSPE, research output, and letters of recommendation have been the key factors used to rank applicants. With USMLE Step 1 changing to a Pass/Fail scoring system in January 2022, that often-emphasized objective measure has been removed from the residency selection process. A recent survey of program directors has found that there is a need for objective measures to separate applicants [1]. With this change, “research output” (quantified as an applicant’s abstracts, posters, presentations, and publications) has become of increased importance and applicants are adjusting to value research as more important to their own applications [1]. Across all medical specialties, applicants who successfully match have a higher number of total research outputs to peer applicants even one decade ago [3]. This shift has been termed “research fever” and is most evident in the most competitive specialties, namely neurosurgery and dermatology [3–5]. A recent study showed that medical students publishing greater than or equal to 3 papers before intern year were significantly more likely to match into a top-25 residency program in neurosurgery [6].

As research output may become an increasingly valued factor in residency selection, medical students will need to gain further training in research to be prepared for the UME-GME transition [2, 5, 7]. Many medical students are not actively taught research, critical thinking, or analytical skills as part of their formal curriculum. Further, medical students may find it challenging to become adequately engaged in research for the shifting incentive of placing higher value on “research output” [3]. As “research” is classically thought of from the basic science lens, there can be some hesitance to become involved in research as a medical student. Research has a very broad definition and can be conducted in a variety of fields beyond the basic sciences (such as public health, clinical science, translational science, social science, health care economics).

Once a research project has been conducted and the focus shifts to disseminating the results as a research output, there can be hesitance on the part of medical students in how to proceed. The process of publishing is murky because the concepts of journal reputation, impact factor, article processing fees, and open access are often poorly understood by medical students and veteran researchers alike. Medical students can fall prey to “predatory publishers” that will accept and publish any work if their “article processing fee” is paid. The term “predatory publishing” was attributed to Jeffrey Beall, a librarian at the University of Colorado [8]. Established in 2010, Beall’s List was an online repository that listed open access journals exhibiting pre-established characteristics common to predatory journals (such as lack of peer review and false operating location) [9]. In a recent survey of medical residents in the United Arab Emirates, with most residents in the sample having at least one publication on their curriculum vitae, 95% of the surveyed trainees were unaware of Beall’s List or other resources to evaluate journal credibility [10]. In January 2017, Beall’s List and its accompanying blog were taken offline, due in part to legal threats from publishers who were listed on the site [9]. Since it was shuttered, alternatives to Beall’s List, including Cabell’s Predatory Reports, have attempted to address the gap in curating a list of predatory journals for academic researchers to reference before choosing to publish their scholarly work [11–14].

Many medical schools have attempted to foster medical student experiences in research. As recently reported by Bauler and Jones [15], the Western Michigan University Homer Stryker M.D. School of Medicine has offered a week-long elective for medical students since 2018 to teach medical students the process of writing and publishing a case report. At UACOM-T and several other medical schools, medical students are required to complete a Scholarly Project, which was designed to help students become more actively engaged in research. As a 4-year longitudinal course, students work with a faculty mentor to define a research question, survey the current literature and scientific evidence, obtain data to answer the question, and make conclusions. The primary research outcome from the Scholarly Project is to present the findings of the research to a diverse audience. As publishing findings in an academic journal is the primary objective for most research projects, medical students at UACOM-T and beyond would benefit from increased familiarity and understanding of the publishing process.

In this report, we describe a student-led educational initiative to increase medical student understanding of and comfort with the academic publishing process through a pre-session survey distributed broadly to the UACOM-T student body, a supplemental interactive lecture, and post-session survey. The interactive lecture incorporated concepts through consultation with research faculty, including basic science researchers, translational researchers, and clinical researchers. After attending this supplemental lecture on academic publishing, medical student attendees reported increased comfort on their ability to identify a predatory journal and their overall comfort in the academic publishing process.

Materials and Methods

Intervention

We began with three main hypotheses: [1] Most medical students at UACOM-T are actively involved in research projects, [2] Most medical students disagree or significantly disagree that research should be an important component of residency selection, and [3] Most medical students are not comfortable with navigating the publication process. To gauge the existence of a potential gap, we set out to distribute a pre-session survey to assess the environment at UACOM-T. To fill the anticipated gap, we planned to host a supplemental interactive lecture on the topic of academic publishing, including assessing journal quality, predatory journals, journal business models, and the review process. Subsequently, we planned to distribute a post-session survey to attendees of our interactive lecture to determine if any aggregate change was appreciable. The pre-session survey, lecture slides, and post-session survey can be reviewed as a supplement to this manuscript (Supplemental Material). Research faculty from basic science, translational research, and clinical research were consulted on their feedback in the development of the surveys and lecture slides at various stages of the process. Overall, the aim was to educate the medical student body at UACOM-T on both the publication process and the role of “research outputs” in residency selection. We submitted our proposed project to the University of Arizona Institutional Review Board (IRB) and were granted exempt status on March 19, 2024 (IRB Submission ID: STUDY00004025). Additionally, approval from UACOM-T curricular leadership was obtained to evaluate the information provided in the supplemental lecture for accuracy and suitability.

Pre-Session Survey

An anonymous online pre-session survey was distributed to the broad UACOM-T student body through individual year of graduation listservs. The survey was developed in Qualtrics (Qualtrics, Provo, UT). The only inclusion criterion for participation in the survey was active enrollment as a medical student at UACOM-T. As such, all medical students of all ages, genders, and ethnicities were eligible. Non-medical students at the University of Arizona or other schools and medical students at other institutions were ineligible for this study, serving as exclusion criteria. The survey was developed by the authors to collect survey population demographics (only medical student year was asked on the survey) and answer the primary hypotheses (as outlined in the “Intervention” section). For determining the sample size, there were an estimated 540 medical students enrolled in M1–M4 years at UACOM-T including dual-degree students. Based on the current estimates for average response rate for email survey-based studies with minimal relationship between investigator and respondents, a reasonable response rate would be between 10 and 30% [16]. In hoping for a 10–15% response rate to the survey, the target sample size was about n = 54–81.

The primary questions on the survey included medical school year of graduation, involvement in research currently and before medical school matriculation, opinion on research as a component of residency selection, familiarity with the term “predatory journal,” comfort in identifying a predatory or fake journal, identifying a reputed journal among a list of four options, comfort in the publishing process, and familiarity with the acronym “APC” (article processing charges) in the context of publishing. The question assessing opinion on research as a component of residency selection measured agreement on a 5-point Likert scale (“strongly disagree,” “somewhat disagree,” “neither agree nor disagree,” “somewhat agree,” and “strongly agree”). The questions assessing comfort in identifying a predatory or fake journal and comfort in the publishing process measured comfort on a 5-point Likert scale (“very uncomfortable” or “extremely uncomfortable,” “somewhat uncomfortable,” “neither comfortable nor uncomfortable,” “somewhat comfortable,” and “very comfortable” or “extremely comfortable”). The question assessing familiarity with the acronym “APC” in the context of academic publishing measured familiarity on a 5-point Likert scale (“definitely not,” “probably not,” “might or might not,” “probably yes,” and “definitely yes”).

Interactive Supplemental Lecture

The student-led supplemental interactive lecture was held approximately 1 week after distribution of the pre-session survey. The entire UACOM-T medical student body was invited to the lecture. The lecture focused on the changes in the UME-GME transition, the academic publishing process, authorship definitions, measures of journal quality (impact factor, academic society backing, speed of review, indexing, journal reputation, and journal business model), predatory publishers, article processing charges (APC), the review process, journal proofing, and tips for medical students along the way. These topics were chosen to focus on aspects medical students would face as they take their own research projects to the “research output” stage. The lecture slides can be viewed as Supplemental Material for this manuscript. Considering the consensus definitions of “predatory journals,” the criteria discussed in the lecture for distinguishing predatory and non-predatory journals were presented as follows: academic society backing, speed of review, indexing, journal reputation, impact factor, journal business model (subscription-based, open access, or hybrid), publication standards, website appearance, journal name, and presence on a “blacklist” of predatory publishers such as Beall’s List or similar resources [8–11, 14]. Following the didactic component of the interactive session, there was a question-and-answer period and an open discussion where third- and fourth-year students reflected on their experiences to advise first- and second-year students in the audience. On the last slide of the lecture, a post-session survey was distributed as a QR code.

Post-Session Survey

An anonymous online post-session survey was distributed to the attendees of the in-person interactive supplemental lecture through a QR code displayed on the final lecture slide. Similarly, the survey was developed in Qualtrics (Qualtrics, Provo, UT). We assumed that 20–40% of medical students who completed the pre-session survey would attend the lecture given the busy nature of medical students, setting an estimated attendance for our lecture to be between 11 and 32 students. Those who did not attend the lecture were ineligible for the post-session survey, serving as exclusion criteria. Assuming 100% of medical students who did attend the in-person lecture would complete the post-session survey, the anticipated sample size for post-survey was n = 11–32. Efforts were made to ensure the same students who the pre-session survey before attending the lecture and completing the post-session survey.

The primary questions on the survey included medical school year of graduation, confirmation of attendance of the lecture, opinion on research as a component of residency selection, familiarity with the term “predatory journal,” comfort in identifying a predatory or fake journal, identifying a reputed journal among a list of four options, comfort in the publishing process, familiarity with the acronym “APC” (article processing charges) in the context of publishing, and assessment of the didactic session. The same Likert scale was used for responses to questions that were on both surveys. For the final question assessing the utility of the session, respondents were asked if they agreed that their comfort in the publishing process increased after attending the session on a 5-point Likert scale (“strongly disagree,” “somewhat disagree,” “neither agree nor disagree,” “somewhat agree,” and “strongly agree”).

Data Analysis

Survey answers were managed using Qualtrics (Qualtrics, Provo, UT). To accurately quantify survey data, survey responses were converted from a 5-point Likert scale to a numerical scale with 1 = strongly negative, 2 = somewhat negative, 3 = neither positive nor negative, 4 = somewhat positive, and 5 = strongly positive. As needed, responses were reverse coded so that the numeric scale was consistent across similar questions. An unpaired t test was used to determine the statistically significant difference between the means for each response comparing pre-session and post-session questions. We conducted statistical analyses in Prism 10.2 (GraphPad Software, La Jolla, CA). The University of Arizona IRB determined this study was exempt from IRB oversight.

Results

Table 1 summarizes the demographic data collected of the respondents to the pre-session and post-session surveys. For the pre-session survey, out of a total of 71 respondents, 20 (28%) were first-year, 14 (20%) were second-year, 14 (20%) were third-year, and 20 (28%) were fourth-year medical students. An additional 3 (4%) respondents were medical students taking additional time to graduate medical school, either for personal or professional reasons, including pursuing a double-degree (MD-PhD, MD-MBA, MD-MPH, etc.). For the post-session survey, out of a total of 23 respondents, the vast majority were first-year medical students. There were 13 (57%) first-year, 2 (9%) second-year, 1 (4%) third-year, 6 (26%) fourth-year, and 1 (4%) “Other” post-session survey medical student respondents. Overall, these data show that first- and fourth-year medical students were represented more in our study sample for both the pre-session and post-session surveys.

Table 1.

Collected survey demographic information

	Pre-session survey	Post-session survey
Variable	Participants (N = 71)	Participants (N = 23)
Medical student year of graduation, no. (%)
MS1	20 (28%)	13 (57%)
MS2	14 (20%)	2 (9%)
MS3	14 (20%)	1 (4%)
MS4	20 (28%)	6 (26%)
Other	3 (4%)	1 (4%)

Comparison of pre-session survey respondents and post-session survey respondents. There were 71 respondents to the pre-session survey and 23 survey respondents to the post-session survey. Breakdowns of these respondents based on academic year in medical school are provided. “Other” represents students taking additional time to graduate medical school for personal or professional reasons, including dual-degree students and those taking additional “research years”

To assess our first hypothesis that most medical students at UACOM-T are actively involved in research projects, we asked medical students to disclose their participation in research projects both actively and before matriculation into medical school on the pre-session survey. Out of 71 respondents, 61 (86%) were actively involved in research while in medical school compared to 62 (87%) respondents who were involved in research before matriculating into medical school. These data provide evidence to suggest that most medical student survey respondents at UACOM-T are involved in research projects at a level similar to their participation before matriculation into medical school.

To assess our second and third hypotheses, we compared pre-session survey responses to post-session survey responses. Table 2 shows these comparisons between pre-session and post-session survey responses. Our second hypothesis was that most medical students disagree or significantly disagree that research should be an important component of residency selection. Based on the survey results, medical student survey respondents at UACOM-T neither agreed nor disagreed in aggregate with this statement, with mean survey response quantification of 3.17 in the pre-session survey sample and 3.61 in the post-session survey sample. The difference between these two was not statistically different when assessed by using an unpaired t test. From these data, we can interpret that medical student survey respondents at UACOM-T in aggregate neither agree nor disagree with the use of research as a metric for residency selection and that their position on this matter did not change after attending an educational lecture on academic publishing.

Table 2.

Comparison of pre-session and post-session survey results

-	Pre-session survey	Post-session survey	-	-
Variable	Participants (N = 71)	Participants (N = 23)	t-ratio for unpaired t-test	p value
Agreement, mean (SD)	-	-	-	-
Do you agree that research should be an important component of residency selection?	3.17 (1.16)	3.61 (1.20)	1.541	0.13
Comfort, mean (SD)	-	-	-	-
How comfortable are you in identifying a predatory or fake journal?	2.46 (1.24)	3.87 (0.97)	5.637	< 0.01*
How comfortable are you with the publishing process (i.e., taking a research topic from idea to published manuscript in a peer-reviewed journal)?	2.56 (1.43)	3.43 (1.04)	3.159	< 0.01*
Survey item	Proportion of “yes” or correct responses N (%) out of N = 71	Proportion of “yes” or correct responses N (%) out of N = 23	-	-
Are you familiar with the term “predatory journals”?	34 (47.8%)	23 (100%)	-	-
Which of the following is a reputed academic journal?	64 (90%)	22 (95.6%)	-	-
Are you familiar with the acronym “APC” in the context of publishing?	7 (9.8%)	23 (100%)	-	-
Utility of the Session, proportion of “somewhat agree” or “strongly agree”	-	-	-	-
I am more comfortable now with the publication process after attending the didactic session compared to before	-	23 (100%)	-	-

Statistical comparison of pre-session and post-session survey results. Where appropriate, p values are provided

^aVariable responses converted from a 5-point Likert scale to a numerical scale with 1 = strongly negative, 2 = somewhat negative, 3 = neither positive nor negative, 4 = somewhat positive, and 5 = strongly positive. “Positive” and “negative” are adjusted appropriately for each variable. For example, to assess agreement with a provided statement, strongly disagree was quantified as 1 point while strongly agree was quantified as 5 points

^bp value compares pre- and post-session means via unpaired t-test. Significant* at p < 0.01

Our third hypothesis was that most medical students are not comfortable navigating the academic publishing process. Based on the survey results, most medical student survey respondents’ comfort in identifying a predatory or fake journal was quantified with a mean survey response quantification of 2.46 in pre-session survey sample and 3.87 in the post-session survey sample. The difference between these two was determined to be statistically significant using an unpaired t test, indicating an increase in comfort from attending the supplemental educational lecture on academic publishing. Comfort in the publishing process as a whole was quantified with a mean of 2.56 in pre-session and 3.43 in post-session, also deemed statistically significant. Based on these data, most medical student survey respondents at UACOM-T were not comfortable with academic publishing and identifying a predatory journal before attending the supplemental educational lecture on academic publishing, which increased their comfort on these topics.

Additional measures compared between the pre-session survey and post-session survey are familiarity with the term “predatory journals,” ability to identify a reputed medical journal, and familiarity with the acronym “APC” in the context of academic publishing. These measures were not compared for statistical significance, but a trend can be gleaned from the data. Approximately 34 out of 71 (47.8%) pre-session survey respondents were familiar with the term “predatory journals” compared to 23 out of 23 (100%) post-session survey respondents who were familiar. Approximately 64 out of 71 (90%) pre-session survey respondents were able to identify a reputed medical journal among a list of four options compared to 22 out of 23 (96%) post-session survey respondents. Approximately 7 out of 71 (9.8%) of pre-session survey respondents were familiar with the term “APC” in the context of academic publishing compared to 23 out of 23 (100%) of post-session survey respondents.

To evaluate the utility of the interactive lecture, we asked post-session survey respondents to answer if they agreed that their comfort in academic publishing increased due to attending the didactic lecture. The results of this post-session survey question are displayed in Fig. 1. The vast majority of post-survey respondents, all of whom attended the lecture, agreed that they were more comfortable with academic publishing after the session compared to before. Approximately 16 out of 23 (70%) strongly agreed that their comfort increased, 6 out of 23 (26%) somewhat agreed, and 1 out of 23 (4%) did not agree or disagree. Overall, these data from Table 1, Table 2, and Fig. 1 suggest that the supplemental lecture was beneficial for medical student study participants and may be beneficial for the broader UACOM-T student body.

Fig. 1.

Utility of student-led interactive lecture. Utility of the interactive lecture as reflected in the post-session survey question: I am more comfortable now with the publication process after attending the didactic session compared to before

Discussion

This report describes the medical student research landscape at UACOM-T, particularly among those who responded to the survey, as well as describes the utility of a student-led supplemental interactive lecture on a variety of topics related to academic publishing. This report delineates our experiences with sending out a survey to the broad student survey (referred to as the “pre-session survey”), developing and hosting the supplemental interactive lecture, and distributing a survey (referred to as the “post-session survey”) to attendees of the lecture in order to assess its utility. The main takeaway points from this report are that the majority of medical student study participants at UACOM-T are actively involved in research, have mixed opinions on the role of research in residency selection, and are not comfortable at baseline in the academic publishing process. Attendance at the student-led supplemental interactive lecture served to increase attendees’ comfort in this process based on their post-session survey responses.

As shown in this report, medical student survey respondents at UACOM-T are actively involved in research at levels similar to their involvement before enrolling in medical school. This points to medical students continuing to prioritize research while in medical school, potentially at the expense of community service or other extracurricular opportunities given that time is a limited resource. The research atmosphere at UACOM-T exemplifies the growing phenomenon of Research Fever in the UME-GME transition in the USA, with medical students seeking to participate in research to bolster their applications to earn positions in residency training programs [3–5]. Commentary by Ranganathan and Vervoort in JAMA Surgery indicates that Research Fever is a growing concern internationally as well [17].

Medical student survey respondents at UACOM-T have mixed opinions on the use of research as a measure for residency selection. Additional medical education research is needed to assess medical students’ opinion on research as a criterion for residency selection. As of now, it is unclear how much of their involvement is due to genuine interest in their chosen research topic versus need to participate in research due to fears of being overtaken by peers with more research on their resume when applying to residency training programs. A broad, multi-institution survey to assess the greater UME-GME transition landscape is needed.

There are no studies to our knowledge beyond this report that detail interventions aimed at increasing medical student education on research ethics and pitfalls. This study fills this gap by delineating a single-institution experience with a student-led supplemental interactive lecture on the topics related to academic publishing, including authorship definitions, measures of journal quality, predatory publishers, article processing charges (APC), the review process, and journal proofing. As part of being a supplemental lecture, this lecture is not part of the primary preclinical or clinical curriculum even though some topics may be covered at various levels of depth throughout. Attendees of the supplemental lecture in aggregate became more confident in their ability to identify a predatory or fake journal and became more confident in their grasp of the academic publishing process as a whole. Further broad, multi-institution survey-based studies are needed to evaluate medical students’ grasp of academic publishing given the importance placed on research in residency selection.

Moving forward, we hope to continue delivering this interactive lecture annually to students at UACOM-T and explore potential opportunities to integrate the concepts of the lecture into the formal UACOM-T curriculum through evidence-based medicine (EBM) instruction, which is a longitudinal thread throughout the preclerkship curriculum. We hope that this intervention serves to increase awareness of the need to educate medical students on these topics. Findings from this report are likely not unique and medical students at peer institutions are likely equally lacking in confidence in the academic publishing process.

The primary limitation of this study is that it is a single-institutional experience with a small sample size. There may be differences in opinion among medical students internally at UACOM-T as well as at peer institutions across the USA. As a survey-based study, there are limitations and biases associated with the nature of the study. Furthermore, the self-reported nature of the survey may introduce bias in that students who are more engaged or interested in research at baseline are more likely to complete the survey and attend the lecture. Our primary pre-session survey was distributed via email, leading to a likelihood of response bias with individuals responding to the survey not fully representing the opinions of non-responders. After attendance of the lecture, there may be a risk of response bias in attendees over-estimating their improved confidence in academic publishing and positive feelings of utility of the lecture. Given these limitations, further study, replication of this study with a larger sample size, and further study with improved follow-up assessments are warranted, particularly if studies can be performed at institutions across the USA.

Conclusions

In summary, most medical students who participated in the study at the University of Arizona College of Medicine – Tucson were actively involved in research and were at baseline not comfortable with the academic publishing process. Attending a supplemental student-led interactive lecture on academic publishing increased medical student attendees’ comfort in being able to identify a fake or predatory journal and increased their overall comfort in the academic publishing process. Educating medical students on academic publishing is particularly important given the rise in research output as a component in residency selection in the UME-GME transition. Similar interactive lectures can help bridge the gap in medical students’ understanding of topics in academic publishing.

Supplementary Information

Below is the link to the electronic supplementary material.

Author Contribution

All authors had access to the data and played a significant role in writing this manuscript.

Data Availability

None of the data was acquired from outside sources. Data can be obtained by contacting the corresponding author.

Declarations

Ethics Approval

This project was certified exempt by the University of Arizona Institutional Review Board (IRB) (IRB Submission ID: STUDY00004025, March 19, 2024).

Conflict of Interest

The authors declare no competing interests.