Nationwide Analysis of Medical Student Publications upon Entry into Residency

Abstract

Research participation has been increasingly emphasized in undergraduate medical education, but limited data are available to help students formulate realistic and attainable goals for scholarly productivity. This study provides an objective, all-specialty, nationally representative estimate of PubMed-indexed publications among the 2022 cohort of new interns in the USA, representing their scholarly productivity during medical school. Only 39% of interns included in the analysis had any publications during medical school, and mean number of publications (1.4 ± 3.9) was well below the mean self-reported total of abstracts, presentations, and publications attributed to the same cohort based on residency application data (7.9).

Introduction

Research is an important element of undergraduate medical education, and the recent change to pass-fail scoring of the United States Medical Licensing Examination (USMLE) Step 1 and Comprehensive Osteopathic Medical Licensing Examination of the United States (COMLEX-USA) Level 1 exams has led to predictions that research will become more important for residency matching [1]. As such, medical schools are challenged to provide appropriate guidance about how much research experience students should attain. Available sources of benchmark data on medical student research are subject to significant limitations. For example, the Association of American Medical Colleges (AAMC) tracks self-reported research participation via the Graduation Questionnaire, but does not include data on accepted articles [2]. Meanwhile, the National Residency Matching Program (NRMP) tracks the number of publications based on applicant self-report, but this number also includes other scholarly works, such as conference abstracts, non-peer reviewed articles, and articles that were submitted but not yet published [3].

Some studies have attempted to objectively measure the number of medical student publications, but only for single specialties (e.g., averaging 1.7, 2.0, and 5.5 in otolaryngology, urology, and neurosurgery, respectively) [4–6]. These analyses have found far lower numbers of PubMed-indexed publications than the volume of scholarly work self-reported to the NRMP. To provide all-specialty benchmark data, this study used a national provider database to identify intern physicians in all medical specialties and determined the distribution of PubMed-indexed publications among this cohort. The secondary aim was to compare PubMed-indexed publications by physician gender and degree.

Methods

This study was approved with a waiver of consent by the local Institutional Review Board. Incoming interns in the entering class of 2022 were identified using the July 2022 version of the Centers for Medicare and Medicaid Services (CMS) National Plan and Provider Enumeration System (CMS NPPES) database, which contains information on healthcare providers in the USA. The search strategy and its validation are detailed in Appendix 1. Inactive registration records, and physicians whose last name was not unique within the sample, were excluded from the analysis. The database was also queried for degree (MD, DO, PhD) and gender. Specialty data were not consistently entered for new interns in this database.

Each intern’s number of publications was determined using a PubMed search for the combination of last and first name. Results published between 1 January 2019 and 31 December 2022 were included, while results published before 2019 were excluded, as these likely represented research conducted before entering medical school. Following prior research [7], publications from July to December of 2022 (after entry into residency) were credited towards the number of medical school publications, recognizing the potential for delays in the publication process. Publications were also stratified based on whether the intern was the first author.

Continuous data were summarized as medians with interquartile ranges (IQR) and means with standard deviations (SD), and categorical data were summarized as counts with percentages. The number of publications and a dichotomized measure of having any publications were compared by physician gender and degree using rank-sum tests and chi-square tests. The mean number of publications was compared to the NRMP mean using a one-sample t-test. Data analysis was completed using Stata/SE 16.1 (StataCorp, LP), and p < 0.05 was considered statistically significant.

Results

The NPPES database included 23,552 interns with an enumeration date within the qualifying period. Eight cases with a deactivated NPI number and 11,555 interns with a non-unique last name were excluded. Among the remaining sample (N = 11,989; 50%/50% female/male; 78%/22% MD/DO; 2% PhD), a PubMed search on 2 January 2023 identified 15,521 unique articles. In the overall sample, 39% of interns had at least one PubMed-indexed publication (median 0; IQR: 0, 1), and 17% had at least one first-authored publication. The mean number of total publications (1.4 ± 3.9) was significantly lower than the mean number of abstracts, presentations, and publications among matched applicants in the 2022 cycle (7.9, one-sample t-test, p < 0.001).

Table 1 compares of publication data by gender and degree, showing that men, MDs, and physicians with PhDs had more total publications than women, DOs, and physicians without PhDs, respectively. Results were similar when comparing first-authored publications (Table 2). A manual Web search was performed to determine the specialties of the 100 interns with the highest number of publications, finding that most interns in this group were in surgery or surgical subspecialties (n = 44), while internal medicine was the single most common specialty (n = 22; detailed results are shown in Appendix 2). Due to the limited sample size of this manual review, it may not fully represent differences among specialties in rates of high scholarly productivity.

Table 1.

Comparison of total attributed publications by intern characteristics (N = 11,989)

Characteristica	Number of interns	Number (%) with any publicationb	Median (interquartile range) of number of publicationsc	Mean (standard deviation) of number of publications
Gender
Women	6031	2196 (36%)	0 (0, 1)	1.1 (2.7)
Men	5958	2490 (42%)	0 (0, 2)	1.7 (4.9)
Medical degree
MD	9353	4064 (43%)	0 (0, 2)	1.6 (4.2)
DO	2636	622 (24%)	0 (0, 0)	0.6 (2.7)
PhD degree
No	11,747	4506 (38%)	0 (0, 1)	1.3 (3.8)
Yes	242	180 (74%)	3 (0, 6)	5.0 (7.4)

^aAll group differences are statistically significant at p < 0.001

^bGroups compared using chi-square tests

^cGroups compared using rank-sum tests

Table 2.

Comparison of attributed first-authored publications by intern characteristics (N = 11,989)

Characteristica	Number of interns	Number (%) with any first-authored publicationb	Median (interquartile range) of number of first-authored publicationsc	Mean (standard deviation) of number of first-authored publications
Gender
Women	6031	925 (15%)	0 (0, 0)	0.3 (1.0)
Men	5958	1119 (19%)	0 (0, 0)	0.5 (1.7)
Medical degree
MD	9353	1845 (20%)	0 (0, 0)	0.5 (1.5)
DO	2,636	199 (8%)	0 (0, 0)	0.1 (0.8)
PhD degree
No	11,747	1,909 (16%)	0 (0, 0)	0.4 (1.4)
Yes	242	135 (56%)	1 (0, 2)	1.6 (2.3)

^aAll group differences are statistically significant at p < 0.001

^bGroups compared using chi-square tests

^cGroups compared using rank-sum tests

Discussion

Medical student motivation to pursue research experiences and scholarly publications during their training creates a tension with the constraints of the medical school curriculum, where time for research activities is often limited, unless students extend their training by pursuing advanced research degrees or dedicated research years. Therefore, it is essential for medical students to set realistic and attainable goals for scholarly activity during medical school. This study provides an objective, nationally representative description of PubMed-indexed publications among a cohort of new interns, estimating that only 39% of interns had any publications during medical school. Differences in publication rates by gender mirror previously reported gender differences in self-reported research productivity among medical students [8], while differences in publication rates by degree (MD or DO) may reflect differences in available research resources, research opportunities, or exposure to research in the curriculum across school types.

Self-reported applicant data on scholarly activity heading into the Match shows matched applicants had a mean of 7.9 publications, presentations, and abstracts in 2022 [3]. However, research on matched applicants in competitive specialties has consistently demonstrated that the count of self-reported research items is much higher than the number of PubMed-indexed publications [7, 9, 10]. This discrepancy also means that, as the total number of all research items increases across cohorts, this number tends to increase faster than the number of indexed publications [7, 11]. The present analysis extends these findings to incoming interns in all specialties. In particular, these data indicate that publishing a single PubMed-indexed article, especially a first-author article, is a significant scholarly accomplishment for medical students and can be used to benchmark scholarly goals regardless of intended specialty.

The gulf between PubMed-indexed publications and total of self-reported publications, presentations, and abstracts also requires further attention, as it may represent opportunities to guide students’ effort towards higher-yield scholarly endeavors. A recent analysis of mandatory medical student research projects in the Netherlands has found that only 28% resulted in publication [12], with similar data being reported from elective research experiences in the USA [13, 14]. While publication is not the sole goal of research or scholarly activity, the completion of a published project gives trainees broader exposure to all elements of the research process. For students interested in research experiences, emphasizing the importance of substantive engagement with a few (or even just one) projects leading to publication can also help manage the total time devoted to extracurricular scholarly activity and balance this with success in the curriculum and pursuit of other extracurricular opportunities.

The present study is subject to several limitations, including imperfect ascertainment of intern physicians in the NPPES database and exclusion of interns with common names to reduce the risk of publication misattribution, which may have downwardly biased estimates of research productivity. Publication counts were also subject to error based on misattribution of publications by other researchers with the same name and exclusion of publications not indexed in PubMed. The large size of the database precluded manual checking and confirmation of author and publication eligibility for the entire sample. Therefore, the analysis of specialties among the top 100 authors with the most attributed publications should be considered exploratory rather than definitive.

Despite these limitations, this analysis provides new and more generally applicable estimates of medical student publication output, supporting guidance on research participation for all medical students in the USA, regardless of intended specialty. Future research should replicate and extend this work to stratify publication output by specialty and to track trends in publication output in more recent cohorts affected by the transition of licensing exam scoring to pass-fail.

Appendix 1. Database Search Strategy and Validation

This study defined residents as health care providers with a Type 1 (individual provider) National Provider Identifier (NPI) in the NPPES database, a MD or DO degree listed, and a primary or secondary NPI provider taxonomy code of 390200000X (“students in an organized healthcare or training program,” a code which includes resident physicians who have not yet been licensed by a state board of medicine). NPI numbers are unique to each health care provider and issued only once. Therefore, the sample was limited to resident physicians with a NPI enumeration date between the first day after the military Match and the last day before new residents begin work in the US (i.e., 9 December 2021 to 30 June 2022), and the remaining sample was considered to represent only interns.

Relative to the 2022 cohort of matched graduating medical students from the author’s institution (N=84), this search strategy successfully identified the corresponding NPI number for 53 interns (63%). In this cohort, the most common reasons for not having an eligible NPPES registry record were absence of “MD” credential (18%) in the registry record, and absence of the student taxonomy code (13%).

To quantify potential misattribution of PubMed-indexed publications by other researchers with the same name as each intern, we manually reviewed a random subsample of 100 intern names before conducting the main search. A PubMed query using the parameters described above returned 97 results for 28 intern names. After conducting a Web search of public records to confirm author affiliations, we determined that 22 of 28 interns (79%) were correctly classified as having published at least one article during the study period, while 74 of 97 articles identified in our search (76%) were correctly attributed to one of the intern physicians in this cohort.

Appendix 2. Manual Search to Determine the Specialty of the 100 Interns with the Most Publications Attributed by the Study

The author manually searched the Web in February 2023 to determine the specialties of 100 profiles with the highest number of total publications. This manual review excluded 5 profiles of physicians who completed residency training abroad; 4 profiles of physicians in non-clinical research positions; 3 profiles appearing to represent attending physicians; 1 profile appearing to represent a current medical student; and 1 profile of an intern physician whose specialty could not be determined. The remaining 86 interns had between 20-134 publications (median: 29) attributed by the PubMed search strategy. This group included 69 men and 17 women; 78 MDs and 8 DOs; and 10 PhDs. The specialty distribution of the 86 interns is shown in the Table below.

Specialty	Number of residents
Internal Medicine	22
Neurological Surgery	10
Orthopedic Surgery	10
General Surgery	9
Plastic Surgery	7
Pediatrics	5
Ophthalmology	3
Transitional Year	3
Urology	3
Dermatology	2
Neurology	2
Otolaryngology	2
Preliminary Year (not otherwise specified)	2
Anesthesiology	1
Diagnostic Radiology	1
Emergency Medicine	1
Family Medicine	1
Pathology	1
Psychiatry	1

Data Availability

Source data are publicly accessible and replication code is available from the author upon request.

Declarations

Conflict of Interest

The author discloses salary support through research and quality improvement grants from the Kate B. Reynolds Charitable Trust and Lilly and Co., Inc., which are unrelated to the present work. The author has no other relevant financial or non-financial interests or competing interests to declare.