Association Between Preresidency Peer-reviewed Publications and Future Academic Productivity or Career Choice Among Ophthalmology Residency Applicants

This study evaluates the correlation between the number of preresidency peer-reviewed publications and subsequent peer-reviewed publications or career choices of ophthalmology residency graduates.

Key Points

Question

Is preresidency publication history predictive of future academic productivity or career choice in ophthalmology residency applicants?

Findings

This cross-sectional study found preresidency publication history has at least a weak positive correlation with intraresidency, postresidency, first authorship, and publication in journals with an impact factor score of 3 or more. There was a difference in preresidency characteristics associated with academic vs community-based ophthalmologists.

Meaning

Preresidency publications may be correlated with future research or choice of an academic career in ophthalmology, but admissions committees should recognize the current limitations of these investigations.

Abstract

Importance

Ophthalmology-residency selection committees require robust metrics to review applicants. Participation in research activities is a core component of the application process for its perceived association with future academic productivity.

Objective

To evaluate the correlation between the number of preresidency peer-reviewed publications (PPPs) and subsequent peer-reviewed publications or career choices of ophthalmology residency graduates.

Design, Setting, and Participants

In this cross-sectional study, names of ophthalmology residency graduates were obtained. PubMed-indexed publication records were generated and publications were categorized as preresidency, intraresidency, and postresidency. First author and journal publications with an impact factor (IF) score of 3 or more were recorded. Current academic and community-based career statuses were designated. Names were obtained from cohort and alumni lists on residency program websites or by emailing program directors. Participants included US Accreditation Council for Graduate Medical Education–accredited ophthalmology residency graduates from 2013 to 2016.

Main Outcomes and Measures

The primary outcome measure was association of PPPs with later publications, first authorship, and journal publications with an IF score of 3 or more. The secondary outcome measure was difference in characteristics associated with academic vs community-based ophthalmologist.

Results

A total of 964 ophthalmologists (52% of graduates) were studied and most (85.5%) had PubMed-indexed publications. First authorship (ρ = 0.71; 95% CI, 0.67-0.74; P < .001) had a strong positive correlation with intraresidency publications, while journal publications with an IF score of 3 or more (ρ = 0.56; 95% CI, 0.51-0.60; P < .001) and PPPs (ρ = 0.38; 95% CI, 0.32-0.43; P < .001) had moderate and weak positive correlations, respectively. For postresidency publications, journal publications with an IF score of 3 or more (ρ = 0.86; 95% CI, 0.84-0.87; P < .001) had the strongest positive correlation followed by first authorship (ρ = 0.77; 95% CI, 0.74-0.79; P < .001) and PPPs (ρ = 0.26; 95% CI, 0.20-0.31; P < .001). Preresidency (t = 3.3; P = .001), intraresidency (t = 4.1; P < .001), postresidency (t = 7.5; P < .001), first author (t = 6.6; P < .001), and journal publications with an IF score of 3 or more (t = 5.9; P < .001) were greater for academic ophthalmologists compared with community-based ophthalmologists.

Conclusions and Relevance

Preresidency publication history is at least weakly correlated with future publications or work in an academic setting among ophthalmologists. Multiple factors associated with academic productivity were evaluated; however, adjustment for multiple analyses was not done and further testing is required to prove whether these factors are predictive.

Introduction

Research experience has been considered an important part of the residency selection process and identifies an applicant’s commitment to their field and aptitude for academic pursuits.¹ Although the weight placed on these experiences can vary between residency programs and across medical specialties, conducting research is now considered an important part of becoming a physician.² Many medical schools and residency programs now set aside time for scholarly endeavors as part of their training with recent reports showing an increase in the total number of resident publications as a result.³

In recent years, objective measures, such as preclinical course grades and US Medical Licensing Examination (USMLE) step 1 scores, have moved to pass/fail scoring. As a result, it is believed that other key factors, such as letters of recommendation, research experience, and USMLE step 2 clinical knowledge, will begin to have increased importance during the residency application process.⁴ Faced with the challenge of reviewing an increasing number of applications each year, program directors are looking for quantifiable data to compare candidates and predict their future success. Whether higher publication numbers before or during medical school correlate with being a more productive resident is unclear; however, multiple studies have pointed to the limited predictive value of using conventional objective measures (eg, grades, USMLE scores, medical school rank) to predict overall resident success.^5,6

Prior studies have demonstrated that physicians who published as medical students were associated with increased publications during residency and a higher propensity for pursuing an academic career.^7,8,9 Cruz et al¹⁰ studied whether the subject and setting of preresidency publications was associated with postresidency academic productivity among graduates of Wilmer Eye Institute. This study seeks to expand upon previous work by analyzing multiple factors associated with academic productivity in graduates from 75 different residency programs.

According to the Association of American Medical Colleges, ophthalmology applicants should aim for high levels of research experience compared with the average medical student.¹¹ Therefore, this study was designed to analyze correlations between preresidency research productivity and subsequent research output or career choice for ophthalmologists who graduated residency between 2013 and 2016.

Methods

Ethics and Permissions

As this study was composed of both publicly available and private data, approval for the project was obtained through the institutional review board of University of Miami. Ophthalmology program directors were contacted via email to request the names of their 2013 to 2016 graduates and the graduates themselves were never directly contacted. These names were then used to search publicly available data. The study was conducted in compliance with Health Insurance Portability and Accountability Act regulations and the need for institutional review board approval was waived by the University of Miami. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guidelines were followed.

Participants

A database was assembled of ophthalmologists who graduated from a US-based residency program from 2013 through 2016 from cohort and alumni lists on residency program websites or by emailing program directors directly requesting names of graduates. All 124 Accreditation Council for Graduate Medical Education–accredited residency programs as of March 2021 were searched online or contacted via email.

Data Collection

Each ophthalmologist was then searched in PubMed by full name to find all associated publications. To account for potential variations in name and organization, multiple permutations (including a person’s first name, last name, first initial and last name, the word “ophthalmology,” and residency program name) were used when searching through PubMed. Additionally, each ophthalmologist was searched on an internet search engine (Google) to address possible name changes or variation, such as following marriage. The alternate names were recorded in the database and used for the search as described above.

Publications were then analyzed for supporting characteristics to improve the accuracy of the selection process. Characteristics considered included but were not limited to middle initial, residency program or medical school affiliation, location, and associated field of ophthalmology. For difficult-to-discern publications, outside references, including LinkedIn and online curriculum vitae, were cross-referenced for validation. Search results of common names with numerous results (John Smith, Jing Wang, etc) that could not be validated with the above method were excluded from the database to increase the specificity of the analysis.

Noteworthy characteristics, such as first author and impact factor (IF) of 3 or more, were also collected. An IF score of 3 or more was chosen to reflect the added work required to publish in 1 of the 17% of ophthalmology-related journals (21 of 127) with this IF score. While no one metric can be used to assess academic quality, journal IF score was used as the data are available for each publication reviewed in this study. IF scores were determined using the Resurchify database and were accessed in March 2021. Additionally, each ophthalmologist was subsequently searched using an internet search engine (Google) to determine whether they had an academic or community-based practice as perceived by the authors at the time of data collection. Publication and employment data collection concluded on June 10, 2021, and no publications after this date were included in this study.

Classification

Ophthalmology residency programs operate on a 4-year cycle, beginning July 1 each year. Intraresidency publications were defined as 3 years before ones’ residency graduation date to a 1-year postgraduation date to capture work likely completed during residency but published after. Preresidency was defined as any publication prior to the intraresidency period and postresidency was defined as the intraresidency period end date to present. Articles were categorized according to their publication date on PubMed.

Statistical Analysis

The mean and median number of publications preresidency, intraresidency, and postresidency were calculated for each ophthalmologist by year. Correlations were performed comparing preresidency vs intraresidency, intraresidency vs postresidency, and preresidency vs postresidency. Correlational analyses were performed for first author and an IF score of 3 or more publications between the preresidency, intraresidency, and postresidency windows. The mean number of publications stratified by graduation year and as a total for preresidency, intraresidency, and postresidency were compared in order to establish a trend by year. Independent-samples t tests were performed to compare preresidency, intraresidency, postresidency, first author, and publications with an IF score of 3 or more for academic and community-based ophthalmologists. All P values were 2-sided and no adjustment was made for multiple analysis.

Results

The data set yielded 7789 PubMed-indexed publications from 964 ophthalmology graduates identified from 75 of 124 residency programs (60.5%). The number of graduates across the 4 years was distributed as follows: 201, 222, 244, and 297 from 2013 to 2016, respectively. These 964 graduates represent at least 52% of the potential graduates in that time frame based on SFMatch data indicating 1846 filled ophthalmology resident positions in those years. The number of total graduates was estimated from SFMatch data of positions filled and was distributed as 455, 460, 464, and 467 from 2013 to 2016, respectively. A total of 824 ophthalmologists (85.5%) had PubMed-indexed publications, 503 (52.1%) published before starting residency, 596 (61.8%) published during residency, and 606 (62.9%) published following residency. A total of 628 published as a first author (65.1%), 588 published in a journal with an IF score of 3 or more (61.0%), and 316 pursued an academic career (32.8%).

The mean number of publications during the preresidency, intraresidency, and postresidency period was 1.58 (SD, 3.08), 2.03 (SD, 2.97), and 4.46 (SD, 9.81) for 2013 to 2016, respectively (Table 1). Correlation between publications was weak for preresidency vs intraresidency, moderate for intraresidency vs postresidency, and weak for preresidency vs postresidency, as seen in Table 2. Correlation between publications was weak for first author vs preresidency, strong for first author vs intraresidency, and strong for first author vs postresidency, as seen in Table 2. Correlation between publications was moderate for IF scores of 3 or more vs preresidency, moderate for IF scores of 3 or more vs intraresidency, and very strong for IF scores of 3 or more vs postresidency, as seen in Table 2. Mean numbers of preresidency publications were 1.63 (SD, 2.33), 1.6 (SD, 4.56), 1.87 (SD, 2.56), and 1.3 (SD, 2.47) from 2013 to 2016, respectively. Mean numbers of intraresidency publications were 2.24 (SD, 3.33), 2.2 (SD, 3.08), 2.02 (SD, 2.78), and 1.76 (SD, 2.27) from 2013 to 2016, respectively. Mean numbers of postresidency publications were 6.29 (SD, 14.21), 5.36 (SD, 11.86), 3.95 (SD, 6.81), and 2.95 (SD, 5.22) from 2013 to 2016, respectively (Table 3).

Table 1. Number of Publications by Residency Stage and Total.

Residency stage	Mean (SD)	Median (IQR) (range)
Preresidency	1.58 (3.08)	1 (2) (0-63)
Intraresidency	2.03 (2.97)	1 (3) (0-21)
Postresidency	4.46 (9.81)	1 (4) (0-120)
Total	8.08 (12.85)	4 (10) (0-133)

Table 2. Correlations Between Residency Stage Publications, First Author, and Impact Factor (IF) Score of 3 or More.

Characteristics	ρ (95% CI)	P value
Preresidency vs intraresidency publications	0.38 (0.32-0.43)	<.001
Intraresidency vs postresidency publications	0.49 (0.44-0.53)
Preresidency vs postresidency publications	0.26 (0.20-0.31)
First author vs preresidency publications	0.40 (0.35-0.45)
First author vs intraresidency publications	0.71 (0.67-0.74)
First author vs postresidency publications	0.77 (0.74-0.79)
IF score ≥3 vs preresidency publications	0.56 (0.51-0.60)
IF score ≥3 vs intraresidency publications	0.56 (0.51-0.60)
IF score ≥3 vs postresidency publications	0.86 (0.84-0.87)

Table 3. Number of Publications per Residency Stage and Total, Stratified by Year.

Residency stage	Mean (SD)
	2013	2014	2015	2016	Total
Preresidency	1.63 (2.33)	1.60 (4.56)	1.87 (2.56)	1.30 (2.47)	1.58 (3.08)
Intraresidency	2.24 (3.33)	2.20 (3.08)	2.02 (2.78)	1.76 (2.75)	2.03 (2.97)
Postresidency	6.29 (14.21)	5.36 (11.86)	3.95 (6.81)	2.95 (5.22)	4.46 (9.81)

Independent-samples t test showed a difference in the means of preresidency (mean difference, 0.87; 95% CI, 0.46-1.28; t = 3.3; P = .001), intraresidency (mean difference, 0.93; 95% CI, 0.53-1.32; t =  4.1; P < .001), postresidency (mean difference, 6.33; 95% CI, 5.08-7.59; t = 7.5; P < .001), first author (mean difference, 2.14; 95% CI, 1.57-2.70; t = 6.6; P < .001), and IF scores of 3 or more (mean difference, 3.33; 95% CI, 2.49-4.17; t = 5.9; P < .001) for journal publications for academic and community-based ophthalmologists with ophthalmologists in academic careers having more publications than community-based ophthalmologists (Table 4).

Table 4. Independent-Samples t Test of Academic vs Community-Based Ophthalmologists.

Characteristics	University-employed (no = 0; yes = 1)	Mean (SD)	t	Mean difference (95% CI)	2-Sided P value
Preresidency	1	2.16 (4.43)	3.31	0.87 (0.46-1.28)	.001
	0	1.29 (2.08)
Intraresidency	1	2.65 (3.53)	4.15	0.93 (0.53-1.32)	<.001
	0	1.73 (2.60)
Postresidency	1	8.72 (14.60)	7.49	6.33 (5.08-7.59)	<.001
	0	2.39 (5.14)
First author	1	4.17 (5.17)	6.59	2.14 (1.57-2.70)	<.001
	0	2.03 (3.62)
IF score ≥3	1	5.25 (9.65)	5.95	3.33 (2.49-4.17)	<.001
	0	1.92 (3.56)

Abbreviation: IF, impact factor.

To determine whether selection bias was present when emailing residency program directors requesting names of ophthalmology graduates, the study team analyzed the region and number of residents in each program and compared those who replied with those who did not. Independent-samples t test showed no statistical difference between responsive vs unresponsive programs when comparing number of residents in each program (mean difference, 0.02; 95% CI; −1.00 to 1.04; t = 0.031; P = .49). χ² Analysis was then performed to compare the proportion of responsive with unresponsive programs for each region (West, Midwest, Northeast, and South) and we found no statistical difference between regions: West vs Midwest (difference, 24.17; 95% CI, −10.04 to 57.50; χ² = 1.7; P = .19), West vs Northeast (difference, 16.45; 95% CI, −16.17 to 50.75; χ² = 0.76; P = .38), West vs South (difference, 11.03; 95% CI, −18.03 to 45.09; χ² = 0.38; P = .54), Midwest vs Northeast (difference, 7.72; 95% CI, −19.85 to 31.98; χ² = 0.33; P = .56), Midwest vs South (difference, 13.14; 95% CI, −14.13 to 32.35; χ² = 1.0; P = .31), and Northeast vs South (difference, 5.42; 95% CI, −19.83 to 26.26; χ² = 0.19; P = .66).

Discussion

The current study sought to evaluate the association between factors commonly considered to represent academic productivity. Given the evolving landscape surrounding the residency application process related to objective measures, such as examination scores and clinical rotation performance, evaluating factors used to guide selection committees is important. A better assessment of the qualities associated with ophthalmologists who pursue an academic career is valuable for many selection committees, depending on how a given residency program prioritizes promoting university-based careers among their alumni. Academic medicine provides opportunities for research, care of patients with complex medical problems, and education of physicians-in-training. Despite these attractions, there have been growing concerns about a perceived diminished workforce in academic medicine, including ophthalmology.^12,13 Therefore, it is particularly important to understand factors that predict this career pathway.

Publishing preresidency was found to have at least a weak positive correlation with publishing during and after residency. These data suggest that a history of academic projects may be predictive of future academic productivity. These results are well aligned with prior studies conducted in other specialties, including otolaryngology, pediatrics, dermatology, neurology, urology, plastic surgery, and radiology.^{7,8,9,14,15,16,17} Medical students who successfully published early in their career may garner experience and skills performing research, which can be leveraged during residency and into their career as responsibilities and workload continue to grow. Parker et al¹⁸ demonstrated that participation in an organized research program is associated with the pursuit of future research. Furthermore, the study reported that students from more highly ranked colleges may gain exposure to more opportunities to conduct research, gaining a better understanding of how to perform in a research setting. Although ophthalmology residents who have no preresidency publications may still develop the skillset to perform high-quality research during residency and their career, these findings suggest that they may benefit from structured programs to facilitate this growth. Additionally, care must be taken by selection committees to understand that the limited resources available to students from less research-robust medical schools do not necessarily preclude them from future research productivity.

In a prior study, Lin et al¹⁹ found in their analysis of first-year ophthalmology residents from 2009 that having an additional graduate degree, between 1 and 3 publications, or 4 or more publications were associated with higher future research productivity. Still, as there are many factors that likely influence the decision to pursue an academic career, looking only at the volume of research produced may have limited utility in predicting which applicants will pursue an academic career. Our study found that both publication as a first author or in an journal with an IF score of 3 or more was associated with increased research productivity during and after residency or pursuit of an academic career. A similar study of dermatology residents¹⁷ demonstrated the same finding regarding first authorship and publications in a high IF journal and additionally noted a negative association with the number of case reports and pursuit of an academic career. In contrast, a recent study by Cruz et al¹⁰ found that only peer-reviewed publications (PRPs) related to projects performed during undergraduate and medical school were associated with academic productivity and neither the subject of the PRP nor order of authorship was associated with future academic productivity in their cohort. One possible explanation for the different results was in defining academic productivity. Cruz et al defined academic productivity based upon faculty status and academic productivity score, a combination of various achievements and activities that are important for a successful academic career, such as postresidency research publications, faculty status, grants, editorial board membership, reviewer appointment, and mentorship. This study, like others,^14,20 defined academic productivity based on the number of PubMed indexed PRPs, as this provides an objective quantitative score and is an academic standard.²¹

It is unclear if exposure to research and publishing during training fosters an interest in an academic career or whether these scholarly activities are done to achieve a pre-established career goal of entering academic medicine. Furthermore, applicants who publish more are likely to have better access to mentorship and research support, which subsequently encourages more research publications. Programs with a greater focus on academia may also rank applicants with a history of research and academic aspirations higher, creating selection bias favoring those future ophthalmologists’ development into university-based clinicians. For example, Hang et al²² found that attending a highly ranked program where resources, culture, and proper infrastructure are more readily available was the most predictive measure of a resident’s research productivity.

Limitations

Several limitations exist in the current study. First, though this study sought to include every graduate from 2013 to 2016 from an Accreditation Council for Graduate Medical Education-accredited residency program, not every graduate could be identified as there was an approximately 52% capture rate of graduates (964 of 1846 graduates) from 60.5% of residency programs (75 of 124 residency programs) after emailing program directors or reviewing residency webpages. The number of ophthalmology graduates from 2013 to 2016 was estimated using data from SFMatch, assuming positions filled approximates the number of graduates each year. As the number of positions was seen to increase each year and not every resident who starts residency ultimately completes training, this number likely overestimates the number of ophthalmology graduates in the years sampled.

Selection bias is another limitation inherent to survey-based research. It is very possible that residency programs that have residents who are more prolific before, during, or after residency were more likely to respond to the request to provide the names of resident graduates. Similarly, programs that produced more academic ophthalmologists may have been more likely to respond. Still, after analyzing by region and number of residents, we found no statistical difference between the programs that were included in this analysis and those that were not, supporting the idea that our sample is likely representative of the overall population. Graduates from residency programs which closed prior to this study but were open between 2013 to 2016 may not have been captured due to the method of identifying graduates. Thus, the graduates included in this study may not be fully representative of the field.

While generating publication histories of the graduates, great care was taken to validate that those publications belonged to the graduate of interest. In some cases, the graduate’s name was too common to validate the publications and these individuals were removed from the study in an effort to increase the specificity of the study. Another limitation in this study is the use of only PubMed-indexed journals, which may not capture every article published by the ophthalmologist in this study, including magazine articles or book chapters. Research productivity may also be measured by poster presentations, abstracts, and oral presentations, none of which were captured in this study. The study did also not distinguish ophthalmology-related publications from nonophthalmology-related publications and did not delve into whether publications were associated with additional years off during medical school for research, whether it be in the process of obtaining an additional postgraduate degree or unrelated to a formal program.

Additionally, academic ophthalmologists and community-based ophthalmologists were defined according to their current position at the time of this study; however, some may have worked in the alternate position prior to this study. The nature of their practice was determined as perceived by the authors and is subject to error as some ophthalmologists work in centers affiliated with an academic institution while not engaging in academic endeavors. Where possible, level of affiliation, including active professorship, was determined to inform the decision-making process.

Conclusions

In conclusion, this study demonstrated that research output before residency was at least weakly correlated with increased publications during and after residency for ophthalmologists graduating between 2013 and 2016 across the US. Publication as a first author or publication in a journal with an IF score of 3 or more was likewise associated with increased future academic productivity or with holding an academic position. The use of multiple analysis without adjustment limits this study’s ability to prove the predictive power of the factors studied. Selection committees interested in attracting future academic ophthalmologists may find these results useful when considering what to include in a holistic review of candidates.

Supplement.

Data sharing statement.