Urology Resident Publication Output and Its Relationship to Future Academic Achievement

Abstract

Purpose

Scholarly research is considered by many to be an important component of residency training but little is known about the quantity and types of publications produced by urology residents. To our knowledge whether publication efforts during residency predict future academic publication performance is also unknown. We evaluated resident productivity, as measured by peer reviewed publication output, and determined its relation to future publication output as junior faculty.

Materials and Methods

We assembled a list of graduating residents from 2002 to 2004 who were affiliated with the top 50 urology hospitals, as ranked in 2009 by U.S. News & World Report. PubMed® was queried to determine the publication total in the last 3 years of residency of each individual and during years 2 to 4 after residency graduation. Resident publication output was stratified by research time and fellowship training. The relationship between resident productivity and future achievement was assessed.

Results

We assessed the publication output of 251 urologists from a total of 34 training programs affiliated with the top 50 urology hospitals. Subjects published a mean total of 3.5 and a mean of 2.0 first author papers during training. Greater research time during residency was associated with increased productivity during and after residency. Publication during training correlated with publication during the early academic career.

Conclusions

Publication output correlated with increasing dedicated research time and was associated with the pursuit of fellowship training and an academic career. Publication during residency predicted future academic achievement.

Academic scholarship in the form of peer reviewed publications is an important component of training during urology residency¹ and yet barriers to resident research exist, such as limited time, funding and research mentors.² Understanding the quantity and types of publications produced by urology residents may aid in evaluating trainees and developing residency program guidelines but little is known about resident publication output. Also, the significance of resident publication rates in predicting future academic achievement is not clear.

Others have begun to quantify resident publications, primarily using voluntary surveys.^3–5 In orthopedic surgery the amount of dedicated research time correlated with resident publication output and the attainment of research grants but did not affect whether residents pursued an academic career.^4,5 In radiation oncology Morgan et al found that trainees published an average of 1 first author paper during residency and were more productive at departments with more trainees.⁶

In urology a group quantified resident publication output.³ Using voluntary surveys they found that most residents published at least 1 first author manuscript during training and the publication rate correlated with the amount of dedicated research time. However, to our knowledge no study has objectively quantified urology resident publications or correlated this productivity with future achievement.

We quantified the peer reviewed publication output of urology residents during residency and in their early careers. We hypothesized that the amount of dedicated research time during residency correlated with resident publication and increased resident productivity correlated with future academic success.

METHODS

Study Population and Variables

We assembled a list of urology residency programs affiliated with the 50 top urology hospitals in 2009, as ranked by U.S. News & World Report (www.health.usnews.com). Institutional review board exemption was obtained. The names of residents who graduated in 2002, 2003 and 2004 were acquired from program websites or by contacting program administrators directly. We obtained information on publication output during and after residency, the amount of dedicated research time during residency, whether and where residents went on to fellowship training, whether they pursued an academic career, current academic rank, and the h-index from departmental and physician websites. Dedicated research time during residency was analyzed as a categorical variable (0, 3 to 4, and 6 and 12 months) since the research blocks of programs stratified into these discrete groupings.

Publication Data and h-Index

Data on the peer reviewed publications of each graduated resident were obtained by PubMed query using multiple search terms, including 1) full name, 2) last name plus first and middle initial, 3) last name plus first initial and 4) last name plus first initial plus urology. We attributed unique publications resulting from these queries to a given urologist if they met 2 criteria, that is they 1) pertained to a urological topic and 2) were affiliated with an institution where the urologist had spent professional time. If only 1 of the 2 criteria were met, the original manuscript was reviewed to ensure that the author name was an exact match to that of the urologist in question before it was included as a publication by that urologist. If neither criterion was met (the publication was neither urology related nor affiliated with an institution where the urologist had spent time), the publication was discarded. For each study subject the compiled list of publications was reviewed by a urology physician to determine the type of each publication (original research, review article, case report or editorial) and the order of resident authorship (first, second, middle or last). In 2 of the 878 publications reviewed there was a question of whether a manuscript should be categorized as editorial or original research and in these 2 situations we sought the opinion of one of us (PRC). To determine the relative residency publication output all PUBMED entries published during the calendar year of the residency graduation and 2 years previously were totaled (fig. 1). Similarly all PubMed articles published during the second, third and fourth calendar years after residency completion were totaled as a measure of subject academic productivity during their early careers. For example, for a subject completing residency in 2004 all articles published in 2002 through 2004 were included as well as those published in 2005 through 2007. Articles published during year 1 after residency were excluded from analysis to exclude those likely completed during fellowship.

Figure 1.

Publication analysis timeline of resident graduating in June 2002. Publications were totaled during graduation calendar year and 2 prior years, and during years 2 to 4 years after graduation. Hatched bar indicates publications during residency. Dotted bar indicates publications after residency.

As a measure of the overall academic achievement of each subject we calculated the h-index⁷ in the Scopus® database (www.scopus.com) by accessing the citation index. Defined as the number of publications (h) by an author that have been cited a minimum of h times, the h-index of a researcher is a measure of the publication record that combines aspects of quantity and quality. For example, a researcher who has published 8 manuscripts that have each been cited at least 8 times would have an h-index of 8. Any additional publications that have not been cited at least 8 times would be ignored. While the impact factor assesses the merits of a given journal, the h-index assesses an individual researcher. the h-index has gained increasing popularity and was recently used to examine academic rank among academic urology faculty.⁸ It performs better than total citation count, citations per paper and total paper count to predict the future publication output of a researcher.⁹

Statistical Analysis

We used descriptive statistics to characterize the study population. Since our data were not normally distributed, the mean, median and IQR are presented. The Cuzick nonparametric test of trend for ranks across ordered groups was used to assess for trends in continuous variables. The t test was used to compare publication outputs among those who did and did not obtain fellowship training. The nonparametric Spearman rank correlation coefficient was used to determine the relationship between residency publication totals and post-residency publication totals. We produced a scatterplot of publications during and after residency, and present academic rank. Multivariate logistic regression analysis was performed in subjects who pursued an academic career. The outcome was academic rank (assistant or associate) and the a priori selected variables were resident publication output and year of graduation. Subjects with missing data were excluded from multivariate analysis. Statistical significance was defined at p <0.05 and all tests were 2-sided. Stata® 11 was used for all analysis.

RESULTS

Subject Characteristics

Of the 50 urology hospitals considered for analysis 37 supported urology residencies. The names of graduating residents were available for 34 programs but 3 programs that were ranked 12, 32 and 48, respectively, did not provide names or respond to our inquiries. The study cohort included 251 urology residents, of whom 163 (65%) had dedicated research months during residency, 102 (41%) completed fellowship training and 68 (27%) ultimately pursued an academic career (table 1).

Table 1.

Characteristics of 2002 to 2004 urology resident graduates from top 50 urology hospitals

	No. Subjects (%)
Graduation yr:
2002	83 (33)
2003	84 (33.5)
2004	84 (33.5)
Gender:
M	220 (87.7)
F	31 (12.4)
Research time (mos):
0	88 (35.1)
3–4	30 (11.9)
6	40 (15.9)
12	93 (37)
Fellowship:
None	149 (59.4)
Oncology	25 (10)
Infertility	7 (2.8)
Laparoscopy	34 (13.6)
Pediatrics	16 (6.4)
Female	12 (4.8)
Reconstruction	6 (2.4)
Transplantation	2 (0.8)
Current post-residency position:
Private practice	183 (72.9)
Assistant	55 (22)
Associate	13 (5.2)

Publications

During residency

During the last 3 years of residency subjects published a mean of 3.5 total articles (median 2) and 2.0 first author articles (median 1) (table 2). Future pursuit of an academic career, eventual fellowship training and increased dedicated research time during residency were significantly associated with greater resident publication (table 2). Residents with 12 months of research time published almost 3 times the number of manuscripts of residents without research time (p <0.001). Figure 2 shows the type and number of publications in the last 3 years of residency, stratified by research time.

Table 2.

Residency productivity by research time, fellowship status and academic appointment

	Mean/Median Publications (IQR)
	First Author	Total
Overall	2.0/1.0 (0–3)	3.5/2.0 (1–5)
Research time (mos):*
0	0.9/0.0 (0–1)	1.6/1.0 (0–3)
3–4	1.5/1.0 (0–2)	2.4/2.0 (1–3)
6	1.8/1.0 (0–2)	3.4/2.0 (1–4)
12	3.3/2.0 (1–4.5)	5.8/4.0 (2–8.5)
Fellowship:†
No	1.1/1.0 (0–2)	2.2/1.0 (0–3)
Yes	3.3/2.0 (1–5)	5.5/4.0 (2–7)
Appointment:*
Private practice	1.2/1.0 (0–2)	2.3/2.0 (0–3)
Assistant	3.3/2.0 (1–5)	5.5/5.0 (2–8)
Associate	7.2/6.0 (3.5–10)	13.1/13.5 (6.5–15.5)

^*Cuzick nonparametric test for trend p <0.001.

^†t Test p <0.001.

Figure 2.

Mean resident publication totals stratified by dedicated research time. Blue bars indicate original research. Green bars indicate case reports. Red bars indicate reviews. Orange bars indicate editorials.

After residency

During years 2 to 4 after residency urologists in academic practice produced a mean of 12.1 total and 4.7 first author publications. Research time during residency, fellowship training and pursuit of an academic career significantly correlated with productivity after residency (table 3). Figure 3 shows the relationship between subject publication rates during and after residency (Spearman’s rank correlation coefficient 0.5, p <0.001). The subject h-index differed significantly depending on research time during residency and current academic rank (table 3).

Table 3.

Productivity after residency by research time, fellowship type and academic appointment

	Mean/Median First Author Publications (IQR)	p Value	Mean/Median Total Publications (IQR)	Mean/Median h-Index (IQR)
Overall	1.6/0.0 (0–2)	—	4.3/1.0 (0–5)	5.2/3.0 (1–7)
Research time (mos):
0	0.9/0.0 (0–1)		2.2/0.0 (0–2)	3.0/2.0 (1–4)
3–4	1.5/0.0 (0–2)		3.6/0.5 (0–4)	4.6/4.0 (2–6)
6	1.7/0.0 (0–1)		4.9/1.0 (0–4)	5.3/3.0 (1.5–8)
12	2.3/0.0 (0–2.5)	0.002 (Cuzick nonparametric test for trend)	6.2/2.0 (0–9)*	7.5/5.0 (2.5–11)*
Fellowship:
No	0.3/0.0 (0–0)		1.2/0.0 (0–0)	2.9/2.0 (1–4)
Yes	3.6/2.0 (1–5)	<0.001 (t test)	8.9/6.0 (2–11)†	8.7/7.0 (5–11)†
Appointment:
Private practice	0.4/0.0 (0–0)		1.2/0.0 (0–1)	3.1/2.0 (1–4)
Assistant	4.1/3.0 (1–6)		10.1/7.0 (4–14)	9.4/8.0 (6–13)
Associate	7.5/3.5 (1.5–10.5)	<0.001 (Cuzick nonparametric test for trend)	22.7/16.0 (8.5–35)*	17.8/18.0 (13–22.5)*

^*Cuzick nonparametric test for trend p <0.001.

^†T test <0.001.

Figure 3.

Publication output during vs after residency (Spearman’s rank correlation coefficient 0.5, p <0.001). Circles represent private practice. Diamonds indicate associate professor. Triangles indicate assistant professor.

We performed logistic regression on the 68 former residents with current academic appointments to determine the influence of resident publication and graduation year on attaining the title of associate professor. In the cohort that published 9 or greater manuscripts during residency the odds of becoming an associate professor were increased by 21 (95% CI 2.0–218.3) compared to that in residents who publishing 0 to 3. As anticipated, time since graduation influenced academic advancement. Compared to 2002 graduates, 2004 graduates were 0.1 times (95% CI 0.02–0.9) as likely to achieve associate rank.

DISCUSSION

Producing peer reviewed research is an important component of urology residency for some. However, data are lacking on resident publication output and its relationship to future achievement. We report the publication rate in a large cohort of urology residents using objective citation analysis. Residents published a mean of 3.5 total and 2.0 first author manuscripts during training. Productivity significantly correlated with resident research time, future fellowship training and pursuit of an academic career. These factors were also associated with the future publication rate and the h-index during urologist early careers. There was a significant correlation between subject publications during and after residency.

A research curriculum was identified as an important component of urology residency¹⁰ and creating guidelines for resident manuscript publication may have a key role in designing such a curriculum. Determining publication production during residency may help set reasonable benchmarks for trainee requirements, or for resident or program evaluation. Hellenthal et al reported a slightly lower mean publication rate of 1.6 manuscripts during training in a cohort of American and Canadian urology residents who provided self-reported data via a questionnaire.³ Our higher publication averages likely reflect the high representation of academically oriented programs in the U.S. News & World Report list of best urology hospitals.

Our finding that increased dedicated resident research time, and future fellowship training and academic careers correlate with increased resident publication corroborates the results of others in urology³ and in other medical specialties.^4–6 Residents who eventually pursue fellowship training and academic careers publish more articles. This is a logical finding that likely reflects greater personal interest in scholarly pursuit. Similarly the fact that increased research time during residency portends a greater publication rate during training is logical since trainees may otherwise lack the time to complete scholarly projects. Research time during residency also correlated with the future publication rate after subjects had already graduated. This may reflect that residents with stronger academic interests self-select into training programs with more dedicated research time but it may also suggest that residency programs can instill greater academic productivity into trainees by providing more time to develop and hone analytical and writing skills.

To our knowledge what is novel about our approach is the ability to quantitatively correlate urology resident scholarly output with future academic achievement. These data may be useful to fellowship programs or academic departments when evaluating prospective candidates for fellowship or faculty positions. We confirmed the assumption that more productive residents become more productive new faculty members and we found a relationship between resident publication and the likelihood of achieving early associate professor status. Such data linking resident publication rate to future academic achievement validate the ongoing support of resident research by training programs seeking to nurture academic urologists.

The h-index has become an increasingly popular measure of the total scholarly work of an individual.⁷ It is defined as the number of papers (h) that have been cited by at least h other papers and it combines aspects of the quantity and quality of publications. In our series the h-index served as a current measure of academic achievement. In urology the h-index of faculty members is significantly associated with academic rank.⁸ Similarly we found that associate professors have a significantly higher h-index than assistant professors. That h-index in our cohort correlated with resident research time may be further evidence that providing trainees with such support fosters greater future success. However, the incremental h-index increase with research time may simply reflect additional publications completed during training and may not actually represent greater achievement after graduation. The h-index is subject to the multiple limitations documented previously, such as the favoring of authors who participate in large collaborative projects, its confounding by self-citation, its inability to discern whether a manuscript was cited in a positive or a negative light, and the possibility of age and gender biases.^{7–9,11–14}

We acknowledge limitations of this analysis. 1) The study cohort represents urology residents affiliated with the top 50 American urology hospitals, as ranked by U.S. News & World Report, and our findings may not represent residency programs outside this cohort. 2) Due to the temporal delay between manuscript preparation and publication, and the varied timing of fellowship training there may have been be some inaccuracy in categorizing publications as representing work completed during residency or during the early faculty career. By tracking publications by calendar year we included publications up to 6 months after graduation, which should have incorporated most but not all work completed during residency. By excluding the first calendar year after graduation we aimed to exclude most work done during fellowship training. Research time was self-reported by the program. Those who entered private practice may still be affiliated with academic training programs and have a critically important role in academic urology. 3) We assessed only publication records and academic employment, and recognize that they are only 2 components of academic success. We did not evaluate activities such as expert clinical care and education, which are critically important academic missions.

CONCLUSIONS

Urology residents from highly rated training programs published an average of 3.5 total and 2.0 first author peer reviewed manuscripts during residency. Publication output correlated with increasing dedicated research time and was associated with the pursuit of fellowship training and an academic career. Publication during residency predicted future academic achievement.

Abbreviations and Acronyms

h-index

Hirsch index