Long-Term Academic Outcomes of Triological Society Research Career Development Award Recipients

Abstract

Objectives:

Each year, the Triological Society awards several Research Career Development Awards (CDAs) to support early-career otolaryngologists. The objective of this study was to evaluate academic outcomes of CDA recipients including National Institute of Health (NIH) funding acquisition and h-index. A secondary objective was to appraise gender differences in outcomes among awardees.

Methods:

Recipients’ practice setting, degree type, academic rank, and leadership titles were determined through review of academic and private practice profiles in October 2019. NIH funding was assessed using the NIH Research Portfolio Online Reporting Tool (NIH-RePORTER) and h-index was calculated using the Scopus database.

Results:

Between 2004 and 2019, 70 investigators received a CDA. Of the 65 awardees prior to 2019, 26 (40.0%) obtained NIH grants after the CDA. Having an MD/PhD or MD/master’s was not associated with NIH funding attainment (p=0.891) nor with higher funding total (p=0.109). However, funding total was significantly higher for full professors compared to assistant professors (p=0.022). Median h-index among awardees was 16 (IQR 11–21) and differed significantly by academic rank (p<0.001). Moreover, 23 CDAs (32.9%) were awarded to women. However, fewer female recipients obtain NIH funding after the CDA compared to men (10.5% vs. 52.2%, p=0.002) and they had significantly lower h-indices than men (10 vs. 17, p<0.001).

Conclusion:

As a cohort, CDA awardees achieve higher academic success than academic otolaryngologists in general. However, female CDA recipients lag behind their male colleagues, highlighting the need for more research to uncover contributors to gender differences and ways to foster equity in research.

Introduction

Physician-scientists are critical to sustaining medicine’s progress. Their unique position as physician and researcher informs the other, allowing physician-scientists to bridge the gap between basic science and clinical practice in order to answer the most pressing questions in medicine.

Despite programs such as the National Institute of Health (NIH) Institutional Training Grant (T32) that provides residents with research support, there has been a steady decline in the number of physician-scientists over the last few decades.¹ A report by the NIH found that in 2012, only 13,717 physicians reported research as their primary activity compared to 14,467 physicians who stated the same in 2003.¹

The field of otolaryngology-head and neck surgery has seen a similar decline in surgeon-scientists.² Early career otolaryngologists and residents cite potential lost income, family and work-life balance, length of training, and poor departmental support as obstacles to pursuing a surgeon-scientist career.³ Additionally, a study by Narahari et al. found that NIH funding to surgeon-scientists, including otolaryngologists, has decreased over the last several years making it even more challenging for surgeons to have successful research careers.⁴

To mitigate this, a number of organizations have established career development awards for young faculty members interested in becoming surgeon-scientists.^5–7 Within otolaryngology, the Triological Society awards Research Career Development Awards (CDAs) to assistant and associate professors who have completed graduate training within the past 7 years.⁸ This $40,000 award is designed to support recipients in developing research projects and facilitate the pursuit of their long-term research goals.

To assess the CDA’s impact, this study evaluated the long-term academic outcomes of CDA recipients including acquisition of NIH funding after CDA receipt, NIH funding total, and h-index. A secondary aim was to appraise gender differences in academic outcomes among CDA awardees.

Materials and Methods

Study Design

Cross-sectional study

Study Protocol

Names of CDA recipient were obtained from the Triological Society website (www.triological.org) in October 2019. CDAs were awarded each year between 2004 and 2019, with the exception of 2015, when no awards were given (B. Slovinski, written personal correspondence, July 2019). Medical practice setting, degree type, academic rank, and departmental leadership titles were determined through review of institutional websites and academic and private practice profiles. Individuals possessing a master’s degree, including Master of Science, Master of Science in Clinical Research, Master of Health Science, Master of Public Health, and Master of Business Administration, were collectively analyzed as MD/master’s. If recipients possessed more than one advanced degree, only the most advanced degree (MD/PhD > MD/master’s) was recorded.

Practice setting was categorized as academic versus other. Academic rank was categorized as assistant professor, associate professor, and full professor. The following departmental leadership titles were recorded: department chair, vice chair, division chief (head of a division within their respective otolaryngology department), and residency program director.

NIH funding was determined using data from the NIH Research Portfolio Online Reporting Tool (NIH-RePORTER). If a recipient was listed as principal investigator (PI) or project leader, the number, type, and funding total of NIH grants was recorded. NIH Career Development Awards (K08 and K23 awards), which differ from Triological Society CDAs in funding source, were collectively analyzed as “K awards”. R01 and R01-equivalent grants (DP1, DP2, DP5, R01, R37, R56, RF1, RL1, U01 and R35 MIRA as defined by the NIH) were analyzed jointly as “R01 awards”.⁹ Only grants acquired beginning the calendar year after CDA receipt were recorded (e.g. if a CDA was awarded in 2014, grants were recorded beginning in 2015). This method was chosen because the primary aim of the study was to assess the impact of the CDA on academic outcomes; it is of the authors’ opinion that it was unlikely that NIH funding received prior to or in the same year as the CDA award was impacted by CDA receipt. As such, because data collection occurred in 2019, NIH funding status was assessed only for those who received a CDA prior to 2019 (n=65).

H-index was chosen as the primary measure of scholarly impact as it represents a reliable method of assessing scholarly impact within otolaryngology.¹⁰ An author’s h-index is defined as the number of their publications (h) with at least h citations. For example, an h-index of 15 indicates that an author has at least 15 publications with a minimum of 15 citations per publication.¹¹ H-index was calculated using the Scopus database (www.scopus.com).

Data Analysis

Statistical analysis was performed using STATA 16 (StataCorp, College Station, Texas, United States) software. Where appropriate, Fisher’s exact tests, Wilcoxon’s rank-sum tests, and Kruskal-Wallis tests were used to assess NIH funding status, NIH funding total, h-index, and gender differences. Due to the skewedness of NIH funding totals and h-indices, results and variabilities were reported as medians and interquartile ranges, respectively.

Population means for academic otolaryngologists were extrapolated from a previous study and compared to academic CDA recipients (n=67) using a one-sample t-test.¹² Both population and sample means are presented in the relevant figures.

Results

Recipient Characteristics

Between 2004 and 2019, 70 investigators at 47 different institutions received CDAs (Table 1). Half of the recipients (50.0%) held advanced degrees in addition to MDs: 17 with MD/PhDs (24.3%) and 18 with MD/master’s (25.7%). Sixteen recipients were identified as practicing within neurotology and otology (22.9%), 15 in head and neck oncology (21.4%), 13 in pediatric otolaryngology (18.6%), 9 in laryngology (12.9%), 9 in rhinology (12.9%), 3 in sleep medicine (4.3%), 3 in general otolaryngology (4.3%), and 2 in plastics and reconstruction (2.9%).

Table 1.

Characteristics of CDA recipients.

Characteristics	n (%)
Total	70 (100.0)
Degree type
MD-only	35 (50.0)
MD/PhD	17 (24.3)
MD/master’s	18 (25.7)
Subspecialty
Neurotology and otology	16 (22.9)
Head and neck oncology	15 (21.4)
Pediatric otolaryngology	13 (18.6)
Laryngology	9 (12.9)
Rhinology	9 (12.9)
Sleep medicine	3 (4.3)
General otolaryngology	3 (4.3)
Plastics and reconstruction	2 (2.9)
Practice setting
Academic practice	67 (95.7)
Other	3 (4.3)
Academic rank (n=67)
Assistant professor	19 (28.4)
Associate professor	31 (46.3)
Full professor	17 (25.4)
Departmental leadership title (n=67)
Vice chair	7 (10.4)
Division chief	7 (10.4)
Residency program director	3 (4.5)

Sixty-seven recipients (95.7%) practiced in an academic setting and the majority (72.9%) remained at the institution where they received their CDA. Of those in academics, 19 were assistant professors (28.4%), 31 associate professors (46.3%), and 17 full professors (25.4%). Several also held departmental leadership titles including vice chair (n=7, 10.4%), division chief (n=7, 10.4%), and residency program director (n=3, 4.5%).

NIH Funding

Of the 65 awardees that received a CDA prior to 2019, 26 (40.0%) obtained NIH grants after the CDA, including 6 who received more than one award (Table 2). Median grant total was $1,061,174 (range $97,669 to $10,288,565). Grant types included 16 K awards, 21 R01 awards, and 10 NIH grants of other types. Sixteen recipients received a K award within a median 1.5 years, and of these, six converted their K awards to an R01 in a median 5.0 years (Figure 1). Meanwhile, 9 investigators received an R01 award without first receiving a K award and half of them did so within 4.0 years of CDA receipt.

Table 2.

NIH funding status, NIH funding total, and h-index by degree type and academic rank.

	NIH Funding Status, n (%)		p-value	Median NIH Funding Total	p-value	Median h-index (IQR)	p-value
	Received	Not received
Total†	26 (40.0)	39 (60.0)		$1,061,174		16 (11–21)
Degree type			0.891		0.109		0.626
MD-only (n=34)	14 (41.2)	20 (58.8)		$1,002,708		16 (10–21)
MD/PhD (n=16)	7 (43.8)	9 (56.3)		$2,249,262		18 (14–20)
MD/master’s (n=15)	5 (33.3)	10 (66.7)		$633,509		15 (8–21)
Academic rank‡			0.945		0.045*		<0.001*
Assistant professor (n=15)	6 (40.0)	9 (60.0)		$592,339		11 (6–15)
Associate professor (n=30)	11 (36.7)	19 (63.3)		$1,522,110		16 (11–19)
Full professor (n=17)	7 (41.2)	10 (58.8)		$1,923,467		23 (18–30)

^†NIH funding status was assessed for CDA recipients prior to 2019 (n=65), NIH funding total for those who received funding (n=26), and h-index for the cohort as a whole (n=70).

^‡Three recipients were not in academic practice, thus not included in the analysis by academic rank: NIH funding status, n=62; NIH funding total, n=24; h-index, n=67.

^*Significant difference, p<0.05.

Figure 1.

Progression of grant acquisition for CDA recipients. Bar graphs indicate number of recipients by award type. R01 awards are divided into number of recipients who received an R01 after K award (bottom bar) and those who received an R01 without a K award first (top bar). Box plots depict distribution of years to award acquisition for: CDA to R01 award (no K award) (green/vertical lines), CDA to K award (blue/checkered), and K to R01 award (purple/diagonal stripes). Arrows are origin independent and represent the time between awards as indicated by arrow base and head positioning. From left to right, black bars on box plots represent minimum, median, and maximum values. Left and right box borders represent 25^th and 75^th percentile, respectively, and dots represent outliers (values 1.5 times the interquartile range above the 75^th percentile or below the 25^th percentile).

Nearly half of NIH funding grantees (46.2%) had advanced degrees. However, having an MD/PhD or MD/master’s was not associated with acquisition of NIH funding (p=0.891) nor was it associated with higher funding total (p=0.109). Moreover, most NIH grant recipients were in academic practice (n=24, 92.3%) and these include 6 assistant professors, 11 associate professors, and 7 full professors. While academic rank was not associated with NIH funding status (p=0.945), median grant total was significantly higher for full professors compared to assistant professors (p=0.022) (Figure 2).

Figure 2.

H-index (blue/solid lines) and NIH funding totals (green/dotted lines) for recipients collectively and by academic rank, as of October 2019. Middle line is the median, and upper and lower box limits are the 75^th and 25^th percentile, respectively. Top and bottom lines represent maximum and minimum values, respectively, and dots represent outliers (values 1.5 times the interquartile range above the 75^th percentile or below the 25^th percentile). The ‘x’ represents the mean h-index of CDA recipients in academics and the ‘△’ represents the mean h-index of all academic otolaryngologists (data extracted from Okafor et al., 2019).¹²

Scholarly Impact

Median h-index among awardees was 16 (IQR 11–21) and was similar between recipients with advanced degrees and those without (p=0.626) (Table 2, Figure 2). Furthermore, median h-indices increased significantly by academic rank (p<0.001): assistant professors 11 (6–15), associate professors 16 (11–19), and full professors 23 (18–30).

Gender Differences

Of the 70 CDAs granted, 23 (32.9%) were awarded to women (Table 3). Women received one to two awards each year between 2004 and 2014, no awards in 2016 and 2017, two in 2018, and four in 2019.

Table 3.

Recipient characteristics and outcomes by gender.

	Women, n (%)	Men, n (%)	p-value
Total	23 (32.9)	47 (67.1)
Degree type			0.754
MD-only	10 (43.5)	25 (53.2)
MD/PhD	6 (26.1)	11 (23.4)
MD/master’s	7 (30.4)	11 (23.4)
Academic rank†			0.893
Assistant professor	7 (31.8)	12 (26.7)
Associate professor	10 (45.5)	21 (46.7)
Full professor	5 (22.7)	12 (26.7)
NIH funding status‡			0.002*
Received	2 (10.5)	24 (52.2)
Not received	17 (89.5)	22 (47.8)
Median NIH funding total§	$3,687,763	$1,002,708	0.124
Median h-index (IQR)	10 (7–16)	17 (12–23)	<0.001*

^†Three recipients were not in academic practice, thus not included in the analysis by academic rank (women, n=22; men, n=45).

^‡Only for CDA recipients prior to 2019 (women, n=19; men, n=46).

^§Only for those who received funding (women, n=2; men, n=24).

^*Significant difference, p<0.05.

Similar proportions of men and women with and without advanced degrees received a CDA (p=0.754). Additionally, the proportion of female recipients promoted to associate (45.5%) and full professors (22.7%) mirrored the proportion of men promoted to each position (46.7% and 26.7%, respectively; p=0.893). However, only 10.5% of women received NIH funding after the CDA compared to 52.2% of male recipients (p=0.002). Moreover, female recipients had significantly lower h-indices than their male colleagues (10 vs. 17, p<0.001) (Figure 3).

Figure 3.

H-index by gender, as of October 2019. Middle line is the median, and upper and lower box limits are the 75^th and 25^th percentile, respectively. Top and bottom lines represent maximum and minimum values, respectively, and dots represent outliers (values 1.5 times the interquartile range above the 75^th percentile or below the 25^th percentile). The ‘x’ represents the mean h-index of CDA recipients in academics and the ‘△’ represents the mean h-index of all academic otolaryngologists (data extracted from Okafor et al., 2019).¹²

Discussion

Each year, the Triological Society grants several Research Career Development Awards to young otolaryngologists to support their research endeavors. This study explored the long-term academic outcomes of recipients as well as gender differences among awardees.

Our analysis found that CDAs have supported researchers from academic institutions across the United States and subspecialties of otolaryngology. Most CDA recipients (96%) remain in academic practice. This is similar to the proportion of physician K awardees at academic centers (90%) and much higher than the proportion of boarded, practicing otolaryngologists in academics (<20%).^12–14 Academic practice affords access to protected research time, research mentors, collaborators, and university resources that are often unavailable in other practice settings. This facilitates further surgeon-scientist career development and likely contributes to CDA recipients remaining in academics.

NIH Funding

After receiving a CDA, 40% of awardees obtained NIH funding. Many were in the form of K awards, and the majority of recipients who obtain a K award did so within 1.5 years after the CDA. Interestingly, 50% of those who obtain an R01 award without first receiving a K award obtained it within 4 years and 25% did so within 1.5 years. This represents a rapid transition from a CDA to an R01 award. A possible explanation may be that some recipients had NIH funding before receiving a CDA, which better positioned them for additional grant funding. However, further analysis is needed to assess factors that contributed to this finding.

Furthermore, an MD/PhD degree was not associated with obtaining NIH funding in this cohort (p=0.891). This differs from findings by Bobian et al. where academic otolaryngologists with PhDs were 8.6 times more likely to receive NIH funding than their MD-only counterparts.¹⁵ This contrast suggests that receiving a CDA may be particularly beneficial for MD-only physicians, possibly giving them the skills, support, and funding needed to match the NIH grant success of their MD/PhD colleagues.

Our study also found that median NIH funding total was greater for full professors compared to assistant professors (p=0.022). This is to be expected as it typically takes many years to attain full professor rank, thus allowing more time for grant accumulation. In addition, while Bobian et al. found NIH funding acquisition was associated with increasing academic rank, our cohort did not reproduce these results (p=0.945).¹⁵ Instead, similar proportions of CDA awardees at each academic rank had NIH funding. We hypothesize that this may be because the CDA mechanism gives early career faculty the resources needed to obtain later NIH funding. However, more research exploring these differences is needed.

Scholarly Impact

Mean h-index was significantly higher among our cohort’s academic otolaryngologists than academic otolaryngologists in general (16.5 vs. 12.5, p<0.001) (Figure 2).¹² This trend persisted for assistant professors (10.3 vs. 5.5, p<0.001) and associate professors (15.9 vs. 10.7, p<0.001), though this was not seen at the full professor rank (24.4 vs. 24, p=0.432). As h-index is often used as a measure of scholarly impact, this highlights that CDA recipients tend to have greater scholarly impact early in their career than the average academic otolaryngologist. While our study design precludes a conclusion of causation, we postulate that the CDA helps contribute to high early career impact, which, in turn, may lead to greater grant attainment and academic rank.

Gender Differences

Gender disparities in academic outcomes are pervasive throughout otolaryngology; women are less likely to be in leadership positions, receive less NIH funding, are less likely to be full professors, and have lower h-indices than men.^12,16–18 Therefore, we sought to determine if these differences were present within the CDA cohort.

Our study found that the proportion of CDAs presented to women (32.9%) was higher than the proportion of active, board-certified female otolaryngologists (17.1%) and of women in academic otolaryngology (23%).^12,14 Moreover, women were equally as likely to be full professors as men in our study (p=0.893) and more likely to be full professors than female academic otolaryngologist in general (23% vs. 16%).¹²

Nevertheless, similar to previous studies, female recipients were significantly less likely to have acquired NIH funding than male recipients (p=0.002) and had significantly lower h-indices than men (p<0.001) (Figure 3).^12,17 This was in spite of the fact that during the first decade of the award, one to two CDAs were consistently presented to women each year, making lead time bias an unlikely primary cause of these differences.

Despite the Triological Society’s actions of awarding CDAs to females at rates higher than the national prevalence of female academic otolaryngologists and despite the remarkable scholarly impact and high academic rank of these women, a gender difference remains. These disparities may be accounted for by other, non-CDA related factors. In previous studies, researchers have identified lack of mentorship, familial responsibilities, and greater appointment to educational activities than research time as potential contributors to gender inequalities in academic achievement.^19–21 However, further research is needed to explore consequent factors in observed gender differences and to identify ways to foster gender equity and promote the research careers of women in otolaryngology.

Limitations

There are several important limitations to this report. First, the cross-sectional nature of the study precludes ascription of academic success specifically to the CDA. Another limitation is that this study was reliant on information listed on academic and private practice profiles and was subject to incorrect or out-of-date information listed on websites. Moreover, our analysis was limited by the use of h-index as a measure of academic success after CDA. H-index is calculated cumulatively over one’s research career and not confined to a specific time period such as after CDA receipt.

Additionally, our study utilized NIH funding as the primary funding outcome and did not assess other non-NIH sources of funding. We did this for several reasons. First, the NIH-RePORTER is a standard reporting measure for NIH grants that facilitates assessment of funding. Moreover, the NIH is the largest contributor to health research in the world and acquisition of its grants is often considered the highest-standard for academic productivity.²² Finally, NIH funding is one of the most commonly used methods of assessing research contribution in medicine, meaning its use as a primary measure of success in our study facilitates comparisons with other studies.^17,23–28 Nevertheless, we recognize that sole NIH-RePORTER assessment likely underrepresents the funding and academic productivity of the CDA recipients (e.g. recipients may have gone on to receive significant funding from agencies outside of the NIH).

Furthermore, the NIH-REPORTER database only maintains funding data for recipients listed as PI/project leader. This may understate academic contributions of CDA awardees as recipients may have contributed significantly to NIH research as co-investigators or may have joined research teams with significant funding, but were not explicitly listed as PI/project leader in the database. However, the NIH-RePORTER is an up-to-date, publicly available, and objective tool that allows for consistent funding appraisal and thus, is a useful for assessing funding.

Conclusion

Overall, recipients of the Triological Society Research Career Development Award tended to remain in academic practice and were often promoted to senior faculty. Many, including MD-only otolaryngologists, obtained NIH funding after receiving a CDA, and as a whole, they had significantly higher h-indices than academic otolaryngologists in general. However, female CDA recipients lag behind their male colleagues in a number of academic outcomes. Therefore, more research is needed to uncover contributors to this difference and to identify ways to foster gender equity in research.