Academic productivity in surgical oncology: Where is the bar set for those training the next generation?

Abstract

Background and Objectives

Promotion and tenure are important milestones for academic surgical oncologists. The aim of this study was to quantify academic metrics associated with rank in surgical oncologists training the next generation.

Methods

Faculty were identified from accredited surgical oncology fellowships in the United States. Scopus was used to obtain the number of publications/citations and h-index values. The National Institutes of Health (NIH) RePORT website was used to identify funding history.

Results

Of the 319 surgeons identified, complete rank information was obtained for 308. The majority of faculty were men (70%) and only 11% of full professors were women. The median h-index values were 7, 17, and 39 for assistant, associate, and full professors respectively. While 50% of full professors have a history of NIH funding, only 26% have had RO1s and 20% have current NIH funding. On multivariate analysis, years in practice, h-index, and a history of NIH funding were associated with academic rank (p<0.05).

Conclusion

Objective benchmarks such as the median h-index and NIH funding provide additional insight for both junior faculty and leadership into the productivity needed to attain promotion to the next academic rank for surgical oncologists.

Introduction

Academic surgery is a competitive field and the promotion and tenure processes are frequently vague without definitive milestones or benchmarks. Young academic surgeons develop their clinical, teaching, and research portfolios in the hopes of moving up in rank so that they are eventually promoted to full professor [1]. Traditionally, academic scholarship has been a major determinant of successful promotion. These contributions have been quantified by the number of publications/citations, impact factor of journals where work is published, and grant funding from the National Institutes of Health (NIH) or other extramural sources [2]. In addition to the traditional tenure stream faculty position, many surgical departments have developed other tracks which focus on educational or clinical contributions to the department [3]. Therefore, other factors such as clinical expertise and volume, teaching prowess, and administrative roles are also considered [4, 5]. Other intangibles like one’s regional or national reputation, character, and leadership skills also go into the selection of candidates for promotion.

Looking only at the number of publications or citations can provide a skewed sense of a researcher’s academic accomplishments. The Hirsch Index (h-index) was developed as a means to quantify the cumulative impact and relevance of a researcher’s work [6]. In this model, the index is equal to h number of papers that have been cited at least h times. This is a tool that has been used in other aspects of medicine and surgery to evaluate individual productivity, and it has also been used on a wider scale to evaluate entire departments or institutions as a whole [7–10]. In fact, the h-index has even been shown to be more predictive of future scientific achievement when compared to number of citations or publications [11].

Within surgical oncology, little data has been published on objective benchmarks for academic promotion [9]. This creates not only a challenging scenario for junior faculty who are actively seeking to be promoted, but also for department and institutional leadership who are evaluating these candidates, especially as compared to their non-clinical tenure-track counterparts. In this study, we sought to quantify metrics associated with academic rank in surgical oncologists who are training surgical oncology fellows at accredited programs.

Methods

All data collection occurred during a two week time period in September 2017.

Faculty Selection

The twenty-two Accreditation Council for Graduate Medical Education (ACGME) accredited surgical oncology fellowships within the United States were identified from the Society of Surgical Oncology (SSO) website (http://www.surgonc.org). From those programs, individual faculty members were identified from departmental websites. Only surgeons trained through a general surgery pathway were included. Faculty trained in otolaryngology, gynecology, neurosurgery, or urology were not included. Those individuals clearly identified as research-only faculty (without a clinical practice) were excluded. Surgeons were identified as instructor, assistant professor, associate professor, or professor.

Demographics

Gender, degree (MD, MD/PhD), and years in practice were recorded for each surgeon. As a surrogate for years in practice, the year of initial American Board of Surgery (ABS) certification was obtained from the ABS website (http://www.absurgery.org). Additionally, it was noted if a respective surgeon’s institution: has a general surgery residency program, is a US News & World Report Top 25 cancer hospital (https://health.usnews.com/best-hospitals/rankings/cancer), or is a National Comprehensive Cancer Network (NCCN) institution (https://www.nccn.org/members/network.aspx).

Bibliometrics

The Scopus database (http://www.scopus.com) was searched by author name to obtain the number of citations, number of publications, and h-index for individual faculty members. In the event of similar author names, the institution affiliations (previous and current) were cross-referenced to ensure the correctness of the gathered data.

NIH Funding

The NIH Reporter database (https://report.nih.gov/) was used to obtain grant funding information. Surgeons were recorded as having any history of NIH funding or current federal funding. For the purposes of this study, any NIH awards (R, T, K, etc) were counted so long as the surgeon was listed as the corresponding principal investigator.

Statistical Analysis

Descriptive statistics were performed for the variables in question. A multivariate analysis was performed using an ordinal regression to analyze factors that contribute to academic rank. A value was considered significant when p<0.05.

All statistical analyses were performed with STATA 15, (College Station, Texas).

Ethics

All of the information obtained for use in this study was obtained from publically available information sources. Therefore, no institutional review board approval was sought.

Results

After querying the SSO website to identify ACGME-approved complex general surgery oncology programs, a total of 22 institutions were identified. From these programs, a total of 319 surgeons were identified and complete rank information was obtained for 308. Of these, there were 103 assistant professors, 83 associate professors, and 122 professors (Table 1). Instructors (n=3) were not included in the final statistical analysis. There were 216 males, which comprised approximately 70% of the total. Notably, amongst the full professors, 108 (89%) were male.

Table 1.

Demographic information for assistant, associate, and full professors

	Assistant (n=103)	Associate (n=83)	Professor (n=122)
Years in Practice, median (SD)	6 (5.4)	12 (6.7)	25 (9.1)
Degree, n (%)
MD	92 (89)	77 (93)	106 (86)
MD PhD	9 (9)	6 (7)	16 (14)
DO	2 (2)
Gender, n (%)
Male	56 (54)	52 (63)	108 (89)
Female	47 (46)	31 (37)	14 (11)
Cancer Center, n (%)
NCCN	62 (60)	48 (58)	67 (55)
Residency, n (%)
General Surgery	56 (54)	40 (48)	65 (53)
US News, n (%)
Top 25	55 (53)	43 (52)	63 (52)

Assistant professors were found to have been in practice for a median of 6 years, while associate and full professors had a median number of years in practice of 12 and 25 respectively. The vast majority of surgeons (89%) had only a MD degree, while 10% had MD PhD’s.

Approximately 52% of surgeons are at a site that has a general surgery residency program. Additionally, 57% of surgeons are employed at NCCN member institutions, and 52% of surgeons are at US News and World Report Top 25 Cancer Hospitals.

For all surgical oncologists in this study, the median number of publications was 49 (range 1–1014). The median number of publications was 16 (range 1–124) for assistant professors, 49 (range 6–225) for associate professors, and 162 (range 12–1014) for professors (Table 2). In total, the median number of citations was 1303 (range 1–73,459). The median number of citations was 220 (range 1–3385) for assistant professors, 1229 (range 57–12,899) for associate professors, and 6033 (range 166–73,459) for professors (Table 2). The median h-index was 17 (range 1–138) for the entire sample of surgeons. The median h-index was 7 (range 1–30) for assistant professors, 17 (range 3–44) for associate professors, and 39 (range 6–138) for professors (Table 2).

Table 2.

Quantitative bibliometric data for assistant, associate, and full professors

	Assistant (n=103)	Associate (n=83)	Professor (n=122)	All Faculty (n=308)
Publications, median (range)	16.0 (1–124)	49.0 (6–225)	161.5 (12–1014)	49 (1–1014)
Citations, median (range)	220.0 (1–3385)	1229.0 (57–12899)	6032.5 (166–73459)	1303 (1–73459)
h-index, median (range)	7.0 (1–30)	17.0 (3–44)	39.0 (6–138)	17 (1–138)

A total of 28% of all surgeons have a history of NIH funding, with only 13% being currently funded. Fifty-one percent of those at the full professor rank have a history of funding (Figure 1).

Figure 1.

NIH Funding Across Academic Ranks.

Funding histories were obtained for surgeons to evaluate if there was any history of NIH funding, a history of R01 award, or a current NIH grant. The percentage of surgeons (within each academic rank) are displayed on the y-axis.

Univariate analysis demonstrated that gender, years in practice, h-index, and NIH funding correlated with academic rank (p<0.05) (Table 3). A subsequent multivariate analysis demonstrated that years in practice (coefficient: 0.01, 95% CI: 0.08–0.12), h-index (coefficient: 0.07, 95% CI: 0.05–0.08), and history of NIH funding (coefficient: 0.54, 95% CI: 0.01–0.98) were the only variables with a statistically significant association with academic rank (p<0.05) (Table 3).

Table 3.

Univariate and multivariate analysis of factors affecting academic rank

	Univariate			Multivariate
Factor	Coefficient	95% CI	p	Coefficient	95% CI	p
Gender
Male	0.76	0.49 to 1.03	< 0.01	0.38	−0.01 to 0.68	0.56
Female
Degree
MD	1.38	−0.26 to 3.02	0.10	−0.26	−3.33 to 2.81	0.87
MD PhD	1.52	−0.16 to 3.21	0.08	−0.12	−3.23 to 2.98	0.94
Years in Practice	0.13	0.11 to 0.15	< 0.01	0.01	0.08 to 0.12	< 0.01
h-index	0.09	0.07 to 0.11	< 0.01	0.07	0.05 to 0.08	< 0.01
NIH funding
Yes	1.24	0.93 to 1.55	< 0.01	0.54	0.01 to 0.98	0.02
No

Discussion

Getting promoted to the next highest academic rank is a significant milestone in the career of any surgical oncologist. Unfortunately for many individuals, the process surrounding these decisions can be somewhat unclear and lack an element of objectivity. In this study, we sought to identify objective criteria that were associated with academic rank among those surgical oncologists at ACGME-approved complex general surgical oncology fellowship sites.

As might be expected, there was a stepwise increase in productivity with ascending academic rank. This trend was apparent for the number of citations and publications, h-index, history of NIH funding, and current NIH funding. For example, the median h-index for an assistant professor is 7, while that of a professor is 39. The design of this study did not allow for capturing the h-index at the time of an individual surgeon’s academic promotion, but we believe that the median values obtained from each rank still provide critical insight to better understand the framework for the promotion and tenure processes. Additionally, 5% of assistant professors have a history of obtaining NIH funding, while 51% of professors have had NIH funding at some point in their career. Furthermore, our multivariate analysis demonstrated that years in practice, h-index, and history of NIH funding were significantly associated with academic rank. Here, the most influential factor associated with rank was NIH grant funding.

Since it was first published in 2005, the h-index has been widely adopted as a measure of the overall impact of an individual’s research contributions. According to Hirsch, to have a h-index of 12, a researcher must have 12 publications with at least 12 citations each, while the remainder of his/her publications will have ≤ 12 citations [6]. The index has the advantage of being relatively insensitive to lowly cited articles, while it is easily computed with existing citation databases [12]. Since it has gained a reputation for being an objective measure of scholarship, several surgical subspecialties have demonstrated a direct correlation between the h-index and increasing academic rank [9, 13–16]. Table 4 provides a sample of h-index values with respect to academic rank across various surgical subspecialties [8, 10, 15, 17, 18]. While it is difficult to compare h-index values across different disciplines [19], this table highlights the varying degrees of academic productivity necessary to achieve high academic standing within select specialties. This highlights the importance of evaluating surgical oncologists amongst their respective colleagues rather than an entire department of surgery or academic institution.

Table 4.

Published data of mean h-index as a function of academic rank across a sampling of surgical specialties

Author	Year	Specialty	Assistant	Associate	Professor
Therattil	2016	Plastic Surgery	4.6	9.1	15.3
Svider	2013	Otolaryngology	4.6	8.1	15.6
Benway	2009	Urology	8.0	13.8	21.9
Susarla	2017	Craniofacial Surgery	5.6	9.5	18.8
Khan	2013	Neurosurgery	10.3	14.9	28.1
LaRocca	2018	Surgical Oncology	8.4	18.8	43.5

The h-index itself has been subject to certain criticisms. For example, it does favor older surgeons as they will have had a longer time interval to accumulate numbers of citations and publications [20]. Additionally, it does not take into account order of authorship or type of publication into account [12, 21]. Furthermore, there have been concerns that self-citation can affect one’s h-index; however, this has not been demonstrated to be a reproducible finding [22].

At some institutions, obtaining NIH or other extramural funding is a requirement for promotion to higher academic ranks. R01 grants are particularly impressive to department leadership and promotion committees, as they are often used as a marker of a researcher’s independent scientific scholarship. However, being awarded an R01 or other federal funding is growing more difficult for surgeons. Multiple recent studies have demonstrated that surgical grant proposals are less likely to get funded (compared to non-surgical proposals), and if they are funded it is often for a lesser amount [23–25]. While there may be increasing barriers for surgeon-scientists, being able to successfully obtain funding is associated with increasing academic rank and scholarly impact, which has been shown across multiple specialties [26–29]. Additionally, not only has research funding of an individual been linked to high academic rank, but the funding history of a surgical chairperson has also been linked to the degree of departmental NIH funding [30].

The relative value unit (RVU) is one method to gauge the clinical productivity of a physician or surgeon. In many practices, there are incentives or bonuses for those individuals who perform above a certain threshold on an annual basis. In recent years, there has been discussion regarding potential ways to incentivize academic productivity among surgical faculty [31, 32]. To this end, many centers have developed a version of an “academic” relative value unit (aRVU) that seeks to parallel that of its clinical counterpart [33]. This trend is not unique to surgery, as many other specialties have tried to develop methods to increase academic productivity and reward successful efforts [34, 35]. For example, one academic radiology department noted that the components of their aRVU included include teaching, publications, administrative, and research funding. The total aRVU was then calculated by weighing the percent effort for a given academic contribution as well as the assigned academic value (which was decided upon by departmental leadership) [36]. While each aRVU system must be adjusted to the unique needs and goals of a surgical department, it may provide another objective means to quantify academic productivity and a serve as a tool to facilitate the promotion process.

One of the trends that became apparent with increasing academic rank was a widening gender disparity. At the assistant professor level, there were 56 males and 47 females. In the associate professor rank, there were 52 males and 31 women. However, at the professor level, there were 108 males and 14 women. This observation has been reported across multiple medical specialties (including surgery), and more research will be needed to determine the underlying contributing factors within the field of surgical oncology [37–39].

There are limitations to this study. The data set is based upon publically available information across multiple websites. We are unable to control the frequency with which those websites are updated and the overall accuracy and completeness of the information that they contain. While we made every effort to capture the data in a limited time period to obtain a “snapshot,” there still is some potential bias.

To obtain bibliometric information, there are multiples sources that can be used. The three broadly available and searchable databases include Scopus, Web of Science, and Google Scholar. In general medical journals, these three databases produced qualitatively and quantitatively different citation counts [40]. None of the search mechanisms will capture all of the publication and citations for any individual faculty member [41], and the h-index values will differ slightly based upon the databased that is being queried.

The NIH Reporter website only listed grant information from 1992 and onward, so it is possible that certain grants for more senior faculty were not captured. Additionally, faculty may have had grants from other extramural funding sources like the Department of Defense (DOD) or research organizations such as the American Association of Cancer Research (AACR). However, unlike NIH funding, there is no easily searchable database from which to gather such funding information from these organizations.

Conclusion

Academic surgical oncologists devote themselves to clinical care of patients, teaching, scholarship, and other leadership and administrative roles. The promotion process is not always clear cut, and objective benchmarks have not been well studied in the field of surgical oncology. We demonstrated that academic rank is associated with a surgeon’s number of years in practice, h-index, and history of NIH grant funding. These objective measures may help to provide some insight and clarity for this promotion process.

Synopsis.

Academic promotion and tenure are important milestones for surgical oncologists. In this study, we analyzed the academic productivity of surgical oncology fellowship faculty to identify objective factors that are associated with academic rank.