Academic Surgical Oncologists’ Productivity Correlates with Gender, Grant Funding, and Institutional NCI Comprehensive Cancer Center Affiliation

Abstract

Background.

A scholar’s h-index is defined as the number of h papers published, each of which has been cited at least h times. We hypothesized that the h-index strongly correlates with the academic rank of surgical oncologists.

Methods.

We utilized the National Cancer Institute (NCI) website to identify NCI-designated Comprehensive Cancer Centers (CCC) and Doximity to identify the 50 highest-ranked general surgery residency programs with surgical oncology divisions. Demographic data of respective academic surgical oncologists were collected from departmental websites and Grantome. Bibliometric data were obtained from Web of Science.

Results.

We identify 544 surgical oncologists from 64 programs. Increased h-index was associated with academic rank (p < 0.001), male gender (p < 0.001), number of National Institutes of Health (NIH) grants (p < 0.001), and Affiliation with an NCI CCC (p = 0.018) but not number of additional degrees (p = 0.661) or Doximity ranking (p = 0.102). H-index was a stronger predictor of academic rank (r = 0.648) than total publications (r = 0.585) or citations (r = 0.450).

Conclusions.

This is the first report to assess the h-index within academic surgical oncology. H-index is a biblio-metric predictor of academic rank that correlates with NIH grant funding and NCI CCC Affiliation. We also highlight a previously unexpected and unappreciated gender disparity in the academic productivity of US surgical oncologists. When academic rank was accounted for, female surgical oncologists had lower h-indices compared with their male colleagues. Evaluation of the etiologies of this gender disparity is needed to address barriers to academic productivity faced by female surgical oncologists as they progress through their careers.

Research productivity often is used as a criterion for decision-making in faculty recruitment, promotion, compensation, tenure, and grant support.^1–3 Traditionally, scientific eminence has been quantified by total publication or citation count. However, these cumulative metrics do not sufficiently evaluate the quality of one’s scholarly output. For example, total publication count does not account for journal impact factor, and total citation count may be skewed if an author has one highly cited paper amongst other less frequently cited articles. Consequently, Dr. Jorge Hirsch from UC San Diego developed the h-index to assess research output.⁴

The h-index is defined as the number of h papers published, each of which has been cited at least h times.⁴ Thus, an author with an h-index of ten has published ten papers that have each been cited at least ten times. Hirsch has demonstrated that the h-index is superior in predicting future scientific productivity when compared with total publication and citation count.⁵

Since its advent, the h-index has been analyzed in the setting of various medical specialties including otolaryngology, plastic surgery, neurosurgery, orthopedic surgery, craniofacial surgery, urology, radiology, radiation oncology, and emergency medicine.^6–15 Although the h-index has been validated as a predictor of academic rank within these given fields, it varies considerably across medical specialties.^6–15

To date, no studies have assessed the h-index within academic surgical oncology. Thus, the primary objective of this cross-sectional study is to evaluate the association between surgical oncologists’ h-indices and academic rank. We hypothesized that the h-index more strongly correlates with the academic rank of surgical oncologists than total publications or total citations. In addition, we characterized associations between the h-index and various personal and institutional factors including: gender, number of additional degrees, number of National Institutes of Health (NIH) grants, National Cancer Institute Comprehensive Cancer Center (NCI CCC) Affiliation, and Doximity ranking.

METHODS

Identification of Study Sample

We obtained a list of all NCI-designated CCCs from the NCI website and queried Doximity Residency Navigator to identify the 50 highest-ranked general surgery residency programs with surgical oncology divisions according to research output. Programs from all US regions and training environments (urban vs. rural) were included. Programs lacking an academic ranking system were excluded (Fig. 1a).^16,17

FIG. 1.

Identification algorithm of study sample. a Venn diagram of study sample (n = 64 programs). b CONSORT flow diagram of study sample (n = 544 surgical oncologists)

Next, we searched through these programs’ departmental websites to identify respective academic surgical oncologists. For programs with separate Divisions of Surgical Oncology and Breast Surgery, faculty from both divisions were included. Inclusion criteria encompassed full-time tenure-track faculty with an MD or DO degree. Lecturers/instructors, adjunct faculty, and faculty emeriti were excluded.

Bibliometric Variables

We used Web of Science to collect each surgical oncologist’s h-index, total publications, and total citations. In cases of duplicate author entries, we cross-referenced each surgeon’s education and training history to refine the results.¹⁸

Demographic Variables

We examined individual surgical oncologists’ personal attributes (academic rank, gender, number of additional masters and/or doctorate degrees, total number of past and/ or current NIH grants), as well as variables which described their institutions (NCI CCC Affiliation, program rank). Demographic data were collected from departmental faculty profile pages, Grantome, the NCI website, and Doximity, respectively.¹⁹

Statistical Analysis

Descriptive analyses were computed to summarize the data. Bivariate analyses (ANOVA, Chi square) were performed to determine associations between variables. Multivariate regression analysis was performed to control for confounding factors and confirm statistical significance of variables. All statistical analysis was performed using SPSS, with statistical significance set as p < 0.05.

RESULTS

We identify 544 surgical oncologists from 64 programs who met the inclusion criteria (Fig. 1b), with an overall mean h-index of 21 ± 17 (median: 17, range: 0–111). The following descriptive and bivariate analyses are summarized in Table 1.

TABLE 1.

H-index distribution of US surgical oncologists by personal and institutional demographic variables

Demographic variable	H-index
	Mean ± SD	Median	Range	p value
Personal
Academic rank
Assistant professor (n = 186)	7.9 ± 5.5	7	0–34	< 0.001
Associate professor (n = 128)	18 ± 8.9	17	3–52
Professor (n = 142)	32 ± 18	28	2–107
Division chief (n = 68)	33 ± 17	29	7–84
Department chair (n = 20)	48 ± 26	42	8–111
Gender
Male (n = 331)	26 ± 19	21	0–111	< 0.001
Female (n = 213)	13 ± 11	11	0–78
Additional degrees
0 (n = 412)	22 ± 18	17	0–111	0.661
1 (n = 122)	18 ± 13	15	1–65
2 (n = 9)	31 ± 18	28	4–63
3 (n = 1)	33	33	33
NIH grants
0 (n = 369)	15 ± 12	12	0–84	< 0.001
1–5 (n = 152)	32 ± 20	28	2–111
6–10 (n = 14)	43 ± 20	37	24–97
> 10 (n = 9)	48 ± 11	47	27–67
Institutional
NCI-designated comprehensive cancer center affiliation
Yes (n = 438)	22 ± 18	17	0–107	0.018
No (n = 106)	17 ± 17	13	0–111
Doximity rank*
1–10 (n = 114)	22 ± 18	17	0–107	0.102
11–20 (n =104)	22 ± 18	18	0–111
21–30 (n = 77)	20 ± 18	16	0–83
31–40 (n = 73)	17 ± 14	16	0–86
41–50 (n = 63)	20 ± 16	15	0–64

*Rank of general surgery residency programs with divisions of surgical oncology

Academic Rank

Mean h-index significantly increased with academic rank (p < 0.001): 7.9 ± 5.5 (median: 7, range: 0–34) for assistant professors (n = 186); 18 ± 8.9 (median: 17, range: 3–52) for associate professors (n = 128); 32 ± 18 (median: 28, range: 2–107) for professors (n = 142); 33 ± 17 (median: 29, range: 7–84) for division chiefs (n = 68); and 48 ± 26 (median: 42, range: 8–111) for department chairpersons (n = 20). In addition, there was a stronger association between academic rank and h-index (r = 0.648) than total publications (r = 0.585) or total citations (r = 0.450; Table 2).

TABLE 2.

Association between bibliometric measures and academic rank

Bibliometric measure	Pearson's r	p value
H-index	0.648	< 0.001
Total publications	0.585	< 0.001
Total citations	0.450	< 0.001

Gender

Of the 544 surgical oncologists, 331 were male (61%) and 213 were female (39%). The mean h-index was 26 ± 19 (median: 21, range: 0–111) for male surgeons versus 13 ± 11 (median: 11, range: 0–78) for female surgeons. Male gender and h-index were significantly associated (p < 0.001, r = 0.362).

Stratification by academic rank revealed that male faculty had higher median h-indices than female faculty across all ranks. There was a significant association between gender and h-index at the levels of assistant professor (p < 0.001), associate professor (p = 0.006), and professor (p = 0.002) but not division chief (p = 0.305) or department chairperson (p = 0.115). The h-index distribution of male and female faculty by academic rank is summarized in Fig. 2 and Supplemental Table 1. This gender discrepancy in h-index may be attributed to the fact that male faculty also had higher median total publications and citations across all ranks. There was a significant association between gender and total publication and citation counts at the levels of assistant professor (p < 0.001; p = 0.008), associate professor (p = 0.010; p = 0.049), and professor (p = 0.007; p = 0.041), but not division chief (p = 0.533; p = 0.481) or department chairperson (p = 0.096; p = 0.244). Supplemental Fig. 1a, b and Supplemental Table 2a, b show the total publications and citations of male and female faculty by academic rank, respectively.

FIG. 2.

Median h-indices of male and female faculty by academic rank

Additional Degrees

The mean h-indices were 22 ± 18 (median: 17, range: 0–111) for faculty with no additional degrees (n = 412); 18±13 (median: 15, range: 1–65) for one additional degree (n = 122); 31 ± 18 (median: 28, range: 4–63) for two additional degrees (n = 9); and 33 (median: 33) for three additional degrees (n = 1). Number of additional degrees and h-index were not significantly associated (p = 0.661).

NIH Grants

The mean h-indices were 15 ± 12 (median: 12, range: 0–84) for faculty with no NIH grants (n = 369); 32 ± 20 (median: 28, range: 2–111) for 1–5 NIH grants (n = 152); 43 ± 20 (median: 37, range: 24–97) for 6–10 NIH grants (n = 14); and 48 ± 11 (median: 47, range: 27–67) for < 10 NIH grants (n = 9). Number of NIH grants and h-index were significantly associated (p < 0.001, r = 0.379).

NCI Comprehensive Cancer Center Affiliation

Of the 64 identify programs, 48 (75%) were affiliated with an NCI CCC. The mean h-index of the 438 surgical oncologists with NCI CCC Affiliation was 22 ± 18 (median: 17, range: 0–107). In comparison, the mean h-index of the 106 surgical oncologists without NCI CCC Affiliation was 17 ± 17 (median: 13, range: 0–111). NCI CCC Affiliation and h-index were significantly associated (p = 0.018, r = 0.101).

Program Rank

The bivariate analysis of h-index and program rank only included the 50 highest-ranked general surgery residency programs with divisions of surgical oncology from Doximity. The mean h-indices were 22 ± 18 (median: 17, range: 0–107) for faculty from programs ranked 1–10 (n = 114); 22 ± 18 (median: 18, range: 0–111) for rank 11–20 (n = 104); 20 ± 18 (median: 16, range: 0–83) for rank 21–30 (n = 77); 17 ± 14 (median: 16, range: 0–86) for rank 31–40 (n = 73); and 20 ± 16 (median: 15, range: 0–64) for rank 41–50 (n = 63). Program rank was not significantly associated with h-index (p = 0.102).

Multivariate Analyses

To confirm the significance of academic rank, gender, NIH grants, and NCI CCC Affiliation, multivariate regression analysis was performed with the h-index as the dependent variable. Academic rank (p < 0.001), gender (p < 0.001), NIH grants (p < 0.001), and NCI CCC Affiliation (p = 0.012) all remained significantly associated with h-index (Table 3).

TABLE 3.

Multivariate regression analysis of the association between h-index and academic rank, gender, NIH grants, NCI Comprehensive Cancer Center (CCC) affiliation

Dependent variable	Independent variable	p value
H-index	Academic rank	< 0.001
	Gender	< 0.001
	NIH grants	< 0.001
	NCI CCC affiliation	0.012

DISCUSSION

Research productivity is a hallmark of career advancement in academic surgery; therefore, it is critical to standardize the bibliometric tools that are used to guide faculty promotions across institutions. Traditionally, institutional promotions committees, funding agencies, and surgical societies have utilized total publication count to measure productivity. However, this fails to account for the quality, impact, and acceptance of an individual’s body of work within the academic community. Thus, we evaluated the h-index as a stronger bibliometric predictor of academic rank than total publication or citation count.

We now demonstrate that h-index was the most accurate bibliometric predictor of academic rank in a cohort of academic surgical oncologists. In fact, h-index had a stronger association with academic rank than total publications and citations by 11 and 44%, respectively. This supports Hirsch’s finding that the h-index was a superior predictor of future research productivity compared to total publications and total citations.^4,5 We also evaluated the association between the h-index and several individual and institutional demographic variables. The h-index was significantly associated with gender, number of NIH grants, and NCI CCC Affiliation, but not number of additional degrees or Doximity program ranking. Multivariate analyses confirmed the significance of academic rank, gender, NIH grants, and NCI CCC Affiliation when all other variables were accounted for.

The finding that gender predicts h-index in academic surgical oncology highlights limitations of this metric. Our study aligns with previous reports demonstrating that male healthcare providers (i.e., general surgery, gynecologic oncology, radiation oncology, urology, otolaryngology, gastroenterology, anesthesiology, pediatrics, and psychology) and STEM (science, technology, engineering, and mathematics) researchers (i.e., ecology, evolutionary biology, industrial engineering and molecular biology) have significantly higher h-indices than their female colleagues across professorial ranks (all p < 0.05), but is the first to analyze this gender disparity within academic surgical oncology.^10,20–28 The etiologies underlying all of these disparities are likely multifactorial. First, the number of citations an article receives has been shown to correlate with male gender of authors.^29–31 One potential explanation for this is the phenomenon of research ‘‘networking,’’ as investigators tend to have networks comprised of their same gender.³² Thus, male trainees and faculty in male-dominated fields such as academic surgical oncology (69% male according to our data) may have increased access to collaborators and mentors compared to their female colleagues. In turn, this has been associated with higher rates of inclusions as co-authors on publications, as well as more citations from peers as noted above.^32,33 This correlation between gender and research networking potentially creates a positive feedback cycle that reinforces the gender disparity in academic surgical oncology. Additionally, females are less likely to apply for grant support, have lower success rates in obtaining funding, and are awarded smaller sums than their male counterparts.^25,34 It also is possible that female faculty may have authored prior publications under their maiden names earlier on in their careers, which may not be accounted for in their h-index calculations.³⁵ Female physicians’ increased time commitments to home and child-bearing and rearing responsibilities, coupled with male physicians’ longer career durations, offer alternative potential explanations for this discrepancy in academic productivity.^{20,21,23,36,37} Finally, another explanation may be that females are more likely to pursue career advancement through clinical excellence and teaching responsibilities, which are more difficult to objectively quantify and are not accounted for in the h-index. For example, lower rates of academic productivity among female surgical oncologists may originate from differential career choices of faculty track (i.e., research vs. clinical track).^38,39 Thus, further studies are needed to better understand and mitigate gender disparities in the academic productivity of academic surgical oncologists. The h-index needs to be carefully weighed in assessing academic productivity given that one major unmodifiable attribute, namely gender, independently factors into this bibliometric parameter. By solely relying upon it, we risk ‘‘perpetuating, exacerbating, or even exploiting’’ existing gender disparities while undervaluing the contributions of those who excel at teaching and clinical excellence.²⁶ Still, the data should not be dismissed as it may serve as an important metric for evaluating how successful academic programs are at nurturing the careers of all young surgeons equally.

Other studies have also examined the impact of NIH funding and advanced graduate degrees on surgical faculty’s bibliometrics.³ Valsangkar et al. found that the presence of a PhD degree significantly correlated with increased publications and citations for assistant professors (p < 0.001), associate professors (p < 0.001), and professors (p < 0.05).³ The same study also determined that history of NIH funding significantly correlated with increased academic productivity.³ Our results also show that the number of NIH grants was significantly associated with h-index. This suggests a cyclical process, wherein increased NIH funding leads to greater academic productivity, which in turn increases the success rate of obtaining more grant support. This underscores the importance of early career development awards, such as NIH T- and K-series awards, to jumpstarting one’s funding potential. However, contrary to this study, which reported an increase in total publications and citations with an additional PhD degree, we found that the number of additional degrees was not significantly associated with the h-index. One potential explanation is that we included all masters and doctorate degrees, rather than just PhD degrees, which are traditionally more heavily research-oriented.

We also analyzed how institutional factors beyond personal attributes contribute to surgical oncologists’ academic productivity. NCI CCC Affiliation was significantly associated with h-index, which is consistent with the fact that NCI CCC designation is dependent upon expertise in laboratory, clinical, and behavioral/populations-based research.¹⁶ Taken together, our findings suggest that surgical oncologists at high volume research institutions (i.e., those with NCI CCC Affiliation) with high volume NIH funding produce more robust and impactful research (i.e., higher h-index).

Our findings also provide insight into how the h-index distribution within surgical oncology compares to other medical specialties. The h-indices of surgical oncologists appear to be higher across all academic ranks compared to neurosurgeons, otolaryngologists, plastic surgeons, radiation oncologists, and orthopedic surgeons.^6–8,12,14 This aligns with a previous report that faculty from the Division of Surgical Oncology were among the most academically productive within their Departments of Surgery, as measured by total publications, citations, and NIH funding.³ Surgical oncologists’ higher h-indices likely reflect the presence of institutional research resources (within cancer centers), more readily available external funding including federal and foundational support, and perhaps the advent of genomic technologies and personalized oncotherapeutics.⁴⁰

Overall, there were several strengths unique to the present study. All data were collected within a 2 week period (July 2017) from a single database to minimize discrepancies across the study sample. Web of Science advantageously indexes publications dating back to 1900, whereas SCOPUS only includes publications from 1995 onwards. In cases of ambiguous authorship due to common names, we carefully refined the results by cross-referencing each surgeon’s education and training history. Moreover, we advanced the discussion by determining that both personal attributes and institutional factors are significant predictors of surgical oncologists’ academic productivity. The present study also highlights a previously unappreciated gender disparity in academic surgical oncology and underscores the need for further exploration of this phenomenon.

There were limitations to our study that warrant consideration. The cross-sectional design only provides a snapshot of the data, but the h-index is continually in flux as one’s work becomes increasingly cited. Additionally, the frequency at which departmental webpages are updated is variable and cannot be controlled for. Since we utilized Web of Science, comparisons to other studies that referenced SCOPUS may not be exact. However, we did not have the option to reference SCOPUS due to a lack of UC San Diego institutional access. We also acknowledge the inherent limitations of the h-index, which does not account for authorship position nor the practice of self-citation. However, Rad et al. determined that the h-index did not change for 77% of authors as a result of self-citation.¹¹ Similarly, Lee et al. performed a weighted calculation of the h-index based on authorship position (full credit for first and last author; half credit for second author; quarter credit for other) and found no significant difference between weighted and nonweighted h-indices, with a correlation coefficient of 0.99 between the two.⁷ Despite these limitations, we provide several new insights into the landscape of academic productivity and professorial rank in surgical oncology.

CONCLUSIONS

Our results demonstrate that the h-index is strong bibliometric predictor of surgical oncologists’ academic rank. In addition, we highlight associations between the h-index and other demographic factors that are personal (gender and number of NIH grants) and institutional (NCI CCC Affiliation) in nature. Given the h-index’s association with gender, its application as a measure of academic productivity should be exercised with caution. While it serves as a marker of one’s scientific impact, it should be used alongside evaluation of clinical acumen, teaching excellence, and administrative leadership to guide faculty promotions in academic surgical oncology. Future studies are needed to further evaluate and counteract the factors contributing to this newly identify disparity in academic productivity between male and female surgical oncologists.