The Impact of Fellowship Training on Peer-Reviewed Publication Productivity in Academic Anesthesiologists as Evaluated by the h-Index

Abstract

Background:

The effect of anesthesiology fellowships on scholarly contributions has been minimally studied. In this study we analyzed differences in h-index between fellowship-trained and non–fellowship-trained anesthesiologists, as well as by type of fellowship, academic rank, and years in practice.

Methods:

All anesthesiologists on staff between September 1, 2021, and August 31, 2022, were included in this study. The variables collected were fellowship training status, h-index, total number of publications, years in practice, academic rank, and years at the institution. For analysis, the anesthesiologists were divided into 2 groups: those with fellowship training and those without.

Results:

Among 78 anesthesiologists, 40 were not fellowship-trained and 38 were, with 10 types of anesthesiology fellowships identified. The h-index and number of publications did not differ between fellowship-trained and non–fellowship-trained anesthesiologists, and the number of publications per years in practice did not differ by fellowship type. The number of publications per years in practice was higher in the fellowship-trained group than in the non–fellowship-trained group (1.2 ± 1.1 vs. 0.71 ± 0.6; P = .04), as was the number of publications per years in practice at our institution (1.5 ± 1.1 compared with 0.9 ± 0.9; P = .0093).

Conclusion:

Fellowship training among academic anesthesiologists was not associated with a difference in h-index. However, fellowship training was associated with a higher number of publications per years in practice. Further research could elucidate the usefulness of h-index to support career development and contributions of anesthesiologists in academia.

Introduction

Fellowship training is designed to provide increased expertise in a field, including scholarly work such as research. Accreditation Council for Graduate Medical Education (ACGME)-accredited fellowships in the field of anesthesiology are increasing, including fellowships in adult and pediatric cardiac anesthesiology, critical care, pain medicine, pediatric anesthesiology, regional and acute pain, obstetric anesthesiology, addiction medicine, clinical informatics, and hospice and palliative medicine.¹ There are also many non–ACGME-accredited anesthesiology fellowships. Those who complete a fellowship may choose to then practice in the academic setting, where promotion is influenced by a multitude of factors: clinical service, research, grant funding, teaching, mentoring, leadership, administrative duties, and type and length of service. Scholarly publication productivity is weighed heavily as an important component of the contribution to academics and thus may greatly influence promotion.² Whether subspecialty training in anesthesiology results in increased publication contributions is therefore of interest because it can influence recruitment, anesthesiologist appointments, professional growth, and promotions.

However, the measurement of academic activity is not quantitatively well defined. One proposed metric to evaluate the comprehensive impact and quality of research output by an individual is the h-index.³ Hirsch proposed the h-index in 2005 to quantify the cumulative impact of an individual’s academic output. Although Hirsch developed the h-index specifically for physicists, he suggested that it could also be useful for other fields, including biological sciences.³ The h-index is a bibliometric statistic calculated as the resulting number h of the author’s publications having at least h citations each. Therefore, the h-index is primarily determined by the number of times a publication is cited, which presumably would reflect the quality of the collective work and its impact. While the h-index is one of the most frequently utilized metrics in quantifying scholarly productivity, it does have limitations, such as favoring researchers who have been in their field longer who have had more time to publish.⁴

Analyzing the h-index of anesthesiologists who are fellowship-trained compared with those who are not fellowship-trained is of interest because of the increasing number of specialized fellowships. Clinical anesthesia fellowship training programs are in addition to the 3-year anesthesiology residency and generally entail monthly rotations in clinical specialties under the guidance of a program director. In addition, fellowships require the completion of an academic project such as a published manuscript or case series as part of the program requirements. We hypothesized that fellowship-trained anesthesiologists have higher publication productivity, as determined by the h-index, compared with non–fellowship-trained anesthesiologists at a highly specialized US tertiary cancer center. In the current study, we analyzed differences in h-index between clinical fellowship-trained and non–fellowship trained anesthesiologists, as well as among different types of fellowships in anesthesiology and by departmental rank, years in practice, and years at the study institution.

Methods

This retrospective study was approved by the Investigational Review Board at The University of Texas MD Anderson Cancer Center, Houston, Texas (protocol 2022-0836). Our study cohort comprised active faculty anesthesiologists at MD Anderson from September 1, 2021, to August 31, 2022, who had participated in an ACGME or non-ACGME clinical fellowship training program lasting for 1 year. These anesthesia fellowships included cardiovascular, pediatric, critical care, regional anesthesia and acute pain medicine, cancer anesthesia, pain management, general clinical anesthesia, neuroanesthesia, graduate medical education, and combined obstetric and liver. Given that the inherent structure and duration of research fellowships often differ from that of clinical fellowships, anesthesiologists who had participated in research fellowships were excluded from our analysis.

The primary objective of our investigation was to compare the h-index between fellowship-trained and non–fellowship-trained anesthesiologists. Secondary objectives were to evaluate (1) the mean number of publications per years in total practice and per years in practice at MD Anderson, (2) the h-index associated with specific types of fellowships, and (3) h-index based on academic rank. Each faculty member’s h-index and number of publications were determined by the public Scopus website. Faculty members’ curricula vitae were used to determine their years in practice and to confirm and validate publications. Departmental databases were used to determine their years of employment at MD Anderson and academic rank (ie, assistant professor, associate professor, or professor).

Anesthesiologists were grouped into fellowship-trained anesthesiologists and non–fellowship-trained anesthesiologists, and subject characteristics were summarized using descriptive statistics. The Fisher exact test was used to evaluate the association between 2 categorical variables, and the Wilcoxon rank-sum test or Kruskal-Wallis test was used to compare location parameters of continuous distributions between or among subject groups. Spearman correlation coefficient was used to evaluate the correlation between 2 continuous variables. Statistical software SAS 9.4 (SAS, Cary, NC) was used for all analyses.

Results

A total of 78 anesthesiologists were included in the analysis, of which 38 were fellowship-trained and 40 were not. Fellowship types identified are summarized in Table 1. Among the 10 different fellowships identified, the most common fellowship training was in cardiovascular anesthesia (n=13). Other fellowships included pediatric anesthesia (n=6), critical care (n=4), regional anesthesia (n=4), cancer anesthesia (n=3), pain management (n=3), general clinical anesthesia (additional year, n=2), neuroanesthesia (n=1), obstetric/liver anesthesia (n=1), and graduate medical education (additional year, n=1). In our analysis, we combined the graduate medical education, obstetric/liver, neuroanesthesia, cancer anesthesia, and general clinical anesthesia fellowships into a single group (other), and the regional anesthesia and pain management fellowships were also combined into 1 group.

Table 1.

Characteristics of anesthesiologists included in our study (N=78)

Characteristic	No. (%)
No fellowship	40 (51)
Fellowship	38 (49)
Critical care	4 (11)
Cardiovascular	13 (34)
Other	8 (21)
Pediatric	6 (16)
Regional/pain	7 (18)
Current rank
Assistant	8 (10)
Associate	30 (38)
Professor	40 (51)

In the entire cohort, overall, the h-index (mean ± SD) was 6.4 ± 4.8, and the number of Scopus publications was 15.2 ± 14.9. The other academic characteristics of the entire cohort are summarized in Table 2. Academic characteristics by fellowship training status are summarized in Table 3. The mean ± SD h-index of the fellowship-trained group (6.5 ± 5.0) was not statistically different (P = .94) from that of the non–fellowship-trained group (6.3 ± 4.6). There was also no statistical difference in the mean number of Scopus publications between the fellowship-trained group (17.8 ± 18.1) and the non–fellowship-trained group (12.8 ± 10.7; P = .329). However, statistically significant differences were found in the number of publications (mean ± SD) per years of practice between the fellowship-trained group (1.2 ± 1.1) and the non–fellowship-trained group (0.7 ± 0.6; P = .039), as well as in the number of publications per years of practice at MD Anderson (fellowship-trained: 1.5 ± 1.1; non–fellowship trained: 0.9 ± 0.9; P = .009).

Table 2.

Academic characteristics of the entire cohort of anesthesiologists (N = 78)

Variable	No.	Mean	SD	Min.	Q1	Median	Q3	Max.
Years in practice	78	18.3	9.2	1.3	12.3	17.2	24.1	43.8
Years at MD Anderson	78	14.8	7.7	0.6	9.9	15.4	19.8	32.6
Scopus publications	78	15.2	14.9	0.0	5.0	11.0	19.0	95.0
h-index	78	6.4	4.8	0.0	3.0	6.0	9.0	28.0
Publications/years in practice	78	1.0	0.9	0.0	0.3	0.7	1.2	4.1

Table 3.

Academic characteristics of fellowship-trained (n = 40) and non–fellowship-trained (n=38) anesthesiologists included in our analysis

Variable	Fellowship	No.	Mean	SD	Min.	Q1	Median	Q3	Max.	P
Years in practice Years at MD Anderson Scopus publications h-index	No	40	19	6.9	2.3	14.3	17.3	22.8	38.3	.166
	Yes	38	17.5	11.1	1.3	8.2	14.8	27.7	43.8
	No	40	16.3	6.5	0.6	11.6	17.6	20.5	31.1	.063
	Yes	38	13.3	8.7	1.5	6.4	12.5	19.5	32.6
	No	40	12.8	10.7	0	5	10.5	17.5	54	.329
	Yes	38	17.8	18.1	0	7	11.5	23	95
	No	40	6.3	4.6	0	3	6	9	22	.940
	Yes	38	6.5	5	0	3	6	8	28
Publications/year in practice Publications/year at MD Anderson	No	40	0.7	0.6	0	0.2	0.6	1	2.7	.039
	Yes	38	1.2	1.1	0	0.6	0.9	1.4	4.1
	No	40	0.9	0.9	0	0.3	0.8	1.3	5	.009
	Yes	38	1.5	1.1	0	0.7	1.2	1.9	5.3

Both h-index (P = .0003) and number of publications (P = .0001) differed by academic rank (Table 4). In addition, anesthesiologists had more publications in Scopus as academic rank increased (P = .0001). There were no differences (P = .567) in the number of publications per years of practice based on fellowship type (Table 5). However, h-index differed by fellowship type; with those who had participated in an obstetric/liver anesthesia, cancer anesthesia, neuroanesthesia, general clinical anesthesia, or graduate medical education fellowship (grouped together as “other”) had the highest h-index (mean ± SD 11.4 ± 7.6; P = .049).

Table 4.

Number of publications and h-index of anesthesiologists included in our analysis, by academic rank (N = 78)

Variable	Current rank	No.	Mean	SD	Min.	Q1	Median	Q3	Max.	P
Scopus publications	Assistant	8	3.1	3.1	0	0	3	5.5	8	0.0001
	Associate	30	11.3	7.9	2	7	10	15	40
	Professor	40	20.6	17.9	1	9	16.5	28	95
h-index	Assistant	8	2	2.2	0	0	1.5	3.5	6	0.0003
	Associate	30	5.3	2.9	1	3	5	7	13
	Professor	40	8.1	5.5	0	5	7	10.5	28

Table 5.

Number of publications per years of practice and h-index by type of fellowship (n = 38)

Variable	Type of fellowship	No.	Mean	SD	Min.	Q1	Median	Q3	Max.	P
h-index	Critical care	4	2.8	2.5	0	1	2.5	4.5	6	0.049
	Cardiovascular	13	5.5	2.9	2	3	5	7	11
	Othera	8	11.4	7.6	5	5.5	10	13.5	28
	Pediatric	6	5.8	3.3	0	6	6	7	10
	Regional/pain	7	5.6	3.9	1	3	4	9	12
Publications per years of practice	Critical care	4	0.6	0.6	0	0.1	0.5	1.1	1.4	0.567
	Cardiovascular	13	1.1	1.1	0.1	0.6	0.7	1.0	4.0
	Other	8	1.6	1.3	0.5	0.8	1.1	2.1	4.1
	Pediatric	6	1.3	1.4	0	0.3	1.0	2.1	3.6
	Regional/pain	7	1.4	1.0	0.2	0.6	1.0	2.3	3.1

^aOther: includes graduate medical education, obstetric/liver, neuroanesthesia, cancer anesthesia, and general clinical anesthesia fellowships.

Discussion

The current study showed that fellowship training is not associated with differences in publication productivity as measured by the h-index in the field of anesthesiology at a US tertiary cancer center. However, for those who were fellowship-trained anesthesiologists, we did observe a higher number of publications per years at MD Anderson, and per total years in practice. Additionally, h-index significantly differed by academic rank.

Since Hirsch’s introduction of the h-index in 2005, there have been several studies to evaluate the applicability of the h-index to anesthesiology in academic centers. For instance, anesthesiologists at randomly selected academic programs in the United States had a reported median h-index of 1, with a range up to 44.² More specifically, Pagel and Hudetz studied cardiothoracic anesthesiologists working in programs with accredited fellowships, as well as the influence of transesophageal echocardiographic credentials on h-index in the United States.⁵ Their research showed that the average h-index was 6 (SD ± 7) and was influenced by rank, increasing from instructor to professor. Similarly, we observed an average h-index of 6 in our study, and h-index also increased according to rank. Pagel and Hudetz discouraged comparison of different medical specialties by h-index given that some highly specialized fields do not have the wide audience of more generalizable fields. Anesthesiology fellowships are highly specialized and for this reason may not have a wide audience. Our study adds to the literature as it analyzes various anesthesia-related fellowships in a center where anesthesiologists have access to similar resources including statistical support, scientific writing experts, data analysts, and departmental mentoring.

In a separate study, Eloy et al evaluated the h-index in non–fellowship-trained academic anesthesiologists compared with fellowship-trained anesthesiologists and found them to be similar (2.98 compared with 2.88, respectively).⁶ That study showed that critical care fellowship-trained anesthesiologists had the highest average h-index, at 5.78, and regional and pain medicine fellowship-trained anesthesiologists had the lowest average h-index, at 1.18. In another study, O’Leary et al evaluated the h-index as a measure of research output in pediatric anesthesiologists in Canada, reporting a median h-index of 2, with a range of 0-32.⁷ In contrast with our study, our findings showed a significant difference in h-indexes (11.4) for those who had participated in an obstetric/liver anesthesia, cancer anesthesia, neuroanesthesia, general clinical anesthesia, or graduate medical education fellowship. Our results further showed that those anesthesiologists who had participated in fellowships in cancer anesthesia had all completed the fellowship at our institution. Given the relevance of cancer anesthesia fellowships to our practice, this may have resulted in a higher h-index for these anesthesiologists.

Gender differences have also been studied. For example, Pashkova et al found that male anesthesiologists had higher h-indexes than female anesthesiologists, but when organized by rank, this difference was seen only in full professors.⁸ Likewise, Pagel and Hudetz found that male anesthesiologists had higher h-indexes than female, but the number of citations per publication was similar between both; specifically, the absolute number of publications that was responsible for the difference in the h-index.² These findings, along with gender differences reported in other studies, highlight the need for cultural shifts within academia, promoting more holistic measures of academic productivity to foster equitable advancement opportunities.⁹

Although these types of quantitative indexes can be useful tools to evaluate scholarly output, there are multiple limitations to this type of evaluation of scholarly work. For instance, the h-index does not differentiate the authorship order of a publication, so a first or last author is regarded at the same level as an author in a different position of authorship. Moreover, h-index does not differentiate among original research publications, basic science, clinical research, and review articles, which may be cited more often.⁶ The more specialized the field, the smaller the number of readers, and therefore, the less likely the publications will be cited by a larger audience, resulting in lower h-indexes for the authors. Furthermore, consideration should be given to a time factor, specifically, a lead-time bias. Because the h-index is a compilation of cumulative work, it does not represent annual productivity, so an h-index may increase even if the author has been inactive for years.⁵ The h-index also does not include publications that are not peer-reviewed.

Several search engines can be used to search by h-index, including Web of Science, Scopus, and Google Scholar. These search engines may show different h-indexes because each engine indexes different sources than other sites. In addition, the search engines may index different journals, and authors who have used different initials to publish may have an omission of pertinent publications. Marriage or divorce may lead to name changes, including hyphenation, resulting in additional omission of publications. It has therefore been recommended that multiple search engines should be used to calculate the h-index.^6,10 In addition, a study among French anesthesiologists showed that a presence on X (formerly Twitter) is associated with higher citations and publications, and therefore social networks may affect the h-index.¹¹ Self-citation cannot be excluded from the h-index and could potentially affect the final h-index.³ The h-index does not specify if the citation is positive or negative in nature, as in a controversial article, and therefore a citation in this case does not indicate quality but quantity.⁵

Furthermore, a quantitative index, such as the h-index, fails to dynamically capture the academic productivity and scholarly activity of academicians. For instance, it does not include other academic activities such as mentoring, teaching, grant funding, innovations, development, and leadership.¹² Other factors such as mentorship for the faculty, departmental resources, and personal drive and ambition play a substantial role in determining an anesthesiologist’s academic productivity.¹³ Moreover, this index cannot delineate individuals who have dedicated research time from those who do not, nor can it capture the percentage of clinical duties required in specialties. Historically, academic physicians have been expected to function not only as clinicians but also as educators and scientists, and other roles for academic physicians now include administrative functions and community service. Because there is no agreed-upon gold standard for objectively measuring academic productivity, measurements such as the h-index may fail to capture the qualitative facets of academic productivity accomplished while fulfilling these other roles. For example, in very heavily clinically oriented departments, an anesthesiologist’s h-index may not be high even if the anesthesiologist had a fellowship. Of note, there are other bibliometrics to consider, including the g-index and the modified impact index. The g-index is used to provide weight to more highly cited papers, avoiding the pitfall of the h-index when there is a small number of very impactful publications resulting in a low h-index. The modified impact index corrects for time to publication, which is a limitation of the h-index.¹⁴

In addition to the general limitations of the h-index and other bibliometric indexes, our study has specific limitations. We did not assess other contributions made by faculty to the academic mission. In addition, the included fellowship list was limited and did not include all fellowships that are available within the specialty of anesthesiology. Our study also did not evaluate differences in h-index based on gender, which can be a confounder as noted above. Finally, we did not evaluate differences in h-index based on age, but evaluating h-index by years in practice may be a more accurate reflection of time that could be devoted to anesthesiology research.

In conclusion, our study found that fellowship training in anesthesiology is not associated with increased academic productivity as measured by the h-index in a tertiary cancer center. However, the results did show that fellowship-trained anesthesiologists produced statistically significantly more publications per year in practice, suggesting that fellowship programs, which primarily focus on refining clinical expertise, may still positively influence scholarly productivity. Future research should study larger cohorts from multiple academic centers, to study a broader range of factors that influence academic potential.¹⁵ In addition, further studies on the h-index area could help elucidate the usefulness and limitations of indexes, as well as shed light on the mechanisms behind this relationship and provide insights into how to support the career development and contributions of anesthesiologists in academia. Factors such as protected time, clinical demands, mentorship, and collaborations will need to be assessed to better understand the drivers of academic productivity. While no perfect measure or gold standard of academic productivity exists, the h-index may serve as a useful reference to anesthesiologists when navigating career development in academia.