Abstract
Introduction:
Assessing scholarly contributions in academic orthopaedic surgery remains challenging, despite the importance of publication productivity in evaluating academic leaders. The Hirsch (h)-index and its derivative, the m-index, offer objective measures to compare and evaluate publication productivity. The primary aims of this study were to compare h- and m-indices of department chairs and residency program directors of orthopaedic training programs in the United States and assess the association between h-index and program rank.
Methods:
A publicly available Doximity report was used to identify and rank orthopaedic surgery residency programs in the United States. Internet-based searches of institution-specific websites were performed to confirm current department chairs and residency program directors as of January 2024. H-index data were collected from a publicly available database known as Scopus. The m-index was calculated as the quotient of h-index and years in practice. Gender and years in practice were obtained through available biographies on institution-specific websites.
Results:
Department chairs had significantly higher h- and m-indices at both the career and 5-year time intervals compared with program directors. Upon subgroup analysis, department chairs and program directors at top-10 ranked programs had significantly higher h- and m-indices compared with the entire population of department chairs and program directors. Linear regression analyses demonstrated a direct linear association between department chair/program director h- and m-indices and program rank.
Conclusions:
Publication productivity was higher for department chairs than residency program directors. Orthopaedic leaders at top-10 institutions had higher publication productivity compared with those at institutions outside of the top 10, per Doximity rankings. Finally, there is substantial variation in publication productivity of orthopaedic surgeons in leadership roles at academic institutions in the United States.
Clinical Relevance:
This study highlights the publication productivity of orthopaedic surgery department chairs and residency program directors and describes the linear association between publication productivity of academic orthopaedic leaders and program rank. This information can be utilized by those interested in learning more about the publication productivity among academic orthopaedic surgeons across different institutions and can assist those who are involved in the hiring and promotion process at these academic institutions.
Introduction
The goal of academic departments and teaching programs is threefold: to provide excellence in patient care, teach the next generation of surgeons, and contribute impactful scientific research. While clinical contributions can be assessed through metrics such as relative value unit generation, case volume, complications, readmission rates, and number of clinic visits per year, objective measurement of scholarly contributions is challenging.
The Hirsch (h)-index was developed as an objective formula to track and compare publication productivity and scholarly contributions1. The h-index not only considers an individual's quantity of publications but also their influence through the number of times each publication has been cited2. The h-index is calculated by first ordering a researcher's publications by the number of citations in descending order. The h-index is the number of publications, X, for which an author has been cited at least X number of times1. Table I presents an example of this calculation.
TABLE I.
Example of H-index and M-index Measurement of a Researcher Who Has Been Publishing Research for 5 Years
| Publications (N = 5) | Cited |
|---|---|
| 1 | 10 |
| 2 | 5 |
| 3 | 3 |
| 4 | 1 |
| 5 | 1 |
| H-index | M-index (H-index/# of Research Years) |
|---|---|
| 3 | 0.6 |
Bolded text represents the h-index for this example researcher.
Because the h-index is highly dependent on how long an individual has been performing research, the m-index was created to help mitigate the impact of time. The m-index is the ratio of h-index to the number of years since an author's first publication1. Table I presents an example of this calculation. The utility of m-index is apparent when comparing candidates with varying career lengths, as an author's h-index inevitably increases over time. The m-index accounts for and controls for time, which allows comparisons between individuals irrespective of their tenure in academic medicine.
Many fields of medicine use h- and m-indices as a component of the hiring process and for promotional considerations to assess a candidate's scholarly contributions3-5. Previous work has explored the applicability of h-index in orthopaedic surgery and demonstrated a positive correlation of h-index with increasing academic rank and experience6-9. Ence et al. examined a large number of orthopaedic surgeons at 142 academic institutions, defining the threshold between a junior orthopaedic surgeon and one of more senior status to be h-indices of 12 and m-indices of 0.57. While these studies demonstrate that h- and m-indices may be meaningful metrics when evaluating publication productivity, to date, to the best of our knowledge, no study has investigated the research output for academic orthopaedic leaders, specifically Departments Chairs and residency program directors/Vice Chairs of Education. Furthermore, there is a paucity of data that have investigated the influence of publication productivity on program prestige/rankings.
The primary aim of this study was to compare h- and m-indices of department chairs and residency program directors of orthopaedic training programs in the United States. Our secondary aim was to assess how h- and m-index metrics evolve as program rank is taken into consideration.
Methods
A publicly available Doximity report was used to identify and rank 209 allopathic and osteopathic orthopaedic residency programs in the United States. Doximity program rankings are based on reputation surveys completed by physicians, resident satisfaction surveys, and objective measures of alumni research output. A study by Meade et al. showed that the Doximity rankings among orthopaedic residency programs are associated with the academic productivity of those programs10-12. Internet-based searches of institution-specific websites were performed to confirm current department chairs and residency program directors as of January 2024.
The h-index, both career and prior 5-year metrics, for orthopaedic department chairs and residency program directors was recorded from a publicly available database known as Scopus. The m-index was calculated as the quotient of h-index and years in practice. H-indices were reported as whole integer numbers, while m-indices were reported as decimals. Gender and years in practice were obtained through available biographies on institution-specific websites.
The Statistical Package for the Social Sciences version 29 was used for statistical analyses. A two-sample t-test was used to compare independent means of research metrics. A χ2 test was performed to assess the association between gender and occupying a department chair or program director position. Finally, a linear regression analysis was performed to test for associations between Doximity-reported ranking of orthopaedic surgery programs and publication productivity metrics of both department chairs and residency program directors.
Results
One hundred fifty-one orthopaedic department chairs and 204 orthopaedic residency program directors were identified on Scopus data collection of research metrics. Following exclusions for missing data, the mean number of years in practice for 143 orthopaedic department chairs was 25 ±9 years and the mean number of years in practice for 179 residency program directors was 17 ±9 years. There was a significant difference between years in practice for department chairs and program directors (p < 0.0001). Among orthopaedic department chairs, 142 of 151 (94%) were male and 9 of 151 (6%) were female. Among residency program directors, 170 of 195 (87%) were male and 25 of 195 (13%) were female. There was a significantly lower female representation among department chairs compared with residency program directors (p = 0.0335).
H-Index
H-index data were available for 149 orthopaedic department chairs and 171 program directors through Scopus. The mean career h-index was 26 ± 21, and the mean prior 5-year h-index was 7 ± 6 for department chairs. The mean career h-index was 11 ± 8, and the mean prior 5-year h-index was 5 ± 4 for program directors. There was a significant difference in not only career h-index but also prior 5-year h-index when comparing department chairs with program directors, where chairs had greater h-index metrics (Table II).
TABLE II.
Total and 5-year H- and M-indices of Orthopaedic Department Chairs and Residency Program Directors
| Department Chairs (All) | Program Directors (All) | T-tests | p-value | |
|---|---|---|---|---|
| H-index Total | 26 ± 21 (N = 149) | 11 ± 8 (N = 174) | 8.6286 | <0.0001 |
| 5-year | 7 ± 6 (N = 149) | 5 ± 4 (N = 174) | 4.6121 | <0.0001 |
| M-index Total | 1.1 ± 0.8 (N = 141) | 0.8 ± 0.6 (N = 162) | 10.3716 | <0.0001 |
| 5-year | 1.4 ± 1.1 (N = 147) | 0.9 ± 0.8 (N = 174) | 4.7558 | <0.0001 |
Bolded results are significant at the 5% level of significance.
M-Index
M-index data were available for 141 orthopaedic department chairs and 162 orthopaedic residency program directors through Scopus. The mean career m-index was 1.1 ± 0.8, and the mean prior 5-year m-index was 1.4 ± 1.1 for department chairs. The mean career m-index was 0.8 ± 0.6, and the mean prior 5-year m-index was 0.9 ± 0.8 for program directors. There was again a significant difference in not only career m-index but also prior 5-year m-index when comparing department chairs with program directors, with chairs demonstrating greater m-index metrics (Table II).
Subgroup Analysis—Top 10 vs. All Chairs
For department chairs at top-10 ranked programs, the mean number of years in practice was 28 ± 8 years, which was not significantly different from the mean years in practice for all department chairs in this study (25 ± 9 years). For top-10 department chairs, the mean career h-index was 57 ± 27 and the mean prior 5-year h-index was 13 ± 5, both of which were significantly greater than the mean career h-index (26 ± 21) and prior 5-year h-index (7 ± 6) for all department chairs in this study (total h-index: p < 0.0001|prior 5-year h-index: p = 0.0031). The mean career m-index was 2.1 ± 0.9, and the mean prior 5-year m-index was 2.5 ± 1.1, both of which were also significantly greater than the mean career m-index (1.1 ± 0.8) and prior 5-year m-index (1.4 ± 1.1) for all department chairs in this study (total m-index: p < 0.0001|prior 5-year m-index: p = 0.0036).
Subgroup Analysis—Top 10 vs. All Program Directors
For program departments at top-10 ranked programs, the mean number of years in practice was 16 ± 10 years, which was not significantly different from the mean years in practice for all program directors (17 ± 9 years). For top-10 program directors, the mean career h-index was 20 ± 17 and the mean prior 5-year h-index was 9 ± 6, both of which were significantly greater than the mean career h-index (11 ± 8) and prior 5-year h-index (5 ± 4) for all program directors in this study (total h-index: p = 0.0021|prior 5-year h-index: p = 0.0006). The mean career m-index was 1.5 ± 1, and the mean prior 5-year m-index was 1.8 ± 1.3, both of which were also significantly greater than the mean career m-index (0.8 ± 0.6) and prior 5-year m-index (0.9 ± 0.8) for all program directors in this study (total m-index: p = 0.0028|prior 5-year m-index: p = 0.0006). Data from our subgroup analyses are summarized in Tables III and IV.
TABLE III.
Total and 5-year H- and M-indices of Orthopaedic Department Chairs at Top 10 Programs by Doximity
| Department Chairs (All) | Department Chairs (Top 10) | T-test (Top 10 vs. All) | |
|---|---|---|---|
| Years in Practice | 25 ± 9 (N = 143) | 28 ± 8 (N = 10) | p = 0.2895 |
| H-index Total | 26 ± 21 (N = 149) | 57 ± 27 (N = 10) | p < 0.0001 |
| H-index 5-year | 7 ± 6 (N = 149) | 13 ± 5 (N = 10) | p = 0.0031 |
| M-index Total | 1.1 ± 0.8 (N = 141) | 2.1 ± 0.9 (N = 10) | p < 0.0001 |
| M-index 5-year | 1.4 ± 1.1 (N = 147) | 2.5 ± 1.1 (N = 10) | p = 0.0036 |
Bolded results are significant at the 5% level of significance.
TABLE IV.
Total and 5-Year H- and M-indices of Orthopaedic Residency Program Directors at Top 10 Programs by Doximity
| Program Directors (All) | Program Directors (Top 10) | T-test (Top 10 vs. All) | |
|---|---|---|---|
| Years in Practice | 17 ± 9 (N = 179) | 16 ± 10 (N = 10) | p = 0.8401 |
| H-index Total | 11 ± 8 (N = 174) | 20 ± 17 (N = 10) | p = 0.0021 |
| H-index 5-year | 5 ± 4 (N = 174) | 9 ± 6 (N = 10) | p = 0.0006 |
| M-index Total | 0.8 ± 0.6 (N = 162) | 1.5 ± 1 (N = 10) | p = 0.0028 |
| M-index 5-year | 0.9 ± 0.8 (N = 174) | 1.8 ± 1.3 (N = 10) | p = 0.0006 |
Bolded results are significant at the 5% level of significance.
Regression Analyses
Linear regression analyses were used to assess the association between h- and m-index metrics among orthopaedic department chairs and residency program directors, with respective program ranking plotted on the x-axis and productivity metrics plotted on the y-axis. There was an observed linear relationship for both h- and m-indices among both department chairs and program directors as program reputation and rank, as reported by Doximity, improved. These data are graphically depicted in Figures 1 through 4.
Fig. 1.
Regression analysis for orthopaedic department chair total and prior 5-year Scopus h-index by Doximity rankings.
Fig. 4.
Regression analysis for orthopaedic program directors total and prior 5-year Scopus m-index by Doximity rankings.
Fig. 2.
Regression analysis for orthopaedic department chair total and prior 5-year Scopus m-index by Doximity rankings.
Fig. 3.
Regression analysis for orthopaedic program directors total and prior 5-year Scopus h-index by Doximity rankings.
Discussion
H- and m-indices have widely been accepted as an objective representation of a researcher's impact and scholarly contribution and used in multiple disciplines of medicine1,13. These metrics have recently been demonstrated to be useful in objectively comparing candidates for academic promotions and recruitment14. This investigation sought to report the h- and m-indices for academic orthopaedic leaders as well as report on any observed associations of publication productivity with institution ranking. For the purposes of this study, we utilized Doximity ranking as a proxy of institutional reputation or program prestige.
We found that department chairs overall had higher h- and m-indices when compared with residency program directors. This may be explained by the generalization that most department chairs are full professors and occupy a position that can be viewed as more influential and may afford more connections and opportunities to make scholarly impacts through research. To test for associations between Doximity-reported program rank and publication productivity, we performed subgroup analyses of department chairs and residency program directors at top-10 institutions. For department chairs at top-10 institutions, h- and m-index scores, both total and prior 5-year, were significantly greater compared with all department chairs. For residency program directors, there were also greater h- and m-index scores, both total and prior 5-year, among top-10 institutions as compared with all residency program directors. Based on these results, it can be inferred that orthopaedic surgeons in leadership positions at higher ranked institutions have a greater number of publications and scholarly influence as compared with those at lower ranked institutions. These initial findings coincide with the fact that publication output influences Doximity rankings and is not surprising to see that the program leadership in top-10 institutions has more publication productivity compared with that outside of the top-10. To further investigate and expand on our findings, linear regression analyses were utilized. There were significant linear associations between all research metrics (h- and m-indices; total and prior 5-year) and Doximity program rank. The linear regression models demonstrated stronger associations between publication productivity and program ranking for department chairs than for residency program directors. This may indicate that the publication productivity of department chairs at higher ranking institutions is of greater magnitude of importance or weight than for residency program directors. However, it is important to remember that Doximity rankings are multifactorial in nature and, in addition to research output, also involve survey responses assessing the reputation of the program from board-certified physicians in the corresponding specialty, current residents, and recent alumni. The subjectivity of the rankings is a potential confounder, especially if more academically inclined programs influence the way their residents and alumni respond to these surveys.
Although our study provides important insight into evaluating and comparing contemporaries within the field of orthopaedic surgery, there are important limitations with the indices to note. A major intrinsic limitation of the h-index is the inability to differentiate the authorship position of a researcher for an article, so the amount of research impact or contribution to the article is not clearly delineated. In addition, the h-index fails to account for hyperauthorship, which is when all members of a consortium are listed as authors on scientific contributions skewing impact analyses. Although this is not commonly observed within orthopaedic surgery, it can still occur15. Next, more well-established and known authors tend to be cited repeatedly in the literature as compared with those who are not as well known. This inevitably skews the research metrics discussed. Furthermore, the basis of citation is not necessarily always associated with a positive connotation. Often, articles are cited with disapproval, which is not obviously discernible with the h- or m-index score. Finally, self-citation is not currently controllable in the h- or m-index calculation, which may lead to some inflation bias.
While there are general trends in publication productivity such as higher metrics for department chairs than residency program directors, there is great variation in publication productivity of orthopaedic surgeons in leadership roles at academic institutions in the United States. In addition, our study demonstrates that there may be a correlation between publication productivity and reputation/perceived prestige of academic departments. While research is valued in hiring and advancement, there are likely other factors also driving these decisions. For example, years in practice likely holds a great deal of value, as supported by the demographics data collected in this study that shows many of the orthopaedic surgeons in leadership positions are in their second or third decade of practice. At many training programs, publication productivity is often considered a prerequisite for academic promotion and consideration for executive leadership positions. The h-index and m-index allow for tracking and comparing publication productivity and scholarly contributions between individuals for the purpose of advancement and recruitment. However, other factors such as teaching experience, leadership qualities, emotional intelligence, and interpersonal skills are also critical factors to consider. These other factors may ultimately prove to be more important and impactful for predicting successful senior leaders in our field.
Footnotes
Disclosure: The Disclosure of Potential Conflicts of Interest forms are provided with the online version of the article (http://links.lww.com/JBJSOA/A676).
Contributor Information
Andrew Gaetano, Email: agaetano@luc.edu.
Krishin Shivdasani, Email: krishin.shivdasani001@luhs.org.
Nickolas Garbis, Email: ngarbis@lumc.edu.
Dane Salazar, Email: dsalazar@lumc.edu.
References
- 1.Hirsch JE. Does the H index have predictive power? Proc Natl Acad Sci U S A. 2007;104(49):19193-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.DeLuca LA, Jr, St John A, Stolz U, Matheson L, Simpson A, Denninghoff KR. The distribution of the h-index among academic emergency physicians in the United States. Acad Emerg Med. 2013;20(10):997-1003. [DOI] [PubMed] [Google Scholar]
- 3.Zimarino M, Benedetto U, Pelliccia F. The H-index in the road for academic progression. Int J Cardiol. 2024;408:132119. [DOI] [PubMed] [Google Scholar]
- 4.Nocera AP, Boudreau H, Boyd CJ, Tamhane A, Martin KD, Rais-Bahrami S. Correlation between H-index, M-index, and academic rank in urology. Urology. 2024;189:189-150. [DOI] [PubMed] [Google Scholar]
- 5.Ashfaq A, Kalagara R, Wasif N. H-index and academic rank in general surgery and surgical specialties in the United States. J Surg Res. 2018;229:108-13. [DOI] [PubMed] [Google Scholar]
- 6.Lopez J, Susarla SM, Swanson EW, Calotta N, Lifchez SD. The association of the H-index and academic rank among full-time academic hand surgeons affiliated with fellowship programs. J Hand Surg Am. 2015;40(7):1434-41. [DOI] [PubMed] [Google Scholar]
- 7.Ence AK, Cope SR, Holliday EB, Somerson JS. Publication productivity and experience: factors associated with academic rank among orthopaedic surgery faculty in the United States. J Bone Joint Surg Am. 2016;98(10):e41. [DOI] [PubMed] [Google Scholar]
- 8.Atwan Y, Charron BP, Sidhu S, Cavanagh J, Degen R. Publication productivity among academic orthopaedic surgeons in Canada. Cureus. 2020;12(6):e8441. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Scanaliato JP, Green CK, Polmear MM, Orr JD, Dunn JC. The association of H-index and academic rank among full-time academic foot and ankle surgeons affiliated with fellowship programs. J Surg Orthop Adv. 2022;31(4):263-7. [PubMed] [Google Scholar]
- 10.Wilson AB, Torbeck LJ, Dunnington GL. Ranking surgical residency programs: reputation survey or outcomes measures? J Surg Educ. 2015;72(6):e243-50. [DOI] [PubMed] [Google Scholar]
- 11.Doximity residency navigator survey methodology. Available at: https://s3.amazonaws.com/s3.doximity.com/mediakit/Doximity_Residency_Navigator_Survey_Methodology.pdf.
- 12.Meade PJ, Amin SJ, Stamm MA, Mulcahey MK. Doximity orthopaedic surgery program rankings are associated with academic productivity. JBJS Open Access. 2023;8(1):e22.00081. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Bastian S, Ippolito JA, Lopez SA, Eloy JA, Beebe KS. The use of the h-index in academic orthopaedic surgery. J Bone Joint Surg Am. 2017;99(4):e14. [DOI] [PubMed] [Google Scholar]
- 14.Sharma B, Boet S, Grantcharov T, Shin E, Barrowman NJ, Bould MD. The h-index outperforms other bibliometrics in the assessment of research performance in general surgery: a province-wide study. Surgery. 2013;153(4):493-501. [DOI] [PubMed] [Google Scholar]
- 15.Koltun V, Hafner D. The h-index is no longer an effective correlate of scientific reputation. PLoS One. 2021;16(6):e0253397. [DOI] [PMC free article] [PubMed] [Google Scholar]