A 15-Year Bibliometric Analysis of Sports Medicine Studies in The Journal of Bone and Joint Surgery

Abstract

Background:

Orthopaedic sports medicine is among the most popular subspecialties. Understanding the trends in sports medicine research over time can offer insight into progress and innovation within the field. The purpose of this study was to assess both the quality of the current literature and trends in gender diversity and inclusion by evaluating publishing characteristics of sports medicine studies in The Journal of Bone and Joint Surgery-American Volume (JBJS-A) from 2007 to 2021.

Methods:

Sports medicine studies in JBJS-A from 2007 to 2021 were identified using JBJS subspeciality tags for “sports medicine” articles and organized by study type, number of authors, sex of the authors, academic degree(s) of the first and last authors, level of evidence, country of publication, citations, and use of patient-reported outcomes (PROM).

Results:

A total of 784 studies were reviewed, and 513 met inclusion criteria. Clinical therapeutic studies were the most common publication (48%). There was an increase in the publication of clinical prognostic studies (17%-25%, p = 0.037) and a significant increase in the use of PROM measures over time (13%-47%, p < 0.001). The total number of authors increased over the study period (4.8-6.3), but there was no significant increase in female authorship. Only 15% of the 784 studies included a female author, with an average of 0.8 female authors per article (range 0-8) compared with 4.6 males (range 1-14).

Conclusion:

The significant increase in the use of PROMs in sports medicine studies indicates that the quality of research has improved over the 15-year period. The gender disparity in authorship has remained stagnant. Only 11% of all first authors and 9% of senior authors were female. The number of included international studies improved over time; however, the United States remains the most prolific publisher. Despite these areas of growth, this study suggests that there is room for improvement of authorship gender diversity in orthopaedic sports medicine research.

Level of Evidence:

Level III. See Instructions for Authors for a complete description of levels of evidence.

Introduction

Over the past 2 decades, there has been an increase in female medical students, representing 55% of matriculating medical students in 2023 compared with only 9% in 2001¹. Despite these growing numbers, specialties such as orthopaedic surgery have disproportionately low numbers of female trainees. In 2019, 6% of American Academy of Orthopaedic Surgeons (AAOS) members and 17% of orthopaedic surgery residents were female^1-3. Recent initiatives aim to increase diversity in the field, such as the Perry Initiative, National Football League Diversity Program, and the AAOS Inclusion, Diversity, Equity, and Access Grant program⁴. These programs offer early exposure, mentorship, and research opportunities for medical students interested in a career in orthopaedic surgery. Involvement with research has become a very important factor in successfully matching into orthopaedic surgery residency^5-7. One measure to determine whether these programs are meeting the goal of increasing diversity in the field is to assess whether there has been a subsequent increase in female authorship.

Previous studies have demonstrated that despite increased total authorship numbers and research productivity, female authorship rates in orthopaedic surgery have remained low and relatively stagnant^5-12. A retrospective review of 6 major orthopaedic journals from 1987 to 2017 found the publishing rate of female first authorship to be approximately 4% and senior authorship at only 2%¹¹. A more recent 20-year review of 14 sports medicine journals including the American Journal of Sports Medicine (AJSM) and British Journal of Sports Medicine found that only 16.8% of senior authors on sports medicine publications were female, and the rate of increasing female authorship per year over this period was only 0.5%¹⁰. These studies suggest that representation of female authors in research is not in line with the increasing number of female orthopaedic surgeons¹¹. In addition, a 2018 study by Bendels et al. demonstrated that even when women authors are published in prestigious high-impact journals (e.g., Nature), there were distinct gender-specific differences in citation rates, journal categories, and country of origin¹³. Their review of 293,557 articles in 54 high-impact journals found an annual increase rate of female authors per year was 0.7%, demonstrating a gender-specific dichotomy in academia¹³. Interpretation of these trends may allow for a better assessment of diversity in the field and may identify areas for improvement to increase recruitment and retention of females to orthopaedic surgery and sports medicine^12,14,15.

To the best of our knowledge, this is the first assessment of sports medicine-specific authorship trends by sex in a journal that is not solely dedicated to sports medicine. The purpose of this study was to assess both the quality of the current literature and trends in gender diversity and inclusion by evaluating publishing characteristics of sports medicine studies in The Journal of Bone and Joint Surgery-American Volume (JBJS-A) from 2007 to 2021. We aimed to determine whether changes to female authorship had occurred, specifically in sports medicine, as the number of both female medical students and orthopaedic surgery residents has increased in the past decade^1,2. We hypothesized that there would be increased female authorship and improved quality of research because previous studies evaluating authorship trends in orthopaedic surgery have indicated that total authorship and quality of studies are increasing^5,6.

Methods

The JBJS-A advanced search function was used to search all articles with the “Sports Medicine” subspecialty tag. All other subspecialties were deselected. This feature is like “MESH Terms” in PubMed searches, where categories are grouped relative to a specific term or tag to allow for ease of filtering. The publication date range was set from 2007 to 2021 in the advanced search function. Studies were then organized into 3 different 5-year groups; January 1, 2007, to December 31, 2011, January 1, 2012, to December 31, 2016, and January 1, 2017, to December 31, 2021. Studies were excluded if they were editorials, orthopaedic forums, specialty updates, concept reviews, commentaries, announcements, events, errata, retracted manuscripts, instructional lectures, and letters to the editor.

The following data were collected from each eligible study: total number of authors, sex of authors, professional academic degrees of first and last authors, study type, country of origin, the level of evidence (LOE) as reported by JBJS-A, the number of references used by the study, the number of times the study was cited as noted by PubMed search, and the use of patient-outcome measures (PROM)^15,16. Three independent reviewers (medical students A.W., T.M., and J.B.) assessed the studies and recorded the publishing characteristics. In cases where JBJS-A did not assign an LOE and study type, the methodology provided by JBJS-A in the 2015 updated guidelines was used, whereas for studies older than the recent JBJS-A guidelines, the methodology proposed by Spindler et al. was used^17,18.

Ten studies (0.02%) listed authorship group names rather than individually named authors. Therefore, individual authorship sex was not recorded for these studies, and the 10 studies were removed from the total number of studies in authorship sex analysis. However, studies that listed authorship groups were included in the total number of studies for analysis of number of authors per article. Group authorship was recorded as if there was only one author in the study. The author's sex was identified using a Google search of the author's name. Finally, international medical graduate degrees as recognized by the American Medical Association’s Education Commission of Foreign Medical Graduates were classified as MD equivalents (e.g., Bachelor of Medicine and Bachelor of Surgery, MBBS).

Qualitative Analysis

The quality of publications over time was interpreted by assessment of publishing characteristics collected including use of PROMs, number of references per study, number of times a study was cited by another, and LOE. These factors were used as an indicator of an increase in the quality of research, as modeled by Alexander et al. in 2020¹⁹.

Statistical Analysis

Publication characteristics were summarized with the median and the interquartile range for the number of references and the number of citations per study. All other categorical characteristics were summarized with frequencies and percentages. The Kruskal-Wallis test was used to test significant differences in the distribution of the number of references and the number of citations per study over time. Chi-square analysis was used to test for significant differences in percentages of female first and second authorship and study type over time, as well as to assess differences in LOE by female first and last authorship. Statistical analyses were conducted in SAS.

Results

Total Number of Studies and Study Type

Seven hundred eighty-four studies were identified, of which 513 met inclusion criteria and were analyzed. There were 248 (48%) clinical therapeutic studies, the most popular study type over time. The proportion of clinical prognostic studies increased slightly from 18% in 2007 to 2011 to 25% in 2017 to 2021, whereas clinical diagnostic studies significantly decreased over time from 20 (10%) in 2007 to 2011 to 2 (2%) in 2017 to 2021 (p = 0.003). The total number of case reports, clinical economics, and meta-analyses was too low to be analyzed individually, so they were combined for statistical purposes (Table I). Together, these studies accounted for a slight increase over time with 18.8% of studies from 2007 to 2011 compared with 24% from 2017 to 2021, with a peak of 55 studies (26%) between 2012 and 2016 (Table II).

TABLE I.

Authorship, References, Citations, and Study Design Trends for 2007 to 2021

	2007-2011	2012-2016	2017-2021
Total no. of studies published	202	209	104
Median no. of references per study	31	31	36
Median no. of citations per study	83	61	16
Average total authors per study	4.8 (SD: ± 1.8)	5.6 (SD: ± 2.2)	6.3 (SD: ±2.1)
Average # of male authors per study	4.1 (SD: ± 1.7)	4.7 (SD: ±2.2)	5.4 (SD: ±2.2)
Average # of female authors per study	0.6 (SD: ± 0.9)	0.9 (SD: ± 0.6)	0.8 (SD: ± 1.1)
First authors by sex	22 F, 180 M	26 F, 181 M	10 F, 94 M
Last authors by sex	12 F, 188 M	26 F, 178 M	8 F, 94 M
Type of study (no. [%])	108 (53.4)	89 (42.6)	51 (49.0)
Clinical therapeutic	36 (17.8)*	42 (20.0)	26 (25.0)*
Clinical prognostic	20 (9.9)*	23 (11.0)	2 (1.9)*
Clinical diagnostic	38 (18.8)	55 (26.3)	25 (24.0)

^*p < 0.05.

TABLE II.

Level of Evidence Trends by Sex of First and Last Authors for 2007 to 2021^*

LOE	1	2	3	4	Total
First authors by sex
Female	5	12	5	8	30
Male	49	37	60	103	249
Total no. of first authors by LOE	54	49	65	111	279
Last authors by sex
Female	4	7	10	12	33
Male	50	42	55	99	244
Total no. of last authors by LOE	54	49	65	111	279

^*LOE = Level of Evidence.

References and Citations

The median number of references was 31 for the periods of 2007 to 2011 and 2012 to 2016 (2007-2011 range = 3-144, interquartile range [IQR] = 22-41; 2012-2016 range = 0-136, IQR = 24-39). Between 2017 and 2021, the median number of references increased slightly to 36 (range 16-115, IQR = 29-46). Overall, the average number of references per study was 34.9 (SD ±18.6) and did not change significantly over time.

The number of times a study was cited by another study decreased with time. The median number of citations from 2007 to 2011 was 83 (range: 1-931, IQR = 39-162), as compared to an average of 16 citations (range 0-290; IQR = 5-40) between 2017 and 2021 (p ≤ 0.0001).

Level of Evidence

LOE was reported for 279 (51.3%) of the studies reviewed. Most studies were Level IV (111; 40%). The least common study was Level II (49; 18%). There were no Level V studies reported across all periods (Table III). There was a slight correlation between higher LOE studies (I or II) and female first authorship, with 40% of female first authors having studies of LOE I or II (p = 0.007) (Table II).

TABLE III.

Sex of First and Last Author by Study Design 2007 to 2021

Study Type	Clinical Therapeutic	Clinical Prognostic	Clinical Diagnostic	Other	Total
First author by sex
Female	24	13	7	14
Male	221	91	38	105
Total	245	104	45	119	513
Last author by sex
Female	22	14	3	7
Male	221	88	40	111
Total	243	102	43	118	506

Authorship: Sex and Degrees

The number of authors increased over time from an average of 4.8 (SD: ±1.8; range = 1-12) authors in 2007 to 2011 to 6.3 authors (SD: ±2; range = 1-14) in 2017 to 2021 (Table I). Notably, there were substantial differences in male and female authorship averages across time. From 2007 to 2021, 89% of first authors (n = 249) and 90% (n = 244) of last authors were male, compared with the respective 11% (n = 30) and 9% (n = 33) of females. Female authorship averaged less than one per study, 0.69 (SD: 0.9, range: 0-4) and increased to 0.84 (SD: 1.13, range 1-5) from 2017 to 2021. Female first authorship remained consistent across time at about 10% (Table I). The majority of first and last authors' professional degrees for male and female authors combined were MD (75%, n = 386) or MD equivalents (70%, n = 351) over time. The second most common professional degree was a PhD for both first (12%, n = 62) and last authors (17%, n = 84). Only 25 (5%) first authors had a professional degree of MD/PhD, versus 47 (9%) last authors. Finally, only 26 (5%) first authors had a bachelor's or bachelor's equivalent degree.

Country of Origin

The United States was the most common country of origin across time, accounting for 329 (64%) studies, followed by the United Kingdom with 87 (17%) and Canada with 51 (10%).

PROMs

The use of PROMs significantly increased over time (p > 0.001), from 26 (13%) studies reporting the use of PROMs from 2007 to 2011 to 48 (48%) from 2017 to 2021.

Discussion

This study reviewed the publishing characteristics of 513 sports medicine studies in the JBJS-A across a 15-year period (2007-2021). To the best of our knowledge, this is the first assessment of sports medicine specific authorship trends by sex in a journal that is not solely dedicated to sports medicine. The findings of this review reflect that female authorship in a high-impact general orthopaedic journal remains limited, despite attempts to increase authorship diversity.

Number of Studies and Study Type

Clinical therapeutic studies remained the most common publication type, accounting for 248 (48%) of the total publications. Clinical prognostic studies increased over time, with 104 (20%) publications overall. This growth could be explained by the increasing importance of evidence-based medicine²⁰. There was a decrease in clinical diagnostic studies from 20 (10%) in 2007 to 2011 to 2 (2%) in 2017 to 2021 (Table I).

References and Citations

Citation analysis has been used to establish the frequency and patterns of citations in scientific literature. Measuring the number of times a study is referenced can provide insight into its academic influence^21-23. The number of citations per study decreased from 2007 to 2011 to 2017 to 2021. This trend may be due to newer publications being available for less time compared with older studies. The number of references per study has increased with time, from an average of 34 (range: 3-144, median: 33, IQR: 22-41) in 2007 to 2011 to a peak of 40 (range: 16-155, median: 36, IQR: 29-46) in 2017 to 2021. This could be attributed to the growing availability of publications in electronic formats^24,25.

Level of Evidence

Higher LOE studies correlate with increased evidence-based medicine practices, with the goal of improved patient outcomes²⁶. Historically, LOE has been used as a method of determining the quality of research and the long-term impact of study findings^27,28. In a review of LOE trends in JBJS over the past 30 years, LOE IV was the most common publication, consistent with our findings²⁶. However, LOE classification systems have changed over the years leading to some inconsistencies in classification which can skew LOE results^7,29.

Authorship Sex and Degrees

Over the study period, the number of female orthopaedic sports medicine surgeons has increased, and we anticipated a similar increase in female authorship; however, it remained stagnant at less than one female author per study^30,31. These findings are consistent with a study that evaluated the number of female authors in Clinical Orthopaedics and Related Research, JBJS-A, and the AJSM from 2006 to 2017²⁸. The authors found that of 6,292 studies included, only 13% had a female as the first author and 10% had a female as the last author, similar to the rates found by our study²⁸. The slight trend of increasing first female authors over the last author may be indicative of the recent increase of females into medicine and orthopaedic surgery at a more junior level. By contrast, stagnant last authorship rates may be reflective of older males in senior leadership in orthopaedics^10,29-32. There remains a significant gap between male and female authorship, with only about 10% of all authors being female, which is a consistent finding with other previous studies such as Hiller et al. (2017)²⁸. Low rates of female medical students entering the field of orthopaedic surgery and the lack of female representation at senior levels may limit female sports medicine authorship opportunities^15,33,34. This study finds female authorship correlated with increased LOE, a finding that remained consistent across time intervals (40% of all studies with a female author were LOE I or II; p = 0.007) (Tables II and III). This indicates that although there are fewer female authors, they may engage in high-level and impactful research. This speculation is supported by a 46-year review of orthopaedic sports medicine publications from 1972 to 2018, where Kim et al. found that most female authors were more likely to be full-time researchers, with PhD degrees, rather than physicians³⁵.

Country of Origin

International publication rates in JBJS have increased over time. The United States accounts for about two-thirds of publications analyzed. Representation from Asian, Middle Eastern, and South American countries remains lower comparatively. This may reflect a potential self-curating bias by international orthopaedic surgeons who may be more likely to identify as surgical or joint-focused specialists rather than “sports medicine specialists.” The percentage of sports medicine studies published in the United States in this study aligns with the overall publishing trends reported in a JBJS bibliometric analysis of orthopaedic journals^36-38.

PROMs

This study shows there is an increasing focus on the quality of the patient experience over time, based on a significantly increased use of PROMs. PROMs allow physicians to assess patient perspectives of clinical outcomes, which can help inform patient care decisions and evaluate the effectiveness of a treatment plan^37,39-41.

Limitations

This study was limited to sports medicine publications, and its conclusions may not be reflective of trends across the other 12 orthopaedic subspecialties. Our final analysis of last author trends by sex did not account for large group practice publications where author sex information was unavailable. In addition, in 2015, the JBJS-A LOE guidelines were amended to better classify and evaluate the methodological rigor of a study¹⁹. Studies published before 2015 may have reported LOE differently than those published after the amended guidelines, and all reported LOE publication trends should be interpreted with caution. Furthermore, the SARS-COV-2 (COVID-19) pandemic posed significant changes to the lives of medical researchers and professionals, with significant disruptions to noncritical research⁴². The impact of the COVID-19 pandemic on research and academic publishing is yet to be understood, and further evaluation beyond the pandemic period would be an interesting next direction. Overall, future directions should look at increasing the breadth of journals, length of time, and assessment of quality metrics beyond this focused review.

Conclusion

The significant increase in the use of PROMs in sports medicine studies indicates that the quality of research has improved over the 15-year period. The gender disparity in authorship has remained stagnant. Only 11% of all first authors and 9% of senior authors were female. The number of included international studies improved over time; however, the United States remains the most prolific publisher. Despite these areas of growth, this study suggests that there is room for improvement of authorship gender diversity in orthopaedic sports medicine research.