The Competitiveness of Orthopaedic Surgery Residency Programs: A Twenty-year Analysis Utilizing a Normalized Competitive Index

Highlights

• •The competitiveness of orthopaedic surgery residency programs has grown over time.
• •Match rates for in orthopaedic surgery residency have remained constant throughout time.
• •All applicant metrics have improved, and research output has quadrupled.
• •The NCI may be a valuable tool for applicants to gauge the competitiveness of orthopaedic surgery residency.

Abstract

Objective

To apply a normalized competitive index over a 20-year period to analyse applicant trends in orthopaedic surgery residency programs.

Design

A retrospective analysis of orthopaedic surgery residency program data from the National Resident Matching Program (NRMP) Main Residency Match data from 2003 to 2022 and NRMP Charting Outcomes data from 2007 to 2021 was performed. A competitive index (CI) was utilized by dividing the number of programs ranked per applicant, multiplied by available positions, and by the match rate for each year. Next, the index was normalized to a value of 1 to establish a normalized competitive index (NCI) by dividing the yearly CI by the average CI.

Setting

National Resident Matching Program (NRMP) Main Residency Match data from 1992 to 2021 and NRMP Charting Outcomes data from 2007 to 2021.

Participants

Orthopaedic surgery residency applicants.

Results

The NCI was significantly different over time (R²= 0.95, p < 0.001) with an upward trending NCI slope. Total applicants have nearly doubled over the past two decades (2003, 830 vs. 2022, 1460; p<0.001). The match rate did not significantly change over the two decades (2003−2012 vs. 2013–2022; 70% vs. 70%; p = 0.76). However, in 2022, the match rate dropped to 59.5%. The 2022 applicant cycle had 14% more applicants, while positions increased by 0.8% compared to the year prior. The USMLE Step 1 and Step 2 scores of matched applicants increased over time (R²>0.83, p < 0.001). Research output quadrupled over the 2007–2021 period (3 vs. 14, p < 0.001).

Conclusion

Despite match rates remaining constant over time, the NCI metric demonstrated the increasing competitiveness of orthopaedic surgery residency programs over twenty years. Match rates have inherent limitations as a singular metric and should only be used in conjunction with additional metrics. Establishing a more comprehensive index with applicant metrics provides insight into the growing competitiveness of orthopaedic programs and a more comprehensive outlook for future applicants.

Introduction

Orthopaedic surgery residency applicants have competitive USLME scores and publication volumes [1], [2], [3]. This pool of applicants and relatively few spots in residency programs has made orthopaedic surgery amongst the most competitive residencies to obtain [1,4]. In addition, the increase in the number of applications suggests that applicants believe applying to more programs is necessary to increase their probability of matching [4], [5], [6]. As a result, the number of applications received by programs has increased by more than double in the last several years [2,3,5]. This is leading to a burden on program and applicant resources, with a potential decline in the proportion of applications receiving an in-depth review [3,4].

With the current state of the match cycle for orthopaedics and a recent transition to pass/fail exams, many applicants and programs are concerned with how to assess applicant competitiveness [7]. Medical students will likely use metrics such as match rate, number of positions offered per applicant, volume, and quality of research output to determine their competitiveness [8]. Additionally, with the changes in the number of positions, trainee preferences, and changes in reporting results of standardized examinations, competitiveness is a fluid metric that is constantly changing [9,10]. A study by Yong et al. found that 50% of orthopaedic applicants would have applied to fewer programs if they had more program information and metrics [11]. Thus, an additional metric is needed to help medical students determine their competitiveness and reduce the applicant and program burden.

An assessment of competitiveness using a competitive index (CI) has been previously studied to provide another metric for applicants applying to general and vascular surgery training programs [9]. Treffalls et al. further developed a normalized competitve index (NCI) that provided insight into the competitiveness of surgical subspecialty programs [[9], [12]]. This study aims to apply the normalized competitive index over 20 years to analyze applicant trends in orthopaedic surgery residency programs.

Methods

Data collection

Using the publicly accessible National Resident Matching Program (NRMP) Main Residency Match data (Main page: https://www.nrmp.org/match-data-analytics/residency-data-reports/; Archive: https://www.nrmp.org/match-data-analytics/archives/), a retrospective study was performed. The Main Residency Match Data were used to compile information on orthopaedic surgery residency programs from all recognized United States (U.S.) medical schools from 2003 to 2022. For each year, information was gathered on the number of applicants, programs, positions, filled programs, filled positions, match rate, and the number of rated positions.

In addition, the Association of American Medical Colleges (AAMC) Report on Residents data, published every two to three years, and the NRMP Charting Outcomes were used to compile the orthopaedic surgery applicant metrics from 2007 to 2021 [13]. United States Medical Licensing Exam (USMLE) Step 1, Step 2, research experiences, research output (publications and presentations), volunteer experiences, and work experiences were amongst the applicant metrics. Because no patient information was used and the data was publicly available, no Institutional Review Board (IRB) approval was required.

Competitive index

The number of programs ranked per applicant, divided by the match rate, was used to produce a competitive index (CI) for individual years between 2003 and 2022 (Eq. (1)). The number of programs ranked for each applicant was then divided by the overall match rate to produce a cumulative competitive index (Eq. (2)). Finally, the annual CI was divided by the cumulative CI to determine the NCI, normalized to a value of 1 (Eq. (3)). Normalizing the CI allowed for the CI to be adjusted for the significant increase in positions and corresponding increases in the number of programs ranked per candidate [12].

$$\mathit{C}\mathit{I}=(\#\mathit{of}\mathit{Prog}\mathit{rams}\mathit{Rank}\mathit{ed}\mathit{per}\mathit{Appl}\mathit{icant}*\mathit{Posi}\mathit{tions})/\mathit{Match}\mathit{Rate}$$ (1)

$$\mathit{A}\mathit{v}\mathit{e}\mathit{r}\mathit{a}\mathit{g}\mathit{e}\mathit{C}\mathit{I}=(\mathit{Avg}\#\mathit{of}\mathit{Prog}\mathit{rams}\mathit{Rank}\mathit{ed}\mathit{per}\mathit{Appl}\mathit{ican}\mathit{ts}*\mathit{Avg}\mathit{Posi}\mathit{tions})/\mathit{Avg}\mathit{Match}\mathit{Rate}$$ (2)

$$\mathit{N}\mathit{C}\mathit{I}=\mathit{CI}\mathit{per}\mathit{Year}/\mathit{Aver}\mathit{age}\mathit{CI}$$ (3)

Statistical analysis

Statistical analysis was performed using GraphPad Prism v8.0 (GraphPad Software Inc, San Diego, CA, USA). Data on match characteristics were divided into two periods: 2003–2014 and 2015–2022. Two research periods (2007–2013 and 2014–2021) were used to group the applicant metrics. Mann-Whitney U tests and analysis of variance (ANOVA) were used. The predetermined threshold for statistical significance was p<0.05. Data were presented as mean and standard deviation (S.D.). Linear regressions were applied to the NCI across the study period (2003–2022) to determine a long-term competitiveness trend. For the 20 years, linear regressions were performed on the number of applicants ranked per school and the percentage of orthopaedic surgery matches. After performing linear regressions over time, applicant metrics were plotted across time (2007–2021). The line of best fit (R²) and the statistical difference were identified for each regression.

Results

Overall characteristics

The mean number of programs increased (2003–2012 vs 2013–2022; 157.4 [3.1] vs 178.1 [20.5]; p<0.005) with minimal increase in programs on longitudinal analysis (R²=0.64, slope=0.43, p<0.001). Similarly, there was a mean increase in positions (2003–2012 vs 2013–2022; 629.0 [34.4] vs 762.4 [73.0]; p<0.001) with linear increase across time (R²=0.91, slope=1.05, p<0.001). Orthopaedic surgery residency total applicants have nearly doubled in the past two decades, with 1.7 times the number of applicants in 2022 compared to 2003 (830 vs. 1470; p<0.001). The mean total applicants also increased (2003–2012 vs 2013–2022; 924.6 [75.2] vs 1120.8 [151.6]; p = 0.002) with significant linear increase in total applicants (R²=0.71, slope=3.16, p<0.001) (Table 1).

Table 1.

Orthopaedic Surgery Residency Match Characteristics from 2003 to 2022.

Mean (SD)	2003–2012	2013–2022	p value
Orthopaedic Surgery Programs	154.4 (3.1)	178.1 (20.5)	0.005
Orthopaedic Surgery Positions	629 (34.4)	762.4 (73)	<0.001
Matches into Orthopaedics	625 (36.5)	760 (72.6)	<0.001
Total Applicant Match Rate	70 (0)	70 (0)	<0.001
MD Senior Match Rate	80 (0)	80 (0.1)	0.46
Total Applicants	925 (75.2)	1121 (151.6)	0.002
MD Senior Applicants	748 (59.3)	886 (76.7)	<0.001
Ranked Programs per Applicant	8.7 (0.4)	9.4 (0.4)	0.001
Applicant/Position	1.5 (0.1)	1.5 (0.1)	0.976
NCI	0.9 (0.1)	1.1 (0.1)	<0.001

SD, Standard deviation; M.D. Senior, U.S. allopathic medical graduates; Total Applicants, U.S. allopathic and osteopathic medical graduates, U.S. international medical graduates, and foreign medical graduates; NCI, Normalized competitive index.

In addition, the mean MD senior applicants increased (2003–2012 vs 2013–2022; 747.7 [59.3] vs 885.6 [72.6]; p<0.001); however, this occurred at a slower rate compared to total applicants (R²=0.77, slope=1.84, p<0.001) (Fig. 1). While applicants per position were not significantly different between the two decades (2003–2012 vs. 2013–2022; range 1.3–1.5 vs. range 1.4–1.7; p = 0.98), the rate of change between the two is significantly different (Table 1). Total applicants have increased at a faster rate over time (R²=0.73, slope=3.16, p<0.001) compared to total positions available (R²=0.91, slope=1.05, p<0.001).

Fig. 1.

Linear regressions over a 20-year period of orthopaedic surgery applicants and program data, including total orthopaedic applicants (R²=0.73, p<0.001), United States MD senior applicants (R²=0.77, p<0.001), and total orthopaedic residency positions (R²=0.91, p<0.001).

The match rate did not significantly increase over the two decades (2003–2012 vs. 2013–2022; 70% vs. 70%; p = 0.76). Similarly, the US-MD match rate remained stable over the two decades (2003–2012 vs. 2013–2022; 80% vs. 80%; p = 0.46). However, the overall match rate dropped to 59.5% in 2022. The 2022 applicant cycle had 14% more applicants with 0.8% more positions than the previous year. Lastly, the number of programs ranked per applicant has increased over time (2003–2012 vs 2013–2022; 8.7 [0.4] vs 9.4 [0.4]; p = 0.001) (Table 1).

Normalized competitive index

The average NCI significantly increased over the last two decades (0.9 vs. 1.1; 2003–2012 vs. 2013–2022 p <0.001). The NCI reached a value of 1.35 in 2022, 35% higher than the mean NCI of 1.0. When longitudinal trends were analysed, the NCI was significantly different across time, with an upward trending NCI (R²=0.95, p <0.001) (Fig. 2).

Fig. 2.

A linear regression of the orthopaedic surgery normalized competitive index (NCI) across a 20-year period. The NCI significantly increased over time. (R²=0.95, p<0.001).

Applicant metrics

In addition to the increasing competitiveness demonstrated by the NCI, most applicant metrics increased across time. USMLE scores increased over time for matched applicants with a 2.9% increase in Step 1 scores (R²=0.83, p<0.001) and a 4% increase in Step 2 scores (R²=0.85, p<0.001) (Fig. 3A). The mean Step 1 score was 239 (4.6) between 2007 and 2014 and 246 (0.4) between 2015 and 2022 (p = 0.01) (Table 1). Research output quadrupled over the 2007–2021 period (3 vs. 14, p < 0.001) with a significant upward slope over time (R²=0.96, p<0.001). Research experience (3.0 [0.5] vs 4.0 [0.3], p = 0.005; R²=0.90, p<0.001) and work experiences (2.7 [0.3] vs 3.3 [0.2], p = 0.018; R²=0.86, p<0.001) increased over the time period (Table 2, Fig. 3B). Volunteer experiences were not significantly different upon univariate analysis (6.4 [0.7] vs. 7.3 [0.5], p = 0.06). However, there was a significant linear trend across time (R²=0.88, p<0.001) (Fig. 3C).

Fig. 3.

Linear regressions were performed on each applicant metric across time: (A) United States Medical Licensing Exam Step 1 (R²=0.83, p<0.001) and Step 2 (R²=0.85, p<0.001), (B) Research output (R²=0.96, p<0.001) and research experiences (R²=0.90, p<0.001), and (C) Volunteer experiences ((R²=0.88, p<0.001 and work experiences (R²=0.86, p<0.001).

Table 2.

Orthopaedic Surgery Residency Applicant Metrics from 2007 to 2021.

Mean (SD)	2007–2014	2015–2021	p value
USMLE Step 1 score	239 (4.6)	246 (0.4)	0.011
USMLE Step 2 score	243 (6.7)	253 (0.4)	0.013
Research Experiences	3 (0.5)	4 (0.3)	0.005
Research Output	4.6 (1.6)	11 (2.3)	0.002
Volunteer Experiences	6.4 (0.7)	7.3 (0.5)	0.061
Work Experiences,	2.7 (0.3)	3.3 (0.2)	0.018

SD, Standard Deviation; USMLE, United States Medical Licensing Exam.

Discussion

While the rate of matching into orthopaedics has remained constant over the last two decades, the requirements and characteristics of applicants have changed significantly. Over the years, successful matching into orthopaedics has been linked to research experience and board scores, with particular emphasis on the USMLE Step 1 exam [4,5,14]. With USMLE Step 1 transitioning to pass/fail, a significant increase in applicants, increased participation in research years, increasing numbers of program applications, and a substantial drop in this year's match rate, it is unclear how medical students should evaluate the competitiveness of programs and applicants [6,[15], [16], [17], [18]]. Over 20 years, our study assessed applicant metrics and integrated a previously established NCI to provide a comprehensive metric to better assess orthopaedic surgery residency's competitiveness. We discovered that the competitiveness of orthopaedic surgery residency programs has continued to rise, with this year reporting the lowest match rate in the last twenty years. This study also identified increases in both applications and applicant metrics over time. This data demonstrates the growing competitiveness of orthopaedic surgery residency programs and applicants.

Despite minimal changes in match rate, the NCI of orthopaedic surgery residency programs has increased significantly. While this is the first time an NCI has been used in orthopaedics, it has previously been used to assess surgery specialities and fellowships, such as vascular surgery, and general surgery residency competitiveness [[9], [10], [12]]. The NCI stagnated from 2016 to 2019, with a dramatic increase in 2020 and a peak in 2022. The rise in 2020 could be attributed to the merger of the Accreditation Council for Graduate Medical Education and COVID-19 pandemic's restrictions on student rotations and the conduction of virtual interviews [15,19]. Previously, when the NCI was applied to surgical and vascular surgery residencies and fellowships, most programs showed an increase in NCI, similar to our findings [9]. The NCI is expected to rise in the coming years as applicants become more competitive and medical students' interest in surgery grows [20]. In the future, applying the NCI to orthopaedic residencies could be a helpful tool for applicants to use in conjunction with other match metrics to determine the competitiveness of orthopaedic residencies.

According to previous research, the match rate is an unreliable indicator of residency competitiveness [8]. In our study, we discovered no increase in the match rate over time, which could be interpreted as a level of competition stagnation. The growing number of applicants, positions ranked per applicant, NCI, and applicant metrics, on the other hand, all contradict this finding. Although there have been more residency openings since 2003, the number of applications has nearly doubled, with a record high of 1470 candidates for 875 positions this year. According to a survey, orthopaedic residency applicants submitted an average of 83.1 applications [4]. Both residency programs and applicants are struggling to keep up with the increase in applications contributing to match competitiveness [4,5,16]. Because so many variables are involved, accurately measuring competitiveness in terms of a match rate is difficult. Unmatched applicants traditionally rank approximately half the number of programs as those who have successfully matched [1,21]. To provide a more accurate assessment of the level of orthopaedic residency competitiveness, we drew on a previously studied metric that considers the match rate in conjunction with the number of programs ranked per candidate. The originators of the NCIs also proposed that the metric could be applied to individual programs to provide applicants with a better understanding of their chances of matching and minimizing the applications and resources of applicants and programs. Although this data is not publicly available, programs should consider making it available to combat the overwhelming increase in applications [4,9,10].

In addition to our NCI results, we found that applicant metrics such as USMLE Step 1, Step 2, research experience, research output, volunteer activities, and employment experience increased significantly over the 20-year period. Orthopaedic surgery candidates have been reported to have more competitive applicant metrics over time, which is consistent with our findings [1,2,4,5,16,22]. With the current Match state, program directors will focus more on indicators such as Step 2, research experiences, audition electives, preference signaling and letters of recommendation [[5], [7], [23], [24]]. While it is known that medical students more commonly seek research, the research trend was surprising, as it had almost quadrupled over the study period. Recent graduates and medical students often look for extra strategies to stand out from the competition, as it is increasingly common for students to take a research gap year. These research gap years are taken during medical school or after graduation to strengthen residency applications, further emphasizing that residency programs highly value research [7,25,26]. In addition to current literature, our results help elucidate the definition of a competitive applicant with importance likely to be placed on Step 2 scores, research out and experiences, volunteer experiences, audition performance, and letters of recommendation.

This study had several limitations. First, the investigation was conducted retrospectively, utilizing only publicly accessible NRMP website data. Second, various programs may interpret the notion of a competitive applicant differently. For example, some programs may emphasize research and audition performance more than USMLE Step 2 results. Third, although we utilized a normalization method to study the match rate and the number of programs evaluated per applicant, this does not account for application metrics or other applicant characteristics. In addition, the NRMP's publication Charting Outcomes in the Match is derived from applicant-reported USMLE Step-1 and two scores: the number of research, volunteer, and work experiences, which may contain discrepancies. Finally, until the M.D. and D.O. match processes were merged in 2020, application data were reported as "U.S. MD Seniors" and "All Applicants." Since the merger, the NRMP data have become more granular, differentiating between M.D., D.O., and International Medical Graduates (IMG). Consequently, we were unable to distinguish between these applications. As more information becomes available, more research will be required to elucidate differences in applicant competitiveness.

Conclusion

Our study demonstrated the increasing competitiveness of orthopaedic surgery residency programs over twenty years as measured by the NCI. Match rates have inherent limitations as a singular metric and should only be used in conjunction with alternative metrics. A limited number of positions leads to increased competition, as evidenced by significant increases in applicant USMLE scores and research output. This study provides insight into the growing competitiveness of orthopaedic programs, future applicants, and a potential metric to aid in improving the match process for programs and applicants.

Author contributions

Conception and design: VM, JA, MS

Analysis and interpretation: VM, JA

Data collection: VM, JA, MS, RR, SG

Writing the article: VM, JA, MS, RR, SG

Critical revision of the article: VM, JA, MS, RR, SG

Final approval of the article: RR, SG, JA, MS, VM

Statistical analysis: VM

Overall responsibility: RR

Disclosures

The authors have no disclosure.

Source of funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.