Assessing the Impact of COVID-19 on the 2021 Orthopaedic Surgery Match Outcomes

Background:

The purpose of this study was to assess the influence of the COVID-19 pandemic on the 2021 orthopaedic surgery residency match outcomes. Because in-person away rotations and interviews were canceled during the 2020 to 2021 application cycle, we hypothesized that applicants would match at their home program at a higher rate in 2021 than in previous years.

Methods:

We queried allopathic orthopaedic surgery residency websites and social media accounts for names of residents and medical school information for cohorts matching from 2017 through 2021. To assess availability of and participation in virtual away rotations, we administered a survey to Accreditation Council for Graduate Medical Education–accredited allopathic orthopaedic surgery residency programs. The primary outcome was the annual proportion of applicants matching at the program affiliated with their medical school (“home program”). Subgroup analyses were stratified by Doximity reputation ranking and availability of a virtual away rotation.

Results:

We identified 2,632 residents who matched between 2017 and 2020 and 698 residents who matched in 2021. Overall home program match rate and likelihood of home matching were higher in 2021 compared with 2017 to 2020 (28% vs. 20%; odds ratio [OR] = 1.51, 95% confidence interval [CI] 1.24-1.82, p < 0.001). The increase in the home match rate at programs ranked in the top 30 (27% vs. 20%, p = 0.034) was similar to the increase at programs ranked outside the top 30 (28% vs. 20%, p < 0.001). Of the 66 (48%) programs that responded to the survey, 16 (24%) offered a virtual away rotation. Programs with a higher Doximity ranking were more likely than lower-ranked programs to offer a virtual away rotation (OR = 6.75, 95% CI 1.95-23.4, p = 0.003). Home match rates did not differ significantly between programs that offered a virtual away rotation and those that did not (26% vs. 32%, p = 0.271).

Conclusions:

A higher proportion of orthopaedic surgery residency applicants matched at their home program in 2021 compared with previous years. Limitations on in-person activities due to the COVID-19 pandemic may have contributed to this rise.

Level of Evidence:

N/A.

Introduction

The COVID-19 pandemic disrupted several key factors in the orthopaedic surgery residency match process. When evaluating applicants, residency programs historically have valued high US Medical Licensing Examination (USMLE) scores, research productivity, and applicants' performance on visiting subinternships (“away rotations”) outside of their home institution^1-4. Away rotations can be especially impactful because they facilitate programs' assessment of applicants and applicants' evaluation of prospective programs^5,6. Due to the pandemic, however, many applicants had their USMLE testing dates delayed and research projects interrupted⁷. Nearly 90% of applicants surveyed in one study reported that pandemic-related restrictions limited their clinical exposure to orthopaedics before they applied for residency⁸. The same study noted that women and African American students were less likely than men and non-Hispanic White students to apply for an orthopaedics residency due to the pandemic⁸. Finally, and most notably, in-person away rotations and interviews were universally canceled for the 2020 to 2021 orthopaedics residency application cycle^9,10.

Applicants and programs were forced to adapt to these major changes. To compensate for the loss of vital in-person experiences, many programs developed novel virtual alternatives such as formal virtual away rotations, entirely virtual interviews, and virtual happy hours with current residents^11-13. In addition, Wang et al. reported a 355% rise in use of social media by residency programs looking to recruit potential applicants and convey program strengths¹⁴. Meanwhile, applicants applied to more programs than in prior years and were encouraged to bolster their applications by maximizing their USMLE Step 2 Clinical Knowledge score, securing strong letters of recommendation, and focusing on extracurricular activities such as research and volunteering^15,16. Previous studies have characterized the responses of programs and applicants during this unprecedented application cycle, but limited information exists on how the pandemic ultimately affected the 2021 match outcomes.

The primary aim of this study was to assess the influence of the COVID-19 pandemic on the 2021 orthopaedic surgery match outcomes. We hypothesized that the home institution match rate would be higher in 2021 than in prior years because of the lack of in-person experiences available at other institutions. Secondarily, we evaluated the relationships between match outcomes, program ranking, and participation in virtual away rotations.

Methods

Resident Sample and Home Match Rate

We identified allopathic orthopaedic surgery residency programs using the Accreditation Council for Graduate Medical Education (ACGME) website¹⁷. Subsequently, we collected resident information from the program websites and official social media platforms (Instagram, Facebook, and Twitter) for cohorts matching from 2017 to 2021. Collected data included resident names, medical schools, and residency class size. This information was used to determine which residents matched into their home program (“home match rate”). Home program was defined as any orthopaedic surgery residency program affiliated with an applicant's medical school.

If a program did not report the medical schools attended by their residents, that program was excluded from the home match analysis and subanalyses for that year. Programs affiliated with the US military were also excluded because of differences in selection criteria for their residents¹⁸.

Program Ranking and Virtual Away Rotations

We performed subanalyses by categorizing all programs based on Doximity reputation rankings and on whether they offered a formal virtual away rotation in 2020. Doximity is a popular digital platform used by medical professionals, and its orthopaedic surgery residency reputation rankings are determined annually by survey responses from board-certified orthopaedic surgeons¹⁹. These rankings are important to applicants and have been shown to affect match list rankings^20,21. We recorded the numerical ranking of each program. A lower numerical value indicated a higher ranking, with a ranking of 1 being the highest (e.g., the third-ranked program was assigned a value of 3, and the twelfth-ranked program was assigned a value of 12). Programs were further divided into 2 tiers: those ranked in the top 30 and those ranked lower than the top 30.

In addition, we administered a survey to all included ACGME-accredited orthopaedic surgery residency programs. Both program directors and program coordinators were contacted for study participation to maximize the response rate, and their emails were acquired through the ACGME website. To characterize the geographic distribution of survey respondents, we grouped programs by their geographic region (South, Northeast, Midwest, West) as defined on Doximity²². Emails explained the academic purpose of the survey and posed questions to determine (1) whether the program offered a formal virtual away rotation for visiting medical students during the 2020 to 2021 application cycle and (2) whether the medical students who matched into the program in 2021 participated in the virtual away rotation if one was offered. No incentives were offered for survey participation. Email reminders were sent 4 times, 14 days apart, to programs that had not yet completed the survey.

Statistical Analysis

For this study, we analyzed data using bivariate statistical analysis and provided descriptive statistics. For continuous variables, a 2-tailed Student t test was used to compare groups. Chi-squared tests were used for dichotomous variables, and the Fisher exact test was used for variables with fewer than 5 observations. Odds ratios were generated using univariate logistic regression to identify the likelihood of matching at one's home institution by the tier of program Doximity rank, matching in the 2021 year, and whether the institution offered a virtual rotation in 2020. A p-value of less than 0.05 was considered statistically significant. We used Stata Statistical Software, 17.0 SE (StataCorp LLC; 2021), to conduct all analyses.

Results

Home Match Rate

Table I presents the annual home match rates for orthopaedic surgery residency programs between 2017 and 2021. The number of matched applicants and the number of programs for which residents' medical school information was available each year are listed. The overall home match rate during this period was 21.9%. The annual home match rates were 22.2% in 2017, 19.5% in 2018, 19.6% in 2019, 20.2% in 2020, and 27.8% in 2021. The annual home match rates were not significantly different (p > 0.05) between the Doximity top-30 programs and programs ranked outside the top 30 (Fig. 1).

Fig. 1.

Home match rates between 2017 and 2021 based on Doximity reputation ranking. No significant differences (p > 0.05) were found in any year when comparing programs in the top 30 vs. programs below the top 30.

TABLE I.

Home Match Rates for Allopathic Orthopaedic Residency Programs, 2017 to 2021^*

	2017	2018	2019	2020	2021	Overall
Included programs (n)
Overall	127	131	133	131	139	661
DR 1-30	26	26	27	27	29	135
DR 31-NR	101	105	106	104	110	526
Total residents matched (n)
Overall	634	663	668	667	698	3,330
DR 1-30	183	190	193	202	210	978
DR 31-NR	451	473	475	465	488	2,352
Home match (n)
Overall	141	129	131	135	194	730
DR 1-30	41	36	35	44	57	213
DR 31-NR	100	93	96	91	137	517
Home match rate (%)
Overall	22.2	19.5	19.6	20.2	27.8	21.9
DR 1-30	22.4	18.9	18.1	21.8	27.1	21.8
DR 31-NR	22.2	19.7	20.2	19.6	28.0	22.0
p-value for DR 1-30 vs. DR 31-NR	0.949	0.834	0.540	0.513	0.801	0.898

^*DR = Doximity reputation rank, and NR, not ranked.

The home match rate also did not vary significantly between 2017 and 2020 (p = 0.838). The overall home match rate during this 4-year period was 20.4%. In the 2021 match, however, the home match rate increased significantly to 27.8% (p < 0.001) (Fig. 2). Compared with previous years, programs ranked in the top 30 (27.1% vs. 20.3%, p = 0.034) and lower-ranked programs (28.0% vs. 20.4%, p < 0.001) experienced significantly higher home match rates in 2021 (Fig. 2). Overall, students matching in 2021 were 51% more likely to match at their home institution compared with students who matched between 2017 and 2020 (odds ratio [OR] = 1.51, 95% confidence interval [CI] 1.24-1.82, p < 0.001).

Fig. 2.

Comparison of home match rates between 2017–2020 and 2021. Significant differences between groups are indicated by an (*) if p ≤ 0.05, (**) if p ≤ 0.01, or (***) if p ≤ 0.001.

Virtual Away Rotation

Responses were obtained from 66 of the 139 programs (47.5%) that received the survey. The geographic distribution of programs that responded was 28.6% South, 28.6% Midwest, 23.3% Northeast, and 19.5% West. The median Doximity reputation ranking of these programs was 67 (interquartile range [IQR] 28-97). Seventeen of these programs (25.8%) were ranked in the Doximity top 30, and the remaining 49 (74.2%) were ranked lower. The median number of students who matched at these programs in 2021 was 5 (IQR 4-6). In total, these programs matched 339 of the 849 applicants (39.9%) who successfully matched into orthopaedic surgery in 2021.

Sixteen of these programs (24.2%) reported offering a formal virtual rotation for visiting medical students during the 2020 to 2021 application cycle. The programs that offered virtual away rotations matched 93 residents in 2021. Among these residents, 49 (52.7%) completed a virtual away rotation at the program they matched into, 24 (25.8%) matched into their home program, and 20 (21.5%) matched at programs with which they were unaffiliated. In addition, the home match rate among all respondents was 30.3% (102/339). The home match rate was not significantly different between programs that offered a virtual away rotation and those that did not (25.8% vs. 32.0%, p = 0.271).

The average Doximity reputation ranking of programs that offered a virtual away rotation (36.2, 95% CI 20.9-51.4) was significantly higher (p < 0.001) than the average ranking of programs that did not offer a virtual away rotation (78.1, 95% CI 65.6-90.6). Compared with lower-ranked programs, programs ranked in the Doximity top 30 were significantly more likely (OR = 6.75, 95% CI 1.95-23.4, p = 0.003) to offer a virtual away rotation.

Discussion

Orthopaedic surgery continues to be one of the most competitive residency fields for medical students to match into, with just 64.9% of senior medical students in the United States matching successfully in 2022²³. Navigating the match process became more challenging during the COVID-19 pandemic, which severely limited in-person interactions between applicants and programs. To our knowledge, this is the first study to examine the possible effects of the pandemic on 2021 orthopaedic surgery residency match outcomes. Our findings validated our hypothesis that a greater proportion of students applying for orthopaedics residency in 2021 would match at their home program compared with previous application cycles. These results mirror the findings of similar recent studies in plastic surgery, urology, otolaryngology, and dermatology^24-28.

In this study, approximately 20% of students who matched into an orthopaedic surgery residency between 2017 and 2020 matched at their home program. Across this period, home match rates were comparable between the Doximity top 30 programs and programs ranked lower. The home match rate for orthopaedics residencies seems to have remained rather stable over the previous decade. Cox et al. reported that 21% of orthopaedics applicants matched into their home program between 2011 and 2015, with no significant changes to home match rates occurring in that period²⁹. Affiliated students, faculty, and residents generally gauge compatibility through formal subinternships, research projects, or informal shadowing. Importantly, students are afforded an extended look into resident camaraderie and program culture at their home institution, and programs have more opportunities to assess students' work ethic and teamwork skills. Greater familiarity with these critical intangible factors may facilitate matches between students and their home programs.

For the 2021 orthopaedic surgery match, however, we found that the overall home match rate increased to 28%, which was significantly higher than in prior years. This trend was similar across programs, regardless of their Doximity reputation ranking. COVID-19–related changes may have increased the likelihood of orthopaedics applicants matching at their home programs. In a typical match cycle, in-person away rotations play a major role in guiding the preferences of applicants and programs^3-6. During the 2020 to 2021 cycle, however, 82% of applicants altered their rank list due to the lack of in-person away rotations¹¹. Applicants and program directors also felt that residency program social events during the cycle were not replicated well in the virtual setting¹¹. Without adequate opportunities to evaluate each other, unaffiliated programs and applicants may have weighed home program affiliation more heavily when they developed their match rank lists³⁰.

In addition, we found that program ranking was strongly associated with likelihood of offering a virtual away rotation. Virtual away rotations have rapidly emerged as an innovative response to the cancellation of in-person aways during the pandemic. However, less than half of programs surveyed in this study reported offering a virtual away rotation in 2020, which was similar to the findings of previous studies^11,14. Furthermore, programs ranked in the Doximity top 30 were more likely than lower-ranked programs (OR = 6.75) to offer a virtual away rotation. Although the reason for this finding was not assessed in this study, higher-ranked programs might have expected to receive greater interest from unaffiliated students and therefore might have been more motivated to develop a virtual away rotation to accommodate these prospective applicants. Despite their inherent limitations, virtual rotations can offer applicants greater flexibility to connect with more programs of interest while mitigating the financial burden and stresses associated with securing in-person away rotations^31,32. The results of this study suggest that applicants to lower-ranked programs might have been less likely to experience the benefits of virtual away rotations during this application cycle.

Among applicants who matched at programs that offered a virtual rotation, 53% completed a virtual rotation at the program they matched into. Slightly higher match rates have been reported for students who completed in-person away rotations in prior cycles. For example, a survey of students in the 2013 orthopaedics match found that 58% matched at a program where they performed an in-person away rotation³³. A possible explanation for this difference is that programs may have placed less emphasis on virtual rotations when considering applicants during the 2020 to 2021 application cycle. Indeed, program directors ranked virtual rotations in the 2020 to 2021 cycle as a less important selection criterion than away rotations in the previous cycle¹⁴. Programs were also less likely to invite back virtual away rotators for an interview compared with in-person rotators in prior years^11,12. It remains unclear how student performance on virtual away rotations will factor into the orthopaedics applicant evaluation process as in-person away rotations resume.

This study was not without limitations. Determination of annual home match rates depended on the availability of information found on program websites and social media, and the lack of data for residents at some programs precluded a comprehensive assessment. The study also did not assess the relationship between 2021 match outcomes and certain historically important selection criteria such as clerkship grades, research productivity, or Alpha Omega Alpha status. Furthermore, the Doximity reputation rank system is one of the multiple systems used to judge the quality of orthopaedics residencies. Another example is the Jones ranking, first described by Jones et al., which ranked programs based on the number of citations received by publications from the orthopaedics department from 2005 to 2015³⁴. The Doximity system was chosen because it reflects the opinions of orthopaedic surgeons; accordingly, it must be interpreted in light of its inherent subjectivity. In addition, this study considered formal virtual away rotations as a single entity, but the components of virtual away rotations—such as duration, didactic sessions, and interaction with attending physicians—may have varied between programs^12,13. The findings related to virtual away rotations may also be subject to response bias due to the survey response rate of 48%. However, the response rate was comparable with previous studies surveying orthopaedics residency programs^14,35,36. The median program ranking, program size, and geographic distribution of programs that responded may also enhance the generalizability of our results.

Conclusion

Without in-person away rotations or interviews due to the COVID-19 pandemic, orthopaedic residency programs matched applicants from affiliated medical schools at a higher rate in 2021 than in prior years. Home match rates increased similarly regardless of program ranking. Although higher-ranked programs were more likely to offer a virtual away rotation, offering a virtual away rotation did not significantly affect the home match rate. The pandemic accelerated the evolution of the orthopaedic surgery match landscape by prompting the creation of virtual substitutes for customarily in-person experiences. This study offers insight into the effect of these changes on match outcomes. Future studies may evaluate how match outcomes are affected when both in-person and virtual away rotations are offered during the same application cycle.