The Value of Research in the Orthopaedic Surgery Residency Applicant: A Pilot Survey of Orthopaedic Surgery Residency Program Directors

Abstract

Introduction:

The role of research in the orthopaedic surgery residency application process has become increasingly important, particularly in the wake of the United States Medical Licensing Examination (USMLE) Step 1 examination transitioning to pass/fail. The purpose of this study was to determine the value that orthopaedic surgery residency program directors and other faculty place on various research elements included in residency applications.

Methods:

An anonymous cross-sectional survey was distributed to program directors and other faculty involved with the review of orthopaedic surgery residency applications within the Collaborative of Orthopaedic Education Research Group in February 2024. The survey asked questions related to the value placed on publication types, authorship position, dedicated research years, and the general importance of research.

Results:

Ten respondents, including 7 program directors, 2 assistant program directors, and 1 director of research, participated. Eight (80%) indicated that the number of publications is important, with 4 (40%) identifying a minimum number of publications of 2 to 3 to be considered competitive at their program. Respondents also identified the following components as important: journal impact factor (9; 90%), authorship position (5; 50%), and study type (7; 70%). Clinical research within orthopaedic surgery was most highly regarded (9; 90%), while case studies were deemed least valuable (9; 90%). Six respondents (60%) agreed that research is more important now that USMLE Step 1 is pass/fail.

Conclusions:

Since the transition of Step 1 to pass/fail, research has become an increasingly important part of an orthopaedic surgery resident's application. Residency programs value research productivity, particularly clinical research within orthopaedics. While the minimum number of publications varies, 2 to 3 are often considered competitive. Factors contributing to the perceived value of research include journal impact factor, authorship position, and study type. With this knowledge, applicants can align their research activities with the values of orthopaedic surgery residency programs.

Level of Evidence:

Level IV. See Instructions for Authors for a complete description of levels of evidence.

Introduction

In the 2024 match, approximately 39% (577/1,492) of orthopaedic surgery residency applicants did not match¹. With the transition of the United States Medical Licensing Exam (USMLE) Step 1 to pass/fail, applicants have lost 1 highly predictive objective marker of their application strength². The timing of Step 2 does not align with the strategic planning necessary for orthopaedic surgery residency applications, though it will serve as an important new metric. Many applicants take Step 2 following third year and with recent changes this year by the USMLE, Step 2 scores may not be released until up to 8 weeks following the examination³. Without a clear barometer of an applicant’s strength up to this point, applicants who fail to achieve scores competitive for orthopaedic surgery are left with just months to consider and prepare applications for alternative specialties. It is increasingly important for applicants to understand how various components of their application are valued, as they attempt to gauge their competitiveness in the setting of fewer standardized metrics.

Research plays a crucial role for matching into orthopaedic surgery residency, yet the specific ways in which programs evaluate research productivity is unclear⁴. Current literature regarding orthopaedic surgery residency applicant research is limited by scale (single institution studies) and does not accurately reflect the current climate of the postpass/fail USMLE application process^5,6. Despite these limitations, current literature demonstrates the increasing emphasis on research in the orthopaedic surgery application. In 2023, Martinez et al. examined trends in research of orthopaedic surgery residency applicants from 2007 to 2021 and found that the research productivity of the average applicant had quadrupled during this period⁷. Past studies examining orthopaedic surgery applicant research reported that matched applicants had significant differences in total research items compared with unmatched applicants; however, this finding no longer holds true in the current application climate. A 2023 study by Sudah et al. evaluating orthopaedic surgery residency applicants found no significant difference in the mean number of research products for matched and unmatched US MD seniors⁴. Many recent studies on this topic use the “total research items” to summarize applicant trends; however, this metric includes various items such as posters, presentations, and publications, with no distinction between these subgroups. The purpose of this study was to determine the value that orthopaedic surgery residency program directors and other faculty place on various research elements included in residency applications.

Materials and Methods

Study Design

Following Institutional Review Board approval, an anonymous cross-sectional survey design was used to investigate the role of research in the review of an orthopaedic surgery applicant’s application. At a monthly meeting of the Collaborative of Orthopaedic Education Research Group (COERG), 10 orthopaedic surgery residency programs agreed to participate in this survey. The survey was distributed in February 2024 through email to the 10 agreeing orthopaedic surgery residency programs. The email included a brief explanation of the study and a link to the anonymous Microsoft form survey, along with an informed consent statement. Study participants had 3 weeks to complete the survey. Participants included orthopaedic surgery residency program directors and other faculty who review residency applications. This study design was evaluated using the Checklist for Reporting Results of Survey Studies (CROSS)⁸.

Survey Instrument

The anonymous survey was developed using Microsoft Forms on a secure institutional account. The survey consisted of 28 questions divided into sections assessing programs' perceptions of various research elements (e.g., publication type, authorship position). A full copy of the survey is included in Appendix 1. Questions focused on publication types, authorship position, the importance of early research, and the impact of research postpass/fail USMLE Step 1. Additionally, open-ended questions provided space for qualitative comments.

Data Collection

Anonymous survey responses, as well as standard demographic data (e.g., age, sex, home institution, race), were compiled in a password-protected database. To ensure confidentiality, no personal identifiable information was collected. Data collection was performed over a period of 6 weeks. Only those directly involved in the study had access to this database.

Data Analysis

Statistical analysis was performed using Statistical Package for the Social Sciences for Macintosh, Version 28.0 (IBM Corp).

Results

A total of 10 surgeons responded to the survey (Table I), 7 (70%) of whom were Program Directors, 2 (20%) were Assistant Program Directors, and 1 (10%) was the Director of Research. These 10 respondents constituted 100% of the programs that agreed to participate but represent only a fraction of all orthopaedic surgery residency programs nationally. The largest proportion of respondents were from academic-based practices (60%; 6), followed by community-based practices (30%; 3), and 1 respondent was from a private-academic practice (10%; 1). Geographically, most respondents were from the Northeast (50%; 5), the West Coast and South each represented 20% (2) of participants, and 1 respondent was from the Midwest (10%). Regarding gender identity, most participants identified as male (80%; 8), with 1 participant identifying as female (10%; 1) and 1 as a transgender female (10%; 1). Six respondents (60%) indicated having 0 to 4 years of experience, with the remaining 40% (4) having 5 to 9 years of experience in their current position.

TABLE I.

Demographics of Survey Participants

Demographics of Study Participants
	Count (N)	Response %
Position
Program Director	7	70.0
Assistant Program Director	2	20.0
Director of Research	1	10.0
Total	10
Practice type
Academic-based practice	6	60.0
Community-based practice	3	30.0
Private-Academic practice	1	10.0
Total	10
Geographical region
Northeast: CT, ME, MA, NH, NJ, NY, PA, RI, VT	5	50.0
West: AK, AZ, CA, CO, HI, ID, MT, NV, NM, OR, UT, WA, WY	2	20.0
South: AL, AR, DE, (DC), FL, GA, KY, LA, MD, MS, NC, OK, SC, TN, TX, VA, WV	2	20.0%
Midwest: IL, IN, IA, KS, MI, MN, MO, NE, ND, OH, SD, WI	1	10.0
Total	10
Gender
Male	8	80.0
Female	1	10.0
Transgender female	1	10.0
Total	10
Years of experience
0-4 years	6	60.0
5-9 years	4	40.0
10+ years	0	0.0
Total	10

Publications

Most respondents (80%; 8) “somewhat agree” or “strongly agree” that the number of publications is important to be considered a competitive applicant for the orthopaedic surgery match. While there was no clear agreement on a minimum number of publications required for competitiveness, the largest percentage of respondents (40%; 4) indicated that “2 to 3 publications” were sufficient to be considered competitive (Fig. 1). Interestingly, 2 programs (20%) did not specify a minimum number of publications, while some programs appeared to have higher research expectations, with 1 indicating a preference for applicants with 10 or more publications.

Fig. 1.

Minimum number of publications to be considered a competitive orthopaedic surgery residency applicant.

The impact factor of the journal of publication was considered important, with 8 respondents (80%) selecting “somewhat agree” and 1 (10%) selecting “strongly agree.” An applicant's authorship position was also important, with 5 respondents (50%) selecting “somewhat agree” or “strongly agree”; however, 4 (40%) selected “neutral” regarding authorship position’s importance. The type of study (prospective vs retrospective vs case report) was important as well, with 7 respondents (70%) selecting “somewhat agree.”

Type of Research

Clinical research studies within orthopaedic surgery were most highly valued, with 9 respondents (90%) ranking them as their first choice. Clinical research in a nonorthopaedic field was the second most valued type of research for many of the respondents (40%; 4), followed by educational studies in orthopaedic surgery (40%: 4). Basic science studies and case reports were less preferred, with the majority of rankings (90%; 9) placing case reports as the least important research type (Fig. 2).

Fig. 2.

Research study-type ranking by respondents.

Presentations

The type of conference (local, regional, national) was considered important, with 9 respondents (90%) selecting “somewhat agree” or “strongly agree.” While many of the respondents (50%; 5) agreed that being the presenting author at a conference was important, a significant cohort (40%; 4) were neutral on this topic, suggesting that presenting authorship may not be a universally required criterion. Podium presentations were favored over poster presentations, with 6 respondents (60%) selecting “somewhat” and 3 (30%) selecting “yes, strongly.”

General Research Topics

There was no strong consensus among participants when asked if research before medical school was viewed the same as research performed during school. Responses were evenly divided between “disagree” (20%; 2), “neutral” (20%; 2), “somewhat agree” (30%; 3), and “strongly agree” (20%; 2). One respondent selected “strongly disagree” (10%; 1). Six respondents (60%) indicated that research has become more important when reviewing applications since USMLE Step 1 transitioned to pass/fail. There was no clear consensus regarding whether a dedicated research year improves applicant competitiveness; however, slightly more respondents disagreed with this statement (Fig. 3).

Fig. 3.

Does a dedicated research year increase an applicant's competitiveness for orthopaedic surgery residency?

Regarding time spent reviewing the research section of an application, 6 respondents (60%) indicated spending 3 to 5 minutes, 2 (20%) spending marginally less (1-3 minutes), and 2 (20%) spending marginally more (5-10 minutes).

Discussion

The purpose of this study was to determine the value that orthopaedic surgery residency program directors and other faculty place on various research elements included in residency applications.

The key findings from our survey include the following: (1) 80% of respondents indicated that the number of publications is important, with 40% identifying 2 to 3 publications as the minimum to be considered competitive; (2) clinical research within orthopaedic surgery was most highly valued; (3) journal impact factor, authorship position, and study type were considered important factors; (4) 60% of respondents agreed that research has become more important since USMLE Step 1 became pass/fail; and (5) premedical school research and dedicated research years did not hold universal value among program directors.

Our findings of the increased importance of research since the USMLE Step 1 transition to pass/fail align with and extend the narrative reported by Martinez et al., which documented a rapid increase in research items among applicants over the last decade⁷. Unlike previous studies that used broader metrics such as “total research items” (including abstracts, presentations, and publications), our findings provide more specific guidance. Program directors expressed diverse opinions regarding minimum publication requirements, ranging from no specific number to over 10. Despite this variation, the most commonly reported threshold was 2 to 3 publications, suggesting that applicants should aim for at least this number to be competitive. This aligns with Abbas et al.'s findings, which reported a mean of 4.12 peer-reviewed publications among the class of 2027 residents⁹. Research years have been an increasingly popular option for applicants hoping to bolster their publication record; however, they may not be universally desirable or beneficial¹⁰. Our findings demonstrate a lack of clear consensus among program directors and suggest that applicants should carefully weigh their individual goals, application, and career aspirations before deciding whether a dedicated research year is the optimal path.

The survey also revealed important preferences for the types of research, with a clear preference for work published in orthopaedic journals, as 90% of participants ranked it highest. This finding contrasts with prior literature by Toci et al., which found no significant difference in the matched program tier between applicants with orthopaedic-related publications and those with publications in other fields, concluding that orthopaedic-specific research was not significantly more beneficial in the match.⁵ In addition, the quality of research published mattered, as 9 respondents (90%) considered journal impact factor to be important. These findings suggest that medical students should strive for clinical orthopaedic publications in high-impact journals.

Many of the respondents' preferences closely aligned with the rigor of the research item, favoring podium presentations over poster presentations, and 9 (90%) finding the type of conference to be important (presumably with a preference for national conferences, albeit not explicitly asked). However, the data also suggests that applicants can still gain value from their work being presented, even if they are not the presenting author, as 4 respondents (40%) remained neutral when explicitly asked the importance of the applicant presenting the research. In addition, while 5 respondents (50%; 5) found authorship position to be an important selection factor, just under half (40%; 4) of respondents rated authorship position’s importance as neutral. These findings are reassuring and suggest medical students should contribute to projects even in supporting roles.

Study participants were also provided space to share general comments on research and red flags for applicants, out of these comments 2 major themes arose:

• Theme #1: The Essential Role of Research

Overall, a clear consensus emerges among program directors that research plays a central role in ranking orthopaedic surgery residency applicants. While there's no need for excessive publications, demonstrating some research initiative is now considered essential. As 1 program director states, “An empty research section” is a red flag, while another emphasizes, “Need some to be a competitive candidate.” Research has become a key way to differentiate orthopaedic surgery applicants in the postpass/fail Step 1 era, with 1 program director noting, “It is an important factor, but not the most important.”

• Theme #2: Quality Over Quantity

Program directors emphasize a nuanced evaluation of research, placing significant value on an applicant's depth of knowledge and skills rather than mere volume. “A couple well thought out research contributions by an applicant who is clinically ready for residency is all that is necessary,” 1 respondent remarks. Another echoes this sentiment: “I'd rather someone build skills… but have less deliverables rather than having more deliverables but no specific… skills.” Red flags include a lack of understanding of one's research during interviews and questionable verifiability of the research itself. Some programs are wary of potential exploitation in research years and look favorably on initiatives that demonstrate the development of practical skills such as coding or research design. However, there is a recognition that access to research opportunities varies across institutions and this should be contextualized.

In the postpass/fail USMLE Step 1 landscape, research has become a cornerstone factor in differentiating a competitive orthopaedic surgery residency applicant. Understanding the nuanced ways in which research is evaluated empowers future applicants with invaluable insights to optimize their positioning. By emphasizing clinical orthopaedic surgery research, formal publications, conference presentations, and impactful authorship, applicants can significantly improve their chances of securing a highly coveted residency position within orthopaedic surgery.

Limitations

Although these results provide a valuable snapshot of how some programs weigh research, the small sample size limits generalizability. Given our sample of 10 programs, this investigation should be viewed as a pilot study offering preliminary insights. Future efforts to survey a wider array of orthopaedic residency programs, potentially involving the entire COERG membership or other national consortia, could strengthen these findings and offer a more comprehensive perspective. In addition, as with any survey-based research, there is potential for response bias among program directors. However, the detailed open responses from participants offer valuable contextual insights, supplementing the survey and strengthening the robustness of our findings despite the sample size constraints.

Conclusion

Our study finds that since the transition of Step 1 to pass/fail, research is an increasingly important part of an orthopaedic surgery resident’s application. Orthopaedic surgery residency programs value research productivity, with clinical research within orthopaedics being the most highly regarded. While the minimum number of publications varies, 2 to 3 publications are often considered competitive. In addition, several factors contribute to the perceived value of research, including journal impact factor, authorship position, and study type. With this knowledge, applicants can make informed decisions about their research activities, ensuring their efforts align with the values of orthopaedic surgery residency programs.

Appendix

Supporting material provided by the authors is posted with the online version of this article as a data supplement at jbjs.org (http://links.lww.com/JBJSOA/A831). This content was not copyedited or verified by JBJS.