Current State of Research Gap-Years in Orthopedic Surgery Residency Applicants: Program Directors’ Perspectives

Abstract

Background

The purpose of this study was to determine how orthopedic residency program directors (PDs) evaluate residency applicants who participated in a research gap-year (RGY).

Methods

A 23 question electronically administered survey was created and emailed to all Accreditation Council for Graduate Medical Education (ACGME) orthopedic residency PDs for the 2020-21 application cycle. PDs were emailed directly if active contact information was identifiable. If not, program coordinators were emailed. The survey contained questions regarding the background information of programs and aimed at identifying how PDs view and evaluate residency applicants who participated in a RGY. Descriptive statistics for each question were performed.

Results

Eighty-four (41.8%) of 201 PDs responded. Most respondent programs (N=62, 73.8%) identified as an academic center. The most common geographic region was the Midwest, N=33 (39.3%). Few programs (N=3, 3.8%) utilize a publication “cut-off” when screening residency applicants. When asked how many peer-reviewed publications were necessary to deem a RGY as “productive,” responses ranged from 0-15 publications (median interquartile range 4.5 [3-5]). Forty-one (53.3%) PDs stated they would council medical students to take a RGY with USMLE Step 1 scores being the #1 factor guiding that advice. More PDs disagree than agree (N=35, 43.6%; vs N=22, 28.2%) that applicants who complete a RGY are more competitive applicants, and 35 PDs (45.5%) agree research experiences will become more important in resident selection as USMLE Step 1 transitions to Pass/Fail.

Conclusion

Program directors have varying views on residency applicants who did a RGY. While few programs use a publication cutoff, the median number of publications deemed as being a “productive” RGY was approximately 5. Many PDs agree that research experiences will become more important as USMLE Step becomes Pass/Fail. This information can be useful for students interested in pursuing a RGY and for residency programs when evaluating residency applicants.

Level of Evidence: IV

Introduction

Orthopedic surgery continues to be one of the most competitive fields in medicine. The percentage of applicants successfully matching to an orthopedic residency position is consistently between 75%-82%.^1-5 Numerous studies have analyzed National Residency Match Program (NRMP) publicly available residency applicant data to determine applicant factors associated with match success.^1,3 Commonly reported findings include higher United States Medical Licensing Exam (USMLE) Step 1 scores (mean 248 for matched applicants compared to mean 239 for unmatched in 2020),^1,5,6 a greater mean number of applicant self-reported research activities in matched applicants (4.6 vs 3.0 for matched compared to unmatched applicant data 2007-2014),⁶ membership in Alpha Omega Alpha (AOA) honor society,^5,6 and attendance at a top-40 National Institutes of Health funded medical school.^5-7 USMLE Step 1 scores have received significant attention as an objective, standardized metric to screen applicants as the number of applications submitted per available residency position is by far the highest across all medical specialties (124 applications per position²).^8,9 On February 12, 2020, it was announced that the USMLE Step 1 scoring will become pass/fail as early as January 1, 2022. This change removes one of the most commonly utilized objective evaluation metrics used by residency program selection committees.^9,10 A recent survey study of orthopedic residency program directors (PDs) inquired about how this change might affect the weight of other applicant variables, including research productivity, in resident selection.⁹ The results of the study by Cohn et al. demonstrated that USMLE Step 2 clinical knowledge (CK) will become the factor that increases the most in importance; however, several other variables, including published research experience, were noted to increase in importance as compared to a similar study conducted in 2002.^9,11

In recent years, the residency selection committee at our institution has noticed a significant number of applicants who have done research gap-years (RGY) during medical school, often resulting in a robust number of peer-reviewed publications in addition to cultivating mentor relationships. Other competitive medical specialties have reported 16%-33% of applicants taking a year off of medical school to accrue research experience.^12-14 The goal of a RGY for many medical students is to strengthen their residency application; however, it is unknown how PDs view and evaluate this aspect of a student’s application. To date, the sole orthopedic study investigating this topic was a review of a single, large academic institution’s 18-year experience offering RGY opportunities. The authors noted a higher match rate for students completing the RGY at their institution compared to published NRMP data.¹⁵

The purpose of this study was to determine how orthopedic surgery PDs evaluate residency applicants who participated in a RGY. A secondary aim was to evaluate for any differences in how residency applicants who participated in a RGY are evaluated based on if a program identified as an academic center or not. The hypothesis was that PDs view students who had productive RGYs as stronger applicants for orthopedic surgery residency selection and academic centers will view RGY applicants more favorably than non-academic centers.

Methods

This study received exemption status from our Institutional Review Board. Accredited orthopedic surgery residency programs for the 2020-2021 application cycle were identified through the Accreditation Council for Graduate Medical Education (ACGME). Program websites and contact information for program coordinators (PCs) and PDs were obtained through numerous methods including residency program websites, Doximity, and back-tracing contact information from PubMed. A total of 201 ACGME-accredited orthopedic residency programs were identified. We were unable to identify an active PD email address for 18 programs (9.0%) and therefore PCs were emailed instead at those programs. No programs were excluded from this study.

Survey Content

The authors collectively formulated a 23 question electronically administered survey (SurveyMonkey®, San Mateo, CA). Nine questions regarding the background information of the responding PDs and their programs were adapted from a recent study by Cohn et al.⁹ These questions inquired about the geographic location of the residency program, the type of program (academic, private practice, community, or county), residency class size, research requirements of the program, if a research track is offered during residency, number of current residents who participated in a research gap-year, years served as PD, age of the PD, and gender of the PD. Twelve questions were created by consensus of all authors a combination of yes/no questions, Likert scales, multiple choice, and free text all aimed at identifying how PDs across the country view and evaluate residency applicants who participated in a research gap-year. The complete survey can be found in Appendix I.

Survey Administration

Similar to Cohn et al.,⁹ a survey link embedded in a short email was sent to all PDs and PCs, with identifiable contact information, by the senior author who is also the PD at the study institution (GJL). The survey was open for a total of 40 days with one follow up email sent to PDs if they had not responded within the first 14 days, and one follow-up phone call if no response within 28 days.

Statistical Analysis

Prior to analysis, all data were assessed for normality using Shapiro-Wilks tests. Descriptive statistics including means and standard deviations, or medians and interquartile ranges were calculated for continuous variables depending on whether the assumption of normality was met. For categorical data, frequencies and percentages were reported. Questionnaire responses were reported in narrative text, tabular format, and visually with pie charts. Likert plots were constructed for select questions using the likert (v1.3.5; Bryer & Speerschneider, 2016) package.¹⁶ Sub-analysis was performed comparing responses between programs that identified as academic centers vs programs that did not identify as academic centers. For the purposes of analysis, community programs, private practice, and county programs were considered a single group, non-academic. Wilcoxon Rank Sum Tests were used to compare continuous variable responses between the academic centers and non-academic subgroups. Categorical response data was evaluated using Chi-squared tests and Fisher’s exact tests when observed counts were less than 5. P <0.05 was considered statistically significant. All statistical analyses were performed using RStudio software version 4.0.4 (R Foundation for Statistical Computing, Vienna, Austria).

Results

Respondent Program Information & Program Director Characteristics

Eighty-four (41.8%) of 201 total PDs and 45.9% (84 of 183) of PDs with known contact information responded to the survey and were included in analysis. It took PDs on average, 4 minutes and 27 seconds to complete the survey. The most common geographic region of respondent programs was the Midwest, N=33 (39.3%).

The median number of residents per class of respondent programs was 5 (range 2-14) with a mode of 5 residents. There was a wide range in the number of years respondent PDs have been in their role (1-30 years). Table 1 details program background information and PD characteristics for respondent programs.

Table 1.

General Residency Program and Program Director Characteristics

Characteristic	No. (%)
Total number of respondentsa	84 (41.8)
Geographic region
Northeast	20 (23.8)
Southeast	7 (8.3)
South	7 (8.3)
Midwest	33 (39.3)
Northwest	5 (6.0)
Southwest	12 (14.3)
Program type
Academic center	62 (73.8)
Private practice	4 (4.8)
Community	18 (21.4)
County	0
No. of residents per residency class (median [IQR])	5 [4-6]
Research year required or offered to residentsb	17 (20.5)
Minimum research requirement to graduate
No research requirement	3 (3.6)
1 research project whether published or not	50 (59.5)
1 published manuscript	14 (16.7)
2 published manuscripts	12 (14.3)
3 or more published manuscripts	5 (6.0)
Require residents to submit at least 1 grant during training	8 (9.5)
No. of current residents who completed a research gap year (median [IQR])c	2 [0-3]
No. of years spent as PD (median [IQR])d	6.0 [3.0-12.0]
PD age (median [IQR])e	51.0 [43.0-58.0]

^aSurvey administered to a total of 201 program directors

^bOne program director (1.2%) did not answer this question

^cFive program directors (6.0%) did not answer this question

^dFive program directors (6.0%) did not answer this question

^eSeven program directors (8.3%) did not answer this question

Abbreviations: IQR, interquartile range; PD, program director; No.: number

Research Gap Year Information

Very few programs (N=3, 3.8%) utilize a “cut-off” for the number of published research articles when screening residency applicants. There was a wide range of responses to the question regarding how many peer-reviewed publications were necessary to deem a RGY as “productive.” Responses ranged from 0-15 publications (median interquartile range (IQR) 4.5 [3-5]) with 7 PDs responding in prose. Those responses can be found in Table 2.

Table 2.

Program Director Prose Responses to Question #14: How Many Peer-Reviewed Publications Are Necessary For You to Consider a Research Gap-Year as “Productive?”

Program Director Textual Responses
Letters of recommendation from that research year is more important
No strict cut-off number
Not sure
No set minimum
Depends on location of gap year
At least one! We will give a nod to presentations (natl mtgs) in place of PMIDs
10 submitted
It depends on the type of research, the journals that are published, and where they are doing the research. If at a big power house and they didn’t publish a lot it is a red flag
Likely 4-5, with about 10 ‘deliverables’ which includes abstracts/posters/podiums. More important: learning skills they can continue to apply

Abbreviations: natl: national; mtgs: meetings; PMID: PubMed Identification Numbers

Regarding the importance of letters of recommendation (LOR) from RGY mentors, most PDs (N=47, 61.0%) responded that clinical LOR are stronger, 26 (33.8%) responded that research LOR and clinical LOR are of equal value, and 4 (5.2%) believe research LOR are stronger. More respondent PDs (N=35, 44.9%) either “strongly disagree” or “disagree’ that applicants who complete a RGY are more competitive applicants compared to those who “agreed” or “strongly agreed” (N=22, 28.2%). Thirty-five PDs (45.5%) agree research experiences will become more important in residency applicant selection as USMLE Step 1 transitions to Pass/Fail. Figure 1 demonstrates in graphic format the responses to questions regarding LOR, competitiveness of applicants who have a RGY experience in comparison to those who do not, and if research experiences will gain further importance in residency applicant selection as USMLE Step 1 transitions to Pass/Fail in 2022.

Figure 1.

Likert plot representation of program director responses to questions #10, #18, and #20. Corresponding colors are immediately to the left of each response level.

Only 16 PDs (20.5%) responded that published research is either “extremely important” or “very important” in creating their applicant rank-lists. In contrast, 35 PDs (45.5%) felt that an applicant’s explanation for why they did a RGY in framing how the PD views the impact of that year was either “extremely important” (N=18) or “very important” (N=17). Figure 2 is a Likert plot representation of responses to these two questions.

Figure 2.

Likert plot representation of program director responses to questions #11 and #17. Corresponding colors are immediately to the left of each response level.

There was a wide range of responses regarding the most beneficial time for applicants to seek a RGY experience. The most common response (N=33, 33.8%) indicated that after USMLE Step 1 was the most optimal time. Eleven PDs, 14.3%, prefer applicants do not do a RGY at all. Figure 3 and Figure 4 graphically depict PD responses to questions 15 and 16.

Figure 3.

Pie chart displaying the responses to Question 15: “How do you compare applicants who have a research gap-year experience with those who don’t?” Scenario: There are two applicants with similar USMLE Step scores, grades, letters, and experiences. Student 1 has no research gap-year experience but has participated meaningfully in a research project while Student 2 has what you consider a “productive” research gap year. Unabridged “Depends on reason” response: The competitiveness of Student 2’s application is in part determined by the reason for taking the research gap-year.

Figure 4.

Pie chart displaying the responses to Question 16: “When do you think a research gap-year experience is most beneficial to increase the competitiveness of an applicant for a residency position, if at all?” Unabridged “After M4 year” and “Prefer no gap-year” responses: “After M4 year, if an applicant does not match” and “I prefer applicants do not do a research gap-year”, respectively.

There was a near equal response amongst PDs regarding whether they counsel medical students to seek RGY opportunities. For the 41 PDs (53.3%) who responded “yes,” the most important variables guiding that advise was USMLE Step 1 score, lack of research experiences, and clerkship grades. Table 3 summates responses to those questions.

Table 3.

Program director responses to questions 21 and 22 regarding counseling of medical students to seek research gap-year opportunities

		No. (%)
Question #21: “Do you ever counsel medical students to seek a research gap-year opportunity?”a		Y: 41 (53.3) N: 36 (46.8)
Rankb (N = 38)		Median [IQR]
1	USMLE Step 1 score	1.0 [1.0-3.0]
2	Lack of research experience	3.0 [2.0-4.0]
3	Clerkship grades	3.0 [2.0-5.0]
4	USMLE Step 2 score	4.0 [3.0-5.0]
5	Class rank	4.5 [4.0-6.0]
6	Life circumstances	5.0 [3.0-6.8]
7	Medical school reputation	6.0 [4.0-7.0]

^aN = 77 responses (91.7%)

^bFor those responding yes, what component of their application indicates to you that a research gap-year would make them more competitive? Factors rated from 1 to 7, with 1 being the most important and 7 the least important

The final question (#23) was a free text response asking PDs “if you have any further comments regarding research gap-years for medical students, please feel free to enter them below.” Sixteen PDs (19.0%) responded with comments other than commenting on their interest in the results of the survey and can be found in Appendix 2.

Sub-analysis Based on Program Type

A total of 62 programs (73.8%) identified as an academic center and 22 (26.2%) identified as either a community program, private practice, or county program (collectively termed “non-academic centers”). There were no significant differences between programs who identified as academic centers and non-academic centers based on responses to questions about minimum research project requirement for residents, a requirement of residents to submit a grant application, and if students who participated in a RGY were considered more competitive applicants (P>0.05, each). Further, there were no differences in responses to the scenario question comparing a RGY applicant to an applicant who did not participate in a RGY, and no difference in responses to when is the most beneficial time to take a RGY (P>0.05, each). No significant findings were identified regarding PD’s response on strength of RGY letters or importance of an applicant’s reason for why they took a RGY (P>0.05, each). Thirty-two programs (55.2%) identifying as academic centers responded that they counsel applicants, if deemed appropriate, to take RGY compared to 9 non-academic programs (47.4%; p=0.744). There were two notable findings. Academic centers were found to have significantly more residents who did a RGY (median IQR: 2 [1.0-5.0]) as compared to non-academic centers (median IQR: 0.5 [0.0-1.0]; P<0.001). In addition, academic centers reported a greater number of peer-reviewed publications to consider a RGY as “productive” (median IQR: 5 [4.0-6.0]) as compared to non-academic centers (median IQR: 3 [2.8-4.3]; P=0.007).

Discussion

The main findings of this study collectively demonstrate that program directors’ views on residency applicants who took a RGY vary widely. Few programs (N=3, 3.8%) utilize a “cut-off” for the number of published research articles when screening residency applicants. There was a wide range of responses to the question regarding how many peer-reviewed publications were necessary to deem a RGY as “productive.” Responses ranged from 0-15 publications with the median response at around 5 publications. Most PDs responded that clinical LOR are stronger than research mentor LOR. In addition, more PDs disagreed than agreed that RGY applicants are stronger residency applicants. Only 53.3% of respondent PDs stated they would council medical students to take a RGY, with the applicant’s USMLE Step 1 score being the most important factor guiding that advice. Finally, 45.5% of PDs agreed research experiences will become more important in residency applicant selection as USMLE Step 1 transitions to Pass/Fail in 2022. While few significant differences were noted between PD responses from academic centers and non-academic centers, programs identifying as academic centers were found to have a greater number of current residents who participated in a RGY than non-academic centers.

Orthopedics remains one of the most competitive specialties in medicine and will likely continue to be for the foreseeable future.^1-5 It has been our experience that more applicants have taken a year out from medical school to pursue research experiences, and it has been challenging to know how to evaluate these students. Of the 36 orthopedic PDs who responded to the NRMP’s 2020 PD survey regarding factors important when ranking applicants, the mean importance rating for “evidence of continuous medical education without gaps” was rated 5% on a 0-100% scale.¹⁷ This indicates that PDs who responded to the NRMP survey do not necessarily believe taking a gap year is adversarial to an applicant’s competitiveness. The results of this survey study of ACGME accredited orthopedic residency PDs highlights the variability of how these RGY experiences are viewed. A similar wide range of perspectives clearly expressed in the written responses of 16 PDs to the final question of the survey, which prompted PDs to add any additional comments. One PD wrote, “A research gap year with our department is considered favorably,” while another responded, “I and my program see this as a weakness.” However, a common theme was notable in many responses. Specifically, the PDs’ perception of these applicants depends in large part on the reason for pursing a RGY. While this was already apparent from the written comments, 18.2% of PDs also responded, “Depends on reason” to the scenario question comparing an applicant who did a “productive” RGY to a similarly competitive applicant who did not pursue a RGY. Some PDs noted that they have found these applicants to have weaknesses in other areas of their applications and may be hoping a productive RGY and strong connections increase their match success. In addition, on sub-analysis by program type, academic centers reported having significantly more current residents who pursued a RGY suggesting that certain types of programs may view RGY applicants more favorably. These findings while interesting, should be interpreted with caution given the small sample size (N=22) of non-academic centers.

Within the orthopedic literature, there is a dearth of research on the topic of RGY in general, but specifically as it pertains to whether these experiences lead to higher match rates. Within plastic surgery, a recent national survey study of integrated plastic surgeon residency applicants from 2013 to 2016 (N=198 respondent applicants) reported that 25% of all applicants did a RGY, and those who did a RGY had a 97% match success rate into plastic surgery compared to 81% for those who did not (P<0.05). Further, in a cross-sectional study of otolaryngology-head and neck surgery (OHNS) applicants from 2014-2015 to 2019-2020, 16% of all applicants had a RGY experience on their application. However, OHNS applicants who had a RGY experience did not have significantly greater odds of matching than those applicants who did not do a RGY (86.9% vs 83.5%, P=0.161). The authors did identify a significantly greater odds of matching into a top 25 OHNS residency program as ranked on Doximity in applicants who participated in a RGY (predicted probability: 58.6% vs 30.5%, adjusted odds ratio: 3.24, P<0.001).¹³ These data demonstrate that among other highly competitive surgical subspecialties, RGY are common and, for certain specialties, those applicants who did a RGY may have higher match rates into top programs than those applicants who did not.

The timing of a RGY may also provide context as to why it was pursued by an applicant, and in turn, influence how programs view that experience. In this study, the most common response to the question of “When do you think a research gap-year experience is most beneficial to increase the competitiveness of an applicant for a residency position?” was after USMLE Step 1. This was a response we were anticipating based upon previously published data demonstrating those with a higher USMLE Step 1 score have a higher match success rate (mean 248 for matched applicants compared to mean 239 for unmatched in 2020).^1,5,6 The importance of a student’s USMLE Step 1 score in deciding on pursing a RGY, or not, is demonstrated in the responses to the survey question regarding counseling of medical students. Of the 53.3% of respondent PDs who stated they do counsel medical students to pursue RGY, if appropriate, the USMLE Step 1 score of the applicant is the most common variable guiding that advice. As USMLE Step 1 becomes pass/fail, it is unknown how this advice may change, and whether it may affect both an applicant’s decision to pursue a RGY and the timing of a RGY. Similar to findings by Cohn and colleagues, 45.5% of PDs in the present study agreed research experiences may become more important once USMLE Step 1 becomes pass/fail.⁹

There was little PD consensus on what constituted a “productive” RGY. When asked directly regarding how many peer-reviewed publications during a RGY that they would consider “productive,” the median response was 4.5 publications with a range from 0-15. We allowed respondents to comment on their answer, and several PDs noted that they have no set minimum when evaluating RGY applicants. On sub-analysis by program type, programs identifying as academic centers did report a higher median number of publications necessary to deem a RGY as “productive” (5 publications vs 3 publications) relative to non-academic centers. Others noted the importance of where the student did their RGY, as this may affect the ease with which a greater number of publications are achievable in a single RGY. In addition, one PD noted that, “letters of recommendation from that research year are more important.” This opinion can be explained by the fact that research mentors may spend more time with applicants during a RGY than clinical instructors during a clerkship. This may allow research mentors to comment more accurately on important attributes such as work ethic, social skills, time management, and ability to follow through on tasks. When asked about how LOR are evaluated from a RGY mentor in comparison to clinical letters, the majority of PDs (N=47, 61.0%) responded that clinical letters of recommendation (LOR) are stronger, 26 (33.8%) responded that research LOR and clinical LOR are of equal value, and 4 (5.2%) believe research LOR are stronger. This data highlights that the majority of PDs weight letters based on applicant’s clinical performance more heavily.

Strengths of this study include the diversity of survey questions and responses on an important topic with a paucity of existing data. Further, the response rate was comparable to a recent study of orthopedic PDs across the country.⁹

Limitations

This study is not without limitations. The data presented herein is reflective only of those program directors who responded to the survey. The data may not be representative of all programs and may be subject to selection bias. Further, only 41.8% of all ACGME accredited programs responded, and an active email address was unable to be identified for 18 PDs. Similar to the study by Cohn et al.,⁹ PDs were surveyed in this study, and their responses may not be representative of all members of their programs’ residency selection committee. Responses were anonymous, so further analysis based on research prowess of programs was not performed.

Conclusion

Program directors have varying views on residency applicants who did a RGY. While few programs use a publication cutoff, the median number of publications to have a “productive” RGY was about 5. Many PDs agree that research experiences will become more important as USMLE Step 1 becomes Pass/Fail. This information can be used by applicants who may be interested in pursuing a RGY and programs when evaluating residency applicants.

APPENDIX - Survey of Orthopedic Residency Program Directors

Program Information (Okay for program coordinator to complete this section)

1. Please enter the geographic region of your program based on the above diagram. (circle one)

   1. Northeast

   2. Southeast

   3. South

   4. Midwest

   5. Northwest

   6. Southwest

2. How would you identify your program (majority of the time spent)?

   1. Academic center

   2. Private practice

   3. Community

   4. County

3. Please enter the number of residents per residency class in your program.

   1. Free text

4. Does your residency program have a research year requirement or offer one for residents who are interested?

   1. Yes

   2. No

   3. Sometimes (25-50%)

   4. Almost Never (<25%)

5. What is the minimum research project requirement to graduate from your residency program?

   1. No research requirement

   2. 1 research project whether published or not

   3. 1 published manuscript

   4. 2 published manuscripts

   5. 3 or more published manuscripts

6. Do you require residents to submit at least 1 grant application during residency?

   1. Yes

   2. No

7. How many current residents in your program did a research gap-year?

   1. (free text)

   2. Yes - Once at first follow-up

   3. Yes - Once at end of treatment

   4. No - Unless reinjury, continued pain or complication occurs

   Program Director Section

   PD Information

8. How many years have you been in the role of Program Director?

   1. Free text

9. Please enter your age:

   1. Free text

   Research Gap Year Information

10. Please indicate the extent to which you agree or disagree with the following statement: in general, applicants who complete research gap-years are more competitive than those who do not.

    1. (5 option response scale: strongly disagree, disagree, unsure, agree, strongly agree)

11. How important is published research in selecting a candidate for your residency rank list?

    1. (5 option response scale: strongly disagree, disagree, unsure, agree, strongly agree)

12. Do you utilize a minimum “cut-off” for the number of published research articles when screening residency applicants? (yes/no)

13. If you answered “Yes” to question #12, please indicate how many publications (accepted or in print) on a CV you consider as your cut-off?

14. How many peer-reviewed publications are necessary for you to consider a research gap-year as “productive?”

    1. Free text #

15. How do you compare applicants who have a research gap-year experience with those who don’t? Scenario: There are two applicants with similar USMLE Step scores, grades, letters, and experiences. Student 1 has no research gap-year experience but has participated meaningfully in a research project while Student 2 has what you consider a “productive” research gap-year experience. (Circle one)

    1. I view these candidates as equally competitive

    2. Student 1 is a more competitive applicant

    3. Student 2 is a more competitive applicant

    4. The competitiveness of Student 2’s application is in part determined by the reason for taking the research gap-year

16. When do you think a research gap-year experience is most beneficial to increase the competitiveness of an applicant for a residency position, if at all?

    1. Before medical school

    2. Before USMLE Step 1

    3. After USMLE Step 1

    4. After M4 year, if an applicant doesn’t match

    5. Timing does not matter

    6. I prefer applicants do not do a research gap-year

17. How important is an applicant’s explanation for taking a research gap-year in framing how you view the impact of that year on their application?

    1. (5 option response scale: Extremely important, very important, somewhat important, not so important, not important at all)

18. Please indicate the extent to which you agree or disagree with the following statement: in general, applicants have stronger letters of recommendation as a result of their research gap year.

    1. (5 option response scale: strongly disagree, disagree, unsure, agree, strongly agree)

19. How do you compare strength of letters of recommendation, that are equally glowing in nature from a research gap-year mentor commenting solely on performance with research related tasks to those from clinical faculty?

    1. Research letters are stronger

    2. Clinical letters are stronger

    3. The letters are of equal value

20. Please indicate the extent to which you agree or disagree with the following statement: With USMLE Step 1 scoring changing to pass/fail, research experiences will increase in importance when evaluating candidates for your residency program.

    1. (5 option response scale: strongly disagree, disagree, unsure, agree, strongly agree)

21. Do you ever counsel medical students to seek a research gap-year opportunity?

    1. Yes

    2. No

22. If you answered “Yes” to question #21, what component of their application indicates to you that a research gap-year would make them more competitive? Please rank the factors below in order of importance (1=most important and 7=least important). Click and drag variables.

    1. USMLE Step 1 score

    2. USMLE Step 2 CK score

    3. Clerkship grades

    4. Lack of research experience

    5. Life circumstances (death of loved one, growing family, medical leave, couple’s match)

    6. Class Rank

    7. Medical School Reputation

23. If you have any further comments regarding research gap-years for medical students, please feel free to enter them below.

    1. Free text

APPENDIX 2 - Program director prose responses to question #23: if you have any further comments regarding research gap-years for medical students, please feel free to enter them below.

Program Director Textual Responses

• I understand why applicants are doing it more often and why PDs are stressing research productivity more. I don’t see a strong correlation between productive research experience and quality of resident quite yet but that may change because now a days even the strong candidates are doing research years whereas usually a gap year was done only if you were considered an otherwise weaker applicant.

• Most students, in my experience, who seek research gap-years do so because their applications are otherwise relatively weak for orthopedic applicants. What are the historical matching data for “gap-year” applicants compared to applicants in general?

• Gap year research should be seen as a positive but when students that are not competitive in SEVERAL of the other parameters that are measured (not just test scores but grades, class rank, letter of recommendation) it is seen as a last-ditch effort and then I believe a research gap year should be discouraged. Research gap year for a student with a solid application can be very helpful, shows level of interest and commitment and helps this type of candidate tremendously.

• I personally did a research gap year, and found it valuable because I was interested in research, which I have continued. I did not need the research fellowship to get into residency. (I was accepted and postponed residency 1 year.) Research gap year is most valuable for individuals who are authentically interested in research. Research gap year can help someone who has poor Step 1 scores be more visible, and someone with weaker 3rd year grades. However, research gap year will not make a poor candidate a strong one. I evaluate research gap year as one factor out of all in evaluating applicants. Our APD also did a research gap year as well as having an MPH. Both of us look very carefully at applicants who did a gap year, with the goal of not being lulled into considering them better/stronger only because of doing a year of research. Many citations in one’s CV does not necessarily make a good candidate.

• For our community-based program, someone who chooses a research gap year because they love research will not be a good fit. If they did the gap year to reapply, then we expect it to be productive.

• I can’t help but think some research gap year applicants are hiding a less than spectacular application coming out of medical school.

• I and my program see this as a weakness. We, as academicians must balance research with clinical (especially in trauma) and most of us do not have ‘protected time’ which I have never seen work in incentivizing anyway. thus, the idea that to be involved in research, while only a med student with so few obligations and stress, requires time off is a good indicator that they do not “have the drive”. there are exceptions, but rare.

• None of this is as important on the resident selection process as audition rotations.

• A research gap year with our department is considered favorably.

• Research experience was useful in the past when it was unusual and allowed a candidate to separate themselves from others. It is no longer useful in that capacity with many having experience and many schools having built-in research requirements. A research year cannot overcome bad grades, bad class rank or bad scores at our institution. This may change in the future when these all become pass/fail.

• Research gap-years after not matching typically are out of desperation (although may give somebody an opportunity to prove themself). A research gap-year between M2 and M3 or before medical school (assuming acceptance was already secured prior to starting gap-year) indicates genuine interest in research and increases importance of gap-year.

• I think a research year that’s unproductive is more detrimental than not doing a research year at all. In my mind, the utility of an applicant’s ability to do research is that if they can publish, do well on scores and on their clerkships SIMULTANEOUSLY it shows good ability to time manage and prioritize which i think are extremely important in residency. The ability to publish research during a year devoted to research especially at a place that is structured to pump out research is less impressive to me. If a resident has a real interest in research, I think it’s great, but that doesn’t prove to me that someone will be a strong resident.

• I am at a big research powerhouse program and there are some on the committee who put a lot of weight on the research. There is NOTHING in our residents that have shown any increased in productivity or quality of residency performance with a gap year. I think it’s ridiculous however, it does make students more competitive for programs and if a student has lower scores (now going away), middle/lower tiered med school with average grades, I encourage them to take a year because of how it is perceived by many programs. We have had some years where we have had a lot of residents who have done research and others not so much and only a few have continuing on with productivity during residency. Nice study.

• The training pipeline is already so long. If a student wants to get a Masters to help with future career, I think that is good. But a cottage industry of research fellowships I think is not good for the students and not good for research in orthopaedics overall. Better, I’d say, to have a six or 12 month track open to students in residency. Being productive with research during a research year, and being productive as a resident or future faculty, are very different! I remind applicants that a year of their life, is not a small thing. I suspect most competitive applicants would still be competitive applicants without a research year. Likewise, many who aren’t that competitive won’t suddenly become more competitive through the extra time. It is a narrow group for whom I think it is helpful for getting into an orthopaedics residents - not zero, but not very big.