ABSTRACT
Introduction
The residency application process is a critical time for medical students. The COVID-19 pandemic prompted changes to the residency recruitment procedures with the conversion of interviews to a virtual format. For medical school advisors guiding students on an all-virtual residency application process brought uncertainty to their advising practices. Thus, this study aimed to identify advising practices during the 2021 virtual application cycle.
Methods
We administered an IRB-exempt national survey through the Clerkship Directors in Internal Medicine to 186 internal medicine core/co-/associate/assistant clerkship directors and sub-internship directors representing 140 Liaison Committee on Medical Education-accredited U.S./U.S.-territory-based medical schools in spring 2021. The 23-question survey was designed and pilot-tested by faculty-educators and leaders with expertise in undergraduate medical education. Data analysis included paired t- and z-tests and thematic analysis of open-ended questions.
Results
The institutional response rate was 67% (93/140) and individual rate 55% (103/186). Half of the respondents felt prepared/very prepared (40% and 13% respectively) for their advising roles. Compared to pre-pandemic cycles, respondents advised a typical student in the middle-third of their class at their institution to apply to more residency programs (mean 24 programs vs 20, p < 0.001) and accept more interviews (mean 14 interviews vs 12, p < 0.001). Sixty-three percent (64/101) of respondents spent more time on student advising; 51% (51/101) reported more students asked them for informal advice. Fifty-nine percent (60/101) of respondents reported their advisees were able to assess a residency program ‘somewhat well;’ 31% (31/101) expressed that residency recruitment should remain entirely virtual in the future.
Conclusion
The transition to virtual residency recruitment due to COVID-19 prompted advising practices that may have contributed to application inflation and increased advising workload. Future studies should explore longitudinal outcomes of virtual interviews on student success to guide best practices in how to advise students during residency recruitment.
KEYWORDS: Medical student advising, clerkship, clerkship directors, sub-internship directors, residency interviews, residency application, virtual residency recruitment
Introduction
The residency application process is a stressful time for medical students. Advising can help students navigate the process and maximize their success in the Match [1,2]. The COVID-19 pandemic prompted changes in the residency recruitment processes with the conversion of interviews to a virtual format. These changes raised concerns about the impact of virtual interviews on residency application inflation [3,4], and match success rates [5], and questioned best student advising practices in virtual residency recruitment. Thus, the purpose of this study was to identify advising practices during the 2021 virtual application cycle through a national survey administered by the Clerkship Directors in Internal Medicine (CDIM) and offer a potential reference for medical student advisors.
Methods
From March to May 2021, we administered an online survey about the virtual IM residency application process to 186 IM core and co-/associate/assistant CDs, and sub-internship directors representing 140 Liaison Committee on Medical Education fully-accredited U.S./U.S.-territory-based medical schools. The 23-question survey was designed, revised after multiple iterations, and pilot-tested by CDIM faculty-educators and leaders with expertise in undergraduate medical education. Eleven of the survey questions focused on student advising for residency (Appendix 1). Data analysis was conducted in Stata 16.1 and included paired t- and z-tests to compare application cycle outcomes between the pre-2020-21 and the 2020–21 application cycles. For open-ended questions, thematic analysis was conducted following an iterative approach. The study was deemed exempt by Pearl IRB (U.S. DHHS OHRP #IRB00007772).
Results
The institutional response rate was 67% (93/140) and individual rate 55% (103/186). Almost all respondents (98%, 101/103) served as student advisors for residency. However, only half felt prepared/very prepared (40% and 13% respectively) for their advisor roles.
Compared to pre-pandemicIM residency application cycles, respondents advised a typical student in the middle-third of their class at their institution to apply to more residency programs (mean 24 programs vs 20, p < 0.001) and accept more interviews(mean14 vs 12, p < 0.001); spent more time advising students (63%, 64/101); and received more requests from students (37%, 31/84) and other individuals (19%, 15/79) to contact residency programs and advocate for students. Half of respondents (51%, 51/101) reported that more students asked for informal advice during the 2020–21 cycle (Table 1).
Table 1.
Advising practices for internal medicine residency applicants during the virtual application cycle 2020–2021.
| Comparing the 2020–21 Virtual Application Cycle with Prior Application Cycles (n = 101)* | Decreased No. (%) | About the same No. (%) | Increased No. (%) |
|---|---|---|---|
| Total number of students advised | 4 (4) | 75 (74) | 22 (22) |
| Number of students who asked for informal advice | 2 (2) | 48 (47) | 51 (51) |
| Time spent advising students | 0 (–) | 37 (37) | 64 (63) |
| Number of students who asked advisor to contact a residency program or advocate on their behalf** | 5 (6) | 48 (57) | 31 (37) |
| Number of requests from individuals (e.g., other than students) to advocate for a student*** | 9 (11) | 55 (70) | 15(19) |
| Number of residency programs respondents contacted on behalf of student(s)^ | 5 (6) | 52 (63) | 25 (31) |
| n = 101 | Pre-2020-2021 application cycle | During 2020–2021 application cycle | |
| Mean (SD) | Mean (SD) | P-value^^ | |
| Number of programs a typical student in the ‘middle-third of their class ’ was advised to apply to | 20 (7) | 24 (9) | <0.001 |
| Number of interviews a typical student in the ‘middle-third of their class’ was advised to accept | 12 (3) | 14 (5) | <0.001 |
SD: standard deviation.
*For 101 respondents who reported to advise students at their institution about internal medicine residency applications. The question asked: ‘How did the following change during the 2020–2021 residency application cycle?’
**Excludes 17 respondents who reported ‘Not applicable:’ n = 84.***Excludes 22 respondents who reported ‘Not applicable:’ n = 79.^Excludes 19 respondents who reported ‘Not applicable:’ n = 82.
^^Paired t-test and paired z-test (using standard deviation of the difference): n = 101.
Fifty-nine percent of respondents (60/101)reported that their advisees could assess a residency program during the virtual interview day ‘somewhat well,’and 9% (9/101) ‘very well.’ From free text responses to a question about challenges their advisees experienced due to virtual residency interviews, 81% (61/75) of respondents felt the virtual interview format made it difficult for students to ‘get a good feel’ of a residency program and its culture, the hospital, and the location. Additionally, 31% (31/101) of respondents expressed that residency recruitment should remain entirely virtual.
Discussion
The transition to virtual residency recruitment due to the COVID-19 pandemic brought uncertainty and questioned best advising practices. Although, many organizations (e.g., Alliance for Academic Internal Medicine) offered resources for virtual interview preparation for students, advisors, and programs [6,7], our results showed that only half of the respondents felt very prepared/prepared for their roles as student advisors.
During the 2021 virtual residency recruitment, respondents advised a typical student in the middle-third of their class to apply to and interview at a higher number of IM programs which may have contributed to application inflation [3,8,9]. Their advice misaligned with data from the National Residency Matching Program [10] showing that 99% of students would match to IM when their rank list includes 12 or less programs, and that 8–10 interviews would suffice. It is possible that lack of familiarity with these recommendations and uncertainty about the impact of virtual interviews on student candidacy may have been responsible for these findings.
Moreover, advisors received more requests from students and other individuals to contact residency programs, and were asked for informal advice by an increased number of students. One possible explanation may be the ambiguity and angst among students and educators about the virtual residency recruitment process. However, these requests increased advising workload for clerkship leaders raising concerns about burnout [11,12]. Our results underscore the need for institutional support and faculty development interventions to better prepare educators for their advising roles [13].
This study had limitations. Although our respondents were broadly representative of IM clerkships’ advising practices, our findings represent a single specialty and may not be generalizable to other specialties or disciplines. Our survey focused only on advising practices for middle-third ranked students, and variability in student ranking systems among IM clerkships might explain differing responses. Additionally, responses were self-reported which could be subject to perception bias.
In summary, the transition to virtual residency recruitment during COVID-19 prompted advising practices that may have contributed to higher application numbers and increased advising workload. Future studies should explore longitudinal outcomes of virtual interviews.
Acknowledgments
The authors wish to thank the Clerkship Directors in Internal Medicine (CDIM) Survey and Scholarship Committee of the Alliance for Academic Internal Medicine (AAIM) for its assistance with developing and editing the survey questions that informed this research. The authors also thank Jordan Ortiz of AAIM for her technical and administrative support with this project.
Appendix 1.
2021 CDIM Spring Survey of Internal Medicine Clerkship and Sub-Internship Directors
Start of Block: Landing Page
2021 CDIM Spring Survey on the Virtual Residency Application Process
The purpose of this survey is to 1. understand the perspectives of internal medicine (IM) clerkship and sub-internship directors (as student advisors) about the 2020–21 residency application cycle when interviews were conducted virtually; 2. identify perceptions of how your advising might have changed; and 3. gather data essential for providing medical schools and residency programs with recommendations about the virtual application process.
The survey results will be compared to the 2021 APDIM Spring Survey of Residency Program Directors on Virtual Interviewing, to provide medical educators with more holistic information. Summary results from both surveys will be available at IM.org in the months after survey closure. Upon completing this survey, you will receive your responses by email.
The survey should take approximately 15 minutes to complete. At any point, you may exit and return without losing your data. Please use the unique survey link in your email invitation; you will be returned to where you left off. The survey software will alert you of any unanswered questions but you may skip any that you do not wish to answer.
This study (#21-AAIM-118) is exempt by Pearl IRB (U.S. DHHS OHRP #IRB00007772) under FDA 21 CFR 56.104 and 45 CFR46.104(b)(2). You are invited to participate as an IM core clerkship or sub-internship director whose institution is a CDIM member: co-/associate/assistant clerkship directors are included. Participation is voluntary; refusal to participate will not affect your/your institution’s membership. Upon survey closure, all personal and institutional identifiers will be removed by Alliance for Academic Internal Medicine Surveys staff, who manage data collection.
If you encounter technical problems, no longer are a clerkship or sub-internship director, or have questions about the survey content, contact surveys@im.org or 703–341-4540. If you feel that your participant rights have not been upheld, please contact Pearl IRB at info@pearlirb.com or 317–602-5917:
**SURVEY NAVIGATION**
1. DO NOT USE your browser’s ‘Back’ or ‘Forward’ buttons to navigate the survey. Please use the survey <PREVIOUS> and <NEXT> buttons at the bottom of each page.
2. This survey is compatible with most tablet devices but if you encounter technical problems please check that your device’s operating system is updated. Smartphones use is discouraged, due to programming that might cause unexpected survey navigation problems. Further technical support FAQ’s may be viewed here (a separate browser window/tab will open).
Q1 By clicking below, you acknowledge that your participation is voluntary.
▢ Click ‘PROCEED’ (below) to begin
Display This Question:
If Q1 ! = Click ‘PROCEED’ (below) to begin
Q2 TO CONFIRM: Do you acknowledge that your participation is voluntary?
▢ Yes: BEGIN the survey
▢ No (you will EXIT the survey and not be able to return)
▢
Skip To: End of Survey If Q2 = No (you will EXIT the survey and not be able to return)
End of Block: Landing Page
Start of Block: Section I: Student Advising About IM Residency Applications
Student Advising for Internal Medicine Residency Applications
Q3 Do you advise students at your institution about internal medicine residency applications? Answer carefully; your response will be used for subsequent questions.
o No
o Yes
Skip To: End of Block If Q3 = No
Display This Question:
If Q3 = Yes
Q4 For the periods below, enter whole numbers only.
On average, for how many of the following did you advise a typical ‘middle-third-of-the-class’ student at your institution?
| … Number of PROGRAMS to APPLY to? | … Number of INTERVIEWS to ACCEPT? | |
|---|---|---|
| PRIOR to the 2020–2021 application cycle (pre-COVID-19) | ||
| DURING the 2020–2021 application cycle (COVID-19) |
End of Block: Section I: Student Advising About IM Residency Applications
Start of Block: Section II: IM CDs and Sub-I Directors’ Perceptions of their Advising Practices
Display This:
If Q3 = Yes
Perceptions of Your Advising Practices and Residency Application Process Before and During the Pandemic
Display This Question:
If Q3 = Yes
Q5 Please answer for the pandemic application cycle (2020–2021), compared to PRIOR cycles (pre-COVID-19). How did the following change during the 2020–2021 residency application cycle?
| Decreased | About the same | Increased | Not Applicable | Unsure | |
|---|---|---|---|---|---|
| Total number of students you advised | o | o | o | o | o |
| Number of students who asked you for informal advice | o | o | o | o | o |
| Your time spent advising students | o | o | o | o | o |
| Number of students who asked you to contact a residency program or advocate on their behalf | o | o | o | o | o |
| Number of requests you received from individuals (e.g., other than students) to advocate for a student | o | o | o | o | o |
| Number of residency programs you contacted on behalf of student(s) | o | o | o | o | o |
Display This Question:
If Q3 = Yes
Q6 How prepared were you for your advisor’s role during the 2020–2021 residency application cycle when interviews were conducted virtually?
o Very unprepared
o Unprepared
o Neutral
o Prepared
o Very prepared
o Do not know/Unsure
Display This Question:
If Q3 = Yes
Q7 Answer to the best of your ability: Compared with pre-COVID-19 application cycles, how likely were your current advisees to …
| Less likely | Neither less likely nor more likely | More likely | Do not know/Unsure | |
|---|---|---|---|---|
| Receive invitations for out-of-state interviews? | o | o | o | o |
| Match at your institution’s residency program? | o | o | o | o |
| Match at regional residency programs? | o | o | o | o |
Display This Question:
If Q3 = Yes
Q8 Generally speaking, how well could your current advisees assess a residency program(s) during the virtual interview day?
o Not at all well
o Somewhat well
o Very well
o Other (please explain): _________
o Do not know/Unsure
Display This Question:
If Q3 = Yes
Q9 If onsite interviews are an option, do you think that the residency recruitment process should remain entirely virtual next year?
o No
o Yes
o Other (please explain): _________
Display This Question:
If Q3 = Yes
Q10 If unrestricted travel is allowed next year, what aspect(s) of virtual interviewing would you like to retain for the 2021–2022 residency interview season? Check all that apply.
▢ Option for virtual interviews with faculty (some or all)
▢ Option for virtual interviews with residents
▢ Option for a full virtual interview cycle
▢ Other (please explain): _________
▢ ⊗Do not know/Unsure
▢ ⊗None of the above
Display This Question:
If Q3 = Yes
Q11 What challenges did your students experience due to virtual residency interviews? Answer to the best of your ability. ____________
End of Block: Section II: IM CDs and Sub-I Directors’ Perceptions of their Advising Practices
Funding Statement
This work was supported in part by the University of Chicago Bucksbaum Institute for Clinical Excellence article.
Disclosure statement
No potential conflict of interest was reported by the author(s).
Data availability statement
The data that support the findings of this study are openly available in Harvard Dataverse (‘Replication Data for: 2021 AAIM/CDIM Survey on the Virtual Residency Application Process’) at https://doi.org/10.7910/DVN/Y05ULM, reference number UNF:6:7hhL0iDwfeOXP4HAyviCBw== [fileUNF].
Ethical approval
The study was deemed exempt by Pearl IRB (U.S. DHHS OHRP #IRB00007772).
References
- [1].Aagaard E. Medical student advising: informed individualized advice is the key. J Grad Med Educ. 2015;7(3):486–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [2].Chretien KC, Elnicki DM, Levine D, et al. What are we telling our students? A national survey of clerkship directors’ advice for students applying to internal medicine residency. J Grad Med Educ. 2015;7(3):382–387. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [3].Beshar I, Tate WJ, Bernstein D. Residency interviews in the digital era. Postgrad Med J. 2021. DOI: 10.1136/postgradmedj-2021-140897 [DOI] [PubMed] [Google Scholar]
- [4].Marbin J, Hutchinson YV, Schaeffer S. Avoiding the virtual pitfall: identifying and mitigating biases in graduate medical education videoconference interviews. Acad Med. 2021;96(8):1120–1124. [DOI] [PubMed] [Google Scholar]
- [5].Tawfik AM, Imbergamo C, Chen V, et al. Perspectives on the orthopaedic surgery residency application process during the COVID-19 pandemic. J Am Acad Orthop Surg Glob Res Rev. 2021;5(10). DOI: 10.5435/JAAOSGlobal-D-21-00091 [DOI] [PMC free article] [PubMed] [Google Scholar]
- [6].Chretien KC, Raj JM, Abraham RA, et al. AAIM recommendations for the 2020-2021 internal medicine residency application cycle in response to the COVID-19 pandemic. Am J Med. 2020;133(10):1223–1226.e6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [7].Alliance for Academic Internal Medicine . AAIM guidance for virtual residency interviewing. [cited 2022 Apr 22]. Available from: https://www.im.org/resources/ume-gme-program-resources/guidance-virtual-interviewing
- [8].Huppert LA, Santhosh L, Babik JM. Trends in US internal medicine residency and fellowship applications during the COVID-19 pandemic vs previous years. JAMA Network Open. 2021;4(4):e218199. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [9].Simmons RP, Ortiz J, Kisielewski M, et al. Virtual recruitment: experiences and perspectives of internal medicine program directors. Am J Med. 2022;135(2):258–263. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [10].National Resident Matching Program . Charting outcomes in the match: senior students of U.S. MD medical schools. 2020. [cited 2022 Apr 22]. Available from: chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://www.nrmp.org/wp-content/uploads/2021/08/Charting-Outcomes-in-the-Match-2020_MD-Senior_final.pdf
- [11].Dyrbye LN, Shanafelt TD, Thomas MR, et al. Brief observation: a national study of burnout among internal medicine clerkship directors. Am J Med. 2009;122:310–312. [DOI] [PubMed] [Google Scholar]
- [12].Ratcliffe TA, Jasti H, Ortiz J, et al. Finding meaning: internal medicine clerkship director burnout and professional fulfillment. Am J Med. 2022;135(22):654–760. [DOI] [PubMed] [Google Scholar]
- [13].Coalition for Physician Accountability . The coalition for physician accountability’s Undergraduate Medical Education-Graduate Medical Education Review Committee (UGRC): recommendations for comprehensive improvement of the UME-GME transition. [cited 2022 Jul 14]. Available from: https://physicianaccountability.org/wp-content/uploads/2021/08/UGRC-Coalition-Report-FINAL.pdf
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The data that support the findings of this study are openly available in Harvard Dataverse (‘Replication Data for: 2021 AAIM/CDIM Survey on the Virtual Residency Application Process’) at https://doi.org/10.7910/DVN/Y05ULM, reference number UNF:6:7hhL0iDwfeOXP4HAyviCBw== [fileUNF].