Abstract
This survey study describes the perceived implications of virtual-only recruitment and the preferred application process for residents and fellows.
Residency and fellowship recruitment converted from nearly exclusively in-person interviews to virtual interviews after the onset of the COVID-19 pandemic. The virtual recruitment process has advantages, including convenience and decreased costs for both applicants and programs1,2; the reduced cost likely makes the process more equitable for applicants with less resources. At the same time, virtual recruitment may contribute to application inflation1,2,3,4 and reduce applicants’ ability to assess the fit of a training program and location.1,2
The Association of American Medical Colleges (AAMC) and other organizations have recommended virtual-only recruitment for the 2022 to 2023 residency and fellowship recruitment season regardless of pandemic-related travel restrictions.5,6 Little is known about applicants’ perspectives on a virtual-only recruitment season. In this survey study, we sought to assess internal medicine residents’ opinions regarding the first virtual-only recruitment cycle in the US.
Methods
A confidential, optional survey (eAppendix in Supplement 1) was administered to all residents immediately after completing the American College of Physicians Internal Medicine In-Training Examination (IM-ITE) in the fall of 2021. The survey included only internal medicine residents in US programs, including Puerto Rico. The Duke University School of Medicine Institutional Review Board deemed this study exempt from review because it posed minimal risk. Participants checked a box indicating consent for their responses to be shared as deidentified results. We followed the AAPOR reporting guideline.
The survey asked residents for their opinions about how virtual-only recruitment affected the number of programs medical students applied to and the number of interviews medical students accepted, and for their preferences for future recruitment seasons (eAppendix in Supplement 1 provides questions and response options). We separated responses by postgraduate year (PGY) to allow comparison between PGY-1 residents who completed virtual-only recruitment and PGY-2 and PGY-3 residents who had experienced in-person recruitment. We also compared responses by medical school type (US medical school graduate [USMG] vs international medical graduate [IMG]) and by sex.
We used Pearson χ2 tests of independence to analyze group comparisons. Two-sided P < .05 indicated statistical significance. Data were analyzed in Q 5.14.2.0 (Q Research Software).
Results
Of the 28 892 residents who took the IM-ITE, 22 310 (12 336 males [55.3%], 9941 females [44.6%]) completed the survey (77.2% response rate) (Table 1 and Table 2). Eighty percent of respondents believed that applicants applied to more or many more programs in virtual vs traditional recruitment. More PGY-1 residents than PGY-2 and PGY-3 residents believed that applicants applied to many more programs in the virtual format (51% vs 43% and 41%; P < .001). Three-quarters of residents perceived that applicants accepted more or many more interviews in virtual recruitment, with more PGY-1 residents than PGY-2 and PGY-3 residents selecting many more interviews (42% vs 33% and 31%; P < .001) (Table 1). The USMGs were more likely than IMGs to respond that applicants applied to more or many more programs (85% vs 72%; P < .001) and accepted more or many more interviews (78% vs 65%; P < .001).
Table 1. Residents’ Opinions About the Implications of Virtual Recruitment for Program Applications and Interviewsa.
| Survey response option | Residents, No. (%) | |||||
|---|---|---|---|---|---|---|
| All (n = 22 310) | PGY-1 (n = 7593) | PGY-2 (n = 7738) | PGY-3 (n = 6979) | P value for PGY-1 vs PGY-2 | P value for PGY-1 vs PGY-3 | |
| No. of programs medical students applied to | ||||||
| Many more programs | 10 082 (45) | 3871 (51) | 3317 (43) | 2894 (41) | <.001 | <.001 |
| More programs | 7865 (35) | 2398 (32) | 2865 (37) | 2602 (37) | <.001 | <.001 |
| The same number | 3017 (14) | 912 (12) | 1073 (14) | 1032 (15) | <.001 | <.001 |
| Fewer programs | 909 (4) | 264 (3) | 341 (4) | 304 (4) | .003 | .006 |
| Many fewer programs | 437 (2) | 148 (2) | 142 (2) | 147 (2) | .60 | .50 |
| No. of interviews medical students accepted | ||||||
| Many more interviews | 7924 (36) | 3220 (42) | 2575 (33) | 2129 (31) | <.001 | <.001 |
| More interviews | 8420 (38) | 2486 (33) | 3062 (40) | 2872 (41) | <.001 | <.001 |
| The same number | 3635 (16) | 967 (13) | 1383 (18) | 1285 (18) | <.001 | <.001 |
| Fewer interviews | 1670 (7) | 634 (8) | 549 (7) | 487 (7) | .004 | .002 |
| Many fewer interviews | 661 (3) | 286 (4) | 169 (2) | 206 (3) | <.001 | .007 |
Abbreviation: PGY, postgraduate year.
Residents were asked “Compared to traditional in-person recruitment, how do you think virtual recruitment affected the number of residency programs [that] medical students applied to?” and “Compared to traditional in-person recruitment, how do you think virtual recruitment affected the number of interviews that medical students accepted?” Analysis included residents from US residency programs, including in Puerto Rico, who allowed their responses to be analyzed.
Table 2. Resident Preferences for Future Recruitment Seasonsa.
| Survey response option | Residents, No. (%) | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| All (n = 22 310) | PGY level | Medical school type | Sex | ||||||
| PGY-1 (n = 7593) | PGY-2 (n = 7738)b | PGY-3 (n = 6979)b | USMG (n = 14 776) | IMG (n = 7534) | Male (n = 12 336) | Female (n = 9941) | Other (n = 32)c | ||
| Recruitment should be only in person (if safe due to COVID-19) | 7372 (33) | 1752 (23) | 3135 (41) | 2485 (36) | 5098 (35) | 2274 (30)d | 4198 (34) | 3166 (32)e | 8 (25) |
| Virtual option should be offered even if in-person is the norm again | 5790 (26) | 2116 (28) | 1945 (25) | 1729 (25) | 3674 (25) | 2116 (28)d | 3117 (25) | 2662 (27)f | 10 (31) |
| Recruitment should remain virtual with option for subsequent on-site visit | 4456 (20) | 2320 (31) | 989 (13) | 1147 (16) | 3104 (21) | 1352 (18)d | 2295 (19) | 2154 (22)e | 7 (22) |
| Recruitment should be only virtual | 592 (3) | 291 (4) | 128 (2) | 173 (2) | 376 (3) | 216 (3) | 379 (3) | 211 (2)f | 2 (6) |
| Unsure | 4100 (18) | 1114 (15) | 1541 (20) | 1445 (21) | 2524 (17) | 1576 (21)d | 2347 (19) | 1748 (18)e | 5 (16) |
Abbreviations: IMG, international medical graduate; PGY, postgraduate year; USMG, US medical school graduate.
Residents were asked “For future residency recruitment seasons, which of the following best reflects your opinion?” Analysis included residents from US residency programs, including in Puerto Rico, who allowed their responses to be analyzed.
P < .001 for comparisons with PGY-1 residents.
Other was a checkbox option for sex, along with Male and Female options. No descriptors were provided. One resident did not indicate sex.
P < .001 for comparison between USMGs and IMGs.
P < .001 for comparison between females and males.
P = .01 for comparison between females and males.
Residents’ opinions about the best future recruitment approach varied, but only 3% of all residents and 4% of PGY-1 residents selected the virtual-only recruitment option. The most commonly selected response among PGY-1 residents was virtual interviewing with subsequent on-site visit option (31%), while in-person only was commonly selected by PGY-2 and PGY-3 residents (41% and 36%) (Table 2). Results were not markedly different between USMGs and IMGs or between males and females.
Discussion
Only 3% of respondents preferred virtual-only recruitment, the AAMC recommendation for the current season. These findings suggest that a virtual-only recruitment process compared with in-person recruitment may be a factor in decisions to apply to and interview with more programs. A study limitation was the survey lacked data on race and ethnicity, geography, or application or interview numbers. The findings support the development of new recruitment approaches that better balance equity, costs, and convenience; control application inflation; and allow for opportunities to visit programs.
eAppendix. Survey Questions Included
Data Sharing Statement
References
- 1.Labiner HE, Anderson CE, Maloney Patel N. Virtual recruitment in surgical residency programs. Curr Surg Rep. 2021;9(11):25. doi: 10.1007/s40137-021-00302-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Simmons RP, Ortiz J, Kisielewski M, Zaas A, Finn KM. Virtual recruitment: experiences and perspectives of internal medicine program directors. Am J Med. 2022;135(2):258-263.e1. doi: 10.1016/j.amjmed.2021.10.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Association of American Medical Colleges. Electronic Residency Application Service. Accessed December 27, 2022. https://www.aamc.org/media/39376/download
- 4.National Resident Matching Program . Virtual experience and holistic review in the transition to residency: an examination of the 2021. and 2022 main residency matches. Accessed October 31, 2022. https://www.nrmp.org/wp-content/uploads/2022/07/2022-Virtual-Exper-Research-Brief_Final.pdf
- 5.Association of American Medical Colleges. AAMC interview guidance for the 2022-2023 residency cycle. Accessed October 31, 2022. https://www.aamc.org/about-us/mission-areas/medical-education/aamc-interview-guidance-2022-2023-residency-cycle
- 6.Alliance for Academic Internal Medicine . AAIM guidance for virtual residency interviewing. Accessed October 31, 2022. https://www.im.org/resources/ume-gme-program-resources/guidance-virtual-interviewing
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eAppendix. Survey Questions Included
Data Sharing Statement