Virtual Residency Interviews: A Survey of Anesthesiology Program Director Perspectives Amidst the COVID-19 Pandemic

Abstract

Background:

The COVID-19 pandemic caused a rapid pivot from in-person to virtual residency interviews across the United States. We present a survey we conducted about the attitudes and opinions of anesthesiology program directors with regard to the 2021 virtual interview process.

Methods:

This was a 13-question online survey disseminated to 142 anesthesiology residency program directors in February 2021, asking them to compare the most recent interview cycle to their experience with prior cycles.

Results:

There were 46 (37%) respondents. Generally, respondents saw an increase in applicants and reported perceiving worse interpersonal relationships with applicants, significantly so in programs with small resident classes.

Conclusions:

Past research has focused on the benefits of virtual interviews for the applicant, but these should be evaluated in tandem with increased difficulties for the interviewers.

Introduction

The travel restrictions and limitations to in-person gathering caused by the COVID-19 pandemic disrupted the traditional residency application process for the 2020–2021 application and match process. The National Resident Matching Program (NRMP) adopted a recommendation from the Academy of American Medical Colleges (AAMC) to implement virtual interviews for all residency programs.¹

It is often emphasized to medical students that more interviews correlate with a higher chance of matching into a residency program. This is borne out in data from the NRMP, where successful anesthesiology applicants in 2019–2020 ranked an average of 15.1 programs, compared to 5.1 among unsuccessful applicants.² Among successful anesthesiology matches in 2018–2019, graduates from US allopathic medical schools submitted a median of 35 applications, and graduates from US osteopathic medical schools and international medical graduates submitted a median of 58.³ Data gathered prior to the start of the 2020–2021 cycle suggested that applicants were both increasing their number of applications, as well as their number of anticipated interviews because of the virtual format.⁴ With the logistical barriers involved with in-person interviews removed, there was concern that this would lead to a small pool of top-tier applicants occupying the majority of interview slots, with the consequence of applicants not matching and programs not filling.

In this brief report, we present the results of a survey sent to anesthesiology program directors, looking at their early experience with a fully virtual interview cycle. We hypothesized that the majority of Anesthesiology program directors would find the sudden pivot to a mostly virtual interview format difficult to accommodate, which would manifest as increased dissatisfaction with the 2020–21 Match Cycle.

Materials and Methods

A 13-question survey (Appendix) was conceived by a senior anesthesiologist with experience in graduate medical education, and then piloted within a focus group consisting of 3 members: our residency program director and 2 administrators involved in the recruitment process. The survey was sent to 142 anesthesiology residency program directors who participated in the 2018–2019 interview cycle across the United States using REDCap (Vanderbilt University, Nashville, Tennessee), a secure, web-based platform designed to support data capture for research studies. The survey was sent in late February 2021, which was approximately the end of the 2020–2021 residency interview cycle. The survey was timed to capture the immediate perception of and reaction to the new interview format before the results of the interview either reassured or disappointed program directors. In order to maximize response, the survey was sent twice more, at weekly intervals. This study was deemed exempt by the George Washington University Institutional Review Board (NCR2023203).

Apart from 2 demographic questions (location and size of resident intake), there were 9 Likert-scale questions and 2 open-ended questions asking respondents to compare their previous experiences to the 2019–2020 interview cycle. Four questions used a 3-point Likert-scale (decreased, stayed the same, increased), and the remaining a 5-point Likert-scale (much worse to much better). The substance of the 3 questions related to the pattern of application sources graduates from US allopathic medical schools, graduates from US osteopathic medical schools, international medical graduates, and the 5 questions explored the perceptions of virtual interview process in comparison to the 2020 interview cycle.

For the 5-point Likert-scale questions, outcomes were dichotomized (before data collection) to compare worse/much worse to no change/better/much better. Further post-hoc analysis of responses was performed by dividing respondents’ programs by size of resident intake, as larger programs may have been better equipped to deal with the infrastructure changes required for a fully virtual interview cycle. Data analysis was performed using the χ² test in STATA v15 (StataCorp, College Station, Texas). Analysis of open-ended questions was conducted and themes extracted by an independent coder method. Themes were determined by consensus.

Results

The overall response rate was 37% (46/142). The programs represented were geographically diverse. There was an average residency intake size of 13 (SD ± 6.13). There were 25 programs with less than 13 positions, and 21 programs with 13 or more positions (Table 1).

Table 1.

Demographics

Parameter	Value
Total number of respondents, n/N (%)	46/142 (37)
In which region is your hospital located? n (%)
Northeast	18 (39)
South	9 (28)
Midwest	13 (20)
West	6 (13)
Average size of CA-1 class, mean ± SD	13 ± 6.13
Number of respondents with CA-1 < 13, n (%)	25 (54)
Number of respondents with CA-1 ≥ 13, n (%)	21 (46)

Abbreviation: CA-1, first-year clinical anesthesia resident.

With regard to number of applications (Table 2), 89% (41/46) of respondents reported an increase, mostly mediated by applications from US medical students. This effect did not vary significantly with size of residency intake (P = .2). With regard to length of interviews, 9% (4/46) held longer interviews, 30% (14/46) shorter, and the remaining 61% (28/46) reported that interview times remained the same when compared to the 2020 interview cycle.

Table 2.

Perception of Quantitative Parameters of Interview Cycle, N (%)

	Increased	Stayed the Same	Decreased
Number of applicants		5 (11)
Number of IMG applicants	19 (42)	25 (54)	2 (4)
Number of DO applicants	27 (59)	19 (41)
Length of interview sessions per applicant	4 (9)	27 (59)	15 (32)

Abbreviations: DO, Graduate from US Osteopathic School; IMG, International Medical Graduate.

The responses to the qualitative survey statements are presented in Table 3. Of note, 39% (18/46) of respondents indicated that their perception of applicant commitment levels was somewhat worse or much worse. A larger proportion of small-sized programs (54%) reported a significantly worse perception of this in comparison to larger programs (24%; P = .04). In general, more small-sized programs perceived a worse experience with virtual interviews, but there was no significant difference regardless of class size in the rest of the questions asked.

Table 3.

Proportion of Respondents Who Answered Somewhat Worse or Much Worse to the Following Qualitative Survey Statements, Presented as Total and by Resident Intake Size

Statement	All Respondents (n, %), n = 46	Small (<13) (n, %), n = 25	Large (≥13) (n, %), n = 21	P Valuea
Applicants have sufficient information about your program	9 (20)	7 (28)	2 (10)	.13
Interpersonal relationships with applicants	30 (65)	16 (67)	14 (67)	1.00
Applicant commitment to your program	18 (39)	13 (54)	5 (24)	.04
Your program’s ability to gauge the candidacy of an applicant	21 (46)	13 (54)	8 (38)	.28
The ability of applicants to properly gauge the merit of your program	22 (48)	13 (54)	9 (43)	.45

^aBoldface values indicate statistical significance.

The most common themes among challenges experienced by the respondents were “difficulty in gauging candidate interest” (16 responses), “inability to observe candidates interacting with people other than interviewers” (9 responses), and “technological difficulties” (8 responses).

Discussion

This was a survey of anesthesiology program directors exploring their early attitudes toward virtual interviews, having just completed a cycle of entirely virtual recruitment without precedent. The majority of respondents also found making personal connections with interviewees more difficult, and this was especially true of programs with a small residency intake. Difficulty gauging candidate interest was also a common theme in the open-ended question section.

The sudden diminution of human interaction cannot be overlooked: consecutive anesthesiology program director surveys produced by the NRMP rank interpersonal skills and interaction with faculty as among the top 2 factors for ranking applicants.⁵ An AAMC survey in 2015⁶ reported that program directors rely on the interview process to gauge an applicant’s professionalism, integrity, teamwork, and reliability, all characteristics difficult to gauge from a paper application and complex to define in a purely virtual setting.

Virtual interviews offer both programs and applicants financial savings and reduction in missed clinical workdays, and for applicants it allows for the possibility to interview more broadly without being bound by travel expenses and geography.⁷ Preliminary disadvantages of the virtual format included technical difficulties, limited interactions with current residents, inability to gauge program culture, potential for increased interview cancellations, and inability to see the hospital and/or city. Of note, a larger proportion of programs with small residency intakes we surveyed perceived worse experiences with virtual interviews. This may have been because of a lack of resources (eg, fewer personnel to facilitate interviews, create online content, provide administrative support) or a proportionally larger number of applicants per available position, compared to larger programs.

Previous research into hybrid models incorporating virtual interviews done prior to the pandemic suggest that, with adequate and intentional preparation, virtual interviews do not affect the rate of admission into anesthesiology residency.^8,9 Moreover, with adequate preparation, faculty interviewers in another study expressed satisfaction with virtual interviews and highlighted the efficiency and convenience of the format, although the majority still preferred live interviews.¹⁰

This paper was limited by the relatively small survey response rate, which may have suggested selection bias. Further, the survey was filled out toward the end of the recruitment cycle prior to its conclusion, and some responses may have therefore been premature.

Conclusion

Our survey identified that interpersonal relationships remain an issue with the virtual interview format, likely exacerbated by the increase in applicant numbers. Future research should focus on how that gap can be bridged in the virtual space.

Appendix

Appendix. Survey Questions

General

1. In which region of the country is your program located?

2. How many residents do you have in your CA-1 class?

12. What adjustments have you made to the new virtual setting with your program?

13. What were some new challenges you faced as a program director due to interviews being virtual?