Zooming In Versus Flying Out: Virtual Residency Interviews in the Era of COVID‐19

The 2019 coronavirus disease (COVID‐19) pandemic has prompted graduate medical education (GME) programs to revisit the concept of virtual interviews for applicants given uncertainty over the duration of social distancing measures and travel restrictions. A “virtual interview” refers to the process of conducting interactions over a video‐conferencing platform instead of the traditional model of traveling to an on‐site location with face‐to‐face interactions.

The interview for GME training programs has multiple purposes. While the evidence is mixed regarding its predictive value for training outcomes, the interview interaction is weighted heavily by program directors in their decisions regarding applicant selection ¹ , ² The interview is felt to provide insights into applicants’ interpersonal communication skills and professionalism, which are otherwise poorly represented in application materials. ³ From the applicants’ and programs’ perspectives, the interviews and surrounding recruitment events provide critical information for their decisions including highly valued casual interactions. ⁴ , ⁵

Given the emphasis placed on the interview and surrounding interactions by the primary stakeholders of the recruitment process, it is important to understand the strengths and limitations of any transition to a virtual platform. The current model of in‐person interviews already has a number of challenges beyond the current pandemic, including high costs as well as significant time commitments and scheduling limitations. ⁶ , ⁷ , ⁸ , ⁹ In light of recent recommendations by the Association of American Medical Colleges (AAMC) to transition to virtual interviewing during the COVID‐19 pandemic and potentially beyond, this paper summarizes the existing knowledge base regarding virtual interviews and proposes potential best practices for programs. ¹⁰

CURRENT EVIDENCE AROUND VIRTUAL INTERVIEWS

There are several published examples of virtual interviews in GME. Applicants to a single urology program completed a crossover study with both video and in‐person interviews. The authors identified benefits to the video interview in terms of time and cost. Participants overall reported reduced ability to represent themselves in the virtual interaction; however, they favored continuing it as an adjunct to in‐person interviews. ¹¹

A 2014 study of gastroenterology fellowship applicants had four in‐person interviews and a single video interview with a remote faculty member. Eighty‐one percent of applicants agreed that their video interview met or exceeded expectations. Twenty‐five percent responded that their video interview was at least equivalent to their in‐person interview, and 87% agreed that video interviews should continue. From these findings, the authors concluded that Web‐based video conferencing has the potential to be an effective screening tool or an acceptable alternative to in‐person interviews. ¹²

An observational study at an anesthesiology residency program allowed applicants to complete either face‐to‐face interviews (75%) or video interviews (25%). The study noted a higher proportion of nonlocal applicants in the video pool who were also more likely to complete a later campus visit. The follow‐up survey showed selection of the video format was driven by geographic and travel concerns as well as conflicts with interview dates. Only 4.2% of applicants who selected face‐to‐face interviews worried that a video interview would negatively impact their chances of matching. Similar proportions from both groups were in the top half of the rank list and in their matched class. Overall, the video participants felt the virtual interview met or exceeded their expectations. ¹³

Virtual interviewing has been more extensively used in business than medicine. These fields may provide transferrable insights and lessons for conducting residency virtual interviews. Hospitality managers favored video interviewing for convenience and cost savings, but found it disadvantageous due to the lack of in‐person contact and potential technological malfunctions. ¹⁴ Another older study found that the number of job offers positively correlated with face‐to‐face interviews. ¹⁵ However, a recent study of medical school admissions demonstrated equivalent acceptance rates between the two interviewing methods. ¹⁶

The research world also provides insights into differences in interviewer–interviewee dynamics with the virtual format. Krouwel et al. found similar interview content but longer duration of in‐person interactions. ¹⁷ The interviewer also spoke substantially more during in‐person interviews. Overall, they slightly favored in‐person interviews for qualitative research studies due to the increased richness of interview content. Another clinical research study suggested that interviewees of younger age and higher education levels may prefer video interviews. ¹⁸

Although distinct from a synchronous virtual interview and no longer utilized within emergency medicine residency applications, the standardized video interview (SVI) developed by the AAMC provides valuable processes to inform virtual interviews in GME. The SVI was intended to provide information about interpersonal communication skills and professionalism to allow for screening of applicants. Key elements include attention to unconscious bias with the use of trained raters. ¹⁹ , ²⁰ It was field tested for 2 years prior to its cancellation due to lack of prognostic value. ²¹ Our intent is not to advocate for the return of the SVI, but rather to acknowledge insights gained.

ADVANTAGES AND DISADVANTAGES OF VIRTUAL INTERVIEWS

Applicants participating in virtual interviews may accrue advantages such as time, finance, and flexibility; however, disadvantages are also present such as the inability to tour the campus, loss of meaningful casual interactions, and the introduction of potential biases. The key advantages and disadvantages of virtual interviews are described in Table 1.

Table 1.

Advantages and Disadvantages of Virtual Interviews to Applicants and Programs

Advantages

Disadvantages

Allows GME* interview process to continue during times of social disruption. Decreased cost for applicants and programs due to absence of travel and hosting expenses. Applicants may be able to attend more interviews due to decreased cost. Less potential for disruption of interview days due to travel problems. Potentially minimizes time away from medical student rotations. Increased flexibility for interview times and dates. May benefit applicants on the wait lists due to ease of scheduling. Programs may be able to use virtual interviews as a screening tool. Allows alumni of the program practicing in a different geographic location to interview applicants. Virtual interviews could be augmented with an in‐person second look.

Loss of interactions which may provide insight about interpersonal skills and professionalism. Loss of opportunity to directly observe program culture, form relationships, and visit location. Technical difficulties can impact the interview interaction and influence impressions. Computer literacy with platforms can vary. Also, residency programs may utilize different platforms. Computer proficiency with platforms can vary. Scheduling challenges may occur with different time zones. Applicants may still feel obligated for an on‐site visit. Virtual strategies could amplify disparities among programs. Potential for misrepresentation and misunderstanding of the training environment. Introduction of unanticipated sources of bias, which may amplify disparities.

^*GME = graduate medical education.

All interviewers, regardless of mode of interaction, require training in unconscious bias. This may have particular importance with virtual interviews as the video format may introduce novel information about the applicant from glimpses of their environment including religious symbols, evidence of family structure, or the physical state of their environment, which may reflect socioeconomic status. In addition, novel biases may occur such as bias against the applicants who appear to struggle with technology or prefer one format over the other.

BEST PRACTICES FOR IMPLEMENTING VIRTUAL INTERVIEWS

Virtual interviewing offers substantial advantages for both applicants and programs in the current environment. We propose the following best practices for GME programs planning to implement virtual interviews. These recommendations fall within broad categories including use of technology, interview format, and social interactions, which are listed in Table 2.

Table 2.

Best Practices for Implementing Virtual Interviews

Use of technology Interviewers should be trained in advance with the platform and troubleshooting issues that may arise. Both interviewers and applicants should have access to reference materials for operating the software. Virtual interviews should be designed to maximize interviewee and interviewer confidentiality. This should include disabling of recording functions; providing individualized, nonshareable links; and using passwords or a virtual waiting room that requires approval by the interviewer to join the meeting. Programs should prepare backup plans in the event of technology failure, such as a telephone call. Programs utilizing virtual interviews should perform ongoing quality improvement and adjustments to the process through real‐time feedback from all participants including interviewers, applicants, and administrative personnel. Consider use of neutral backgrounds including professional virtual backgrounds.

Interview format and schedule Programs should provide interviewers and interviewees with a specific itinerary for the interaction including time zones, password‐protected links, and a contact person who is facile with troubleshooting. If a choice of in‐person or virtual interview is allowed, programs should implement practices to decrease resultant biases. Programs should communicate clearly with applicants regarding expectations for scheduling and an explicit delineation of required and optional activities. While the environment of the interview has changed, legal and regulatory considerations remain in place. Remain mindful of “illegal” questions and National Resident Matching Program (NRMP) regulations. 22 Programs may want to consider hybrid models of virtual interviews and in‐person interactions. If these models are used, applicants should be clearly informed of plans and expectations.

Social interactions Interviewers should be trained in facilitating video interviewing and the ways that it may differ from in‐person interviews. This may include attention to body language on a virtual interface, awareness of vocal tone over electronic media, and appropriate pacing of the interview and questions. Interviewers should also be trained in recognizing personal biases including those which may be introduced with video observations. Programs should provide honest resources for applicants that attempt to replicate critical features of the in‐person interview day. These may include a program overview, facilities tour, or less structured interactions with trainees or other members of the program. Programs may consider archiving these resources to allow applicants a virtual “second look” at a later date.

CONCLUSIONS

Although virtual interviews may not completely replace in‐person interactions for GME interviews, they may offer distinct advantages including lower cost, reduced travel, and scheduling flexibility. The existing literature demonstrates that even prior to the COVID‐19 pandemic, virtual interview strategies have shown promise. However, virtual interviews are not without potential pitfalls. Additional research needs to rigorously assess the impact of virtual interviews on all stakeholders and the GME selection process. We proposed some initial best practices for programs as they seek to trial this approach. However, truly effective and fair incorporation of virtual interviewing will require the NRMP to explicitly provide guidelines and adapt its existing regulations around “second looks” and postinterview communication. ²² The uncertain future of social distancing restrictions and financial consequences of the pandemic will force training programs to adapt in the short term. Even when society returns to “normal,” there will still be a role for virtual interview strategies. Virtual interviews can offer a number of advantages to residency programs and applicants, either in isolation or as part of a hybrid model.

AEM Education and Training 2020;4:443–446