Residency and Fellowship Program Leaders' Perceptions of Virtual Recruitment and Interviewing

Abstract

Background

Virtual recruitment is a new and more cost-effective alternative to traditional in-person recruitment in academic medicine. However, little is known about the perceived repercussions of the switch across a variety of training settings.

Objective

To describe the perceptions of graduate medical education program leaders about virtual matching and preferred format for future recruitment within an integrated health care delivery system sponsoring residency and fellowship programs at both university- and community-based primary teaching sites.

Methods

We surveyed program leadership of 136 Accreditation Council for Graduate Medical Education programs at a single sponsoring institution in April 2021, following residency match results but before matched applicants began programs. The 40-item survey pertained to various aspects of recruitment. Select questions were assessed using a 5-point Likert scale. Descriptive statistics, Student's t test, and ordinal linear regression models were used for analysis.

Results

Out of 136 programs, 129 (94.8%) responded. Overall, preferred format for recruitment was neutral, although there was wide heterogeneity of responses. Programs felt that virtual recruitment marginally decreased their ability to describe strengths but did not affect the strength or diversity of their matched class. Community sites preferred in-person recruitment.

Conclusions

Programs did not perceive that virtual recruitment affected the strength or diversity of their 2021 matched class, although community programs were more likely to prefer in-person formats.

Introduction

Graduate medical education (GME) rapidly adopted virtual recruitment and interviewing in 2020 in response to guidance from the Association of American Medical Colleges. While more than half of program directors intend to continue with virtual recruitment in some form, many worry about impediments to nuanced assessments of applicants (such as assessment of applicant interpersonal skills and interest in a particular program) as well as the overall success of recruitment with virtual platforms.^1,2 These fears may be particularly pronounced for programs that are smaller or lack the name recognition of a university affiliation. Programs that rely on in-person visits to showcase physical aspects of their program or community may be disadvantaged as well.³ Further research from a greater breadth of training programs is needed to ascertain the trade-offs with virtual formats given their known cost saving benefits. The purpose of this study was to evaluate virtual match perceptions and preferred format for future recruitment from a breadth of diverse residency and fellowship programs.

Methods

Settings and Participants

In 2021, we surveyed residency and fellowship program directors representing 136 Accreditation Council for Graduate Medical Education–accredited academic and community programs affiliated with the University of Pittsburgh Medical Center (UPMC) at 11 separate hospitals across urban and rural areas in Pennsylvania. Surveys assessed perceived match outcomes for the 2021-2022 academic year and plans for recruitment in the upcoming 2022 match cycle.

UPMC committed to virtual recruiting and interviewing for all residency and fellowship positions beginning in the spring of 2020, consistent with recommendations from the Coalition for Physician Accountability.⁴ Specifics of the interview/recruitment process were left to individual program directors. To support programs, the central GME office provided multiple educational sessions about using online interviewing and recruiting. Microsoft Teams was our preferred platform for interviewing. Research questions were included in an annual survey of program directors and coordinators affiliated with the sponsoring institution. The survey was administered in April 2021 following National Resident Matching Program Main Match results but prior to the arrival of matched applicants in the summer of 2021. Questions were to be answered jointly by program director and coordinator and submitted as a single response. The survey included 40 items pertaining to virtual recruitment and interviewing, which were developed through consensus by a group of 7 program directors, the designated institutional official (DIO), and 3 associate DIOs. They were pilot-tested and refined for clarity. Most questions utilized a 5-point Likert scale for agreement (see online supplementary data for the survey instrument).

Analysis

We used descriptive statistics to report interview day content and structure. Student's t test was used to compare Likert data according to program type (residency vs fellowship) and logistic regression to understand predictors of preference for the upcoming interview season. A sizable portion of our fellowship programs (18%) recruit outside of a dedicated match. Recognizing these programs are unique, they were excluded from our analyses about program perceptions of virtual recruitment and interview success and logistic regression analysis. Hypothesized predictor variables included program type (residency vs fellowship), size of the incoming class, primary teaching site (specialty/referral hospital compared to a community-based hospital), number of faculty who interviewed each applicant, and whether trainees were involved in interviewing. Regression analysis was also used to examine program directors' perceived success in recruiting a diverse class, perceived ability to describe the strengths of programs through virtual formats, and association with future format preferences. All statistical analyses were performed using R, version 4.0.3 (The R Foundation for Statistical Computing). This analysis was considered exempt by the University of Pittsburgh Institutional Review Board.

Results

Out of 136 programs, 129 (94.8%) responded. Program recruitment characteristics are provided in Table 1. Programs' perceptions of virtual interviewing and recruiting are shown in Table 2. Opinions were similar when analyzed according to program type, with the exception of perceptions of faculty time investment (residency mean=2.89, SD=0.85 vs fellowship mean=2.50, SD=0.86, P=.02). Programs noted virtual interviewing and recruiting took slightly less time investment from faculty and current trainees. Virtual interviewing and recruiting were less expensive than in-person recruitment, though we did not track discrete dollar amounts. Programs felt virtual recruiting marginally decreased their ability to describe the strength of their program (mean=2.67, SD=0.78); however, programs did not notice any meaningful change in the diversity or perceived strength of their recruited class (mean=3.10, SD=0.69, and mean=3.09, SD=1.34, respectively).

Table 1.

Residency and Fellowship Programs' Recruitment Characteristics

Characteristics of Individual Programs	n (%)
Program type (N=129)
Residency	43 (33.3)
Fellowship	86 (66.7)
Participating Match (N=127)
NRMP Main Match	41 (32.3)
NRMP Specialty Match	52 (40.9)
San Francisco Match	9 (7.1)
Specialty Match	2 (1.6)
Out of Match	23 (18.1)
Primary teaching site (N=129)
Tertiary care, referral hospital	111 (86.0)
Community-based hospital	18 (14.0)
No. of positions offered annually, mean (SD)	5.61 (8.52)
No. of applicants who applied to the program, mean (SD)	406.8 (812.4)
No. of applicants interviewed, mean (SD)	50.6 (68.1)
Platform used for interviews (N=129)
Microsoft Teams	97 (75.2)
Zoom	20 (15.5)
Skype	1 (0.8)
Thalamus	4 (3.1)
Combination (other)	7 (5.4)
Did program use prerecorded materials about program? (N=124)
Yes	91 (73.4)
No	33 (26.6)
Did the program director/APD meet with candidates individually or in group setting to describe program? (N=126)
Yes	116 (92.1)
No	10 (7.9)
How were applicants interviewed? (N=126)
Individual faculty interviews	95 (75.4)
Group faculty interviews	8 (6.3)
Mixed	23 (18.3)
For programs using individual interviews, how many faculty interviewed each applicant? mean (SD)	5.4 (3.66)
Interview duration in minutes, mean (SD)	24.5 (8.1)
Current trainees participated as interviewers (N=129)	79 (61.2)

Abbreviations: NRMP, National Resident Matching Program; APD, assistant program director.

Table 2.

Program Perceptions of Virtual Interviewing and Recruiting

Questions	Overall Mean (SD)	Residency Mean (SD)	Fellowship Mean (SD)	P Value
How difficult was virtual interviewing and recruitment compared to traditional in-person? 1=very difficult; 5=much easier	3.38 (2.15)	2.98 (1.12)	3.66 (2.6)	.06
How did virtual recruitment and interviewing impact faculty time investment compared to traditional in-person recruitment and interviewing? 1=much less faculty time; 5=much more faculty time	2.66 (0.87)	2.89 (0.85)	2.50 (0.86)	.02
How did virtual recruitment and interviewing impact current trainee time investments compared to traditional in-person recruitment and interviewing? 1=much less trainee time; 5=much more trainee time	2.57 (1.23)	2.56 (1.31)	2.57 (1.19)	.96
How did virtual recruitment impact your program's expenditures on recruitment vs prior years? 1=much less expense; 5=much more expense	1.31 (0.75)	1.19 (0.55)	1.40 (0.86)	.11
Compared to prior recruiting season, how well did the virtual format allow you to describe the strength or your program? 1=a lot worse; 5=a lot better	2.67 (0.78)	2.56 (0.83)	2.75 (0.74)	.24
Did virtual interviewing and recruiting affect the strength of your recruited class? 1=much worse matched applicants; 5=much better matched applicants	3.06 (0.48)	3.05 (0.30)	3.07 (0.57)	.82
Did virtual recruitment and interviewing affect diversity in your recruited class? 1=much less diversity; 5=much more diversity	3.10 (0.69)	3.11 (0.85)	3.10 (0.54)	.91
Based on your experience, how interested are you in continuing virtual recruitment and interviewing in comparison to returning to in-person recruitment and interviewing? 1=strong preference for in-person interviewing/recruiting; 5=strong preference for virtual interviewing/recruiting	3.09 (1.34)	3.28 (1.39)	2.95 (1.30)	.22

Overall preferences for continuing virtual recruiting vs in-person recruiting in the future were neutral (mean=3.09, SD=1.34). However, this reflects heterogeneity of opinion. Fourteen percent (14 of 103) had a strong preference for in-person interviewing, 31% (26 of 103) had a slight preference for in-person interviewing, 14% (14 of 103) had no preference, 25% (26 of 103) had a slight preference for virtual interviewing, and 16% (17 of 103) had a strong preference for virtual interviewing and recruitment. This mirrors programs' varying preferences for the ideal structure of the upcoming interview season: 31.1% (32 of 103) of programs preferred to offer both formats and let the applicant choose. Approximately double the number of programs would conduct exclusively virtual recruitment (n=21, 20.4%) rather than exclusively in-person recruitment (n=10, 9.7%), and the remainder would choose a combination (n=40, 38.8%).

Using logistic regression, we analyzed predictors of in-person recruitment and found that community sites were 4 times more likely to prefer in-person vs virtual interviewing and recruiting (P=.013, 95% CI 1.34-12.39). There were no other statistically significant predictors of preference.

Discussion

The dramatic shift to virtual recruiting and interviewing was unexpected and likely to cause enduring changes in how programs recruit physicians. Few studies have analyzed the impact of fully virtual recruitment across a spectrum of GME programs. Our results demonstrate that while program preference on recruitment methods varied substantially, overall, virtual methods were not perceived to be detrimental to recruitment and interviewing outcomes.

Our data suggest that community and university-affiliated programs have different recruitment needs. Community sites are often desirable for the close trainee and faculty connections and practice settings that closely mirror trainees' ultimate desired setting. However, applicants may have limited exposure to community sites during their medical training, as many medical schools are university-based in urban or suburban settings. Consequently, interview experiences and recruitment events may play a more prominent role in informing and matching applicants into community training programs.

Diversity and inclusion recruitment efforts are not perceived to be adversely affected by the virtual format in our population. Though there has been concern about the virtual format amplifying and potentially introducing new sources of bias,⁵ programs' self-reported measures of matched class diversity in 2021 were comparable to prior years. Our sponsoring institution tracks the percentage of matched trainees who meet Association of American Medical Colleges underrepresented in medicine definitions,⁶ and according to internal unpublished data between 2018 and 2021, this number ranged from 9% to 11%, with 11% of matched applicants meeting criteria from the 2020 recruitment season. Though more research is needed to confirm whether an impact exists on diversity and inclusion, virtual recruitment remains an attractive option to decrease the match process's financial burden on applicants.

Limitations of this study include a single center design and that fellowship programs may be overrepresented in this sample. However, this center represents both academic and community programs as well as a diverse spectrum of residency and fellowship programs, which increases generalizability of our findings.

Conclusions

Our data show that while virtual interviewing is less expensive and may use slightly less faculty time, it is less enthusiastically embraced by community-based programs.

Funding Statement

Funding: The authors report no external funding source for this study.