Virtual Interactions and the 2020-2021 Residency Application Cycle in General Surgery: A Look Ahead

Abstract

Introduction

General surgery residency training programs adapted to the COVID-19 pandemic by going online instead of in-person, through virtual interviews, social media engagement, and virtual open houses. The impact of these virtual interactions is unknown. We sought to understand their effectiveness as per residency program directors and assistant program directors.

Materials and methods

An institutional review board approval was obtained to conduct this anonymous survey. A Qualtrics XM survey containing multiple-choice and short-answer questions was distributed to 590 residency program and assistant program directors through the Association of Program Directors in Surgery (APDS) listserv on July 6, July 13, and July 20.

Results

We observed a response rate of approximately 11% across the 590 surgeons contacted. Nearly all (90%) respondents offered virtual preinterview interactions, primarily virtual open houses, virtual facility tours, and virtual question and answer (Q&A) sessions with residents and faculty; 48% of respondents were unsure of the utility of virtual interactions and the majority (54%) felt that virtual interaction limits a program's ability to evaluate applicants. Virtual Q&As were ranked to be the most effective interaction (7.6/10); 80% of respondents felt that visiting rotations were “somewhat important” to “very important,” the two highest options available. In addition, 74% felt that applicants missed out on fully experiencing the program by forgoing these rotations. Most respondents (78%) noted that evaluation of applicants’ preinterview did not change as a result of virtual interactions. Nearly half (48%) of the respondents offered more interview days due to the virtual format. A fifth (21%) of respondents stated that virtual interactions resulted in a change in the rank position of an applicant. Respondents ranked Twitter and Instagram higher in applicant engagement than Facebook. Factors that impacted interview or rank order list the most were late/absent step two CK scores (33%) and a lack of away rotations (31%), both being limitations largely due to the pandemic. With respect to future application cycles, most (71%) raised concerns regarding disparities between applicants applying in-person and virtually if both or either are offered.

Conclusions

Our study suggests that program directors and associate program directors have reservations about the use of virtual interactions with applicants. Interestingly, these data suggest that visiting subinternships are useful for programs in evaluating applicants. This may encourage students to pursue rotations at other institutions at the expense of already-limited resources. It remains unclear whether virtual interactions will be used in the future, but respondents largely agreed that the virtual means of interacting with and disseminating information to the applicants of the 2020-2021 general surgery Match were a success.

Introduction

In an unprecedented effort to limit transmission in the context of the COVID-19 pandemic, United States medical students were largely barred from partaking in many of their usual hospital-based clinical duties for a substantial portion of 2020 and 2021.¹ These measures were necessary for the mutual safety of students and patients alike, although their impact on the 2021 Association of American Medical Colleges residency applicant pool and programs respective responses have been equally unprecedented. Efforts were taken to bridge the gap for students who were unable to participate in standard in-person clinical clerkships, namely simulated shelf-like questions and virtual group learning activities.^{2 , 3} However, the limited availability of extracurricular academic interaction, namely in-person interviews and rotations at other institutions, were largely unaccounted for.

Visiting subinternships were brought to a halt, limiting students’ ability to assess and be assessed by programs outside of their home institutions.⁴ Rotations at outside institutions are of particular significance among more competitive institutions and historically competitive specialties.^{5 , 6} Students reported significantly higher levels of stress surrounding the virus and fear of transmission and these changes in the academic environment are likely to have compounded that stress.⁷

In response to the COVID-19 pandemic, residency programs appeared to enhance their online presences through the creation of virtual opportunities and social media accounts.8, 9, 10 Some specialties saw the introduction of a virtual subinternship, although these virtual subinternship opportunities represent a small fraction of applicants compared to prior years.^{11 , 12} A recent study estimated the prevalence of away rotators to be 59% of all fourth-year medical students.¹³ The same study reported 36% of applicants matching at an institution where they had rotated, either their home institution or one at which a visiting rotation was performed. These unique experiences offer students a useful insight into programs that they would likely not otherwise have access to.

Given the absence of in-person interviews and the near-elimination of subinternships, residency applicants in 2020 were left without two primary mainstays of interacting with programs other than their home institution. This constraint hindered students’ ability to assess institutions where they may be interested in matching and assess other programs in relation to programs known to them. Program directors are similarly limited and their programs face an equally difficult task of adequately evaluating individual applicants and managing ever-growing applicant pools.¹⁴ The effectiveness of virtual interactions remains unclear. Significant opportunities exist to investigate the quality of virtual interactions between residency programs and prospective applicants.¹⁵ This article seeks to enumerate both how programs accommodated COVID-19 restrictions and their retrospective perception of the successes and failures of those accommodations. These perspectives may offer a unique perspective on general surgery residency program directors’ impressions of the 2020 Match and a possible insight into what elements of virtual interaction may remain in the application years to come, independent of the progress and hopeful resolution of the ongoing global pandemic.

Methods

Institutional board exemption (IRB #300006990) was obtained to conduct this anonymous survey. A Qualtrics XM survey containing multiple-choice and short-answer questions was distributed to 590 residency program and assistant program directors through the Association of Program Directors in Surgery (APDS) over the course of three consecutive weeks. No identifying data were collected. The survey was closed 1 week following the last distribution email. A full list containing the survey's questions is provided within the supplemental document (Survey Questions).

Results

Approximately 11% of the 590 APDS members contacted responded to the survey with generally even distribution across regions of the United States (Table 1 ). Among respondents, almost all (90%) offered virtual preinterview interactions, primarily open house and question and answers (Q&As), with residents and faculty. Of preinterview offerings, the most highly rated were Q&A sessions with residents and faculty, 7.6/10 and 7.1/10, respectively (Fig. ). Of our respondents, 43% thought these preinterview virtual interactions were a success while a slightly greater portion (48%) was unsure of their utility. Of those who thought preinterview virtual events were not a success (9%), the major reason was difficulty assessing applicants over a virtual platform. Most general surgery residency program directors and assistant program directors (78%) concluded that their evaluations of students did not change because of preinterview virtual interactions. However, those whose evaluations of students did change due to preinterview virtual interactions reported that virtual opportunities allowed the opportunity to gauge students’ interest and ability to interact with others.

Table 1.

Respondent demographics.

Respondent demographics	%	Count
Region
Northeastern (Connecticut, Delaware, Maine, Maryland, Massachusetts, New Hampshire, New Jersey, New York, Pennsylvania, Rhode Island, Vermont, Washington, D.C.)	38.33%	23
Southern (Alabama, Arkansas, Florida, Georgia, Kentucky, Louisiana, Mississippi, North Carolina, Oklahoma, Puerto Rico, South Carolina, Tennessee, Texas, Virginia, West Virginia)	26.67%	16
Central (Illinois, Indiana, Iowa, Kansas, Michigan, Minnesota, Missouri, Nebraska, North Dakota, Ohio, South Dakota, Wisconsin)	20.00%	12
Western (Alaska, Arizona, California, Colorado, Hawaii, Idaho, Montana, Nevada, New Mexico, Oregon, Utah, Washington, Wyoming)	15.00%	9
Total	100%	60
Role
Program director	83.33%	50
Associate program director	16.67%	10
Total	100%	60
Response rate
Completed at least some of the survey	10.5%	62
Received but did not complete survey	89.5%	528
Total	100%	590

Fig.

Histogram with standard deviation of responses to Q25: “Please rate the effectiveness of each preinterview virtual opportunity you offered. (0, Not influential at all; 10, The most beneficial metric available to you this application cycle).”

Of respondents, 69% said their virtual events were conducted to provide information about their programs to applicants, whereas 45% of responses reported that the events were used to advertise their programs to applicants (Table 2 ), and 43% said they were held to facilitate resident-student interactions. A minority, 26%, of programs answered that events served as a way to get to know applicants and determine their fit within the program. Although most programs developed social media accounts for their residency programs after March 1, 2020, their interactions with applicants via those accounts largely did not affect how applicants were viewed (mean 3/10). The respondents who said they evaluated candidates differently in this cycle were more likely to have offered virtual internships and online didactics and less likely to have offered other less structured events like Q&As and meet and greets (Table 3 ).

Table 2.

Responses to Q20: “Why did you offer these preinterview virtual opportunities? Select all that apply.”

#	Answer	%	Count
1	To provide information about our program to applicants	30.08%	40
2	To get to know applicants	11.28%	15
3	To facilitate resident and applicant interaction	18.80%	25
4	To evaluate applicant interest in our program	11.28%	15
5	To determine applicant fit in our program	8.27%	11
6	To advertise our program	19.55%	26
7	Other	0.75%	1
	Total	100%	133

Table 3.

Responses to Q16: “In response to the COVID-19 pandemic, did you change your preinterview selection criteria for candidates during the 2020-2021 application cycle? How so?”

Number of responses	Question	Yes (%), we evaluated candidates differently in this application cycle than in prior cycles		No (%), we evaluated candidates in a similar manner as in prior cycles		Total
1	Virtual open houses	24.14%	7	75.86%	22	29
2	Virtual subinternships	100.00%	1	0.00%	0	1
3	Didactic lectures	100.00%	3	0.00%	0	3
8	Virtual grand rounds	50.00%	3	50.00%	3	6
4	Q&A session with program director and faculty	18.52%	5	81.48%	22	27
5	Q&A session with residents	19.44%	7	80.56%	29	36
6	Virtual facility tour	25.00%	7	75.00%	21	28
7	Other	25.00%	1	75.00%	3	4
9	We did not offer any preinterview virtual opportunities	0.00%	0	100.00%	2	2

Respondents reported factors that impacted interview decisions or applicants' ranking most significantly were late/absent Step two CK scores (33%) and a lack of visiting subinternships (31%). Of respondents, 75% said they did not change preinterview selection criteria, and 74% of respondents felt applicants missed out on fully experiencing their program by not doing an away rotation there. Of the responding program directors and assistant program directors, 54% believed virtual interviews limit a program's ability to gauge applicants, although residency programs remain divided on whether absence of an away rotation opportunity would limit their ability to assess applicants: 37% answered “yes;” and 46% answered “no.” However, a large majority of respondents (80%) believed visiting rotations were at least somewhat important and 35% believed they were very important.

Among those who answered, 50% said the expense of in-person interviews will impact how they interview students in the future, whereas 30% said it will not, and 20% remained unsure. Respondents remain equally split regarding how cost will affect future structures; 40% said expense will impact the decision to offer virtual versus in-person interviews, whereas 40% said it will not, and 20% remained unsure.

Discussion

In this study, we sought to characterize the changes made for the 2021 Residency Match process and residency program leadership impressions of their degrees of success and viability for future cycles. The restrictions surrounding travel and in-person interaction in the era of COVID-19 have affected physicians in nearly every corner of their practice. Residency applicants were faced with changes to the traditional aspects of application and their application cycle. All United States medical students have experienced some degree of limitation in their clinical exposure in the era of COVID-19, even within their own institutions. These limitations may vary from institution to institution, but Association of American Medical Colleges restrictions on rotations at visiting institutions and in-person interviews stand out as a broad and uniformly limiting change to a once-crucial aspect of applying to the Match, especially in more competitive fields like surgery.

The most highly rated virtual preinterview interactions were Q&A sessions with residents and faculty. Faculty viewed the interactions primarily as a means of familiarizing applicants with their programs, rather than a method of primary evaluation of applicants for preinterview vetting. For the minority who said that preinterview interactions altered their perceptions of applicants, a majority said their ranks of these students were positively affected, causing them to be ranked higher. Of identifiable areas affected by COVID-19 restrictions, program directors and assistant program directors cited absent or late Step two CK scores and a lack of visiting subinternships as areas that negatively affected the strength of students' applications. Although Step two CK testing was not directly affected in the way visiting subinternships were, many students experienced unexpected test date cancellations and had difficulty in rescheduling tests at nearby proctored locations.¹⁶ Visiting subinternships were virtually eliminated altogether for most students. It would be difficult to estimate all the impacts COVID-19 had on applicants, but the lack of visiting subinternships and affected Step two CK testing stand out among respondents and may merit a continued examination. Although this study evaluated the perspective of residency program leadership on the virtual offerings only for the 2020-2021 application cycle, the results remain relevant because they pose implications for current and future application cycles given the unexpected length of the COVID-19 pandemic. Cycles in the future stand to benefit from this information and will be well-served to consider the efficacy of various virtual interactions as they plan for an uncertain future. With these changes, we hope to see a greater degree of transparency in program directors’ plans for applicant interactions so that students can also prepare appropriately. Concerns facing students are academic, logistical, and more now than ever, financial.

Outside of academic concerns, the changing landscape of cross-institutional interactions before the residency Match raises the issue of equity in access to visiting subinternships and interviews. Of the responding general surgery residency program directors and assistant program directors, 71% worry about disparities between applicants applying in-person and virtually. Visiting subinternships provide an excellent opportunity for exposure to specialties that may not be present at a student's home institution and an opportunity to potentially audition for a spot at an outside institution. Like many aspects of medical education, however, these subinternships require students to incur a significant personal expense. A 2016 study estimated the cost of a single visiting rotation to be $958.¹³ With many students, particularly those applying to surgery and surgical subspecialties applying for multiple visiting subinternships, these expenses can easily reach several thousands of dollars.¹⁷ Total expenditures among specialties ranged from $1312 to $3465.¹³ Respondents to our survey are largely divided on how to address student expenses, with only half saying student expenses will affect how they interview in the future, including the decision to offer virtual or in-person interviews. In one sense, the restriction of visiting subinternships controls for financial privilege among applicants. On the other hand, it disadvantages students coming from smaller institutions without a large availability of subspecialty rotations. In-person interviews present another disparity in access by requiring students to incur costs of travel and lodging. Rather than assuaging this disparity, virtual interviews allow wealthier applicants to apply to as many programs as they choose without concerns surrounding the expense of extra applications and without having to consider the logistic implications of in-person interviews. The COVID-19 restrictions on travel offer a unique kind of case study on virtual interactions and their impact on access and outcomes and thereby warrant further investigation.

Continuous studies evaluating these online opportunities can better provide applicants and programs with understanding of the impact of these interactions and how to improve in the future. Limitations to this study include a small sample size, potential unknown bias in response, possible subjectivity in the survey itself, and the uncertainty of the accuracy of retrospect among respondents. Our response rate was particularly low at 11.5% (Table 1). We are unaware of any other concomitant surveys at the time of issuance, but time to complete survey and a lack of incentive or compensation likely played some role in the poor turnout. Further studies may benefit from more concise surveys, possibly multiple. This small sample size reduces the statistical power of our study. However, as a purely descriptive study, our results still provide an insight into the experiences of those who chose to participate. These data are based on a collection period that may not have allowed all programs to fully develop their response for the next cycle of the residency Match. Future directions following this study could include postmatch interviews with applicants on their impressions of social media interactions, online open houses, and virtual away electives in comparison with a similar survey given prior to any of these virtual interactions. In addition, as the opportunity cost of advanced medical training and higher education continues to grow out of proportion to students ‘and applicants’ financial means, future studies would be well-served to further characterize the impact of financial burden on United States medical trainees before and after the global COVID-19 pandemic and possible accommodations that may be adopted beyond the era of COVID-19 in the future.

There remains much to be seen regarding what will be done in future application cycles, not to mention what faculty and applicants believe ought to be done. Choosing between fully virtual and fully physical interviews makes an implicit argument about the significance of the interview itself. In-person interviews indirectly assert the importance of first-hand observation of social interaction not otherwise seen on a visiting rotation. Virtual interviews, however, may suggest that applicants have been vetted socially by their letters of recommendation. Allowing applicants to choose presents the problem of stratifying applicants by their presumed interest based on whether they chose to invest the time and resources needed for a physical interview over a virtual one. Programs may benefit from offering two rounds of interviews, the first being virtual and the latter physical. This would allow applicants greater freedom to apply to more programs without a fear of being limited by travel resources. The secondary interview would be in-person, acting to gage an applicant's compatibility with the program and serve as a demonstration of committed interest in a program. In-person interviews could also possibly be waived if a student had already completed a visiting rotation in the department to which they are applying.

Disclosure

None declared.

Funding

None.

Availability of Data

Not applicable.