Applicant perceptions of virtual interviews for society of urologic oncology fellowships during the COVID-19 pandemic

Abstract

The COVID-19 public health emergency forced the conversion of in-person SUO fellowship interviews into virtual interviews. We sought to understand applicant perspectives and preferences related to virtual interviews and whether programs should consider virtual interviews in the future. We distributed a survey to 2020 SUO Fellowship interview participants at 4 SUO urologic oncology fellowship programs. Response items were on a Likert scale scored 1–5 with higher scores indicating greater agreement with the survey item construct. Survey responses were collated and thematic mapping used to describe open text responses. Descriptive statistics were used for analysis of survey and open text results.

Fifty-eight SUO fellowship applicants completed the survey. Virtual interviews successfully promoted interaction with SUO fellowship program faculty (mean 4.6, SD 0.6), outlined program research opportunities (mean 4.5, SD 0.7), and proffered opportunities to ask questions about the fellowship (mean 4.7, SD 0.5). Applicants exhibited weakly positive orientation to the adequacy of the virtual format (mean 3.5, SD 1.1). 63% of applicants would prefer a virtual format in the future. Qualitative feedback noted the benefits of virtual interviews were lower cost and reduced time away from residency. SUO fellowship applicants exhibited mixed preferences for virtual and in-person interviews. Although virtual fellowship interviews have benefits such as cost savings and time efficiency, notable weaknesses included challenges observing the culture of the programs. Following the pandemic, SUO fellowship programs may consider virtual interviews but should consider incorporating opportunities for informal interactions between faculty, fellows, and fellow applicants.

Introduction

The COVID-19 public health emergency has necessitated drastic national lockdowns that impacted graduate medical education on multiple levels, including processes of recruitment. In March 2020, many municipalities and states instituted shelter-in-place orders and many academic institutions imposed travel restrictions in order to ensure public safety and attempt to limit spread of the novel SARS coronavirus 2. In response to these orders, fellowship programs in urologic oncology transitioned from in-person interviews to virtual formats.

These virtual interviews represented both an opportunity and a threat. Fellowship interviews are burdensome on applicants due to the time commitment required during a foundational senior clinical year of residency as well as due to the cost of self-funded trips. Reducing the cost and time commitment could facilitate interviews at a greater number of—or more geographically dispersed—institutions. However, decisions regarding fellowship often center around the clinical experience, research experience, career development, and culture of the program, and it may be difficult to garner a full sense of the program through limited remote interactions.

We sought to understand applicant perspectives related to virtual SUO fellowship interviews. We hypothesized that due to the reduced time and cost commitment of virtual interviews that applicants would prefer virtual to in-person interview formats. The results would inform our plans for SUO fellowship interviews in the future.

Materials and methods

We constructed a survey that queried applicants regarding aspects of fellowship interview interactions hypothesized to influence decision-making regarding fellowship ranking. We focused on the constructs of faculty and fellow interaction, degree of information transfer related to the clinical and research programs, perception of the program culture, and preferences around virtual vs. in-person formats. Candidate responses were selected from a Likert scale ranging from “strongly agree” to “strongly disagree” with the exception that respondents were queried with yes/no responses about preferences for virtual versus in-person interviews. We further asked survey participants an open text question around reasons why virtual vs. in-person formats are preferred.

SUO fellowship applicants to 4 urologic oncology fellowship programs were queried and asked to complete the survey electronically. No respondent characteristics were collected to prevent identification of individual SUO fellowship applicants. Surveys were distributed 1 week after the interview dates. Responses were aggregated and analyzed with descriptive statistics. Open text responses were analyzed with thematic mapping and collated with descriptive statistics. Illustrative text responses were paraphrased to prevent identification of respondents including their ability to recognize their own responses.

Results

The survey was distributed to all participants in SUO urologic oncology fellowship interviews at 4 SUO-accredited fellowship programs in Spring 2020. We received complete responses from 58 fellowship applicants (median 14.5 per fellowship program).

Survey responses are tabulated quantitatively in Table 1 . In general, virtual interviews successfully conveyed the clinical, research, and didactic structure of the fellowship programs. Highest affirmative responses were for ratings of the quality of interactions with the fellowship faculty, the ability to have questions answered about the fellowship, and the quality of review of the research opportunities in the fellowships. There was little variation by SUO fellowship program with respect to survey responses for these items.

Table 1.

Responses of SUO fellowship applicants regarding 2020 virtual fellowship interviews regarding their satisfaction with the following interview constructs.

Mean (SD)	Overall	Program 1	Program 2	Program 3	Program 4
Survey respondents	58	10	17	12	19
Faculty interaction⁎	4.6 (0.6)	4.7 (0.5)	4.6 (0.5)	4.8 (0.4)	4.4 (0.8)
Fellow interaction⁎	4.4 (0.8)	4.4 (0.7)	4.7 (0.5)	4.3 (1.0)	4.2 (0.8)
Fellowship culture⁎	4.3 (0.8)	4.7 (0.5)	4.2 (0.8)	4.3 (0.9)	4.2 (0.9)
SUO Fellowship presentation Overview⁎	4.4 (0.6)	4.6 (0.5)	4.5 (0.5)	4.3 (0.9)	4.3 (0.7)
Clinical program⁎	4.3 (0.9)	4.4 (0.5)	4.4 (0.9)	4.1 (1.1)	4.2 (1.0)
Research opportunities⁎	4.5 (0.7)	4.7 (0.5)	4.6 (0.5)	4.5 (0.7)	4.3 (0.8)
Didactic program⁎	4.1 (0.9)	4.1 (0.7)	4.3 (0.7)	3.8 (1.2)	4.0 (0.9)
Fellowship questions⁎	4.7 (0.5)	4.9 (0.3)	4.7 (0.5)	4.8 (0.5)	4.6 (0.5)

^⁎Responses were scored 1 through 5 from “strongly disagree” to “strongly agree”.

Responses to questions about the review of the didactic program and the ability to understand the clinical opportunities in fellowship garnered more tempered responses and this appeared to be consistent across fellowship programs.

Enthusiasm for the virtual format was weakly positive (Table 2 ). The mean response from SUO fellowship applicants aligned with a response between “neither agree nor disagree” and “agree” that the virtual interview was an adequate replacement for an in-person interview with minimal variation in response by fellowship program.

Table 2.

Responses of SUO fellowship applicants regarding 2020 virtual fellowship interviews regarding their preferences for virtual versus in-person interview.

Mean (SD)	Overall	Program 1	Program 2	Program 3	Program 4
Survey respondents	58	10	17	12	19
Adequacy of virtual format⁎	3.5 (1.1)	2.9 (1.1)	3.6 (1.3)	3.5 (1.1)	3.7 (1.0)
Prefer virtual format	63%	50%	76%	58%	61%

^⁎Responses were scored 1 through 5 from “strongly disagree” to “strongly agree”.

Reasons given for preferring in-person interviews (Table 3 ) included the ability to visit the program hospital (n = 19) and the surrounding city (n = 16), observing informal interactions with program faculty (n = 15) and fellows (n = 14), and observing the general culture of a program (n = 11). Several applicants (n = 6) also noted that in-person interviews also allow for important interactions between fellow applicants.

Table 3.

Qualitative themes from SUO fellowship applicant feedback regarding virtual versus in-person fellowship interviews in rank order of frequency of mention of themes.

Themes favoring virtual interviews	Exemplar quotes
Expense of in-person interview	“Candidates save a significant amount of money in travel expenses”; “Virtual interviews save an incredible amount of money for the applicant”
Increases fellowship program applications	“Applicants are able to interview at more programs”; “Would enable people to interview at more places and make a more informed decision”
Maintenance of clinical responsibilities	“Time does not need to be taken away from clinical activities for interviews”; “The flexibility in the time constraint is helpful”; “Residency requirements are always an issue”
Themes favoring in-person interviews	Exemplar quotes
Hospital visit	“A tour of the facilities is an integral experience of an interview”; “Tours really do make a difference”
City visit	“Important to see the surrounding area where fellows will live and commute from”; “You lose the ability to experience the city”
Informal interactions with faculty	“Being able to observe interactions between faculty and fellows”; “Getting a grasp of the department and their interpersonal interactions”
Informal interactions with program fellows	“Having more extended time to talk to fellows could influence one's rank list”
Appreciation of program culture	“Difficult to gauge the environment and culture”; “Valuable moments occur when there is an opportunity to socialize”; “Appreciate the intangibles that cannot be appreciated online”
Informal interactions with other fellow applicants	“Get to meet your cohort of fellows”; “You benefit from hearing about other fellows’ experiences at various programs”

Reasons given for favoring virtual interviews included the frequently cited theme of expense (n = 19), in which applicants save a substantial amount of money on travel and lodging as well as the expense of time. Other themes elicited included that virtual interviews allowed applicants to have less time away from resident clinical responsibilities (n = 6) and allowed applicants to interview at a greater number of SUO fellowship programs (n = 6). A minor theme elicited (n = 2) was that fellowship applicants perceived that more program faculty were able to participate in virtual interviews than otherwise would have participated in in-person interviews.

Discussion

The COVID-19 public health emergency upended normal educational processes including the 2020 SUO fellowship interviews and match. Importantly, this was a largely successful process wherein faculty and fellow applicants were able to connect over virtual platforms. Fellowship applicants acknowledge that they were able to save significantly on the expense and time required to participate in fellowship interviews across the country. However, enthusiasm for continuing in virtual formats was tempered by an inability to fully appreciate the facilities, regional area, and culture of a fellowship program.

Interview costs can be tremendous. Urology residency interviews, which intrinsically may involve interviews at a greater number of residency programs than subspecialty fellowship interviews, incurred a median total expenditure between $7000 in a 2015 multicenter survey study [1] of 173 applicants and $9725 among 64 applicants interviewing at a single residency program [2]. However, subspecialty fellowship interviews in other disciplines are similarly costly. The mean cost of orthopedic surgical fellowship interviews was $5875 [3] and 22% of applicants for general surgery fellowships spent more than $8000 for applications and travel for interviews [4]. Applicants that responded to our survey most commonly cited cost as a reason to continue with virtual formats for SUO fellowship interviews. That reduced cost and the virtual format also potentially facilitated interviews at a greater number of fellowship programs.

Similarly, interviews incur the cost of time away from resident responsibilities. Residents interviewing for subspecialty urology fellowship are typically in their senior years of training, a critical developmental time for surgical skills and judgement. Fellowship interview-associated travel leads to an additional 7-10 days away from residency training, time which requires coverage plans and opportunity cost for foundational experience [3,4].

Virtual interviews were able to successfully present the formative experiences that are integral to SUO fellowship such as the research opportunities and clinical program. These are typically laid out in PowerPoint presentations during in-person interviews as well and would most certainly translate to a virtual format. However, opportunities to observe informal interactions between faculty, between faculty and fellows, and between fellows and residents precluded important observations about program culture. Fellows noted that fit and culture are important determinants in their rankings, and these informal discussions offer tremendous value to the decision-making process. This was a common theme in other disciplines as exposure outside of the interview itself offers applicants valuable insights into program culture [5,6].

Limitations of our study include the possibility for overlapping survey respondents, the timing of surveyed applicants early in the pandemic, and the lack of information regarding program director and faculty perceptions of the virtual interviews. Because we did not collect identifying information from survey respondents, we possibly obtained survey responses from the same applicant through different fellowship programs. However, the 4 fellowship programs represent 4 geographically dispersed institutions (West, Midwest, East) and may have non-overlapping cohorts. These interviews took place within the first 2 months of the pandemic, in March and April 2020. Since that time, many of us have become more familiar and comfortable with virtual interactions for clinical care, scientific meetings, and interviews. If we were to survey participants in the 2021 SUO urologic oncology fellowship match, our results may differ and be more supportive of virtual formats going forward. Lastly, we did not gather the perspectives of faculty participating in these virtual interviews. A survey of female pelvic medicine and reconstructive surgery fellowship program directors who participated in virtual interviews necessitated by the COVID-19 public health emergency actually demonstrated similar results as our fellowship applicants [7]. Although the virtual interviews were felt to effectively promote interaction with fellow applicants, more than two-thirds of program directors preferred in-person interviews.

Conclusions

These survey results provide valuable insights that inform fellowship interview planning in a post-COVID educational environment. Virtual interviews offer important benefits to applicants given the typically burdensome expense associated with interviewing. Yet some applicants would prefer a return to in-person interviews when feasible. In the future, SUO fellowship programs may consider hybrid interview processes that allow for virtual options. This may allow for applicants to consider programs that otherwise might be geographically constraining for interview planning given the time and cost required. In considering how to operationalize these survey responses, any virtual option must consider how to incorporate informal interactions and virtual social events to support a greater exposure to fellowship program culture.