Utilization of a Virtual Information Session to Increase Engagement With Prospective Applicants in the Setting of COVID-19

Abstract

Purpose

This study aimed to assess the efficacy of a virtual information session hosted by a diagnostic radiology residency program at addressing applicant concerns about the 2020-2021 interview cycle and highlighting key aspects of the residency program.

Methods

Participants were recruited to attend the virtual information session over a 2-week period via social media and communication with medical school radiology interest groups. Attendees were able to submit questions or topics of interest prior to the session. The virtual information session was hosted by trainees and faculty from a radiology residency. Data regarding the demographics of the attendees and the efficacy of the session were obtained through interactive live polling during the virtual session and a voluntary anonymous postsession survey.

Results

A total of 171 attendees participated in the virtual information session. Of the attendees, 42% learned about the session from Twitter and 72% were fourth-year medical students applying for residency. Among topics addressed during the session, attendees indicated that they were most interested in learning about “Application strategies during COVID-19” during an in-session poll. On the post-session survey, 96% of attendees reported being more knowledgeable about the residency program culture and the breadth of research and educational opportunities.

Conclusion

Given the virtual nature of the 2020-2021 residency application cycle, utilization of web-based platforms for recruitment will be essential. Virtual information sessions can be effective at providing insight into aspects of a residency program that are typically gained during the in-person interview experience.

Introduction

The COVID-19 pandemic has had significant effects on the 2020-2021 radiology residency recruitment and application cycle. In accordance with recommendations from national organizations including the Coalition for Physician Accountability, the Association of Program Directors in Radiology, and the Association of Program Directors in Interventional Radiology, interviews and program “visits” will be conducted virtually.^{1 , 2} Both students and residency programs must interview and make ranking decisions without information typically gathered in person during the interview day.³

This paradigm shift has led to challenges and questions for both applicants and residency programs. In the absence of an in-person interview experience, key aspects of residency programs, including the culture and sense of community, may be difficult to convey effectively. In addition to learning about educational and professional opportunities, applicants traditionally use the interview day to assess their “fit” for a program. Proposed strategies to highlight these aspects of residency programs during the 2020-2021 virtual interview cycle have centered on establishing a strong online presence. Specific suggestions have included creating videos to highlight resident and faculty experiences, maintaining an up-to-date residency webpage, and increasing social media engagement among faculty and residents.3, 4, 5

Despite these challenges, the virtual interview cycle may also offer potential benefits for both prospective applicants and residency programs. For example, transitioning to a virtual platform for recruitment and interviewing may lead to significant cost savings for both the applicants and programs. In addition, it also presents the opportunity to increase interaction between applicants and members of the residency program via social media or other web-based platforms.5, 6, 7

Since the 2020-2021 application cycle is uncharted territory for both programs and applicants, hosting a virtual information session to clarify the structure of the virtual interview experience allows applicants to voice their concerns and enables programs to showcase residency culture and available educational opportunities. Here, we present survey data and insights from hosting a virtual information session for potential radiology residency applicants in the setting of COVID-19.

Methods

The goal of the virtual information session was to provide a forum for medical students to learn more about the 2020-2021 application cycle and the institution's diagnostic radiology program. The target audience was fourth-year medical students applying for diagnostic radiology residency in the 2020-2021 application cycle.

Recruitment of Virtual Information Session Participants

Registration was required prior to the virtual information session but was open to all medical students. Participants were recruited over the period 7/20/2020-8/2/2020 through several different methods. A flyer for the virtual information session with relevant information about the date, topics, and speakers was created by the media services division of the radiology department. The flyer and registration link were posted on social media platforms, including Twitter and Instagram, by the organizers, panelists, and radiology residency accounts. Emails with registration information were also sent directly to medical schools across the country via various professional organization listservs or personal contacts. Information on how to register for the virtual information session was also posted directly to the department's radiology residency website.

Virtual Information Session Logistics

The virtual information session was hosted on the Zoom webinar platform (Zoom Video Communications, San Jose, CA) on Sunday August 2, 2020 at 7 pm EDT. All registrants were required to complete a preregistration form that included name, email, and a free-response topic or question of interest that they would like addressed during the information session. Registrants received an individual participation link via email after completion of the registration form.

A total of 7 faculty and 17 trainees, including residents and fellows, from the radiology department participated in the information session. Faculty participants included the radiology department chair, residency program director, both associate program directors, and 2 recent graduates of the residency who had been hired as faculty. A PowerPoint (Microsoft Corporation, Redmond, WA) presentation was prepared prior to the event. Trainees and faculty members were responsible for specific slides and discussion topics. The advancement of slides and introduction of speakers during the session was coordinated by one of the associate program directors. Panelists signed on in “practice mode” prior to the start of the broadcast to test individual audio and video.

During the virtual information session, only panelists had audio and video broadcast capability. Attendees were able to interact with the panelists in 2 ways. First, the “Public Polling” function of the Zoom platform was used to pose a series of multiple-choice questions to the attendees. Second, attendees were able to submit typed questions via the “Question and Answer” function of the platform. Panelists responded to the questions live and the written answers were immediately visible to all participants.

The virtual information session broadly covered the following topics: anticipated structure of virtual residency interviews during COVID-19, application strategies during the 2020-2021 cycle, and an overview of the institution's diagnostic radiology program and resident life, which highlighted resident wellness and life in the city/region of the residency program.

Postvirtual Information Session Survey

A voluntary, anonymous online survey was distributed to the information session attendees immediately after completion of the session and was emailed to the attendees again the following day. The online survey was hosted on Google Forms (www.google.com/forms) and included a combination of multiple choice and open-ended questions. Survey results were downloaded 1 week after completion of the session and exported to an Excel spreadsheet (Microsoft Corporation, Redmond, WA) for aggregate analysis.

This project was undertaken as a Quality Improvement Initiative, and as such was not formally supervised by the Institutional Review Board in compliance with their policies.

Results

Virtual Information Session Data

The total virtual information session run time was 81 minutes. No significant audio or technical difficulties were encountered. There were 257 registrants and 171 attendees, yielding an overall attendance rate of 67%. There was a maximum of 163 concurrent viewers in the session. The average amount of time spent in the session per attendee was 58 minutes.

The most effective method for recruiting participants for the virtual information session was via Twitter, with 42% of attendees responding via live poll that they had heard about the session on the social media platform (Table 1 ). The second most common source of information was an email from their medical school interest group or from a current radiology resident in the program (32%).

Table 1.

Information session live poll data

In-webinar poll data	n (%)
How did you hear about the virtual information session? (n = 130)
Twitter	54 (42)
Email from medical school or current radiology resident	41 (32)
Professional society	6 (5)
Other social media platform (Instagram/Facebook)	6 (5)
Radiology residency website	2 (2%)
Other	21 (16)
What of the following topics would you most like to learn about during the information session? (n = 130)
Application strategies during COVID-19	54 (42)
Resident experience structure of virtual interviews	31 (24) 23 (18)
Residency opportunities	11 (9)
Ways to interact with residents	10 (8)
Other	1 (1)
Which of the following are you most concerned about regarding the 2020-2021 application process? (n = 136)
Increased number of applicants to each program	55 (40)
Difficulty expressing interest in programs	32 (24)
Lack of away rotations	17 (13)
Obtaining letters of recommendation	12 (9)
Lack of research opportunities	7 (5)
Other	13 (10)
What specific aspect of the residency program would you most like to learn about? (n = 143)
Clinical education and curriculum	31 (22)
Resident life	27 (19)
Diversity, equity, and inclusion initiatives	23 (16)
Professional development opportunities	20 (14)
Research opportunities	14 (10)
Wellness initiatives	9 (6)
Fellowship placement	8 (6)
Other	11 (8)

Participant Demographics

A total of 55/171 (32%) participants responded to the voluntary postsession survey. Participant demographics are detailed in Table 2 . Most respondents attended medical schools located in the Northeast (35%), South (35%), or identified as International Medical Graduates (16%). The majority of survey respondents (72%) were fourth-year medical students, which was the target population for the presentation. In addition, the majority of the respondents (72%) indicated that they intended to apply into diagnostic radiology residency. The remainder of respondents planned to apply to the integrated interventional radiology residency (13%) or was undecided (15%). Most respondents (73%) indicated that their home institution had a dedicated radiology core clerkship.

Table 2.

Participant demographics

Participant demographics	n (%)
Gender
Male	28 (51)
Female	27 (49)
Medical school region
Northeast	19 (35)
South	19 (35)
Midwest	5 (9)
West	3 (6)
IMG*	9 (16)
Year in medical school training
Fourth year	40 (73)
Third year	4 (7)
Second year	3 (6)
First year	1 (2)
Already graduated	7 (13)
What specialty do you plan to pursue?
Diagnostic radiology	40 (73)
Integrated interventional radiology	7 (13)
Undecided	8 (15)
Is there a core radiology clerkship at your home institution?
Yes	40 (73)
No	15 (17)

Virtual Information Session Topics of Interest

A total of 294 questions were submitted by registrants prior to the virtual information session. These prewebinar questions were categorized into 7 general themes agreed upon by consensus between 3 of the authors (Table 3 ). The most common theme pertained to the format of virtual interviews for the 2020-2021 application cycle, which accounted for 40% of all questions submitted. Additional topics of interest included how residency applications will be reviewed during the 2020-2021 cycle (14%), how to learn more about residency programs virtually (12%), and questions related specifically to aspects of the diagnostic radiology residency program (12%).

Table 3.

Prewebinar question themes

Prewebinar question themes (n = 294)	n (%)
Format of virtual interviews	118 (40)
How residency applications will be reviewed	41 (14)
How to learn more about residency programs virtually	36 (12)
Questions about the residency program (culture, research, educational opportunities, etc.)	35 (12)
Ways to strengthen application to residency	22 (8)
Ways to demonstrate interest in a program without away rotations	14 (5)
Other	28 (10)

There was a high response rate to the in-webinar poll questions, with greater than 80% attendee participation. Responses to the in-webinar polls indicated that the topic that attendees were most interested in learning about was “Application strategies during COVID-19” with 42% of votes (Table 1). The second most selected topic of interest was “Learning about the radiology resident experience” with 24% of votes. Among the specific aspects of the residency program, attendees were most interested in hearing about the clinical curriculum (22%) and resident life (19%). The attendees were most concerned about the potential for increased number of applicants to each program during the 2020-202 interview cycle (40%) and possible difficulties expressing specific interest to residency programs (24%).

Postvirtual Information Session Survey

Results from a postwebinar survey to assess the effectiveness of the information session at delivering information on a range of topics are presented in Table 4 . The session was most effective at informing participants about the culture of the residency program as well as breadth of research and educational opportunities available to residents, with 96% of respondents reporting being either “somewhat more knowledgeable” or “significantly more knowledgeable” after the session. The information session was relatively less effective at presenting information regarding application strategies for the 2020-2021 virtual interview cycle, with 74% of respondents reporting being either “somewhat more knowledgeable” or “significantly more knowledgeable” after the session.

Table 4.

Postwebinar survey results—information session effectiveness

Postwebinar survey: information session effectiveness	1 n (%)	2 n (%)	3 n (%)	4 n (%)	5 n (%)
Structure of virtual interview day	1 (2)	0	8 (15)	24 (45)	20 (38)
Ways to learn about residency programs virtually	0	1 (2)	8 (15)	22 (42)	22 (42)
Ways to express interest in residency programs virtually	1 (2)	1 (2)	7 (13)	25 (47)	19 (36)
Application strategies in COVID-19 Era	0	1 (2)	13 (25)	21 (40)	18 (34)
Resources to prepare for virtual interviews	1 (2)	0	7 (13)	26 (49)	19 (36)
Culture of residency program	0	1 (2)	1 (2)	10 (19)	41 (77)
Research opportunities at residency program	0	1 (2)	1 (2)	4 (8)	47 (89)
Educational opportunities at residency program	0	1 (2)	1 (2)	5 (9)	46 (87)
Knowledge about residency program Location (cost of living, things to do, etc.)	0	1 (2)	8 (15)	11 (21)	32 (60)

1 = Significantly less knowledgeable.

2 = Somewhat less knowledgeable.

3 = Same level of knowledge.

4 = Somewhat more knowledgeable.

5 = Significantly more knowledgeable.

The majority or 79% of respondents felt that the information session length of 81 minutes was “just right.” Of the remaining respondents, 17% felt that the length was “too long” and 4% felt that it was “too short” (Table 5 ). There was significant interest among participants in additional virtual content, as 96% of respondents stated that they were either “very likely” or “somewhat likely” to attend additional virtual information sessions during the 2020-2021 interview season (Table 5).

Table 5.

Postwebinar survey results—information session feedback

Postwebinar survey: information session feedback	n (%)
How did you feel about the length of the webinar?
Much too long	0 (0)
Too long	9 (18)
Just right	42 (79)
Too short	2 (4)
Much too short	0 (0)
How likely are you to participate in additional webinars related
To the upcoming virtual interview season?
Very likely	41 (76)
Somewhat likely	11 (20)
Somewhat unlikely	1 (2)
Very unlikely	1 (2)
Unsure	0 (0)

Discussion

As radiology residency programs navigate the 2020-2021 application cycle in the setting of COVID-19, maintaining a robust online presence is critical for providing timely and reliable information to applicants about the interview process and program. Virtual information sessions/webinars hosted by radiology residency programs can increase engagement with prospective applicants, address concerns about the 2020-2021 interview cycle, and showcase important aspects of a residency program.

One of the primary goals of the virtual information session was to provide insight into the culture of the residency program because of the inability to have in-person interviews in the upcoming application cycle. The authors believed that the best way to demonstrate the residency culture was to include a wide range of resident and faculty perspectives. Therefore, a total of 17 trainees and 7 faculty participants were featured during the session. The survey data suggest that the session was effective at achieving this goal, with 77% of survey respondents indicating they were “significantly more knowledgeable” about the culture of the residency after the webinar. In a study by Pretorius et al, radiology residency applicants cited information gathered during their interview day as the most reliable and important data used to determine ranking decisions, with specific emphasis on resident interactions, feel of the city, and interactions with the staff.⁸ In another large survey of applicants across specialties, resident morale was one of the most important factors that applicants considered when choosing a residency program.⁹ With preinterview dinners and in-person interactions with residents and faculty on the interview day not feasible for the 2020-2021 cycle, the ability to convey these features of a program through virtual means is critical.

Substantial coordination may be required to organize virtual information session speakers and create the presentation. In order to minimize potential technological glitches, virtual platforms should be vetted and a trial run implemented. In addition, presenters should log onto the platform prior to the broadcast to test audio and video equipment. Gaining familiarity with new technology platforms is a recurring theme during the COVID-19 pandemic given social distancing efforts and will continue to be important as residency programs plan the logistics of the virtual interview process.

Utilization of a range of web-based strategies to engage with applicants is essential for residency programs for the 2020-2021 application cycle. We employed several different approaches to recruit attendees and disseminate information about the event. The most effective method for recruiting participants in this study was through the social media platform Twitter, which is increasingly utilized in radiology for communication between clinicians/researchers, promotion of ideas, and residency recruitment. Heitkamp et al previously outlined the influence that social media, and in particular Twitter, can have in residency recruitment by program directors.¹⁰ In addition, early efforts to update the radiology residency website were undertaken to promote the virtual information session and ensure that the information provided about the 2020-2021 interview season was accurate. Residency applicants utilize residency program websites to gain insight into the interview process, curriculum, and research opportunities at different programs.¹¹ Consistent maintenance of timely and accurate information, especially regarding the evolving interview logistics in the setting of COVID-19, will be important over the course of the upcoming application cycle. In summary, a multifaceted online approach to engaging applicants will continue to be a focus of the 2020-2021 residency recruitment period and was a major factor that allowed the authors to recruit a large audience for the session.

The robust attendance at this information session was also likely in part due to the heightened levels of anxiety many applicants are feeling as they apply to residency amidst the COVID-19 pandemic. Concerns about the ability to assess the culture of a program remotely are particularly prevalent. Importantly, without the experience of an in-person interview, applicants from groups that are underrepresented in radiology may have difficulty assessing the safety, support, and inclusiveness of the program environment. Further, applicants may be applying to more programs this year because of apprehension over the unfamiliar virtual interviewing paradigm and decreased cost-barriers from travel expenses.⁷ Higher numbers of applications create new stress on the residency selection process and many applicants have questions about how this will be addressed.^{3 , 5} Finally, many medical students have anxiety about the format of the virtual interviews. Fortunately, several resources have been published to address many of these concerns.^{6 , 12}

Given these concerns, applicants will undoubtedly continue to be eager for information about the application process and the structure of the virtual interviews. To the author's knowledge, this was the first virtual information session hosted by a radiology residency program for the 2020-2021 residency application cycle aimed at addressing applicant concerns. Based on the survey data indicating that 96% of respondents were either somewhat or very likely to attend more webinars during the 2020-2021 interview season, the authors anticipate that additional virtual information sessions hosted by other residency programs will continue to be welcomed and attended by applicants. A second virtual information session was scheduled 4 weeks after the first webinar for interested medical students who were unable to attend the first session. A separate virtual information session was also scheduled for the integrated interventional and diagnostic radiology residency program.

This study had several limitations. Although the webinar platform reported the total number of participants who joined, it did not assess how engaged the attendees were during the presentation. Participation in the voluntary postwebinar survey was also only 32% of the total number of participants in the virtual information session. This rate of participation may have led to selection bias in the survey results, as participants who were more engaged in the session may have been more likely to participate in the postwebinar survey and record more positive feedback. In addition, specific aspects of the residency program discussed during the session may be unique to the authors’ institution and not applicable to other residency programs.

Conclusion

Our initial experience demonstrates that radiology residency program virtual information sessions can increase engagement with prospective applicants and showcase unique aspects of a program. Virtual information sessions can be highly effective at providing insight into the intangible aspects of a residency program that are traditionally gained during the in-person interview experience. Maintaining a robust online presence will be critical for radiology residency programs in providing timely and reliable information to applicants in the setting of the COVID-19 pandemic.

Author Contributions

Mari E. Tanaka: Conceptualization, Methodology, Investigation, Writing – Original Draft, Visualization, Project Administration. Holly R. Brideau: Conceptualization, Software, Data Curation, Investigation, Writing – Review and Editing, Visualization, Project Administration. Thomas J. An: Methodology, Software, Formal analysis, Data Curation, Investigation, Writing – Original Draft, Visualization. Theresa C. McLoud: Conceptualization, Methodology, Investigation, Visualization, Writing – Review and Editing, Supervision. Brent P. Little: Conceptualization, Methodology, Validation, Investigation, Visualization, Writing – Review and Editing, Supervision. Hillary R. Kelly: Conceptualization, Methodology, Software, Validation, Investigation, Visualization, Writing – Review and Editing, Supervision, Project Administration.