Abstract
Rationale and objectives
In response to the COVID-19 pandemic reducing medical student presence on clinical services and in classrooms, academic institutions are utilizing a virtual format to continue medical student education. We describe a successful initial experience implementing a virtual elective in interventional radiology (IR) and provide the course framework, student feedback, and potential improvements.
Materials and methods
A 2-week virtual IR elective curriculum was created utilizing a combination of synchronous and asynchronous learning and the “flipped” classroom educational model. Students virtually participated in daily IR resident education conferences, resident-led case review sessions, and dedicated lectures. Asynchronous prelearning material consisted of text and video correlating to lecture topics. Anonymous precourse and postcourse surveys were sent to all participating students (n = 10).
Results
Ten students (100%) completed precourse and seven (70%) completed postcourse surveys. Enrolled students were considering residencies in surgery (50%), internal medicine (40%), interventional radiology (30%), and/or diagnostic radiology (30%). Students’ understanding of what IRs do and the procedures they perform (p < 0.001), when to consult IR for assistance in patient management (p = 0.005), and the number of IR procedures students could recall (p = 0.015) improved after the course. Case-review sessions and virtual lectures ranked as having the highest education value. Students recommended additional small-group case workshops.
Conclusion
This successful virtual IR elective provides a framework for others to continue IR medical student education during the pandemic and grow the specialty's presence within an increasingly virtual medical school curriculum. The described model may be modified to improve IR education beyond the COVID-19 era.
Key Words: Medical student education, Interventional radiology education, Virtual learning, COVID-19
Abbreviations: IR, interventional radiology; PBL, problem-based learning
INTRODUCTION
While the COVID-19 pandemic has disrupted medical education at all levels of training, its effect on medical student education has been especially profound. On March 17th, 2020, the Association of American Medical Colleges recommended the suspension of medical student clinical rotations and has reaffirmed this position in subsequent statements (1). In addition to the suspension of clinical rotations, preclinical curricula have migrated to a virtual format to maintain distancing of students (2).
The current disruptions to medical student education pose unique challenges to interventional radiology (IR). Currently, IR is not well represented within the preclinical curriculum and this status quo is unlikely to change during the transition to a virtual curriculum (3, 4, 5, 6). The lack of preclinical IR experience results in students relying on clinical IR electives and/or IR exposure integrated into diagnostic radiology or surgical clerkships to gain exposure to the field (7). Decreased clinical time during the pandemic has therefore disproportionately reduced medical student exposure to IR. This proves a disservice to students interested in pursuing IR training as well as those bound for other specialties who will interact with IR while caring for their patients.
IR educators must adapt to virtual learning and find new ways to educate medical students, as has been done in diagnostic radiology, surgery, and other specialties (4,8,9). This article describes an initial experience in creating and facilitating a virtual IR elective at an academic tertiary care medical center and includes a description of the course framework, student feedback, and suggestions for future improvements.
MATERIALS AND METHODS
Institutional review board approval was obtained for this survey study. The virtual IR elective consisted of a remotely-attended two-week course offered to third and fourth year medical students on a recurring basis. The course was one of over 30 virtual electives offered by the medical school, which ranged across disciplines including various medical, surgical, and subspecialty areas. Students were registered for the course based on their ranking of elective preferences. The course was capped at a maximum 10 students per iteration to allow for interactive virtual group sessions.
Course Curriculum
The course curriculum was reviewed and approved by the medical school, which required virtual courses to include at least 40 hours of “student effort” per week in the form of synchronous and asynchronous learning activities. Synchronous learning refers to learning in which students and instructors engage in real-time, typically utilizing videoconferencing and/or chat software to allow for live interaction, while asynchronous learning refers to learning that occurs at different times for each student, without real-time interaction, utilizing resources such as assigned readings or prerecorded videos provided by the instructor. The virtual elective consisted of approximately 50% synchronous and 50% asynchronous content. A “flipped-classroom” model, in which students were provided asynchronous educational material to review prior to participating in a synchronous lecture on the same topic, was employed throughout the course (5,10).
Synchronous Learning Activities
Synchronous activities occurred in real-time through institutionally-licensed and Health Insurance Portability and Accountability Act compliant videoconferencing software (BlueJeans, Mountainview, California; Microsoft Teams, Redmond, Washington). Each morning, students virtually attended the daily IR resident education conferences (part of the ongoing IR resident training curriculum). Resident-led case-review sessions modeled after morning “film rounds” (Table 1 ), during which a resident would review the previous day's cases and discuss patients’ medical history, work-up, imaging, procedural details, and follow-up, were held with students at various times depending upon resident availability. Students also attended between one and three daily virtual lectures. These were predominantly attending-led and delivered specifically for medical students enrolled in the course (Table 2 ). Attending lectures were scheduled according to academic availability. Resident-led sessions were scheduled during resident academic time when able, otherwise, residents were excused from clinical service to participate in the course. At the conclusion of the course, students virtually participated in an end-of-course student presentation session, in which each student presented on a topic of interest within IR that was not directly covered in the course.
Table 1.
Daily Interventional Radiology (IR) Resident Education Conferences (Part of the Ongoing IR Resident Training Curriculum) Were Being Held Virtually Prior to the Start of the Medical Student Elective
| IR Resident Education Conferences | Description |
|---|---|
| Chief Rounds | Bi-weekly case-based conference led by the IR division chief during which trainees work through the pre-, intra-, and postprocedural aspects of cases |
| Resident Lecture | Traditional didactic lectures provided on a weekly basis by IR faculty and based on the interventional radiology residency didactic curricula |
| Journal Club | Topic-focused attending-moderated sessions during which residents present 3–4 articles, followed by discussions of study design, methods, validity, and role in the practice of evidence-based IR |
| Liver Tumor Board | Multidisciplinary conference run by the transplant department which discusses diagnostic and treatment options for patients |
| Morbidity and Mortality Conference | Resident-led discussion of procedural and patient-care complications attended by IR faulty and trainees, with a focus on safety and quality improvement |
| Film Rounds* | Daily review of each case performed the day prior – residents typically present the cases they specifically performed, highlighting anatomy, procedural technique, thought process, and educational highlights |
Medical students attended these resident conferences as part of the virtual elective.
For the virtual elective, separate case-based reviews (film rounds) were held specifically for medical students every 1–2 days, during which an IR resident reviewed selected cases with the students, ensuring the discussion was appropriate to their education level and allowing more time for questions.
Table 2.
Virtual Medical Student Lecture Topics and Associated Asynchronous Prelearning Text Materials
| Category | Lecture Topic | Asynchronous Text Material | |
|---|---|---|---|
| Intro | IR basics |
|
|
| Arterial interventions | Peripheral arterial disease |
|
|
| Acute limb ischemia |
|
||
| Trauma |
|
||
| Venous interventions | Venous disease |
|
|
| IVC filters |
|
||
| Interventional oncology | Ablation | NA* | |
| Lung |
|
||
| Kidney |
|
||
| Liver |
|
||
| Transarterial chemoembolization |
|
||
| Hepato-biliary | TIPS |
|
|
| Biliary drainage |
|
||
| Additional topics | Uterine artery embolization and varicocele treatment |
|
|
| PAVM embolization |
|
||
| Lymphatics and thoracic duct embolization |
|
||
AJR, American Journal of Radiology; CVIR, Cardiovascular and Interventional Radiology; IR, interventional radiology; IVC, inferior vena cava; JVIR, Journal of Vascular and Interventional Radiology; NA, not applicable; NEJM, New England Journal of Medicine; PAVM, pulmonary arteriovenous malformation; TechVIR, Techniques in Vascular and Interventional Radiology; TIPS, transjugular intrahepatic portosystemic shunt.
Abbreviated citations are provided for articles and the complete citation for IR playbook is as follows: Keefe, Nicole A., et al, eds. IR Playbook: A Comprehensive Introduction to Interventional Radiology. Springer, 2018.
These materials were provided with the ablation lecture (lung, kidney, liver) that was delivered first.
Asynchronous Learning Activities
Asynchronous learning consisted of prelearning materials in text and video formats to provide background knowledge on topics prior to their corresponding lecture (Table 2) (3, 4, 5,10). Text material was available to students through the institution's library. Videos predominantly consisted of narrated IR procedures available online. Students were also given access to optional IR webinars organized by other institutions.
Course Organization and Evaluation
Content was organized online via course management software used by the medical school (Canvas by Instructure, Salt Lake City, Utah). Students were sent a daily email with the following day's schedule, assigned prelearning content, and videoconferencing links. An example daily schedule is provided in Table 3 . Performance was assessed using an open-book mid-elective quiz focusing on key teaching points covered in the case-review sessions and the end-of-course student presentations. Lecture attendance was also monitored. Completion of asynchronous prelearning material was not directly monitored. The course had a pass and/or fail grading system.
Table 3.
Example Generic Daily Schedule
| Educational Activity | Facilitator | Attendees | Time Commitment | |
|---|---|---|---|---|
| Morning | IR Resident Education Conference | IR attending | Enrolled medical students and IR residents | 1 hour |
| Virtual Medical Student Elective Lecture #1 | IR attending* | Enrolled medical students | 1 hour | |
| Asynchronous Prelearning | - | - | 2 hours | |
| Break | ||||
| Afternoon | Virtual Medical Student Elective Lecture #2 | IR attending* | Enrolled medical students | 1 hour |
| Case-review Session (“film rounds”) |
IR resident | Enrolled medical students | 1 hour | |
| Asynchronous Prelearning | - | - | 2 hours | |
IR, interventional radiology.
The 8-hour daily commitment ensured approximately 40 hours per week of “student effort,” as required by the medical school.
Lectures were predominantly given by attending IR physicians, with the exception of “Basics of IR” and “Trauma,” given by residents.
Surveys and Statistical Analysis
Anonymous web-based surveys consisting of multiple choice, Likert scales, and free response were sent to all enrolled students before and after the course. Survey participation was optional. Both surveys had questions regarding knowledge and interest in IR. The pre-elective survey had additional questions regarding prior exposure to IR. The postelective survey had questions regarding the effectiveness of different course elements. Survey responses were analyzed for trends and compared using the Student's t-test, when applicable.
RESULTS
Ten students had participated in two consecutive iterations of the course (five students per iteration) at the time of manuscript preparation. All students passed the course.
Ten students completed the precourse survey (100%). Seven completed the postcourse survey (70%).
Interest in IR
Participants reported first learning about or developing interest in IR during a prior medical school lecture (n = 4), IR research experience (n = 2), discussion with peers (n = 1) or family (n = 1), or during review of virtual elective course offerings (n = 2). Three students were Society of Interventional Radiology members and one had previously attended an IR national conference. When asked to provide up to three specialties students were considering for residency, students primarily reported surgery or surgical subspecialties (n = 5), internal medicine or medical subspecialties (n = 4), interventional radiology (n = 3), and diagnostic radiology (n = 3) (Fig 1 ).
Figure 1.
Pre-elective and postelective survey response data. Questions marked by an asterisk (*) were asked only during the pre-elective survey. IR, interventional radiology.
IR Knowledge
Regarding IR knowledge prior to the elective, 50% of students (n = 5) felt they “had barely been introduced to IR,” 40% (n = 4) felt “as familiar with IR as any other specialty,” and one student was “slightly more familiar with IR than any other specialty.” After the course, responding students felt they were either “as familiar with IR as any other specialty” (57%, n = 4) or “slightly more familiar with IR than any other specialty” (43%, n = 3). A statistically significant improvement was found in students’ understanding of what interventional radiologists do and the procedures they perform (p < 0.001), when to consult IR for assistance in patient management (p = 0.005), and the number of IR procedures students could recall (p = 0.015) (Fig 1). No difference was found in interest in IR as a specialty choice (p = 0.89).
Student and Educator Feedback
When asked to rank different course elements in order of educational value, case-review sessions, and virtual medical student lectures ranked highest, each with three number one rankings and mean rankings of 2.6 and 2.7, respectively. These were followed by student presentations, IR resident education conference, and asynchronous content, with mean ranks of 2.9, 3.6, and 3.7, respectively. Students provided positive feedback regarding case-review sessions, lectures, course organization, and efforts made to tailor the course to specific student interests. Suggestions for improvement included developing additional small-group case workshops (Table 4 ). IR residents and faculty participating in the course reported entirely positive experiences, with some faculty volunteering to give multiple additional lectures. Faculty reported returning to medical student education in the midst of the pandemic as a “return to normalcy” despite its virtual format and multiple faculty were impressed by students level of interest in IR and the quality of their prelecture preparation, evidenced by student knowledge of lecture topics, and a high level of participation during lectures.
Table 4.
Anonymous Student Feedback
| Anonymous Student Feedback |
|---|
|
IR, interventional radiology.
Feedback was minimally edited for spelling, grammar, and readability.
Discussion
The COVID-19 pandemic and resulting limitations on in-person medical student education has forced educators to rapidly adapt to the “new normal” of remote online learning. While the core IR clinical experience comprised of hands-on procedures and patient-care cannot be replaced, the challenges of finding new ways to engage learners and showcase the scope of IR offers opportunities to positively reshape the future of IR medical student education. The described experience demonstrates that a highly effective virtual IR elective can be created and implemented.
Simulating the clinical experience in a virtual format was one of the foremost goals of the course design. Prior to the COVID-19 pandemic, rotating medical students and residents attended daily IR resident education conference each morning. These conferences were being held virtually to maintain resident education during the pandemic, and students in the elective joined them in virtual attendance, maintaining this aspect of their educational experience. Resident involvement in medical student education was a crucial aspect of the virtual elective. Since residents and medical students work closely during clinical rotations, residents as teachers within virtual electives restore normalcy for both parties, offer future IR applicants’ opportunities to interact with IR residents, and allow residents to hone teaching skills (11). Case-review sessions, which simulate clinical patient-care scenarios more effectively than didactic lectures, allow for increased student interaction and are an effective means of virtual education (4). Students highlighted the resident-led case-review sessions as the best aspect of the virtual elective. In addition to holding case-review every 1-2 days, student feedback recommended the addition of problem-based small group learning opportunities (Table 4). Problem-based learning (PBL), in which groups of students guide themselves through different case-based scenarios while applying their medical knowledge and clinical reasoning skills, provides a student-centered educational approach (12). The creation of successful small-group sessions utilizing PBL is labor-intensive, limiting their inclusion in such a rapidly implemented course, and potential difficulties facilitating multiple groups of students in the virtual setting must be considered. However, PBL is often considered superior to more traditional teaching methods and is well-suited to teaching IR clinical scenarios (12). As a result, virtual PBL sessions are currently being developed for use in IR education at our institution, including within the preclinical curriculum.
The elective applied the “flipped-classroom” model to lectures throughout the course, which encourages asynchronous self-learning followed by synchronous sessions focusing on the application of knowledge (3, 4, 5,10). The number of virtual lectures was much greater than the typical number of lectures in traditional IR elective and represented one of the largest differences between the two experiences. Providing students with asynchronous prelearning reading and video materials to establish background knowledge on a subject allowed facilitators to focus on clarifying difficult concepts, sharing clinical pearls, and engaging with students during virtual lectures. Lectures ranked as one of the course elements with the highest educational-value, likely due in part to the benefits of the “flipped-classroom” model.
The lack of student involvement in clinical patient care and procedures was the largest difference between the traditional IR elective and virtual lR elective. Exposure to procedures could be improved with the creation of new high-quality video content available as prelearning and/or live streaming of cases to students, as has been reported in virtual surgical rotations (13). As students return to clinical service and participate in cases, a “blended-learning” model may be considered to encourage physical-distancing whenever possible. Blended-learning combines online education, either in the form of virtual synchronous or asynchronous learning, with in-person place-based teaching methods (14). This would allow students to be physically present for IR clinical activities and procedures, an irreplaceable aspect of traditional IR clinical rotations, while participating in morning conference, didactic lectures, and other learning activities virtually. The success of the virtual IR elective is encouraging in that students found virtual learning to be effective and would likely be open to its inclusion in future electives utilizing a blended-learning model.
Engagement was further encouraged by tailoring sessions to individual interests. For example, many students in the first iteration of the course were interested in applying to IR and/or diagnostic radiology residencies, thus a residency information session was held. In the second iteration, several students were interested in urology, thus additional genitourinary cases and concepts were covered in the case-review sessions. In fact, many students who planned to apply to a specialty other than IR specifically enrolled in the IR virtual elective, rather than one of many other available electives, with the goal of becoming more familiar with IR's role in patient care as it will relate to their future careers. This unexpected finding highlights the need for more accessible IR rotations for upper-year medical students pursuing nonradiologic careers.
Prior to the pandemic, few practical options regarding IR education existed for students at our institution who did not wish to pursue the traditional 4-week IR elective. Prior studies have shown that medical students are eager to learn about IR and that interest in a 2-week IR elective is high, regardless of career choice (14, 15, 16). Students pursuing non-radiologic specialties may prefer a shorter IR experience tailored towards their specific educational needs. However, only a minority of programs currently offer a 2-week IR elective option (7). In the post-COVID era, offering a 2-week elective targeting students bound for nonradiologic specialties who are unlikely to commit to a 4-week clinical IR elective may allow IR physicians to more effectively educate future colleagues and more broadly increase awareness of IR throughout the medical community. A blended-learning model could also be employed in this setting, in which students spend part of their time on the IR clinical service and the remainder of their time utilizing synchronous and/or asynchronous resources geared towards those entering nonradiologic specialties. Alternatively, a 2-week IR selective can be offered as a subspecialty option within the core surgical clerkship, as is currently done at some institutions (7). Previously, the IR elective at our institution was limited to only the students who had completed the core clerkships, however, the virtual IR elective was offered more broadly, as core clerkships had been suspended This transiently increased IR exposure to students early in their clinical training and may hopefully lead to more opportunities for IR within the core clerkship curriculum. Additional future considerations include offering an abridged course to first and second year medical students to increase the presence of IR within the preclinical curriculum and offering an “away” virtual course to students from other institutions.
Limitations
This study is limited by the small size of the surveyed cohort and the inherent limitations and biases of a survey study. The generalizability of our institutional experience may be limited by the availability of adequate videoconferencing software, ability of IR attendings and residents to participate in synchronous teaching sessions, and access to previously created educational content at other institutions. How participation in a virtual elective impacts student's abilities to obtain letters of recommendation or impacts their application to IR residency was not evaluated. Requests to participate in the elective were received from students from outside institutions, however, no institution-approved mechanism for participation of such students were in place during this initial experience.
CONCLUSION
Interventional radiologists must leverage their ability to embrace technology in order to adapt to the increased emphasis on virtual learning in the wake of COVID-19. This successful initial experience with a virtual IR elective provides a framework for continuing IR medical student education during the pandemic and growing the specialty's presence within an increasingly virtual medical school curriculum.
Disclosures
Scott O. Trerotola is a paid consultant for BD Bard (Covington, Georgia), B Braun Melsungen AG (Melsungen, Germany), W.L. Gore & Associates (Flagstaff, Arizona), Adrenas Therapeutics, Inc (Palo Alto, California), MedComp (Harleysville, Pennsylvania) Cook, Inc (Bloomington Indiana) and Teleflex (Wayne, Pennsylvania) and receives royalties from Cook, Inc, and Teleflex.
Funding
None.
Acknowledgments
The authors thank Dr. Soulen, Dr. Shlansky-Goldberg, Dr. Stavropolous, Dr. Itkin, Dr. Mondschein, Dr. Hunt, Dr. Gade, Dr. Reddy, and Dr. Khaddash (Department of Radiology, Division of Interventional Radiology, Perelman School of Medicine at the University of Pennsylvania) for participating in the course and Dr. Vairavamurthy (Department of Radiology, Division of Interventional Radiology, Keck School of Medicine at the University of Southern California) for inviting students to participate in their COVID-19 webinar series.
Footnotes
Submission type: Medical student education.
References
- 1.Whelan A, Prescott J, Young G, et al. Guidance on medical students’ participation in direct patient contact activities. Assoc Am Med Coll. 2020:1–6. Available at: https://lcme.org/wp-content/uploads/filebase/March-17-2020-Guidance-on-Mediical-Students-Clinical-Participation.pdf. Published March 17 2020. Accessed June 1 2020.
- 2.Rose S. Medical student education in the time of COVID-19. JAMA. 2020;323(21):2131–2132. doi: 10.1001/jama.2020.5227. [DOI] [PubMed] [Google Scholar]
- 3.Wolff M, Wagner MJ, Poznanski S. Not another boring lecture: engaging learners with active learning techniques. J Emerg Med. 2015;48(1):85–93. doi: 10.1016/j.jemermed.2014.09.010. [DOI] [PubMed] [Google Scholar]
- 4.Ehrlich H, McKenney M, Elkbuli A. We asked the experts: virtual learning in surgical education during the COVID-19 pandemic: shaping the future of surgical education and training. World J Surg. 2020;44:2053–2055. doi: 10.1007/s00268-020-05574-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Bilello LA. Turning the tables on tradition: flipped high-fidelity simulation to potentiate learning. Adv Med Educ Pract. 2019;10:959. doi: 10.2147/AMEP.S205967. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.DePietro DM, Kiefer RM, Redmond JW. Increasing medical student exposure to IR through integration of IR into the gross anatomy course. J Vasc Interv Radiol. 2017;28(10):1455–1460. doi: 10.1016/j.jvir.2017.06.040. [DOI] [PubMed] [Google Scholar]
- 7.Goldman DT, Magnowski A, Rochon PJ. The state of medical student teaching of interventional radiology: implications for the future. J Am Coll Radiol. 2018;15(12):1761–1764. doi: 10.1016/j.jacr.2018.07.016. [DOI] [PubMed] [Google Scholar]
- 8.Gomez E, Azadi J, Magid D. Innovation born in isolation: Rapid transformation of an in-person medical student radiology elective to a remote learning experience during the COVID-19 pandemic. Acad Radiol. 2020;27(9):1285–1290. doi: 10.1016/j.acra.2020.06.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Calhoun KE, Yale LA, Whipple ME. The impact of COVID-19 on medical student surgical education: implementing extreme pandemic response measures in a widely distributed surgical clerkship experience. Am J Surg. 2020;220(1):44–47. doi: 10.1016/j.amjsurg.2020.04.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Young TP, Bailey CJ, Guptill M. The flipped classroom: a modality for mixed asynchronous and synchronous learning in a residency program. West J Emerg Med. 2017;15(7):938. doi: 10.5811/westjem.2014.10.23515. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Slanetz PJ, Parikh U, Chapman T. Coronavirus disease 2019 (COVID-19) and radiology education: strategies for survival. J Am Coll Radiol. 2020;17(6):743. doi: 10.1016/j.jacr.2020.03.034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Vernon D.T., Blake R.L. Does problem-based learning work? A meta-analysis of evaluative research. Acad Med. 1993;68(7):550–563. doi: 10.1097/00001888-199307000-00015. [DOI] [PubMed] [Google Scholar]
- 13.Chao TN, Frost AS, Brody RM. Creation of an interactive virtual surgical rotation for undergraduate medical education during the COVID-19 pandemic. J Surg Educ. 2020 doi: 10.1016/j.jsurg.2020.06.039. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Darras KE. Undergraduate radiology education during the COVID-19 pandemic: a review of teaching and learning strategies. Can Assoc Radiol J. 2020 doi: 10.1177/0846537120944821. [DOI] [PubMed] [Google Scholar]
- 15.O'Malley L, Athreya S. Awareness and level of knowledge of interventional radiology among medical students at a Canadian institution. Acad Radiol. 2012;19(7):894–901. doi: 10.1016/j.acra.2012.03.009. [DOI] [PubMed] [Google Scholar]
- 16.Commander C.W. Assessing medical students’ knowledge of IR at two American medical schools. J Vasc Interv Radiol. 2014;25(11):1801–1807. doi: 10.1016/j.jvir.2014.06.008. [DOI] [PubMed] [Google Scholar]