Necessity Is the Mother of Invention: Virtual Medical Education

The coronavirus disease (COVID-19) pandemic arrived during a time of medical education revision at institutional and national levels, with an ongoing deemphasis of exam scores and an increased focus on experiential learning, individualized curriculum, and reorganized basic science curriculum (1, 2). At the same time, many medical schools were gradually integrating virtual learning into existing live curricula (3). The COVID-19 pandemic and its resultant social distancing requirements led to a rapid, large-scale transition into the virtual space. With this drastic increase in virtual medical education, its optimal delivery, assessment, and broader impact on learners remain active areas of research.

In this issue of ATS Scholar, Nilaad and colleagues report the impact of their COVID-19 virtual format (CVF) curriculum compared with a predominately in-person version one year prior (4). In a standard core biomedical science course, the authors adapted a wide range of preexisting, in-person material for online delivery, including live sessions with interactive components, mandatory small-group sessions, and optional weekly review sessions for focused content review. Notably, there were no major content changes between cohorts, with the same lecture topics and most speakers and small-group facilitators returning from the prior year. Curricula were compared using scores from midterm and final exams. Accordingly, the hybrid-year exams were delivered in person, and the CVF exams were completed online, requiring an anticheating agreement for the virtual examinees. Exams were similar, but not identical, with 28% of the virtual format exam material representing modified or replaced questions. In addition to exam scores, data on content use patterns were also collected. The main outcome was no difference in test scores between the hybrid and CVF groups. As expected, there were significant increases in the number of hours of online course material viewed and in the number of students who completed online material in the CVF group, but there was no observed relationship between the total hours of lecture material viewed and exam scores. The authors did observe a strong correlation between consistent practice quiz download (and presumed use) and exam scores in both the hybrid and CVF groups.

This study demonstrates the comprehensive delivery of a virtual biomedical science course while reporting consistent learner engagement and short-term knowledge acquisition, comparable with a predominantly in-person curriculum. Features of this study’s design, as well as some of its findings, provide useful insight into the development and assessment of a virtual curriculum. There was no correlation between exam scores and lecture viewership in the CVF group, a finding that is consistent with prepandemic research, which has failed to demonstrate a consistent association between in-person lecture attendance and multiple-choice exam scores (5). This observation suggests that virtual didactic lectures are no more effective (and also no worse) than in-person didactics and offers an opportunity for curricular innovation, with an increased focus on an active, experiential curriculum on the basis of adult learning principles (6). Unfortunately, the virtual space often feels at odds with active learning, as student engagement, group work, and real patient assessment can be particularly challenging. New strategies are needed to facilitate higher yield virtual learning, and fortunately, this is an area of active innovation, with recent studies incorporating telemedicine visits and virtual operating room participation for learners (7, 8).

One of this study’s primary observations is a clear association between practice quiz download and exam scores that was present in both hybrid and CVF groups. Although this may suggest that practice quizzes lead to improved knowledge and competency, the quizzes were modeled to reflect exams and may have led to improved exam scores through short-term memorization as opposed to true comprehension. The use of exam scores to assess the effect of a diversified curriculum also has its own pitfalls. As an isolated outcome, exam scores may fail to comprehensively assess learner experience and durable competence (9, 10). Metrics used to assess virtual curricula in other industries, including program usability, cost-effectiveness, and user satisfaction, will be helpful in assessing future virtual curricula (11). Medicine-specific outcomes are also important but often challenging to assess. For example, this study converted the small-group practice-of-medicine experience to a virtual environment, but its impact on student experience and patient interactions is unknown. Student performance on standardized patient encounters and clerkship evaluations may help assess this portion of a virtual curriculum.

Increased use of virtual curricula may lead to unintended negative consequences, and the authors acknowledge potential unmeasured impacts on identity development, social engagement, and learner motivation. Participation in medical education is important in students’ doctor identity formation and professional development, and it is unknown how these may be affected by virtual learning (12, 13). Another area of concern in the broad integration of virtual medical education, and a limitation of this study’s generalizability is its potential to exacerbate preexisting socioeconomic disparities. Access to course material and live online sessions at home is dependent on students’ having a computer and Internet access, which may not always be available (14). These economic and technologic barriers vary among institutions and student populations and present a potential limitation to the use of virtual curricula across all medical schools, especially in the absence of preimplementation needs assessment and barrier reduction strategies.

The COVID-19 pandemic has served as a catalyst for rapid expansion and innovation within the virtual learning space. With these innovations, there are significant challenges in the delivery of effective virtual medical education. Although this study has demonstrated the successful implementation of a virtual biomedical science curriculum, there is more work to be done to optimize the integration of virtual technology into our evolving concept of competency-based medical education.