COVID-19 online medical education and outcomes on Internal Medicine In-Training Examination scores

Abstract

Background

COVID-19 presented a unique opportunity to explore new methods to deliver medical education virtually due to requirements for social distancing. We provided webcams and microphones in each of our core teaching team rooms. We used existing teleconferencing systems with share screen, polling, and audio/video capabilities to continue fostering a group learning environment.

Methods

The Internal Medicine In-Training Examination (IM-ITE) was used as a surrogate measurement of the effectiveness of virtual medical education, comparing composite scores from 2015 to 2019 (pre-COVID, in-person conference) to 2020 (post-COVID, virtual conference) for each postgraduate class.

Results

No statistically significant differences between the mean or median scores on the IM-ITE were noted for all three classes.

Conclusion

Although COVID-19 presented many challenges to residency programs across the United States, our pilot study demonstrated that virtual medical education did not result in lower IM-ITE scores and may be an innovative solution to bridge the education gap during COVID-19.

Coronavirus disease 2019 (COVID-19) caused by the novel virus SARS-CoV-2 has changed the landscape of graduate medical education across the United States.¹ At the onset of COVID-19, residents across the country were called upon to serve as frontline workers in the fight against this global pandemic. The unprecedented spread of COVID-19 impacted education of residents and fellows. The Centers for Disease Control and Prevention’s recommendation for social distancing to limit the spread of the disease essentially canceled traditional didactics for training programs.¹ This strongly challenged the current paradigm of graduate medical education that includes in-person educational didactics, formal lectures, and bedside teaching.

Prior to COVID-19, our weekly didactic included morning report (Monday to Thursday) using a case-based format, didactic noon conference (Monday, Thursday), internal medicine grand rounds (Friday), and academic half day during our clinic week. We also emphasized bedside teaching with our faculty and used grand rounds to discuss innovative teaching methods.

We made a quick transition to a virtual platform, first with Microsoft Teams (Microsoft Corporation, Redmond, WA) and later with WebEx (Cisco, Milpitas, CA), which was already available through our hospital system. We continued to schedule our morning report and noon conferences through these virtual platforms and emphasized participation through polling, quizzes, and interactive discussions, which helped to mimic in-person conferences. The implementation of a virtual learning environment posed many challenges, including difficulties with participation, accountability of participants, loss of interpersonal relationships, and connectivity issues. We tried to mitigate these issues by providing a webcam in each room to allow for virtual participation and interactions with residents. We also recorded all the lectures except those with protected information, such as morbidity and mortality conferences, and stored these lectures via OneDrive (Microsoft Corporation, Redmond, WA), accessible by all residents for later review.

COVID-19 has provided a unique opportunity to evaluate the effectiveness of a virtual learning platform. We aimed to understand how the virtual learning environment impacted education during the COVID-19 pandemic.

METHODS

The Baylor Scott and White Health System institutional review board determined this study to be exempt.

From July 2020 to June 2021, medical education transitioned to an all-virtual experience. Using the American Board of Internal Medicine blueprints as the outline for the academic year, the internal medicine conference chief developed a virtual curriculum to transition in-person didactics to a virtual platform. As a surrogate objective measure of the success of the virtual platform, we compared the Internal Medicine In-Training Examination (IM-ITE) for residents who received in-person conferences from 2015 to 2019 and those who received virtual conferences from 2020 to 2021. The IM-ITE is taken by all internal medicine residents in August to September. The sample size varied for each postgraduate year (PGY), with the PGY-1 class having 22 participants; the PGY-2 class, 19; and the PGY-3 class, 20. The comparison sample composite of 2015 to 2019 had 103, 106, and 106 participants, respectively, for the PGY-1, PGY-2, and PGY-3 classes.

Means, medians, and standard deviations were used to describe 2015 to 2019 and 2020 scores. A Wilcoxon rank sum test was used to assess whether there was a statistically significant difference for each PGY class between residents who received an in-person medical education (2015–2019) or a virtual medical education (2020). Statistical significance was determined at a P value <0.05. All statistical analysis were performed in SAS 9.4.

RESULTS

As shown in Table 1, for the PGY-3 class, the aggregate 2015–2019 IM-ITE had a mean score of 61.61 and median score of 65.0, whereas for 2020 the mean score was 62.27 and the median score was 69.5. For the PGY-2 class, the aggregate 2015–2019 IM-ITE had a mean score of 65.14 and median score of 68.0, and for 2020 the mean score was 57.42 and the median score was 61.0. For the PGY-1 class, the aggregate 2015–2019 IM-ITE had a mean score of 66.11 and median score of 68.00. For 2020, the mean score was 62.50, and the median score was 59.5. The 2020 score was 0.66 points higher for the 2020 PGY-3 group, but the difference was not significant. The 2020 scores were lower for the PGY-2 and PGY-1 groups (−7.72 and −3.61, respectively), but these differences also were not statistically significant.

Table 1.

Comparison of In-Training Examination scores for pre-COVID and post-COVID education groups

PGY	Cohort	N	Mean	SD	Min	Max	Median	P value
3	2015–2019	103	61.61	23.04	10	99	65.0
	2020	22	62.27	28.33	7	97	69.5
	Difference		0.66				4.5	0.72
2	2015–2019	106	65.14	21.72	11	99	68.0
	2020	19	57.42	21.69	16	88	61.0
	Difference		−7.72				−7.0	0.18
1	2015–2019	106	66.11	22.14	11	98	68.0
	2020	20	62.50	21.22	27	90	59.5
	Difference		−3.61				−8.5	0.44

DISCUSSION

The education of medical professionals remains largely unchanged since critiques about the American medical education system were introduced by Abraham Flexner in the Flexner Report.² After publication of this report, the American medical education system focused on mainstream scientific evidence-based research and didactics, resulting in the closure of many non–evidence-based medical schools throughout the United States.² Prior to COVID-19, we used a combination of formal didactic sessions and bedside teaching rounds, which transitioned to a fully virtual platform at the start of the academic year when the Centers for Disease Control and Prevention recommended social distancing and limiting of in-person conferences.^1,3–5

The results of this study suggest that the virtual platform is a viable option for learning. The transition to a fully virtual platform helped the residents continue to foster a sense of community among their colleagues. Almarzooq et al published four key features of a virtual platform including integration, collaboration, education, and communication.³ Both Microsoft Teams and WebEx are available as an application on the phone and computer, allowing residents to be able to login anywhere with internet or cellphone service, which helped disseminate conferences to more residents than in-person conferences. Both of these platforms allowed for simultaneous sharing of content for anyone logged into the meeting, allowing both residents and attendings to fully collaborate on presentations. These meetings were recorded, and the content was immediately available for review after the presentation via a password-protected link. Both platforms also allowed for polling of the audience, sharing of files, and marking attendance or absence of residents. In our study, we assessed attendance via a login link which automatically marked attendance, and expected a >80% rate, which was achieved. In addition, engagement was assessed via multiple-choice questions/open-ended questions, and participation was marked in the chat-box, giving credit for the “house-cup” competition to encourage participation.

There were several limitations to the transition from an in-person to virtual platform. Both residents and faculty struggled with the new technology, including use of the integrated webcam and microphone, poor bandwidth connections due to the rapid increase in telemedicine and virtual medical education, and difficulty with imaging and videos embedded into the presentations. However, as the year continued, the virtual platform was generally well received, as residents were able to view the content from anywhere with an internet connection, content was delivered virtually to our two primary teaching sites without need to travel between locations, and content was immediately available for review following the presentation.

Though COVID-19 was disruptive, the quick transition to preexisting platforms helped smooth the transition from in-person to virtual. New innovative teaching methods were implemented, including the flipped classroom, one-minute clinical preceptor, SNAPPS model of education, and ultra-brief teaching after a brief introduction during grand rounds. We would recommend a short teaching session regarding utilization of the intended platform prior to use, as this helped orient those new to virtual learning. Further investigation is needed to determine the optimal learning environment during a pandemic.

Limitations of this study include a single-center study with a relatively small sample size, which may not be generalizable to other institutions. Another potential limitation is the variable transition to virtual education for the 2020 PGY-1 class. Given that COVID-19 spread across America starting in January, there would have been a variable transition time depending on each feeder institution; however, they would have received uniform virtual education starting in July when they started as internal medicine residents.

COVID-19 provided an opportunity for growth in the area of graduate medical education. Quickly transitioning from an in-person didactic and bedside teaching environment to virtual education challenged graduate medical education across America. However, despite the challenges associated with virtual medical education, it is a viable alternative to in-person teaching during a pandemic and may be a hybrid option for the future.

Disclosure statement/Funding

No funding or potential conflict of interest was reported by the authors.

The authors report no funding.