Abstract
Objective
The COVID-19 pandemic has disrupted graduate medical education, impacting Accreditation Council for Graduate Medical Education (ACGME)-mandated didactics. We aimed to study the utility of 2 methods of virtual learning: the daily National Surgery Resident Lecture Series (NSRLS), and weekly “SCORE School” educational webinars designed around the Surgical Council on Resident Education (SCORE) curriculum.
Design and Setting: NSRLS
The National Surgery Resident Lecture Series was a daily virtual educational session initially led by faculty at an individual surgical residency program. Thirty-eight lectures were assessed for number of live viewings (March 23, 2020-May 15, 2020).
SCORE School
Attendance at eleven weekly SCORE educational webinars was characterized into live and asynchronous viewings (May 13, 2020-August 5, 2020). Each 1-hour live webinar was produced by SCORE on a Wednesday evening and featured nationally recognized surgeon educators using an online platform that allowed for audience interaction.
Results: NSRLS
There were a mean of 71 live viewers per NSRLS session (range 19-118). Participation began to decline in the final 2 weeks as elective case volumes increased, but sessions remained well-attended.
SCORE School
There were a range of 164-3889 live viewers per SCORE School session. Sessions have most commonly been viewed asynchronously (89.8% of viewings). Live viewership decreased as the academic year ended and then rebounded with the start of the new academic year (range 4.9%-27%). Overall, the eight webinars were viewed 11,135 times. Each webinar continues to be viewed a mean of 43 times a day (range 0-102). Overall, the eleven webinars have been viewed a total of 22,722 times.
Conclusions
Virtual didactics aimed at surgical residents are feasible, well-attended (both live and recorded), and have high levels of viewer engagement. We have observed that careful coordination of timing and topics is ideal. The ability for asynchronous viewing is particularly important for attendance. As the COVID-19 pandemic continues to disrupt healthcare systems, training programs must continue to adapt to education via virtual platforms.
KEY WORDS: Surgical Education, COVID-19, Virtual Learning, SCORE, medical knowledge, practice-based learning and improvement, systems-based practice
Competencies: Patient Care, Medical Knowledge, Interpersonal and Communication Skills, Practice-Based Learning and Improvement
INTRODUCTION
The COVID-19 pandemic of 2020 caused by severe acute respiratory syndrome coronavirus 2 has had a significant impact on medical education in numerous ways, leading to unique challenges for surgical education.1 A complete cessation of nonurgent surgery mandated in many states by major United States surgical organizations and by the Surgeon General has negatively impacted the operative experience of surgical trainees. In response, the American Board of Surgery and ACGME have introduced flexibility into certification and accreditation standards for trainees and programs.2 , 3 Additionally, disruption of clinical schedules, requirements for quarantine after potential exposure, and disaster staffing models have interrupted many ACGME-mandated group didactic curriculums at many residencies. For example, in one large academic medical center, trainees’ in-hospital staffing was reduced by 67%, forcing changes to learning platforms.4 As hospitals emerged from clinical shutdowns, many leaders planned for a surge in clinical volume as the backlog of postponed elective surgeries were rescheduled. This, coupled with the uncertainty of additional future surges in COVID-19 and ongoing restrictions on in-person didactic sessions, presents ongoing challenges to the availability of faculty and trainees for nonclinical educational activities.
During periods of change, asynchronous learning can become particularly important. In response to widespread clinical and educational disruption in the Spring of 2020 due to COVID-19, the National Surgery Resident Lecture Series (NSRLS) was created at Virginia Commonwealth University, to serve as a replacement for in-person didactics. Following this, the Surgical Council on Resident Education (SCORE) launched a free, weekly “SCORE School” webinar series to respond to the need for enhanced online learning modalities for surgical trainees. SCORE offers a standardized and accessible curriculum for surgical trainees,5 divided into weekly sections called “This Week in SCORE” (TWIS). Prior usage data has illustrated that residents increasingly use SCORE on mobile platforms,6 across programs of varying sizes and types,6 and engage with SCORE-based social media for learning.7
In this study, we aimed to assess the utility of these 2 methods of virtual learning during the COVID-19 pandemic by evaluating viewership data.
METHODS
NSRLS
The National Surgery Resident Lecture Series was a daily virtual educational session led by faculty initially at an individual surgical residency program, which then expanded to include 50 multi-institutional subject-expert faculty. Broad lecture categories were created from a list of sub-specialties and special topics known to be important to general surgery residents for education and board preparation. Faculty with expertise were chosen from each identified discipline. The exact topic to be presented was then refined by group discussion and recent literature was incorporated to maximize the relevant impact of the session. Sessions were conducted at 12:00 PM Eastern Time for live viewing only via Zoom (Zoom Video Communications Inc., 2016). Over the course of the series, which aired from 3/23/20-5/15/20, zoom meeting invitations were emailed out to as many as 53 surgical program directors; as knowledge of the series spread, additional programs were added to the email list. The thirty-eight lectures which aired during this time were assessed for number of live viewings. Viewings were obtained from the list of participants logged into each session. Although we have assessed viewership data, the true denominator of all residents who received the lecture invites is not known.
SCORE
Under the leadership of the American Board of Surgery, SCORE was formed in 2004 by principal organizations involved in US surgical education to improve the education of surgery residents through the development of a national curriculum.8 , 9 The vast majority of US surgical residency programs now use the SCORE portal (http://portal.surgicalcore.org/) to help organize their program didactics and provide self-study resources for their trainees.10 Having observed the success of the NSRLS, SCORE launched “SCORE School” on 5/13/20. These weekly, hour-long, freely accessible interactive didactic sessions feature a volunteer surgical expert presenter along with a SCORE moderator on a webinar platform (freeconferencecall.com). The presenter focuses on modules within that week's TWIS content, discussing important concepts and tying them to clinical scenarios and self-assessment questions. The interactive “chat” feature of the platform allows attendees to respond to case-based questions and ask questions for further discussion. In order to minimize the chance for technical failure, each SCORE School session was conducted with one surgeon moderator, a backup surgeon moderator, and a technical lead from SCORE's staff. The webinars were advertised to surgical program directors by email through the APDS list-serve and were freely available for anyone to attend. Following the live airing, the webinars were made available on the SCORE website without a login required. Attendance at 11 weekly SCORE educational webinars (5/13/20-8/5/20) was analyzed. Participant numbers were tabulated by individual logins to the webinar platform, with repeat identical logins removed.
Analysis
Attendance at each SCORE webinar and NSRLS session was quantified. For SCORE sessions, which aired live and were then made available on the SCORE portal for viewing asynchronously, attendance was categorized into live or asynchronous. Trends over time were analyzed. Descriptive statistics were performed.
RESULTS
NSRLS Lectures
At the 38 NSRLS lectures, there were a mean of 71 live viewers per session (range 19-118). Participation began to decline in the final 2 weeks as elective case volumes increased (Fig. 1 ). Additionally, as sessions aired during the workday, the numbers of participants may have been underestimated as several residents may have viewed the lecture from a single screen. Overall, attendance was significantly higher than expected, given their debut towards the end of the academic year, and their airing during the normal workday. Participation in the multi-institution NSRLS sessions remained fairly steady throughout the course of the series, indicating that virtual didactics were feasible and could function as a useful replacement for in-person didactics for residents.
FIGURE 1.
NSRLS viewings by session. National Surgery Resident Lecture Series Participant numbers for each of the 38 virtual lectures. The lowest number of participants occurred on lecture 9 as the incorrect Zoom code was emailed to viewers.
SCORE Webinars
To date, there have been eleven SCORE School webinars. Because each webinar is separated in time by one week, there are sequentially fewer views for each webinar ranging from the highest viewings for the first webinar, on the pancreas, to the lowest total viewings for the most recent webinar, on the liver (Fig. 2 ). The total number of views per webinar has ranged from 164 (Session 11, Liver) to 3889 (Session 1, Pancreas). Sessions have been most commonly viewed asynchronously rather than live, with a mean proportion of 10.2% live viewings and 89.8% asynchronous viewings. Viewership data is shown in Figure 2. Live viewership decreased as the academic year ended in June 2020, at 2.5% (Session 6, Interpersonal Skills), but increased slightly at the start of the new academic year to 4.6% (Session 7, Preoperative Evaluation 1). Overall, the eleven webinars have been viewed a total of 22,722 times. Each webinar continues to be viewed by a mean of 43 people per day (range 0-102).
FIGURE 2.
SCORE Webinar Viewing Data: Total views per webinar displayed, in addition to proportion of live vs. asynchronous viewings.
DISCUSSION AND CONCLUSIONS
In the face of the unprecedented disruption to surgical training caused by COVID-19, these 2 learning collaboratives were able to rapidly adapt to produce a new series of all-virtual learning experiences. The introduction and maintenance of weekly expert-led interactive webinars based on the TWIS curriculum was used by a large number of surgical residents and may function as a supplement to ongoing self-directed learning even after social-distancing guidelines are relaxed. The success of the NSRLS, initially led by an individual institution, was an important proof of concept for adapting formerly in-person didactics to a virtual format with a steady level of resident participation. The finding that trainees continue to view the SCORE School webinars several weeks after the initial live webinar suggests that a web-based didactic curriculum that allows for independent, asynchronous viewing is important to trainees as they study their craft. Several important lessons can be gleaned from these 2 projects.
Producing didactic webinars with consistent high quality is labor-intensive and can be difficult to maintain over the long-term, particularly when offered without cost to viewers. After 38 NSRLS sessions, the VCU organizers found they could not maintain quality, consistency, and multi-institutional participation while tending to their clinical practices, particularly after resumption of surgical volume across the Northeast and Mid-Atlantic region. Based on the success of the first few months of SCORE School, and anticipated ongoing disruptions to surgical education nationwide, SCORE has decided to continue this series through the end of the 2020 to 2021 academic year, however, given the high proportion of asynchronous viewerings, the series has been transitioned to a recorded webinar with no live component. As the COVID-19 pandemic continues to unfold, we will critically evaluate these virtual didactic modalities moving forward and ensure that the webinars produced are meeting the needs of the residents. Each of these weekly sessions requires recruitment and coordination between SCORE surgeons, staff, and the invited speaker, and demands several hours of time. Attention to speaker diversity has been an important principle for this project.
Linking these webinars to TWIS has proved to be very helpful as the subject matter is predictable and relevant to trainees’ practice and learning. Asking that the speakers organize their sessions around SCORE modules provides a consistent framework around peer-reviewed and timely content.
The reliability of the technological interface has been a paramount concern, not only for the live sessions, but also for asynchronous viewing. This particular vendor was chosen primarily because of its proven stability, as well as pre-existing security against uninvited and potentially disruptive attendees (a phenomenon observed early in the pandemic, which has subsequently been mitigated through software upgrades by most remote-meeting applications). An added benefit of this particular platform was the ability to collate reliable usage statistics—although this feature has also been incorporated into most other large applications.
After so much investment of time and effort for both projects, the ability to view these sessions asynchronously and with a rubric of organization was also desirable. The SCORE platform already hosts videos and multi-media content. Thus, storing them on the portal was fairly effortless for SCORE's professional staff. Understanding the tremendous financial hardships that have befallen hospitals and health-care workers, this content was intentionally placed “in front” of the SCORE paywall to allow free accessibility for anyone interested.
The implementation of a national curriculum for surgery has significantly enhanced the way in which trainees learn. It has also provided a resilient resource for all surgical trainees, regardless of their individual environments and resources. The recent COVID pandemic required a pivot to virtual learning. Following the lead of the NSRLS, SCORE was well-positioned through its already robust portal to be able to provide a popular and easily accessible curricular tool to ensure that the education of current graduate surgical trainees continues unabated. Individual residency programs have also had success in shifting to online didactics, and residents may rely on both institutional adaptations and the resources made available through SCORE during this time to support their ongoing pursuit of mastery of their craft.
FUNDING
The project described was supported by the National Center for Advancing Translational Sciences, National Institutes of Health, through grant number UL1 TR001860 for author CMT. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
DECLARATION OF COMPETING INTEREST
Drs. Joshi, Chahine, and Klingensmith, along with Mr. Hickey, Ms. Barret, and Ms. Bradley serve on the SCORE executive committee. Drs. Theodorou, Boyd, Stern, and Anand and Mr. Tilak have no disclosures to declare.
REFERENCES
- 1.Fong ZV, Qadan M, McKinney R. Practical implications of novel coronavirus COVID-19 on hospital operations, board certification, and medical education in surgery in the USA. J Gastrointest Surg. 2020;24:1232–1236. doi: 10.1007/s11605-020-04596-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.American Board of Surgery COVID-19 Updates. American Board of Surgery. Available at: https://www.absurgery.org/default.jsp?news_covid19updates. Published 2020. Accessed September 12, 2020.
- 3.ACGME Response to Pandemic Crisis. Accreditation Council for Graduate Medical Education. Available at: https://www.acgme.org/COVID-19. Published 2020. Accessed September 12, 2020.
- 4.Lancaster EM, Sosa JA, Sammann A. Rapid response of an Academic Surgical Department to the COVID-19 Pandemic: implications for patients, surgeons, and the community. J Am Coll Surg. 2020;230:1064–1073. doi: 10.1016/j.jamcollsurg.2020.04.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Onufer EJ, Trolard A, Hickey M. SCORE – leveling the playing field for surgical training programs. J Surg Educ. 2019;76:e146–e151. doi: 10.1016/j.jsurg.2019.07.009. [DOI] [PubMed] [Google Scholar]
- 6.Joshi ART, Salami A, Hickey M, Barrett KB, Klingensmith ME, Malangoni MA. What can SCORE web portal usage analytics tell us about how surgical residents learn? J Surg Educ. 2017;74:e133–e137. doi: 10.1016/j.jsurg.2017.10.002. [DOI] [PubMed] [Google Scholar]
- 7.Cassidy DJ, Mullen JT, Gee DW. #SurgEdVidz: using social media to create a supplemental video-based surgery didactic curriculum. J Surg Res. 2020;0:1–7. doi: 10.1016/j.jss.2020.04.004. [DOI] [PubMed] [Google Scholar]
- 8.Sachdeva AK, Bell RH, Britt LD, Tarpley JL, Blair PG, Tarpley MJ. National efforts to reform residency education in surgery. Acad Med. 2007;82:1200–1210. doi: 10.1097/ACM.0b013e318159e052. [DOI] [PubMed] [Google Scholar]
- 9.Bell RH. Surgical council on resident education: a new organization devoted to graduate surgical education. J Am Coll Surg. 2007;204:341–346. doi: 10.1016/j.jamcollsurg.2007.01.002. [DOI] [PubMed] [Google Scholar]
- 10.Joshi ART, Klingensmith ME, Malangoni MA. Best practice for implementation of the SCORE portal in general surgery residency training programs. J Surg Educ. 2018;75:e11–e16. doi: 10.1016/j.jsurg.2018.04.014. [DOI] [PubMed] [Google Scholar]