Abstract
Background:
During the COVID-19 emergency, medical students were mandated to remain home, creating challenges to providing education remotely for third-year clinical rotations. This study aims to assess student reception and investigate objective outcomes to determine if online learning is a suitable alternative.
Methods:
Medical students enrolled in the third-year surgical clerkship during COVID-19 were asked to participate in a survey. 19 of 27 (70%) students participated. Content, faculty-led lectures, and resident-led problem-based learning (PBL) sessions were assessed using a ten-point Likert scale. National Board of Medical Examiners (NBME) examination, weekly quiz, and oral examination scores were compared to previous years. Student t-tests compared the groups.
Results:
The median age was 25 years. Comparing in-person to electronic sessions, there was no difference in effectiveness of faculty sessions preparing students for NBME (6.2 vs. 6.7, P = .46) or oral examinations (6.4 vs. 6.8, P = .58); there was also no difference in resident-led PBL sessions preparing students for NBME (7.2 vs. 7.2, P = .92) or oral examinations (7.4 vs. 7.6, P = .74). Comparing this group to students from the previous academic year, there was no difference in weekly quiz (85.3 vs. 87.8, P = .13), oral examination (89.8 vs. 93.9, P = .07), or NBME examination (75.3 vs. 77.4, P = .33) scores.
Discussion:
Surgical medical didactic education can effectively be conducted remotely through faculty-led lectures and resident-led PBL sessions. Students did not have a preference between in-person and electronic content in preparation for examinations. As scores did not change, electronic education may be adequate for preparing students for examinations in times of crisis such as COVID-19.
Keywords: surgical education, general surgery
Introduction
The novel coronavirus (SARS-CoV-2) and the associated respiratory illness (COVID-19) led to the United States limiting social gatherings and preventing nonessential travel.1–4 As physicians and hospital systems braced for the influx of COVID-19 cases, there emerged a need to adjust the education provided to students across the country, including medical students.5 The Association of American Medical Colleges (AAMC) issued recommendations concerning clinical clerkships and suggested that all education be converted to remote learning until the health risks abated.6
Medical students typically structure their education into 2 parts, preclinical education in the classroom and clinical education in the inpatient and outpatient settings.7 The final years of medical education focus on clinical education, where students rotate through medical specialties in a hospital or outpatient-based setting, learning to care for patients and refining clinical skills. A variety of innovations have been made over the past few decades, creating new ways to deliver educational content to students. For the initial 2 years, problem-based learning (PBL) and team-based learning have been suggested.8,9
While many degrees can be obtained through online media, the doctorate of medicine degree is one that has required in-person didactic sessions, hands-on and skill-based learning, and a team-based approach.7 During the COVID-19 crisis, our department of surgery implemented innovative strategies, altering the traditional in-person curriculum to an electronic web-based approach. Although many other medical schools have exclusively pivoted to an online curriculum, there is little literature to investigate the validity of this method of teaching for medical students on clinical rotations.10 This study aims to examine the student perception of the effectiveness of this teaching media, new to clinical medical education clerkships, and to quantitatively compare objective outcomes, such as examination scores, across multiple years to determine if electronic-based learning hinders student performance. Additionally, a secondary aim was to confirm if students in our cohort prefer PBL sessions to faculty-led lectures as has been suggested by other studies.11,12 We believe this to be the first study examining these metrics in the medical student surgical education literature.
Methods
Data Source and Patient Selection
Institutional review board approval was obtained, and a waiver of informed consent was granted. A total of 27 students were enrolled in the third-year medical student surgical clerkship for the fifth block of the year, February 17-April 10, 2020. Students completed education and training in the hospital for 4 weeks, until March 15th, after which they completed the remainder of their 8-week education electronically. Of the 27 students enrolled, 19 completed the electronic survey distributed. The survey was anonymous and optional and had been piloted earlier in the year for in-person education but was adapted to accommodate the online-related questions. The survey was distributed on the last day of the clerkship, and the students were given 1 week to complete it. Additionally, the grades of students who completed the surgical clerkship from the previous academic year (2018–2019) were also obtained. In order to reduce confounding, students in the fifth block were compared to students from the fifth block in 2018–2019 as exam scores tend to vary throughout the academic year.
Design of Curriculum
Traditionally, the surgical rotation lasts 8 weeks and has several important components. A faculty member delivers weekly lectures on topics pertaining to general surgery. The topics include trauma and vascular surgery, thyroid and parathyroid, inguinal hernia, esophagus, gallbladder, appendix, breast, and colon. The students have a ten-question quiz with information they learned from the faculty lecture as well as required readings from the textbook (Essentials of General Surgery, fifth edition, Peter F. Lawrence-Lippincott Williams & Wilkins, Philadelphia, Pennsylvania). This year, we have also implemented weekly, resident-led, PBL sessions. The cases are designed to mirror the content of the weekly lecture and readings, following a patient from diagnosis, through the appropriate workup, to the surgical management, and finally the postoperative follow-up. Senior surgical residents moderate the sessions, and students are divided into small groups of about 10 students for these sessions. To encourage participation from all students, video features were enabled during sessions.
In addition to weekly quizzes administered based on the content of lectures and readings, students are assessed formally at the end of the rotation. A faculty member administers an oral examination, where a student must discuss the workup, treatment, and follow-up of a patient with a surgical disease. Finally, a standardized written National Board of Medical Examiners (NBME) examination is administered.
Due to the unforeseen circumstances of COVID-19 requiring students to remain home, the surgical rotation underwent alterations. Faculty-led lectures and PBL sessions were transitioned online using the Zoom (Zoom Video Communications, Inc., San Jose, California) platform to allow for discussion among participants.
Variables
Sociodemographics, student preparation for examinations, effectiveness of in-person and electronic sessions, effectiveness of faculty-led lecture and resident-led PBL sessions, and overall quality of these sessions were collected. After completion of the course and all required examinations, student grades, including weekly quizzes, oral examinations, and NBME examinations, were analyzed. These examination grades were also obtained for students from the 2018–2019 group.
Statistical Analysis
Descriptive statistics were calculated for sociodemographics and student preparation for examinations. A 10-point Likert scale was used to assess the effectiveness of in-person sessions and electronic sessions, as well as to assess the overall quality of these sessions. Student t-tests were used to compare in-person and electronic sessions, stratified by type of session (faculty-led lecture or resident-led PBL), and faculty-led and resident-led sessions, stratified by mode of delivery (in-person or electronic). Student t-tests were also used to compare weekly quiz, oral examination, and NBME examination scores. P values less than .05 were considered statistically significant. Statistical analysis was performed using SPSS version 26 (IBM Corporation, Armonk, New York, copyright 2019).
Additional Analysis
An additional group of students, whose educational content was completely remote, was given this survey to examine their responses. These students completed the PBL and lectures remotely, in the sixth block of the academic year, and were given the NBME shelf examination following this electronic content. They had the opportunity to return to the hospital for clinical experiences once it was safe for students to reenter the hospital. There were a total of 19 students on the total remote clerkship, and 14 of the 19 completed the survey. An identical ten-point Likert scale was used to assess the effectiveness of electronic sessions, as well as to assess the overall quality of these sessions. Finally, their NBME examination scores were compared to students in a similar block from the previous year.
A final group of students was collected as well. This group of students had completely in-person educational content prior to the COVID-19 pandemic. A total of 69 out of 79 students completed their surveys.
Results
Sociodeomgraphics and Preparation for Examinations
A total of 19 of 27 students completed the end-of-rotation survey. All 27 students completed their weekly quizzes, oral examination, and NBME examination. For those who completed the survey, the median age was 25 years (IQR: 22,28), and 9 (47.4%) were women (Table 1).
Table 1.
Student Demographics and Preparation for Examination Obtained From Postcourse Survey.
| All students (n = 19) n (%) | ||
|---|---|---|
| Student demographics | ||
| Age (median, 95% CI) | 25 (24–27) | |
| Gender | Female | 9 (47.4) |
| Male | 8 (42.1) | |
| Race | White | 9 (47.4) |
| black | 0(.0) | |
| Asian | 6 (31.6) | |
| Ethnicity | Hispanic | 3 (15.8) |
| Student preparation for examinations | ||
| Studying per week(mean, SD hrs) | 8.1 ± 4.0 | |
| Media used for studying for shelf exam | Textbook | 3 (15.8) |
| Online questions | 17 (89.5) | |
| NBME exam | 11 (57.9) | |
| Other | 3 (15.8) | |
| Media used for studying for oral exam | Textbook | 6 (31.2) |
| Online questions | 6 (31.2) | |
| NBME exam | 1 (5.3) | |
| Other | 10 (52.6) | |
Abbreviation: NBME, National Board of Medical Examiners.
In-Person Compared to Electronic Sessions
Perceptions of faculty-led lectures and resident-led PBL sessions were analyzed, and comparisons were made between in-person and electronic sessions. There was no perceived difference in faculty-led sessions when comparing in-person to electronic sessions for effectiveness of lectures to prepare for the NBME or oral examination (Table 2). There was also no difference in the overall perceived quality of the sessions.
Table 2.
Comparison of In-Person and Electronic Sessions, Stratified by Type of Session(Faculty or Resident-Led). In Addition, All In-Person and All Electronic Clerkship Feedback was Included.
| Half in-person, half-electronic group |
||||||
|---|---|---|---|---|---|---|
| Combined (n = 19) |
In-person sessions (n = 19 |
Electronic sessions (n = 19) |
All in-person (n = 69) |
All electronic (n = 14) |
||
| Mean ± SD | Mean ± SD | Mean ± SD | P | Mean ± SD | Mean ± SD | |
| Faculty-led lectures | ||||||
| Effectiveness to prepare for shelf exam | 6.4 ± 1.8 | 6.2 ± 1.7 | 6.7 ± 1.9 | .462 | 6.0 ± 2.1 | 6.9 ± 2.4 |
| Effectiveness to prepare for oral exam | 6.6 ± 1.8 | 6.4 ± 1.5 | 6.8 ± 2.0 | .581 | 6.2 ± 2.2 | 6.9 ± 3.1 |
| Overall quality | 7.3 ± 1.6 | 7.3 ± 1.4 | 7.3 ± 1.8 | 1.000 | — | 7.6 ± 1.8 |
| Resident-led problem based learning | ||||||
| Effectiveness to prepare for shelf exam | 7.2 ± 1.8 | 7.2 ± 2.1 | 7.2 ± 1.4 | .915 | 6.8 ± 2.1 | 8.4 ± 1.4 |
| Effectiveness to prepare for oral exam | 7.5 ± 1.2 | 7.4 ± 1.2 | 7.6 ± 1.2 | .737 | 7.9 ± 2.0 | 8.5 ± 1.5 |
| Overall quality | 8.4 ± 1.3 | 8.9 ± 1.0 | 7.9 ± 1.5 | .032 | — | 9.1 ± 1.1 |
Examining the resident-led PBL sessions, there was no perceived difference in effectiveness of in-person or online sessions to prepare for the NBME or oral examinations. There was, however, a greater perceived overall quality for in-person PBL sessions than electronic PBL sessions.
Resident-Led Compared to Faculty-Led Sessions
Results from in-person and electronic sessions were analyzed, and comparisons were made between faculty-led and resident-led sessions. Regarding the in-person sessions, students preferred resident-led PBL sessions to faculty lectures for their effectiveness to prepare them for NBME examinations, their effectiveness to prepare them for oral examinations, and their overall quality (Table 3).
Table 3.
Comparison of Resident-Led Problem-based Learning to Faculty Lectures, Stratified by Media of Delivery (In Person or Electronic).
| Combined (n = 19) |
Resident-led problem-based learning (n = 19) |
Faculty-led lectures (n = 19) |
||
|---|---|---|---|---|
| Mean ± SD | Mean ± SD | Mean ± SD | P | |
| In-person sessions | ||||
| Effectiveness to prepare for shelf exam | 6.7 ± 2.0 | 7.2 ± 2.1 | 6.2 ± 1.7 | .148 |
| Effectiveness to prepare for oral exam | 6.9 ± 1.4 | 7.4 ± 1.2 | 6.4 ± 1.5 | .036 |
| Overall quality | 8.1 ± 1.5 | 8.9 ± 1.0 | 7.3 ± 1.4 | <.001 |
| Electronic sessions | ||||
| Effectiveness to prepare for shelf exam | 6.9 ± 1.7 | 7.2 ± 1.4 | 6.7 ± 1.9 | .326 |
| Effectiveness to prepare for oral exam | 7.2 ± 1.7 | 7.6 ± 1.2 | 6.8 ± 2.0 | .173 |
| Overall quality | 7.6 ± 1.6 | 7.9 ± 1.5 | 7.3 ± 1.8 | .229 |
Regarding the electronic sessions, there was no perceived difference comparing resident-led PBL sessions to faculty lectures in their effectiveness to prepare them for NBME examinations, their effectiveness to prepare them for oral examinations, and their overall quality.
Outcomes on Examinations
Twenty seven students who completed the clerkship during this calendar year were compared to 24 students from the previous (2018–2019) academic year in the same course block. There were no differences in the students’ weekly quiz, oral examination, or NBME examination scores (Table 4) when comparing students on their rotations during the 2020 COVID-19 crisis to students from the previous 2018–2019 academic year in block 5.
Table 4.
Comparison of Examination Scores Between Historic Examinees (2019) to Current Examinees During the COVID-19 Crisis.
| Combined(n = 51) |
2019(n = 24) |
2020(n = 27) |
P | |
|---|---|---|---|---|
| Mean ± SD | Mean ± SD | Mean ± SD | ||
| Weekly quizzes | 86.5 ± 5.8 | 87.8 ± 5.5 | 85.3 ± 5.8 | .130 |
| Oral examination | 91.7 ± 8.0 | 93.9 ± 6.9 | 89.8 ± 8.6 | .068 |
| NBME examination | 76.3 ± 7.5 | 77.4 ± 8.1 | 75.3 ± 6.9 | .328 |
Abbreviation: NBME, National Board of Medical Examiners.
Additional Analyses
A total of 19 students completed the clerkship in a totally remote environment, and 14 completed the survey (Table 2). Comparing NBME scores of students from this totally remote learning to NBME scores from the same sixth block in 2018–2019, there was no statistically significant difference.
A total of 79 students completed the clerkship in a totally in-person environment (prior to the COVID-19 pandemic), and 69 completed the surveys (Table 2).
Discussion
This study has shown that third-year medical students on a surgical clerkship perceive the online modality of learning as equally effective compared with in-person content for preparing them for their examinations. Additionally, objective measures of performance, including quizzes, faculty-proctored oral examinations, and nationwide NBME examinations, did not show a significant decrease in performance when switching to an online mode of teaching. While many medical schools have moved to online educational content due to the COVID-19 crisis, very little is known regarding the validity of this method for medical students on clinical surgery rotations.
An aspect of the study worth noting is that students included had the opportunity to experience both modalities of teaching, in person and electronic. Although studies on the effectiveness of online lectures in clinical rotations are scarce, there is the literature, in addition to the present study, to support noninferiority of basic science content for medical student education.13 Other studies have pointed to potentially improved performance with online medical education,14 but there is little content on clinical education using an online format. It may be difficult to extrapolate these findings as current students have not also been exposed to in-person surgical education content, and this may represent a limitation of this study. In order to combat this limitation, however, we included data, in Table 2, showing the feedback of students prior to the pandemic (all in-person content) and after the pandemic had led to an all-electronic education for students. Possible discussions across different clinical disciplines may also aid in planning future curricula.
Although examination scores did not differ significantly compared to examination scores over the previous academic year, there are several potential confounding factors. The group of students discussed spent the final 4 weeks of their rotation at home. Their didactic and group session requirements were greater than the normal clerkship, but they may have had significantly greater time to study since they were no longer rotating in clinical settings. While they missed out on important educational clinical content potentially relevant to their examinations, they had greater time at home to study.
Another potential confounder in this study is that students had 4 weeks in the hospital before they were removed due to the spread of COVID-19. It is possible that students were able to ascertain all the clinically relevant information useful for their NBME examinations during this initial 4-week period. In order to address this, we also analyzed the subsequent group of students, who did not have any time in the hospital due to COVID-19, and compared their NBME grades to the same group of 2018–2019 students from the previous academic year. A total of 19 students were enrolled in this group spanning from April 13 to May 8, 2020. Comparing their scores to the previous year’s cohort, there was also no difference in their NBME examination scores. Although they did not have inpatient education, they did have increased time each day to study, which may have offset any decrease in their scores. While this again suggests that at-home electronic learning may be sufficient to prepare students for examinations, there are other confounding factors which may have affected study habits. More investigation for future groups will be needed.
While online lectures may be suitable for teaching didactic content, it is difficult to conceive that it can exclusively replace in-person learning. Medical students rely on others to collaborate and work with throughout their medical school careers.15 Additionally, technical skills also require in-person learning. On the surgical clerkship, for example, there are a variety of procedures students are expected to master by the end of rotations. We have required our students to view online content to become familiarized with these skills, but they will need to practice in order to become comfortable performing such tasks.
Not surprisingly, other medical schools have taken a similar approach to medical student surgical education during the coronavirus pandemic with one notable addition. Apart from a similar model to the one we have been using at our institution, one allopathic medical school in the northeast has expanded content to include a variety of surgical topics, including common plastic surgery procedures and the surgical management of burn victims. Additionally, they have attempted to address one of the pitfalls of an online program discussed previously, the absence of in-person exposure to procedures. This medical school aided students in obtaining suture materials and tools, subsequently holding a virtual suture workshop hosted by the faculty. With this, they attempted to provide some of the technical skills students would normally be exposed to on their surgical clerkship. Anecdotally, students found this to be an engaging and helpful session and were appreciative of the opportunity to experience some of what they were missing clinically. Several challenges of this approach include the availability of the materials, as well as the financial burden associated with this session. The supplies and shipping costs can make for an expensive activity depending on institutional finances.
A final point worth considering is quantifying the effect that at-home education will have on students in the future. Students may not pursue medical specialties where they did not have the opportunity to engage with faculty and patients in person. An interesting future study would be to identify the residency specialties our students pursue and if they had a decreased propensity for pursing specialties in which the clerkship was predominately remote. While this information was not collected initially, it would be worthwhile to gather these details prior to residency applications. An additional concern with a remote curriculum is the preparedness of our students for residency. Third-year clerkships not only help to shape our career selections but also help to impart the knowledge and skills to become an effective future resident and physician. With a large portion of clinical education delivered remotely, during the COVID-19 pandemic, students may be ill prepared to embark on postgraduate training. Additionally, future surgical trainees may have missed out on critical early development of surgical and technical skills (eg, suturing, knot tying, nasogastric tube placement, and urinary catheter insertion) due to their predominately remote clerkship. While these effects may not be felt for a few years, we must remain vigilant to ensure that graduating students are equipped to effectively care for patients throughout their careers.
The cessation of stay-at-home orders will no doubt occur in the near future, and we will be faced with an interesting decision. Medical school faculty and administrations will need to decide if it is best to return to the traditional format of in-person learning or if we will continue the online, web-based, learning activities (or perhaps a hybrid approach). It is reassuring, however, to see such promising results for electronic education. Students perceive that these modalities are as effective as in-person learning for preparing them for examinations, and their examination scores did not decrease as a result of the transition. This form of education should continue to be investigated but can potentially serve as an adjunct to traditional learning formats. Technology has helped to innovate many other aspects of our lives over the past decades, so it seems reasonable that it would find its way into the realm of surgical medical education as well.
Acknowledgments
Our thoughts are with the families of those who have succumbed to COVID-19. Thank you to the frontline health care workers who have worked tirelessly and heroically through this global pandemic.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the National Institute of Health under a training grant, T32CA211034.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article
References
- 1.CDC. What you need to know about coronavirus disease 2019 (COVID-19) United States Department of Health and Human Services. 2020. https://www.cdc.gov/coronavirus/2019-ncov/downloads/2019-ncov-factsheet.pdf. Published Accessed April 14, 2020. [Google Scholar]
- 2.Callaway E, Cyranoski D, Mallapaty S, Stoye E, Tollefson J. The coronavirus pandemic in five powerful charts. Nature. 2020;579(7800):482–483. [DOI] [PubMed] [Google Scholar]
- 3.CDC. 15 Days to slow the spread. White House. 2020. https://www.whitehouse.gov/articles/15-days-slow-spread/.Published Accessed April 14, 2020. [Google Scholar]
- 4.CDC. The president’s coronavirus guidelines for America. White House. 2020. https://www.whitehouse.gov/wpcontent/uploads/2020/03/03.16.20_coronavirus-guidance_8.5×11_315PM.pdf. Published Accessed April 14, 2020. [Google Scholar]
- 5.Jawaid A Protecting older adults during social distancing. Science. 2020;368(6487):145. [DOI] [PubMed] [Google Scholar]
- 6.AAMC. Guidance on Medical Students’ Participation in Direct In-person Patient Contact Activities. Washington, DC: Association of American Medical Colleges. 2020. https://www.aamc.org/system/files/2020-08/meded-August-14-Guidance-on-Medical-Students-on-Clinical-Rotations.pdf. Published Accessed August 18, 2020. [Google Scholar]
- 7.AAMC. Report on American Allopathic Medical School Curricula. Iowa City, IA: Association of American Medical Colleges. 2010. https://www.aamc.org/system/files/c/2/178392-iowareport_medicalschoolcurricula.pdf. Published Accessed April 29, 2020. [Google Scholar]
- 8.Burgess A, Kalman E, Haq I, Leaver A, Roberts C, Bleasel J. Interprofessional team-based learning (TBL): How do students engage?. BMC Med Educ. 2020;20(1):118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Chang BJ. Problem-based learning in medical school: A student’s perspective. Anna Med Surg. 2016;12:88–89. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Brockfeld T, Müller B, de Laffolie J. Video versus live lecture courses: A comparative evaluation of lecture types and results. Med Educ Online. 2018;23(1):1555434. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Alaagib NA, Musa OA, Saeed AM. Comparison of the effectiveness of lectures based on problems and traditional lectures in physiology teaching in Sudan. BMC Med Educ. 2019;19(1):365. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Carson R, Mennenga H. Team-based learning and the team-based learning student assessment instrument (TBL-SAI): A longitudinal study of master of occupational therapy students’ changing perceptions. Am J Occup Ther. 2019; 73(4):7304205010p1–7304205010p7. [DOI] [PubMed] [Google Scholar]
- 13.Eisen DB, Schupp CW, Isseroff RR, Ibrahimi OA, Ledo L, Armstrong AW. Does class attendance matter? Results from a second-year medical school dermatology cohort study. Int J Dermatol. 2015;54(7):807–816. [DOI] [PubMed] [Google Scholar]
- 14.Zhang X-M, Yu J-Y, Yang Y, Feng C-P, Lyu J, Xu S-L. A flipped classroom method based on a small private online course in physiology. Adv Physiol Educ. 2019;43(3): 345–349. [DOI] [PubMed] [Google Scholar]
- 15.McGeorge E, Coughlan C, Fawcett M, Klaber RE. Quality improvement education for medical students: A near-peer pilot study. BMC Med Educ. 2020;20(1):128. [DOI] [PMC free article] [PubMed] [Google Scholar]