Combing pre-workshop, web-based learning and hands-on workshop as a flipped classroom clinical skill training model during the COVID-19 pandemic

Abstract

Objective

Due to the COVID-19 pandemic, there is a dramatic drop in in-person residency training due to the implementation of isolation and triage measurements. Here, we propose a new flipped classroom clinical skill training model utilizing a combination of pre-workshop, web-based learning and hands-on workshop that shortens in-person contact time to maintain residency training during the pandemic. Effectiveness of this training model was evaluated with a pre-test and post-test skills competency assessment, two-way feedback, and a five-point Likert scale structured survey questionnaire.

Materials and methods

The workshop was conducted in a flipped classroom fashion by the obstetrics and gynecology (OBGYN) department of a single tertiary teaching medical center covering topics from five OBGYN subspecialities. Every topic consisted of a pre-workshop, web-based, mini lecture (PWML) followed by a hands-on workshop (HW). All first to fourth year OBGYN residents were invited to attend the workshop. All the trainees were required to complete the PWML prior to the day of HW. The workshop consisted of rotational station for each topic and was conducted within one afternoon. A 0-100-point scale pre-test and post-test skills competency evaluation were performed for each station and was assessed by the attending doctor or fellow doctor of each subspeciality. Two-way feedback was done after the post-test evaluation. A five-point Likert scale structured survey instrument consisting of participant’s perceptions of the workshop design, relevance to clinical practice, and quality of instructors and materials was created during the curriculum development process and sent to each participant one month after the workshop.

Results

A total of 19 residents including five first-year, six second-year, three third-year, and five fourth-year residents completed the entire pre-workshop lecture, workshop, pre-test, and post-test. For all residents, the average post-test score of 5 stations was 95 and was significantly higher than the pre-test score of 60 (p < 0.001). For both junior residents and senior residents, the average post-test scores of 5 stations were also significantly higher than pre-test scores (p < 0.001). Survey generated one month after the workshop showed a high overall satisfaction with the workshop instructors on their professional knowledge, communication skills, and interactions between the instructors and trainees. The average satisfaction scores for manipulation of vaginal breech delivery (VBD), semen analysis (SA), cervical conization (CC), obstetrics anal sphincter injury (OASIS), and laparoscopic suture techniques (LST) were 4.84, 4.96, 4.92, 4.88, and 4.92, respectively The average score for practical application of the training materials, class design and teaching method, overall satisfaction of the session, and time scheduling was 4.84, 4.96, 4.96, and 4.48, respectively. The entire HW was completed within 180 min and was carried out within half a day.

Conclusion

With the implementation of isolation and triage measures in the COVID-19 pandemic, there is a dramatic drop in in-person exposure to all aspects of the residency training, in particular, non-emergent surgeries. Utilization of PWML saved 1/3 of in-person time and the entire workshop was completed within 180 min that could be carried out within half a day. The decrease of person-to-person contact time during the COVID-19 pandemic is necessary while still providing curriculum-based residency training in spite of decreased hands-on experience.

Introduction

Since the first establishment of surgical residency in Johns Hopkins Hospital by Dr William Stewart Halsted in 1890, the teaching methods of “see one, do one, teach one” has not changed [1]. However, this traditional teaching methods has flaws in the modern world. As the population ages, patient comorbidities and complications are more complicated. The expending new knowledge and treatment guidelines changes exponentially. Patient expectations, increase in standard of care, and paralegal issues make it harder for residents to operate primarily even under supervision. At the same time, new conservative treatment methods causing decrease in surgical volume and restricted residency working hours reduce opportunities for residents to master the necessary surgical techniques [2]. This is further impacted by the COVID-19 pandemic with the implementation of isolation and triage measurements [3].

The fundamental purpose of residency training is to equip a surgeon with the knowledge and skills required to retool and adopt throughout the aftercoming 30-year career [4]. It is vital that physicians and surgeons alike will be able to serve the population as medicine evolves. The basis of surgical training consists of (1) understanding of the scientific basis of a surgical disease, (2) acquiring skills in patient management and, (3) the technical operations in an environment with increasing complexity, enhanced responsibility, and independence [4], which are affected by shortening of work hours.

The establishment of curriculum-based skills training workshop can overcome the lack of experience. This allows all residents to reach approximately the same threshold and limits variability and uncertainty of skills acquisition [5].

In flipped classroom, students prepare in advance prior to person-to-person meeting by learning content independently through online learning modules, textbooks, or journal articles [6]. The didactic material in the form of pre-recorded video lecture, per se, can be accessed remotely at any time, and students can cover the materials on their own pace promoting active, deeper learning, longer retention, and life-long learning skills [3,6]. The in-class, in-person, student-centered, peer–faculty interactions allows for application and critical thinking, allowing integration of new knowledge with existing knowledge [6].

The current study is to report a new design of workshop for resident’s clinical skill training and its effect. To shorten the person-to-person contact time of workshop, lectures of each topic were given in a pre-class flipped classroom method followed by a hands-on workshop that shortens in-person contact time in compliance to the isolation and triage measurements implemented during the COVID-19 pandemic. The primary objective of the current study is to evaluate the effectiveness of this new type of hands-on workshop through a pre-test and post-test skills competency assessment comparison. The secondary objective is to evaluate the trainee’s satisfactory of combining a pre-workshop, web-based, mini lecture (PWML) with hands-on workshop (HW) with a two-direction direct feedback, and a five-point Likert scale structured survey questionnaire.

Materials and methods

The workshop was conducted in a flipped classroom fashion by the obstetrics and gynecology (OBGYN) department of a single tertiary teaching medical center. The workshop covered topics from five OBGYN subspecialties including maternal–fetal medicine (MFM), endocrinology and reproductive medicine (ERM), gynecologic oncology (GO), female pelvic medicine and reconstructive surgery (FPMRS), and minimal invasive surgery (MIS).

The topics chosen were from the clinical skill aspects of our residency program milestones to be completed prior to graduation, including manipulation of vaginal breech delivery (VBD), semen analysis (SA), cervical conization (CC), primary repair of obstetrics anal sphincter injury (OASIS), and laparoscopic suture techniques (LST). With the flipped classroom method, every topic consisted of a PWML followed by a HW. The trainees were asked to complete the PWML prior to the HW to provoke learner-centered learning and reduce in-person contact during the pandemic. Pre-test and post-test evaluation of information and skills covered in the PWML and HW was assessed by the instructors composed of attending doctor or fellow doctor of each subspeciality. The contents covered in each topic by the PWML, HW, and evaluation are listed in Table 1 . At the end of the HW, there was a two-way feedback session. A structured survey instrument was sent to each participant and generated anonymously one month after the workshop.

Table 1.

Contents of PWML, HW, pre- & post-test evaluation of each topic.

Topics	PWML	HW	Pre- & post-test evaluation
VBD	▪Classifications of breech presentation ▪Identification of fetal presentation ▪Route of delivery decision-making ▪Breech delivery extraction	▪Breech delivery extraction on a pregnant mannequin	▪State contraindications for VBD ▪Perform modified Praque maneuver ▪Perform Mauriceau maneuver ▪State managements for entrapment of aftercoming head
SA	▪Indications for semen analysis ▪Overview of spermatogenesis ▪Sample collection guideline ▪Semen morphology nomenclature ▪Semen anaylsis parameters	▪Process of semen analysis ▪Preparation of semen ▪Macroscopic analysis ▪Microscopic analysis ▪(Performed on porcine semen)	▪Ability to perform macroscopic examinations ▪Operate the microscope ▪Identify semen morphology ▪Diagnosis of abnormal semen analysis
CC	▪Anatomy of the cervix ▪Comparison of conization methods ▪Operative technique ▪Postoperative care ▪Related complications	▪Loop electrosurgical excision on hotdogs mimicking cervix placed in pelvic mannequin	▪Pap smear interpretations ▪Conization indications ▪Conization technique ▪Postoperative care follow-up
OASIS	▪Anorectal anatomy ▪Identification of laceration ▪Clinical significance and complications associated with OASIS ▪Updated current recommendations for repair of OASIS ▪Step-by-step guide to surgical repair	▪Video demonstration of 4th degree laceration repair ▪Hands-on repair of porcine anorectal fourth-degree laceration	▪Identification of anal mucosa ▪Identification of internal sphincter ▪Identification of external sphincter ▪Completion of OASIS repairment within the allocated time
LST	▪Video and tips on needle holding ▪Video and tips on needle manipulation ▪Video and tips on suture looping ▪Video and tips on knot typing	▪Needle holding ▪Needle manipulation ▪Suture looping ▪Knot typing ▪(Performed on laparoscopic training boxes)	▪Ability to perform all four maneuvers within a time frame

PWML: pre-workshop, web-based, mini lecture, HW: hands-on workshop, VBD: manipulation of vaginal breech delivery, SA: semen analysis, CC: cervical conization, OASIS: primary repair of obstetrics anal sphincter injury, LST: laparoscopic suture techniques.

All first to fourth year OBGYN residents, including those on duty were invited to attend the workshop. Junior residents are defined as first- and second-year residents, and senior residents composed of third- and fourth-year residents. All the trainees were required to complete the PWML prior to the day of HW. This was confirmed with attendance and study time record through the hospital’s online teaching system. The PWML consisted of 5–10 slides from each subspecialty that were accessible online. Each PWML required approximately 15 min if given as lectures. In this learner-centered, flipped classroom model, lectures were given beforehand, allowing time during the HW for direct feedback and Q&A for hands-on simulation techniques.

The HW consisted of rotational station for each topic and was conducted within one afternoon. The instructors of each station were attending doctors and fellow doctors from each subspecialty. Trainees were divided into two large rotations and five groups. Rotation A consists of thirty minutes of MRM, ERM, and GO stations each. Rotation B consists of forty-five minutes of FPMRS and MIS stations each. After ninety minutes, the two rotations were switched (Fig. 1 ).

Fig. 1.

Rotation of hands-on workshop. MRM: maternal fetal medicine, ERM: endocrine and reproductive medicine, GO: gynecologic oncology, FPMRS: female pelvic medicine and reconstructive surgery, MIS: minimal invasive surgery.

The pre-test and post-test evaluations were performed on a scale of 0–100, accordingly to the trainee’s performance for each station. Each station’s evaluation consisted of four scoring items and were assessed by the single instructor. The scoring items for the pre-test and post-test were the same for each station. The pre-test was performed in the beginning of the HW, and post-test evaluated after each trainee had actual hands-on practice. An overall average score of the entire workshop was also calculated. Direct feedback was given after the post-test evaluation. Before the end of the workshop, a two-way feedback was conducted. A structured survey instrument consisting of participant’s perceptions of the workshop design, relevance to clinical practice, and quality of instructors and materials was created during the curriculum development process. The satisfaction questionnaire on the instructors included professional knowledge, communication skills, and interaction between the instructors and trainees. The satisfaction questionnaire on the training course covered the practical application of training materials, class design and teaching methods, time scheduling, and overall satisfaction with the entire course and each rotation. The five-point Likert score (1: strongly disagree, 2: disagree, 3: neutral, 4: agree, 5: strongly agree) was used for the questionnaires. The survey questions and feedbacks were sent to each participant and generated anonymously one month after the workshop.

This study (IRB No. 202100320B) has been reviewed and exempted from Chang Gung Medical Foundation Institutional Review Board review according to the “Educational assessments or tests, or evaluation of teaching skills or efficacy in a general teaching environment” and the regulation of Ministry of Health and Welfare, Taiwan. The HW was funded by the Medical Education Department of Linkou Chang Gung Memorial Hospital.

Statistical analysis

Demographics and peri-operative data were reported using descriptive statistics. Paired-samples t test was applied for comparison of pre- and post-testing score continuous data. A p-value of <0.05 was considered statistically significant. All statistical methods were performed using commercial software Microsoft® Excel, version 16.49 (© 2021 Microsoft).

Results

A total of 31 residents, 17 junior residents and 14 senior residents, were invited to attend the workshop. Six residents were unable to attend the workshop, and another six residents attended part of the workshop due to emergent clinical situations. A total of 19 residents including five first-year, six second-year, three third-year, and five fourth-year residents completed the entire pre-workshop lecture, hands-on workshop, pre-test, and post-test. All available scores of these 19 residents were analyzed for the efficacy of training course.

For all residents, the average post-test score of 5 stations was 95 and was significantly higher than the pre-test score of 60 (p < 0.001). For both junior residents and senior residents, the average post-test scores of 5 stations were also significantly higher than pre-test scores (p < 0.001) as shown in Table 2 .

Table 2.

The pre-test and post-test scores of junior and senior residents.

	Junior resident			Senior resident
	Pre-test	Post-test	P-value	Pre-test	Post-test	P-value
VBD	59 ± 15.4 (49.5–67.7)	88 ± 5.3 (85–91.4)	<0.001	61 ± 10.1 (53.6–67.6)	97 ± 5.6 (93.1–100)	<0.001
SA	67 ± 15.5 (58.2–76.4)	93 ± 9.6 (87–98.4)	<0.001	61 ± 3.3 (59–63.6)	98 ± 6.6 (92.9–100)	<0.001
CC	66 ± 10.8 (59.1–71.9)	99 ± 3.1 (96.8–100)	<0.001	78 ± 15.8 (67.2–89)	98 ± 6.6 (92.9–100)	0.005
OASIS	42 ± 9.6 (36.3–47.7)	100 ± 0 (a)	<0.001	44 ± 11.6 (36–52)	100 ± 0 (a)	<0.001
LST	49 ± 18.2 (37.8–59.4)	81 ± 14.3 (72.5–89.3)	<0.001	83 ± 7.9 (77–88)	99 ± 1.7 (98.3–100)	<0.001
Average	57 ± 7.3 (52.2–60.8)	92 ± 3.1 (90.3–93.9)	<0.001	66 ± 6.9 (60.8–70.4)	98 ± 2.5 (96.6–100)	<0.001

VBD: manipulation of vaginal breech delivery, SA: semen analysis, CC: cervical conization, OASIS: primary repair of obstetrics anal sphincter injury, LST: laparoscopic suture techniques.

Data are listed as mean ± standard deviation with 95% CI in parentheses.

Paired t-test.

^a95% CI not available due to standard deviation is zero.

Twenty-five (14 junior and 11 senior) residents responded to the structured survey one month after the workshop to reflect the benefits of their clinical practice. The response rate was 100% for those who attended the hands-on workshop. Fig. 2 showed a high overall satisfaction score with the workshop instructors on their professional knowledge, communication skills, and interactions between the instructors and trainees in a 5-point Likert scale. All residents expressed that they would re-invite the same instructors for future workshops. The average satisfaction scores for VBD, SA, CC, OASIS,and LST were 4.84, 4.96, 4.92, 4.88, and 4.92, respectively (Fig. 3 ). The average score for practical application of the training materials, class design and teaching method, overall satisfaction of the session, and time scheduling was 4.84, 4.96, 4.96, and 4.48, respectively.

Fig. 2.

Overall mean satisfaction with the instructors, n = 25.

Fig. 3.

Overall mean satisfaction with the course, n = 25. VBD: vaginal breech delivery, SA: semen analysis, CC: cervical conization, OASIS: obstetrics anal sphincter injury, LST: laparoscopic suture techniques, PA: practical application of training materials, CD: class design and teaching methods, OS: overall satisfaction, AT: adequate time allocation.

In the feedback, residents hoped they had longer time to practice. Eighty percent of residents (20/25) felt the course content were highly correlated with the topics, and 20% (5/25) felt the course content were moderately correlated with the topics. When being asked if the lesson learnt can be applied clinically in the future, nineteen residents (76%) responded definitely and six residents (24%) responded equivocal. In a multiple-choice question, 17, 4, 9, 12, and 20 residents responded that the VBD, SA, CC, OASIS,and LST rotations, respectively, were helpful with their clinical work.

The entire HW was completed within 180 min and could be carried out within half a day. Without the PWML, the workshop would have required at least 225 min (15 min per lecture) and needed a whole day to conduct. The design of PWML saves 1/3 of in-person time which is more feasible in a busy teaching hospital. The decrease of person-to-person contact time during the COVID-19 pandemic is necessary. All trainers and trainees wore facial masks during the entire activity.

Discussion

The change in generation shift of today’s residents for the desire of a work-life balance should be taken into consideration. Increase in family commitments, two professional parents, and societal changes have significant impacts on training and the mentality of residents [7]. The Accreditation Council for Graduated Medical Education implemented rules limiting working hours for all residents in 2003. After its enforcement in Taiwan 2019, the restricted weekly duty hours is a concern on the adequacy of residency training, especially clinical skills.

The alarming decrease in hands-on experience during residency makes it difficult for trainees to acquire skills required to become a competent surgeon [8]. It is challenging to overcome with less learning time, decreased surgical volume, and greater learning curve for increased complexity of surgical techniques. Furthermore, clinical, operative settings should not be a place for residents to acquire new skills. Newer models of residency training emphasis on curriculum-based training with competency-based advancements. This also includes skills acquisition and assessment in nonclinical environments. HW is an effective clinical skill training for residents.

The learning of a surgery requires the following steps: acquisition of technical skills such as suturing techniques and knot tying, knowledge of surgical field anatomy, steps of the surgery, efficiency of motions, and recognition and management of unanticipated findings and complications [4]. All these steps can and should be learnt and assessed objectively in a nonclinical environment first. Use of HW allows familiarity and development of dexterity of instrumentation. Acquisition of surgical skills such as minimal invasive laparoscopy outside the operating room has shown to shorten the learning curve [9]. With the aid of simulation, fundamental principles of procedural techniques and operative steps can avoid naive residents acquiring basic skills in the operating room that puts patients at risk. Deliberate practice of a well-defined task followed by an immediate detailed feedback on performance and with ample opportunities to practice repeatedly generates expert performance [10]. Lenchus et al. found that the utilization of procedural instruction curriculum allows residents to perform their first procedure on patients such as central venous placement with more confidence and skills [11]. The utilization of workshop with hands-on simulation is an important teaching modality for procedural based learning that allows the learning curve to shift to the left. In the current study combing HW and immediate feedback, the post-test scores were significantly better than pre-test scores in all grade of residents. In a given rotation, senior residents outperformed junior residents. This is consistent with clinical findings where senior residents have had much more opportunity and experience with surgical techniques.

With the implementation of isolation and triage measures in the COVID-19 pandemic, there is a dramatic drop in in-person exposure to all aspects of the residency training, in particular, non-emergent surgeries [3]. By taking advantage of technology and by modifying delivery of teaching methods, residency training can be maintained. Shortening the time of HW is important since it is a privileged time counted towards work hours and allows less person-to-person contact during the COVID-19 pandemic. Each topic’s lecture was designed to correspond to a 15-min teaching. The utilization of PWML saves 1/3 of the in-person training time.

To fully maximize time efficacy and promote active learning, the use of flipped classroom teaching modality has become popular in recent years in not only medical, but also in postgraduate medical education [12]. In Stanford Medical University, the attendance rate in core biochemistry course rose from 30% to 80% when the class with flipped [13]. Students learnt through online conditions performed better than those learnt through traditional lecture model with greater effects if the online learning was reenforced with in-person instructions [14]. Additionally, Mayo Clinic found residents to demonstrate improved quality knowledge and the in-class application sessions were valued [12]. Our design combining pre-workshop flipped classroom and hands-on workshop showed good results of the evaluation scores and high trainee satisfaction.

The use of HW and flipped classroom modules is not without fault. Utilization of flipped classroom modules alone is insufficient for clinical skills acquisition without hands-on training [15]. Validated studies with assessment tools that address skills acquisition from simulation, transfer of the skills to an operating table, and retention of skills over time are still lacking [16].

Online learning and tracking allows more flexibility and efficient learning [7]. Resident’s competency can be evaluated and ensures certain benchmarked are met. The utilization of pre-workshop online learning and hands-on workshop has been success in our experience. All residents responded that they are willing to attending similar workshop in the future. They have found the lessons learnt in the workshop can be applied clinically. However, the trainees wished there were longer time allocated to each rotation and that the topics can be stratified accordingly to the different grade of residency. The limitation of our study includes lack of validation for our pre-test and post-test evaluation and structured survey questionnaires, small case numbers, lack of tailored topics for different levels of residency (junior versus senior), and direct teaching method comparison to the traditional “see one, do one, teach one” learning. Although we were unable to compare the effect of PWML on HW outcome directly, it is still essential as it decreased the total HW time by 1/3.

In conclusion, combing PWML and HW as a clinical skill training model is time efficient, effective, and has a high resident satisfaction that can provide an alternative way of delivering teaching methods during the pandemic. Not only that, it can also be utilized in the future as the standards of residency training shift towards a curriculum-based, competency-based advancements program including simulation-based skills training. A larger scale of training items and validated evaluations will be constructed in the further practice.

Declaration of competing interest

The authors have no conflicts of interest to disclose.