Abstract
Background
During the coronavirus disease 2019 (COVID-19) pandemic, medical colleges in China had to use online teaching. This study explored the effect of COVID-19 knowledge learning online in a flipped classroom based on micro-learning combined with case-based learning (CBL).
Methods
There were 74 undergraduate medical students who were randomly grouped to an observation group and a control group with 37 participants in each virtual classroom on the Network Teaching Platform. Students learning in the control group utilized face-to-face lecture with PowerPoint pre-provided, while students learning in the observation group were conducted in a flipped classroom based on micro-learning combined with CBL. We compared the effect of both formats of COVID-19 knowledge learning online and the impact on clinical practice attitude in two groups.
Results
All 74 students (100%) responded pretest, posttest and retention test, and completed the questionnaire online. Both formats significantly improved COVID-19 knowledge acquisition at the conclusion of online COVID-19 curriculum. Students’ knowledge test scores including total score and scores of five knowledge dimensions of COVID-19 were significantly higher in the observation group than those in the control group (P<0.05). Compared with students in the control group, students in the observation group performed better in retention test and had a significantly more positive clinical practice attitude (P<0.05 in all items).
Conclusion
A flipped classroom based on micro-learning combined with CBL showed greater effectiveness in COVID-19 knowledge gain in undergraduate medical students and made their attitude toward clinical practice more positive.
Keywords: coronavirus disease 2019, online learning, flipped classroom, micro-learning, case-based learning
Background
In December 2019, cases of serious illness causing pneumonia and death were first reported in Wuhan, the capital of Hubei, China.1 The World Health Organization (WHO) announced the official name of the disease as “coronavirus disease 2019 (COVID-19)”. Affected by this COVID-19 pandemic, medical colleges in China had to shut down campus teaching to restrict COVID-19 spread. Alternative teaching methods were urgently on the agenda because traditional offline face-to-face lectures could not be implemented.
Previous studies in low- and middle-income countries show that utilizing e-learning can result in greater educational opportunities for students while simultaneously enhancing faculty effectiveness and efficiency.2 Beneficial effects have been found in empirical studies of the flipped classroom approach,3 micro-learning4 or case-based learning (CBL)5 in some health profession educations including medical education.
This study explored the online teaching method of a flipped classroom based on micro-learning combined with CBL on a virtual learning platform in undergraduate medical students acquiring COVID-19 knowledge using prospective randomized study.
Methods
Setting and Participants
In the academic year 2019–2020, we invited 74 undergraduate medical students who have finished their fourth grade courses coming to the clinical bedside practice in fifth grade to participate in this study. The average age of participants was 22.0 years, and 47.3% (35 of 74) were female. The study protocol was approved by the Institutional Ethical Committee for clinical research of Shanghai University of Medicine and Health Sciences affiliated Zhoupu Hospital, and all participants provided written informed consent prior to participation.
Design
This was a prospective, controlled educational research study. Using the random number table, participants were divided into an observation group or a control group with 37 participants in each group. Neither age nor sex differed between the two groups. Pretest scores suggest that there was no significant difference in baseline knowledge between the two groups (Table 1, P<0.05). Figure 1 displays our research design and interventions.
Table 1.
Knowledge Assessment Scores in Observation Classroom versus Control Classroom
| Pretest | Posttest | Retention Test | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Observation Classroom (n=37) | Control Classroom (n=37) | t value | P value | Observation Classroom (n=37) | Control Classroom (n=37) | t value | P value | Observation Classroom (n=37) | Control Classroom (n=37) | t value | P value | |
| Clinical manifestations (20), mean (SD) | 13.89 (2.37)c | 13.95 (2.30)d | 0.100 | 0.921 | 16.81 (1.33)a | 16.03 (1.66)a | 2.243 | 0.028 | 15.08 (1.34)b | 13.81 (1.33)b | 4.091 | <0.001 |
| Etiological detection methods (20), mean (SD) | 16.03 (2.23)c | 16.14 (2.14)c | 0.213 | 0.832 | 18.84 (1.34)a | 18.03 (1.66)a | 2.311 | 0.024 | 17.16 (1.46)b | 15.08 (1.67)b | 5.696 | <0.001 |
| Serological detection antibody titer characteristics (20), mean (SD) | 12.59 (2.57)c | 12.70 (2.58)c | 0.181 | 0.857 | 15.97 (2.62)a | 14.27 (1.35)a | 3.517 | 0.001 | 14.51 (3.07)b | 11.41 (1.14)b | 5.772 | <0.001 |
| Diagnostic criteria (20), mean (SD) | 12.32 (2.25)c | 12.08 (2.56)c | 0.434 | 0.666 | 17.05 (2.26)a | 14.84 (2.92)a | 3.651 | <0.001 | 15.00 (2.43)b | 12.86 (2.08)b | 4.060 | <0.001 |
| Clinical typing (20), mean (SD) | 12.24 (1.96)c | 11.97 (2.40)c | 0.530 | 0.597 | 17.54 (1.12)a | 16.62 (0.95)a | 3.800 | <0.001 | 14.27 (1.54)b | 14.00 (1.39)b | 0.792 | 0.431 |
| Total score (100) | 67.08 (10.70)c | 66.84 (11.08)e | 0.096 | 0.924 | 86.22 (5.74)a | 79.78 (7.12)a | 4.279 | <0.001 | 76.03 (6.49)b | 67.16 (5.05)b | 6.558 | <0.001 |
Notes: aCompared with pretest in the same group, P < 0.05. bCompared with posttest in the same group, P < 0.05. cCompared with retention test in the same group, P < 0.05. dCompared with retention test in the same group, P = 0.739. eCompared with retention test in the same group, P = 0.832.
Figure 1.
Study design and protocol.
Notes: Both observation group and control group utilized the Network Teaching Platform built and provided by Shanghai University of Medicine & Health Sciences to carry out a 120-minute online COVID-19 lesson. All 74 interns (100%) responded pretest, posttest and retention test, and completed the questionnaire online.
Abbreviation: PPT, PowerPoint.
Interventions
Both the observation group and the control group utilized the Network Teaching Platform built and provided by Shanghai University of Medicine & Health Sciences to carry out the research design respectively in two virtual classrooms. Educational content of our COVID-19 lesson was from “Program of Diagnosis and Treatment of COVID-19 (trial 7th edition)” issued by National Health Committee of the People’s Republic of China.6 The lesson for the observation group was set as a flipped classroom with micro-lessons combined with case-based learning guided by the teacher and discussion with questions and answers between students in a virtual classroom on the Network Teaching Platform. The lesson for the control group was set as a face-to-face traditional lecture utilizing lecture PowerPoint (PPT) slides in a virtual classroom on the Network Teaching Platform.
To maintain the consistency of the teaching content, a teaching team consisting of 3 clinical teachers was responsible for each format of a 120-minute online COVID-19 lesson. One teacher prepared all teaching materials for both formats. The other two teachers held the online lesson respectively at the same time during the intervention protocol. All teaching materials were peer-reviewed by all teachers, and the teaching team held a collective lesson preparation meeting before online instruction. Trial lessons for the observation group or the control group were approved at the collective lesson preparation meeting. The COVID-19 knowledge was decomposed into several sections, each of which focused on one or two foundational knowledge points and was made into micro-lesson videos. These micro-lesson videos were utilized as micro-learning materials provided to participants in the observation group in advance. In terms of knowledge coverage, the full content of traditional lectures was included. Teachers of the teaching team were only responsible for organizing teaching COVID-19 lessons, and they were not involved in research design and research.
One week before the online COVID-19 lesson, lecture PPT slides were uploaded to the virtual classroom for participants in the control group to preview, while micro-lesson videos were uploaded to the virtual classroom for participants in the observation group to self-study. Each participant logged into the virtual platform with a unique account password to enter his or her grouped classroom and could not exchange visits. The two groups of students study simultaneously in two different classrooms on the same virtual learning platform, with the same learning duration. The online COVID-19 lesson time length was set up as 120 minutes for both groups. The format of the control group was a 120-minute online face-to-face lecture given by a clinical teacher utilizing PPT slides, while the format of the observation group was micro-learning combined with CBL which contained a 60-minute discussion of typical clinical cases guided by a clinical teacher and a 60-minute interactive question session responded by students themselves.
A knowledge pretest was administered before classroom division, followed by a knowledge posttest immediately at the end of their online COVID-19 lesson, a questionnaire following the posttest and a retention test 2 months later. Pretest, posttest, retention test and questionnaire were issued at the same time in both grouped classrooms. Answers were required to be submitted within the same limited time on the Network Teaching Platform. The contents of the three tests and the questionnaire were designed by our research team, which were cut off from the teachers giving COVID-19 lessons.
Data Collection
We collected the demographic characteristics of students when the participants were invited. A 25-item multiple-choice knowledge test on five dimensions of COVID-19, consisting of clinical manifestations, etiological detection methods, serological detection antibody titer characteristics, diagnostic criteria and clinical typing, was developed to measure the knowledge benchmark (pretest), acquisition (posttest) and knowledge retention (retention test). The total score of the knowledge test was set as 100 and composed of five dimensions with 20 per dimension. For clarity, length and difficulty, we piloted the knowledge test on PGY-1 residents and faculty in primary care, respiratory and infectious disease who did not participate in the study. The same test with varied question order was utilized for maintaining consistency among pretest, posttest and retention test. A questionnaire utilized Yes or No form including willingness to acquire the knowledge of infectious diseases, practice hand hygiene, practice wearing self-protection and return to internship rotation was developed to assess participants’ attitudes toward clinical practice.
Outcomes and Analysis
We compared mean scores in pretest, posttest and retention test between observation group and control group using independent-samples t-test. We performed paired samples t-test for each dimension knowledge and overall outcome, comparing the change in scores from pretest to posttest, and from posttest to retention test in the same group. A χ2 test was used to compare attitudes toward clinical practice between the two groups.
Results
The response rate was 74 of 74 (100%) in all 3 knowledge tests and questionnaire.
Knowledge Acquisition and Retention
The mean total scores were 66.96, 83.00 and 71.59, respectively, in pretest, posttest and retention test. The mean total scores of the 3 knowledge tests were significantly different in pairwise comparison (all P < 0.001). For participants in the observation group classroom, their own mean total scores were both higher in posttest (86.22 versus 67.08; t = 12.310; P < 0.001) and in retention test (76.03 versus 67.08; t = 6.088; P < 0.001), respectively, compared to those in pretest, and in posttest differed from that in retention test (86.22 versus 76.03; t = 20.297; P < 0.001) in the paired analysis. For participants in the control group, as depicted in Figure 2, their mean total score had similar trend changes from pretest to posttest (66.84 versus 79.78; t = 8.245; P < 0.001) and from posttest to retention test (79.78 versus 67.16; t = 18.889; P < 0.001) compared to those changes in the observation group, but were not significantly different between their own pretest and retention test (66.84 versus 67.16; t = 0.213; P = 0.832).
Figure 2.
Knowledge acquisition and retention.
Notes: Group mean total scores on knowledge assessment test at baseline (pretest), immediately after the online COVID-19 lesson (posttest) and 2 months later (retention test). The change in scores between baseline and immediately after the lesson represents knowledge acquisition and is significantly better in the observation classroom. The change in scores from immediately after the intervention until end of 2-month follow-up represents knowledge retention and is significantly better in the observation classroom.
Mean total score (66.84 versus 67.08; t = 0.096; P = 0.924; Figure 2) and each dimension knowledge score (all P ≥ 0.05; Table 1) in pretest were not significantly different between the two groups. The observation group performed significantly better than the control group for both posttest and retention test for either mean total score (all P < 0.001; Figure 2) or five dimensions scores (all P < 0.05; Table 1).
Attitudes Toward Clinical Practice
Compared with participants in the control group, participants in the observation group had higher willingness to acquire the knowledge of infectious diseases, practice hand hygiene, practice wearing self-protection and return to internship rotation (all P < 0.05; Table 2).
Table 2.
Intern Attitudes Toward Clinical Practice in Flipped Classroom versus Standard Classroom
| Item | Flipped Classroom n (Yes%) | Standard Classroom n (Yes%) | χ2 Value | P value |
|---|---|---|---|---|
| Willingness to learn the knowledge of infectious diseases | 28 (65.12) | 15 (34.88) | 9.382 | 0.004 |
| Willingness to practice hand hygiene | 22 (68.75) | 10 (31.25) | 7.929 | 0.009 |
| Willingness to practice wearing self-protection | 24 (63.16) | 14 (36.84) | 5.409 | 0.036 |
| Willingness to return to internship rotation | 28 (66.67) | 14 (33.33) | 10.792 | 0.002 |
Discussion
To our knowledge, this is the first report of a prospective educational research study in a rotation-suspended setting comparing the effects of a mixed learning approach to a traditional lecture format, with regard to knowledge change on COVID-19 and attitude toward clinical practice. We found all students had significant improvement in knowledge acquisition and retention compared with their knowledge benchmark; both mean total score and five dimensions in the mixed learning approach were higher than those in the face-to-face group, and there was no significant difference between the knowledge benchmark and retention within the face-to-face group. This indicates that although both face-to-face and mixed learning approaches may improve the knowledge acquisition, the latter could make the students better understand and consolidate the new disease knowledge. It might be based on better understanding of the new disease knowledge that students in the mixed learning group showed more positive practice attitude and better prepared them for clinical practice, such as showing higher willingness to acquire the knowledge of infectious diseases, practice hand hygiene, practice wearing self-protection and return to internship rotation. It is suggested that this mixed learning approach could improve the learning initiative of students in clinical practice, and thus help to indirectly improve the level of clinical practice in the future.
With the popularization and development of internet and computer technology, it is feasible and realistic to utilize this mixed learning approach. In micro-learning, the content for the learners is short and concise, encouraging them to use the fragmentation time to study independently. Recently, a review explored how micro-learning can be used to increase retention in learners on mobile applications.7 In CBL, typical cases engage learners in active learning using course concepts to solve important problems.8 Studies have shown CBL is a teaching tool used in a variety of medical fields using human cases to impart relevance and aid in connecting theory to practice.9 The impact of CBL can reach from simple knowledge gains to changing patient care outcomes.5 Jhala and Mathur found CBL was useful for deep learning and emphasized its role in lifelong medical learning.10 In the flipped classroom, the roles of teachers and learners are exchanged, learners completing self-study before lesson start, and classroom time is used for interactive learning and problem solving. Findings from a meta-analysis including six studies with only one prospective randomized study11 suggest that the flipped classroom may be associated with minimal gains in student knowledge compared to a lecture.12 There are several practical problems associated with flipped classroom, particularly prominent of which is more faculty time, faculty development or resources to ensure content to be effectively delivered.13–15 We finished the video recording in the lecture preparation stage, and according to the knowledge units decomposed into micro-lesson videos. Generally, clinical teachers collect typical clinical cases conveniently and have the ability to guide classroom discussions on clinical cases. Therefore, we did not spend more faculty time, including teacher development and preparing learning resources in our study. In other words, this mixed learning approach might be utilized at minimal cost.
The students’ clinical experience was insufficient, and the bedside learning could not carry on during the rotation-suspended period, so it was inevitably difficult to achieve the ideal effect with a traditional lecture. The original intention of our study was to provide a feasible way for clinical teachers to teach students gaining new clinical knowledge effectively during a special period. A flipped classroom based on micro-learning combined with CBL might form a resultant force in our study, promoting the effect of COVID-19 knowledge acquisition and retention.
This study has limitations. It was a single-center study, and was assessed only in undergraduate medical students in grade 4 with a highly compact topic, which limits generalizability. As a mixed learning approach, our research results do not delineate whether the main effect of this approach derived from flipped classroom, micro-learning or CBL. Additionally, both control and observation formats and all evaluations in our study were entirely online, which needs network communication and technology support.
The same teacher prepared all teaching materials, which was peer-reviewed by all teachers in the teaching team, for two different formats, and conducted the collective lesson preparation. Teachers of the teaching team were not researchers in this study, which made the knowledge assessment scores in pretest, posttest and retention test more credible. We might be able to ensure the same and full coverage of learning knowledge points in both classrooms, but we could not ensure that the same content was covered and that materials were totally the same. For example, it was impossible that the clinical cases in the mixed learning classroom were fully covered in the traditional face-to-face online lecture. This might cause bias of the result.
Further research should investigate whether the positive effect can be replicated in other medicine lessons, including offline or online combined with offline lectures on a broader scale and explore whether the main effect comes from the flipped classroom, micro-learning or CBL by detailed cohorts study and preference survey. More study would be useful to understand details of interaction and enhancement. In future studies, we will expand the depth and breadth of research, including increasing the number of learning sessions. In the future, we will refine the separate role of different learning techniques in network teaching.
Conclusion
A flipped classroom based on micro-learning combined with CBL showed greater effectiveness in COVID-19 knowledge acquisition and retention without sacrificing more faculty time or development, and made the attitude toward clinical practice more positive in undergraduate medical students. This mixed learning approach could be recommended to use as a teaching strategy to acquire knowledge of new diseases online during special periods.
Funding Statement
This study was supported by grants from Shanghai University of Medicine and Health Science (HMSF-17-22-036, ZPJXJS-SJY-06) and Pudong Health Bureau of Shanghai (PWRd2019-03).
Disclosure
The authors report no conflicts of interest in this work.
References
- 1.Li Q, Guan X, Wu P, et al. Early transmission dynamics in Wuhan, China, of novel coronavirus-infected pneumonia. N Engl J Med. 2020;382(13):1199‐1207. doi: 10.1056/NEJMoa2001316 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Frehywot S, Vovides Y, Talib Z, et al. E-learning in medical education in resource constrained low- and middle-income countries. Hum Resour Health. 2013;11(1):4. doi: 10.1186/1478-4491-11-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Hew KF, Lo CK. Flipped classroom improves student learning in health professions education: a meta-analysis. BMC Med Educ. 2018;18(1):38. doi: 10.1186/s12909-018-1144-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Isba R. When I say … micro learning environment. Med Educ. 2015;49(9):859‐860. doi: 10.1111/medu.12736 [DOI] [PubMed] [Google Scholar]
- 5.McLean SF. Case-Based learning and its application in medical and health-care fields: a review of worldwide literature. J Med Educ Curric Dev. 2016;3:JMECD.S20377. doi: 10.4137/JMECD.S20377 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Office of National Health Committee, Office of National Administration of Traditional Chinese Medicine. Diagnosis and treatment of COVID-19 (trial 7th edition). Available from: http://www.nhc.gov.cn/yzygj/s7653p/202003/46c9294a7dfe4cef80dc7f5912eb1989/files/ce3e6945832a438eaae415350a8ce964.pdf. Accessed July21, 2021.
- 7.Shail MS. Using micro-learning on mobile applications to increase knowledge retention and work performance: a review of literature. Cureus. 2019;11(8):e5307. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Crowe S, Cresswell K, Robertson A, et al. The case study approach. BMC Med Res Methodol. 2011;11(1):100. doi: 10.1186/1471-2288-11-100 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Greenberg-Worisek AJ, Campbell KA, Klee EW, et al. Case-based learning in translational biomedical research education: providing realistic and adaptive skills for early-career scientists. Acad Med. 2019;94(2):213‐216. doi: 10.1097/ACM.0000000000002470 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Jhala M, Mathur J. The association between deep learning approach and case based learning. BMC Med Educ. 2019;19(1):106. doi: 10.1186/s12909-019-1516-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Anderson HG Jr, Frazier L, Anderson SL, et al. Comparison of pharmaceutical calculations learning outcomes achieved within a traditional lecture or flipped classroom andragogy. Am J Pharm Educ. 2017;81(4):70. doi: 10.5688/ajpe81470 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Gillette C, Rudolph M, Kimble C, et al. A meta-analysis of outcomes comparing flipped classroom and lecture. Am J Pharm Educ. 2018;82(5):6898. doi: 10.5688/ajpe6898 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.McLaughlin JE, Roth MT, Glatt DM, et al. The flipped classroom: acourse redesign to foster learning and engagement in a health professions school. Acad Med. 2014;89(2):236–243. doi: 10.1097/ACM.0000000000000086 [DOI] [PubMed] [Google Scholar]
- 14.Koo CL, Demps EL, Farris C, et al. Impact of flipped classroom design on student performance and perceptions in a pharmacotherapy course. Am J Pharm Educ. 2016;80(2):Article 33. doi: 10.5688/ajpe80233 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Graham KL, Cohen A, Reynolds EE, et al. Effect of a flipped classroom on knowledge acquisition and retention in an internal medicine residency program. J Grad Med Educ. 2019;11(1):92‐97. doi: 10.4300/JGME-D-18-00536.1 [DOI] [PMC free article] [PubMed] [Google Scholar]