Abstract
Aim
Evaluate the fully online flipped classroom's effects during the pandemic.
Design
A comparative descriptive study with historical control design.
Methods
In an internal medicine nursing course, the fully online flipped classroom instruction was used with 53 undergraduates in 2020. Their final examinations were compared with the 50 students taught with traditional offline methods in 2019. Online interactions and students' experiences were described. Pass rates in both classes were over 90% (Χ 2 = 0.276, p = 0.60), but the median score in 2019 was higher than in 2020 (Z = −2.491, p = 0.01). There were 996 online interactions and 734 valid interactions in total. All 49 students believed the online flipped classroom schedule was reasonable and all but three said it was helpful. However, 19 students (39%) felt traditional teaching is more effective.
Conclusions
The fully online flipped classroom method was fairly effective during the pandemic. This model also did increase class participation and sufficient faculty–student interactions in remote education. However, fewer students earned outstanding scores, with possible reasons including the online flipped classroom, lack of clinical practice, stress from COVID‐19 and the shortened exam time. Overall, the method is worth recommending under public health emergencies like COVID‐19, and future research exploring potential concerns about scores is necessary.
Keywords: COVID‐19, flipped classroom, nursing education, online teaching, teaching effectiveness
1. INTRODUCTION
The highly infectious coronavirus disease‐2019 (COVID‐19) has become a public health event affecting the world. For nursing students, the pandemic has not only affected students' normal in‐class learning and clinical placements but also has had a significant impact on students' emotions and well‐being (Rasmussen et al., 2022). Some studies indicated that nursing students experienced uncertainty and anxiety during the pandemic (Hamaideh et al., 2022; Roldán‐Merino et al., 2022).
The government adopted a series of coping strategies to prevent its spread, such as limiting public gatherings and delaying school opening (Zhou et al., 2016). The COVID‐19 outbreak coincided with the winter holiday for most students in China. The holiday was supposed to end in the middle of February for most schools. Still, due to the need for pandemic prevention and control, the school opening time was suspended indefinitely. China's Ministry of Education emphasized that all the schools should not be opened until the pandemic was largely under control. Universities were encouraged to implement online teaching, encouraging teaching innovation.
2. BACKGROUND
2.1. Considerations in online teaching
Different online teaching formats were tried, for example massive online open courses (MOOCs), online teaching using social media like the WeChat app, internet live teaching relying on various meeting software Zoom and so on (Jia et al., 2019; Wang et al., 2017). However, some issues that may affect the effectiveness of online teaching have to be considered. Firstly, the accessibility of online teaching resources was important (Zhou et al., 2020). Could all the enrolled students access the online teaching? It required a certain level of network access and hardware (Naresh & Reddy, 2015). Secondly, what constitutes a feasible online teaching plan? The online teaching plan should be step‐by‐step so students can cope with the learning intensity (Gallagher‐Lepak et al., 2009). Thirdly, quality control of the teaching plan and its implementation was another critical consideration. That required an effective teaching management mechanism to ensure the transmission of the information at an appropriate pace (Porter et al., 2020). Process control during online teaching especially tracking the students' online learning behaviours was essential to ensure the smooth implementation and timely adjustment of the teaching plan (Keengwe & Kidd, 2010). Lastly, teacher–student communication was also an important consideration (Uijl et al., 2017). Compared with classroom teaching, online teaching lacks face‐to‐face discussion, which may be a gap that affects students' participation and the effectiveness of online teaching.
2.2. Flipped classroom in teaching
The flipped classroom is the teaching process in which the teacher gives the students pre‐prepared documents, tests and video materials according to the course content and teaching aims, and the students learn autonomously before class. Then the students need to cooperate and discuss with other students and the teacher together to solve any problems and clear up doubts, which could help the students internalize the knowledge (Bergmann & Sams, 2012). The flipped classroom has changed the traditional mode of teacher teaching and students passively receiving knowledge (Schmidt & Ralph, 2016). The core feature of the flipped classroom is that the students need to learn the material before class and then have student‐led and teacher‐assisted interactive learning activities in class. In general, the flipped classroom is thus a strongly student‐oriented teaching technique (Persky & McLaughlin, 2017), which can enrich the learning experience and stimulate valuable interactions among teachers and students. And the key components of flipped classroom teaching are teaching resources for the students' autonomous learning (Shi‐Chun et al., 2014), tests for finding problems and a teaching plan setting out what parts of the course content are to be ‘flipped’ and which are not (Schmidt & Ralph, 2016).
2.3. Online flipped classroom attempts
The typical flipped classroom has been offline, but several scholars have been trying to establish a fully online flipped classrooms (OFC) model. The fully OFC model allows students to complete online pre‐class activities asynchronously and meet online synchronously for active learning using Web‐based videoconferencing media or communication tools rather than in a physical classroom (Jia et al., 2023). The entire process of full OFC learning is online, whether pre‐class preparation or classroom activities. The OFC model has the following advantages during the pandemic. Firstly, an OFC model can overcome learning obstacles related to the physical environment better than a typical flipped classroom. Secondly, the model is expected to reduce the sense of remoteness in online teaching and facilitate online interactions between teachers and students. Students learning satisfaction and performance were higher with higher professor–student interactions in the online flipped classroom (Cho & Kim, 2021). Moreover, these interactions can not only avoid students' concentration loss and immersion difficulties when listening to online lectures for a long time but also alleviate the situation of ineffective communication and Q&A in online courses (Kang & Zhang, 2020; Tang et al., 2020). Thirdly, the OFC model can promote students to higher learning levels, such as application, analysis and synthesis of knowledge, as defined by Bloom's Taxonomy, which is also a timely and effective assessment of the students' mastery of the basics (Beason‐Abmayr et al., 2021).
The study indicated that medical students in the OFC group showed better effects of COVID‐19 knowledge learning, compared with online lecturing (Qian et al., 2021). The qualitative research also reported that the implementation of a fully online flipped classroom in pharmacy practice education was well received by medical students (Annamalai et al., 2021). In nursing teaching, there was a study with a pre‐test–post‐test design applying the OFC method to cultivate students' self‐directed learning readiness and metacognitive awareness. The results showed a significant improvement compared with before the intervention (Khodaei et al., 2022). In the study of teaching immunization courses for nursing students, it was mainly found that participation in an OFC method had a positive effect on academic achievement compared to online lectures (Holm & Beckman, 2022). However, Phillips and O'Flaherty (2019) reported similar course grades for both methods when using the OFC method versus online–offline‐blended teaching for nursing students in Australia, and half of the students expressed dissatisfaction with the online flipped delivery. At present, although a few research using the online flipped classroom approach have been reported, further research is needed to assess the effects of this model on learning outcomes and application experience in nursing courses.
Considering the advantages of the OFC model and teaching conditions during the pandemic, this study was designed to establish and implement this teaching method in the internal medicine nursing (IMN) course during the COVID‐19 outbreak. The IMN course is one of the core nursing courses taught for nursing undergraduates, which involves learning about multiple systemic diseases. To help students understand the course content and complete the pre‐class preparation, MOOC of IMN was the primary means of content delivery. It ensures the availability of systematic learning in IMN courses where students have immediate and convenient access to any topic (Jia et al., 2019). Meanwhile, WeChat, as the most popular social media, is suitable for carrying out classroom activities in the OFC model.
Traditionally, the teaching methods for the course have included lectures, clinical probation and case discussions. With the unexpected delay in the university's opening, an online flipped classroom program was designed instead. The OFC program was carried out based on MOOC, WeChat applet and WeChat groups for the topics that would normally have been delivered in lectures. This study aims to establish the full OFC model of internal medicine nursing courses, evaluate the effects of the OFC method in IMN courses during the pandemic, explore whether the OFC model can serve as a substitute for nursing education during the pandemic and lastly, provide suggestions for the future implementation of online flipped classroom teaching.
3. METHODS
3.1. Design and participants
This study was a comparative descriptive study with the historical control design evaluating the online flipped classroom (OFC) teaching effectiveness. It had a convenience sample and included two groups. The OFC group comprised full‐time third‐year nursing undergraduates participating in the fully online flipped classroom course in the spring of 2020. The traditional teaching group consisted of the previous year's undergraduates who received traditional teaching in the spring semester of 2019. Students have not received an OFC teaching method in previous learning. The students who did not take the final examination for any reason (such as applying for a suspended exam, suspension of school, etc.) were excluded from the study.
The sample size was estimated using the G*Power version 3.1 software (Copyright © 2020 HHU.). When the effect size was entered as 0.3 in the Χ 2 tests, with the power (1 − β) of 80%, a significance level (α) of 0.05, and a degree of freedom (Df) of 1, the minimum sample size calculated by the G*Power was 88. Considering 10% dropouts, a sample size of approximately 98 was determined.
3.2. The internal medicine nursing course
The course tested was Internal Medicine Nursing II (IMN II). Due to a large amount of content, to reduce the learning burden on the students, Internal Medicine Nursing (IMN) is divided into two courses, IMN I and IMN II, which are conducted in the autumn semester and the following spring semester respectively. The teaching in both courses includes lectures on theoretical knowledge, clinical probation and clinical case discussions. Generally, IMN I lasts for the whole autumn semester and covers the respiratory, cardiovascular, digestive, urinary, and blood systems, all given in lectures. The clinical work and discussions focus mainly on the first three systems, and the examination at the end of IMN I also covers only those three. IMN II lasts only half of the following spring semester. The theoretical content of IMN II deals with the endocrine system and rheumatic diseases, and the clinical work and discussion in IMN II emphasize the urinary system and the blood system from the previous semester, plus the endocrine system and rheumatic diseases. The students' performance in both IMN I and IMN II is composed of the final examination score (50%) and regular performance, including case documentation and tests (50%). In this study, the OFC method taught IMN II in the first half of the spring semester of 2020.
3.3. Teaching process
In 2019, for the traditional group, IMN II was arranged in the first half of the spring semester and lasted 9 weeks for students in Grade 2016. There were two weekly lessons, each lasting for 2–3 learning hours (each learning hour is 45 min). Theoretical knowledge about the endocrine system and rheumatic diseases was delivered in lectures during the first 3 weeks. Clinical work and discussions about the urinary, blood, endocrine and rheumatic systems were arranged during the following 6 weeks (Table 1). Students' performances in the IMN II were composed of the final examination (50%) and regular performances (50%). The final examination was closed‐book and lasted 2 h. Case documentation after clinical probation and unit test results contributed to half of the final score.
TABLE 1.
The plan of traditional offline teaching.
| Week | Contents | Day and time | Actions |
|---|---|---|---|
| Week 1–Week 3 |
|
Each Wednesday 14:30–16:00 |
|
| Each Thursday 14:30–16:00 | |||
| Week 4–Week 9 |
|
Each Wednesday 14:30–16:45 |
|
| Each Thursday 14:30–16:45 |
|
In 2020, for the OFC group, the OFC technique was delivered for the nursing undergraduates in Grade 2017. In the first 4 weeks, knowledge about the endocrine system and rheumatic diseases was provided through self‐instruction in a MOOC platform (https://www.icourse163.org), supplemented by online interactions in WeChat conferences, and quizzes and tests administered using MOOC Classroom software (Lu et al., 2020). Afterwards, because the examination would cover those topics, the urinary and blood systems from the previous term, weekly WeChat conferences were provided online for the next 6 weeks to help the students review those previous topics. The clinical probation and case discussions had to be cancelled because of the COVID‐19 outbreak.
The following strategies were adopted for the online flipped classroom of IMN II:
The teaching plan for the OFC method (Table 2) referred to the previous offline course. New knowledge was delivered in the first 4 weeks. Planned reviews were arranged in the last 6 weeks.
All the online learning resources in MOOC, including videos and documents developed by three associate professors of nursing and two lecturers of medicine in 2017, were issued according to the teaching plan through the MOOC platform (MOOC platform URL). Moreover, the online learning resources in MOOC of IMN II were similar to the 2019 traditional course.
A curriculum management structure was established. The corresponding author, the course leader for IMN II with flipped classroom experience, was in complete charge of the online flipped classroom. Two other teachers assisted the corresponding author after receiving the OFC training through the online conference. One student acted as a class liaison who was responsible for contacting and organizing all the students using WeChat. Two other students who were originally the class learning monitors assisted the teachers with tracking the students' learning behaviours.
Tracking students' online learning behaviours. The two class learning monitors need to log in to the MOOC course management system every week to check the number and duration of videos watched by students, and the number and situation of the week's quizzes completed by students. The students who did not meet this week's learning on time were identified and were contacted individually by the learning monitors to remind them and help them resolve difficulties.
WeChat is the phone app most commonly used among Chinese students, so a WeChat group was established that covered all the students and teachers. Weekly WeChat conferences were conducted to maintain communication between the teachers and students about classroom activities.
All students were required to download and log on to the WeChat applet MOOC Classroom which was used to issue quizzes and unit tests to specific groups.
TABLE 2.
The teaching plan of the online flipped classroom.
| Week | Contents | Day and time | Actions |
|---|---|---|---|
| Week 1–Week 4 |
|
Each Monday 10:00 |
|
| Each Wednesday 10:00 |
|
||
| Each Thursday 10:00–11:00 |
|
||
| Week 5–Week 10 |
|
Each Monday 10:00 |
|
| Each Thursday 9:00–11:00 |
|
The course leader regularly posted group announcements in the WeChat group to guide the students' online self‐learning based on the teaching plan. The videos and files about the endocrine system and rheumatic diseases were issued on the MOOC platform, and the quizzes and unit tests were given through the WeChat applet MOOC Classroom. WeChat conferences were held in the WeChat group every Thursday for online interactions. In the WeChat conferences, the teachers first discussed the questions with the highest error rates. Then the students could freely raise questions in the group and communicate with each other and the teachers. At the end of the WeChat conferences, a unit test would be issued through the MOOC Classroom, which the students could use to test themselves.
Because the clinical probation was cancelled during the COVID‐19 outbreak, the usual clinical case documentation was also cancelled. The students' performance in the course was based entirely on the final examination, the regular quizzes, and unit tests, all conducted online. The usual tests were given through the WeChat applet MOOC Classroom. The final examination was performed using the Wenjuanxing software, an online examination system (https://www.wjx.cn/). During the examination, the students were sent an examination link and required to complete the examination independently and closed book. Objective questions like multiple‐choice questions could be answered directly online. For the subjective questions, the students were required to write down their answers on paper, photograph them and upload the images. Online invigilation typically requires the students to prepare two cameras set in front and back simultaneously during the exam. Still, due to the imbalance in the students' family economic situations, online invigilation was not carried out in this examination. Also, the examination's duration was reduced to 90 min from 2 h to reduce the possibility of students looking for reference data.
3.4. Outcome measures
3.4.1. Final examination
The final examination scores in 2019 and 2020 were compared. The total possible score in both years was 100. Each exam consisted of 40 multiple‐choice questions, five explanations of terminology, four essay questions and two case analyses. The knowledge about the urine, blood and endocrine‐rheumatic systems each accounted for about one‐third. Because it is not permitted that the same questions to appear in two successive final examinations in the university, the following techniques were used to ensure equivalence between the two examinations: (1) the questions came from the same question bank and were composed by the same teacher, (2) before it was administered, two experienced teachers of the course compared the 2020 paper with the examination in 2019 together and modified some questions to ensure similar difficulty, (3) after the final examination in 2020, the quality analysis of tests, including the item‐difficulty (sometimes termed item‐facility) index and the item‐discrimination index, was computed and compared with 2019.
3.4.2. Interaction in the WeChat group
The number and duration of students' interactions in the WeChat conferences were compiled in 2020. They reflected on how actively the students participated in the OFC teaching.
3.4.3. Survey about online flipped classroom teaching
An online survey was conducted at the end of the course, soliciting the students' opinions about the OFC learning experiences. The self‐developed questionnaire consisted of six items with responses on a 5‐point Likert scale and two open‐ended questions.
3.5. Statistical analysis
Microsoft Office Access (Copyright © 2003 Microsoft, Inc.) and Testan software (Copyright © 2015, the university where the corresponding author worked.) were used to conduct the quality analysis of the final examination. The difficulty and discrimination of the final examinations in 2019 and 2020 will be described and compared. The item‐difficulty index quantifies the exam's degree of difficulty. It ranges from 0 to 1, with larger values indicating greater difficulty (Tavakol & Dennick, 2011). An item‐discrimination index quantifies how well a test discriminates between students who perform well and those who do not. The discrimination index ranges from −1.00 to 1.00, with higher values indicating better differentiation between students who scored high and those who scored low (Tavakol & Dennick, 2011). Generally, when the discrimination value is greater than 0.39, the discrimination level is considered excellent. And if the value is between 0.2 and 0.39, it is mediocre, and teachers are suggested to check and review the examination papers. Lastly, when the value is smaller than 0.2, the level is poor, and the examination should be discarded or reviewed in depth (Escudero et al., 2000).
SPSS 25.0 software (SPSS Inc.) was used in the statistical analysis. Means, standard deviations, frequencies and percentages were used to describe the nursing students' demographics and performance in the final examinations. Χ 2 tests were used to compare the pass rates and distribution of scores in the final examination between the two groups. T tests were used to examine the difference in mean scores. When the data were not normally distributed, the Wilcoxon test was performed. Interactions in the WeChat conferences and the survey results about OFC teaching were also analysed for the 2020 class.
4. RESULTS
4.1. Test paper analysis of the final examination
After the online final examination in 2020 was implemented, the test paper analysis of IMN II final examination was computed. The difficulty of the 2020 paper was assessed as 0.1799, which was viewed as very easy. The discrimination index was 0.6973, which was considered excellent. The difficulty and discrimination of the previous year's paper were 0.2073 and 0.6917, respectively, which were on the same level as the paper in 2020.
4.2. The balance examination in 2019 and 2020
There were 53 nursing undergraduates who took the final examination in 2019 compared with 50 in 2020. The average age of students in 2019 was 22.32 ± 0.70, and the average age in 2020 was 21.48 ± 0.97. In both grades, the age ranged from 19 to 24, and the majority were female. Table 3 shows that their origin places were in all parts of China. There were a few ethnic minority students in both groups. There were no significant differences between the two groups about gender distribution, places of origin, ethnic group distribution, or previous academic performances.
TABLE 3.
Comparison of baseline variables in 2019 and 2020.
| Variable | Categories | 2019 class (n = 53) | 2020 class (n = 50) | Χ 2/Z | p |
|---|---|---|---|---|---|
| Frequency (%) | Frequency (%) | ||||
| Gender | Male | 5 (9.4) | 11 (22.0) | 3.096 | 0.078 |
| Female | 48 (90.6) | 39 (78.0) | |||
| Place of origin | Eastern Provinces | 5 (9.4) | 5 (10.0) | 0.676 | 0.755 |
| Central Provinces | 10 (18.9) | 16 (32.0) | |||
| Western Provinces | 34 (64.1) | 28 (56.0) | |||
| Hong Kong, Macao or Taiwan regions China | 4 (7.6) | 1 (2.0) | |||
| Ethnic group | Han Nationality | 41 (77.4) | 42 (84.0) | 0.725 | 0.394 |
| Other Minority Ethnic Groups | 12 (22.6) | 8 (16.0) | |||
| Previous academic performance (GPA a ) | ≥3.0 | 21 (39.6) | 22 (10.0) | 0.213 | 0.975 |
| 2.5 ≤ GPA <3.0 | 16 (30.2) | 14 (38.0) | |||
| 2.0 ≤ GPA <2.5 | 10 (18.9) | 9 (42.0) | |||
| <2.0 | 6 (11.3) | 5 (10.0) |
GPA is a student's grade point average on a scale of 1–5.
4.3. Performance comparison of the final examination in 2019 and 2020
The passing rate in 2020 reached 92%, and the passing rate in 2019 was 96%. The difference in passing rates between the two groups was not statistically significant (Χ 2 = 0.276, p = 0.60). The examination median score in 2019 (85.0) was nearly 5 points higher than that (80.5) in 2020 (Z = −2.491, p = 0.01). The difference in the distribution of scores between the two classes was statistically significant (Χ 2 = 6.655, p = 0.04).
The score distribution chart (Figure 1) showed that the overall score distributions of the two classes were consistent, in which excellent and poor students accounted for a small number, and medium and good students accounted for the majority. But note that the 2019 class had more students rated good (52.83%) than medium rank (18.87%). In 2020 the percentages of the students with good ranks decreased obviously (40.00%), which was basically the same as those with medium scores (42.00%). That made 2020s overall average lower than that of 2019.
FIGURE 1.
Distribution of final examination scores in 2019 and 2020.
4.4. Online interactions in 2020
Teachers and students sending messages, pictures or voices to contact each other in a WeChat group are defined as online interaction on WeChat. There were 996 online interactions totally during the online flipped classroom phase (with 50 students and 2 teachers). Raising or answering a question about nursing expertise in a WeChat group is a valid interaction on WeChat. There were 734 valid interactions in total.
As shown in Table 4, in the first 4 weeks when the new knowledge was being taught, there were 704 online interactions (328 times by teachers and 376 times by students) and 460 valid interactions (245 times by teachers and 215 times by students) in that stage. In the last 6 weeks, when the previous knowledge was being reviewed, 292 online interactions (181 times by teachers and 111 times by students) and 274 valid interactions were generated (168 times by teachers and 106 times by students).
TABLE 4.
Times of interactions during the online flipped classroom.
| Types of interactions | Teaching new knowledge (weeks 1–4) | Reviewing previous knowledge (weeks 5–10) |
|---|---|---|
| Number of times teachers contact | 328 | 181 |
| Valid interactions of teachers | 245 | 168 |
| Number of times students contact | 376 | 111 |
| Valid interactions of students | 215 | 106 |
4.5. The survey on the online flipped classroom teaching
Forty‐nine students participated in the survey about their learning experience under OFC teaching (Table 5). All of them said they thought the OFC schedule was reasonable, and 14 students (84%) said they could follow the lesson in progress most of the time. All except three (94%) said they were satisfied with their degree of engagement during the online study and were confident about passing the final examination. Taking the weekly quiz and unit tests and having weekly WeChat conferences were considered helpful by over 46 (94%) students. Although 61% of the students believed this method's effects were the same and better than offline learning, 39% still perceived worse learning results than offline teaching.
TABLE 5.
The survey about online flipped classroom teaching in 2020 (n = 49).
| Items | Frequency (%) |
|---|---|
| 1. What do you think of your mastery of the knowledge you learned this semester? | |
| No grasp at all or master a little | 3 (6.12) |
| Not bad. I think I will pass | 36 (73.47) |
| Better mastery | 10 (20.41) |
| Complete mastery | 0 (0) |
| 2. Are you satisfied with your engagement in the flipped classroom online learning this semester? | |
| Not satisfied at all or less satisfied | 3 (6.12) |
| Generally satisfied | 28 (57.14) |
| More satisfied | 17 (34.69) |
| Completely satisfied | 1 (2.04) |
| 3. Do you think the online flipped classroom schedule for Internal Medicine Nursing II is reasonable? | |
| Totally unreasonable and less than reasonable | 0 (0) |
| Generally reasonable | 4 (8.16) |
| Mostly reasonable | 35 (71.43) |
| Completely reasonable | 10 (20.41) |
| 4. Can you follow the teacher's teaching progress during the flipped classroom online study? | |
| Not at all or occasionally | 2 (4.08) |
| Often | 6 (12.24) |
| Most of the time | 23 (46.94) |
| Completely | 18 (36.73) |
| 5. Do you think the weekly quiz and unit tests are helpful for mastering theoretical knowledge? | |
| Not helpful at all or a little helpful | 1 (2.04) |
| Generally helpful | 2 (4.08) |
| Fairly helpful | 30 (61.22) |
| Completely helpful | 16 (32.65) |
| 6. Do you think the weekly WeChat conferences help students master theoretical knowledge? | |
| Not helpful at all or a little helpful | 0 (0) |
| Generally helpful | 1 (2.04) |
| Fairly helpful | 15 (30.61) |
| Completely helpful | 33 (67.35) |
| 7. How do you rate this semester's online flipped classroom compared with previous offline teaching? | |
| Completely inferior to teaching in the classroom | 1 (2.04) |
| A little worse than teaching in the classroom | 18 (36.73) |
| The same as teaching in the classroom | 8 (16.33) |
| Better than teaching in the classroom | 20 (40.82) |
| Much better than teaching in the classroom | 2 (4.08) |
5. DISCUSSION
During the pandemic, we implemented a pedagogical practice for the OFC model based on MOOCs and the social media WeChat app. The results showed that the effects of this method were fairly effective. Students maintained the same high pass rate as traditional teaching. However, slightly more students were ranked as the medium, and slightly fewer were rated as good compared with traditional teaching. This study provided experiences for other practitioners when students cannot meet the lecturers in traditional teaching. Secondly, the study reported that the OFC model was effectively interactive compared with other online teaching methods. Thirdly, the students showed their satisfaction with this teaching model. Lastly, we also identify the limitations that need to be addressed. Compared with traditional teaching, fewer students had excellent grades in the examination, which includes many reasons other than the online flipped classroom. This deserves further exploration in future research.
The overall median examination score of the students in 2020 with OFC teaching was lower than that in 2019 with traditional teaching in this study. This is inconsistent with previous research that showed no statistical difference between the effects of traditional teaching and OFC teaching in health assessment courses of second‐year nursing undergraduates (Wang et al., 2022). The main reason may be that the study provided supplementary lessons on health assessment practicum skills after students returned to university, which facilitates the understanding and applying knowledge. Secondly, unlike our study, the study combined live lectures with videoconferencing software and the OFC method instead of just applying the OFC method in all chapters of the health assessment course. Therefore, for the OFC model, it is necessary to determine the optimal number of chapters to be flipped to produce more benefits for learners. Joseph et al. (2021) reported that flipped classrooms should take longer than 2 weeks, but the threshold for students to accept flipped chapters has not yet been determined. In general, the teaching effects of the full OFC model on nursing courses are currently inconclusive and worthy of more research and exploration.
Students in the OFC teaching model showed the same excellent pass rates as those in the traditional teaching model, but more students were distributed in the medium rank than in traditional teaching. There are several possible reasons for this. Firstly, online learning lacks links with clinical learning and practice such as clinical probation and case discussion of specific internal medicine diseases. That is important for strengthening students' memory, understanding, and application of medical nursing knowledge (Yong et al., 2010). Especially for 3rd‐ and 4th‐year students, their coursework is primarily based on clinical practice. They also attach great importance to clinical practice and link it to acquired competence (Ramos‐Morcillo et al., 2020). Moreover, supportive clinical learning and practice can increase students' positive learning interests and experiences (Carlson & Bengtsson, 2014). Therefore, the lack of clinical practice hinders the improvement of students' performance to some extent. Secondly, the flipped classroom is a student‐led learning mode in which students have to learn related teaching videos and documents by themselves before class. It requires better self‐learning ability and self‐discipline (Shi‐Chun et al., 2014). And, of course, not all students display the same good initiative and self‐discipline. Thirdly, some students may not prepare appropriately in time to switch to online mode due to the sudden pandemic (Ramos‐Morcillo et al., 2020). Lastly, online learning lacks the offline learning atmosphere among peers and the supervision of teachers. After all, the external environment is well known to influence students' learning behaviour (Jack & Punch, 2001).
It is worth noting, though, that the online flipped classroom generated a lot of online interactions, including valid and non‐valid interactions in which all students participated. Non‐valid interaction is conducive to establishing active professor–student relationships and communication, and valid interactions facilitate the application, analysis and synthesis of internal medicine nursing knowledge by students (Cho & Kim, 2021). These two kinds of interactions also effectively make up for the lack of face‐to‐face interaction to some extent. When faced with an emergent event like COVID‐19, online learning may be the best resolution for maintaining education. There are different online learning methods, like MOOCs, internet live teaching and so on, all offer convenience, flexibility and fewer space limitations (Stone & Perumean‐Chaney, 2011). However, encouraging effective interactions between students and teachers in online teaching is not easy (Uijl et al., 2017). Beason‐Abmayr et al. (2021) reported that when using Zoom to discuss, fewer students participated in class discussions, and were more hesitant to show their faces and speak up in Zoom, resulting in a poor discussion effect. In this study, selecting WeChat seems to have been an essential facilitator of satisfactory interaction. WeChat was used to build an OFC community for the students because it is the most popular social medium for the Chinese (Zeng et al., 2016). All students and teachers already had WeChat accounts. It satisfied the needs of online learning in accessibility. Secondly, the well‐organized weekly WeChat conferences helped a lot. Learning resources and the corresponding quizzes were issued before the WeChat conferences. The teachers would first analyse the quiz questions with high error rates in each conference. Then the students asked questions and communicated with the teachers freely. When the conference was closing, a unit test was issued to close out the topic for the week. In the WeChat conferences, both the students and the teachers came prepared. In the student‐led teaching activities, there were established not only tasks like quizzes and tests but also opportunities for accessible communication and discussion. This may be why the professor–student online interactions were satisfactory.
All the students thought the OFC arrangement reasonable and the weekly WeChat conferences helpful in mastering knowledge. They have high overall satisfaction with the internal medicine course with full OFC teaching. Several studies also reported that nursing undergraduates showed positive acceptance and satisfaction with the OFC model (Cho & Kim, 2021; Joseph et al., 2021). However, 39% of students believed that OFC's learning effect was worse than offline teaching. Those subjective feelings of the students were consistent with some students' actual performances. After all, nursing is a practice‐based discipline. Nursing practice not only helps students understand and apply theoretical knowledge but also guides the coming professional development of nursing students (Carlson & Bengtsson, 2014). Some learning activities, such as clinical practice and case discussion, are essential for effective nursing education. It may be necessary to conduct offline supplementary courses after the pandemic.
5.1. Limitations
Firstly, the study may not control the confounding variables well. The OFC group was accompanied by the pandemic, which had a larger impact on nursing students than the traditional teaching group. Then, the OFC technique was implemented in only one of the kinds of nursing curriculums in one university. The effects of this teaching mode should be examined with more different universities and more diverse nursing content. Besides, we did not implement strict monitoring in the final online examination. Strict invigilation for online examinations requires students to own at least two cameras (one is at the back of the student and the other one is in front of the student), but not all students can meet the requirements. Instead, we shortened the examination duration to 30 min than before to limit the possibility of students searching for references and communicating with others. At the same time, there is also a limit to evaluating the effectiveness of education only with examination scores in the study. Case discussions after clinical probation and non‐objective indicators could be supplemented.
6. CONCLUSION
The fully online flipped classroom method was fairly effective. It helps students become qualified learners for internal medicine nursing courses during the pandemic. The reasonable teaching plan and curriculum management structure, rich online learning resources and effective interactive tools are beneficial for high pass rates of students. However, fewer students earned outstanding grades, which can be explained by the lack of supervision and clinical practice, stress from COVID‐19, and shortened examination time. This potential concern about the scores deserves attention and should be explored in future research. It is worth noting that the fully online flipped classroom teaching did yield sufficient and effective online interactions, which increased the involvement of students in class and professor–student interactions and ensured student satisfaction. This model can, to some extent, compensate for the limitations of online teaching through the interaction between students and teachers. Overall, the fully online flipped classroom is worth implementing in public health emergencies like a pandemic, and whether this model can be used as a substitute in nursing education during the pandemic needs further exploration and evaluation for different teaching objects and courses.
AUTHOR CONTRIBUTIONS
Jingrong Du and Xiaoyi Chen collected and analysed the data and drafted and revised the manuscript for publication. Tong Wang and Juanjuan Zhao were involved in the analysis and interpretation of data and the revision of the manuscript. Kun Li organized the project design, implementation, quality control and manuscript revision. All authors take responsibility for their portions of the content and agree to be accountable for all aspects of the work.
FUNDING INFORMATION
The study was supported by the Undergraduate Teaching Quality and Teaching Reform Project of Guangdong Province (grant number [2023] 4) and Undergraduate Teaching Quality Project of Sun Yat‐sen University (grant number [2021] 93). The sponsors had no role in the study design; in the collection, analysis and interpretation of data; in the writing of the report or in the decision to submit the article for publication.
CONFLICT OF INTEREST STATEMENT
There was no conflict of interest in the study.
ETHICS STATEMENT
The survey was approved by Sun Yat‐sen University's Department of Education Administration and Sun Yat‐sen University's ethics committee. That the work in the study was carried out in accordance with the Declaration of Helsinki, including the anonymity of participants being guaranteed and the informed consent of participants being obtained.
ACKNOWLEDGEMENTS
The authors thank all students who participated in the study and give feedback. The authors would like to express special thanks to Lejian He, who helped to contact and organize all the students using WeChat as the teaching assistant, and Mengmeng Jia, who managed and issued the MOOC learning resources of the Internal Medicine Nursing course.
Du, J. , Chen, X. , Wang, T. , Zhao, J. , & Li, K. (2023). The effectiveness of the fully online flipped classroom for nursing undergraduates during the COVID‐19: Historical control study. Nursing Open, 10, 5766–5776. 10.1002/nop2.1757
Jingrong Du and Xiaoyi Chen contributed equally to this work.
DATA AVAILABILITY STATEMENT
The data that support this study are available from the corresponding author upon reasonable request.
REFERENCES
- Annamalai, N. , Tangiisuran, B. , & Athirah Daud, N. A. (2021). Transitioning to online clerkship during unprecedented times: An innovative online flipped in‐patient clerkship. Innovations in Education and Teaching International, 59, 690–700. 10.1080/14703297.2021.1931406 [DOI] [Google Scholar]
- Beason‐Abmayr, B. , Caprette, D. R. , & Gopalan, C. (2021). Flipped teaching eased the transition from face‐to‐face teaching to online instruction during the COVID‐19 pandemic. Advances in Physiology Education, 45(2), 384–389. 10.1152/advan.00248.2020 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bergmann, J. , & Sams, A. (2012). Why you should flipped your classroom (chapter 3). In Flip your classroom: Reach every student in every class every day (1st ed.). International Society for Technology in Education. https://so03.tci‐thaijo.org/index.php/jliwu/article/view/107655 [Google Scholar]
- Carlson, E. , & Bengtsson, M. (2014). The uniqueness of elderly care: Registered nurses' experience as preceptors during clinical practice in nursing homes and home‐based care. Nurse Education Today, 34(4), 569–573. 10.1016/j.nedt.2013.07.017 [DOI] [PubMed] [Google Scholar]
- Cho, M.‐K. , & Kim, M. Y. (2021). Factors affecting learning satisfaction in face‐to‐face and non‐face‐to‐face flipped learning among nursing students. International Journal of Environmental Research and Public Health, 18(16), 8641. 10.3390/ijerph18168641 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Escudero, E. B. , Reyna, N. L. , & Morales, M. R. (2000). The level of difficulty and discrimination power of the basic knowledge and skills examination (EXHCOBA). Advanced Pharmaceutical Bulletin, 2(1), 2. [Google Scholar]
- Gallagher‐Lepak, S. , Reilly, J. , & Killion, C. M. (2009). Nursing student perceptions of community in online learning. Contemporary Nurse: A Journal for the Australian Nursing Profession, 32(1–2), 133–146. 10.5172/conu.32.1-2.133 [DOI] [PubMed] [Google Scholar]
- Hamaideh, S. H. , Al‐Modallal, H. , Tanash, M. A. , & Hamdan‐Mansour, A. (2022). Depression, anxiety and stress among undergraduate students during COVID‐19 outbreak and" home‐quarantine". Nursing Open, 9(2), 1423–1431. 10.1002/nop2.918 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Holm, P. , & Beckman, L. (2022). Flipped or traditional online teaching? Two different strategies to handle teaching in nursing education during the COVID‐19 pandemic. International Journal of Nursing Education Scholarship, 19(1), 20210119. 10.1515/ijnes-2021-0119 [DOI] [PubMed] [Google Scholar]
- Jack, Y. , & Punch, K. F. (2001). External environment and school organisational learning: Conceptualising the empirically neglected. International Studies in Educational Administration, 29(3), 28. [Google Scholar]
- Jia, C. , Hew, K. F. , Jiahui, D. , & Liuyufeng, L. (2023). Towards a fully online flipped classroom model to support student learning outcomes and engagement: A 2‐year design‐based study. The Internet and Higher Education, 56, 100878. 10.1016/j.iheduc.2022.100878 [DOI] [Google Scholar]
- Jia, M. , Gong, D. , Luo, J. , Zhao, J. , Zheng, J. , & Li, K. (2019). Who can benefit more from massive open online courses? A prospective cohort study. Nurse Education Today, 76, 96–102. 10.1016/j.nedt.2019.02.004 [DOI] [PubMed] [Google Scholar]
- Joseph, M. A. , Roach, E. J. , Natarajan, J. , Karkada, S. , & Cayaban, A. R. R. (2021). Flipped classroom improves Omani nursing students performance and satisfaction in anatomy and physiology. BMC Nursing, 20(1), 1–10. 10.1186/s12912-020-00515-w [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kang, X. , & Zhang, W. (2020). An experimental case study on forum‐based online teaching to improve student's engagement and motivation in higher education. Interactive Learning Environments, 31, 1029–1040. 10.1080/10494820.2020.1817758 [DOI] [Google Scholar]
- Keengwe, J. , & Kidd, T. (2010). Towards best practices in online learning and teaching in higher education. Journal of Online Learning and Teaching, 6(2), 533–541. [Google Scholar]
- Khodaei, S. , Hasanvand, S. , Gholami, M. , Mokhayeri, Y. , & Amini, M. (2022). The effect of the online flipped classroom on self‐directed learning readiness and metacognitive awareness in nursing students during the COVID‐19 pandemic. BMC Nursing, 21(1), 22. 10.1186/s12912-022-00804-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lu, L. , Shuai, Q. , & Li, J. (2020). Online teaching practice of analytical chemistry based on the combination of China university MOOC, MOOC classroom and QQ group. University Chemistry, 33(5), 15–18. [Google Scholar]
- Naresh, B. , & Reddy, B. S. (2015). Challenges and opportunity of E‐learning in developed and developing countries‐a review. International Journal of Emerging Research in Management and Technology, 4(6), 259–262. [Google Scholar]
- Persky, A. M. , & McLaughlin, J. E. (2017). The flipped classroom–from theory to practice in health professional education. American Journal of Pharmaceutical Education, 81(6), 118. 10.5688/ajpe816118 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Phillips, C. , & O'Flaherty, J. (2019). Evaluating nursing students' engagement in an online course using flipped virtual classrooms. Student Success, 10(1), 59–72. 10.5204/ssj.v10i1.1098 [DOI] [Google Scholar]
- Porter, J. E. , Barbagallo, M. S. , Peck, B. , Allen, L. , Tanti, E. , & Churchill, A. J. (2020). The academic experiences of transitioning to blended online and digital nursing curriculum. Nurse Education Today, 87, 104361. 10.1016/j.nedt.2020.104361 [DOI] [PubMed] [Google Scholar]
- Qian, Q. , Yan, Y. , Xue, F. , Lin, J. , Zhang, F. , & Zhao, J. (2021). Coronavirus disease 2019 (COVID‐19) learning online: A flipped classroom based on micro‐learning combined with case‐based learning in undergraduate medical students. Advances in Medical Education and Practice, 12, 835–842. 10.2147/AMEP.S294980 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ramos‐Morcillo, A. J. , Leal‐Costa, C. , Moral‐García, J. E. , & Ruzafa‐Martínez, M. (2020). Experiences of nursing students during the abrupt change from face‐to‐face to e‐learning education during the first month of confinement due to COVID‐19 in Spain. International Journal of Environmental Research and Public Health, 17(15), 5519. 10.3390/ijerph17155519 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rasmussen, B. , Hutchinson, A. , Lowe, G. , Wynter, K. , Redley, B. , Holton, S. , & McTier, L. (2022). The impact of covid‐19 on psychosocial well‐being and learning for australian nursing and midwifery undergraduate students: A cross‐sectional survey. Nurse Education in Practice, 58, 103275. 10.1016/j.nepr.2021.103275 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Roldán‐Merino, J. , Hurtado‐Pardos, B. , Molina‐Raya, L. , Bande, D. , Casas, I. , & Farrés‐Tarafa, M. (2022). Psychological impact of the COVID‐19 pandemic on nursing students in the first wave: A cross‐sectional survey. Nursing Open, 9(4), 2003–2012. 10.1002/nop2.1207 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schmidt, S. M. , & Ralph, D. L. (2016). The flipped classroom: A twist on teaching. Contemporary Issues in Education Research, 9(1), 1–6. 10.19030/cier.v9i1.9544 [DOI] [Google Scholar]
- Shi‐Chun, D. , Ze‐Tian, F. , & Yi, W. (2014, April). The flipped classroom–advantages and challenges . Paper presented at the 2014 international conference on economic management and trade cooperation, China. 10.2991/emtc-14.2014.3 [DOI]
- Stone, M. T. , & Perumean‐Chaney, S. (2011). The benefits of online teaching for traditional classroom pedagogy: A case study for improving face‐to‐face instruction. MERLOT Journal of Online Learning and Teaching, 7(3), 393–400. https://jolt.merlot.org/vol7no3/stone_0911.pdf [Google Scholar]
- Tang, T. , Abuhmaid, A. M. , Olaimat, M. , Oudat, D. M. , Aldhaeebi, M. , & Bamanger, E. (2020). Efficiency of flipped classroom with online‐based teaching under COVID‐19. Interactive Learning Environments, 31, 1077–1088. 10.1080/10494820.2020.1817761 [DOI] [Google Scholar]
- Tavakol, M. , & Dennick, R. (2011). Post‐examination analysis of objective tests. Medical Teacher, 33(6), 447–458. 10.3109/0142159X.2011.564682 [DOI] [PubMed] [Google Scholar]
- Uijl, S. , Filius, R. , & Ten Cate, O. (2017). Student interaction in small private online courses. Medical Science Educator, 27(2), 237–242. 10.1007/s40670-017-0380-x [DOI] [Google Scholar]
- Wang, F. , Xiao, L. D. , Wang, K. , Li, M. , & Yang, Y. (2017). Evaluation of a WeChat‐based dementia‐specific training program for nurses in primary care settings: A randomized controlled trial. Applied Nursing Research, 38, 51–59. 10.1016/j.apnr.2017.09.008 [DOI] [PubMed] [Google Scholar]
- Wang, T. , Du, J. , Jia, M. , Zhao, J. , Cheng, L. , & Li, K. (2022). Design of online flipped classroom teaching for “Health Assessment” and its practice. Journal of Nurses Training, 37(15), 1373–1378. 10.16821/j.cnki.hsjx.2022.15.006 [DOI] [Google Scholar]
- Yong, T. , Haoyu, Y. , & Qian, J. (2010). Study on application effect of two teaching methods in clinical probation of surgical nursing. Chinese General Nursing, 15, 1402–1403. 10.3969/j.issn.1674-4748.2010.15.064 [DOI] [Google Scholar]
- Zeng, F. , Deng, G. , Wang, Z. , & Liu, L. (2016). WeChat: A new clinical teaching tool for problem‐based learning. International Journal of Medical Education, 7, 119–121. 10.5116/ijme.5708.e5c4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zhou, Y. , Hu, Y. , & Jia, S. (2016). Study on mixed mode teaching and classroom teaching effect of “evidence‐based nursing”. Chinese Nursing Reserach, 30(2A), 440–443. 10.3969/j.issn.1009-6493.2016.04.017 [DOI] [Google Scholar]
- Zhou, T. , Huang, S. , Cheng, J. , & Xiao, Y. (2020). The distance teaching practice of combined mode of massive open online course micro‐video for interns in emergency department during the COVID‐19 epidemic period. Telemedicine and E‐Health, 26(5), 584–588. 10.1089/tmj.2020.0079 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The data that support this study are available from the corresponding author upon reasonable request.