Abstract
The flipped classroom has become a fundamental methodological option in recent years to provide more personalized education opportunities. The aim of this study is to investigate the academic performance and the perception of the students on the flipped classroom model (FCM) in a course of Didactics of Music in Childhood Education. One of the two groups enrolled in this subject received a quarter of classroom hours. A total of 51 students were selected by means of convenience random sampling, who were divided during a semester into an experimental group (n = 24, flipped classroom methodology) and a control group (n = 27, traditional methodology). Using ANOVA and ANCOVA models and cumulative ordinal categorical models with various functions of the statistical software R, the results indicate that the group has studied under a traditional methodology showed statistically significant differences in the variables of learning climate, self-efficacy and cognitive commitment, but with no impact on academic performance. Students who have studied under the flipped classroom model achieved higher academic performance globally. At the end of the study, the results obtained in the light of other similar investigations are discussed.
Keywords: Flipped Classroom, Educational technology, Academic performance, Music education
Flipped classroom; educational technology; academic performance; music education.
1. Introduction
In recent years, we have witnessed a progressive change in the way of teaching in higher education, combining -or progressively replacing-a traditional teacher-based model with other active and student-centred models. Technological evolution is supporting this change thanks to the variety of digital resources available, decisively influencing the emergence of new methodological approaches. Among these active learning models, the FCM (flipped classroom, flipped learning) is considered an effective model for students to participate in active learning and promote meaningful interactions among themselves and with the teacher (Pluta et al., 2013).
The FCM was born as a methodology of a mixed nature, where face-to-face and online teaching are combined, and where students work on the contents outside the academic environment, spending much more time in class to problem solving and more practical and participatory work (Mengual-Andrés et al., 2020). The teacher usually designs the contents in some audio-visual format, mainly video (Zainuddin, 2018). These materials are usually hosted on digital platforms so that students can easily access them at any time before attending class (Abeysekera and Dawson, 2015). In this way, a very practical learning mediated by technology takes place (Froehlich, 2018).
2. Literature review
At present, several empirical studies have demonstrated the potential of the FCM to improve the academic performance of students, in courses of English (Lee and Wallace, 2017), Mathematics, Physics and Chinese (Lo et al., 2018) or using augmented reality (Chang and Hwang, 2018). For Zainuddin et al. (2019), the FCM helps students prepare for classes and exams, and this preparation is reflected in their academic performance. Studies such as those by Albert and Beatty (2014) and Sahin et al. (2015) measured the effectiveness of the FCM -according to exams scores-against a traditional methodology. The results showed that the FCM produced significantly higher scores than those obtained with the traditional model, although not in all the exams. Önce and Kara (2019) analyzed the academic performance of a complete communication systems class under an FCM compared to the results of the two previous classes (traditional methodology). The results revealed a significant improvement in the learning level and a positive perception of students with the methodology used. A study by Ryan and Reid (2016) with higher General Chemistry students (control group and traditional methodology versus experimental group and FCM) showed a significant increase in academic performance in one third of the students in the experimental group. However, meta-analyses such as that by Strelan et al. (2020) highlighted that new studies are needed to examine these effects and to analyse how the different aspects of the FCM can moderate them.
When the FCM is used, there are authors who claim that this model causes a decrease in students’ motivation (Yilmaz, 2017). However, studies such as that of Thai et al. (2017) highlighted the opposite. Motivation towards studying is usually accompanied by a greater commitment to tasks both in and outside the classroom (Huang et al., 2019), making pre-class content an essential aspect of motivation when working in a flipped classroom setting. Some studies (Chuang et al., 2018; Nel, 2017) reported that courses with pre-class content (based on online videos) and the strategy of conducting tests before class had a positive impact on motivation of students, since they are previously prepared to participate actively in the activities in the classroom.
When managing a flipped classroom project, online learning platforms such as LMS (Learning Management Systems) and tools from Web 2.0 play a fundamental role. Thus, LMS such as Moodle, WiziQ or Docebo, among others, facilitate: 1) access to content selected by the teacher for use at any time; 2) interaction between students and teachers outside the classroom; 3) collaborative activities; and 4) options for monitoring students’ progress and online evaluation and feedback, among many other options (Ahmed and Osman, 2020; Prada et al., 2019). These features are essential when students must be self-sufficient outside the classroom, as in flipped teaching. For Dilani et al. (2013), these virtual platforms can improve student engagement and, consequently, increase their performance and motivation.
Learning resources in the FCM are essential to self-regulate learning. The FCM requires students to dedicate time to acquire basic knowledge and prepare for the face-to-face class (Burke and Fedoreck, 2017), and video is one of the most used resources for this (Xiu et al., 2019). Although video is usually the most used resource, Sletten (2017) showed that the participating students did not have a very positive opinion of the training videos as pre-class material. On the contrary, Porcaro et al. (2016) highlighted the very positive assessment of videos by more than 80% of the students, who demonstrated that they were well prepared for face-to-face classes. Videos act as a ‘substitute’ for the teacher, as they usually include the content that the teacher usually imparts in person. According to various studies, video has positive effects on learning (Arguel and Jamet, 2009; Gligora et al., 2014) and increases students' motivation to continue with their teaching process (Bravo et al., 2011). In any case, the main purpose in the design of an educational video should be the delivery of high-quality videos (Plantak Vukovac, Orehovački, and Novosel-Herceg (2016), with a balance between the included content and the visual or aesthetic experience. In this sense, there are some influential aspects such as the length, the type of content, the design, the narrative style, the inclusion of images, the position of the text on the screen or the integration of subtitles (Arguel and Jamet, 2009; Guo et al., 2014).
The FCM requires student's commitment, and this can be approached from different modes (pre-class and face-to-face activities) and dimensions (behavioural, cognitive and affective commitment). Commitment is often used as a synonym for motivation, but it is a consequence of the latter, and can be defined as the ‘effort made by students in their learning community, observable through any type of behavioural, cognitive or affective indicators along a continuum’ (Bond, 2020, p.3). In this study, we took into account cognitive engagement (superficial and deep) and engagement with pre-class activities. Deep commitment refers to personal satisfaction with learning and the students' involvement in their training process. While superficial commitment responds to the need of some students to achieve academic results with minimal effort.
Self-efficacy -understood by Bandura (2006) as the set of self-perceptions that people have about what they are capable of achieving with their abilities and considering it as one more self-regulatory process-also plays a fundamental role in FCM. Authors such as Williams and Rhodes (2016) consider that the indicators ‘could’, ‘ability to…’ or ‘confiance in...’ are consistent with Bandura's definition of self-efficacy and his guide to measure it. Studies such as those by Alegre (2014) and Nasir and Iqbal (2019) have provided strong evidence that a high level of self-efficacy is a good predictor of high performance. On the contrary, a perception of low self-efficacy can impair this performance.
The FCM implies a greater interaction between students and teachers, where the learning climate plays an important role. The ‘learning climate’ variable (Moral and Pareja, 2010) is continually debated in a dichotomy between the learning proposed and the different styles of the teaching staff. McGregor (1994) coined the concept of ‘theory X and theory Y’ to refer to the assumptions where students do not want to learn and we have to control all their actions, which leads to a learning climate based on student's anxiety and blame (theory X). Then we reach theory Y, according to which students are assumed to work and learn best when they are free to use their own judgment. Both positions ‘continue to confront each other, hindering the creation of an optimal learning environment’ (Pareja and Pedrosa, 2012, p. 233).
Studies such as those by Önce and Kara (2019) and McNally et al. (2017) emphasize that we must consider that there are students with little discipline who are reluctant and unwilling to work under an FCM. To minimize that impact on these students, it is necessary for the teacher to present the flipped classroom methodology clearly and concisely, paying special attention to tasks and resources both inside and outside the classroom (Alcaraz et al., 2020; Wanner and Palmer, 2015). In this sense, McNally et al. (2017) suggest that it is important for the teacher to incorporate a theoretical perspective of the methodology and include evaluation tools according to it, in addition to spending the entire school year and not isolated subjects, which can generate confusion among students.
In the field of music, we find experiences and research such as those of Wang (2018), who applied the FCM to collective piano classes. Brownlow (2017) developed an experience with his Music History students, videotaping his classes on stylistic analysis of musical works and getting much more time in class to guide and attend the different work groups. A study by Doi (2016) applied the FCM in a course on musical bibliographic research methods and measured the perception of students about the FCM. A pre-test indicated a preference for face-to-face classes (91% of the students) with practical activities as the preferred learning method, with little acceptance of video viewing. The post-test -after working with the FCM-increased the students’ preference regarding the visualization of the classes recorded on video from 9% to 31.25%.
In the field of studying a musical instrument, a study by Akbel (2018) on cello learning used, among other data collection tools, the video recordings of pre and post student performances. The results highlighted that the students, thanked to the teacher's video recordings, showed superior performance of the FCM over the traditional model. In a similar vein, Sever and Sever (2017) developed an investigation with piano students, based on a research design with pre and post-test (rubrics with variables such as tempo, rhythm, melody and finger technique) of a single group. The students' self-assessment scores and the expert assessment were examined, the latter consisting of the assessment of the students' videotaped performances. The data showed statistically significant differences between the scores issued by the students and teachers at the pre and post time.
In the Spanish university context, a systematic review carried out by Galindo-Domínguez and Bezanilla (2019) highlighted that, although there are not many universities that consider this methodology as a common part of their practices, from the 109 investigations carried out ‘almost half of the Spanish universities have had a first contact with it’ (bib_citation_to_be_resolvedGalindo-Domínguez and Bezanilla, 2019, p. 81). Most of these investigations -based on quantitative designs and techniques-were set as the main objectives to measure the FC model impact on the academic performance of students, and to know their perception and satisfaction with their own experience. From the studies carried out in the field of Education, only eight analysed academic performance and had a control and experimental group. All studies confirmed a positive impact of the FC model on the academic performance of students, at least as effective as with the traditional methodology, as shown by other university-level studies carried out in Spain (Mendaña et al., 2017; Poy et al., 2017).
The purpose of this study was to know students' perception about the flipped classroom model and to check if students’ academic performance improves with this model. To do this, the following research questions were posed:
-
1)
Can we achieve better academic results with the flipped classroom model compared to a more traditional and expository methodology?
-
2)
Is the flipped classroom model capable of improving learning climate, self-efficacy and cognitive engagement of students compared to a more traditional model?
-
3)
How do students perceive the flipped classroom model in terms of learning personalization, their ability to prepare the contents and exams of a course and in terms of the resources used?
3. Method
3.1. Design
The research has been carried out using a pre-experimental design with only post-test control group (Ato and Vallejo, 2015). The choice of this design is justified on the basis that the participants could not be randomly assigned -they were selected for their accessibility and lacked pre-test measures-. The main objective of this study was to analyse the impact that the flipped classroom model had on the students’ academic performance and the perception of the students about it. For this, in the study were analysed the variables of academic performance, as well as the perception of the students about the learning climate, motivation, self-efficacy, degree of commitment to study, and the resources used.
3.2. Participants and context
This study has been carried out in a Centre for Higher Studies in Seville. Fifty-one students have participated who were enrolled in the Didactics of Music in Early Childhood Education subject, a third-year subject of Early Childhood Education Degree. The educational centre of assignation was considered as the sampling unit. The participants were obtained according to a convenience non-random sampling, where the subjects were chosen for their accessibility and belonging to the natural groups established by the educational centre. All the participants were women, whose ages ranged from 21-23 years (87.5%) and 24–26 years (12.5%). At the time the survey was conducted, participants were informed that it would be anonymous, and no confidential data would be disclosed to third parties. Furthermore, Centre of Higher Education CEU Cardenal Spínola approved the study.
All aspects of this research were carried out in accordance with university ethical guidelines. The students were notified and gave their informed consent. The university granted ethical approval for it.
3.3. Data collection tools
For the data collection process, a questionnaire was designed derived from the following validated instruments: LCQ; R-SPQ-2F; MSLQ and the Feedback Questionnaire on Flipped Classroom Activity. From each of them, we have selected the blocks of items that best fit the purpose of this study. The questionnaire consisted of 63 items structured in seven dimensions: self-efficacy (SE = 9 items); learning climate (LC = 6 items); level of commitment to study (deep: D-COM-S = 10 and superficial: S-COM-S = 10 items); intrinsic value (IV = 9 items), methodology (MET = 10 items) and student-resource interaction (SRI = 9 items). The items followed a response format based on a Likert-type scale with five anchor points (1 = Strongly disagree; 5 = Fully agree).
Now we define the variables as contemplated in this study. Self-efficacy should be understood as the ‘ability to’ or ‘confidence in’, that is, the students' belief in what they are able to achieve according to their abilities (Williams and Rhodes, 2016). Learning climate should be understood as the students' perception of the teacher figure as support for their learning process. In this sense, learning climate is based on the principle that students work and learn better when they are free to express their own judgments, and the teacher must contribute to achieving that ideal state of confidence, especially in a scenario with little student presence. On the other hand, we took into account cognitive engagement (superficial and deep) and engagement with pre-class activities. Deep commitment refers to personal satisfaction with learning and the students' involvement in their training process. While superficial commitment responds to the need of some students to achieve academic results with minimal effort. The intrinsic value refers to the students' perception about the importance of the subject of study.
In addition, we considered important to collect information from the experimental group students about the main educational resource used (video), assessing the students’ commitment to its visualization, the advantage of videos to adapt to the work pace of each student, their ability to prepare students for face-to-face classes and other aspects such as their duration and their design. In the same way and in relation to the methodology, students valued aspects such as the materials organization and accessibility, the dynamics of classes and the spaces organization, as well as the teacher role as a guide in the process.
Finally, academic performance refers to the grades obtained by students in the different activities and exams carried out, according to a score of 0–10, with 10 being the highest grade.
The choice of the above variables has been conditioned by the exceptional situation of a group of students (experimental group) who have had to pass the academic year with half the number of face-to-face classes than the control group, hence the authors consider these variables as important for the study.
Other tools used to collect information are:
-
•
Edpuzzle: interaction of the students with the videos (viewing percentages, correct/incorrect answers and viewing time of the videos).
-
•
Google Forms: to collect information about pre-class content.
-
•
Moodle: management of complementary materials.
-
•
Exam, activities and development of the didactic unit: to collect the academic performance expressed in grades with scores from 0 to 10.
3.4. Procedure
The participating groups were assigned the control and experimental condition based on their greater or lesser presence in the classroom:
Control group (CG, n = 27): it participated in the project for 15 consecutive weeks spread over a four-month period (4 h of class per week).
Experimental group (EG, n = 24): 7 teaching weeks distributed throughout the semester (2 h of class per week).
The musical contents on which both groups have worked are structured in five blocks (Table 1).
Table 1.
Curricular content blocks of Didactics of Music in Early Childhood Education subject.
| Blocks | Contents |
|---|---|
| B1 | Music in childhood |
| B2 | Psychology of musical learning (0–6 years) |
| B3 | Didactic principles of hearing in Early Childhood Education |
| B4 | Didactic principles of vocal and instrumental training in Education |
| B5 | Didactic principles of musical creation in Early Childhood Education |
3.4.1. Control group (CG)
CG students used notes as the main study resource. These notes were mainly made up of book chapters, articles and other teaching materials provided by the teacher. During the face-to-face classes, they carried out the theoretical-practical activities, most of which served to reinforce and complement the subject contents. The main types of the developed activities were: analysis of didactic videos, creation and design of games, reading and commentary on articles and dramatizations, among others (Table 2). A relevant activity in this project was the didactic unit, as it globally included all the contents covered in the subject. Finally, students had to pass an exam consisting of multiple-choice questions, didactic analysis of videos and short questions. At the end of the subject, the control group completed the questionnaire Perception of the students of Didactics of Music in Early Childhood Education on the flipped classroom model.
Table 2.
Control group and experimental group sessions schedule.
| Week | Control group |
Experimental group |
|||
|---|---|---|---|---|---|
| Content blocks | Face-to-face activities | Content blocks | Pre-class activities | Face-to-face activities | |
| 1 | Bl.I | Viewing, and discussion, of videos about the importance of music in the children's stage | |||
| 2 | Bl. I | Role-playing | Subject presentation | Video tutorials + summary of Block I | |
| 3 | Bl. II | ||||
| 4 | Bl. II | Poster design with Canva tool | Bl. I |
Follow-up activities Questionnaire in Google Forms + Delivery of the summaries of block I. Theoretical-practical activities - Viewing, and discussion, of videos about the importance of music in the children's stage -Role-playing |
|
| 5 | Bl. II | Articles reading and review | Video tutorials + summary of Block II | ||
| 6 | Bl. III | Video analysis of different teaching music methods (Orff, Kodaly, and others). | Bl. II |
Follow-up activities -Questionnaire in Google Forms + Delivery of the summaries of block II. Theoretical-practical activities - Articles reading and review - Video analysis of different teaching music methods. |
|
| 7 | Bl. III | -Activities and games design about parameters of sound: pitch, duration, timbre and dynamics. -Activities design to develop music listening skills using active learning. |
Video tutorials + summary of Block III | ||
| 8 | Bl. IV | Bl. III |
Follow-up activities Questionnaire in Google Forms + Delivery of the summaries of block III. Theoretical-practical activities - Activities and games design about parameters of sound: pitch, duration, timbre and dynamics. - Activities design to develop music listening skills using active learning |
||
| 9 | Bl. IV | Video tutorials + summary of Block IV | |||
| 10 | Bl. IV | Bl. IV |
Follow-up activities -Questionnaire in Google Forms + Delivery of the summaries of block IV. Theoretical-practical activities -Activities design to develop vocal technique training with children -Song-teaching strategies and technique -Criteria for selecting songs for children |
||
| 11 | Bl. IV | -Activities design to develop vocal technique training with children -Song-teaching strategies and technique |
Video tutorials + summary of Block V | ||
| 12 | Bl. V | -Teaching strategies to work music creativity | Bl. V |
Follow-up activities -Questionnaire in Google Forms + Delivery of the summaries of block V. Theoretical-practical activities' -Teaching strategies to work music creativity |
|
| 13 | Development of didactic unit | Development of didactic unit | |||
| 14 | |||||
| 15 | - Exam - Questionnaire Perception of the students of Didactics of Music in Early Childhood Education on the flipped classroom model |
- Exam - Questionnaire Perception of the students of Didactics of Music in Early Childhood Education on the flipped classroom model |
|||
3.4.2. Experimental group (EG)
For its part, the EG's main study resource was video (33 videos). The video design was based entirely on the subject theoretical notes. They were edited with Camtasia (a screen capture tool), uploaded to YouTube and, later, hosted on the EdPuzzle platform, which made it possible to insert questionnaires in the videos and monitor their viewing (Figure 1). Like the control group, the EG carried out a didactic unit and an exam and, at the end of the subject, completed the questionnaire Perception of the students of Didactics of Music in Early Childhood Education on the flipped classroom model. Table 2 shows the temporal organization and distribution of contents and activities in the control and experimental groups.
Figure 1.
Edpuzzle screenshot (private web platform) of one of the videos used, where we observe the content block, a slide with information explained and developed by the teacher, and an interactive questionnaire within the video.
The methodology and structure of the activities carried out with the EG were different from those of the CG, since the EG had significantly fewer face-to-face sessions (Table 2). Thus, while the CG worked according to a traditional or expository methodology, the EG did so with a flipped classroom methodology. Regarding the activities, the CG carried them out mainly at home, while the EG (Table 2) carried out pre-class activities and face-to-face activities:
-
⁃
Pre-class activities: Before face-to-face session, students studied the videos corresponding to a content block, so they were better prepared for face-to-face session. In addition, the videos hosted on Edpuzzle allowed students to self-assess their learning. After watching the videos, students had to make a summary and deliver it to the teacher during the next face-to-face session for subsequent correction.
-
⁃
Face-to-face activities: In each face-to-face session, students had to deliver the corresponding block summary at the beginning of class and take a questionnaire in Google Forms (which has been considered in this study as follow-up activities). The rest of the session was used to carry out different theoretical-practical activities, always taking into account the reduced time available for the EG with respect to the CG
3.5. Data analysis
All the statistical analysis of the study has been carried out with various R system packages. First, the unidimensionality of the 7 questionnaire scales was tested in order to report reliability measures, using principal component analysis and varimax rotation with R's psych package, and then Cronbach's alpha and Guttman's lambda G6 reliability coefficients were calculated, once the items incompatible with unidimensionality were eliminated (Table 3). The initial set of 63 items was finally reduced to a set of 45 items, achieving a reliability in all scales above 0.75, which is considered the minimum level for a scale with satisfactory reliability.
Table 3.
Scale reliability analysis based on the questionnaire Perception of the students of didactics of music in Early Childhood Education on the flipped classroom model.
| Scale | Initial # of items | Final # of items | Alpha | G6(lambda) |
|---|---|---|---|---|
| I | 15 | 6 | .86 | .88 |
| II | 10 | 5 | .82 | .82 |
| III | 10 | 5 | .86 | .84 |
| IV | 9 | 5 | .90 | .89 |
| V | 9 | 6 | .88 | .88 |
| VI | 10 | 10 | .93 | .95 |
| VII | 7 | 7 | .94 | .96 |
With the refined scales, a descriptive analysis was applied, and all the statistical criteria required for the correct application of the statistical tests were tested. For this purpose, the psych package was also used.
In order to detect differences in the academic performance of both groups, we considered as response variables the weighted score of the final exam with or without covariates, and the students’ direct partial score, also with or without covariates. For the statistical analysis, several ANOVA and ANCOVA models were used for metric variables in all these cases, we also used adjusted R-squared coefficient both as effect-size measures and as more effective indices of the relationship between variables.
In order to know the students' perception of the methodology used, the variables of the questionnaire were tested with categorical models, considering that the Likert scale items are ordinal measures. Specifically, cumulative ordinal models with probit link were used for each item of all scales and multilevel cumulative ordinal models with probit link to control the dependencies between the ratings of the same participant and between the ratings of the same item. For this purpose, we used the brms package, which uses a multi-level Bayesian approach in R with the Stan programming language (Bürkner, 2017).
Three students in the control group refused to complete the questionnaires. For this reason, the initial sample was reduced to 48 participants (24 for the experimental group and 24 for the control group). However, almost all of the scores were completed for their participation in the educational process, including the partial exam score and the final weighted score. In this context, for academic performance, the sample consisted of 24 participants in the EG and 27 in the CG. Some missing scores that were detected in particular actidvities were considered missed completely at random (MCAR) and imputed using an FCS (Fully Conditional Specification) approach with the mice package (van Bureen and Groothuis-Oudshoorn, 2011).
4. Results
In relation to academic performance, the partial exam (EX) score (weighted by 35% for the final score) was used as a response measure to test differences between groups, as well as the resolution of activities (ACT) (weighted by 45% for the final score) and the development of the didactic unit (DU) (weighted by 20% for the final score).
Regarding the exam scores EX, no significant differences were found between groups, but there were in the activities, ACT: F(1; 49) = 35.7, p < .001 (adj R square = .41) and in the development of the didactic units, DU: F(1; 49) = 13.7 (adj. R square = .20). Controlling the two weighted tasks (ACT and DU), an ANCOVA model was applied that was statistically significant: F(3; 47) = 22.41; p < .001 (adj R square = .56), detecting significant differences between groups (p < .001) and between activities (p < .001), but not in the didactic units (p > .05). Lastly, weighted final scores (FG) were used as response variable to test the differences between groups using an ANOVA model, and significant differences were found between both groups: F(1; 49) = 10.9; p = .0018 (adj R square = .17).
Follow-up activities by groups were also object of statistical analysis. In the first place, we analysed whether each of the blocks of follow-up activities (F1 to F5) presented significant differences between groups: all obtained statistically significant differences between groups, favouring in all cases the experimental group over the control group (see Table 4). In addition, the five activity blocks individually explained the performance in the final score, with proportions of explained variance between .12 and .30, but only blocks 1, 2 and 5 individually explained the performance in the partial exam, with explained variance proportions between .05 and .15.
Table 4.
Descriptive and inferential analysis of follow-up activities.
| Group differences | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| n | mean | sd | median | trim | mad | se | F(1; 49) | p | PVE | |
| All the groups | ||||||||||
| F1∗ | 51 | 7.36 | 1.76 | 7.00 | 7.38 | 1.48 | 0.25 | 6.1 | .017 | .093 |
| F2 | 51 | 7.68 | 1.68 | 8.00 | 7.77 | 1.48 | 0.24 | 16.6 | <.001 | .237 |
| F3 | 51 | 6.99 | 1.55 | 7.00 | 6.96 | 1.48 | 0.22 | 8.1 | .006 | .125 |
| F4 | 51 | 6.47 | 1.36 | 6.00 | 6.43 | 1.48 | 0.19 | 5.0 | .030 | .078 |
| F5 | 51 | 7.52 | 1.63 | 8.00 | 7.56 | 1.48 | 0.23 | 9.8 | .003 | .150 |
| Experimental Group | |||||||
|---|---|---|---|---|---|---|---|
| F1 | 24 | 7.98 | 1.51 | 8.00 | 8.12 | 1.48 | .31 |
| F2 | 24 | 8.56 | 1.08 | 9.00 | 8.57 | 1.48 | .22 |
| F3 | 24 | 7.60 | 1.48 | 8.00 | 7.62 | 1.48 | .30 |
| F4 | 24 | 6.90 | 1.25 | 6.75 | 6.72 | 1.11 | .26 |
| F5 |
24 |
8.21 |
1.47 |
8.00 |
8.30 |
1.48 |
.30 |
| Control Group | |||||||
| F1 | 27 | 6.81 | 1.82 | 6.00 | 6.74 | 1.48 | .35 |
| F2 | 27 | 6.89 | 1.74 | 7.00 | 6.91 | 2.97 | .33 |
| F3 | 27 | 6.44 | 1.42 | 6.00 | 6.39 | 1.48 | .27 |
| F4 | 27 | 6.07 | 1.36 | 6.00 | 6.09 | 1.48 | .26 |
| F5 | 27 | 6.89 | 1.53 | 7.00 | 6.96 | 1.48 | .29 |
F1 to F5: Follow-up activities of the five content blocks.
Regarding the data from the questionnaire, the E1 scale (items 1–6) was intended to determine the learning climate perceived by the experimental group, taking into account that their presence was reduced by 75% in relation to the control group. With cumulative models for ordinal response with probit link applied to each of the items on the scale, only item 6 obtained statistical significance between groups, favouring the control group (p < .05, 95% CI: .02–1.55). With multilevel cumulative models, significant differences were found between groups, again favouring the control group (p < .05; CI: .14-.70).
With the S2 (items 7–11) and S3 (items 12–16) scales, the purpose was to evaluate the student's commitment to study. With cumulative models for ordinal response, only item 11 of the S2 scale (p = .05, 95% CI: .01–1.27) and item 13 of the S3 scale (p < .05, 95% CI: .04–1.26) obtained statistical significance, again favouring the control group, but the multilevel cumulative model did not reach statistical significance. None of the items on the S3 scale reached statistical significance.
The objective of the S4 scale (items 17–21) was to check the students' perception of their self-efficacy in the study. Almost all the items on the S4 scale approached statistical significance with the cumulative models for ordinal response, particularly items 20 (p < .05, 95% CI: .11–1.41) and 21 (p < .05, 95% CI: .02–1.25). Significant differences were also detected between groups with a multilevel cumulative model for ordinal response (p < .05, 95% CI: .32-.86), favouring the control group.
In scales S5 to S7 (items 22 to 44), no significant differences were detected between groups neither with the cumulative models for ordinal response nor with the multilevel cumulative models for ordinal response.
Finally, given the results obtained by the cumulative models on the S4 scale (self-efficacy), it was interesting to know in greater detail if there was a relationship between the students' academic performance and their belonging to one of the two groups, controlling their opinion on that scale. To this end, several ANCOVA models were applied, using the partial exam score or the final score as response variables, the group variable as a factor, and each of the items of the scale separately as a covariate. Table 5 presents a tabulation of the students’ ratings by item and group, and the probabilities of the F tests of the ANCOVA models for item and group. It should be noted that, with the partial score as the response variable, no significant differences were found between groups in any of the items, but significant differences in the covariate did appear for items i17, i18 and i19, with proportions of explained variance between 0.05 and 0.13. With the final score as response variable, significant differences were found between groups for each one of the items, but only a significant effect of the covariate was found for item i17, with proportions of explained variance ranging between .12 and .20.
Table 5.
ANCOVA with performance scores by group controlling each item of the Self-efficacy scale.
| Item |
Group |
Tabulation |
Partial exam F(2; 45) |
Final score F(2; 45) |
||||||
|---|---|---|---|---|---|---|---|---|---|---|
| p(item) |
p(group) |
p(item) |
p(group) |
|||||||
| Categories | 1 | 2 | 3 | 4 | 5 | |||||
| i17 | EG | 1 | 2 | 5 | 11 | 5 | <.005 | >.050 | <.050 | <.050 |
| CG | 1 | 1 | 4 | 6 | 12 | |||||
| i18 | EG | 1 | 0 | 7 | 11 | 5 | <.050 | >.050 | >.050 | <.050 |
| CG | 0 | 2 | 2 | 9 | 11 | |||||
| i19 | EG | 0 | 1 | 4 | 14 | 5 | <.050 | >.050 | >.050 | <.050 |
| CG | 0 | 1 | 4 | 5 | 14 | |||||
| i20 | EG | 1 | 5 | 13 | 4 | 1 | >.050 | >.050 | >.050 | <.050 |
| CG | 1 | 3 | 6 | 6 | 8 | |||||
| i21 | EG | 2 | 6 | 12 | 3 | 1 | >.050 | >.050 | >.050 | <.001 |
| CG | 1 | 3 | 10 | 7 | 3 | |||||
5. Discussion and conclusions
Based on the results of this research, firstly, we confirm a better academic performance of the students who followed the FCM. The results indicate a significant effect of the covariates and of the group on the ‘final score’ variable. Almost 91% of the variance of the final score is explained by the group, the activities and the didactic unit. These results -based on circumstances of reduced presence of the EG-coincide with previous research such as that of Baepler et al. (2014), who investigated whether less time in the classroom would translate into worse academic performance. For this purpose, the number of hours that students would spend in the classroom was reduced by two thirds, and they found that academic performance was statistically similar and even higher in some cases. In our study, although there were no statistically significant differences between groups in the exam scores, the general academic performance (weighing partial exam, activities and didactic unit) was higher in the experimental group, suggesting that, even with limited resources and time, the benefits of the flipped classroom model are quantifiable and significant.
In relation to learning climate, the results suggest that it may be conditioned by more or less presence in the classroom. There were statistically significant differences in favour of the control group in their perception of the teacher's concern for knowing their points of view before suggesting new ways of working, an aspect that the experimental group did not value to the same extent, probably due to face-to-face time limitation of the group. This limitation also affected the teacher, who had to develop the same practices as with the control group, but in a quarter of the time. This data allows us to suggest that still the students -despite the results obtained and having performed reasonably in a flipped classroom setting-continue to demand greater presence of the teacher as a guide during the educational process.
Regarding self-efficacy, it should be noted that the CG was perceived as significantly more self-effective than the EG. However, these data were not reflected in overall academic performance. While studies such as those by Nasir and Iqbal (2019) highlight that self-efficacy and academic performance correlate positively in a moderate way (being necessary to study other factors that mediate this relationship), in our study the opposite happened: the EG, despite being perceived less self-effective, showed higher overall academic performance. This could be explained considering that, when students expect lower results (in this case due to little presence in class and work with the FCM), they tend to try harder and to improve their performance and vice versa. This is consistent with studies such as those by Mooi (2007) and Christensen et al. (2002), who observed that students with lower self-efficacy scores achieved higher scores in final exams and in the final score of the course.
Regarding the students' cognitive commitment, we have only observed significant differences in favour of the CG in items 11 and 13 of the deep and superficial commitment scales, but without any impact on academic performance. However, there was statistically significant differences in favour of the EG in the follow-up activities, which is a very revealing result since these activities were based on pre-class tasks. This shows not only the students’ high motivation but also their firm commitment to these tasks, resulting in an improvement in academic performance, in line with studies such as those by Chuang et al. (2018), and Lee et al. (2018). This motivation and commitment to learning could be due to the fact that the FC model favours a better adaptation and personalization of the learning rhythm, as highlighted by Tse et al. (2019).
Although there were no significant differences between EG and CG in their perception of the methodology, it should be noted that the FCM has allowed a personalization of learning, which is one of the pedagogical dimensions proposed by Koh (2019) as a theoretical perspective that considers the FCM as a student-centred model. The FCM helped students prepare for their classes and exams, and this preparation has been reflected in their final scores, in line with recent studies such as those by Zainuddin (2018). One of the advantages of the FCM, as Akçayır and Akçayır (2018) point out, is that it favours students studying at their own pace, and at the most convenient time and place, aspects that have made it easier for a group with 75% less hours of class followed up on the subject in conditions similar to the face-to-face group.
Regarding the student-resource interaction, video as pre-class material (based on aspects such as content design, accessibility and adaptation to the learning rhythm, analogy with face-to-face class, duration, etc.) was highly valuated by the EG. This coincides with the results of previous investigations such as those by Doi (2016), whose study with pre-test/post-test highlighted the positive change in the preferences of students regarding the visualization of videos (9% before to 31.25% after). It should be noted that the EG -with 25% of face-to-face hours in relation to the CG-highlighted that the pre-class videos with the subject contents allowed them to attend classes being well-prepared, regardless of the academic performance mentioned above, in line with studies such as that by Porcaro et al. (2016).
We must emphasize that, from the teaching point of view, we have found ourselves in the need to create specific resources adapted to an unusual educational situation, trying to respond to a group of students with a very limited attendance situation. This has involved a huge effort and investment of time (organization, technology, etc.) that not all teachers are willing to assume, but otherwise it would have been practically impossible to implement this methodology with certain guarantees and results, as shown in this paper. In this sense, we agree with studies such as those by Alcaraz et al. (2020), who emphasize that in the FC model it is essential to select quality didactic resources and design specific materials if they are not available on the internet. This aspect can also be a fundamental key to motivate students, increase the level of commitment and improve their learning experience (Awidi and Paynter, 2019; Ryan &Reid, 2016; McNally et al., 2017), aspects that also were reflected have been reflected in our study.
Furthermore, in this study we have shown that, in order to achieve more active learning in the classroom, we need a two-way feedback between students and the teacher and vice versa, as well as links between activities outside and inside the classroom. Finally, the use of technology has been shown to be essential to transfer classes outside the classroom. We believe that the learning experience proposed herein can be generalized to other university contexts interested in implementing active and student-centred learning environments.
We can conclude that, on a theoretical level, the FC model has meant a paradigm shift in relation to the organization of the teaching-learning process. However, this shift is taking place in a very progressive way. Studies such as those by Freeman et al. (2014) highlighted that teaching in higher education has followed a traditional teacher-centred approach, with students assuming a passive role, which has made it more difficult for these students to develop their full potential throughout the school year and obtain better results in the final grades (He et al., 2016). However, in many cases, this pedagogical approach has been questioned by academics as it only implies a reorganization of the processes to be carried out inside and outside the classroom by students (O'Flaherty and Philips, 2015). Therefore, and on a practical level, as we showed in this study, the model must be enriched and improved to be more useful, adding complementary tasks to be carried out by both teachers and students, both inside and outside the classroom (Murillo- Zamorano et al., 2019), enabling access to digital resources designed for specific situations, with pre-class activities that contribute to better preparation of face-to-face classes, and establishing monitoring and evaluation systems to better understand the real impact of this methodological approach.
Finally, although the results obtained in this study are very satisfactory, we recognize our small sample as a limitation, since there were no more groups for its development. However, the teaching-learning setting with two groups of students with very different percentages of presence in the classroom amply justified the research, by highlighting the importance of the methodology used to respond to this unusual educational setting. By extension, the study results were supported by the high reliability of the scales used. In conclusion, this study is a good starting point for the application and transfer of the flipped classroom model in other educational centres with similar circumstances, which would serve to discover new findings, and as an additional validation process that could contribute to generalize the results obtained.
Declarations
Author contribution statement
José Palazón-Herrera: Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.
Alejandro Soria-Vílchez: Conceived and designed the experiments; Performed the experiments.
Funding statement
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Data availability statement
Data will be made available on request.
Declaration of interests statement
The authors declare no conflict of interest.
Additional information
No additional information is available for this paper.
References
- Abeysekera L., Dawson P. Motivation and cognitive load in the flipped classroom: definition, rationale and a call for research. High Educ. Res. Dev. 2015;34(1):1–14. [Google Scholar]
- Ahmed A.M., Osman M.E. The effectiveness of using WiziQ interaction platform on students’ achievement, motivation and attitudes. Turk. Online J. Dist. Educ. 2020;21(1):19–30. [Google Scholar]
- Akbel B.A. Students' and instructors' opinions on the implementation of flipped learning model for cello education in Turkish music. Journal of Education and Training Studies. 2018;6(8):1–11. [Google Scholar]
- Akçayır G., Akçayır M. The flipped classroom: a review of its advantages and challenges. Comput. Educ. 2018;126:334–345. [Google Scholar]
- Albert M., Beatty B.J. Flipping the classroom applications to curriculum redesign for an introduction to management course: Impact on grades. J Educ Bus. 2014;89(8):419–424. [Google Scholar]
- Alcaraz R., Martinez-Rodrigo A., Zangroniz R., Rieta J.J. Blending inverted lectures and laboratory experiments to improve learning in an introductory course in digital systems. IEEE Trans. Educ. 2020;63(3):144–154. [Google Scholar]
- Alegre A. Autoeficacia académica, autorregulación del aprendizaje y rendimiento académico en estudiantes universitarios iniciales. Propósitos y representaciones. 2014;2(1):79–120. [Google Scholar]
- Arguel A., Jamet E. Using video and static pictures to improve learning of procedural contents. Comput. Hum. Behav. 2009;25(2):354–359. [Google Scholar]
- Ato M., Vallejo G. Pirámide; 2015. Diseños de investigación en Psicología. [Google Scholar]
- Awidi I.T., Paynter M. The impact of a flipped classroom approach on student learning experience. Comput. Educ. 2019;128:269–283. [Google Scholar]
- Baepler P., Walker J.D., Driessen M. It's not about seat time: blending, flipping, and efficiency in active learning classrooms. Comput. Educ. 2014;78:227–236. [Google Scholar]
- Bandura A. Guide for constructing self-efficacy scales. In: Pajares F., Urdan T.C., editors. Self-efficacy Beliefs of Adolescents. Information Age; 2006. pp. 307–337. [Google Scholar]
- Bond M. Facilitating student engagement through the flipped learning approach in K-12: a systematic review. Comput. Educ. 2020;151:103819. [Google Scholar]
- Bravo E., Amante-Garcia B., Simo P., Enache M., Fernandez V. Proceedings of 2011 IEEE Global Engineering Education Conference. 2011. Video as a new teaching tool to increase student motivation; pp. 638–642. [Google Scholar]
- Brownlow A. Vol. 57. 2017. A new approach to Music History pedagogy using iPad technology and flipped learning. (College Music Symposium). [Google Scholar]
- Burke A.S., Fedorek B. Does “flipping” promote engagement? A comparison of a traditional, online, and flipped class. Act. Learn. High. Educ. 2017;18(1):11–24. [Google Scholar]
- Bürkner P.C. Brms: an R package for bayesian multilevel models using stan. J. Stat. Software. 2017;80(1):1–28. [Google Scholar]
- Chang S., Hwang G. Impacts of an augmented reality-based flipped learning guiding approach on students’ scientific project performance and perceptions. Comput. Educ. 2018;125:226–239. [Google Scholar]
- Christensen D., Barnes J., Rees D. Improving the writing skills of accounting students: an experiment. J. Coll. Teach. Learn. 2002;1(1):45–52. [Google Scholar]
- Chuang H.-H., Weng C.-Y., Chen C.-H. Which students benefit most from a flipped classroom approach to language learning? Br. J. Educ. Technol. 2018;49(1):56–68. [Google Scholar]
- Dilani S., John P., Wright N. An analysis of Moodle in facilitating asynchronous activities in a fully online university course. Int. J. Sci. Appl. Inf. Technol. 2013;2(2):6–10. [Google Scholar]
- Doi C. Applying the flipped classroom methodology in a first-year undergraduate Music Research Methods course. Music Ref Serv. Q. 2016;19(2):114–135. [Google Scholar]
- Freeman S., Eddy S.L., McDonough M., Smith M.K., Okoroafor N., Jordt H. Active learning increases student performance in science, engineering, and mathematics. Proc. Natl. Acad. Sci. Unit. States Am. 2014;111(23):8410–8415. doi: 10.1073/pnas.1319030111. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Froehlich D.E. Entornos de aprendizaje no tecnológicos en un mundo tecnológico: la inversión viene a ayudar. J. N. Approaches Educ. Res. 2018;7(2):94–99. [Google Scholar]
- Galindo-Domínguez H., Bezanilla M.J. Una revisión sistemática de la metodología flipped classroom a nivel universitario en España. Innoeduca. International Journal of Technology and Educational Innovation. 2019;5(1):81–90. [Google Scholar]
- Gligora M., Kliček B., Plantak Vukovac D. Proceedings of the 23rd International Conference on Information Systems Development. Faculty of Organization and Informatics; Varaždin: 2014. The effects of multimedia learning materials quality on knowledge acquisition; pp. 140–149. [Google Scholar]
- Guo P.J., Kim J., Rubin R. Proceedings of the FIrst ACM Conference on Learning. Scale Conference. 2014. How video production affects student engagement: an empirical study of MOOC videos; pp. 41–50. [Google Scholar]
- He W., Holton A., Farkas G., Warschauer M. The effects of flipped instruction on out-of-class study time, exam performance, and student perceptions. Learn. InStruct. 2016;45:61–71. [Google Scholar]
- Huang B., Hew K.F., Lo C.K. Investigating the effects of gamification-enhanced flipped learning on undergraduate students’ behavioral and cognitive engagement. Interact. Learn. Environ. 2019;27(8):1106–1126. [Google Scholar]
- Koh J.H.L. Four pedagogical dimensions for understanding flipped classroom practices in higher education: a systematic review. Educ. Sci. Theor. Pract. 2019;19(4):14–33. [Google Scholar]
- Lee G., Wallace A. Flipped learning in the English as a foreign language classroom: outcomes and perceptions. Tesol Q. 2017;52(1):62–84. [Google Scholar]
- Lee J., Park T., Davis R.O. What affects learner engagement in flipped learning and what predicts its outcomes? Br. J. Educ. Technol. 2018:1–18. n/a(n/a) [Google Scholar]
- Lo C.K., Lie C.W., Hew K.F. Applying ‘first principles of instruction’ as a design theory of the flipped classroom: findings from a collective study of four secondary school subjects. Comput. Educ. 2018;118:150–165. [Google Scholar]
- McGregor D. McGraw-Hill; 1994. El lado humano de las organizaciones. [Google Scholar]
- McNally B., Chipperfield J., Dorsett P., Del Fabbro L., Frommolt V., Goetz S.… Flipped classroom experiences: student preferences and flip strategy in a higher education context. High Educ. 2017;73(2):281–298. [Google Scholar]
- Mendaña C., Poy R., González A., Arana M.V., López E. ¿Influye el aula invertida en la motivación y el rendimiento académico de estudiantes universitarios? Infancia, Educación y Aprendizaje. 2017;3(2):660–666. [Google Scholar]
- Mengual-Andrés S., López Belmonte J., Fuentes Cabrera A., Pozo Sánchez S. Modelo estructural de factores extrínsecos influyentes en el flipped learning. Educación XX1. 2020;23(1):75–101. [Google Scholar]
- Mooi T.L. Self-efficacy and student performance in an accounting course. Masalah Pendidikan. 2007;30(2):33–48. [Google Scholar]
- Moral C., Pareja J.A. Dirección de clase. ¿Cómo construir un clima propicio para el aprendizaje? In: Moral C., editor. Didáctica. Teoría y práctica de la enseñanza. 2010. pp. 291–319. Pirámide. [Google Scholar]
- Murillo-Zamorano L.R., López Sánchez J.A., Godoy-Caballero A.L. How the flipped classroom affects knowledge, skills, and engagement in higher education: effects on students' satisfaction. Compute. Educ. 2019;141 [Google Scholar]
- Nasir M., Iqbal S. Academic self-efficacy as a predictor of academic achievement of students in pre-service teacher training programs. Bull. Educ. Res. 2019;41(1):33–42. [Google Scholar]
- Nel L. Students as collaborators in creating meaningful learning experiences in technology-enhanced classrooms: an engaged scholarship approach. Br. J. Educ. Technol. 2017;48(5):1131–1142. [Google Scholar]
- O'Flaherty J., Phillips C. The use of flipped classrooms in higher education: a scoping review. Internet High Educ. 2015;25:85–95. [Google Scholar]
- Önce A.F., Kara A. A flipped classroom in comunications systems: students perception and performance assessments. Int. J. Electr. Eng. 2019;56(3):208–221. [Google Scholar]
- Pareja J.A., Pedrosa B. Mejora de la convivencia a través de la investigación participativa. Profesorado. Revista de currículum y formación del profesorado. 2012;16(3):467–491. [Google Scholar]
- Plantak Vukovac D., Orehovački T., Novosel-Herceg T. Inspecting the quality of educational video artefacts employed in speech-language pathology telerehabilitation: a pilot study. In: Antona M., Stephanidis C., editors. Universal Access In Human-Computer Interaction. Users And Context Diversity. UAHCI 2016. Vol. 9739. Springer; Cham: 2016. (Lecture Notes in Computer Science). [Google Scholar]
- Pluta W.J., Richards B.F., Mutnick A. PBL and Beyond: trends in collaborative learning. Teach. Learn. Med. 2013;25(sup1):S9–S16. doi: 10.1080/10401334.2013.842917. [DOI] [PubMed] [Google Scholar]
- Porcaro P.A., Jackson D.E., McLaughlin P.M., O’Malley C.J. Curriculum design of a flipped classroom to enhance hematology learning. J. Sci. Educ. Technol. 2016;25(3):345–357. [Google Scholar]
- Poy R., Mendaña C., González A., Arana M.V., López E. Una experiencia aplicada de metodología de Flipped Classroom: incidencia en la motivación y en el rendimiento en estudiantes de Educación Superior. Infancia, Educación y Aprendizaje. 2017;3(2):174–179. [Google Scholar]
- Prada R., Hernández C.A., Gamboa A.A. Different scenarios for the teaching of mathematics with the support of virtual platforms: flipped classroom. J. Phys. 2019:1388. Conference Series. [Google Scholar]
- Ryan M.D., Reid S.A. Impact of the flipped classroom on student performance and retention: a parallel controlled study in general Chemistry. J. Chem. Educ. 2016;93(1):13–23. [Google Scholar]
- Sahin A., Cavlazoglu B., Zeytuncu Y.E. Flipping a college calculus course: a case study. Educ. Technol. Soc. 2015;18(3):142–152. [Google Scholar]
- Sever S., Sever G. Students' self-assessments regarding the effects of flip learning practice in Music education. Journal of Turkish Studies. 2017;12(18):505–522. [Google Scholar]
- Sletten S.R. Investigating flipped learning: student self-regulated learning, perceptions, and achievement in an Introductory Biology course. J. Sci. Educ. Technol. 2017;26(3):347–358. [Google Scholar]
- Strelan P., Osborn A., Palmer E. The flipped classroom: a meta-analysis of effects on student performance across disciplines and education levels. Educ. Res. Rev. 2020;30:100314. [Google Scholar]
- Thai N.T., De Wever B.D., Valcke M. The impact of a flipped classroom design on learning performance in higher education: looking for the best ‘blend’ of lectures and guiding questions with feedback. Comput. Educ. 2017;107:113–126. [Google Scholar]
- Tse W.S., Choi L.Y.A., Tang W.S. Effects of video-based flipped class instruction on subject reading motivation. Br. J. Educ. Technol. 2019;50(1):385–398. [Google Scholar]
- van Bureen S., Groothuis-Oudshoorn K. mice: multivariate imputation by chained equations in R. J. Stat. Software. 2011;45(4):1–67. [Google Scholar]
- Wang H. The application of flipped classroom in Colleges and Universities piano collective classes. Creativ. Educ. 2018;9:1021–1026. [Google Scholar]
- Wanner T., Palmer E. Personalising learning: exploring student and teacher perceptions about flexible learning and assessment in a flipped university course. Comput. Educ. 2015;88:354–369. [Google Scholar]
- Williams D.M., Rhodes R.E. The confounded self-efficacy construct: conceptual analysis and recommendations for future research. Health Psychol. Rev. 2016;10(2):113–128. doi: 10.1080/17437199.2014.941998. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Xiu Y., Moore M.E., Thompson P., French D.P. Student perceptions of lecture-capture video to facilitate learning in a flipped classroom. TechTrends. 2019;63(4):369–375. [Google Scholar]
- Yilmaz R. Exploring the role of e-learning readiness on student satisfaction and motivation in flipped classroom. Comput. Hum. Behav. 2017;70:251–260. [Google Scholar]
- Zainuddin Z. Students' learning performance and perceived motivation in gamified flipped-class instruction. Comput. Educ. 2018;126:75–88. [Google Scholar]
- Zainuddin Z., Haruna H., Li X., Zhang Y., Chu S. A systematic review of flipped classroom empirical evidence from different fields: what are the gaps and future trends? Horizon. 2019;27(2) [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Data will be made available on request.

