The role of pre-class and in-class behaviors in predicting learning performance and experience in flipped classrooms

Abstract

The flipped classroom is a well-recognized instructional approach that improves learning performance and experience through two sequential phases: pre-class preparation and in-class collaboration. Despite the large differences between the two phases, few studies have explored their separate effects on flipped learning. This study conducted a hierarchical linear regression to investigate how the two sequential phases and their featured behaviors can predict learning performance and experience in a flipped classroom at the undergraduate level. The results indicated that students' pre-class discussion positively predicted their assignment scores to a moderate degree, while the in-class collaboration was negatively correlated with test scores. However, the two phases had a limited impact on students' flipped learning experience. The results also highlight the influence of students’ initial interest and prior achievement on their flipped learning experience. The findings can extend our understanding of the flipped classroom approach and inform its design and implementation in higher education contexts.

1. Introduction

The flipped classroom is a student-centered instructional approach that leverages digital technologies such as video streaming and learning management systems (LMSs) to achieve effective integration of self-directed and instructor-facilitated learning activities [[1], [2], [3]]. Due to its proven effectiveness and operational flexibility [[4], [5], [6]], the flipped classroom approach has been widely utilized in higher education and emerged as an increasingly mainstream practice of blended learning in the post-COVID era [7,8], further obscuring the lines between traditional and online education [9]. A well-designed flipped classroom is known to yield various learning benefits, including superior learning performance, enhanced behavioral engagement, increased peer interaction, and improved learning experience [[10], [11], [12], [13]].

While there are various definitions of the flipped classroom in the literature [1,4,14], researchers in general agree that this approach consists of two sequential phases: pre-class preparation and in-class collaboration. The pre-class preparation usually takes place in an online learning environment enabled by an LMS where students engaged in self-regulated learning activities such as watching video lectures, reading assigned materials, taking quizzes, or contributing to asynchronous online discussions [[15], [16], [17]]. Recent literature suggests online discussion as a preferable pre-class learning activity because meaningful reflection and social interaction are known to promote active learning and group dynamic [4,18], which are essential for the ensuing in-class collaboration. Consequently, the current study focuses on pre-class discussion as the preparatory work for flipped learning.

The in-class collaboration of the flipped classroom occurs in brick-and-mortar classroom settings. In this phase, students engage in meaningful social dialogue and active knowledge construction through instructor-facilitated activities such as problem-solving, collaborative inquiry, and student presentations [10,13,19]. According to a scoping review by O'Flaherty and Phillips [10], common collaborative learning activities include solving-problems in groups, case-based activities, role-play simulation, joint assignment, peer assessment, and group presentation. In sum, owing to their distinct pedagogic purposes and contextual characteristics, the pre-class and in-class learning behaviors differ substantially in a flipped classroom.

Despite the substantial distinction between its two phases, the flipped classroom approach has often been investigated as a whole, without in-depth examination of the vast difference between pre-class and in-class behaviors and their varying effect on overall flipped learning outcomes and perceptions. Consequently, the existing findings on the benefits and limitations of flipped classrooms cannot be properly attributed at a more detailed level, and little is known regarding the priority of flipped learning behaviors and phases, as well as their intricate interrelationship. To address this research need, the present study conducted hierarchical linear regression analysis to investigate the roles of pre-class and in-class learning behaviors in predicting learning performance and experiences in a flipped classroom at the undergraduate level. The following questions guided our research inquiry:

• 1.
  What are the effects of pre-class and in-class instructional phases on students' flipped learning?

  • a)
    Does pre-class discussion significantly predict learning performance and experiences, and to what extent?
  • b)
    Does in-class collaboration significantly predict learning performance and experiences, and to what extent?
• 2.
  What are the effects of pre-class and in-class learning behaviors on students' flipped learning?

  • a)
    Which pre-class behaviors are significant predictors of learning performance or learning experience, and to what extent?
  • b)
    Which in-class behaviors are significant predictors of learning performance or learning experience, and to what extent?
• 3.What are the possible demographic factors that predict students' flipped learning performance and experience?

2. Literature review

2.1. The flipped classroom and its benefits

The literature in general supports the effectiveness of the flipped classroom as measured by improved learning performance and experience [4,10,20,21]. Students participating in flipped classrooms were often found to outperform those receiving traditional instruction in subject-content exams. For example, Lin [22] conducted an experiment in the context of engineering and showed that the flipped classroom can significantly increase students' final test scores by fifteen percent. Similar findings were reported in instructional contexts such as humanities [21], nursing [23], and chemistry education [24]. With its emphasis on active learning activities and meaningful student–instructor interactions during the class session [2,25], the flipped classroom is known to promote students’ higher-order thinking skills such as analysis, application, evaluation, and creation [26]. For example, Baepler et al. [27] revealed that students demonstrated superior problem-solving skills measured by three instructor-generated midterm tests during flipped learning, and they attributed such learning gain to the enhanced collaboration feature of the flipped classroom. Likewise, Jung et al. [28] associated flipped learning performance with an enhanced level of knowledge construction in the group among students.

In addition to learning performance, many research studies have identified improved learning experience as a major benefit of flipped classrooms. By providing students with more time for their self-study, as well as encouraging students to participate in interactive and higher-level activities, the flipped classroom increases students' school engagement and promotes learning satisfaction [29]. For example, Jamaludin and Osman [30] confirmed that the use of a flipped classroom in an instructional design class promoted active learning and enhanced students' behavioral, emotional, and cognitive engagement. Meanwhile, other studies have demonstrated that students exposed to a flipped classroom had a significantly higher level of satisfaction compared with those in a traditional classroom [31,32]. Enabling a customized learning pace during the pre-class preparation and timely instructor feedback during the in-class collaboration [16] can effectively promote students’ self-efficacy during flipped learning by reducing the perception of task difficulty and learning anxiety [31]. However, most of the literature fails to differentiate the pre-class and in-class behaviors when predicting learning gains in flipped classrooms.

2.2. Pre-class activities in the flipped classroom

The most common pre-class activity is watching lecture videos, some of which are created by instructors [13,33], and some are open educational resources [34,35]. However, simply imparting instructional content might not be sufficient to prepare students for in-class interaction in a flipped classroom [19]. Hence, researchers have explored the use of textual materials as pre-learning resources and achieved good results. For instance, Susantini et al. [36] designed and implemented an e-book to promote students’ metacognitive learning strategies and achieved high validity and readability. In addition, accomplishing the task assigned by teachers and conducting asynchronous online discussions were also common pre-class activities, including reporting the hardest topic and the questions to be clarified [14], answering the questions related to learning content [13], and posting in the discussion forum [37].

Several research studies have described the LMS as a learning space for pre-class learning activities, which provides a seamless, real-time, and non-invasive measure to gather pre-class behavioral data [37,38], including time spent watching videos or browsing the course site, the forum post length, and assignment logs. Students' LMS log data could allow the derivation and interpretation of their learning strategies for interactions in the design context [35], and DeLozier and Rhodes [19] also emphasized that the evaluation of flipped classrooms should be guided by objective measures, and the observable measurement could increase the predictive capabilities of learning performance [39]. In addition, research studies have measured students' readiness for pre-class self-study from the level of cognitive engagement by scoring the content of students’ forum posts [17,40].

2.3. In-class activities in flipped classroom

The literature indicates that the main form of in-class learning activity is group-based collaborative learning through social interaction and knowledge construction [41,42]. Based on constructivist learning theory, the flipped classroom emphasizes students' understanding and construction of knowledge in the process of dialogue and communication with others [43]. The literature mentions that common forms of collaborative learning in in-class include discussion [44], task/project completion [45], and joint inquiry [46,47]. For example, Zheng et al. [13] adopted the flipped classroom in a seventh-grade information technology class, and participants in the class were divided into groups and asked to collaborate on a DVD cover design task. As an internal cognitive process, knowledge construction reflects students’ higher-order thinking skills, such as understanding, evaluation, and creation of knowledge, which usually occur in overt discussions [48,49]. Wilhelm-Chapin and Koszalka [50] have suggested that students can complete knowledge construction in the process of drawing concept maps, knowledge bases, and infographics.

In the process of collaboration, one of the most common behaviors is interaction, which can include providing suggestions, giving feedback, and guiding thinking. When students engage in face-to-face discussion, one person is listening while the others express opinions, and this interaction is helpful for developing problem-solving skills [51]. Previous research has demonstrated that a high level of student–student interaction can improve the learning experience [52] and positively influence learning outcomes [53,54]. Researchers have also observed that leadership behavior, such as distributing tasks and making decisions, emerges from the collaborative process. It is confirmed that students who self-identify as leaders show higher academic performance and social self-efficacy [55]. Moreover, off-task behaviors can also occur during collaboration [38], such as looking around or having sluggish eyes; this can be considered irrelevant behavior. It has been revealed that irrelevant behaviors have negative impacts on learning gains and may be related to students’ negative emotions, such as boredom and frustration [56,57].

2.4. Individual differences in flipped classrooms

The impact of individual difference on learning has been recognized in the literature [58,59]. In particular, learner characteristics such as learning interest and prior academic achievement are known to be important predictors of learning performance and experience. Research studies have identified learning interest as a key construct affecting learning motivation and facilitating cognitive function [60], as well as being associated with academic achievement [61]. High interest may evoke fuller, more adequate, and more creative responses, which can promote more interaction behaviors among students in a class [62].

In addition, academic achievement—as represented by grade point average (GPA) or standardized exam scores, such as the ACT (American College Testing)—not only is able to predict learning performance in traditional classrooms [63] but has also been shown to be a powerful factor affecting success in flipped classrooms [64]. Research studies have demonstrated that, all things being equal, students with higher GPAs may have better study skills, better time-management skills, more motivation, and greater self-regulated skills than students with a lower GPA in technology-enhanced learning environments [65,66]. Consequently, student characteristics such as learning interest and prior GPA were also included as possible predictors of flipped learning in this study.

3. Method

3.1. Participants

The participants in this study were 44 first-year undergraduate students (age: 18–20) majoring in educational technology from Central China Normal University, a top-tier research university in China. Of the participants, 25% were male (n = 11), and 75% were female (n = 33). The mean value of the participants’ GPA in the last school semester was 82.32 (SD = 4.100, Max: 90, Min: 73), suggesting moderate variance among the participants. Participants voluntarily selected their group members, and each group consisted of 5–7 participants.

This study was conducted in accordance with the ethical standards of the Helsinki Declaration. The research protocol and instruments of the present study were reviewed and approved by the Institutional Review Board of Central China Normal University (CCNU-IRB-202103019, approved on 2021/03/16). All participants provided their written informed consent to participate in this study. They were made aware that their participation was completely voluntary, and their personal identifiable information would not appear in any future publications and presentations.

3.2. Research context and procedure

The research study was implemented in the Introduction to Educational Technology course in the spring semester of the undergraduate course. Introduction to Educational Technology focuses on learning theory and instructional design, emphasizing students' understanding and construction of knowledge. The fourth week's lesson employed the flipped classroom approach, which included pre-class self-study, in-class group-based collaborative learning, and after-class testing, as shown in Fig. 1.

Fig. 1.

The flipped classroom process.

The participants were required to complete the pre-class self-study on the Xiaoya platform (http://www.ai-augmented.com/), an LMS that supports a variety of teaching services. The learning content includes two video clips (12 min in total) and five relevant papers uploaded by the instructor. After completing the self-study, participants were required to post their views and reflections on the study materials in the LMS.

In-class activities included two parts: The first part was group collaborative learning. Before the start of the in-class collaboration learning, the research team placed a camera and a microphone in the best shooting position for each group and read out instructions to the participants, informing them that the activities in this class would be filmed but that their learning performance would not affect their grades in the course. The instructor listed 45 key terms related to educational technology, which were classified into four categories: representative scholars (12 items), instructional theories (9 items), instructional media (15 items), and landmark events (9 items). Each group selected a category and discussed how to present these keywords in chronological order. In the second part, the instructor randomly selected four groups responsible for different themes to present their work to the whole class.

After the class, students had one week to review the content, and made a mind map around the relevant content, which they then submitted to the Xiaoya platform. We also had students take a knowledge test and complete the Learning Experience Questionnaire (LEQ) prior to the next class.

3.3. Data collection and analysis

3.3.1. Predictors

This study collected learner characteristics as control variables, and pre-class and in-class behaviors as predictors of flipped learning. The key information for the research variables, such as measuring constructs, operational definitions, and data sources, is listed in Table 1. Learner characteristic variables included students’ interests and self-reported prior GPA, which were collected by the LEQ developed by the research team. The pre-class behaviors included post length and post quality, which were collected from the forum logfile. The in-class behaviors such as interaction and irrelevant actions were captured by the video cameras.

Table 1.

The measurement of learner characteristics, pre-class behaviors, and in-class behaviors.

Construct	Variables	Definition	Source
Learner characteristics	Interests	Learner interest in the educational technology major	LEQ
	Prior GPA	Participant's GPA in the previous academic year	LEQ
Pre-class behaviors	Post length	Number of words posted	Logfile
	Post quality	Rating of post quality for accuracy and reflection	Logfile
In-class behaviors	Interaction behaviors	Number of verbal (conversation) and nonverbal (gesture, eye contact, etc.) interactions	Video recording
	Irrelevant behaviors	Actions irrelevant to learning task (disruption, distraction, chitchat, etc.)	Video recording

Note: LEQ: learning experience questionnaire; GPA: grade point average.

For the pre-class behaviors, the first and third authors counted the length of the posts and rated the posts on the criteria of both accuracy and reflective, each accounted for 50% of the total score (100). Accuracy means that the post accurately answers the discussion question, and reflexivity means that the post not only shows mastery of the knowledge, but also provides reflective analysis and commentary. For example, when asked about the key stages of educational technology, the following post received 90 rating because it correctly identified the development stages in chronological order and included the following reflective argument: “I think the process and results of teaching are very important, and we should always look at the whole process of teaching from a holistic, comprehensive and developing perspective”. Contrarily, if a post simply described the timeline of technology development without commentary would receive zero score on the criterion of reflexivity. After ensuring that the internal consistency coefficient reached above 0.9, the average of the two researchers' scores was used as the post's final score.

We used video cameras to record the whole process of student discussions to examine in-class behaviors. The video capture device used was a DJI POCKET 2, which has 64 megapixels and 4 K resolution, and it works with a microphone to ensure the video and sound are sufficiently clear. The video encoding process consisted of two phases. In the first phase (March–April 2022), we divided the video recordings into 1-min segments, forming the units of analysis, and each 1-min segment was divided into 15-s minimum coding units, because most learning events can be captured in such segments, while the workload remains manageable for manual coding. When participants engaged in one of the target behaviors, we observed the duration (T) and calculated the times (n) according to the following equation: n = T/15 s (remainder = 0), or n = T/15 s + 1 (remainder ≠ 0). For example, if a student continuously talked with others for 17 s, the time for “interaction behaviors” was coded as 2, while if a student remained distracted for 15 s, the time for “irrelevant behaviors” was coded as 1. The total length of the video was 30 min, which was divided into 30 encoded segments.

Using the coding framework in Table 1, the first and third authors freely encoded approximately 30% of the video segments with the aim of further validating and revising the tentative encoding protocol. After completing the encoding protocol, we launched the second phase (April–May 2022), recruiting 14 undergraduate students as volunteers to encode the video segments. Volunteer coders all received 2 h of rigorous coding training and passed coding assessments based on a 15-min sample video. Each video segment was coded by two persons to ensure inter-rater reliability. Any controversial issues and disagreements that arose during the coding process were resolved through weekly discussions by the research team. Upon reaching satisfactory reliability, the mean score of the two coders was used as the final encoding statistic.

3.3.2. Outcome variables

In this study, learning performance and learning experience were selected as the outcome variables. Learning performance was comprehensively evaluated by the multiple-choice knowledge test score and individual assignment score. Informed by the Bloom's taxonomy of cognitive learning objectives [67], the knowledge test was designed to measure participants' lower-order thinking such as recall and comprehension while the assignment to measure higher-order thinking such as analysis and creation. Meanwhile, participants' learning experience was measured in four dimensions (i.e., behavioral engagement, affective engagement, cognitive engagement, and learning satisfaction). It is important to know that the data of all outcome variables were collected at the end of the flipped learning, after the in-class collaboration session. The definition and instruments are listed in Table 2.

Table 2.

The measurement of learning performance and experience.

Construct	Variables	Definition	Instrument
Learning performance	Test score	Score received in a 10-item multiple-choice test	Knowledge test
	Assignment score	Mind map quality rating based on the four evaluation criteria	Mind map
Learning experience	Behavioral engagement	Behavioral participation in pre-class activities and in-class collaboration	LEQ
	Affective engagement	Mean rating of emotion and attitude towards pre- and in-class phases	LEQ
	Cognitive engagement	Mean rating of all items in this sub-scale of engagement	LEQ
	Deep engagement	Perceived engagement in higher-order thinking activities
	Shallow engagement	Perceived engagement in recall and comprehension activities
	Learning satisfaction	Students' evaluation of their flipped learning experience	LEQ

Note: LEQ: learning experience questionnaire.

The knowledge test aimed to examine students’ recall and comprehension of key concepts from the lectures and reading materials, which consisted of 10 multiple-choice questions on the instructional topic (the history of educational technology). The test items were developed by the researchers and revised by two experienced teachers to ensure its test validity.

The mind map assignment was based on the collaborative learning task during the in-class session, which required individual participants to arrange historical events of educational technology (e.g., scholars, events, theories, and technologies) in the correct chronological order depict their proper relations. This assignment was designed from the perspective of generative learning to measure students' reconstruction of knowledge and synthesizing ability. For the assignment score, the first and third authors rated the assignments based on four criteria: completeness, accuracy, integration, and divergence. Completeness refers to whether the mind map involves all key historical events as instructed; Accuracy refers to whether the mind map correctly depicts the chronological order and the relations of the events; Integration means that the mind map is an integrated conceptual presentation of the historical elements rather than a simple list of them; Divergence refers to the degree of variety and originality of the relations and classifications in the mind map. After ensuring that the internal consistency coefficient reached above 0.9, the average of the two researchers’ scores was used as the assignment final score.

The LEQ for the flipped classroom consisted of 36 items (including 30 items scored on a 5-point Likert scale), informed by learning engagement, including four dimensions: behavioral engagement [68], affection [69], cognitive engagement [70], and learning satisfaction (See Appendix A). The total Cronbach's alpha value of the LEQ was 0.955, and Cronbach's alpha values of the first and second parts of the LEQ were 0.946 and 0.844, respectively. Such results support the reliability of the LEQ instrument, showing that it can consistently measure learning experiences in the flipped classroom. The LEQ was created using an online survey tool and could be accessed with mobile devices.

3.3.3. Data analysis

The two main data analysis methods utilized in this study were Pearson's correlation and hierarchical linear regression. We first explored the relationship among variables using Pearson's correlation, and then screened potential variables to be entered into the hierarchical linear regression model. To answer the three research questions, we constructed three hierarchical linear regression models with test score, assignment score, and learning experience rating as the dependent variable for each model, respectively. We put the learner characteristics as control variables in the first block of the regression model and put the pre-class and in-class behaviors variables as independent variables in the second and third blocks, respectively. In addition, we conducted a series of analyses such as Levene's test, the VIF (Variance Inflation Factor) and tolerance values, and Durbin–Watson statistics, to ensure all statistical assumptions of linear regression were met. Data analysis was conducted using IBM SPSS software (version 25).

4. Results

4.1. Descriptive and correlational results

Correlation analysis was performed to determine the relationships among learner characteristic variables, pre-class behavior variables, in-class behavior variables, learning performance, and learning experience. The preliminary results for the descriptive statistics and Person's correlation coefficients are shown in Table 3. In terms of learning performance, prior GPA, post quality, and irrelevant behaviors were not correlated with test scores, while interest was positively significantly correlated with test score (r = 0.340), and post length and interaction behaviors were significantly negatively correlated with test score (r = −0.321). In addition, only the post quality was significantly positively related to the assignment score (r = 0.565). For learning experience, interest and prior GPA were significantly positively correlated with learning engagement (r = 0.537; r = 0.426), while prior GPA was negatively correlated with learning satisfaction (r = −0.346). It is important to note that no significant correlation was found between the test score and the assignment score. A possible explanation is that the knowledge test and mind map assignment were designed to measure different types of learning performance, with the former focusing on lower-order thinking and the latter on higher-order thinking.

Table 3.

Correlation analysis between variables.

	Mean	SD	1	2	3	4	5	6	7	8	9	10
1	3.390	0.579	1	0.002	−0.027	−0.319*	−0.322*	−0.048	0.340*	−0.238	0.539***	0.426**
2	82.323	4.100		1	−0.126	−0.082	- 0.097	0.047	0.226	−0.048	−0.270	−0.346*
3	691	354.072			1	0.654***	0.205	0.136	−0.321*	0.254	0.032	0.114
4	73.466	13.321				1	0.213	0.035	−0.234	0.565***	−0.243	0.154
5	10.886	7.879					1	−0.005	−0.446**	0.088	0.069	0.048
6	0.591	1.304						1	−0.260	0.008	−0.234	−0.093
7	27.61	7.738							1	−0.171	0.092	0.043
8	75.443	17.983								1	−0.221	0.222
9	3.961	0.437									1	0.721**
10	3.671	0.590										1

Note: ^a1: Interest, 2: Prior GPA, 3: Post length, 4: Post quality, 5: Interaction behaviors, 6: Irrelevant behaviors, 7: Test score, 8: Assignment score, 9: Learning engagement, 10: learning satisfaction; ^b*p < 0.05, ***p < 0.001.

4.2. Predictors of learning performance

Hierarchical linear regression was conducted to investigate the predictive effect of pre-class and in-class behaviors on learning performance. As shown in Table 4, when the outcome variable was the test score, the predictive ability of the final model was 28.8%, and learner characteristics and classroom behavior had significant predictive effects on test scores (Adj. R² = 0.126, p < 0.05; Adj. R² = 0.288, p < 0.05). In addition, it was found that interaction behaviors significantly predicted test scores (β = −0.315, p < 0.05).

Table 4.

Hierarchical linear regression analysis results of test score.

	Factors	Adjusted R2	ΔR2	ΔF	SE	β	t
Learner characteristics	Interests	0.126	0.166	4.088*	1.980	0.257	1.736
	Prior GPA				0.048	0.180	1.383
Pre-class behaviors	Post length	0.184	0.094	2.469	0.004	−0.272	−1.496
	Post quality				0.108	0.116	0.624
In-class behaviors	Interaction behaviors	0.288	0.127	3.849*	0.139	−0.315	−2.259*
	Irrelevant behaviors				0.778	−0.225	−1.718

Note: *p < 0.05, ***p < 0.001; SE: Standard Error.

Table 5 presents the regression model results for learner characteristics, pre-class behaviors, and in-class behaviors on assignment scores. As the table indicates, the predictor of this model is very different from the prediction model for test scores. In particular, learner characteristics and in-class behaviors had no predictive ability for the assignment score. By contrast, when pre-class behavior was included, the explained variance was 27.7% (Adj. R² = 0.277, p < 0.001), reaching a statistically significant level; when in-class behavior was included, the explained variance dropped to 23.9%, indicating that in-class behavior is not a good predictor of assignment scores. The result suggests that post quality could significantly predict assignment scores (β = 0.686, p < 0.001).

Table 5.

Hierarchical linear regression analysis results of assignment score.

	Factors	Adjusted R2	ΔR2	ΔF	SE	β	t
Learner characteristics	Interests	0.013	0.059	1.281	4.758	−0.034	−0.221
	Prior GPA				0.115	−0.020	−0.146
Pre-class behaviors	Post length	0.277	0.285	8.484***	0.010	−0.193	−1.026
	Post quality				0.260	0.686	3.556***
In-class behaviors	Interaction behaviors	0.239	0.001	0.027	0.333	−0.032	−0.220
	Irrelevant behaviors				1.869	0.009	0.069

Note: *p < 0.05, ***p < 0.001; SE: Standard Error.

4.3. Predictors of learning experience

Hierarchical linear regression analysis was used to predict the learner characteristics, pre-class discussion, and in-class collaboration on learning experience. It can be seen from the data in Table 6 that learner characteristics and in-class behaviors could significantly predict learning engagement, with a predictive ability of 33.3% and 39.4% (Adj. R² = 0.333, p < 0.000; Adj. R² = 0.394, p < 0.000), respectively. Interests could positively significantly predict learning experience (β = 0.536, p < 0.000); however, neither pre-class nor in-class behaviors predicted learning engagement.

Table 6.

Hierarchical linear regression analysis results of learning engagement.

	Factors	Adjusted R2	ΔR2	ΔF	SE	β	t
Learner characteristics	Interests	0.333	0.364	11.726***	0.103	0.536	3.922***
	Prior GPA				0.002	−0.239	−2.239
Pre-class behaviors	Post length	0.322	0.021	0.826	0	0.146	0.870
	Post quality				0.006	−0.230	−1.334
In-class behaviors	Interaction behaviors	0.394	0.094	3.323*	0.007	0.237	1.839
	Irrelevant behaviors				0.081	−0.207	−1.715

Note: *p < 0.05, ***p < 0.001.

When the outcome variable was learning satisfaction, only learner characteristics had a significant predictive effect (Adj. R² = 0.268, p < 0.000). Table 7 indicates that students' interest in their major and prior GPA could significantly predict learning satisfaction. The higher the students’ interest, the higher the learning satisfaction (β = 0.410, p < 0.000), while the higher the prior GPA, the lower the learning satisfaction (β = −0.321, p < 0.05).

Table 7.

Hierarchical linear regression analysis results of learning satisfaction.

	Factors	Adjusted R2	ΔR2	ΔF	SE	β	t
Learner characteristics	Interests	0.268	0.302	8.870 ***	0.155	0.410	2.697***
	Prior GPA				0.004	−0.321	−2.398*
Pre-class behaviors	Post length	0.260	0.027	0.786	0	0.202	1.081
	Post quality				0.008	−0.211	−1.105
In-class behaviors	Interaction behaviors	0.251	0.027	0.767	0.011	0.153	1.066
	Irrelevant behaviors				0.122	−0.078	−0.578

Note: *p < 0.05, ***p < 0.001; SE: Standard Error.

5. Discussion and conclusion

This study investigated the roles of the two phases of the flipped classroom—pre-class discussion and in-class collaboration—in influencing learning performance and learning experience. The results revealed that pre-class discussion can significantly predict students' assignment scores, and the quality of students' posts in the forum had a significant impact on their assignment scores. However, in-class collaboration could significantly predict students' test scores and learning experiences, and interaction behavior was a significant predictor of test scores. In addition, we examined the influence of learner characteristics, and the results demonstrated that students’ interest in the major and their prior GPA could significantly affect learning performance and experience. This section provides theoretical explanations for the research findings and proposes meaningful practical guidance for the implementation of flipped classrooms.

5.1. Flipped learning phases and their predictive capacity

Both pre-class discussion and in-class collaboration had a limited impact on flipped learning performance in this study, and their impact varied with the different types of learning outcomes. Consistent with a previous study [64], pre-class discussion was found to significantly predict the final learning outcome measured by assignment quality rating. Particularly, the mind map creation assignment indicated that students’ generative learning performance was largely owing to quality pre-class discussion behaviors rather than their social interaction during in-class activities. A possible explanation is that, in the process of in-class collaboration, students tended to focus on the hasty completion of the task at hand due to the limited class time, thus lacking higher-order thinking opportunities such as synthesis and reflection. During the pre-class phase, by contrast, students were able to study the learning materials independently and repeatedly, and active engagement with the learning resources is known to promote generative learning outcomes [50].

Interestingly, inconsistent with previous findings [12], the study results revealed that the in-class collaboration only negatively predicted test scores, indicating students who were more active in in-class collaboration may have lower test scores. One possible reason is that several students only focused on the collaboration process, such as agreeing on the division of labor and determining the presenter, but lacked attention to the knowledge content, thus achieving superficial rather than meaningful collaboration. In addition, because of the 1-h time limit for the in-class collaboration, there was insufficient time for students to internalize the constructed knowledge. Another possible explanation is that the in-class collaboration suffered from instances of irrelevant behaviors such as idle chitchat and mobile phone surfing. The prevalence of such behaviors is known to have a detrimental influence on collaboration performance [71].

Compared with flipped learning performance, students' flipped learning experiences were rarely determined by either of the flipped learning phases (pre-class and in-class). The only exception was that students' in-class collaboration was found to increase the capacity for predicting learning engagement to a small degree (ΔR² = 0.094). One possible reason is that students’ participation in forum discussions was shallow, featuring “essay-type posting” [72] with insufficient meaningful interactions among peers, which is considered a key factor to improve the learning experience [73]. Another explanation could be that although there are many bystanders in the process of in-class collaboration, they can still acquire good learning experience through observing or listening without the pressure of actively participating in group tasks and conversations. Contrarily, active learners might be more anxious to complete the task at hand and thus report inferior learning experience.

5.2. Specific behaviors as flipped learning predictors

In addition, we examined the influence of specific behaviors on flipped learning and identified pre-class post quality and in-class interaction behaviors as important predictors of flipped learning performance. Students who generated high-quality posts during pre-class discussion were more likely to submit a high-quality assignment, suggesting that it is quality rather than quantity of pre-class forum discussion that promotes higher-order learning outcomes [72]. According to Chen and Yeh [74], post quality is a good indicator of a student's willingness and agency to engage in active learning, and students with those traits also tend to hand in high-quality assignments.

One unanticipated finding was the negative correlation between students’ in-class interaction behaviors and their test performance. This correlation might be explained by cognitive load theory [75]: social interactions for the purpose of coordination and management are considered extraneous cognitive load, the increase in which reduces germane cognitive load for processing and rehearsing new information, thus causing poor performance in knowledge recall and retention as measured by objective testing. In addition, the leader with more interaction behaviors does not guarantee improved learning performance, as the added responsibility might negatively affect knowledge acquisition.

5.3. The importance of individual differences in the flipped classroom

Another interesting finding is that learning interest and prior academic achievement significantly predicted learning experience. Students with high interest in learning showed more engagement and higher satisfaction. According to Ning and Downing [76], students with high learning interest generally have high intrinsic learning motivation, and enhanced learning motivation is known to result in improved learning experience. Students with higher prior academic achievement, however, reported worse learning experience marked by lower learning satisfaction. A likely reason is that high-achieving students are more demanding and have higher expectations of the course, while low-achieving students benefit more from cooperation and collaborative practice and thus perceive their flipped learning experience more positively (Sergis et al., 2018).

5.4. Practical implications

Based on the research findings, we propose several implications for creating effective flipped classrooms. First, in the process of implementing a flipped classroom, the importance of pre-class discussion should be emphasized for better learning outcomes. Teachers need to provide well-designed self-study materials and tasks to enhance the quality of pre-class learning, which can lead to better preparation for in-class activities. Second, teachers should be aware of the potential negative effects of excessive social interaction during the in-class learning process, as it might prove to be a distraction to certain students, such as group leaders, who might spend too much time on coordination instead of learning. Third, instructors should seek to diminish the influence of individual differences during flipped learning. Special attention needs to be paid to students with low learning interest and high academic achievement, as they are likely to withdraw from the flipped learning process cognitively due to their negative learning experience.

5.5. Limitations and future research

There are several limitations to this study. First, the participants were selected from a single course offered by a top-tier university using convenience sampling, of which the gender ratio and overall high learning achievement level suggest that this sample might not be representative of the entire college student population, thus undermining the generalizability of the study results. Second, due to the small sample size, the number of predictors in the regression model was limited; the small sample size thus reduced the statistical power of our analysis in this study, making it prone to Type II error. Third, the research instruments for measuring learning performance (i.e., the knowledge test and the assignment grading rubric) were developed specifically for the course content. While the preliminary evidence supports their inter-rater reliability and content validity, the criterion and construct validity need to be further verified. Finally, the pre-class and in-class behaviors occurred sequentially, and their possible synergistic effects on flipped learning cannot be ruled out. Consequently, we recommend conducting future research in varied instructional contexts with more accurate flipped learning constructs, more diversified empirical data, and more advanced statistical methods to enhance the credibility, generalizability, and interpretability of the study results.

6. Conclusion

This study differentiated the impact of two flipped learning phases, pre-class discussion and in-class collaboration, on college students' learning performance and experience in a flipped classroom. Several specific learning behaviors and learner characteristics were identified as significant predictors of student performance and experience, including post quality, interaction behaviors, learning interest, and academic achievement. The study results confirm the importance of pre-class preparation for flipped learning performance, prompting us to re-evaluate the role of in-class collaboration on the flipped learning experience and highlighting the importance of students’ initial interest and prior achievement as confounding factors.

Author contribution statement

Tianjiao Chen: Performed the experiments; Analyzed and interpreted the data; Wrote the paper.

Heng Luo: Conceived and designed the experiments; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Peiyu Wang: Performed the experiments; Analyzed and interpreted the data.

Xin Yin, Jiaxin Yang: Analyzed and interpreted the data.

Funding statement

This work was supported by the “AI + Education” Innovative Teaching Research Grant of Central China Normal University, China, grant number 2022XY017; Teacher Education Specialized Grant of Central China Normal University, China, grant number CCNUTEIII 2021–10; and Hubei Provincial Teaching and Research Project for Higher Education, China, grant number 2021085.

Data availability statement

Data will be made available on request.

Additional information

No additional information is available for this paper.

Declaration of interest's statement

The authors declare no conflict of interest.

Appendix A. Learning Experience Questionnaire

Part One: Basic Information

• 1.Name:
• 2.Gender:
• 3.Age:
• 4.Student ID:
• 5.Prior GPA:
• 6.
  I am very interested in educational technology ( )

  • A.
    Strongly disagree
  • B.
    Disagree
  • C.
    In general
  • D.
    Agree
  • E.
    Strongly agree

Part Two

This section is about views on flipped classroom activities. There are 30 question descriptions below, ranging from “strongly disagree-strongly agree” to 5 levels, please choose according to your actual situation.

Behavioral engagement

• 1.I actively participated in the pre-class discussion.
• 2.I actively participated in the in-class collaboration.
• 3.I completed the pre-class discussion on time.
• 4.I completed the in-class collaboration on time.
• 5.During the pre-class discussions, I interacted with other group members.
• 6.During the in-class collaboration, I interacted with other group members.

Affective engagement

• 1.I was interested in the pre-class discussion.
• 2.I was interested in the in-class collaboration.
• 3.I had a lot of fun during the pre-class discussions.
• 4.I had a lot of fun during the in-class collaboration.
• 5.Regardless the difficulty level and learning task of the pre-class discussion, I believed that I could complete it efficiently.
• 6.Regardless the difficulty level and learning task of the in-class collaboration, I believed that I could complete it efficiently.
• 7.I hope that my participation in the pre-class discussion can benefit my group members.
• 8.I hope that my participation in the in-class collaboration can benefit my group members.
• 9.During the pre-class discussion, I recognized and appreciated the peer contributions.
• 10.During the in-class collaboration, I recognized and appreciated the peer contributions.
• 11.I could feel my value to my peers in the pre-class discussions.
• 12.I could feel my value to my peers in the in-class collaboration.

Cognitive engagement

Deep engagement

• 1.During the pre-class discussions, I tried to integrate different pieces of information from course materials in new ways.
• 2.During the in-class collaboration, I tried to integrate different pieces of information from course materials in new ways.
• 3.When I finished the pre-class discussion, I would analyze it to see if there was more than one way to get the right answer.
• 4.When I finished the in-class collaboration, I would analyze it to see if there was more than one way to get the right answer.

Shallow engagement

• 1.I tried to memorize the steps for solving problems presented in the pre-class discussion.
• 2.I tried to memorize the steps for solving problems presented in the in-class collaboration.
• 3.Reviewing the messages in the pre-class discussion was useful to study for test preparation.
• 4.Reviewing the solved problems in the in-class collaboration was useful for test preparation.

Learning satisfaction

• 1.Overall, I liked the pre-class discussion.
• 2.Overall, I liked the in-class collaboration.
• 3.I would like to recommend the pre-class discussion for future flipped classrooms.
• 4.I would like to recommend the in-class collaboration activity for future flipped classrooms.