Video-based lecture engagement in a flipped classroom environment

Abstract

Purpose

to analyze which factors are associated with students’ engagement and participation in video-based lectures in a flipped classroom environment.

Methods

A single-center experience with video-based lectures in undergraduate medical education is described. The activity was applied to the subject of Neurosurgery during two consecutive courses (2021/22 and 2022/23). The videos were available prior to face-to-face classes through the online application Edpuzzle. Information was obtained from the own platform at the end of each course. Multivariable linear regression analyses were performed to assess the association between different variables and the percentage of video viewing, the early dropout rate, and the percentage of audience retention.

Results

A total of 109 students registered in Edpuzzle (87.2% of all enrolled students). Fifty-one videos were uploaded each course to cover 11 topics. Mean video viewing rate was 41%. Those videos linked to the earliest classroom lessons showed more percentage of viewing and audience retention than those programmed at the end of the course. With mandatory classroom assistance and homework assignments, the seminar videos were viewed more but retained less audience. Shorter videos were associated with higher viewing and audience adhesion, but the presence of questions embedded throughout the clip did not significantly engage students. No significant difference was observed regarding lesson topics.

Conclusions

It is essential to emphasize the importance of designing strategies to initially engage learners since more than half of our students never connected to the clips. Decreasing engagement was associated with the end of the course and video length. Seminar videos were viewed more but retained less audience. Active learning activities such as quizzes embedded throughout the clips did not significantly engage learners.

Background

Innovative methodologies have irrupted in all academic areas, including undergraduate medical education. Massive online open courses (MOOCs), blended learning, or flipped classroom (FC) are a reality nowadays, perhaps accelerated by the covid-19 pandemic [1]. Despite the differences in the methodology and approach, they all share the need to rely on information and communications technology (ICT).

The use of video-based lectures (VBLs) is increasing in this context. MOOCs are online courses available for a broad audience that can be followed worldwide which have the advantages of being self-paced and self-regulated [2]. Blended learning combines face-to-face and online teaching, where online platforms usually deliver complex concepts through digital videos [3]. The FC is considered a type of blended learning, where VBLs are usually used as a pre-class activity. This methodology leaves out of the classroom the most arduous work, the core content, to leave the in-class time to explain complex concepts or to carry out activities that apply in practice the theoretical contents previously “consumed” in the VBLs [4]. In all cases, the most critical problem with VBLs is the learner engagement and dropout rate [5]. The next challenge is to define and measure engagement, the learner’s ability to complete a given task, considering its cognitive, behavioral, affective, and social components [2].

The widespread use of ITC has made it possible to access a large amount of data on the access to, use of, and engagement with video lessons, providing important information. Several online platforms have been used: Youtube [6, 7], Edpuzzle [8], and edX [9]. Thus, learner engagement improves with shorter videos (maximum engagement time of 6 min); faster, enthusiastic speech; explanation of complex concepts; video format based on board drawing (superior to PowerPoint slides); or certificate incentives [8–12]. However, most studies focus on MOOCs, and the motivation and engagement patrons may differ in the FC.

Surveys are frequently used to analyze VBL’s quality, but they offer a subjective perspective [8, 13–16]. In our experience with FC, VBLs were among the lowest-rated activities in a satisfaction survey due to the work overload outside of the classroom. Likewise, the results showed mid-low engagement when analyzing the data on participation and interaction in pre-class activities [17]. Then, we wonder which objective factors make students engage and actively participate in the designed VBLs, and which make students drop out before completing the assignment.

Materials and methods

Study design

A retrospective study was designed, approved by the local Ethics Committee, and conducted in accordance with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. No informed consent to participate in the study was collected since the design was retrospective and the need for consent was waived by the Ethics Committee of Puerta de Hierro University Hospital.

During two consecutive courses (2021/2022 and 2022/2023) the FC methodology was used to teach the Neurosurgery curriculum to fourth-year undergraduate medical students in one of the four academic units that compose the School of Medicine. The subject is shared with Neurology and is programmed during the second semester of the course. Neurology is studied during February and mid-March. Neurosurgery is studied from the end of March to the end of April, when the classes finish to prepare for the final exams, which take place during the first three weeks of May.

VBLs were the main pre-class activity of the FC methodology. Thus, each lesson of the eleven that make up the Neurosurgery curriculum was divided into a variable number of videos that were recorded and uploaded to the Edpuzzle online platform prior to the face-to-face class. All videos were prepared by the first author, following the same methods (narrated PowerPoint videos) and covered the curriculum of the subject. The duration was according to the content and the concept explained, as a matter of coherence. They were available for two weeks during the first course and until the final exam during the second course. Most of the videos had a variable number of multi-choice questions embedded throughout them, intending to provide immediate feedback on previously explained concepts in some cases, and to complement missing content in others (Table 1). In certain cases, the questions in the second course changed as a result of previous experience. A single attempt to answer was allowed in course 2021/22 but multiple attempts were possible in course 2022/23, recording only the last attempt.

Table 1.

Summary of video-based lectures

LESSON	VIDEO	LENGTH (mm: ss)	TOPIC	QUESTIONS N 2021/22	QUESTIONS N 2022/23
1.CEREBROSPINAL FLUID (seminar)	1.1 1.2 1.3 1.4 1.5 1.6 1.7	4:40 6:58 5:14 7:22 7:27 7:59 11:23	Intracranial pressure Intracranial pressure: management Cerebrospinal fluid Hydrocephalus Hydrocephalus: pediatrics Hydrocephalus: adults Syringomyelia	2 2 2 2 1 1 2	2 2 3 1 1 1 1
2.CONGENITAL MALFORMATIONS	2.1 2.2 2.3 2.4 2.5 2.6	4:34 4:25 5:07 3:30 8:36 9:12	Craniosynostosis Dysraphism Hydrocephalus - Chiari Occipital-cervical malformations Open spina bifida Other spinal malformations	2 2 2 2 1 1	2 1 2 1 2 0
3.BRAIN TUMORS I	3.1 3.2 3.3 3.4 3.5 3.6	9:49 3:24 6:42 6:36 2:49 3:52	Introduction Metastasis Glioblastoma multiforme Other gliomas Meningioma Other hemispheric tumors	1 2 1 1 2 2	2 3 2 0 2 1
4. BRAIN TUMORS II	4.1 4.2 4.3 4.4 4.5	8:09 12:05 8:56 9:07 5:57	Intraventricular tumors Pineal region tumors Sellar region tumors Posterior fossa tumors Miscellaneous	2 2 2 2 1	2 2 2 2 0
5. TBI: PATHOPHYSIOLOGY (seminar)	5.1 5.2 5.3	5:28 8:05 6:09	Basic concepts Cranial and epicranial injuries Brain injuries	2 2 2	2 2 2
6. TBI: COMPLICATIONS	6.1 6.2 6.3 6.4 6.5	6:18 7:36 3:51 11:10 8:47	Acute subdural and epidural hematoma Other early vascular complications Other early complications Delayed complications Penetrating brain injury and severe TBI	2 2 1 2 2	3 1 0 1 1
7. VASCULAR PATHOLOGY	7.1 7.2. 7.3 7.4	5:48 21:55 8:26 9:42	Spontaneous intraparenchymal hematoma Aneurysmal subarachnoid hemorrhage. Arteriovenous malformations Other vascular malformations	2 2 2 2	3 0 3 3
8. SPINE TRAUMA (seminar)	8.1 8.2 8.3	17:21 12:27 5:09	Basic concepts Cervical spine injury Dorsal, lumbar, sacral spine injuries	2 2 2	3 3 2
9. SPINAL TUMORS	9.1 9.2 9.3 9.4 9.5	6:15 7:08 5:14 4:49 8:44	Spinal cord compression Vertebral tumors Intradural extramedullary tumors (schwannoma) Intradural extramedullary tumors (meningioma) Intramedullary tumors	2 2 1 1 2	2 1 0 1 1
10. DEGENERATIVE SPINE	10.1 10.2 10.3 10.4	7:20 7:32 14:07 5:47	Cervical disc herniation Cervical spondylosis Lumbar disc herniation Lumbar spinal stenosis	2 2 2 2	3 1 3 2
11. FUNCTIONAL NEUROSURGERY (seminar)	11.1 11.2 11.3	12:24 5:04 5:39	Pain Movement disorders and psychosurgery Epilepsy	2 2 2	3 3 2

During the academic year 2021/22, the videos could not go fast forward, so whenever the student rewound to take notes or listen to the explanation, the entire video must be watched again. However, this option was eliminated during the course 2022/23 so that the students could rewind or advance the video whenever needed.

No video was excluded; all videos were included in the final analysis, regardless of their length, the importance of the content explained, or any other distinguishing feature.

Data information

Data were obtained from the teacher reports and the analytics provided by the Edpuzzle platform at the end of both courses. Participation and registration were voluntary. Therefore, by registering, participants accepted the terms and conditions regarding privacy and data processing on the online platform.

The variables recorded from teacher reports were: academic year, lesson to which the video is linked, if the lesson is a seminar (in such case, assistance is mandatory and, at the end of the class, students have to submit the solution of a clinical case) or not, number of weeks from on-site lesson to the end of classroom sessions (and the beginning of final exams), video identification and position in the series that comprise a lesson, video length, and presence or absence of questions embedded throughout the clip. The variables provided by the platform were: mean percentage of video viewing, mean time spent by students on each video; refined time spent by students on each video (excluding those students that did not interact any second), viewing index (calculated as the mean number of minutes spent on each video over video length –in minutes-), mean percentage of audience retention (calculated as the students that complete the video viewing from those that see at least one second), mean percentage of early drop out (calculated as the students that start the video but stop and leave it before completing at least 10% of video length from all the students that start the video –at least one second of view-), percentage of students that did not interact with the clip (0% viewing), number of questions inserted in each clip, mean percentage of students answering the questions inserted in the clips, and mean correct answer rate. No individual or personal data was exported to the dataset, so participants could not be identified outside of the platform’s access conditions.

The platform records viewing time but pauses the video playback and does not record viewing time if a student switches to another tab or minimizes the window. However, any multitasking activity outside or parallel to the platform may not be directly monitored. Then, viewing time does not necessarily mean the student’s active behavior. There is an option to increase video speed but must be authorized by the teacher (which was not the case in the present study). When students re-watch a section, the platform analytics tracks only the “new” sections of the video.

Statistical analysis

Data file information was processed and analyzed using Stata 17 software (StataCorp 2021. Stata Statistical Software: Release 17. College Station, TX: StataCorp LLC). Numerical variables represented by the mean and standard deviation (SD) were contrasted with the Student-T test. Categorical variables represented by the absolute and relative frequencies were contrasted with the Chi-square test. The Mann-Whitney U test was used to contrast ordinal or continuous variables not normally distributed.

A multiple linear regression analysis was performed to assess which variables were associated with the considered outcomes according to the academic year. This decision was taken since the lesson order was slightly different in the two courses, which could interfere with the variable “number of weeks before the end of the classroom sessions”. A collinearity diagnosis was performed based on the paper published by Belsley in 1980 [18]. In this procedure, the condition of the matrix of independent variables is examined and the condition number of the matrix is calculated. If this number is large (more than 30), there may be collinearity problems. A simple linear regression analysis was used to evaluate the association of video length with different outcomes, considering both courses together. The assumption of linearity was checked by multivariable fractional polynomial models using the ‘’mfp’’ command in Stata. Normality of residuals was checked by plotting a Q-Q plot, a plot of residuals against the expected order statistics of the standard normal distribution, and plotting quantiles of the data against quantiles of a distribution.

The accepted level of significance was 5%. All p values were based on two-tailed tests of significance.

Results

Sample description

Of the 125 students enrolled in the subject, 109 registered on Edpuzzle (55 during 2021/22 and 54 during 2022/23). Fifty-one videos explaining the content of 11 lessons were uploaded to the platform. The videos were the same for the two courses, but the questions in each clip varied in some cases (Table 1). The mean number of videos per lesson was 4.64, and the mean video length was 7.7 min. 31.35% of the clips corresponded to a seminar class (with mandatory assistance and submission of a clinical case solution), whereas the remaining 68.63% were a standard lesson. 76.47% of the videos had questions inserted throughout it during the first course (2021/22), whereas 88.24% of them had questions inserted during the second one (2022/23).

The mean percentage of video viewing was 41% (SD 13). 56% (SD 14) of all registered students did not interact with the videos, and 4.2% (SD 3.5) did not finish the view. Considering the remaining 44% that made any connection, 1.9% (SD 34) of the students prematurely dropped out of the clip before completing 10% of the length. The mean percentage of audience retention was 90% (SD 7.7).

Students spent a mean time of 4.1 min (SD 2.1) viewing each clip. When excluding those students that did not perform any connection or interaction, the mean refined time dedicated per video was 9.8 min (SD 4.7). Then, the mean viewing index was 1.3 (SD 0.14) per video. Finally, the mean number of questions inserted in each clip was 1.6 (SD 1), the mean percentage of students answering the questions was 43% (SD 14), and the mean correct answer rate was 78% (SD 20). Table 2 summarizes the results obtained differentiating per academic year and contrasts both courses.

Table 2.

Description of the outcome variables, differentiation according to the academic year. Contrast analysis between both courses using Mann-Whitney U test

OUTCOME
Course	N	Mean (SD)	Median (p25; p75)	Range	p value
% video viewing					0.469
2021/2022	51	42 (14)	43 (33; 55)	18–69
2022/2023	51	40 (12)	41 29; 46)	19–69
Minutes dedicated per video					0.462
2021/2022	51	4.3 (2.1)	3.9 (2.8; 5.9)	1.2–11
2022/2023	51	4 (2.1)	3.7 (2.3; 5.4)	1.4–11
Refined minutes dedicated					0.192
2021/2022	51	10 (5)	9.3 (6.8; 12)	3.9–30
2022/2023	51	9.2 (4.4)	8.1 (6.2; 10)	3.3–25
Viewing index					< 0.001
2021/2022	51	1.3 (0.13)	1.4 (1.3; 1.4)	0.76–1.6
2022/2023	51	1.2 (0.12)	1.2 (1.1; 1.3)	0.97–1.6
% Audience retention					< 0.001
2021/2022	51	93 (6.8)	94 (90; 100)	64–100
2022/2023	51	87 (7.5)	88 (83; 93)	67–100
% Early drop out (< 10% viewing)					< 0.001
2021/2022	51	0.54 (1.7)	0 (0; 0)	0–9.1
2022/2023	51	3.3 (4)	3.6 (0; 5.9)	0–20
% No interaction (0% viewing)					0.939
2021/2022	51	57 (14)	56 (44; 65)	29–82
2022/2023	51	56 (13)	56 (48; 70)	20–80
N embedded questions					0.381
2021/2022	51	1.5 (1.1)	2 (1; 2)	0–3
2022/2023	51	1.7 (0.96)	2 (1; 2)	0–3
% Students answering questions					0.532
2021/2022	39	42 (13)	40 (33; 51)	18–65
2022/2023	45	44 (14)	44 (31; 52)	22–78
% Correct answer					< 0.001
2021/2022	39	64 (19)	68 (50; 76)	16–100
2022/2023	45	90 (9.3)	92 (88; 96)	46–100

SD: standard deviation; p: percentile; p value: type α error

Analysis of factors

The multiple linear regression analysis performed to assess which factors were associated with different outcomes considered the following variables: if the video belonged to a seminar class, the video length, the number of weeks before the end of the classroom sessions (and, therefore, the beginning of final exams), and if the video included questions throughout it (Table 3). The collinearity diagnosis did not show any condition number > 30. For the 2021/2022 course, the condition number was 7.17 and 8.29 for the 2022/2023 course. The normality of residuals assumption was also checked for all the models.

Table 3.

Multiple linear regression analysis to assess factors associated with percentage of video viewing, audience retention, early dropout and viewing index

OUTCOME	2021/2022 (N = 51)			2022/2023 (N = 51)
Factor	p value	Coefficient (95% CI)	p value	Coefficient (95% CI)
% VIDEO VIEWING		R2 = 0.821		R2 = 0.856
Seminar (yes)	0.02	4.54 (0.73–8.36)	< 0.001	15.00 (11.90–18.06)
Nº weeks before the end of the course	< 0.001	8.21 (6.99–9.43)	< 0.001	4.19 (3.47–4.91)
Video length (min)	0.45	-0.19 (-0.71–0.32)	0.076	-0.37 (-0.78–0.04)
Questions inserted in the clip (yes)	0.676	0.90 (-3.39–5.19)	0.691	0.89 (-3.59–5.37)
% AUDIENCE RETENTION		R2 = 0.146		R2 = 0.174
Seminar (yes)	0.885	-0.29 (-4.29–3.71)	0.028	-5.07 (-9.57 – -0.57)
Nº weeks before the end of the course	0.02	1.54 (0.26–2.81)	0.308	0.54 (-0.51–1.58)
Video length (min)	0.478	-0.19 (-0.74–0.35)	0.169	-0.41 (-1.00–0.18)
Questions inserted in the clip (yes)	0.748	0.72 (-3.78–5.22)	0.966	-0.14 (-6.64–6.36)
VIEWING INDEX		R2 = 0.013		R2 = 0.389
Seminar (yes)	0.967	-0.002 (-0.08–0.08)	< 0.001	0.12 (0.06–0.18)
Nº weeks before the end of the course	0.98	-0.000 (-0.03–0.03)	0.002	0.02 (0.01–0.04)
Video length (min)	0.783	0.002 (-0.01–0.01)	0.581	0.00 (-0.01–0.01)
Questions inserted in the clip (yes)	0.471	-0.03 (-0.12–0.06)	0.716	0.02 (-0.07–0.11)
% EARLY DROP OUT		R2 = 0.054		R2 = 0.052
Seminar (yes)	0.292	-0.55 (-1.59–0.49)	0.661	-0.56 (-3.14–2.01)
Nº weeks before the end of the course	0.536	-0.10 (-0.43–0.23)	0.165	-0.42 (-1.02–0.18)
Video length (min)	0.339	0.07 (-0.07–0.21)	0.762	-0.05 (-0.39–0.29)
Questions inserted in the clip (yes)	0.776	-0.17 (-1.34–1.00)	0.634	0.88 (-2.83–4.60)

p value: type α error; CI: confidence interval; R²: coefficient of determination; min: minutes

Percentage of video viewing

Seminar classes and a higher number of weeks from the final exam were significantly associated with higher video viewing in both courses. The video length was not significant in the course 2022/23 (p = 0.076). A simple linear regression analysis was employed to evaluate both courses, observing an inverse correlation between video length and lower video viewing (p = 0.014; coefficient − 0.91, 95% CI -1,63; -0.18; Fig. 1a).

Fig. 1.

Simple linear regression analysis. a: correlation between video length and percentage of video viewing (R² = 0.058); b: correlation between video length and percentage of audience retention (R² = 0.041)

Percentage of audience retention

The number of weeks from the face-to-face class to the end of the course was the only variable associated with a higher percentage of audience retention in the course 2021/22. In contrast, the seminar class was the only factor identified in the course 2022/23. Thus, during the first course, the engagement (understood as audience retention) was higher the earlier the classroom sessions occurred. During the second course, the adhesion was lower when the videos belonged to a seminar lesson. A simple linear regression analysis was also used to assess the association between video length and audience retention, considering both courses. The results showed that longer videos were significantly associated with lower engagement (p = 0.042; coefficient − 0.44, 95% CI -0.86; -0.15; Fig. 1b). A similar regression compared audience retention depending on the lesson topic. Significantly lower retention was observed in lessons 8 and 11 (seminars) compared with lesson 1 (p = 0.022; coefficients: -8.11, 95% CI -15.01; -1.21 and p = 0.001, -12.36, 95% CI -19.26; -5.46, respectively), which has more and shorter clips, is a seminar with mandatory assistance and usually takes place at the beginning of the subject, far from the end of the course.

Viewing index

No significant association was observed in the factors studied on the time dedicated to each video according to the video length in the course 2021/22. However, a significant association was evidenced during the second course between a higher index and earlier classroom sessions, as well as seminar lessons. The analysis of the relationship between video length and viewing index considering both courses did not reach a statistical association (p = 0.878). When analyzing the video index according to the lesson, only lesson 7 showed a significantly lower index than lesson 1 (p = 0.027, coefficient: -0.14, 95% CI -0.27; -0.02).

Percentage of early dropout

No association was observed between an increased early dropout and the factors assessed in any of the courses evaluated.

Analysis of the video series

The percentage of video viewing and audience retention was evaluated for each clip and considering the position of the video in the lesson series. Results are summarized in Table 4. No contrast test was applied since only two measurements (one per course) were recorded. In lessons 1, 4, and 9, the percentage of video viewing decreased as long as the series advanced. No association was observed in audience retention.

Table 4.

Summary of the percentage of video viewing and audience retention for each clip, considering the position of the video in the lesson series

VIDEO	N	% VIDEO VIEWING		% AUDIENCE RETENTION
		Mean (SD)	Median (p25;p75)	Mean (SD)	Median (p25;p75)
1.1	2	59 (13)	59 (50; 69)	92 (5.7)	92 (88; 96)
1.2	2	58 (14)	58 (48; 68)	90 (4)	90 (87; 93)
1.3	2	55 (9.3)	55 (49; 62)	92 (0.82)	92 (91; 93)
1.4	2	55 (9.2)	55 (49; 62)	92 (0.82)	92 (91; 93)
1.5	2	50 (10)	50 (43; 58)	97 (4.4)	97 (94; 100)
1.6	2	53 (7.3)	53 (47; 58)	90 (3.1)	90 (88; 93)
1.7	2	47 (8.5)	47 (41; 54)	98 (2.4)	98 (97; 100)
2.1	2	53 (9.8)	53 (46; 60)	89 (4.7)	89 (86; 92)
2.2	2	50 (8.5)	50 (44; 56)	94 (8.2)	94 (88; 100)
2.3	2	51 (7.9)	51 (45; 56)	93 (10)	93 (86; 100)
2.4	2	50 (8.3)	50 (44; 56)	89 (11)	89 (81; 97)
2.5	2	51 (10)	51 (44; 58)	83 (16)	83 (71; 94)
2.6	2	50 (12)	50 (42; 58)	90 (9.1)	90 (84; 97)
3.1	2	54 (20)	54 (40; 69)	92 (0.99)	92 (91; 92)
3.2	2	54 (14)	54 (44; 64)	91 (8.9)	91 (85; 97)
3.3	2	54 (12)	55 (46; 63)	91 (8.9)	91 (85; 97)
3.4	2	53 (12)	53 (45; 62)	95 (3.6)	95 (92; 97)
3.5	2	54 (14)	54 (44; 64)	96 (5.7)	96 (92; 100)
3.6	2	53 (12)	53 (45; 62)	95 (3.6)	95 (92; 97)
4.1	2	40 (3.8)	40 (38; 43)	92 (0.49)	92 (91; 92)
4.2	2	38 (1.4)	38 (37; 39)	84 (3.2)	84 (82; 86)
4.3	2	38 (0.41)	38 (38; 38)	98 (3.4)	98 (95; 100)
4.4	2	39 (2)	39 (37; 40)	98 (3.4)	98 (95; 100)
4.5	2	33 (0.02)	33 (33; 33)	92 (3.7)	92 (89; 95)
5.1	2	55 (0.88)	55 (55; 56)	88 (7.9)	88 (82; 94)
5.2	2	51 (2.7)	51 (49; 53)	93 (0.33)	93 (93; 93)
5.3	2	53 (5.4)	53 (49; 57)	83 (4)	83 (80; 86)
6.1	2	46 (3.3)	46 (44; 49)	87 (1.6)	87 (86; 88)
6.2	2	42 (1.1)	42 (42; 43)	92 (0.23)	92 (92; 92)
6.3	2	39 (1)	39 (39; 40)	95 (0.29)	95 (95; 96)
6.4	2	41 (0.92)	41 (40; 41)	98 (3.1)	98 (96; 100)
6.5	2	40 (0.42)	40 (40; 41)	98 (3.2)	98 (95; 100)
7.1	2	32 (5.6)	32 (29; 36)	92 (11)	92 (84; 100)
7.2.	2	30 (7.9)	30 (24; 36)	84 (8.1)	84 (79; 90)
7.3	2	30 (7.9)	30 (25; 36)	86 (1.2)	86 (85; 87)
7.4	2	31 (5.5)	31 (27; 35)	92 (4.8)	92 (88; 95)
8.1	2	40 (9.7)	40 (33; 47)	85 (21)	85 (70; 100)
8.2	2	41 (6.3)	41 (37; 46)	83 (2.6)	83 (81; 85)
8.3	2	42 (4)	42 (39; 44)	87 (6.6)	87 (83; 92)
9.1	2	24 (4.5)	24 (21; 27)	90 (2.9)	90 (88; 92)
9.2	2	23 (4.2)	23 (20; 26)	100 (0)	100 (100; 100)
9.3	2	24 (2.5)	24 (23; 26)	89 (6.2)	99 (85; 93)
9.4	2	22 (5.6)	22 (18; 26)	91 (2)	91 (90; 93)
9.5	2	21 (4.2)	21 (18; 24)	96 (5)	96 (93; 100)
10.1	2	21 (2.1)	21 (20; 23)	85 (7.7)	85 (80; 91)
10.2	2	23 (1.5)	23 (22; 24)	96 (5.4)	96 (92; 100)
10.3	2	19 (0.98)	19 (18; 19)	73 (14)	73 (64; 83)
10.4	2	20 (0.66)	20 (19; 20)	91 (13)	91 (82; 100)
11.1	2	34 (3.9)	34 (32; 37)	84 (7.5)	84 (78; 89)
11.2	2	33 (1.4)	33 (32; 34)	78 (9.6)	78 (71; 85)
11.3	2	34 (2.3)	34 (32; 35)	81 (20)	81 (67; 94)

N: number of measurements; SD: standard deviation; p: percentile

Finally, it was assessed if the absence of questions inserted in a clip was associated with a lower viewing percentage than the other videos of the same series. All lessons 5, 8, 10, and 11 clips included questions. A significant difference was only confirmed in lesson 4, with 33% viewing the clip when no questions were inserted vs. 38% when the video had embedded questions (p = 0.036). The results in lesson 1 did not reach statistical significance (p = 0.051).

Discussion

The two factors that significantly impacted student engagement with VBLs were the timing and the type of face-to-face class. Thus, lectures held earlier in the semester were more likely to be watched and retained larger audience. However, clips associated with seminar-style classes, with compulsory attendance and homework assignment, were viewed more but with lower retention. In addition, shorter videos were associated with a higher viewing percentage and higher retention. Finally, none of the factors analyzed were significantly associated with early dropout.

It is essential to consider the differences inherent to each academic year that, due to unknown factors, do not behave identically even though the subject is the same, the teacher is the same, and the intellectual level and motivation of the students are similar (cut-off score for entering the degree with low variability, less than one point out of 14). Thus, in the course 2022/23, students showed lower audience retention and higher early dropout percentage compared to the previous course. The results observed regarding performance in embedded questions (significantly better during the course 2022/23) must be explained by the possibility of having multiple attempts to answer the question during that course.

Several variables were considered to evaluate the motivation to engage in the videos. The percentage of video viewing was similar in both courses, about 41%, a result comparable to that obtained in other studies (42%) [6] but inferior compared to other cases (67.7%) [16]. It is essential to highlight that more than half of the students registered in the platform did not interact at any time with the clips. In a study performed with doctors and trainees, the main reason referred to by those who did not view the videos was “insufficient time” [16]. Motivation and educational level differ from the present study, performed with undergraduate medical students. Nevertheless, that same reason was argued in another study performed in higher education [13]. One possible explanation in our case is the availability of notes from previous years that are improved successively, considering that Neurosurgery is a “minor” subject of only 11 h of classroom sessions. It may be challenging to make the subject attractive but explaining to the students at the beginning of the course the dynamic of the classes, the learning objectives, and the possibility of training in multi-choice questions could increase the number of students that initially engage with VBLs, as also suggested by other authors [3]. Another intervention might be designing a gamification activity that includes the neurosurgical questions from the last board exam to highlight the profitability of studying the subject.

Seminars were strongly associated with higher viewing. Since there is no difference in the content or the type of concepts explained between seminars and the rest of the classes, the observed difference may be explained by the mandatory nature of attendance and homework assignments. Also, the temporal relationship with the end of the course showed a strong association that has rarely been reported. Lin et al. [10] observed a decrease in the proportion of students accessing VBLs as the semester progressed, but they did not quantify the percentage of viewing, and no temporal change was observed in terms of audience retention (about half of the students who started a video almost completed it, a rate lower than the 90% observed in our experience) [10]. Such association can be explained by the perception of time availability to dedicate to the videos at the beginning of the semester. Thus, the VBLs may be perceived as an overload for the student’s activities outside the classroom. In fact, time-consuming videos were the main complaint in the survey performed in our first experience with FC [17]. According to the results presented, this is a key factor in student engagement. However, the organization of timetable and subjects is complex and involves third parties, so it is controversial to consider it as a modifiable factor. Finally, video length has been previously related to this outcome, a result confirmed in our series [6, 9].

The same variables were associated with audience retention. Unlike other studies [10, 16], in our experience, videos that were linked to an earlier face-to-face session in the semester showed improved audience adhesion, as well as higher viewing rates by participants. In this regard, the influence of the impending end-of-year exams cannot be ignored. Dodson et al. determined a comparable pattern when assessing active video viewing. They observed that students increased playback speed or skimming forward as the course week advanced, which can be interpreted as a time-saving measure [19].

Contrary to the percentage of viewing, those clips linked to a seminar class inversely correlated with students’ retention. Seminar videos were watched more but retained less audience. This is a novel finding, and we hypothesize that the mandatory homework makes more students consult the clip to solve the clinical case assignment. However, once they obtain the answer, students abandon the clip, so it is doubtful that a higher percentage of viewing with lower retention will result in more meaningful learning. Moreover, the mandatory assistance and assignment seems useful for improving access to the videos among undergraduate students, in a compulsory subject. However, this conclusion should be separate from other learning strategies with voluntary access, such as MOOCs, where other kind of external incentives may work differently [20].

The video length also correlated with audience adhesion, and shorter videos retained more students, as observed by other authors [6, 9, 16]. Some studies have shown that retention declined as the video advanced, regardless of length [6, 7]. In our series, retention was high (90%) compared to that observed in MOOCs (49.7%) [6], probably due to the student’s motivation and the higher education context. These same reasons would explain the low dropout percentage observed (< 2%).

The viewing index values the time spent on each video concerning its duration. It, therefore, excludes those students who did not interact at any time with the clip. The index was higher in the first course, probably due to the limitation to forward the clips, as abovementioned. In addition, this index was higher in the videos linked to earlier lessons during the second course. This may be explained by the more significant time availability or students’ lower workload since exams seem far away. Another possible reason is the lesser habit or knowledge of the classes’ dynamics, given that the FC methodology is not used in other subjects in our environment. However, no other study has reported similar findings so it must be considered in future research. Seminars consumed more time, probably due to the same reasons argued in audience retention (homework assignment). Clip length showed no association in this case, so longer videos did not require proportionally more time for comprehension, contrary to the results obtained by Guo et al. [9]. Only one lesson showed a significantly lower index, so we considered no association between complex concepts and a higher viewing index in our sample, a correlation that has been previously obtained [12]. Vavasseur et al. reported a 2.47-fold increase in video connections in three consecutive courses. However, the authors did not explain the hypothesis for this finding [15].

Another consideration was the percentage of video viewing and audience retention depending on the position of the video in the lesson series. In our experience, three of the eleven lessons showed a viewing decline over sequential videos, a trend reported by other authors but without statistical significance [6]. A recent study also observed viewing percentage and audience retention decline for videos placed later in a sequence [21]. Fatigue could be a potential explanation, but reducing the number of videos would imply increasing their length to avoid losing content. However, no effect was observed on audience retention. The last videos of the series must not necessarily include less important content, so those students that start viewing them choose it voluntarily and do not drop out, which is equivalent to the “completers” pattern described by Ferguson et al. [5].

Finally, we assessed the importance of active learning strategies, such as inserting questions within the clip. Rice et al. determined that the most effective position was throughout the video rather than at the end [22], but no association with students’ engagement has been evaluated [22, 23]. No significant effect was observed in the present study after multiple regression analysis. However, the percentage of viewing and audience retention depending on the presence or not of questions was compared between the videos of the same lesson. Four lessons were excluded since all videos included questions. Two of the remaining seven lessons showed significant or borderline significant differences, with lower viewing and retention when no questions were embedded. Nevertheless, we must be cautious with the conclusions considering the small sample size (< 15). Despite no demonstrated association, quiz questions embedded in a video are critical in other aspects, such as immediate feedback or improved test performance [22, 23].

Two of the main limitations of this research is the sample size and the retrospective design. Even though the number of videos is considerable, the number of students must be increased to obtain more reliable conclusions. A more robust design, with a control group, would also strengthen the study’s validity. Engagement is also challenging to define and measure, and the variables used in this study do not necessarily mean active viewing or effective learning. The platform Edpuzzle was chosen due to the possibility to insert questions and obtain immediate feedback, but it does not consider interactive behavior metrics to measure active viewing or determine viewing patterns.

On the other hand, the classroom dynamic changes every year. Students are different, their motivations evolve, and the group behavior varies, making it difficult to control for potential confounding factors. A feasible option would be extending the methodology to the other three academic units of the School of Medicine to increase the sample size. Randomization would also be recommended. Finally, data availability is essential; more complex software may provide more accurate information to discern active viewing or interactive behavior patterns.

Conclusions

The temporal relationship between the VBL classroom session and the course’s end strongly correlated with video viewing and audience retention. Those videos linked to the earliest classroom lessons showed more percentage of viewing and audience retention than those programmed at the end of the course. With mandatory classroom assistance and homework assignments, the seminar videos were viewed more but retained less audience. Shorter videos were associated with higher viewing and audience adhesion, but the presence of questions embedded throughout the clip did not significantly engage students. No significant difference was observed regarding lesson topics, but a viewing decline over sequential videos was observed in 27% of the lessons (considering that no statistical test was applied). Finally, it is essential to emphasize the importance of designing strategies to initially engage learners since more than half of our students never connected to the clips.

Further research may focus on metrics of viewing patterns and active viewing, as well as on the role of embedded questions in the frequency of reviewing videos since any intervention should aim to impact effective learning.

Abbreviations

CI

Confidence interval

FC

Flipped classroom

ICT

Information and communications technology

MOOC

Massive online open course

SD

Standard deviation

VBL

Video-based lecture

Author contributions

RGG is associate professor at the Autonomous University of Madrid and neurosurgeon at Puerta de Hierro Universtity Hospital in Madrid; she has designed the work; acquired the data, drafted the manuscript and approved the final version.AZ is a teaching collaborator and neurosurgeon at Puerta de Hierro Universtity Hospital; he has acquired the data for the work, drafted the manuscript and approved the final version. AR is senior biostatistician at Puerta de Hierro University Hospital; she has analysed the data, reviewed the manuscript for important intellectual content and approved the final version. All authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Funding

None.

Data availability

The dataset generated and analysed during the current study is available in the Zenodo repository, 10.5281/zenodo.8132674, and will be open access when the article is published.

Declarations

Ethics approval and consent to participate

The study was approved by “Puerta de Hierro University Hospital” Ethics Committee and was conducted in accordance with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards (reference 57/695017.9/22). Since data collection and analysis was retrospective, “Puerta de Hierro University Hospital” Ethics Committee waived the need of informed consent.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.