Three Weeks of Team-Based Leaning Do Not Overload Undergraduate Students

Abstract

Context

Team-based learning (TBL) is a flipped-classroom approach requiring students to study before class. Fully flipped curricula usually have fewer in-class hours. However, for practical reasons, several programs implement a few weeks of TBL without adjusting the semester timetable. Students fear that they will be overloaded by the individual and collaborative study hours needed to prepare for TBL.

Methods

We implemented three consecutive weeks of TBL in a 15-week lecture-based course on the renal system. In-class time and assessments were unchanged for all courses. Four hundred fifty-nine first-year undergraduate medical students (229 in 2018; 230 in 2019) were invited to complete weekly logs of their individual and collaborative study hours during lectures and TBL, along with questionnaires on cognitive load and perception of the course. Our program changed from A to E grading in 2018 to pass-fail grading in 2019

Results

Participants (n = 324) spent a similar number of hours studying for TBL vs. lectures with a mean of 3.1 h/week. Collaborative study was minimal outside class (median 0.1 h/week). Results remained similar with pass-fail grading. If in-class time were reduced, 18% of participants said they would have used freed-up time to study for TBL. Studying for TBL generated similar extraneous cognitive load and lower intrinsic load compared to studying for lectures; students were less stressed, and maintained high levels of motivation and self-perceived learning.

Conclusions

Three weeks of lectures were replaced by TBL without reducing in-class time. Students did not report overload in study hours or in cognitive load.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40670-021-01314-x.

Introduction

With the benefits of small group learning within large classes, team-based learning (TBL) can complement lecture-based courses [1–3]. TBL is a flipped-classroom approach used in health sciences education since the 1990s where a single instructor conducts multiple small group discussions and problem-solving activities in one classroom [3, 4]. Knowledge is gained by students through self-regulated learning (SRL) before the class, so that class time can be used for interactive and collaborative consolidation of knowledge [5]. The conceptual model supporting TBL relies on learners’ meaningful engagement with the subject [6]. In other words, students need to be motivated to engage with the TBL course, and should not be overloaded [7, 8].

Fully flipped curricula usually reduce in-class time to give students enough time to prepare for the course [3, 9]. However, for practical reasons, several programs implement a few weeks of TBL without adjusting the semester timetable [10–12]. Students have expressed concerns regarding time management, with the fear that TBL courses are added without concurrently reducing the amount of time allocated for didactic instruction [1, 5, 9, 13, 14]. Students were also concerned that in-class time in TBL was less efficient to prepare for assessments, thus additional efforts would be needed outside class [1]. At this time, the literature on TBL does not demonstrate an effect on study time, nor does it provide clear indications if and how in-class time should be reduced when only few weeks of TBL are implemented [1, 5, 6, 9–11, 15]. Our study aimed to address the gap between student concerns and empirical data.

Students in flipped curricula are also questioning the alignment with the assessment and grading systems [9]. TBL is consistent with a pass-fail grading system, both fostering SRL [16]. However, it brings concerns that students, losing extrinsic motivation and structure, will study less [17, 18]. Studies in pass-fail systems reassuringly show that there is no difference on time utilization and performance [19]. Using study logs, Jessee and Simon [20] showed that students maintained the same commitment to learning. On the other hand, with discriminating grading scales, students are often ‘overstudying’, leading to stress and work overload [16, 21, 22]. Research on grading systems has mostly been devoted to undergraduates studying individually in lecture-based curricula; very little attention has been paid to flipped classrooms [17, 19, 20]. Both known to improve group cohesion, our hypothesis was that pass-fail grading and TBL have synergic effects to increase collaborative study time [21, 23].

Barbosa, Silva [7] proved that study time only predicts academic achievement if undergraduate medical students perceive an appropriate workload. Based on their findings, study time and workload are interacting variables that should be measured concomitantly [7]. We measured workload by combining two complementary methods: cognitive load and perception of the course [8, 24]. Indeed, emotions toward the course, for example feeling motivated or stressed, are closely related to changes in cognitive load and may help to interpret the results [24].

The overall cognitive load is defined as the amount of resources devoted to a specific task within the limits of the working memory [8]. The intrinsic load relates to the task complexity and the amount of interacting elements. Extraneous cognitive load refers to the use of working memory resources on information non-essential to the goals of instruction, such as being unfamiliar with the format of TBL. To optimise the learning process in TBL, extraneous cognitive load should be minimised to allow learners to focus their working memory resources on dealing with the essential elements (i.e. the intrinsic cognitive load) [25].

Kenwright and colleagues [18] suggested that the unfamiliar structure of flipped classroom in medical education may impose extraneous cognitive load. While many students review the content only after a lecture, in TBL, students are assessed to ascertain that they understand important concepts before the class. TBL therefore implies SRL before class. Learners are expected to invest additional time but also cognitive and metacognitive resources, without the guidance and structure usually provided by lectures (e.g. worked examples) [26, 27]. In the literature, validated instruments measuring cognitive load had never been used to compare TBL with other instructional modalities [26].

As modeled by Seufert [26], there is an interplay between the concepts of SRL and cognitive load. Using another version of the questionnaire used in our study [28], Glogger-Frey, Gaus [29] proved that strategy use and monitoring during SRL can increase extraneous load. We can infer that if studying for TBL is too complex to be processed in a limited study time, the use of SRL strategies and the additional monitoring can exceed learners’ capacity to successfully deal with the task and therefore impede learning [26, 29].

In 2018 and 2019, within a full-time undergraduate medical curriculum, we replaced three consecutive weeks of lectures on the renal system by TBL without reducing in-class time. From a practical standpoint, this observational study aimed to answer three research questions on the theme of students’ overload:

1. Will medical students allocate significantly more weekly hours for individual and collaborative study outside class during TBL as opposed to lectures?

2. What is the impact of using pass-fail grading on [question 1]?

3. Are there differences in students’ cognitive load and their perception of the course when studying for TBL as opposed to lectures?

To characterise our sample of participants, we used a questionnaire of self-directed learning abilities and we questioned participants about their learning strategies during TBL. To answer our research questions, participants completed weekly study logs throughout the semester along with four cognitive load and six perception questionnaires, as presented in Table 1.

Table 1.

Schedule of the 2018 and 2019 semesters with number of participants to the weekly study logs and questionnaires

Week	Teaching modality in RS	Teaching modality in DS	Weekly study logs n		Questionnaires (n)
			2018	2019
0	Presentation of the study and informed consent				• Questionnaires on self-directed learning abilities SDLI-FR and learning strategies during TBL (n = 137 in 2018, n = 174 in 2019) • Questionnaires on the perception of the courses (for RS n = 143 in 2018, n = 174 in 2019; for DS n = 142 in 2018, n = 174 in 2019)
1	Lectures	Lectures	99	120
2	Lectures	Lectures	84	147
3	Lectures	Lectures	70	158	• Cognitive load questionnaire CL-FR (for RS, n = 41 in 2018, n = 148 in 2019; for DS n = 34 in 2018, n = 131 in 2019) •  Questionnaires on the perception of the courses (for RS n = 70 in 2018, n = 156 in 2019; for DS n = 70 in 2018, n = 156 in 2019)
4	Lectures	Lectures	61	139
5	Lectures	Lectures	68	139
6	Lectures	Lectures	64	144
7	Exam	Lectures
8	“Spring break”
9	Mock TBL	Exam
10	TBL	Lectures	43	136	• Cognitive load questionnaire CL-FR (for RS n = 43 in 2018, n = 132 in 2019; for DS n = 42 in 2018, n = 120 in 2019) • Questionnaires on the perception of the courses (for RS n = 43 in 2018, n = 132 in 2019; for DS n = 43 in 2018, n = 132 in 2019)
11	TBL	Lectures	37	129
12	TBL	Lectures	36	99
13	Lectures	Lectures	24	115
14	Lectures	Lectures	28	98
15	Exam	Exam

TBL team-based learning, RS renal system course, DS digestive system course, n number of participants who completed a given questionnaire

Methods

Educational Setting

Our study took place from January to May in 2018 and 2019, during 15-week semesters, at University Laval Faculty of Medicine (Quebec City, Canada). First-year full-time undergraduate medical students of a 5-year doctoral program attended a renal system (RS) course combining lectures and TBL. Simultaneously, students attended a digestive system (DS) lecture-based course, a basic sciences lecture-based course, problem-based learning modules, and a discussion-based course on medicine and society. In-class time varied from 16 to 20 h per week.

RS and DS courses taught the physiopathology, semiology, differential diagnoses, and treatments of common pathologies of the renal and digestive systems. Course books were provided at the beginning of both courses. In the first half of the RS course, students learned about difficult concepts of renal physiology (e.g. function of the glomerulus and tubule, electrolytes, homeostasis), relying on limited prior knowledge. In the second half of the RS course, where TBL was implemented, students used the concepts previously learned to solve clinical problems of the renal system (e.g. nephritic syndrome, acute kidney injury, chronic kidney disease). For both courses, students’ final scores in percentage were converted to A to E grading in 2018, and to pass-fail grading in 2019.

Students and faculty report the DS course as the most demanding and having the greatest potential to overload students. Hence, we identified the DS course as a potential confounding factor. Simultaneously studying the DS course also served to blind participants to the focus of the study (i.e. TBL in the RS course).

Team-Based Learning in the Renal System Course

As shown in Table 1, we implemented three consecutive weeks of TBL (weeks 10–12) in the RS course without adjusting the rest of the semester timetable. In other words, the time scheduled to be present in the classroom and the number and scheduling of assessments were unchanged for all courses. During those 3 weeks, the four weekly hours usually allocated for lectures on the RS were replaced by in-class TBL.

The principles and structure of TBL were explained to students at week 9 when students participated in a mock TBL without pre-course preparation. We applied the core principles of TBL detailed by Haidet and colleagues [6]. Students formed teams of five. All teams gathered in an auditorium for 2 h, twice a week. We designed an individual readiness assurance test of 10 multiple-choice questions using ExamSoft©. The questionnaire was completed individually and submitted electronically. The same questionnaire was completed again with their teammates using scratch-off cards to get immediate feedback. Both questionnaire rounds were graded, representing 0.5% (individual tests) and 1.0% (team tests) of the final score of the RS course. Use of books or other materials was permitted. This testing verified that learners were sufficiently prepared to solve a sequence of clinical problems. Following the tests, we assigned the same problem of real-world relevance for all teams to report on, generating intra- and inter-team discussions [6]. The teacher gave complementary information and corrected false reasoning, but did not give a lecture.

Participants

A researcher unrelated to the course (MCM) invited all 459 undergraduate medical students (229 in 2018; 230 in 2019) attending the RS and DS courses to participate in the study at week 0. Participants had no prior experience with TBL in our undergraduate medical curriculum. To minimise desirability bias, participants were not aware that the study measured the impact of TBL, only that we would ask about study hours and their perception of the RS and DS courses throughout the semester.

Participants’ Characteristics and Self-directed learning Abilities

Participants’ abilities for self-directed learning are key factors when studying for TBL. It was measured at week 0, along with participants’ age, gender, and highest obtained degree [5]. We translated into French a questionnaire on self-directed learning abilities (SDLI-FR) developed by Cheng and colleagues [30] and tested with nursing students by Shen and colleagues [31]. SDLI-FR used an agreement scale from 1 to 5 for 20 statements covering four domains: learning motivation, planning and implementing, self-monitoring, and interpersonal communication [28]. Questions 18 and 20 did not apply to this study and were not used.

Weekly Study Logs and Study Strategies

As shown in Table 1, at weeks 1 to 6 and 10 to 14, we sent a study log asking for the number of individual and collaborative (studying with ≥ one colleague, by any means) study hours for the RS and DS courses during the past week. Participants were asked to log separately the number of study hours for RS and DS, without decimals. The logs took the form of 11 successive electronic surveys, created with Limesurvey™. Responses, linked to participants’ ID numbers, were automatically gathered in a spreadsheet, anonymised, and analysed by researchers only after the semester.

Study hours could take place anywhere outside class (e.g. at the university library, at home) and around the clock. We added three questions: (1) if free time was available, for which course students would have studied more; (2) which learning resources were used; (3) did students use test-enhanced learning strategies (flashcards, self-assessment activities, or questions to a colleague)? After the semester, we asked participants to confidentially share their final score, in percentage, for each course. For ethical reasons, researchers did not have direct access to participants’ results (e.g. results for items assessing TBL-based content).

Questionnaires on Cognitive Load

We translated into French a cognitive load questionnaire (CL-FR) developed by Leppink and colleagues [28] and validated with medical students studying outside class [28, 32, 33]. CL-FR used an agreement scale from 0 to 10 for 13 statements [28]. Covering three domains, students self-reported the intrinsic cognitive load, extraneous cognitive load, and self-perceived learning of the course for which they were studying. CL-FR was completed for the RS and DS courses at week 3 (studying for lectures in both courses) and at week 10 (studying for TBL in RS and lectures in DS). Given the time needed to complete the questionnaires and the risk of non-response, we selected weeks 3 and 10 by taking into account local logistical aspects (e.g. major social and academic activities on the day that the questionnaire was sent).

Questionnaires on the Perception of the Course

At weeks 0, 3, and 10 (i.e. before the course, during lectures, and during TBL), we repeated questionnaires on the perception of the RS and DS courses. Responses to four statements recorded agreement from 1 to 5. Questions asked about: importance (This course is important for my medical training), interest (I find the content in this course interesting), intrinsic motivation (I want to study the content of this course), and stress (This course, including its assessments, is stressful).

Seven fourth-year medical students pilot-tested all questionnaires. Both English and French versions of SDLI-FR and CL-FR questionnaires are shown in Appendices 1 and 2.

Rationale for Comparing Weeks 2–4 and Weeks 10–12

RS and DS courses were assessed with multiple-choice written assessments at mid-semester (week 7 for RS, week 9 for DS) and end-of-semester (week 15). In our analysis, we compared out-of-class study hours between weeks 2–4 vs. weeks 10–12. Those weeks were distant from the pre-exam study “binges” and representative of students’ preparation for lectures (weeks 2–4) and TBL (weeks 10–12).

Data Analysis

We used SPSS v21™ for statistical analysis. Paired-samples t-tests were applied to assess the differences between the mean of individual and collaborative study hours during lectures vs. TBL. We applied independent samples t-tests to check whether differences in individual and collaborative study hours differed in 2018 and 2019, given that different grading systems were applied (i.e. pass-fail). Paired-samples t-tests were applied when calculating the differences in cognitive load during lectures vs. TBL. Paired-samples ANOVAs were applied when calculating the differences in perception for RS and DS courses.

For checking the validity of the SDLI-FR scale, we conducted an exploratory factor analysis (Appendix 1). The first model yielded a four-factor solution; however, only item 19 loaded on the interpersonal communication scale. We excluded interpersonal communication from the analysis. In the next model, items 1 and 16 had low loading on a factor other than proposed by the original scale. We excluded these two items. The final factor analysis yielded a 3-factor solution. A confirmatory factor analysis indicated that this model was more suitable than the full-item model. Scale reliabilities were acceptable (Cronbach’s alphas for learning motivation α = 0.699, for planning and implementing α = 0.823, for self-monitoring α = 0.688).

The first factor analysis of CL-FR scale indicated that item 13 did not fit the model well (Appendix 2). After its removal, the three extracted factors fit the originally proposed scale. The results of confirmatory factor analysis demonstrated a good fit of the model. Reliability for all scales was good to excellent (Cronbach’s alphas for intrinsic cognitive load α = 0.888, for extraneous cognitive load α = 0.909, for self-perceived learning α = 0.924).

Ethical Approval

After reviewing our protocol, the Research Ethics Committee of Laval University, applying rule 2.5 for quality improvement of educational projects, waived ethical approval. All participants signed a consent form. None of the authors had access to the data set until all students had received their final grades.

Results

Of the 459 undergraduate medical students, 324 entered the study (response rate of 71% for the questionnaires at week 0): 143/229 in 2018 (62% response rate) and 181/230 in 2019 (79% response rate). Response rates for each week are presented in Table 1. Participants’ demographic data and highest academic degrees for the 2018 and 2019 semesters are presented in Table 2. At week 10, 38/143 (27%) participants in 2018 and 131/181 (72%) participants in 2019 remained in the study (completed study logs, cognitive load, and perception questionnaires). Table 2 helps in the analysis of attrition bias. We present SDLI scores of the sub-group of 169 participants remaining in the study at week 10 in comparison with the scores of the 311 participants at week 0.

Table 2.

Participants’ demographic data and mean scores on the questionnaire of self-directed learning abilities (SDLI-FR) for the 2018 and 2019 semesters

	2018 semester (A to E grading) n = 143		2019 semester (pass-fail grading) n = 181
Age (SD)	21.1 (3.4)		21.6 (4.2)
Gender (ratio)	98 women (69%)		121 women (67%)
Highest degree before medical school College degree Undergraduate university degree Master degree Doctorate (Ph.D.) Others	109 18 2 7 7		133 29 6 6 7
Mean scores of SDLI-FR (completed once, at week 1); 1–5 scale (SD) Learning motivation Planning and implementing Self-monitoring Total	Participants at week 0 (n = 137) 4.3 (0.4) 3.8 (0.6) 3.9 (0.5) 4.0 (0.4)	Sub-group of participants who remained in the study at week 10 (n = 38) 4.3 (0.4) 3.8 (0.5) 3.8 (0.5) 4.0 (0.4)	Participants at week 0 (n = 174) 4.4 (0.5) 3.9 (0.6) 4.0 (0.6) 4.1 (0.4)	Sub-group of participants who remained in the study at week 10 (n = 131) 4.3 (0.4) 3.9 (0.6) 4.0 (0.5) 4.1 (0.4)

n number of participants for a given questionnaire, SD standard deviation

With respect to study strategies, participants (n = 153) used one or two resources to study for TBL: 98% used course notes/books, 7% used personal synthesis and less than 3% used reference books, Web sites, or videos. When studying for TBL, 24% of participants used self-directed testing strategies. Participants’ final scores for the RS course were 92.1% in 2018 (n = 38) and 90.4% in 2019 (n = 107). Participants’ final scores for the DS course were 87.4% in 2018 (n = 38) and 86.3% in 2019 (n = 107).

Answering research question 1, in the RS course, we found a similar mean number of study hours when comparing TBL vs. lectures (weeks 2–4 vs. weeks 10–12), with a mean of 3.1 h/week. Figure 1 aggregates the results of 2018 and 2019 for the mean number of study hours throughout the semester, as self-reported by students in their weekly study logs. Results of paired-samples t-tests are shown in Table 3. In the DS course, the number of individual and collaborative study hours is significantly higher in weeks 10–12 compared to weeks 2–4. In the second half of the semester, participants studied more for the lectures in the DS course (M = 5.5 h/week) than for TBL. Collaborative study was minimal outside class during TBL with a median of 0.1 h/week. If in-class time were reduced in the second half of the semester, 18% of participants said they would have used freed up time to study for TBL, while 34% said they would have used it to study for DS lectures.

Fig. 1.

Individual and collaborative weekly study hours outside class during lectures and team-based learning (aggregated data from 2018 and 2019). TBL team-based learning, RS renal system course, DS digestive system course

Table 3.

Paired-samples t-test results comparing out-of-class study hours between weeks 2–4 (lectures in both courses) and weeks 10–12 (team-based learning in the renal system course)

	2018 semester		2019 semester
	Individual study	Collaborative study	Individual study	Collaborative study
Renal System course	t(43) = 1.684 p = .099	t(43) =  −.547 p = .587	t(136) = .036 p = .971	t(136) =  −2.552 p < .05
Digestive System course	t(43) =  −5.719 p < .001	t(43) =  −5.063 p < .001	t(136) =  −9.978 p < .001	t(136) =  −7.519 p < .001

Answering research question 2, we did not observe a significant change in study hours in the RS course with pass-fail grading (Appendix 3 compares 2018 and 2019 results).

Answering research question 3, cognitive load and self-perceived learning scores for the RS and DS courses are presented in Fig. 2. Studying for TBL, when compared to studying for lectures in the RS course, generated a lower intrinsic cognitive load (M = 4.5 vs. 5.3, on a 0 to 10 scale, p < 0.001), similar extraneous cognitive load (M = 2.0 vs. 2.1, p = 0.732) and higher self-perceived learning (M = 7.2 vs. 6.6, p = 0.006). Studied as a potential confounding factor, the DS course generated a higher extraneous cognitive load that the RS course, with a significant increase from week 3 to week 10 (M = 4.4 vs. 3.6, p < 0.001).

Fig. 2.

Self-reported intrinsic and extraneous cognitive loads and self-perceived learning during lectures and team-based learning, on an agreement scale from 0 to 10 (aggregated data from 2018 and 2019 for renal and digestive system courses). Significant differences between weeks 3 and 10, at p < 0.05: ^at(146) = 4.615, p < .001,^bt(145) =  −2.768, p = .006, ^ct(122) =  −3.942, p < .001

Perceptions of the RS and DS courses before the semester, during lectures, and during TBL are presented in Fig. 3. Comparing TBL with lectures, participants gave a similar importance to the course (M = 4.8 vs. 4.7, on a 1 to 5 scale, p = 108), and maintained interest (M = 4.4 vs. 4.5, p = 0.09) and motivation (M = 4.3 vs. 4.2, p = 0.198). During TBL, they reported less stress related to the course (M = 2.8 vs. 3.1, p = 0.002). Concurrently, in the DS course, participants reported a slight decrease in interest (M = 3.9 vs. 4.1, p < 0.001) and motivation (M = 3.5 vs. 3.7, p < 0.001).

Fig. 3.

Perception of the renal and digestive system courses before the course, during lectures, and during team-based learning; on an agreement scale of 1 to 5. Significant differences before the course, at week 3 and at week 10, at p < 0.05: ^aF(2,322) = 4.762, p = .009, ^bF(2,322) = 6.435, p = .002, ^cF(2,322) = 19.941, p < .001, ^dF(2,322) = 40.467, p < .001, ^eF(2,322) = 14.016, p < .001

Discussion

In this study, we ascertained the safety of TBL when 3 weeks are implemented without redesigning the entire timetable. We used repeated quantitative measures of study hours and cognitive load within the evolving constraints of a full-time undergraduate semester. Five findings confirm that participants were not overloaded in this context. (1) They did not devote more time to study for TBL, although we know from the pre-exam weeks that they can quadruple this amount of time if needed. (2) We did not observe a difference in the study “binge” before the end-of-semester exam, suggesting that students felt as well prepared as before the mid-semester exams (3) Only 18% stated that they would devote additional study time for TBL were it available. (4) Extraneous cognitive load remained low during TBL, while simultaneously increasing in the DS lecture-based course. (5) Stress was reduced during TBL, but not in the DS course.

Previous studies, mostly based on fully flipped curricula, raised concerns that students could be overloaded by flipped-classroom approaches [1, 9, 13, 14, 18]. Both students and teachers misestimate the workload of flipped classroom if subjective measures are used [7, 9]. Using study logs, we measured learning behaviours repeatedly. We reduced bias by concealing the research questions and studying two concurrent courses.

We implemented only three consecutive weeks of TBL in one course. Awaiting dedicated studies, our results should not be extrapolated to fully flipped curricula where high numbers of study hours outside of class are to be expected. For example, in one of the rare studies using study logs in a problem-based undergraduate medical curriculum, students dedicated 33 to 41 h/week to study outside class [34]. In a study by Street and colleagues [14], implementing a fully flipped physiology curriculum, in-class time was reduced by 55%. Of students, 83.4% reported that the amount of freed-up time was appropriate [14]. As suggested by our comparison with the more complex DS course, what is taught in the course largely influences workloads and cognitive loads. Teaching very complex notions with TBL presents a risk of overload that should be monitored with validated questionnaires, ideally in a crossover study.

Changing to pass-fail grading had minimal effect on study hours, confirming a commitment to learning in TBL [20]. Students in medical programs using pass-fail grading have more freedom to allocate time to what they consider important for their future practice and choose their own strategy [14, 16]. As shown in Fig. 3, students found renal diseases important and interesting and it probably explains why they devoted time to it in a pass-fail system. Surprisingly, students did not collaborate to prepare for TBL. However, 3 weeks of TBL may not be sufficient to promote out-of-class collaboration. This outcome remains difficult to prove for all instructional methods supporting collaborative learning [23].

Bouwmeester and colleagues [35] showed that almost all students in flipped classrooms used the same basic material as pre-class study material, mostly text selections and web lectures. Likewise, in our study, almost all participants used teacher-made course books to study for TBL (Web lectures were not available). We believe it saved students time and contributed to reduce the extraneous cognitive load [36]. Bouwmeester’s students used case studies and formative tests to a lesser extent. They reported being overloaded [35]. Although our students were not overloaded, the use of test-enhanced learning strategies (quizzes, flash cards) outside of class was also relatively modest, given that TBL encourages it with readiness assurance tests. Self-perceived learning however remained high during TBL.

We chose open-book readiness assurance tests at the beginning of TBL courses, where students solved clinical problem. Open-book assessments promote critical thinking skills and are more engaging [37]. However, studies on pre-assessment effects would suggest that our participants’ cognitive and metacognitive study strategies were mostly driven by their desire to perform in mid- and end-of-semester exams, representing 94% of their final grade [18, 38].

Limitations

We recruited participants who reported high abilities for self-directed learning and high final scores in both courses. Consequently, students with learning difficulties are underrepresented in our homogeneous sample. Nonresponse attrition totalised 48% of the initial participants. Reassuringly, as presented in Table 2, the SDLI scores of the participants who completed questionnaires at week 10 were very similar to the scores of the participants at week 0. We concluded that the attrition phenomenon did not select more motivated or self-directed learners.

Our measures of extraneous cognitive load during TBL could be slightly underestimated because (1) TBL occurred later in the semester, hence students were more accustomed to the resources of the RS course; (2) students intuitively compared with the DS course for which they reported an increase in extraneous load. Indeed, we noted when conducting TBL that students had integrated the physiologic concepts from the first half of the semester. Renal diseases taught with TBL were moderately complex for our students, as suggested by the moderate intrinsic and low extraneous cognitive loads. Therefore our results only apply if moderately complex notions are taught with TBL.

In future studies, qualitative evidence, for example from focus groups conducted after the semester, would enrich our understanding of participants’ perception of the workload. Key elements for the success of TBL, like stress and motivation, should also be measured using validated instruments [24]. As for other flipped-classroom methods, the positive impacts of TBL are already proven for motivation, engagement, and performance [15, 39]. We did not design our study for that purpose. Moreover, readers should not conclude, based on our results, that TBL does not have a positive influence on individual and collaborative learning habits. This question would be better answered with studies on large-scale implementation of TBL with a profound redesign of the instruction, assessment and grading systems [15, 39].

Conclusion

We replaced 3 weeks of lectures by TBL without reducing in-class time. First-year undergraduate medical students did not report overload in study hours or in cognitive load and maintained a good perception of the course. Consistency without overload, self-directedness, and motivation are generally acknowledged as key factors for success in medical school [7]. Our approach appears safe with respect to not interfering with those factors and dynamics for success.

Supplementary Information

Below is the link to the electronic supplementary material.

Author Contribution

All authors contributed to the design and planning of the study. MCM, MRG and AL handled the logistical aspects. AL wrote the first draft of the manuscript. All authors critically revised drafts and approved the final manuscript for publication. MRG, PRDC, and YC taught the urinary system course in 2018–2019. MRG designed and taught the TBL sessions.

Declarations

Ethics Approval

After reviewing our protocol, the Research Ethics Committee of Laval University, applying rule 2.5 for quality improvement of educational projects, waived ethical approval.

Conflict of Interests

The authors declare no competing interests.