A Randomized Crossover Comparison of Team-based Learning and Lecture Format on Learning Outcomes

Abstract

Objective. To compare learning outcomes and student confidence between team-based learning (TBL) and lecture.

Methods. A crossover study was conducted with 30 students divided into two sections. Each section was taught six therapeutic topics (three TBL and three lecture). There were two assessments of 24 questions each. A survey (Likert scale) assessing student confidence and attitudes was administered at the end.

Results. A significantly higher overall examination score was observed for TBL as compared to lecture. Students were more confident in providing therapeutic recommendations following TBL. Higher survey scores favoring TBL were also seen related to critical-thinking skills and therapeutic knowledge.

Conclusion. Learning outcomes and student confidence in performing higher-order tasks were significantly higher with TBL. The findings of this novel crossover type design showed that TBL is an effective pedagogy.

INTRODUCTION

Traditional lecture may not be the optimal teaching strategy to help students learn and apply therapeutic or scientific content to clinical scenarios, or to other real world situations.¹ Earlier research in nonpharmacy disciplines suggest that active-learning strategies can significantly enhance student learning, even in large courses.^2-4 Today, pharmacy curricula incorporate active-learning pedagogies to promote higher-order learning.^5-7 The University of Michigan College of Pharmacy adopted team-based learning (TBL) as a unifying pedagogy across the 5-semester therapeutic problem-solving course sequence. Team-based learning facilitates active learning and engagement within and among small groups in a single classroom. Studies supporting the effectiveness of TBL are most often either positive or neutral with respect to measurement of short-term learning outcomes, including findings within our own curriculum.^8-14 However, none of these trials were conducted in a randomized, crossover type design, and often soft endpoints (eg, self-reflections, confidence ratings) were evaluated. In addition, investigations of long-term learning outcomes or the impact of TBL on performance in subsequent clinical practice experiences are currently lacking. Despite limited data, especially comparisons to other pedagogies, TBL promotes teamwork and can be effectively implemented at scale in small and large class settings and throughout a curricular sequence.¹⁵

Despite our adoption of TBL, we harbor concerns about the lack of evidence within a rigorous scientific construct showing that learning outcomes are significantly improved with TBL. To begin to address this educational gap we performed the following study comparing an active learning pedagogy (TBL) to a traditional, or punctuated lecture pedagogy evaluated within a rigorous scientific construct on learning outcomes and student confidence.

METHODS

An elective therapeutics course was prospectively designed to compare TBL to lecture pedagogy in a randomized crossover model. Therapeutic topics not covered in required therapeutic course sequence were included in this course. Students were randomly assigned (random numbers table) into either TBL or traditional lecture pedagogies. Following a 3-week course sequence, students were tested and then crossed over to the opposite pedagogy (Table 1). To emulate the team-teaching approach within the curriculum, six different faculty members taught in the course.

Table 1.

Weekly Topic and Examinations Schedule for the Elective Therapeutics Course

We employed a standard TBL method.^11,15,16 Briefly, students within the TBL cohort were assigned to teams of 5-6. Self-guided study materials were provided to students prior to class. At the start of class, a readiness assurance process was conducted and included an individual and a team readiness assurance test (10 questions). Following the initial assessment (individual and team ready assessment tests), the remaining time was devoted to team activities requiring synthesis and application of new knowledge to complete a series of multiple choice questions. The “4S” approach was used during the recitation period (significant problem, same problem, specific choice, simultaneous report).^11,16 For lecture-based pedagogy, faculty members could incorporate either a traditional or punctuated (active-learning techniques or cases embedded in the lecture) style. Each class session was scheduled for a 2-hour time period.

There were two assessment examinations given during the semester, each consisting of 24 questions. Within each examination, 12 questions tested application and 12 questions tested recall. For examination 1, there were 23 multiple-choice questions and one essay-type question. For examination 2, there were 24 multiple-choice questions. Each question was evaluated by three faculty members independently to assess whether the question tested application or recall. Simple majority was employed to assign the question.

A confidential survey (Qualtrics, Provo, UT) was sent to students following the end of the last class to assess student attitudes regarding TBL and lecture and their confidence in mastering the materials presented (Tables 3 and 4). We evaluated four identical Likert scale questions (5=strongly agree, 4=agree, 3=neutral, 2=disagree, 1=strongly disagree) focusing either on TBL or lecture pedagogy. In addition, we evaluated six questions that directly compared TBL to lecture. Preclass preparation time was also assessed.

Table 3.

Team-based Learning (TBL) and Lecture Question Scores

Table 4.

Team-based Learning (TBL) vs Lecture Question Scores

In comparing items in the TBL or lecture classes, we compared their means with paired t tests. Six questions with Likert responses were compared directly. Estimates of time students spent in self-study for each pedagogy were reported as ranges of minutes, and the average was used for comparisons. The six questions directly comparing TBL and lecture classes, and the scores had a midpoint of 3. Using a 2-sided one-sample t test with the null value of 3 (H0: μ=3), we determined whether there were significant differences in the positive or negative direction, and p≤0.05 was considered significant. The effect size for comparison of means was measured using Cohen’s d, which is a difference standardized by the pooled variance. The study was approved by the institution’s Health Sciences and Behavioral Sciences review board.

RESULTS

Thirty students were enrolled in the winter term elective course [28 second-year professional students (P2) and two third-year professional students (P3)]. The P2 students had completed two prior required TBL-based therapeutic courses, and the P3 students had completed four prior required TBL courses. Each instructor in the course had four years of TBL experience and 17 (9) years (range 4-29 years) in lecture experience.

Test scores (% correct) for application, recall, and combined (recall and application) examination questions are shown in Table 2. For each assessment, higher scores were seen for the TBL pedagogy. A significant higher score was seen in the TBL group when both application and recall questions were combined [89.2% (10.6) vs 85% (10.2, p=0.03].

Table 2.

Combined Examination Scores (2 Examinations)

All 30 students responded to the survey assessing confidence and preferences. The mean scores for identical questions are shown in Table 3. The mean score for all questions favored TBL pedagogy. Questions addressing confidence in providing therapeutic recommendations and being prepared to work in teams demonstrated significant differences, favoring TBL. However, there was no difference in student confidence in regard to therapeutic knowledge and recalling facts. When questions directly comparing TBL to lecture were evaluated, the mean scores for all questions favored TBL (Table 4). Five out of the six questions reached significance favoring TBL, including questions regarding critical thinking and therapeutic knowledge. Sixty percent of students preferred TBL to a lecture-based pedagogy compared to 13% of students who preferred lecture to TBL (27% no preference). Finally, the time students spent preparing for TBL class sessions was 134 (65) minutes vs 19 (30) minutes for lecture class sessions (p<0.01). One hundred percent of students spent time preparing for TBL classes whereas only 50% of students spent time preparing for lecture classes.

DISCUSSION

In this randomized crossover trial, significantly higher overall student assessment scores were observed for TBL. For both application and recall type questions, TBL scores were higher compared to lecture. Consistent with these findings were higher student confidence ratings in their therapeutic knowledge and ability to provide therapeutic recommendations following TBL. Students spent more time on self-study for TBL sessions than lecture and a majority of students preferred TBL to lecture format. Importantly, students felt that TBL pedagogy prepared them better to work in a team environment as compared to lecture pedagogy.

Previous data regarding learning outcomes comparing TBL to lecture format show either similar or improved outcomes with TBL format.^8-14 However, most of these studies are observational, cohort type of design. A community need exists to assess learning outcomes comparing different teaching pedagogies in more rigorous constructs. To this end, there are limited data comparing TBL to other pedagogies using a crossover design. However, Thomas and Bowen employed this type of design showed similar results. In their study, three clinical topics were taught by TBL and three by small group lecture to 112 medical students rotating through an ambulatory medicine type clerkship.¹⁷ In each of the two examinations, the TBL group had higher test scores (p≤0.05). Unlike our study, examination questions were not assigned as either application or recall. Despite this, the data from our study and their study show that short-term learning outcomes are improved with TBL compared to lecture when employing a crossover type design with students acting as their own control.¹⁷

One of the advantages of TBL is that this pedagogy focuses on learning and applying information to solve complex problems, whereas, traditional lecture formats tend to emphasize learning and recalling facts related to complex problems. This study was designed specifically to evaluate the performance of students on both application and recall type questions. Previously, we demonstrated that TBL may improve performance more than lecture on application-type questions in a retrospective study.¹¹ Thus, we anticipated that the TBL group would perform at a higher level on application-type questions and the lecture group would perform higher (or the same) on knowledge-type questions. However, our current data show that the TBL group scored similarly to the lecture group on recall and application questions. Unfortunately, there are limited data evaluating student assessments in TBL and lecture formats with regard to application vs recall questions. This is the first randomized crossover trial to show that TBL-based student performance on recall-type questions was similar to lecture-based student performance. Moreover, students’ perception of their ability to recall factual information was numerically higher for TBL compared to lecture. The findings suggest that a perceived limitation to TBL may not be significant. However, the suggested advantage of improved performance for application questions was not observed, though TBL did have approximately 5% higher score compared to lecture and the study had limited power (<50%) (Table 2).

Overall, our data assessing student perception in a number of different areas favored TBL over lecture, including student confidence in performing higher-order tasks (providing therapeutic recommendations and critical thinking). Our findings are also consistent with previous studies.^18-20 For example, in Johnson et al’s multiyear retrospective analysis of TBL in a pharmacotherapeutics course, scores significantly increased in students’ perceived abilities to apply course materials (to improve thinking, problem solving, and decisions).¹⁸

Regarding student preference in our study, students preferred TBL to lecture. Students taking this elective course were concurrently immersed in an integrated 5-semester therapeutics sequence and were well versed in TBL. Our findings may not apply to other circumstances, such as when students are not experienced in TBL, or TBL instruction is alternated with other pedagogies within a course.^15,21

Finally, students in the TBL pedagogy spent more time preparing for class than did lecture-based students. This is not surprising because the TBL-based pedagogy has mechanisms intended to ensure students prepare prior to class. Our data also confirms that students in a lecture-based pedagogy often do not prepare for class. Unfortunately, we did not collect data on how much time students prepared for examinations. Theoretically, TBL students should spend less time than lecture-based students preparing for examinations. It would be interesting to compare total study time between the two pedagogies. In addition, we did not capture student perception on time spent preparing for class. It is critical that preclass work is streamlined and efficient for students.^11-15 Preclass time commitment for incorporating TBL pedagogy needs to be considered within the context of the entire curriculum for effective implementation. We successfully incorporated a 5-semester therapeutic course sequence using TBL pedagogy in part by being cognizant of preclass time commitment.¹⁵ For example, we limited the number of pages that could be assigned from a therapeutic text book.

Limitations of this study include that only 30 students were enrolled in the elective course and that only six topics were covered during the course. In addition, there were only 48 questions assessed, all of which limited our power. For example, post hoc analysis suggest that the study had less than 50% power to detect significant differences in recall and application questions between pedagogies. However, despite these limitations, we did see trends in learning outcomes. Further, the data on student perception showed trends in favor of TBL pedagogy. However, there could be bias from instructors who favor TBL to lecture that may influence student perception. Although internal assessment has consistently shown since first implementing TBL (and at a time when faculty members were questioning choice of pedagogy) that a majority of students “liked” TBL. Another factor difficult to quantify is how effective the teachers were in presenting the material with either pedagogy. However, faculty members had much more experience teaching in a lecture format. More experience within a specific pedagogy may suggest better outcomes, but this was not seen in the study. Additional studies employing crossover type designs and larger sample sizes are required.

CONCLUSION

Our study demonstrated that learning outcomes were significantly improved with TBL compared to lecture. Student confidence in performing higher-order tasks was also significantly higher with TBL, and a majority of students preferred TBL pedagogy to lecture. These findings, based on a randomized crossover design trial, showed that TBL is an effective and desirable pedagogy compared to standard lecture format.