No differences in grades or level of satisfaction in a flipped classroom for neuroanatomy

Abstract

Objective

The intensive nature of a 5- or 6-week teaching block poses unique problems for adequate delivery of content. This study was designed to compare the delivery of a unit of undergraduate neuroanatomy in a short summer school period, as a traditionally taught unit, with a rendition given in the form of the “Flipped Classroom.” The aim was to evaluate the effectiveness of the flipped classroom in the intensive mode classroom.

Methods

The flipped classroom encompassed the same learning outcomes, but students were responsible for covering the content at home in preparation for tutorials that applied their acquired knowledge to higher levels of thinking. The main outcome measures were the final course grades and the level of satisfaction with the course.

Results

There were no significant differences between the 2 cohorts in final grades (p = .259), self-rated knowledge (p = .182), or overall satisfaction with the course (p = .892).

Conclusion

This particular design of the flipped classroom did not add value to the intensive mode experience. It may be that this mode of delivery is ill suited to intensive classes for subjects that carry a lot of content. The use of the flipped classroom requires further research to fully evaluate its value.

INTRODUCTION

A challenge that faces educators in teaching the medical sciences today is an increase in course content and a decrease in teaching time.¹ This is especially the case in the intensive mode teaching of a course, which compresses a traditional teaching program into a shorter session.² Previously, we conducted a comparison between traditional and intensive modes of delivery of a unit of neuroanatomy, and we found that the final grades were lower in the latter group.² This led to the rationale for this present study: to test the “flipped classroom” as a possible effective teaching method in the intensive or compressed delivery class.

The concept of the flipped classroom is attributed to Jonathan Bergmann and Aaron Sams, high school chemistry teachers who created podcasts and screencasts for students in 2006.³ In 2012, they created the nonprofit Flipped Learning Network (FLN) (http://flippedlearning.org/). “Flipping the classroom” means that teaching and learning, which originally occurred in the classroom (basic acquisition of knowledge), take place online in various ways and can be learned in an asynchronous way, and the time on campus is used for active learning,⁴ training the student in concepts that require higher order cognitive skills.⁵ During the on-campus sessions, the students inquire about more difficult aspects of course content, develop skills in applying the knowledge, and interact together in hands-on activity.^6,7 In a sense, this reversal also flips Bloom's revised taxonomy, in that the lower level of cognitive work, knowledge acquisition, is done by the students, and educators work interactively with the students to develop the higher forms of cognition.⁸ Fink's taxonomy of significant learning in acquiring foundational knowledge, application, integration, the human dimension, and learning how to learn is also potentially addressed in this model.⁹

The incorporation of technology in the flipped classroom can be beneficial for providing key content (lecture material, readings, interactive media) that students can access at their own pace in various formats (text, video, audio, multimedia). It is also helpful for providing online interaction and discussion, updates and reminders, immediate revision, and feedback (e.g., quizzes) and capturing data for analysis (http://www.uq.edu.au/tediteach/flipped-classroom/what-is-fc.html). Use of technology is especially suited to intensive mode delivery where the on-campus time is reduced significantly.

An important aspect of the flipped classroom is its ability to appeal to many learning styles. There are preferred learning styles, which can be incorporated into the flipped classroom with a variety of learning modalities.^10,11 Individual learning preferences are based on research that suggests people respond differently to different stimuli in the learning environment (e.g., visual, auditory, read–write, kinesthetic).¹² Thus, learning style is defined in terms of cognitive processes or learning behaviors.¹³ Cognitive style relates to the way in which a person processes information, and learning behavior is the preferred activity to acquire knowledge. Pedagogic literature documents the different learning styles of students in many ways. Bass and Vaughn¹⁴ identified 5 ways of learning: trial and error, perceptual organization, behavior modeling, mediation, and reflection. Kolb et al¹⁵ identified 4 kinds of learning abilities: concrete experience, reflective observation, abstract conceptualization, and active experimentation.¹⁵ There is essentially a continuum from concrete to abstract thinkers. Honey and Mumford¹⁶ derived a similar perception of learning styles as activist, reflector, theorist, and pragmatist. Thus, for some students, learning by activity is best (activist); others are reflectors who want to be given information that they think over before actually being active; while others want to analyze and theorize (theorists); and finally, there are those who cannot easily study something unless they see it is relevant to their future careers.¹⁷

The flipped classroom can attempt to meet these learning needs. Lage et al,⁷ for example, designed a flipped classroom specifically to make their course more compatible with their students' varied learning styles; they saw it as a means of providing the students with a variety of learning options to gain first exposure to material outside of the traditional lecture: readings, online podcasts or videos, PowerPoint presentations with audio, and downloadable PowerPoint slides. The flipped classroom moves away from the face-to-face lecture, for which there is controversy regarding its effectiveness in teaching.^17–19 The flipped classroom gives students more latitude to work at their own pace, and to access learning at times that suit them better. This method fits the lifestyle of today's student. The face-to-face times are interactive, which has been said to favor the millennial students who prefer teamwork and interactive learning.⁵

Krebs et al²⁰ have studied using the flipped classroom in neuroanatomy teaching for medical students and found that 70% of students agreed or strongly agreed that this mode of delivery enhanced their learning. They did find that 43% of the students did not find the workload reasonable, 20% were neutral, and 35% thought that it was reasonable. They concluded that the flipped classroom is hard for both faculty members and students.

But how well does the flipped classroom work in chiropractic teaching? This study evaluates the effectiveness of the flipped classroom for chiropractic students studying undergraduate neuroanatomy and compares it to a more traditional mode of teaching. The objective was to evaluate the effectiveness of the flipped classroom (2013) compared with more traditional teaching (2011), within this session.

METHODS

Study Population and Sample

All students enrolled in a 2nd-year undergraduate neuroanatomy unit at Macquarie University, in session 3 in 2011 and 2013, were invited to participate in this study. Students who do session 3 typically attend the sessions during the year, and then for a variety of reasons choose to pick up 1 or 2 units in the summer break. They are a smaller group compared with groups that go through sessions 1 and 2 of the traditional year.

Materials

The same questionnaire was given to both cohorts. The questionnaire collected basic demographic information, self-reported estimates of average time spent per week in home study, and a self-rating of the level of knowledge and understanding (scale: 0–100) that the students believed they had attained. The survey also asked a number of questions designed to rate the students' satisfaction with the unit overall and with the 3 sections of the course: lectures, laboratory practical classes, and tutorial classes. In addition, we collected the following information from university records: the final grade for the neuroanatomy unit, which is provided as a standard numerical grade (SNG) on a scale from 0 to 100; the university preadmission score of either the University Admission Index (UAI) or the Australian Tertiary Admission Rank (ATAR) on a scale from 0 to 100; and the grade point average (GPA) that students had attained at that point in their degree program on a scale from 0 to 4.

Table 1 provides an overview of the assessment tasks and their relative contributions to the final grade for both the 2011 and the 2013 cohorts. Because of the reduction in time for the unit in 2013, a seminar presentation and midsemester test became impractical. Weekly tutorial-based worksheets and quizzes were allocated a total of 25% of the grade. The final theory examination was structured similarly for the 2 cohorts, and to the same level of difficulty in order to ensure adequate comparison between the 2 years. This final examination, as with all theory examinations, was structured to examine content knowledge, level of understanding, and higher order thinking. The range is intended to effectively challenge students at every level and to provide a good representation of knowledge and ability in the spread of marks. Thus, understanding and integrative ability were tested in both cohorts.

Table 1.

Assessment Tasks

Task	Weighting (%)
	Flipped Classroom Mode, 2013	Regular Intensive Mode, 2011
Weekly revision quizzes	15	0
Seminar presentation	0	20
Weekly worksheets	10	0
Midsemester test	0	20
Practical examination	25	20
Final theory examination	50	40

In order to investigate student satisfaction for each section of the course and for the course overall, students were asked to respond to 5 statements in each section of the questionnaire, using a standard 5-point Likert scale. The statements were adapted from questionnaires used by Macquarie University Learning and Teaching Centre to evaluate units at the end of each semester. The levels of agreement ranged from “strongly agree” (scoring a 4), to “strongly disagree” (scoring a 0). The cumulative satisfaction score would then range from 0 to 20 for each section. The 5 statements were as follows: (1) the (section) assisted my learning; (2) the (section) kept me engaged; (3) the (section) was effective in developing my understanding; (4) the (section) was enjoyable; and (5) sufficient time was allocated for the (section).

Procedure

Session 3 was run over 6 weeks in 2011 and over only 5 weeks in 2013 owing to central changes that were made to the summer session. Face-to-face teaching hours in 2011 were 36 hours per week (4 hours of laboratory practical classes and 2 hours of tutorial classes per week for 6 weeks); in 2013, face-to-face teaching hours were 40 hours per week (4 hours of laboratory practical classes and 4 hours of tutorial classes per week for 5 weeks). This is in contrast to the more typical 78 hours of face-to-face teaching in a traditional session.

The 2 cohorts had identical learning outcomes. In both cohorts, the lectures were only available as audiovisual recordings of the lectures originally delivered to students earlier in the year. These were available as recordings on the university Web-based electronic learning platform, a Moodle Web page that also provided students with presentation slides as a portable document file. The laboratory practical classes were also identical for both cohorts: 3 tutors headed 3 rotating groups that covered human prosection specimens, anatomic models, and living/functional anatomy. The students in both cohorts used a practical manual, which gave week-by-week instructions and activities for the laboratory practical class. However, the orientation of the units was different. In 2013, the flipped classroom was focused around the 3rd component of the course: the tutorials. The time spent in tutorials each week was doubled for this group, from 2 to 4 hours. Students in the flipped classroom cohort were expected to watch specific lectures each week and go through assigned readings ahead of the tutorial classes. They had to complete the weekly quiz based on this work, which constituted a component of their assessable work. They also had to complete worksheets based on this work, which were downloaded from iLearn (Moodle Learning Management System, Macquarie University, Sydney, Australia) and which were handed in at the tutorial classes for marking. Together this work constituted 25% of the grade. There was no allocation of marks to this component in the 2011 cohort.

The tutorial classes were structured to make use of this preparatory work and had the students work together on problem solving and case studies, which were intended to process and integrate the information, with educators acting as guides on the side. The students in the 2011 regular intensive mode cohort did not have the expectation of preparing for tutorial classes ahead of time; the tutorial classes were much more tutor driven and were not reliant on the large component of assessable “homework” the students were expected to complete.

Ethics approval to conduct this research was obtained from the Macquarie University Human Research Ethics Committee (reference numbers: 5201100130 and 5201300691). We obtained informed written consent from all participants at the start of the session, and the questionnaire was administered in the final week of work, prior to the final exam. Data from university records were retrieved after the census dates for each of the sessions.

Statistical Analyses

Analyses of all data were done after the final grades were officially released. Responses to the questionnaires were entered into a spreadsheet and scored. The mean and standard deviation were calculated for all continuous variables (i.e., age, hours of self-directed study, self-assessed knowledge, GPA, UAI/ATAR, SNG, and satisfaction scores), and the Welch 2 sample t test was used to check for group differences. Dichotomous variables (i.e., gender, English/non-English as first language, previous/no previous degree, domestic/international student) were presented as frequencies and proportions, and Pearson's χ² test with Yates' continuity correction was used to check for group differences. All statistical analyses were conducted using R, version 3.0.2 (R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

A total of 64 students were enrolled in the 2 sessions (2013: n = 29; 2011: n = 35). Four students, 1 from the 2013 cohort and 3 from the 2011 cohort, withdrew from the unit or did not complete the course requirements. Of the 60 students who completed the neuroanatomy unit, 56 students agreed to fill out the anonymous questionnaire at the end of the session (2013: n = 23; 2011: n = 33).

Table 2 provides a detailed overview of the baseline characteristics of the 2 cohorts. The 2013 flipped classroom and 2011 regular intensive mode cohorts were not statistically different with regard to age (p = .989), percentage domestic students (p = .411), GPA (p = .965), or ratio of native English speaking to English as a second language students (p = .688). The average university preadmission scores (UAI or ATAR) (p = 0.044) were different. In addition, although the 2013 flipped classroom cohort reported a greater number of hours of self-directed study per week, which would be expected in the preparation needed for the tutorial classes, the difference was not statistically significant (p = .250).

Table 2.

Baseline Characteristics

Characteristic	Flipped Classroom Mode, 2013	Regular Intensive Mode, 2011	p Valuea
Mean age (SD)b	23.18 (10.41)	23.21 (6.30)	.990
Grade point average (SD)c	2.24 (0.77)	2.24 (0.94)	.997
University preadmission score (SD)c	73.83 (8.85)	77.92 (10.58)	.179
Hours of self-directed study per week (SD)b	11.98 (10.28)	9.34 (7.23)	.317
Number of female students (%)c	–	21 (66)	–
Number of domestic students (%)b	22 (96)	33 (100)	.855
Number of students with English as first language (%)b	21 (91)	28 (88)	.994
Number of students with previous degree (%)b	4 (17)	3 (9)	.639

^aWelch 2 sample t test for continuous variables; Pearson's χ² test with Yates' continuity correction for dichotomous variables.

^bData based on questionnaire responses.

^cData based on university records of students that completed the unit.

Table 3 provides an overview of the mean grades, self-rated knowledge, and satisfaction scores for the 2 cohorts. Although the students in the 2013 flipped classroom cohort achieved slightly better grades, the difference was not statistically significant (p = .259). The 2 cohorts did not differ with regard to self-rated knowledge (p = .182) or overall satisfaction with the course (p = .892).

Table 3.

Mean Grades, Self-rated Knowledge and Satisfaction Scores

Outcome Variable	Flipped Classroom Mode, 2013	Regular Intensive Mode, 2011	p Valuea
Self-rated knowledge (SD)b	62.50 (13.25)	67.06 (10.22)	.182
Standard numerical grade (SD)c	58.61 (9.05)	55.28 (13.38)	.259
Satisfaction (SD)b
Overall	14.61 (3.68)	14.48 (2.79)	.892
Lectures	12.96 (4.20)	13.45 (3.45)	.642
Laboratory practical classes	17.91 (2.43)	17.33 (2.39)	.381
Tutorial classes	13.43 (4.87)	13.94 (2.61)	.653

^aWelch 2 sample t test.

^bData based on questionnaire responses.

^cData based on university records of students that completed the unit.

DISCUSSION

Our study found that in the intensive mode setting, the flipped classroom and the traditional mode of delivery resulted in the same grades and level of satisfaction among students in our undergraduate neuroanatomy course.

Neuroanatomy is a vast subject, which poses learning and teaching challenges in the intensive mode. Prober and Khan¹ suggest that the expansion of biomedical knowledge creates a need to “reimagine” medical education in a way that addresses the “digitally empowered learner.” The challenge is to create teaching models that can move some of the learning online, maximize efficient use of the face-to-face times, and engage students in actively managing their own learning. It has been shown that active participation of students enhances retention and understanding.⁵ The flipped classroom appeared to us to be a possible solution in enhancing knowledge acquisition and understanding in session 3. Jon Bergmann and Aaron Sams, describe the flipped classroom as having shifted the emphasis of teaching from the instructor to the learner.²¹ Prober and Khan¹ describe it as a process of building a core content framework, imparting the knowledge in interactive and engaging formats, and encouraging the pursuit of in-depth knowledge in a few chosen areas.

Many authors do report favorable outcomes from this mode of delivery. Lage et al⁷ noted that more information could be covered, the interaction allowed for collaborative acquisition of knowledge, and assessment and feedback were well accommodated. The variety of different exercises offered engaged different learning styles, which enhanced understanding. They also expressly pointed out that their students were motivated by grades and soon realized that they had to take responsibility for their own learning and work continually to keep up with the material in order to get a high grade. At the start of the course, the instructors made it clear that a lot of work was required, and that if the student was not prepared to do this, they should not take the course.

In a study very similar to ours, Tune et al²² compared the traditional delivery of a unit of cardiovascular, respiratory, and renal physiology with a modified flipped classroom delivered to 1st-year graduate students. Both sets of students were given recorded lectures, but the students in the flipped classroom were expected to prepare for class by going through the lecture, and in the class they were given a quiz or homework on the lecture material and problem-solving exercises based on the lecture material. Both sets of students were evaluated with the same final exam. The students in the flipped course did significantly better (p < .05). The authors found the flipped classroom to be highly effective in developing an understanding of key physiologic concepts in graduate students. Jeavens et al²³ similarly compared a cohort in a traditional class of anatomy and physiology for undergraduate nursing students with a flipped classroom and found that although the flipped classroom improved student participation and learning, it had to be carefully managed so that staff and students understood roles and expectations. De Ruisseau²⁴ used the flipped classroom in an undergraduate anatomy and physiology course that required the students to watch e-lectures and review questions outside of class. The review questions were discussed in teams in class, and students prepared the answers for assessment. Case studies were also discussed in these classes. They found that the mean score for the first exam was higher than previous years (79.2 vs 74.3; p < .01) suggesting increased student performance resulting from this method. Thus it is that similar programs to ours have resulted in success.

There are studies that do report difficulties with this method. For example, Jacob Enfield at California State University Northridge did find some resistance to his flipped classroom: “students... believed it was my duty to teach, not direct them elsewhere for information... students... believe they are paying for a structured education where the role of the teacher is to “teach” more than it is to facilitate learning in an independent learning environment.”²⁵ Sawarynski et al²⁶ found the flipped classroom for medical students was generally well accepted, but the students did not perceive video lectures as important and came to class poorly prepared. Missildine et al²⁷ found that examination scores were higher for the flipped classroom but that students were less satisfied with the flipped classroom (p < .001). They concluded, “Blending new teaching technologies with interactive classroom activities can result in improved learning but not necessarily improved student satisfaction.” Chen et al²⁸ found that while there was appeal for the flipped classroom for many reasons, many students had trouble adapting to it. The students commented that they did not have enough time when attempting to finish the preparatory work and had problems understanding some of this work. They felt the amount of work was too heavy, the effort great when there were no instructors around to ask about it, and that if they did not do the preparatory work they were lost in class. In interviews of students, the authors also found that, of those students who were less satisfied, one of the problems was that they resisted adopting the new model because “their old, passive learning habits required less effort.”²⁸

These factors could be contributing to the outcomes of our study. Flipping the classroom requires flipping students' perceptions in order for them to accept the responsibility for teaching themselves and spending their own time online to cover much of the work. It could be that students are reluctant to do the extra work required to prepare before class if they do not see the benefit of doing it. This was reflected in the questionnaire comments made by students, particularly in regard to the tutorials. While this was not an overwhelming finding, there were comments such as “we should be revising instead of doing clinical case studies” and “too much time was wasted in these classes.” Clearly some students did not see these as authentic tasks, and it suggests that there is a preference to passively absorb information rather than engage interactively in developing it. Some students also said they preferred face-to-face lectures to keep them engaged, again indicating a reluctance to change to a new model of learning.

Integral to this discussion of results is the design of the flipped classroom, the quality of the material, and how well the unit is taught. It is necessary to design carefully how face-to-face time is spent and to incorporate effective engagement.⁵ The online preparatory work should be varied and useful.²⁷ Egbert et al²⁹ proposed 8 conditions for optimal technology-supported learning environments in which learners have opportunities to (1) interact and negotiate meaning, (2) interact in the target language with an authentic audience, (3) become involved in authentic tasks, (4) be exposed to and encouraged to produce varied and creative language, (5) have enough time and receive feedback, (6) attend mindfully to the learning process, (7) work in an atmosphere with an ideal stress/anxiety level, and (8) have supported autonomy.

If these are essential for learning, then our flipped classroom was weak in the areas of time, feedback, and keeping stress levels low. We had decreased the session from 6 to 5 weeks but had increased the amount of work the students had to do to prepare for class. There was less time for properly assimilating the information, and also less time for teachers to provide feedback. The short time made it impossible to have an assessment like the seminar presentation assignment given to the 2011 class, which was an opportunity for students to use their minds creatively in a learning process. There was also not enough time to conduct a midsemester exam, which the 2011 cohort received; this would have given the students more feedback on how they were doing. In addition, the online audiovisual lectures could have been more effective in engaging students if they had been broken down into shorter and more frequent podcasts.³⁰

Kim et al³¹ conducted the flipped classroom in 3 different disciplines and extracted successful design principles from surveys. They found that it is important to provide an opportunity for students to gain first exposure prior to class and an incentive for students to prepare for class. Examples of preparing for class might include building in quizzes to the preparatory work or a requirement to post a discussion. Kim and colleagues also share that it is important to provide a mechanism to assess student understanding formatively through the use of tools such as in-class quizzes. It is also important to provide clear connections between in-class and out-of-class activities so that the 2 are cohesive, and to offer clearly defined and well-structured guidance and prompt/adaptive feedback on individual or group works. It is also important to build a learning community by creating a sense of belonging in the classroom.³¹

We suggest that the added responsibility of owning the learning process can be stressful when large amounts of work must be covered in short intensive blocks of teaching. The present study did not add value to the intensive mode experience. Far from being the answer to this type of teaching situation, the flipped classroom may be less favorable for teaching and learning. There is another way to look at this: the students perform to the same level in the intensive mode of delivery regardless of the method used. As we know from our previous research, students do not do as well in intensive mode courses compared with the longer traditional sessions,² and therefore, what the results may indicate is the problem is around the short time period of this session, which is difficult in the case of a content heavy subject like neuroanatomy. The use of the flipped classroom requires further research to fully evaluate its value.

Limitations

The final SNG was used to compare the 2 cohorts. But this grade is dependent on the appropriateness and degree of similarity of the assessment tasks. The schedule of assessments was not the same for both cohorts. The final exam, however, was very similar and covered the same learning outcomes as a way of making it more comparable and maintaining the same high standard across all iterations of the one course. However, this may not reflect attained knowledge and understanding in the 2 groups. With regard to the questionnaire used, this has not been tested for reliability and validity. The statements pertaining to student satisfaction were, however, adapted from official Macquarie University Learner Evaluation of the Unit surveys.

CONCLUSION

The flipped classroom worked as well as traditional teaching in the intensive mode delivery of a unit of undergraduate neuroanatomy. Future research could be centered around the delivery of the flipped classroom; for example, increasing the variety of the online experience, converting long lectures into short podcasts, and preparing students better in order to see the importance of each aspect of the work they are doing.