Comparison of student performance and perceptions of a traditional lecture course versus an inverted classroom format for clinical microbiology

Abstract

Objective:

Student satisfaction and student performance are of primary concern when classroom pedagogy is changed. We determine the equivalence of two teaching methodologies in a clinical microbiology course using test scores as the measure of student performance.

Methods:

The two teaching methodologies examined were a traditional lecture-based method face-to-face (F2F) method and an inverted classroom method (ICM). Student perceptions of the ICM method were measured using a course survey in which students were asked to compare their experiences in the ICM class with experiences in a traditional F2F class. Classroom exams were administered in the same way in the traditional F2F lecture and ICM courses. Student test averages obtained in both pedagogies were compared for equivalence using an independent samples t-test. A six-question survey was developed to assess student perception of the ICM classroom compared to that for the traditional lecture-based classroom.

Results:

Test performance of students in the ICM was equivalent to that of students receiving traditional F2F lectures. Mean difference between test scores for the ICM and traditional F2F groups was 1.9 points (95% confidence interval [CI], −4.0–0.14). Survey responses indicated that respondents feel positively about self-learning in ICM and prefer the flexibility provided by ICM.

Conclusion:

This study provides evidence that the ICM method of teaching clinical microbiology can replace the traditional F2F method without loss of student performance. Respondent perceptions of the inverted classroom were positive, with students favoring the flexibility.

INTRODUCTION

Healthcare education has long relied on traditional lectures to deliver course content to students. In these lectures the instructor typically uses the “stand and deliver” method, wherein students passively receive information pertinent to the course. In recent years, some professional programs have been exploring a variety of teaching and learning technologies. The interest in multimedia resources and techniques has experienced cycles of attentiveness, followed by declining enthusiasm. Currently, many programs use some mix of online and traditional lecture techniques, alternatively termed “blended learning” or “hybrid learning,” in a flipped classroom.^1,2 The term “flipped classroom” often may be applied to primary and secondary education, whereas the term “inverted classroom” is best applied to higher education.³ The inverted classroom model (ICM) is a blended learning model that requires the student to acquire knowledge through self-directed learning online, and then attend face-to-face lecture (F2F) that compels the student to apply the knowledge. This contrasts with a traditional approach, where the F2F lecture conveys the knowledge and the student is expected to self-direct the application of the knowledge outside of class. Thus, the traditional way of learning is inverted.³

Several advantages are noted when an inverted pedagogic method is used, although evidence indicates that groups of students may respond differently to ICM. The inverted classroom allows for students to pace themselves as they move through the online materials. Since students can only concentrate for approximately 10 minutes in a lecture before their attention diminishes, the use of online materials can allow them to break as needed.⁴ Following acquisition of knowledge via the online materials, students assemble as a class to apply the knowledge at the direction of the instructor. A meta-analysis by the United States Department of Education found that this type of blended learning leads to better results than entirely online or entirely classroom teaching.⁵ Student satisfaction is noted to increase in some studies where medical students acquire knowledge ahead of classroom time. This increase in satisfaction may arise for a number of reasons, including ability to self-pace learning and/or use of multiple types of learning tools.⁶ However, nursing students experienced decreased satisfaction when presented with an inverted course, despite scoring higher on learning assessments.⁷ These differences in student satisfaction may be due to course organization or perceived lack of access to the instructor.

Reports of courses using ICM in various professional schools detail experiences with a variety of course content.^2,6,7 A flipped neuroanatomy course was described in a chiropractic college setting with no difference in performance or student satisfaction.⁸ The authors concluded that ICM may not be the appropriate mode of delivery in courses that are concerned with intensive amounts of content. They also noted that more should be done to investigate ICM in chiropractic education.⁸

We detailed the conversion of a traditional F2F clinical microbiology lecture course in a doctor of chiropractic program to an ICM format. The objectives of this report were: (1) to examine student satisfaction with clinical microbiology content delivered online with reduced F2F interaction and (2) to examine the effect both teaching methodologies on student performance. The interchangeability of ICM and the traditional F2F lecture method required similar exam performance in both settings, less than one grade or a 10-point mean difference in test performance between methods. Our hypothesis was that ICM and the traditional F2F lecture method mean test averages were equivalent.

METHODS

Sample

Clinical microbiology is taught to 2nd year chiropractic students in the 5th quarter of the curriculum. The traditional F2F lecture course was taught in fall 2015. The ICM course was taught in fall 2016. Both groups were surveyed and their performance on exams was compared. Characteristics of each group, such as age, sex, and race, were collected. Students in both courses attended a weekly laboratory session. No pedagogic changes were made to the laboratory portion of the clinical microbiology course. This study, with use of the student survey, was identified as exempt from institutional review board oversight by the University of Western States (UWS) institutional review board.

Traditional Course Design

The traditional clinical microbiology course was delivered over a 10-week term using a F2F lecture format where the class met 5 hours per week. Course content was delivered to the students using PowerPoint (Microsoft Inc, Redmond, WA) slides as the instructor spoke to the class. The students followed the lecture content using a course note packet. Exams were given in selected hours of the assigned F2F lecture time.

Online Course Design

The online materials prepared for the clinical microbiology course were carefully constructed in a Moodle platform with strict attention to how the students would receive the information in the absence of the instructor to guide them. The ICM course was designed to meet in person for 1 hour a week. This implies that 4 hours of materials should be prepared for delivery online. Care was given not to increase the amount of workload on the student in the online environment versus the F2F lecture modality. Clinical microbiology course content provided information on a variety of human pathogens. Given that, each week of the online course was organized by dividing content by specific pathogen. Thus, all materials for one organism were grouped together under one heading. Each heading contained at least one audio lecture using Panopto software (Panopto, Seattle, WA), PowerPoint slides from the lecture, text notes, and a short quiz (Fig. 1). The audio lectures were limited to 15 minutes in length. If longer times were required, the content was broken down into appropriate components that could be viewed in succession. Best practice was to have short lecture recordings, at most 20 minutes in length.⁹ The short quiz at the end of each heading was a formative quiz containing three to five questions. Exams for the course were given in the 1 hour of F2F classroom time.

Figure 1.

Screenshot of the organization of the course materials. Each pathogen is listed followed by an audio lecture, slides, notes, and review questions. Audio lecture times are noted; if 10 minutes are exceeded the lecture in broken into two or more videos.

Classroom Meeting in ICM

Once a week, the ICM class was scheduled for a 1-hour in-person meeting. Three of these 10 meetings were used to administer 50-question multiple-choice examinations. These examinations were created from the same question banks used for constructing traditional lecture-based F2F examinations in previous years. The traditional F2F examinations also were 50-question multiple-choice examinations. This eliminated the possibility of having testing methodology as a confounding variable that would obscure the effect of the two types of pedagogic methods on student learning.

The remaining seven weeks of F2F time were devoted to applying the information the student learned online. The learning activity used in the F2F hour was to present clinical cases and use an audience response system to allow students to answer questions regarding the cases. This strategy allowed for two important goals of the F2F time to be achieved. First, it required the students to apply the microbiologic knowledge gained online to answer questions raised in the clinical case scenarios, and second it allowed the students to interact and discuss their responses before and after voting using the audience response system.

Survey Instrument

A survey was developed to secure students' feelings regarding the new ICM format of the clinical microbiology course. Most survey items were adapted from items that appeared in the validated student survey, which was a component of the Blended Learning Toolkit developed by the University of Central Florida and the American Association of State Colleges and Universities.¹⁰ Other survey items were adapted from the Quality Matters Higher Education Rubric (5th edition, 2014) and the Web Learning Project Student Survey.¹¹ Items identified as “best practice” for this type of application also were consulted.^12,13 It should be noted that survey variables are secondary outcomes since the survey was administered only to a single class of students.

The face validity of the survey was established by the director of academic assessment at UWS and by two other faculty members at UWS. These faculty members also teach ICM courses and subsequently used this survey in their courses. Cronbach's α was used to determine the internal consistency of the scaled, closed-ended items (#1, #2, and #3) on the survey. Cronbach's α was calculated using XLSTAT (Addinsoft, New York, NY). The Cronbach's α for this group of survey items was 0.97, indicating excellent internal consistency in the responses.

The survey was administered at the end of the quarter. Participation in the survey did not affect the student's grade and was not linked to the student's name, sex, or any other identifying characteristic. The six-item survey provided students with the opportunity to indicate their level of agreement with various statements about the course, including its design, content delivery, availability/usefulness of materials, and impact on learning. Each statement was phrased positively, reflecting the potential benefits and desired outcomes associated with an ICM course. A six-point Likert scale (strongly agree, agree, neither agree nor disagree, disagree, strongly disagree and don't know/not sure) was used to obtain student feedback in the first three survey items. Below the body of each of these items, students were given the opportunity to provide comments.

In survey item #1, students were asked to indicate their level of agreement to statements that focused on the quantity/quality of learning that occurred in the ICM course compared to their learning experiences in traditional courses. Survey item #2 focused on content delivery and the effectiveness of the instruction that was provided in this course. Desired content delivery and instructional qualities (logical topic sequence, topic organization, benefits derived from lecture, and alignment of online and class activities) were listed in this item and students were asked to express their level of agreement with the statement associated with each quality.

Survey item #3 focused on the online component of this course including the course materials (availability, clarity, usefulness) and specific benefits (increase in quantity and quality of interactions with the instructor) that students were to attain from the online component of the course. In addition students were asked their level of agreement to the following statement: “I possess the time management skills needed to be successful in this type of course format.”

The last section of the survey provided students with the opportunity to express their overall impression of the ICM course format, including what they liked most about this course format (survey item #4) and what they would like to change about this course format (survey item #5). In the last item of the survey (item #6), students were asked the following question: “If you were given a choice regarding how basic science courses were taught, which format would you most prefer?” The course format options ranged from “Entirely face-to-face” to “Entirely online with no face-to-face time.” The other format options listed represented various mixes of web content and F2F class time.

Statistical Analysis of Student Performance

A test of teaching method equivalence was conducted using averages of the four tests administered during the term for the clinical microbiology course. Test performance was the primary outcome. Ten points, or a whole letter grade, was established as the critical difference in student performance between the lecture-based F2F and ICM methods. An independent-samples t-test was conducted to determine if there was statistically significant difference between the test score means of the two classes (SPSS for windows, IBM Corp, Armonk, NY). The null hypothesis was rejected and the equivalence between the ICM and traditional lecture F2F methods accepted, if the 95% confidence interval (CI) for the mean test difference between groups (2-tailed test of significance) was included within the ±10-point threshold of an important difference in test performance.

RESULTS

Sample

The traditional F2F lecture course was taught in fall 2015 to 81 students. The ICM course was taught in fall 2016 to 98 students. Data regarding the demographics of both student populations were obtained from the registrar. This information was obtained as bulk data, the particular characteristic (e.g., sex or age) is not linked to a specific student, since the surveys were anonymous for identifying characteristics (Table 1).

Table 1.

Student Characteristics

Characteristic	Fall 2015		Fall 2016
	N	Mean (SD)	N	Mean (SD)
Age	81	27.2(4.6)	98	26.6(4.0)
	N	%	N	%
Sex
Male	50	61.7%	55	56.1%
Female	31	38.3%	43	43.9%
Race
American Indian/ Alaska Native	0	0.0%	0	0.0%
Asian	6	7.4%	10	10.2%
Black/African American	0	0.0%	1	1%
Pacific Islander	0	0.0%	0	0%
Caucasian	67	82.7%	78	79.6%
More than one race	8	9.9%	7	7.1%
Not specified	0	0.0%	2	2.0%
Ethnicity
Hispanic	3	3.7%	5	5.1%
Not Hispanic	78	96.3%	91	92.9%
Degree
Bachelors	67	82.7%	65	66.3%
No bachelors	14	17.3%	33	33.7%

Student Performance

All students completed their exams so that there were no missing data for the primary outcome. Mean difference between groups was 2.0 (95% CI, −4.0–0.14). As the 95% CI fell within ±10 points, the null hypothesis was rejected and the equivalence for student performance for the two teaching methods accepted. There was no significant difference between the test scores of the lecture-based F2F class (mean [M] = 86.1, standard deviation [SD] = 6.6) and the ICM class (M = 84.1, SD = 7.2); t(173) = −1.91, P = 0.057, d = 0.2.

Survey Results

In fall 2016, 36 of 98 students completed the survey regarding their opinion of the ICM format for learning clinical microbiology. The results of survey item #1 regarding quantity/quality of learning, indicated that 97% of the respondents agreed or strongly agreed that they have better control over the pace of their own learning in the ICM model. Furthermore, 97% agreed or strongly agreed that the ICM format encourages self-directed learning. With regard to understanding of course materials, 70% agreed or strongly agreed that the ICM model allowed for a better understanding of the course content. A lesser percentage of students (61%) agreed or strongly agreed that the ICM format created stronger course engagement. The survey results regarding understanding of course requirements and more opportunities to reflect on learning saw a moderate percentage of individuals that neither agreed nor disagreed: 39% and 28% respectively (Table 2).

Table 2.

Survey Responses for Items 1 to 3

Survey Item	Strongly Agree		Agree		Neither Agree Nor Disagree		Disagree		Strongly Disagree		Don't Know/ Not Sure
	n	%	n	%	n	%	n	%	n	%	n	%
#1 Compared to my experiences with traditional courses,…
This hybrid course is providing me with more opportunities to access and use information.	13	36	13	36	8	22	2	6	0	0	0	0
This hybrid course is providing me with more opportunities to reflect on what I've learned.	14	39	8	22	10	28	4	11	0	0	0	0
The course format is helping me to better understand the course material.	14	39	11	31	5	14	6	17	0	0	0	0
I am more engaged in this course.	12	33	10	28	8	22	4	11	2	6	0	0
I better understand the course requirements.	12	33	8	22	14	39	2	6	0	0	0	0
The format of this course encourages more self-directed learning.	23	64	12	33	0	0	1	3	0	0	0	0
I have better control over the pace of my own learning in this course.	22	61	13	36	0	0	1	3	0	0	0	0
#2 and #3a Please indicate your level of agreement with each of the following statements:
Topics are presented in a logical sequence in this course.	23	66	12	34	0	0	0	0	0	0	0	0
Topics are well organized in this course.	24	69	11	31	0	0	0	0	0	0	0	0
Attending course lectures is beneficial.	16	46	14	40	3	9	1	3	1	3	0	0
There is a clear connection between what I do online and what is covered in class.	21	60	13	37	0	0	1	3	0	0	0	0
Online course materials are always available in a timely manner.	28	80	7	20	0	0	0	0	0	0	0	0
Online course materials are easy to follow.	25	71	10	29	0	0	0	0	0	0	0	0
Online course materials are useful in learning course content.	22	63	10	29	2	6	1	3	0	0	0	0
I possess the time management skills needed to be successful in this type of course format.	18	51	12	34	4	11	0	0	1	3	0	0
The online component of the course enables me to have more interactions with the instructor.	7	20	7	20	8	23	11	31	1	3	1	3
The online component of the course has improved the quality of my interactions with the instructor.	7	20	9	26	12	34	6	17	1	3	0	0

^aItems 2 and 3 have the same prompt to the student.

The results of survey item #2, which was concerned with content delivery and effectiveness of instruction, showed similar levels of student satisfaction. All respondents either strongly agreed or agreed that topics were presented in a logical sequence and were well organized. A large percentage (66%) strongly agreed that topics were presented in a logical sequence in this course. Furthermore, 69% strongly agreed that the course was well organized (Table 2).

Survey item #3 focused on various qualities of effective online instruction and desirable student interactions with the instructor. The results from this item indicated that all respondents agreed or strongly agreed that the course materials were available in a timely manner and were easy to follow. In contrast, 31% of respondents disagreed that the ICM format increased the number of their interactions with the instructor, while 23% neither agreed nor disagreed with this statement. A similar pattern in responses also emerged in the statement focusing on the quality of students' interactions with the instructor. Another statement within this survey item focused on whether the student possessed the time management skills needed to be successful in an ICM course. A majority of respondents (86%) agreed or strongly agreed that they did possess these skills. Only 3% of the respondents strongly disagreed that they possessed the time management skills needed to be successful in this type of course format (Table 2).

Survey items #4 and #5 asked open ended questions. Item #4 asked the student what they liked most about the ICM. The vast majority of the responses centered around the flexibility of learning, being able to learn when and where the student preferred was desirable. Others commented on the organization and the 1-hour of F2F time, “the way the class meetings were structured was great, I definitely learned a lot with the case study examples.” Survey item #5 queried what the student would change about the format. Although many students offered a reply of “nothing, it's great,” a few constructive answers were given. Some commented that they would prefer a balance between the F2F and online components, rather than skewed towards online. Others offered comments about the need for more review questions or reorganization of review questions. The final survey item, #6 concerned the preference of students regarding how all basic sciences courses should be delivered (Fig. 2).

Figure 2.

Students were asked in Item 6: If you were given a choice regarding how basic science courses were taught, which format would you most prefer? (please select one): Extensive use of the online environment, but still some F2F class time. An equal mix of F2F and online content. Minimal use of the online environment, most course content presented in F2F class time. Entirely F2F. No respondent indicated that they favored an entirely online course with no F2F time.

DISCUSSION

The student performance hypothesis tested in this study was purposefully designed to determine if students perform the same using the two teaching methods, the desirable result, rather than to see if students perform differently. The results of the independent-samples t-test indicated that with respect to student performance, the effectiveness of both teaching methodologies was the same, allowing a measure of confidence when considering permanently transitioning to the ICM teaching method for clinical microbiology. This does not, however, indicate that students perform better in the ICM classroom than in the traditional F2F classroom, but only equivalence or noninferiority should be required to support transitioning the course. The neutrality of student performance in ICM vs. traditional courses has been demonstrated in other disciplines outside of chiropractic, including pharmacy education, medical education, and nursing education.^14–16

In retrospect, the statement “I possess the time management skills needed to be successful in this type of course format” could have been presented either as an individual item and/or presented earlier in the survey, because of its contextual importance. Since time management skills is an indicator of student “readiness” for success in this type of course format, students' responses to this statement may have strongly influenced their responses to subsequent survey items.

The responses to the survey indicated that student perception and satisfaction with some aspects of learning are improved when using an inverted classroom model. This study further demonstrated that these students prefer the flexibility of the ICM classroom. This result has been seen in other studies; even when student performance changes are neutral, student satisfaction is improved.^16,17 This study highlighted the need for the online materials to be designed in a student-friendly, accessible manner. Almost 70% of students strongly agreed that the online clinical microbiology materials were presented in a logical, well-organized manner. This stands out as one of the highest percentages of strong agreement in the entire survey. Additionally, students overwhelming strongly agreed that course materials online were easy to follow and available in a timely manner. This demonstrates the need for the ICM course to be user-friendly, and the importance of careful course design.

The clinical microbiology course is concerned with teaching about many different human pathogens. This course was designed around those pathogens, allowing the student to dedicate 20 to 30 minutes to viewing a video and reading the notes for one pathogen. The ability to cover information for one topic completely in a relatively short period may have appealed to the student. This may suggest that course content that can be effectively “chunked” into digestible pieces for the student may lend itself to the ICM format.

The final question in the survey that asked generally about how students might prefer the basic sciences courses to be delivered at UWS was thought-provoking. Only 2.8% preferred the entirely F2F lecture style, while 43% preferred extensive use of online materials. No student wanted any of the courses to be entirely online. This showed that students do, in fact, value interaction with the instructor. However, they desired the freedom afforded by the ICM format. Perhaps the ideal approach is a more equal mix of online and F2F. This clinical microbiology course represents an extreme with 4 hours delivered online and only 1 hour in the F2F classroom.

One limitation of this study was the low survey response rate, 37% for the secondary outcomes. This may be partially explained by the fact that the survey was not linked to any aspect of the grade in the course. This was done intentionally to attempt to garner honest survey responses. Additionally, the ICM format has been used only very recently at UWS and, as such, the data in this study concern only one cohort of students having been taught in the inverted classroom. If the ICM format continues to be used it will be interesting to observe the changes in student performance and perception of this methodology.

CONCLUSION

Clinical microbiology taught as an inverted course resulted in similar student performance when compared to the traditional lecture-only style of delivery. However, the students that responded to the survey preferred to self-teach materials in the online environment and then return to the classroom for 1 hour a week to apply knowledge gained online. An advantage of the ICM format is the flexibility of learning that is afforded to the student.