Comparison of Online Versus Classroom Delivery of an Immunization Elective Course

Abstract

Objective. To compare performance and preferences of students who were randomly allocated to classroom or online sections of an elective course on immunization.

Methods. Students were randomly assigned to either the classroom or online section. All course activities (lectures, quizzes, case discussions, vaccine administration, and final examination) were the same for both sections, except for the delivery of lecture material.

Assessment. Students were surveyed on their preferences at the beginning and end of the semester. At the end of the semester, the majority of students in the classroom group preferred classroom or blended delivery while the majority of students in the online group preferred blended or online delivery (p<0.01). Student performance was compared at the end of the semester. There was no significant difference for any of the grades in the course between the 2 sections.

Conclusion. There was no difference in student performance between the classroom and online sections, suggesting that online delivery is an effective way to teach students about immunization.

INTRODUCTION

Online courses have been offered in higher education in general and pharmacy education specifically for many years. As of 2011, 31% of all higher education students took at least 1 course online, and this percentage has increased steadily since 2003.¹ An online course has been defined as a course in which 80% or more of the content is delivered online.

Historically, the classroom has been the standard venue for delivering information to students.^2,3 There are both advantages and disadvantages to online courses when compared to the classroom setting. Online courses allow flexibility in both when and where students learn material,^2,4,5 in the pace at which they learn course material,² and in the amount of times they can review course material. Offering courses in an online format allows universities to increase course enrollment and to create new courses and degrees.^2,4 Some institutions use the online format extensively to reach traditional and nontraditional students for undergraduate and graduate programs.⁶ Conversely, online courses can lead to social isolation for both students and instructors.^2,4,5 Also, students enrolled in online courses are required to take ownership of their learning and assume a more active role in learning course material.^2,7 Some students have more anxiety with online courses⁸ and many students spend more time on online courses than on face-to-face courses.² There is a start-up cost associated with online courses and there can be technical difficulties, which can cause frustration for students and additional work for universities.⁵

Classroom delivery has been compared to online delivery. A United States (US) Department of Education meta-analysis evaluated the effectiveness of face-to-face compared to online learning in all age groups.⁹ Most of the studies included higher education students, with the most common areas being medicine or health care. Students taking an online course performed moderately better than those learning in a classroom; however, the online courses were often different than the classroom courses in some way, including the time spent on course material.

In the health professions literature, classroom delivery has been directly compared to online delivery with pharmacy, nursing, paramedic, radiologic technology, and dental hygiene students, and with a continuing medical education course.^2,7,10-15 In the pharmacy literature, 1 study compared student performance in a medicinal chemistry course in a Web-based doctor of pharmacy (PharmD) program with a traditional, on-campus PharmD program.¹⁴ Students performed similarly on examinations and in-class assessments; however, the on-campus students performed significantly better on preclass assessment quizzes, case presentations, and final grades. Another study showed that students randomly assigned to either a distance-learning or traditional classroom delivery method for a therapeutics course module performed similarly.¹⁵ There is a lack of evidence in the pharmacy education literature about a direct comparison of online to classroom delivery of course material, especially with the same cohort of students for an entire course.

The Accreditation Council for Pharmacy Education (ACPE) supports colleges and schools of pharmacy in implementing innovative teaching delivery methods for courses and states that institutions must evaluate if the changes made were effective.¹⁶ Additionally, the ACPE standards mention that institutions must use teaching and learning methods that meet the “diverse learning needs of students” and enable “students to transition from dependent to active, self-directed, lifelong learners.”

The University of Wisconsin School of Pharmacy is a traditional 4-year PharmD program. The Comprehensive Immunization Delivery elective course had previously been offered in the spring semester and only third-year (P3) students could take the course, with approximately 70% of students (90 to 100) enrolling. This course allowed students to provide immunizations under the supervision of a pharmacist while they were pharmacy interns and independently once they became a licensed pharmacist. Pharmacy students in Wisconsin could work as interns after completing their second year of the PharmD program. Interns could participate in pharmacy-based immunization programs if they had completed immunization training that fulfilled the requirements in Wisconsin Act 68, which the Comprehensive Immunization Delivery course did.

Several curricular changes were implemented for required PharmD degree courses with the class that began in fall 2011. These changes resulted in second-year (P2) students being able to take an elective course during the spring semester, which they had not been able to do in previous years.

During the spring semester of 2013, both P2 and P3 students could take the immunizations elective course for the first time. The high interest in this elective course created a challenge for the course instructor, as 2 classes of students were able to take the immunizations course in the same semester and the anticipated enrollment of students far exceeded the capacity of the lecture halls. Because all students could not take the course in the classroom setting, an online section option was explored.

At the time of this study, the University of Wisconsin School of Pharmacy had not routinely used online courses. The school had used blended learning for several required courses. Additionally, there had been a campus-wide initiative for educational innovation with the university that supported creative ways of delivering courses and a desire for the development of online courses over the previous year. The primary objective for this study was to determine how online course delivery compared to classroom course delivery in student performance, opinions, and preferences when students were randomly assigned to a section. Additionally, faculty time required to teach an online course was measured.

DESIGN

The Comprehensive Immunization Delivery course met for 50 minutes once a week. The classroom section of the 1-credit course was conducted exactly as it had been in previous years. Although, the course lectures were traditionally offered in the classroom and were not recorded in the lecture capture system, there was 1 week that the class did not meet and the lecture was available online. All students were required to attend a hands-on vaccine administration laboratory session as well as to complete and respond to a written vaccine administration question assignment. There were 2 quizzes in the course (1 in class and 1 online) and a final comprehensive examination. The only additional assignments compared to previous years were completion of 2 survey instruments, 1 at the beginning of the semester and 1 at the end of the semester. All students in the course received points toward their course grade for completing both survey instruments, which assessed their opinions and preferences. No additional coursework was assigned to students who participated in the study.

Besides the location of the lectures, all other course activities were identical for both sections. The classroom lectures were recorded for the online section and were only available to those students registered for that section. The recorded lectures showed the instructor’s presentation slides and included the audio of the presentation. Students in the online section could watch the lecture at any time and as many times as they wanted. All students already were familiar with recorded lectures and the lecture capture system as other courses in the PharmD program recorded all live lectures for student reference. Students in the classroom section had access to the course instructor during class. Both sections could access the course instructor through e-mail and the instructor was available for appointments to discuss any course questions or concerns.

The study included both P2 and P3 students. Students who preregistered for the course and were interested in participating in the study provided written informed consent. Recruitment e-mails were sent to students who did not complete consent forms but had preregistered for the course. Consent forms were accepted until the start of the spring 2013 semester. Students were offered $10 on their university card to cover printing costs as an incentive for participating in the research study. The lecturing faculty member was blinded to the participation status of the students.

All students who enrolled in the course through the preregistration process were randomly assigned to either the classroom or online section using block randomization. Students were informed by e-mail of their randomization and given the registration number for that course. Exceptions to the randomization (eg, scheduling conflicts, personal issues) were not allowed.

Faculty time spent on the online course development was minimal as the in-class lectures were recorded and then posted online for the online section. Information and instructional technology support was instrumental in making sure the online section had access to the recorded lectures in a timely fashion. No new technology was purchased for implementation of the online section. As a result of increased enrollment in the course, there was an increase in faculty time required for grading assignments and conducting the hands-on laboratory sessions. Also, because of the overall increase in course enrollment, additional proctors were required to administer the in-class quiz and final examination.

EVALUATION AND ASSESSMENT

One hundred forty students participated in the study for a participation rate of 83.3% (Figure 1). The survey instrument administered at the beginning of the spring semester included questions about demographic information, technology, and course delivery. Questions were formatted as multiple choice or statements that used a 5-point scale (strongly agree, agree, neutral, disagree, strongly disagree). The survey instrument included open-ended questions regarding perceived advantages and disadvantages of each delivery method. The survey instrument at the end of the spring semester included questions about demographic information and course delivery. Many of the questions were the same in both format and content as those on the first survey instrument; however, some new concepts related to course delivery were explored using the question formats described previously. Both survey instruments had 28 questions. Content ideas were based on previously published studies involving online courses or components of courses.^17-19 Only survey data from students who consented to participate in the study were included in the analysis.

Figure 1.

Consolidated Standards of Reporting Trials (CONSORT) diagram. (Abbreviations: P2s=Second-year pharmacy students; P3s=Third-year pharmacy students.)

At the beginning of the spring 2013 semester, baseline demographic data (Table 1) were obtained from the University of Wisconsin Registrar’s Office about study participants. Demographic and survey data were combined in Excel and analyzed with Excel 2010 and SPSS, version 20 (IBM Corp, Armonk, NY). Descriptive statistics were used to categorize preferences, demographic information for students, and student responses to open-ended questions. Continuous data, such as quiz scores and grades between sections, were compared using t tests. Letter grades earned at the end of the semester were converted to a 4-point scale. Pearson chi-square tests were used to compare nominal-level variable results to each other. Strongly agree and agree responses as well as strongly disagree and disagree responses were collapsed for analysis. For all statistical analyses, a probability less than 0.05 was considered significant. This study was reviewed and approved by the University of Wisconsin Education Research Institutional Review Board.

Table 1.

Demographics of Pharmacy Students Enrolled in the Online or Classroom Section of an Immunization Elective Course

Most students had access to the Internet while off campus and most students owned a laptop computer (97% classroom vs 96% online). Only 41.4% of students had previously taken an online course. Approximately half of the students did not consider themselves technologically savvy, with 46% of students in the classroom group and 55% of students in the online group responding with neutral, disagree, or strongly disagree.

Student preferences for course delivery method were assessed through questions in the student survey instrument. Preferences for course delivery strategies did not differ between sections at the beginning of the semester (Table 2). The majority of students in both groups preferred taking the course in the classroom or a blended setting. At the end of the semester, the majority of students in the classroom group preferred classroom or blended delivery while the majority of students in the online group preferred blended or online delivery (p<0.01). Sixty-four students’ course delivery preference changed from the beginning to the end of the course. Five students changed their preference to classroom, 27 switched their preference to online, and 31 preferred blended learning. There was no significant difference when the changes were broken down by delivery system experienced (p=0.09). Overall, students felt that they would learn more with a blended delivery method, with 51% of students selecting blended.

Table 2.

Preferences of Pharmacy Students Enrolled in the Classroom or Online Sections of an Immunization Elective Course

In response to open-ended questions, students in the classroom group felt that advantages to that delivery method were the ability to ask questions in real time (45%); the schedule, which helped students stay on track with the material (34%); and having face-to-face contact with the professor (14%). Disadvantages to this delivery method were not having access to lecture capture for a variety of reasons (63%) and inflexibility with the course related to student schedules (26%). Students in the online group felt that being able to have a flexible schedule to complete the course at their own pace (88%) and being able to pause or watch a lecture again (45%) were advantages to this delivery method. Disadvantages included getting behind in the course (39%) and not being able to ask questions right away (28%).

Regardless of the delivery method to which students were randomized, they felt that method was effective and helped them learn (Table 3). Students who experienced the classroom delivery of the course felt that method did not allow for flexibility with their schedules while students in the online group felt that method did (p<0.001 for both groups). In both groups, students preferred to complete coursework at their own pace. A relatively large, but not significant increase (p=0.1) in proportion of classroom students indicated that they preferred working at their own pace at the end of the semester compared to the beginning. More students in the classroom group felt that they had an opportunity to ask questions of the instructors throughout the semester compared to the online group (p<0.001). No difference was found between groups in the anticipated amount of time they would spend or the actual amount of time they spent on the class. When the groups were combined and presurvey results were compared to postsurvey results, students thought that they would spend more time than they actually did each week. (2.4±0.86 vs1.6±0.66; p<0.001)

Table 3.

Comparison of Pharmacy Students’ Opinions on Delivery Methods for an Immunization Elective Course^a

Most students (68%) stated they would take an online course again. Eighty-six percent of students agreed with the statement that online courses with some face-to-face interaction should be offered for some courses. Additionally, when asked to respond to the statement that all courses except for laboratories should be delivered online, most students (73%) disagreed.

Student grades were compared between sections at the end of the spring semester. Final course grades were assigned their associated GPA (eg, A=4, AB=3.5, B=3) for analysis. There was no significant difference in course grades between the 2 groups, including grades on the in-class quiz, online quiz, written vaccine information question, final examination, and the final course grade (Table 4). Student attendance was not taken for the classroom course; however, an attendance count was completed at the beginning of the semester (94% of students attended) and at the end of the semester (93% of students attended the class).

Table 4.

Pharmacy Students’ Course Grades for Classroom and Online Sections of an Immunization Elective Course

For the online course, information about how often students viewed the online lectures was recorded. Students accessed an online lecture an average of 1.8 times. Students watched the lectures at the beginning of the semester more often than at the end of the semester, with the average number of times a student accessed a lecture ranging from 0.95 to 2.99.

Eleven students in the classroom group stated they watched 2 or more lectures online. All students in the classroom group did have 1 lecture that was online, and it is unclear whether students considered that lecture when answering the survey question. Only 4 students in the online group reported that they attended a classroom lecture.

DISCUSSION

Students who were randomly assigned to the online or classroom section performed similarly and were generally accepting of the formats. A blended course was defined as a somewhat equal mix of online and in-person lectures and coursework. Blended or hybrid courses have also been defined as 30% to 79% of online coursework, with the remaining percentage face-to-face.¹ All students had taken an introductory pharmacy practice experience course in the first year of the PharmD program that was a blended course, with approximately 70% of the content taught outside of class or online. Students also experienced blended learning in their pharmacotherapy skills laboratories in their second and third years. However, only approximately 50% of students reported they had previously taken a blended course and only approximately 20% of students stated they had taken a course that was mostly online with some face-to-face interaction with faculty members. Additionally, some students believed a blended course would include a classroom lecture that had been recorded and posted online for listening to or watching again and mentioned that being able to do so was a benefit of this type of delivery method, even though these characteristics are not consistent with the definition of blended learning. Given the varying responses regarding blended learning strategies, the responses of the 51% of students who stated they would learn more with blended delivery may have little meaning. From their comments, students liked flexibility with coursework and the ability to access course material as many times as they wanted and on their own schedule. They appreciated the interaction with course faculty members, especially to ask questions in real time. Lastly, while they desired flexibility, they liked some aspect of the course to keep them on task to avoid getting behind with course material. Blended courses would meet many of these student preferences. Because it was not part of this study, we do not know if students would have performed better with a blended course as compared with an online course.

The delivery method that students experienced had an impact on their preferred delivery method. As the students had not taken the final examination or been assigned their final grade when they completed the second survey instrument, students’ experience in the course had the biggest impact on their preferences at the end of the semester. Three times as many students in the online section preferred an online delivery method at the end of the semester compared to the beginning, whereas fewer students in the classroom section preferred taking a classroom course. As students were most familiar with the classroom delivery method, they may not have realized they preferred another delivery method and continued taking classroom-based courses because they knew what to expect and could anticipate how they would perform. Exposing students to different delivery methods could impact their future learning, both in pharmacy programs and beyond, with more students taking an online or blended course.

There was no difference in student performance between the 2 groups. While the US Department of Education meta-analysis showed better performance with the online group, the online courses included differed in several ways from the face-to-face courses.⁹ In our study, the online and classroom sections were identical, except for how the lecture was delivered to students. When course content was kept the same, several studies have shown no difference in student performance,^{2,10,12,13,15,20,21} while 1 study showed that students in the online group performed better.¹¹ Specific to our study, students were randomly allocated to their sections for the entire course. One could assume that students would choose learning strategies with which they were most comfortable in order to ensure their success. Because of the limited classroom seating, students were randomly allocated to either the classroom or online section. This study design adds to the body of knowledge.

For study design purposes, the content presented to the online section of students was the exact same as that presented to the classroom section of students. Except for the online section, the lectures were recorded and posted for students to use. While this worked well for the study, it is not the ideal way to set up a truly online course to engage students in learning. Blended learning and online learning that is collaborative or instructor-directed may be more effective than a standard online course.⁹ Online courses can be enhanced in several ways. Courses should be designed with learning modules for material to assist students in completing the course on their own and to help them stay on task.^22,23 Interactive activities to engage learners, including multimedia technology such as video and animations, case studies, and quizzes, also can be helpful.²⁴ Purposeful student-student and student-instructor interaction should be built into the course to further encourage student engagement.^10,22,25

Course time spent by faculty members was only increased because of the increase in total enrollment. Had the online class included specific learning modules, additional interactive activities, and student-student and student-faculty interaction, this would have taken much more faculty and instructional technology support time to develop and implement. There would also have been an additional cost to the university to support the course. Faculty time, instructional technology support, and cost all need to be considered when developing online courses.

It was not possible to prevent students in the online section from attending a classroom lecture or classroom students from watching an online lecture from a classmate in the online section. This may have increased the number of times a lecture was accessed unless the 2 students watched the online lecture at the same time. Although course enrollment is unlikely to exceed classroom capacity in future years, the delivery method used did not have an impact on student performance, and consideration should be given to students’ preference for flexibility and blended learning.

SUMMARY

Online delivery is an effective way to teach students; in this study, online student performance was similar to that of students who learned the material in the classroom. There was no difference in student performance when students were randomly allocated to the classroom or online section of an elective course on immunization. Students were not opposed to taking a course that was delivered online. Some students changed their preferred course delivery method to online or blended after exposure to their assigned delivery method. Course instructors should critically evaluate whether their course or parts of their course could be delivered online, taking into consideration cost, the technology and instructional technology support available, faculty member setup time, and their plan to engage students in course material.