Time Spent, Workload, and Student and Faculty Perceptions in a Blended Learning Environment

Abstract

Objective. To evaluate student perception and time spent on asynchronous online lectures in a blended learning environment (BLE) and to assess faculty workload and perception.

Methods. Students (n=427) time spent viewing online lectures was measured in three courses. Students and faculty members completed a survey to assess perceptions of a BLE. Faculty members recorded time spent creating BLEs.

Results. Total time spent in the BLE was less than the allocated time for two of the three courses by 3-15%. Students preferred online lectures for their flexibility, students’ ability to apply information learned, and congruence with their learning styles. Faculty members reported the BLE facilitated higher levels of learning during class sessions but noted an increase in workload.

Conclusion. A BLE increased faculty workload but was well received by students. Time spent viewing online lectures was less than what was allocated in two of the three courses.

BACKGROUND

Blended learning is the integration of a classroom face-to-face learning experience with an online learning experience.¹ A blended learning environment (BLE) uses any combination of asynchronous or real-time online instruction and traditional onsite learning.² It allows faculty members to meet learning objectives through learning activities in a more flexible and convenient method than traditional classroom teaching.

Published studies highlight the advantages of blended learning, including the ability to facilitate an independent and collaborative learning experience for students. Studies evaluating outcome data in the pharmacy education literature suggest that this method is an effective and well-received tool for teaching a variety of concepts including cardiovascular and neurologic pharmacotherapy, drug information and literature evaluation, pharmaceutics, and internal medicine.^3-8 Students have an overall high level of satisfaction with a BLE approach and prefer a smaller-class format over a large lecture-based format.^8,9 Smaller-group format increases student accountability for learning outside the classroom, and students state that they preferred reading at their own pace.^9,10 A pharmacokinetics course using BLE demonstrated that initial student concerns were primarily related to communication with the instructor. However, this switched to ability to manage time by the end of the course.¹¹ Data published in other disciplines, including dental and medical education, suggest that BLE may have the ability to enhance the education of a wide variety of health professions students.^12-14

While published data evaluates outcomes and student perceptions,^3-16 considerably less data is available on workload requirements for students and faculty members when implementing this learning strategy. In particular, motivation of students to comply with the time required to invest outside of class for this model to work is unknown. Some faculty members also debate the amount of contact hours to allocate for the online portion of a BLE. A contact hour is “a unit of measure that represents an hour of scheduled instruction given to students.”¹⁷ When determining credit hours per course at our institution, the following conversion is typically used: one credit hour equals one contact hour per week of live lecture, two contact hours per week of workshop or three contact hours per week of independent study. However, this allocation differs among schools, courses, and, even within our own college, there is debate regarding whether a live lecture should be equivalent to an online lecture when calculating contact hours or whether it should count as “independent study.” Additionally, online instructor-guided time (eg, discussion boards, group chats) can also count towards “contact time.” Yet, there is no published consensus on rate of credit equivalency. Having an understanding of the amount of time students spend on the asynchronous activities might be helpful when determining contact hour allocation.

To address some of these questions, we designed a study to measure student and faculty workload when implementing a BLE for five lectures in three different courses at one college of pharmacy. The primary aim of this study was to compare actual student time spent in the BLE to the time allocated for the content by the course director. Secondary aims included comparing actual time to the time reported by students, analyzing the impact of a BLE on faculty workload, and analyzing student and faculty perceptions and attitudes towards the BLE.

METHODS

This study took place from November 2013 to May 2014 at a large private university in the Midwestern United States with an average class size of about 200.The institutional review board at the university approved the study. At this institution, students complete a four professional year pharmacy program and graduate with a doctorate in pharmacy. The most common teaching method used in the college is traditional didactic lectures, although active learning, in the form of turning-point lectures, in-class group exercises, and small-group workshops, is incorporated into most courses.

A BLE was created by combining asynchronous online lectures that students completed on their own time with in-class, active-learning activities centered on the same topic. This approach was used in three different courses at the institution: a small ambulatory care elective course offered to students in the third year of the program; a required cardiovascular pharmacotherapeutics course in the second year of the program; and a required first-year course on evidence-based medicine. In each course, lecture topics covered in previous years as traditional live lectures were converted into interactive, online lectures using Articulate Storyline (Articulate Global, Inc, New York, NY). This software was chosen because it had features that appealed to the investigators, including built in graphics and avatars and the ability to create highly interactive presentations with audio and video. Interactive tools allowed for the creation of in-module assessments and/or case-based scenarios with the ability to customize feedback to the learner based on the choices selected. It was also compatible with Blackboard (Blackboard, Inc., Washington, DC), the university’s learning management system, and it had previously been used successfully at the institution.^18,19 Articulate Online (Articulate Global, Inc., New York, NY) was also used in this study. This online subscription service allows authors of Articulate Storyline presentations to track the amount of time viewers spend on the online module.

In all three courses, students were required to view one or more online lectures prior to coming to class. Online lectures were created using Articulate Storyline 1.0. Online lectures included voice narration by the lecturer. They were self-paced and consisted of interactive material in the form of “click to learn,” self-assessment quizzes, and embedded links to additional reading materials. Students were able to rewatch the lectures as many times as they wanted. Students were given approximately one week to view the material on their own time. After the deadline passed for viewing the online lecture, classroom time was allocated for interactive activities centered on the same material presented in the online lecture.

The format used in the classroom portion of the BLE varied by instructor. Some chose to have the students work through case-based scenarios (elective course), while others chose to use classroom time to review the in-module assessments, answer questions students had about the material, and/or work through additional practice problems or cases (EBM and therapeutics courses). In all cases, the classroom activities were not associated with any points. Table 1 lists the courses and lectures involved in this study as well as the number of contact hours allocated for content. The in-class portion of the elective course incorporated cases that involved all three disease states included in the three online modules. For all three courses, the course director allocated slightly more time for the BLE compared to previous years when the content was delivered by traditional live lecture (Table 1). Classes start at 10 minutes past the hour at our institution; therefore, one contact hour equals 50 minutes of class time, 1.5 hours equals 80 minutes of class time, and two contact hours equals 110 minutes of class time.

Table 1.

Courses and Lectures Involved in the Blended-learning Environment

Rather than spending time training faculty members involved in the study, it was thought that a more efficient model would be to allocate a subject matter expert and a technology design expert. Faculty members then collaborated to develop the online learning modules. Individuals with little to no experience with the Articulate software (subject matter experts) developed a traditional PowerPoint lecture, which was sent to the technology design expert for conversion into an online module. Six faculty members participated in the study. Four faculty members served as subject matter experts and were responsible for updating content from previous years (none of these topics were new lectures). One faculty member served as a technology design expert and was responsible for converting the lectures into online lectures using the Articulate Storyline software, and one faculty member served as both a subject matter expert and a technology design expert. The two technology design experts had previously attended training sessions on how to use the software and had previous experience using the software in other courses at the college.

Articulate Online was used to capture the amount of time students in all three courses spent completing the online lectures. Time reported was cumulative, and students were allowed to go back and view the online lecture an unlimited number of times. Total time spent in the module was used in the analysis. Students were also asked to record the amount of time they spent viewing and completing the online lectures on a form that was provided during the classroom portion of the BLE. To protect subjects’ identity, the forms used to collect self-reported data were anonymous and information that could identify the student (eg IP address, student names, email address) was removed from the Articulate Online reports prior to analysis. For the larger, required courses, participating students were awarded extra credit points. In the EBM class, five extra credit points were awarded (representing less than 1% of the total points available for the course). In the therapeutics course, four extra credit points were awarded (representing less than 1% of the total course points). No extra credit was awarded for participation in the study in the elective course because the class size was small, and the students knew when registering for the course that it was going to involve a BLE.

To assess the impact on faculty workload, all faculty members involved in the study were asked to maintain a log of time spent developing the online lecture, including the time needed to develop new content, update existing content, collaborate with the technology design expert, review the online lecture, and review the online assessments. Faculty members were also asked to denote how much time they spent (on average) in previous years to develop a traditional PowerPoint lecture on the same topic.

A survey, which had been piloted and used in previous research at this institution, was used to assess student and faculty perceptions of the BLE.¹⁸ For students, the survey consisted of two parts. The first series of questions asked them to compare traditional live lectures in the course to the online lectures with regard to their ability to apply the information learned, enhance their understanding of concepts/principles related to the topic, stimulate interest in the topic, sustain interest in the topic, fit into their schedule, and provide feedback while learning. The second set of questions were open-ended and asked respondents to indicate the most and least helpful parts of the online lecture, as well as how online lectures should be used in the future. Faculty members were asked to provide feedback on the advantages and barriers they experienced when using the BLE in their course(s). Answers to each of the open-ended survey questions were reviewed by two of the investigators. Themes were identified independently and then the investigators met to resolve discrepancies and come to a consensus.

To assess long-term knowledge, mean scores on the final examination questions that pertained to the material covered in the BLE for the two required courses were calculated. These scores were compared to the subsequent year when the BLE format was not used. Descriptive and inferential statistics were used to summarize data. The t test was used to compare student reported time to Articulate-measured time. GraphPad 2015 (GraphPad Software, Inc., La Jolla, CA) was used for all statistical calculations. A thematic analysis was conducted by two of the investigators for all open-ended questions in the faculty and student surveys.

RESULTS

Most of the students enrolled in the courses participated in the study. In the elective course, 27/30 (90%) of the students participated. In the therapeutics course, 199/204 (98%) of the students participated, and in the EBM course, 201/209 (96%) participated. The demographics of the students who participated in the postlecture survey are included in Table 2. Most students were female (range: 63-76%), with the majority in their mid-twenties (range: 24-26), with a prior degree prior to starting pharmacy school (range: 52-87%). Women appeared to be slightly overrepresented in our sample; recent vital statistics note that 61.2% of all US pharmacy students are female, and in this study, the percentage of women ranged from 63-76%.²⁰

Table 2.

Student Demographics Across the Three Courses

The time required by students to complete the asynchronous online lectures was less than the time allocated for the lectures for three of the five lectures (p<0.05, Table 3), The magnitude of the difference ranged from 12-22 minutes under the allocated time, which means that students spent 24-28% less than the allocated time for these lectures. For two of the lectures in the elective course, the students spent more time (range 3-5 minutes, representing 4-6% of allocated contact hours), but this difference was not significant (Table 3).

Table 3.

Student Reported Time and Articulate Measured Time to Complete Online Lectures

Student-reported time to complete the online lectures differed from the time measured by Articulate Online for all of the online lectures in all of the courses (Table 3). In all instances, students estimated that it took them longer than it actually did to complete the online lecture. Students overestimated the time to completion by an average of 24.8 minutes per hour lecture (range: 5-34 minutes, Table 3).

Figure 1 shows the distribution of the percentage of students spending between 0%-25%, 26%-50%, 51%-75%, 76%-100%, and >100% of the allocated contact hour for each online lecture topic. For the EBM and the PAD lecture, the amount of time spent on the lecture was highly variable. For the three lectures in the elective course (HLD, HF, and DM), almost all of the students spent more than 50% of the allocated contact hour reviewing the material and for some lectures (HF and DM), many students (>50%) spent much more than the allocated time.

Figure 1.

Percentages of Allocated Time Students Spent on Online Lectures (EBM=evidence-based medicine; PAD=peripheral artery disease; HLD=hyperlipidemia; HF=heart failure; DM=diabetes mellitus).

Attendance was mandatory during the in-class, active-learning portion of the BLE. Table 4 depicts total time spent by students on both the online modules and the in-class portion of the BLE, as well as how this differed from the allocated time and the percentage of allocated time. For all three courses, students spent between 85%-97% of the allocated time completing the activities.

Table 4.

Total Time Spent in a Blended-learning Environment

The four subject matter experts who created the lectures in PowerPoint format reported spending 15 hours updating the lecture content from previous years. This did not differ from previous years. The two design experts, who converted the lectures from PowerPoint format to the online format, reported spending 68 hours converting the Power Point presentations into online modules. The first design expert (who was also a content expert) spent six hours to convert a 1-contact hour lecture to online format. The second design expert spent a total of 60 hours to convert four lectures (seven contact hours total) to online format.

In the postactivity survey, students were asked to identify which lecture format (online lecture or traditional live lecture) was more effective at applying the information learned, enhancing their understanding of concepts/principles related to the topic, stimulating interest in the topic, sustaining interest in the topic, fitting into their schedule, and obtaining feedback while learning. Students were asked to use their experiences with traditional live lectures in the curriculum as a basis for their comparison. The percentage of students who selected an online lecture as their preferred method of learning for each of these principles is depicted in Figure 2.

Figure 2.

Percentage of Students Preferring an Online lecture to Live Lecture (EBM=evidence-based medicine).

Results were consistent across all three courses with regard to students’ preference for an online lecture over a live lecture in the following areas: applying information learned (51-55%), conducive to learning style (44-52%), and ability to fit it into their schedule (60-77%). Student preference was more variable across courses when it came to the other issues assessed including: ability to enhance understanding (range: 36 - 52%), stimulating (range: 38 -56%) and sustaining interest (range: 39 - 52%), and ability to obtain feedback (range: 12 - 47%). A thematic analysis was performed on all open-ended questions by two of the investigators. With regard to what students liked best about the BLE, the following concepts were identified as the most frequently mentioned by students: convenience (n=16), interactive nature (n=3), self-directed learning (n=2), the reinforcement of ideas (n=1) and feedback obtained (n=1).

Regarding suggestions for improvement, the following concepts were identified as the most frequently mentioned by students: difficulty of subject matter (n=10), design of the online lecture (n=7), quality of sound/video (n=3), amount of time needed to complete (n=2), and access issues (n=1). Most students felt that online lectures should be used in other classes.

Faculty perceptions of the BLE, as identified by the six participating faculty members, are depicted in Table 5. In general, most faculty members (n=5) felt that one of the major advantages to having students complete the online modules prior to class is the ability to use classroom time to focus on higher levels of learning (eg, application). Two also noted that use of online lectures might be more convenient and attractive to students, as it allowed them to work at their own pace. The ability to have customized feedback on cases as an advantage was also noted (n=2). The most frequently identified disadvantage identified by the faculty participants was the time and resources needed to build the online module (n=4) or consult with others when developing the online lecture (n=2). There was also concern of technology failures jeopardizing the success of the BLE (n=2).

Table 5.

Faculty Perceptions of a Blended-learning Environment

Use of the BLE did not seem to have an impact on long-term knowledge in this study. For the required first-year evidence-based medicine course, the average score on the final examination questions pertaining to the BLE material was 83% when the BLE was offered compared to 85% the subsequent year when the BLE was not offered. For the required second-year pharmacotherapeutics course, the average score on the final examination questions that pertained to the BLE material when the BLE was used was similar to the subsequent year when the BLE was not used (97%, vs 98%, respectively).

DISCUSSION

This study confirmed the advantages of a BLE highlighted in previous publications.^3-16 While there was no major improvement in student performance, a majority of students across all courses felt that online lectures fit better into their schedules and that BLE should be incorporated into other courses. This convenience factor may have been more appreciated by our student body since we do not record live lectures for future viewing. Students noted that being able to go back and relisten to concepts covered in the online lecture enhanced their ability to learn the material. It is unclear if students’ positive perceptions of the BLE were a result of their preference for this particular format or if their preference was related to the fact that it was new. This “novelty” effect involves a diminished return in achievement gains as students become more familiar with a new educational medium.²¹

This study also highlighted workload considerations and barriers that faculty members may face when developing an effective BLE experience. While most faculty members involved in this study recognized that offering online learning modules allowed for more time to be dedicated to higher levels of learning in the classroom, this was at the cost of increased faculty workload when creating the online lectures. This may be a result in part of the steep learning curve associated with navigating the technology used to create the online lectures. Once faculty members are accustomed to the software and the process, the amount of time needed may decrease.²²

The large role that technology plays in a BLE requires that the software/interface tool used to deliver the online portion must be reliable and user-friendly for both faculty members and students. While we felt that the technology met these requirements, in order for faculty members to create the online learning portion with ease, they must possess some IT literacy and/or have access to IT support throughout the building process. Our study incorporated a subject matter expert and an instructional design expert when developing the online learning modules. Theoretically, a faculty member can fulfill both roles if properly trained on the software used to build the online modules. In a meta-analysis, Cook et al noted substantial variability in the way web-based learning and BLEs are implemented in health care courses.¹⁶ Thus, instructors who choose to use web-based technologies, alone or as part of a BLE, should carefully consider all options and choose the method and media that best matches the needs of their audience as well as their course objectives. Schools who would like to integrate a multi-media BLE into their curriculum might want to consider budgeting for a technology person who could serve as the instructional design expert for all courses.²²

In this study, students who spent less time than was allocated for the online lecture spent 24%-28% less time. This is consistent with published literature, which suggests that computer-based instruction requires, on average, less time than conventional methods of delivery.²³ However, those who spent more time spent as much as 211% of the allocated time on the material. Because allocated time for these online lectures was based on the length of time it took faculty members to deliver this content as a live lecture during the previous year, it is unclear why there was so much variability in viewing time among the students. One potential explanation for why students spent less than the allocated time on the online lecture might be that students may have had some baseline knowledge of the topic being presented and therefore decided to “skim” through some of the slides. Although, in this study, the only content that this might apply to was the three lectures in the advanced ambulatory care elective course and, for two of those three lectures, students actually spent more time than what was allocated. Another explanation may be that students simply did not finish the module prior to the assigned deadline. For each course, students were assessed on the online material through a multiple-choice final examination, which demonstrated equal incentive among the courses for students to want to complete the online lecture in each course.

Students who spent more than the allocated time reviewing the material may have done so because they were having a hard time understanding the material. In such instances, use of online modules may be beneficial, as they allow the student to review the material as frequently as they like. However, it is unknown if all of the time spent was spent actively learning the material. Moreover, students may have had the online lecture on the screen but may have been distracted by something else.

When the total time spent on the BLE was calculated, it was determined that actual student time spent was slightly less than allocated contact hours by 3%-15%. However, this amount of time was still larger than the amount of time allocated in previous years when the material was taught via a traditional live lecture. For this reason, the course directors involved in this study have elected to allocate a conversion of one contact hour = one BLE hour for these courses. For the online portion of the BLE, actual time spent by the students was 72%-76% of the allocated time, and there was variability among students with regard to time spent. A more rigorous evaluation of factors that influence time spent viewing online lectures that are not part of a BLE is warranted before the question of contact hour conversion for online lectures can be answered. In the interim, schools considering contact hour allocation for courses that incorporate online learning should consider online tracking software to determine the amount of time students spend completing asynchronous online lectures. Use of self-reported time is not recommended given the large differences noted in this study between student reported time and actual time spent viewing the online lecture.

Students’ preference for an online lecture over a live lecture varied across the courses used in this study for certain elements including: ability to enhance understanding of the concepts, the ability of the lecture to stimulate the student and sustain interest, and the ability of the learner to obtain feedback. These differences may be related to the topics presented in each class in that some topics may have been more complicated than others. Alternatively, it may also be related to the style of the author for each of the lectures. With regard to obtaining feedback, the elective course did not have embedded questions in the lecture, whereas lectures in the therapeutics and evidence-based medicine course did. This may explain the variability in responses for this item.

With regard to logistical issues, faculty members involved in the study found learning the online lecture software time-consuming. When the subject matter expert was not also the instructional design expert, creating the BLE took significantly longer as material and ideas had to be sent back and forth between both parties. In addition, given the limited use of BLE in our curriculum at the time of this study, students had to be trained on how to access and use the online modules using individual passwords and participation was incentivized with extra credit points. Therefore, is it unknown how many participants would have used the online learning modules if extra credit was not offered in the required courses.

Limitations include the fact that this study was performed at one institution. Faculty and student perceptions, in particular, may be different at other institutions where the educational culture differs. Student perceptions may also differ depending on the level of complexity of the material. Recall of perceptions is subjective and may not always be accurate. Also, including only one elective course and two required courses is a limitation as there was variability in instructor format for in-class activities. In addition, an assumption was made that all logged minutes were “active minutes,” although there is no way to determine if this is actually the case. Just because the student was logged in does not necessarily mean that they were actively learning the material, and just because the student was in workshop does not necessarily mean that the student was actively engaged in the learning process. Another limitation is that the study only spanned eight contact hours. In addition, because the extra credit incentive was not standardized across all courses, it may have introduced a level of bias into student response, although responses were collected anonymously to try to prevent this. Finally, long-term assessment of the impact of a blended learning environment needs to be conducted to rule out the possibility of a novelty effect.²¹

CONCLUSION

The BLE is an instructional method that may allow students to learn at their own pace. Actual time spent by students in a BLE was similar to the time allocated by the course director in this study. Although using a BLE did not lead to improved learning outcomes in the three courses in which it was used at our institution, students appeared to have a generally favorable impression of the BLE, with the most frequent advantage noted being the ability to fit the online lectures into their schedule. Faculty members appreciated that this format allowed more time for interactive discussions and team-based learning in the classroom. However, faculty members also noted a significant learning curve and extensive time required to develop all the resources needed to make this a successful learning experience. To maximize faculty productivity, a full-time expert dedicated to technology and instructional design should be used to create a favorable experience for students and faculty members.