Best Practices for Use of Blended Learning

Abstract

Objective. To determine instructional best practice recommendations for use of blended learning from the students’ perspective.

Methods. Three focus groups were created, one for each of the first three years at a school of pharmacy. The focus group discussions were audio recorded and transcribed for content analysis.

Results. Ten instructional best practices were identified from the focus groups: setting the stage, consistency when team teaching, timeliness in posting materials, time on task, accountability for online activities, use of structured active learning, instructor use of feedback on student preparation, incorporation of student feedback into the course, short reviews of online material during class, and ensuring technologies are user friendly.

Conclusion. Instructors using blended learning should consider incorporating these best practices into their course design and management. More evaluation is needed to see if implementation of these practices affects student performance.

INTRODUCTION

Blended learning courses, or courses incorporating both online and face-to-face activities, often require more preparation time outside of the classroom for students than traditional face-to-face courses.^1-5 Blended learning is effective,^5-9 and students often see the benefit in using a blended learning approach.^1,2,5,7,9-12 However, when multiple courses become blended, the amount of out-of-class time needed for preparation may become excessive and overwhelming to manage for students.¹³

Within pharmacy education, there have been many positive evaluations of blended learning for both student learning outcomes and satisfaction.^1,2,4,5,9,10 However, these evaluations only focus on one course or one experience rather than an entire curriculum. At the University of Wisconsin-Madison School of Pharmacy, many instructors have incorporated the blended learning approach into their classroom in a variety of formats. In the spring of 2013, the first lecture-based course became blended, incorporating required online activities in addition to face-to-face sessions. The UW-Madison School of Pharmacy has a four-year Doctor of Pharmacy (PharmD) program with the fourth year of the program devoted to experiential education. During the 2014-2015 academic year, 14 required lecture-based courses spaced over the first three years of the curriculum incorporated blended learning, with the majority in the second and third years. The manner in which faculty incorporated blended learning was not standardized and included online lectures, simulated patient cases, and online demonstrations either before or after the corresponding face-to-face component. Following this large uptake of blended learning, the first established blended classroom saw a significant negative shift in the course evaluations regarding the use of blended learning and the time requirements for course preparation. Several faculty members also received informal feedback regarding the large number of online lectures and the range in quality of how class time was used. This prompted a school-wide quality improvement effort to determine what strategies instructors can use to improve the student experience with blended learning.

For this programmatic evaluation, a course which uses blended learning was defined by the use of integrated online and face-to-face activities. This did not include reading online handouts before class, or using lecture capture (re-watching a lecture video after it has occurred). The objective of this evaluation was to determine instructional best practice recommendations for use of blended learning from the students’ perspective.

METHODS

This cross-sectional qualitative evaluation utilized content analysis to evaluate a series of three focus groups of pharmacy students’ perceptions of blended learning. A qualitative approach was chosen to allow for a better understanding of what problems students were experiencing with blended learning and to allow for a richer discussion of what works well from their perspective. As this project was undertaken for programmatic evaluation, the UW-Madison Health Sciences Institutional Review Board (IRB) determined this project did not meet the federal definition of research so IRB review was not required.¹⁴

A focus group was conducted for each of the first three didactic years of a four-year PharmD program at the UW-Madison School of Pharmacy. Fourth-year (P4) pharmacy students were excluded from this evaluation because they had minimal exposure to blended learning due to the timing of incorporation. Students were purposefully selected through their participation in the school’s class council program. The first- (P1), second- (P2), and third-year (P3) classes each have their own class council, which meets at least monthly with their academic advisor. Prior to joining a class council, students must submit an application form, which is then reviewed by the class advisors. The class president and vice president are on the class council, but all other council members do not hold other leadership positions within the school. The students who were on class council at the time of the focus groups had varying GPAs with a similar average to their class’ overall GPA.

Class council members were introduced to their classmates via email at the beginning of the school year. Students were encouraged to provide feedback on their learning experiences to class council members through informal discussions or email. Class council members met with the course faculty to share the feedback discussed during the class council meetings and suggest potential changes. Class council members have experience working together and are comfortable providing positive and constructive feedback to faculty on their courses. These attributes made them ideal for the focus groups as they were asked to critique their ongoing courses and offer feedback on what worked and how to improve the courses. These council members had also seen a variety of blended learning courses and techniques. This exposure to variations in blended learning was also an advantage to allow for constant comparison of what worked well and what didn’t during the focus group meetings.

The investigators scheduled time during a class council meeting to review students’ blended learning experiences over the course of the previous school year. The focus group meetings were conducted within the last month of the Spring 2015 semester. Students were informed that the objective of the evaluation was to determine the status of blended learning and determine opportunities for quality improvement. They were also told that the focus group meetings were being audiotaped and that direct quotes may be used but would be de-identified while specific comments from identified individuals would not be reported back to the faculty. All participating students signed an audio release form. Students were also encouraged to share negative experiences regarding blended learning to generate suggestions for improvements to faculty.

After completion of the introduction to the focus group and student completion of the audio release forms, the facilitator began the audio recording. The facilitator was the school’s director of Instructional and Information Technology (IIT). This individual does not have a status relationship with the students (i.e. is not responsible for their grades) and was knowledgeable about technology use and the state of blended learning in the school, which allowed him to ask appropriate probing questions regarding the use of blended learning. He acknowledged students for sharing positive and negative feedback and assured them that their feedback would help faculty improve their course delivery.

Each class had an advisor who also attended the class council meeting. The advisors asked questions from the question guide (Table 1) and took notes during the focus group meeting. The question guide was developed to explore what worked well and where there had been problems from the students’ perspective with how blended learning had been implemented. The question guide was not piloted given the small number of students and focus groups being conducted, but was evaluated by a colleague of the investigators who had expertise in question writing and programmatic evaluation and had some experience with blended learning. Prior to beginning the discussion, the definition of blended learning used for this evaluation was reviewed with the students, including specific examples of what students have experienced thus far in the curriculum. The class advisor used the entire question guide during all three focus groups. The facilitator asked probing questions for clarification. The facilitator’s questions were not part of the question guide and were not standardized.

Table 1.

List of Questions Asked During the Focus Groups

Gisted transcription of the focus group audio recording was conducted by two investigators not present during the focus groups. Both investigators agreed on the transcription and compared it with the class advisor’s notes. For all three focus groups, the transcriptions were found to be consistent with the class advisor notes.

The same two investigators used content analysis methods and coded the transcriptions for common themes regarding the students’ perspective of blended learning practices. If a disagreement in the coding occurred, the two investigators discussed the issue for resolution. If it could not be resolved, the third investigator was available to make the determination; however, this was never needed. The initial eight themes used during the analysis were determined by the facilitator during the focus groups and upon review of the class advisor notes. Throughout the analysis, two additional themes were added and two of the themes were determined to not be consistent across all focus groups; however, both of those themes (consistency in team teaching and instructor feedback on student preparation) were still determined to be appropriate for inclusion in the best practices. The investigators analyzed the transcription for themes within each focus group and for negative cases, or situations where students shared examples that were inconsistent with previously reported information and themes.

RESULTS

Three separate focus groups were held in April 2015 for P1, P2, and P3 students, with eight, nine, and eight students attending the groups respectively. All students present at the class council meeting agreed to participate in the focus groups. The focus group meetings ran for 32, 31, and 25 minutes respectively.

Students identified benefits to blended learning which included receiving guidance on problem solving activities, ensuring all students were at the same knowledge level prior to starting a course activity, reviewing course material at an increased frequency (ie, reviewing the material on off days of class), delivering course material in a variety of ways and formats, applying the material to practical and real-world situations, and increasing the flexibility students had with their schedule. A P1 student said, “It is a good way to make sure we are looking at the material every day – we are doing modules on the off days and then lecture. It is more practice and better for retention and memory. It makes it easier to study for the exam at the end.”

The best practices are provided in Table 2. The first aspect was setting the stage. Students commented that it is helpful when the syllabus includes a schedule of all course activities, especially specific course activities related to blended learning, so they are aware of the course expectations. Students also found the outside-of-class work easier to manage when there was a detailed explanation of the expectations at the beginning of the semester. Students wanted to know how long outside-of-class activities would take for schedule planning purposes. A P2 student said, “The nice thing about having it written out in the syllabus ahead of time is that it won’t, hopefully, slip through the cracks.”

Table 2.

Student Perspectives on Best Practices for Blended Learning

Consistency with team teaching is closely related to setting the stage and was not consistently discussed in all focus groups as the first-year students had not experienced team teaching in a blended course. For classes with multiple instructors, students found that faculty within the same class sometimes gave varying amounts of outside-of- class work and posted online materials at different times in relation to the due date. Students found this harder to manage and would have preferred consistency within a course. Students also described that when multiple faculty members were teaching in a course, they sometimes had different ways of communicating with students (ie, through announcements on the learning management system versus use of a class email), which they found frustrating. Students often discussed consistency with team teaching at the same time as setting the stage. Appropriately, setting the stage may mitigate student frustration related to inconsistencies in blended learning use between instructors within a single course.

Students commented that there was a time range when online materials were posted in relation to the class session or due date. The best practice of timeliness encompasses instructors posting online materials with adequate time for the students to complete. Students in all three focus groups commented that short turnaround times were difficult to manage with other responsibilities, both school and personal in nature. In general, the consensus from all three focus groups was that posting online materials two weeks prior to the class or due date was a reasonable amount of time.

Time students spent completing an online task (ie, time on task) was discussed in detail in all three focus groups. The majority of students felt that they should be compensated for their time completing online course work with cancelled in-class time (ie, use of a replacement model for the blended learning component).¹⁵ Students explained that online lectures took them much longer to complete than the duration of recorded audio, as students paused the lecture to take notes. Additionally, students are still held accountable for online module material for assessment and exams. One focus group articulated that they considered their online materials to be equivalent to lecture or class time. A P3 student said, “A 2 hour lecture [outside of class] corresponds to 4 hours of studying to actually know the material.” In addition, students felt that it was difficult to manage multiple online modules from multiple courses at the same time without being given back some time. Another P3 student said, “I would understand if it was only one professor [in one class] … but when it is every class, it is just a lot.”

However, there were two exceptions to the suggestion for time compensation. The students agreed that if the online assignment was 15 minutes or less in duration, they were willing to complete the assignment prior to class and did not feel cancellation of an in-person class was needed. There was also a negative case regarding time on task where students felt that being compensated with cancelled classes for online lectures hindered their learning. These students would have preferred the additional class or discussion time for more in-depth, practical application of the material. However, the group of students who did not want class to be cancelled had the least amount of blended learning and all of their blended learning courses used a replacement model, with time off, which may explain the deviation from the other two focus groups.

Students discussed the link between timeliness and time on task. Students mentioned various situations when they had multiple online assignments from multiple classes all falling during the same time. This was further compounded when multiple courses gave online preparatory activities with short deadlines. When these situations occurred, students commented they would fall behind with the online materials and therefore didn’t feel attending class with active learning and discussion was worth it, as they weren’t prepared. These situations set them back even further. A P3 student said, “It’s really easy to fall behind too, especially when you have exams, to not watch [the online lectures] and then … miss discussion because you feel like you aren’t going to get much out of it because you’re already behind.”

Students voiced that they had to prioritize which activities they would complete prior to attending class and felt that they sometimes had to forego preparation in one class for another. A P2 student said, “There will be times when our exam schedule, lab schedule, and multiple classes with video things; you have to make this decision about what you put your time towards right now and sacrifice different classes for another class.”

Overall, students preferred when instructors held them accountable for completing online materials prior to attending class. They were motivated when they received points or course credit for completing work on time. If they didn’t receive points or course credit, they often expressed not always completing the online materials, which sometimes meant they didn’t attend the associated discussion period as they weren’t prepared. Examples of how instructors incorporated student accountability into their courses included quizzes and participation credit for viewing an online lecture. A P1 student said, “I think the quizzes are really good too as they force you to do the module instead of a textbook reading where you could skim through it and do it halfway and not really understand [the material]. The quizzes make sure you slow down and are prepared for class.”

In all three of the focus groups, students compared various ways instructors used active learning in the classroom after completing online material as a pre-class assignment. They felt they received the most benefit from true structured active learning based on application of the material. Examples shared by the students included real-world case discussions, additional examples, and practice problems. Active learning techniques could include think-pair-share, polling questions, and large group discussions facilitated by the instructor. Students disliked unstructured discussions (ie, only asking “What questions do you have about the online materials?”) as they were unable to assess where their knowledge was lacking and found it difficult for engagement in a large lecture hall setting without guided activities. They also disliked when instructors skipped the active learning and application of online material and lectured on new material during face-to-face time instead. Students also felt in-class time was most beneficial when the practice problems were application-based instead of a regurgitation of the online material. A P1 student said, “Time spent on my own is better spent listening to a lecture … time with a teacher is better spent being guided towards solving a problem.”

Students also discussed how they were more willing to spend the time on outside-of-class activities depending on what they took away from the in-person class time. One student said, “It also partially depends on how well it is done. If we get a lot out of class, I would be more willing to spend more time out of class knowing that going to class is going to be beneficial. Whereas if we had [a class that we didn’t get a lot out of the face-to-face time], why are we spending all of this time outside of class and then going to class and not getting anything out of it.”

The facilitator initially identified the theme of faculty feedback on student preparation after leading the focus groups. However, it was determined to be a minor theme after focus group transcription and analysis. In one of the focus groups, students described their instructor using the student results from the online materials to guide their active learning discussions. They were the only group that had this experience, and thus, the only group that discussed it. While only one focus group mentioned this, they felt very strongly that it enhanced the face-to-face active learning time with their instructor, as they had targeted review and application of the material. Additionally, the same group of students had an experience where another professor had quiz results prior to the start of class and did not tailor their learning in class, which they found to be a less positive learning experience.

Students appreciated when faculty incorporated student feedback into the course about the blended learning activities in real-time to improve their experience. Examples of real-time changes include asking students the amount of lead-in time needed to complete online activities, clarifications on the course syllabus, and adjustments in the use of face-to-face time. Changes made based on student feedback should be reported back to students because this encourages them to provide feedback to instructors. When it is not feasible to make changes in real time, student feedback can be considered for the following year.

Two additional themes were identified during content analysis. Reviewing the online material during class time came up in all three focus groups with mixed opinions. Two of the focus groups reported liking a short summary of the material presented online before proceeding to the active learning discussion. However, the third focus group recalled an experience where the professor gave a lecture repeating the online material during class and therefore, suggested not reviewing online material during face-to-face time. This suggests a brief review of complicated topics may be warranted at the beginning of a class session, but the majority of class time is best devoted to active learning and application of material.

In all three focus groups, students recalled different forms of technology that had been used for the online material. They appreciated when they could change the speed of the audio and when the menu allowed them to jump between topics. They also mentioned times when technological difficulties had delayed the release of materials for students, suggesting competent information technology support that is willing to meet instructor needs is critical.

DISCUSSION

This evaluation determined 10 best practices instructors can incorporate into their blended learning courses to improve the student experience. While the intent of this evaluation was to find areas for improvement in course design and delivery, student participants still felt that overall blended learning was useful and appreciated the time allowed for structured active learning. They also appreciated the flexibility in determining how and what they study, and not being prescribed what they have to do.^6,7,12

These instructional best practices for blended learning are consistent with other evaluations of students’ perceptions of blended learning.^13,16 Khanova and colleagues conducted a curriculum-wide evaluation of students’ comments and reactions to blended learning at a pharmacy school with a high uptake of blended learning.¹³ Several themes were consistent in their evaluation to the best practices from this evaluation, including consistency with team teaching, time on task, structured active learning, and reviewing online material during class. They also highlighted the quality of the online preparatory materials (ie, monotonous tone and errors in online materials being indicators of poor quality), which was not discussed by the students during these focus groups that led to the best practices. Nematollahi and colleagues described lessons learned from their medical anatomy blended course, including the themes from these best practices of time on task, structured active learning, and accountability.¹⁶ In both evaluations, students suggested approximately 20 minutes of preparatory work outside of class to be most acceptable. However, the curriculum evaluation article also highlighted the difficulty managing multiple online preparatory activities students can encounter when enrolled in multiple simultaneous blended courses.^13,16

Oftentimes an instructor chooses a blended learning course design to better use class time for active learning.^{2,9,10,12,17-19} Incorporation of active learning is especially important as it is likely the mechanism for increased student learning outcomes in blended courses.¹⁹ The use of an active learning pedagogy in curriculum development is listed in the Accreditation Council for Pharmacy Education 2016 Standards (ACPE).²⁰ Using the best practices outlined in this evaluation would allow instructors to incorporate structured active learning techniques to foster deeper learning and application of material to pharmacy practice.^9,10,19,21

This evaluation determined best practices, but was not designed to assess the best practices in relation to student learning outcomes. During the focus groups, students discussed getting behind in the online materials to the point that they would not attend class, as they were not prepared. This is consistent with other evaluations of blended learning courses where over time, students became behind in completing the online components of the course.¹⁷ In one study, less than 25% of students reported completing all or almost all of the 26 online lectures on time.¹⁷ Students were not held accountable through the course grade to complete the lectures on time. Using the best practices may promote completion of the activities on time; especially the recommendations regarding setting the stage, timeliness in posting materials, accountability, and time on task. For example, a blended learning course in which time off was given for online module completion, online activities were consistently posted two weeks prior to the due date, the course schedule included the due dates, and students were held accountable for on-time completion of the online lectures through course credit had 86% of students complete all 14 online lectures on time. If students complete the preparatory work and come prepared to participate in the structured learning activities in class, it can be expected that their learning outcomes would improve.^5,18

This evaluation identified time on task as a potential concern to students when blended learning is used. There are several suggestions on use of the replacement model and how much preparatory time may require a blended course to cut back on face-to-face time.^15,21 When considering this decision, instructors should consider preparation time, time to complete course activities and homework, and expected studying time. It should also be considered that the time it actually takes students to complete this work is often longer than what they or faculty may have initially anticipated.^2,5,8,13,21 Outside-of-class work is considered any work outside of class and may include reading, studying, and solving practice problems.²² While this may vary at different institutions, if out-of-class time is excessive, either the use of a replacement model minimizing the face-to-face time or increase in the credit hours for a class may be warranted.

At schools with multiple concurrent blended learning courses, there may be school-wide initiatives to improve the blended learning experience for students, in addition to individual instructors using these best practices. A school-wide definition of what constitutes blended learning can help with assessment of which courses should be included in these evaluations. School-wide policy determined through faculty agreement on time outside of class may be helpful in managing student workload. Additionally, creation of a blended learning calendar detailing when online activities are due, how long online activities are expected to take, and any time off from class may help students and instructors to better manage students’ workload. Lastly, schools with educational innovation initiatives to create or improve blended learning may consider offering formal faculty training and development in blended learning course design. One publicly available online resource that the UW-Madison currently offers is the Blended Learning Toolkit.²³

There are several limitations to this evaluation. The results may be influenced by the subset of students who participated in the focus groups. Each class of students has their own mindset, personality, and set of experiences with various classes and instructors and student preference for different teaching techniques. This evaluation included a relatively small sample size. By including other students in focus groups, other opinions may have been shared and this list of best practices may be more generally accepted. However, the students invited to participate were from a group considered representative of the general student body and the results from this analysis are consistent with other literature regarding student preferences toward blended learning.¹³ The students at the UW-Madison School of Pharmacy have had multiple simultaneous blended learning courses and these results may not be as generalizable to a school that does not use blended learning as extensively. However, this variation in preferences and variety of experiences with instructors from multiple courses using blended learning has given us a wider range of considerations and comparisons for our results, which may make it more robust.

Student preference may not always be a best teaching practice (eg, students didn’t like being penalized when they failed to complete work; yet they were more likely to complete the online work when credit is given). In some cases, these best practices may not lead to better learning outcomes.²⁴ For example, one author does not believe in giving an extensive review of the online material prior to starting the active learning discussion during the face-to-face time as it may lessen the student accountability for completing the preparatory work. Additionally, giving a class period off for every time a course requires more than 15 minutes of preparatory online work may not be appropriate or feasible, despite it being this group of students’ preference. Despite some of the best practices potentially not being in the students’ best learning interest, many of the best practices revolve around course and time management for students (ie, timeliness in posting and setting the stage), which may facilitate students’ completion of preparatory work prior to class.

Future directions stemming from this evaluation are twofold: evaluation of the best practices on student learning outcomes for an individual course and evaluation of potential school-wide initiatives. To evaluate the impact of the best practices on learning outcomes, a pre-post design implementing the best practices would need to be undertaken. At the UW-Madison School of Pharmacy, we are in the process of implementing several school-wide initiatives, including faculty engagement regarding the best practices, a blended learning calendar, and continued evaluation of student and faculty perceptions of the status of blended learning.

CONCLUSION

Instructors using blended learning should consider incorporating the best practices listed in this article into their course design and management. Schools and colleges of pharmacy should consider initiatives to support their instructors’ use of these practices. More evaluation is needed to determine if implementation of these practices affects student performance.