Impact of Online Lecture-capture on Student Outcomes in a Therapeutics Course

Abstract

Objectives

To examine the correlation between students accessing recorded lecture files (audio and slides) online and course grades and class attendance.

Methods

Second professional year (of 6-year program) students in a therapeutics course had access to recorded online lectures for 72 hours following live lectures. The number and duration of lecture accessions were compared to final course grades and class attendance. Course grades were compared to those of a historical control group. At the end of the semester, students completed a brief survey instrument regarding their use and perceptions of online lectures.

Results

No correlation was found between final course grades and the number of lecture accessions (r = 0.0014) or total number of minutes lectures were viewed (r = 0.033), nor between class attendance and minutes viewed (r = 0.2158). Students with access to recorded lectures outperformed the historical control group on the final examination (p < 0.002). Seventy-two percent of students reported no influence of online files on class attendance.

Conclusions

Posting lectures online did not affect student outcomes, but students did score higher on the final examination.

INTRODUCTION

College students are part of the digital generation, a group which challenges college faculty members to rethink the role of technology in the classroom.¹ Moreover, with recent advances in large classroom technology, faculty members can offer students a variety of lecture enhancements. Pharmacy students often have access to lecture handouts and slides, and students now can access audio files, which are available for viewing and downloading after the lecture has taken place. In a national study of college seniors, 68% reported the availability of course management system software in their classes.² These systems give students access to course materials, allow communication among instructors and students, and also can be used as a venue for posting lecture files for students to view. Faculty members wonder if access to these materials on the Internet will improve or impede academic achievement.³ As such programs become available, their benefits, effects on class attendance, and pharmacy students' reactions to their use should be documented.

Pharmacy students are given an enormous amount of detailed information within a traditional lecture period, and those not attending live lectures neglect learning opportunities. By integrating factual information with active-learning techniques, the typical therapeutics class is designed to engage the students in the topic of the day. Examples of active learning in therapeutics include patient-case scenarios where students are asked to evaluate therapy, select and recommend drug treatment, or monitor therapeutic outcomes in the context of a real-life patient. Other examples of active learning employed by therapeutics lecturers are “think-pair-share” exercises, “muddiest point,” and developing treatment algorithms. Students who attend class also have the opportunity to ask questions. Instructors can sometimes interpret student body language and adjust teaching styles or the progression of the class to ensure student understanding. If absent, the student may not benefit from the full experience the in-class session and activities are meant to provide.

Often, students are expected to complete required reading prior to attending class to keep pace with experts discussing the topic. This can be difficult for some students, as explained by the cognitive load theory which describes learning as a processing system consisting of short-term memory, working memory, and long-term memory.⁴ Information passes from the short-term memory into the working memory, where it can be associated with 1 or more principles already known. Then information is integrated into the long-term memory storage. The working memory is limited both in its capacity and duration of storage. For this reason, if a lecture is poorly organized or contains too much information, the working memory cannot keep up and learning is impeded.⁴ Also, students with poor background knowledge of the material may have a difficult time maintaining attention and taking adequate notes on the information presented.⁵ This may occur if students do not complete reading assignments. Students' inability to handle the demands of notetaking and staying focused may result in notes that contain less than 50% of the ideas presented during a lecture.⁵ Students must comprehend what they hear, write it down, organize the information, and then store and integrate that freshly processed material from existing knowledge.⁶ This requires the working memory to acquire and understand new incoming information, which must interact with already stored knowledge.⁶ To compensate, students often request access to the lecture slides to review and fill in missing information in their lecture notes.

Attending traditional lectures may be influenced by the students' ability to view lectures online at a later date. Faculty members often assume that class attendance will decrease dramatically if they allow students to view lectures online from their home or work. One author reported that students cited access to online notes as playing a “very significant” role in their voluntary absence from class.⁵ Other studies, however, have not shown a negative relationship between students' class attendance and online resource use.³ Common reasons why pharmacy students do not attend class include: being sick, feeling tired or oversleeping, working on an assignment or studying for another course instead, and knowing that attendance is not recorded. Other reported reasons include course content is available from another source, and they do not feel they learn much when they do attend class. ⁷

Course coordinators set out to determine whether providing online lectures (both audio and lecture slides), along with customary lecture handouts, to pharmacy students enrolled in the spring 2009 Therapeutics II course improved student performance. The primary endpoint was the correlation between the total number and duration of accessions of online course lecture files and final course grades. Secondary endpoints included correlation of total number and duration of accessions to weekly class attendance, and a comparison of final examination and course grades to control groups.

METHODS

Students enrolled in the spring 2009 semester of the Therapeutics II course at St. Louis College of Pharmacy were asked to participate in the research project. Informed consent was obtained from 122 of 195 (62.5%) students enrolled in the course. Informed consent was obtained for collecting grade and survey information; however, attendance and online viewing data were not included because these outcomes were tracked in aggregate and were anonymous. The Institutional Review Board approved this protocol.

Therapeutics II was a team-taught course with experts from the pharmacy practice division providing lectures. Handouts for each session were created by each content expert and were available for students to download and print from the course homepage. The handouts provided patient-case scenarios, as well as fill-in-the-blank spaces for students to complete. Therapeutics II met for a traditional 3-hour lecture block once weekly with two 10-minute breaks between each hour. Class attendance was taken in aggregate after the second break throughout the semester and was recorded in a spreadsheet. Class attendance had not been tracked historically. Each lecture was recorded by the Tegrity Campus 2.0 Program (Tegrity USA, Santa Clara, CA) which automatically captured, stored, and indexed the lecture for later review by students.⁸ Lecture files were posted to the course management homepage, and all that was needed to view the files was an Internet connection.⁸ Students had access to these files for 72 hours following each lecture. Because this was the first semester students had access to online lectures, faculty members in the pharmacy practice division deemed a 3-day window for viewing to be appropriate. The pharmacy practice division's intent was to provide short-term availability without negatively influencing class attendance. Also, faculty members were concerned that if posted files were made available for an extended period of time, students might wait to view the files until just prior to examinations, rather than clarifying material after class.

A spreadsheet was created to contain study data that included student identifier, student final course grade, and total number of minutes Tegrity files were viewed throughout the semester. This spreadsheet was stored on a password-protected computer and housed on the restricted faculty hard drive to ensure student confidentiality. Once all information was obtained and the study spreadsheet was updated, student identifiers were purged, making the data anonymous prior to statistical analysis. At the completion of the semester, participating students were asked to complete an anonymous survey instrument regarding their experiences with the Tegrity program. The survey instrument was housed on a password-protected Web page provided by the St. Louis College of Pharmacy information technology department. A link to the survey instrument was sent by study investigators only to those students who signed the informed consent documents. Responders were asked about their number of lecture-file accessions throughout the semester, as well as how helpful the files were when: (a) completing their handouts; (b) studying for examinations; or (c) completing homework assignments. Investigators also asked students to indicate what period of time they would prefer the online lecture files to be available.

Statistical methods included a Pearson's correlation coefficient comparing number and total duration of lecture accessions and final course grades. Study investigators also used a student t test to compare the 2009 Therapeutics II students with 2008 Therapeutics II students, as well as a paired student t test to compare 2009 Therapeutics II students with themselves in their 2008 Therapeutics I course, during which time Tegrity was not used.

RESULTS

No correlations were found for the primary and secondary endpoints (Table 1). Specifically, each lecture viewing was not associated with class attendance (Figure 1). Differences with historical controls were significant for final examination grades but not for final course grades (Table 2).

Table 1.

Correlation Coefficients of Primary and Secondary Endpoints

n = number of students assessed

Figure 1.

Number of Tegrity Viewings Versus Attendance

Table 2.

Grade Comparison of Therapeutics II 2009 to Controls (Self in Therapeutics I and Previous Year Therapeutics 2008) (n = 122)

T2 = Therapeutics II

T1 = Therapeutics I

The average total number of lecture accessions throughout the semester was 3.4 out of a total of 24 available, with a duration range from 0 to 19. Of the 40 hours available, the average viewing time throughout the semester was 2.5 hours. This also had a wide range; some students watched 0 minutes and others viewed almost 14 hours. Seventy-four of 122 students (60.7%) completed the anonymous online survey instrument related to Tegrity at the conclusion of the semester. Results from the survey are depicted in Figure 2.

Figure 2.

Student Responses “Tegrity Helped With …”

DISCUSSION

The investigators found that student use of online, recorded lectures was lower than anticipated. The actual number of accessions and average time students spent viewing the files were low compared to availability of the files. Given that students are technologically savvy, the authors postulated that these numbers would be higher. Never having used lecture-capture software in a therapeutics course before, the course coordinators anticipated that students would use all available technology to enhance their learning experience.

The survey results revealed that the majority of students used theTegrity program to access at least 1 lecture online. Most students reported that they watched an entire 2-hour lecture. In the past, students often expressed concern that they could not keep pace with the lecturer or record all the information in their notes. With recorded lectures available online, the students now had the capability not only to look at the posted slides, but also hear them being explained. This may prove to be a key aspect to posting lectures online. An overwhelming majority of students (86%) stated they would like more lectures available online in the future, despite the low use throughout the semester.

Providing pharmacy students with the resources to access therapeutics lectures online has the potential to influence class attendance as students can access the needed information from home instead of attending class.⁹ However, providing access might improve academic performance by providing students with additional study resources.³

As lecturers and discussion facilitators, faculty members often fear an empty classroom. Student absenteeism can result in inadequate learning and poor academic performance.⁷ Also, pharmacy students observe faculty members and recognize them as role models. If students are chronically absent from class, opportunities to instill professional attitudes and values may be missed.⁷ Student motivation may play a role in who chooses to attend a live lecture and also may be linked to learning and retaining information given during class. Highly motivated students are more likely to persevere in difficult cognitive processes necessary for creating and organizing knowledge. These processes, or knowledge acquisition, also are influenced by classroom factors such as interacting with teachers and peers, creating an optimal learning environment.¹⁰

Faculty members prefer to have students in class over a near empty classroom. Given that today's pharmacy students are part of the “net generation,” they prefer to multi-task, are comfortable with multimedia, and like to interact with each other.¹¹ As technology continues to advance, the capability to include advancements in the classroom will also progress. As changes to courses are implemented, decisions that may impede classroom attendance should not be made lightly. Some examples of ways to promote classroom attendance follow. A compromise can be reached with students regarding when and for how long lectures will be posted online. One example is to wait to post material 10 days after each class session. This allows students to use the material to study for examinations, but also promotes class attendance.⁹ Another idea is to stop the recording approximately 10 minutes before the end of the lecture to review what the examination will cover or to take questions and comments from the class. Students must attend class to participate in the discussion. Another suggestion is to stop posting the recordings to the class homepage if in-class attendance drops dramatically. Of note, a possible benefit to faculty members of lecture-recording systems is that fewer students stop by during office hours (or e-mail the lecturer) with simple questions because students are able to access the slides from the presentation and answer their questions independently.⁹

As technology continues to evolve, lecturers should try to use class time to create a learning community. This will keep students focused on key concepts that are being highlighted, and engaged in the class discussion and notetaking. Other practices that improve attendance and students' learning include an in-class review process, in-class writing exercises, interactive in-class exercises, and discussion.⁷ These methods are often used in therapeutics with real-life patient case examples. The overall goal should be for faculty members to make attending lectures appealing by incorporating active-learning methods into the presentation, giving students time to interact and learn from peers, challenging them to use higher order thinking skills, and filling in missing information from notes.

Limitations include challenges with the online system (Tegrity), attendance issues, the amount of time that files were viewable, and low participation. Tegrity challenges involve the way online use was tracked. The Tegrity program documented use only when a file was playing, making it difficult to differentiate whether students were watching a file or simply left the program running. Also, the amount of time lectures were viewed was documented as an aggregate measure only. There was no way to track how many minutes of lecture were viewed by individual students. Additionally, students may have watched files in pairs or groups and this may have influenced the data collected. The Tegrity program also allowed students to watch files in double-speed, which could have influenced the time viewed and data collected. The large range in both number of accessions and minutes spent viewing the recorded lecture files may have made the correlation coefficient statistic less reliable. Also, the number of viewings by lecture topic was available only in aggregate; therefore, data were presented from all 195 students in the class (Figure 1). The investigators recorded class attendance in aggregate, making it difficult to quantify how each student's attendance may have been influenced by the use of the Tegrity system. However, this was done in an effort not to bias the student's behavior. We felt that if students were required to sign in to the traditional lecture, they would be more likely to attend class. Presumably, the more motivated the student, the more likely they are to attend class. Attending class, however, does not ensure that learning always takes place.

Students' access to lectures was limited to 72 hours after each lecture was recorded. If students had access to each lecture for a longer period of time (for example until that material was covered on an examination, or throughout the semester) presumably the use of Tegrity would increase. Students commented that they would have used the program to review material, help them study, or fill in missing material from the handouts. The precise amount of time students need access to online lecture files has yet to be determined.

Another drawback to the study was the low participation rate. Only 62.5% of the Therapeutics II students participated in the study by signing and returning informed consent documentation. This may have impacted negatively the correlation coefficient results.

CONCLUSIONS

The use of the live lecture capture system (Tegrity) did not affect student grades or attendance in the Therapeutics II course. The total number of lecture accessions or minutes viewing the files did not correlate with students final course grades. Therapeutics II students did have higher scores on the Therapeutics II final examination compared to their Therapeutics I final examination. Students subjectively found the system useful and indicated they would enjoy having more online lecture files available in the future. Further studies are needed to evaluate the optimum amount of time to post online lectures and to compare the impact of viewing online lectures with course outcomes and attendance.