Effect of Live Attendance and Video Capture Viewing on Student Examination Performance

Abstract

Objective. To compare pharmacy students’ performance in a therapeutics course after attending live lectures and/or viewing video-recorded lectures.

Methods. Attendance was taken during seven lectures spaced equally throughout the therapeutics course. Data on students’ viewing of the video-recorded lectures was extracted. Students were grouped based on class attendance and video-viewing behavior; these data were correlated to student performance on examination lecture specific material. The data were also evaluated based on students’ final course grade.

Results. From each lecture for which data were collected, between 346 and 349 students were included in the analysis, resulting in 2,430 data points. Students who were attended lecture and did not access the video-recorded lecture were associated with better performance on the respective examination than students who were absent and accessed the video-recorded lecture only once (grade=71.0 vs 62.3). Students who attended lecture, regardless of whether they subsequently viewed the video online, were associated with better performance on the examination than students who were absent (70.4 vs 64.0). Among all students who attended lecture, those that also used the video-recorded lecture were associated with similar performance on the examination as those who did not access the video (grade=69.1 vs 71.0).

Conclusion. Results from this pilot study demonstrated that live class attendance was associated with higher examination performance than viewing recorded lectures for a therapeutics class. The results of this pilot study can be used to guide future research in understanding how teaching methods affect student performance.

INTRODUCTION

Video capture technology has become increasingly common in higher education as educators try to provide more flexibility for students and appeal to more learning styles.¹ Despite its continued adoption, the impact of video-captured lectures on student attendance and grades remains unclear.^1,2 The Accreditation Council of Pharmacy Education (ACPE) does not specifically require schools to use video capture technology in the classroom, but it does support the adoption of such technology, as noted in standards 18 and 21 of the ACPE Standards 2016.³ While the number of pharmacy programs currently recording lectures and making them available online to students is uncertain, survey data collected in 2011 by the American College of Clinical Pharmacy (ACCP) identified that 62 of 118 programs were using this technology.⁴ Students and faculty members find lecture recording a helpful way for students to stay current with course material and a beneficial study tool to review concepts.^1,5 As ease of use and access to video-capture technology increases, making class attendance optional in some cases, educators are concerned that students’ performance will be negatively impacted.^1,6-8

Several studies conducted within higher education have found mixed conclusions regarding outcomes associated with the availability of video-recorded lectures and class attendance.^1,2,5-16 Specifically in pharmacy education, published studies have demonstrated that students perceive the availability of video-recorded lectures positively, while faculty members are concerned about the use of video-recorded lectures and their subsequent impact on class attendance.^1,6-8,12 Overall, the evidence as to whether making video-recorded lectures available to students provides an academic benefit remains inconclusive.^8,12

Because of dwindling lecture attendance since the implementation of video-capture technology at our institution, this pilot study was designed to evaluate whether student self-selection to attend class and/or watch a video of the lecture impacted their performance on specific examination questions. The primary objective of this pilot observational study was to determine whether class attendance without watching the video-recorded lecture was associated with better performance on specific examination questions compared to performance by students who were absent from class and had accessed the online video of the lecture one time. These two groups were analyzed as both groups would consist of students who had heard the lecture one time, whether virtually or live. Secondary objectives were to determine whether being present or absent from class, regardless of whether the student watched a video of the lecture, impacted student performance on specific examination questions; and to determine whether students who attended class performed better on specific examination questions compared to those who were absent from class, regardless of whether they viewed the recorded lecture. These analyses were also conducted to determine whether class attendance and/or viewing the recorded lecture was more or less likely to impact students in terms of their final grade point averages for the course.

METHODS

At MCPHS University, Therapeutics I and II are mandatory courses in the fall and spring semesters during the second professional year (P2) of the Doctor of Pharmacy (PharmD) curriculum. At the time of this study, there were 353 students enrolled in the 2017 spring Therapeutics II course. The course curriculum covered fundamental pharmacotherapeutic and disease state management principles. Each course provided three credit hours of in-classroom lecture per week: one 100-minute class and one 50-minute class. Lectures in therapeutics were delivered by various full-time and adjunct faculty members, who presented topics relative to their field of practice in a lecture format. Handouts, guidelines for therapy, study guides, and other lecture materials were made available to students prior to class on the course’s Blackboard (Blackboard Inc; Washington, DC) site, an online learning management system. Faculty members were responsible for providing examination questions related to specific material discussed in their lectures to course coordinators. The course coordinators then reviewed each examination question provided by faculty members and attended every lecture to ensure the material covered was both thorough and appropriate for inclusion on the examination. Four paper-based, multiple-choice examinations were administered in the course. Three of these were interim examinations (noncumulative) that were each worth 21% of the overall course grade. The other was a cumulative, final examination worth 32% of the grade. Because of decreased lecture attendance, a 5% attendance component was made a part of the overall course grade to incentivize students to attend every class lecture. This study was designed after incorporation of the attendance grade. A minimum passing grade in the course was a C- (70%).

Video capture technology, specifically Echo360 (Echo360, Inc.; Reston, VA), was adopted at the college in 2013 to enhance student learning by making video recordings of class lectures available online. When used, this software captured the PowerPoint slides, video, and the voice of the lecturer during the entire class. The files were automatically uploaded online immediately following class and were available for enrolled students to view via the course’s Blackboard (Blackboard Inc; Washington, DC) website. Students had unlimited access to the recorded lectures for the duration of the semester. Echo360 (Echo360, Inc.; Reston, VA) users can pause and stop videos at their discretion, skip to sections of interest, and alter the speed at which the recordings played. While all lectures in this therapeutics course were recorded and made available to students, use of this technology was not mandatory. Thus, students accessing this material decided to do so for their own personal benefit. Because of the strictly didactic nature of the course, students received the same exact content, including cases and questions posed in class, whether sitting through the live lecture or watching the recorded lecture online. The one caveat was that students watching the lecture virtually were unable to participate in active-learning activities. However, use of active learning in the course was limited because of the sheer size of the class. At the beginning of the course, all students were encouraged to use the video-recorded lectures to augment rather than replace class attendance. Course coordinators also discussed the video capture technology in class and distributed a message electronically to all students regarding the use of video-captured lectures.

The Therapeutics II course was chosen for this retrospective review because attendance data for the course was readily available. An electronic notification informing students of the investigators’ intentions to perform the study was distributed before the conclusion of the course, and students were given instructions on how to have their data excluded from the study if they so desired. The university’s institutional review board approved this study and all study materials.

Twenty-five lectures were presented in the course over 14 weeks during the spring semester. A note card activity was used to take attendance during seven classes evenly distributed throughout the semester (weeks 2, 3, 6, 8, 10, 11, and 14). To take attendance, each student was given a note card on which to write a response to a question pertaining to the lecture material. The cards were then collected at the end of class. Those students who participated in at least five of the seven attendance-taking activities earned the full 5% toward their overall course grade. Students did not receive any credit for attendance if they missed more than two of the seven classes in which attendance was taken. Students were unaware of when lecture attendance would be taken. Students self-selected which lectures to attend and which lectures to watch on video.

Attendance was taken for the following lectures: diabetes, insomnia, thyroid disorders, peptic ulcer disease/gastroesophageal reflux disease (PUD/GERD), substance abuse, pain management, and depression. Several lecture topics were presented over multiple days, specifically depression, substance abuse, pain management, and diabetes. During these lectures, attendance was taken on only one day of the multi-day presentation. Only the examination questions pertaining to the seven lectures during which attendance was taken were considered for this analysis. All other examination questions were omitted to minimize the risk of confounding results. Students’ performance on these specific examination questions was then correlated to student attendance for each of the seven lectures.

Data were extracted from the Echo360 system to evaluate students’ video viewing behavior for each lecture included in the study. The number of times a student viewed each of the seven lectures included in the study was noted, and this information was then correlated electronically to class attendance on the day of the lecture and performance on specific lecture material.

The study population was initially divided into five categories based on attendance and overall use of the video-recorded lectures. These categories included students who were present for class and used the video-recorded lecture (P+V), students who were present for class but did not access the video-recorded lecture (P-V), and students who were absent from class but did not access the video (A-V). Those students absent from class who used the video-recorded lecture were divided into two categories: viewed the video-recorded lecture once (A+1V) and viewed the video-recorded lecture two or more times (A≥2V). As previously stated, this last category (A+1V) was included to allow for direct comparison with those students who were present for class but did not access the video-recorded lecture (P-V) as both groups would consist of students who had only viewed the lecture once. Students present for the in-class lecture could then be directly compared to students absent from the in-class lecture to determine how attendance impacted students’ overall performance on material covered during these specific lectures. Finally, all groups were evaluated based on their overall final performance within the course (top 50% of the class vs bottom 50% of the class based on final grade). This was done to determine whether the top- or bottom-performing students in the course were more or less likely to be impacted by attendance or use of video-recorded lectures. Those students who were required to make up an examination or received an incomplete for the course were omitted from this analysis.

To detect for significance between and within study groups, repeated-measures analysis of variance with Newman-Keuls post-hoc testing was used. All statistical analyses were conducted using NCSS 2007 software (NCSS, LLC; Kaysville, UT). Results were considered significant if p<.05.

RESULTS

For each of seven lectures, between 346 and 349 students were included in the analysis, which resulted in 2,430 data points. When evaluating all lectures, 1,792 student data points were in the P (present) group and 638 student data points were in the A (absent) group. Five hundred forty-six student data points were in the P+V group, 1,246 were in the P-V group, 198 were in the A-V group, 278 were in the A+1V group and 162 were in the A≥2V group (Table 1). All scores included in the analysis represent percentage correct for examination questions related to each of the seven specific lecture topics as opposed to overall performance on the examination itself. The group was further broken into two cohorts: the top 50% of the class were those students achieving a final course grade of ≥80.5%; whereas, the bottom 50% of the class was determined to be those students achieving a final course grade of <80.5%.

Table 1.

Mean Student Examination Performance Based on Number of Video Capture Views

When evaluating our primary objective, students in the P-V group performed significantly better than students in the A+1V group (examination grade=71.0 vs 62.3, p<.05, effect size=0.51). Additionally, when evaluating attendance with any combination of video use, several differences were noted. Students who were present for a lecture and did not watch the video performed significantly better than both students who were absent from the lecture and viewed the video of the lecture more than once (examination grade=71.0 vs 67.2, p<.05, effect size=0.21) and students who were absent from the lecture and did not view the video (examination grade=71.0 vs 63.7, p<.05, effect size=0.41). Also, those students who were present for the lecture and later watched the video performed significantly better than those who were absent from the lecture and viewed the video only once A+1V (examination grade=69.1 vs 62.3, p<.05, effect size=0.41) and A-V cohorts (examination grade=69.1 vs 63.7, p<.05, effect size=0.31). There was no significant difference noted in examination performance between the P+V and P-V cohorts (examination grade=69.1 vs 71.0; Table 2).

Table 2.

Examination Performance Between Study Groups Based on Lecture Viewing Behavior

When evaluating the group as a whole, regardless of video use, students who were present performed significantly better than students who were absent (examination grade=70.4 vs 64.0, p<.05, effect size=0.37). This difference was also noted when evaluating both the top-performing cohort (76.6 vs 68.5, p<.05, effect size=0.50) and bottom performing cohort (63.7 vs 60.4, p<.05, effect size=0.19; Table 3).

Table 3.

Overall Student Examination Performance Based on Classroom Attendance

Several significant differences were observed when data were further categorized based on the students’ final performance in the course. Of the students in the top 50% of the class, those in the P-V group performed significantly better than those students in the A+1V (examination grade=77.5 vs 64.4, p<.05, effect size=0.86) and A-V (examination grade=77.5 vs 70.8, p<.05, effect size=0.40). Additionally, among the students absent from class, those who watched the video more than once performed significantly better than those who watched the video only once (examination grade=73.7 vs 64.4, p<.05, effect size=0.52). Of the students in the bottom 50% of the class, there were no significant differences observed between any of the groups (Table 2).

DISCUSSION

There has been a rising concern at the MCPHS University regarding the lack of lecture attendance since the introduction of video-capturing technology. This subjective trend of decreased attendance had been observed for multiple years in this therapeutics course and resulted in the addition of an attendance component to the overall course grade. With this graded component, attendance could easily be tracked and compared to overall performance on specific lecture material. Similarly, the functionalities of the video capture technology allowed for tracking of students recorded-lecture viewing. With the availability of these data, it was possible to compare how self-selection regarding attendance and any variation of video-recorded lecture use impacted performance related to the material covered. In addition to this self-selected behavior, we elected to determine whether final overall average achieved in the course impacted any of the previously mentioned outcomes.

There are several studies in higher education that show lecture attendance is directly correlated with positive student outcomes.^9-11 In a meta-analysis including over 9,000 students, Crede and colleagues found that class attendance was a better predictor of grade point average (GPA) than study skills, study habits, Scholastic Aptitude Test (SAT) scores, and high school performance.¹⁰ In survey data, students in healthcare fields report ambivalence as to whether the availability of video-recorded lectures influences class attendance; however, the association between lecture recordings and student outcomes is unclear.^13,14 Johnston and colleagues evaluated grades achieved between two nursing cohorts; one with access to video-recorded lectures and one without access to them.¹⁵ The nursing students with access to the recorded lectures had improved examination performance but decreased overall course performance when laboratory examination grades were taken into account.¹⁵

In pharmacy education, DiVall and colleagues surveyed P3 pharmacy students and faculty members at six colleges of pharmacy to assess the use of technology, including video capture, in the classroom. Results revealed that students considered video-recorded lectures to be “most valuable to their learning” in comparison to other classroom technologies and that professors used the technology “effectively.”⁶ Marchand and colleagues surveyed faculty members and P1, P2, and P3 students at a single pharmacy school to determine their perceptions of video-recorded lectures .¹ In that study, students felt video-recorded lectures augmented their learning, while faculty members felt that the availability of recorded lectures negatively impacted student attendance but did not negatively affect overall student engagement in the course.¹ In a similar survey, Maynor and colleagues found that the majority of students found video-recorded lectures “beneficial” to their learning.⁸ Alternatively, 46% of students reported that they were more likely to miss class if the recorded lecture would be available; however, less than 10% of students reported using video-recorded lectures as a replacement for attending class.⁸ Bollmeier and colleagues evaluated the direct impact of the availability of video-recorded lectures on student attendance and final course grades in a P2 therapeutics course at a single school of pharmacy.¹² No correlation was found between final course grades and the number of times the recordings were accessed; 72% of students with access to the recorded lectures reported attending class.¹² Our study was most similar to that performed by Bollmeir and colleagues as both studies evaluated the effect of having video-captured lectures from a P2 therapeutics course available to students; however, Bollmeir and colleagues evaluated the effect of the total number and duration of time online course lecture files were accessed on final course grades.¹²

Our pilot study was observational in design; thus, the results reflect students’ choices to attend class or watch the video-recorded lecture. This allowed for direct observation and analysis of organic student behavior and related outcomes, rather than unethically forcing students to adopt new class and study habits (forced attendance and/or withholding video-recorded lectures or vice versa) over the course of an entire semester in a required therapeutics course. In addition, the impact of attendance with or without viewing video-recorded lectures was evaluated based on students’ performance on specific examination questions relating to the topics where attendance and video-recorded lectures use was measured. This design differs from previously published literature in that we removed potential confounding of the results and allowing for a more direct correlation of student behavior to performance.^12,15

The primary study outcome was achieved; students in this course who chose to attend the lectures and not view the video-recorded version on the dates on which data were gathered performed significantly better than students who missed class and used the video-recorded lectures. This was achieved with a medium effect size, indicating a moderate to high practical significance. This suggests that within this course, for the topics studied, improved examination performance may be associated more with live class attendance than with watching video-recorded lectures.

The differences recognized when evaluating our secondary objectives remain notable given the significant results, albeit small effect sizes. There was also a difference in the time the students devoted to listening to the lecture. Those present for class spent 50 minutes listening to lecture material, whereas students absent may have devoted significantly more or less time listening to the same material.

A particularly interesting finding in this study was that students absent from class performed better on lecture material when video-recorded lecture was not used compared to when the video-recorded lecture was used only once. This could have resulted from these students employing alternative study methods (eg, study guides, learning from peers’ notes, self-recording) that allowed them to be successful on the respective examination without the need to be present physically or virtually.

There was no significant difference noted in performance on specific examination questions related to the material covered among those students who attended class and watched the video-recorded lecture compared to those who attended class and did not watch the video-recorded lecture. This finding may suggest that, in addition to attending class, some students may need the additional exposure to the course material provided by watching the video-recorded lectures to perform as well as other students. Although students in our study performed similarly despite varying levels of exposure to the lecture material, our study also demonstrated that all students present in class for the lecture performed significantly better on the related examination items than students absent from the lecture who accessed the video-recorded lecture only once or not at all.

The reason why live class attendance yields improved performance on examination questions related to the material covered remains unclear. In a study performed by Owsten and colleagues, students were surveyed regarding various aspects of their in-class behavior (classroom participation, attention span, note taking) when video-recorded lectures were available, and determined there were no significant correlations between the aforementioned behaviors and students’ grades.¹⁶ In our pilot study, we found no conclusive evidence as to why students who attended the live lecture performed better on the subsequent examination compared to those absent from class. Further research is warranted to determine this correlation.

Many of the same results seen in the entire study population were also apparent in the top performing half of students in the class. Alternatively, none of these results were noted in the bottom performing half of students, indicating that within our study population, neither the self-selection of class attendance or video-recorded lecture use had any impact on performance on selected examination questions for students achieving less than 80.5% for their overall course grade.

The authors recognize several limitations regarding the design of this pilot study. Most notably, this study was performed at one school of pharmacy in one therapeutics course over one semester. This may limit the applicability of these results across all pharmacy schools and courses; therefore, these results should be viewed as exploratory. An additional limitation was the student self-selection bias regarding class attendance and use of video-recorded lectures. While inherent to the observational study design, this limitation makes it difficult to determine whether better-performing students are more likely to attend in-class lectures or students who attend in-class lectures are more likely to perform better (regardless of whether they use video-recorded lectures). A true random sample would require randomization of students into the four study groups used (P-V, P+V, A-V, A+1V and A+2V) and then examine this impact on student performance. However as this is a pilot study, randomizing students into these groups could not be justified based on ethical concerns.

Another major limitation of this study was the influence that the class attendance and participation grade may have had on students’ decisions to attend or not attend class. As the marginal increase in students’ final grade point average that would result from the 5% participation grade may have been enough to incentivize attendance, students who self-selected to attend lecture may have done so to gain additional points rather than to benefit from attending a live lecture. Additionally, if a student missed more than two lectures, knowing they would not earn attendance points may have altered their future attendance behavior. Furthermore, earning or not earning the five percentage points for attendance may have influenced how a student was classified in the study using the 80.5% cut-off when the data were stratified based on the top and bottom performers in the course.

For lectures presented over multiple days, attendance was only taken on one day; thus, attendance for the full topic may be overrepresented. Also, the study investigators were only able to view how many times students accessed a recorded lecture online rather than how long they spent viewing the lecture.

Other limiting factors that may have impacted the final results include: students who used their own technology (eg, smart phone, tablet) to record the lecture during class to view later, and students viewing the recorded video online as a group, which would have been recorded by the software as one view. Furthermore, the outcomes for this study are specific to this course's didactic format and may not be applicable to courses with an alternative format, such as a flipped classroom, a team-based learning environment, or additional experiential/laboratory components. In these settings, students who come to class and interact with their peers may have an advantage over those who do not attend class.

For future studies, it would be beneficial to omit the incentive of an attendance grade and assess attendance during every lecture along with all variations of video access to determine how these variables affect performance on specific examination questions. Determining the specific amount of time students spend viewing video-recorded lectures, the speed at which they view the lectures, and the date on which they view them in relation to the live lecture and examination date would also be of interest. This could provide a better understanding of whether students are using the recorded lectures for supplementation of material missed during class or in place of attending the live lecture. Because our study suggests that students who attend class may perform better overall than students who do not, it would be advantageous to evaluate video viewing behavior in a setting where class attendance is mandatory to determine how this would affect students’ overall performance. Future researchers may want to assess these variables in a smaller classroom setting rather than in the large lecture hall setting used in our study. Another concept that should be assessed is the study methods of students who attend class consistently compared to those of students who do not, which could be evaluated via a questionnaire or open-response survey, as these factors may influence student outcomes.

CONCLUSION

This pilot observational study contributes to the current research by evaluating a previously unstudied, alternative approach to understanding student performance within the context of increasing technology use and education. However, several limitations significantly reduced the external validity of this study, and additional research should be conducted to evaluate the effects that class attendance and use of video-recorded lectures have on student performance in higher education. While this study provides insight into the benefits of video capture technology and making recorded lectures available to students after a class, it remains unclear how this type of learning technology impacts students’ long-term retention of lecture material.