Performance of Pharmacy Students in a Communications Course Delivered Online During the COVID-19 Pandemic

Abstract

Objective. To examine pharmacy student readiness, reception, and performance in a communications course during the COVID-19 pandemic and to compare that with the performance of students who completed the same course in person the previous year.

Methods. First-year Doctor of Pharmacy students (2020 cohort) enrolled in a professional communications course completed pre- and post-course surveys regarding their readiness for and changes in perception of online learning. Student learning was assessed using midterm and final examination grades. These grades were then compared with those of students who had completed the same course in person (on campus) the previous year (2019 cohort).

Results. Students’ preference for face-to-face instruction decreased from the pre-course to the post-course survey as indicated by responses made using a five-point Likert-scale (difference in means = −1.59; p < .05). Their comfort level with online learning increased (difference in means = +0.38, p < .05) by the end of the course. Students did not perceive any appreciable changes in rapport with the instructor by the end of the study. Course performance of students in the online cohort did not differ significantly from that of the 2019 cohort (p>.05).

Conclusion. This study demonstrated that first year PharmD students were already somewhat prepared for online learning when they began a communication course, with further adjustment occurring as the quarter progressed. Remote online learning did not seem to impact pharmacy student learning in this communications course conducted during the COVID-19 crisis.

INTRODUCTION

Remote online learning or e-learning refers to an instructional process in a classroom environment where students are physically distant from the instructor.¹ As early as 1993, the Alfred P. Sloan Foundation initiated network-enabled, self-paced learning programs that did not have time and space constraints.² Currently, e-learning consists of two principal categories: Distance learning and Computer-assisted learning.³ Despite some differences in pedagogical methodologies, terms such as web-based, virtual, online, distributed, and internet-based learning, among others, have been used synonymously to refer to e-learning. For the clarity of this communication, the authors use the term online learning to indicate synchronous online learning where the students and instructor are remotely interconnected by computer technology.

Several advantages and barriers to e-learning have been cited in the literature. The advantages include easy accessibility with flexibility,⁴ increased class participation,⁵ and learning outcomes as good as those achieved in traditional instructional settings.^6,7 Some of the limitations noted for e-learning include technical difficulties,⁶ reduced interaction and discussion with peers,⁸ inability to virtualize every aspect of learning (eg, hands-on laboratory experience),⁹ lack of professional growth due to absence of in-person role models,⁸ suboptimal communication skills development,¹⁰ and increased risk of isolation and anxiety.¹¹ Over the last decade, there has been a gradual shift to e-learning in pharmacy education, with both required and elective courses being delivered virtually. These courses may be entirely virtual or a hybrid of virtual and face-to-face instruction.^12-15

Effective communication plays a vital role in the professional practice of a pharmacist. A late 1990s report from the World Health Organization detailed seven crucial roles of a pharmacist in the health care system, one of which is “communicator.”¹⁶ Pharmacists are expected to possess good communication skills, including empathy and critical thinking, with the ultimate objective of improving health outcomes and overall patient satisfaction. A review by Wallman and colleagues summarized various skills and topics taught to students.¹⁷ Among these, oral communication skills were the most often taught, followed by written communication skills. Both simulated and standardized-patient interaction teaching methods are currently employed.¹⁷

The COVID-19 pandemic brought challenges to all facets of life, including teaching and learning. By early summer 2020, journals in various health care professions began publishing papers discussing the unique challenges and opportunities faced by educators in the wake of the pandemic. The Journal devoted its June 2020 issue exclusively to addressing the problems and presenting solutions by pharmacy educators. These communications covered a gamut of exigencies, ranging from accreditation of pharmacy programs to student enrollment to sustainable pharmacy education to facilitating the well-being of students and faculty during the crisis. In particular, Lyons and colleagues provided pragmatic solutions to delivering emergency remote teaching.¹⁸ Their suggestions included providing synchronous lectures through a video conferencing platform, developing quizzes as pass or fail, and using breakout rooms for small group discussions and activities, among other measures. Although much has been discussed about online teaching for educators, there is very little information available from the student perspective about online learning amid the pandemic. For example, were pharmacy students ready to receive course instruction and be evaluated entirely online during the crisis?

A pre-pandemic report by Wei and colleagues discussed the notion of readiness in online instruction.¹⁹ Student comfort with learning resources, degree of self-direction, beliefs about distance education, and a desire for interaction with peers and instructors were identified as components of readiness. Additionally, while Dray and colleagues considered access to, nature of, frequency of usage, and comfort with technology as components of readiness,²⁰ Yu and Richardson viewed factors such as social competencies with peers and instructors, among others, as prime determinants of readiness to online learning.²¹ In the context of the present study, these are all valid components of readiness.

It was vital for the Academy to investigate whether a course such as Professional Communications, which relied heavily on interpersonal development and learning, would fare well when abruptly transitioned to online learning. The students and instructor did not meet in-person at any time during the course because of the social distancing restrictions mandated by the state and university. The primary objective of the present study was to evaluate the readiness of first year Doctor of Pharmacy (PharmD) students for online learning and monitor changes in their receptiveness to a communications course delivered completely online amid the COVID-19 pandemic. The secondary objective of the study was to compare student performance in the online course with that of students who had taken the same course, coordinated by the same instructor, on campus (ie, in person) the previous year.

METHODS

Professional Communications is a required three credit hour course taught in the first quarter of the first year of the PharmD program at the American University of Health Sciences School of Pharmacy. The course introduces students to the concepts and principles of interpersonal and professional communication and strategies for effective communication with patients and other health care providers. In addition, the role of health literacy as well as cultural influence on communication of health information is taught.

In summer 2019, the course was organized into 15 lectures, and in summer 2020, it was organized into 16 lectures. In 2019, the course was delivered face-to-face, with students attending the course in a classroom on campus. Students completed assessments in a traditional manner with the course coordinator proctoring them in a classroom. However, in 2020, because of the COVID-19 pandemic, the course was delivered entirely online using the video conferencing platform, Zoom (Zoom Video Communications), with no in-person interaction between students and the instructor. To maintain the academic integrity of the assessments administered in the online class, students’ test-taking was remotely monitored by Proctorio (www.proctorio.com).²² The class demographics for the two cohorts and data for technology use by the 2020 cohort are presented in Table 1.

Table 1.

Demographics of First Year Pharmacy Students Enrolled in a Communications Course with Either In-Person (2019) or Online (2020) Learning

Student learning in the course for the two cohorts was assessed through the following class activities: conducting a pharmacist interview, writing a professional memo, creating a patient education pamphlet, participating in a role-play of an objective structured clinical examination (OSCE), taking quizzes, and completing midterm and final examinations. The pharmacist interview assignment required students to conduct an in-person interview of a pharmacist regarding their beliefs and attitudes related to the profession. For the professional memo activity students were required to write a memorandum that addressed a situation, topic, or concern to develop professional writing skills. For the patient education pamphlet, students had to create educational material to serve as a tool to communicate medical information to a layperson in easy-to-comprehend language. In the OSCE role-play, students had to apply patient counseling skills by mutually role-playing as pharmacist and patient, which involved exchanging information, demonstrating effective listening skills, expressing sympathy, and ensuring patient advocacy. The quizzes, midterm, and final examination were assessed using conventional multiple-choice questions, whereas the pharmacist interviews, professional memo, patient education pamphlet, and OSCE role-play were assessed using instructor-designed grading rubrics. The topics taught in the two courses and type of assessments used for the two cohorts were similar, but the modes of delivery (face-to-face vs online) were different.

The 2020 cohort of students’ readiness for and reception of online learning were evaluated using anonymous pre-course and post-course surveys. The pre-survey was administered in the first week of the quarter when the course began, and the post-course survey was administered in the last week of the quarter when the course ended. The survey was designed to gather data on student readiness and reception to online learning in three primary domains: transition to online learning (six questions); ease of use of the technology, including taking assessments remotely (three questions); and instructor interaction (three questions). The survey instrument was drafted by gathering items through a review of literature pertaining to distance and online education in the health care field.^12,21,23 The survey instrument underwent two iterations by three faculty members. A final optimized version of the instrument was used in the study. The Google link to the pre- and post-course survey instruments were sent out via emails sent to all students in the 2020 cohort. Additionally, the URL for the survey was made available on the announcement page of the learning management system (Canvas LMS) so students would be reminded to complete it when they logged in to Canvas. Reminder emails were sent a week after the pre- and post-survey administrations. Ten questions requiring Likert-scale responses were the same on both the pre- and post-survey, and one question was unique to each instrument, totaling 12 Likert-scale questions in all (Table 2). The post-course survey contained two additional questions intended to capture technology use by students (ie, type of device and type of internet access; Table 1). Responses to the Likert-scale items were assigned the following quantitative ratings: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree. A comment box was included at the end of both the pre- and post-survey instruments with instructions for students to enter any additional impressions regarding online learning that were not covered by the instrument.

Table 2.

Comparison of Pre- and Post-Course Survey Responses of First Year Pharmacy Students Enrolled in an Online Communications Course^a

Quantitative data are presented as means (SD). Comparisons of means between pre- and post-survey Likert-responses were performed using the Mann-Whitney U test. Paired analysis was not possible because the student responses were anonymized.^24,25 Comparisons of class grades between the two cohorts (summer 2019 vs summer 2020) were performed using the Student t test. Quantitative data analyses were performed using Excel, and statistical analyses were performed using GraphPad Prism, version 9 (www.graphpad.com). For all analyses, a p < .05 was considered statistically significant. Thematic analysis of qualitative data extracted from pre- and post-survey instruments was performed independently by two investigators using the constant comparison method reported in the literature.^26-29 Student comments were read several times to allow the investigators to become familiar with the data. Initial codes were generated manually by highlighting key text. Based on the context, the codes were grouped into categories. Some of these categories were collapsed or expanded as deemed necessary. Care was taken to ensure that the categories remained true to the opinions of the students. Finally, the categories were merged to generate overarching themes. Disagreements between investigators were resolved upon further discussion of the themes and achieving consensus. The data collection process was deemed to be exempt by the Institutional Review Board of the American University of Health Sciences.

RESULTS

Two different cohorts of students were enrolled in Professional Communications during the summer 2019 quarter (face-to-face instruction) and summer 2020 quarter (online instruction). The class size was 25 in 2019 and 32 in 2020. The distribution of sexes was approximately equal in both cohorts (Table 1). Most of the students in each cohort had a bachelor’s degree when they were admitted to the PharmD program and were in their first quarter of the pharmacy program. The average age of students in the 2019 cohort was 36 years, while the average of students in the 2020 cohort was 33 years. While the 2019 cohort did not use computers to receive in-classroom instruction, most students in the 2020 cohort used a personal laptop and a Wi-Fi internet connection to attend online lectures (Table 1).

Out of 32 students in the 2020 cohort, 31 completed the pre-course survey and 26 completed the post-course survey (Table 2). As described in the methods section, three domains of student reception were assessed: transition to online learning, ease of technology use, and instructor interaction. With respect to online learning, in the pre-course survey, student responses were mostly neutral regarding face-to-face instruction (M = 3.32, SD = 1.01). However, their preference for face-to-face instruction decreased by the end of the quarter (difference in the mean = −1.6; p < .001; Table 2). Although students anticipated that online learning would be as effective as face-to-face learning, their perception by the end of the quarter leaned more toward online learning being equally as effective (difference in the mean = +0.63; p = .023). Students also felt the course should continue to be offered online and indicated that their online learning experience met their expectations for the course (M = 4.38, SD = 0.89). With respect to technology, student level of comfort with attending lectures online (difference in the mean = +0.38; p = .03), and taking assessments remotely (difference in the mean = +0.57; p = .004) increased by the end of the quarter (Table 2). However, there was no change in student perception regarding ease of use of technology in online instruction (p = .09). With respect to instructor interactions, student level of comfort increased for posing questions virtually to the instructor (difference in the mean = +0.65; p = .02). However, there were no appreciable changes in students’ perception regarding rapport development with the instructor or believing that the instructor’s role was more important than the manner of course delivery (Table 2).

Eighteen students included responses in open-ended comment box at the end of the pre-course survey, and 17 students included responses in the comment box at the end of the post-course survey. Qualitative thematic analysis of the pre-course survey responses of the 2020 cohort generated themes of reluctance to engage in online learning and general nervousness about the course (Table 3). By the end of the quarter, student perceptions of online learning were more positive. Similar analysis of post-survey responses generated themes of overall acceptance to online learning, and consideration that online learning may be the preferred method of delivery for the Professional Communications course (Table 3). The mean grades on the midterm and final examinations for the 2019 cohort were 95.7 (SD = 5.45) and 84.6 (SD = 5.58), respectively. The interquartile range (IQR) for the midterm grades was 7 (Median = 98; confidence interval (CI) = 95.6%), and the IQR for the final grades was 8 (Median = 85, CI = 95.6%) for the 2019 cohort. The grades of the midterm and final examinations for the 2020 cohort were 94.3 (SD = 3.89) and 78.2 (SD = 11.7), respectively. The IQR for the midterm grades was 4.35 (Median = 95.6; CI = 97%), and the IQR for the final grades was 11.3 (Median = 79.1, CI = 97%) for the 2020 cohort. No significant differences in grades on the two major class assessments were found between the two cohorts (midterm, p = .26; final, p = .08; Figure 1).

Table 3.

Themes from Survey Responses of First Year Pharmacy Students Enrolled in an Online Communications Course^a

Figure 1.

Comparison of Examination Grades Using Box-and-Whisker Plot for the Cohorts in Face-to-Face (2019; N=25) and Online (2020; N=32) Learning in AS736 Course

DISCUSSION

A sudden surge in online teaching and learning occurred in 2020 as a result of the COVID-19 pandemic. While many publications aimed at educators about effectively managing the crisis and continuing instruction have been published since then, very little information is available concerning student readiness and reception to online learning amid this abrupt shift in instructional delivery. The present research is an effort to examine student readiness and monitor changes in reception toward completely online learning in a pharmacy course, along with comparison of student learning performance with that of a pre-pandemic, in-classroom cohort.

Overall, student expectations for the online communications course seem to have been met. Students admitted into the pharmacy program already had some prior exposure to online instruction (Table 2). Nonetheless, their preference for online education had grown (ie, preference for face-to-face instruction declined) by the end of the course. Prior association with online instruction and/or comfort with technology does not necessarily translate into enhanced learning. Cook and colleagues demonstrated only a modest correlation between prior online learning experience and knowledge outcomes in medical residents over a nine-year period.³⁰ Furthermore, there was no association between comfort with computers and enhanced learning in that study. Additional reports have indicated that, despite being technologically savvy, students may lack theoretical knowledge required for a particular profession.³¹ Moreover, student beliefs, expertise, and knowledge were all found to mutually influence one another in relation to student outcomes.³² The findings from the present study regarding preference for online learning resonate with those of a recent study, which captured student perceptions and performance in a blended asynchronous pharmacy course.³³ In that study, there was a decline in student preference for face-to-face instruction, and an increase in the favorability for online learning.

Instructors undoubtedly play a crucial role in student learning. Alsharif and colleagues demonstrated how instructor attitude, enthusiasm, and teaching style impact student learning.³⁴ Therefore, it was essential to investigate whether the students in our Professional Communications course could develop rapport with a remote instructor just as they would in a face-to-face classroom environment. It was reassuring to find that students did perceive developing a rapport with the instructor. In fact, students strongly believed at the beginning of the quarter that the instructor’s role was more important than the manner of delivery (face-to-face vs online). Hence, their post-course impression did not change for these two items on the instrument (Table 2). Furthermore, the present study only considered students’ perception of rapport and not an actual measure of rapport. It may be difficult to isolate the effects of an intervention on rapport development as time will always be a confounding factor. The greater the number of student-teacher interactions, the greater the rapport built, irrespective of the mode of contact. In the context of this discussion, it is worth noting a recent analysis of refocusing instructor presence in higher education during remote learning.³⁵ Three major instructor roles were emphasized: a cognitive role to take into consideration student preparedness and engagement in online learning experience; a social role to sustain and likely improve student-student and student-teacher interaction, and a facilitator role to directly integrate online tools/resources into teaching practices.

Thematic analysis of responses to the open-ended questions on the pre- and post-course survey instruments echoed the data from the corresponding Likert-scale item responses. While it is commonplace for students to be anxious before a transition to online learning, it was surprising to find reluctance to online learning in students’ pre-course evaluation data (Table 3). This likely arose from one or a combination of the following factors. First, students may have had unpleasant experiences with online learning in the past. Second, previous online learning experiences may not have been synchronous (the present study did not capture that information). Third, students may have felt that communication, being a pragmatic skill seemingly requiring in-person interactions to be learned, was not suited for online instruction. This hypothesis stems from the observation that about 15% of students noted along similar lines that “Communications is most important for all pharmacists” (Table 3).

Tarhini and colleagues reported that the effectiveness of e-learning is largely dependent on the end user’s degree of acceptance.³⁶ We noticed that, in the present study, students had developed an attitude of acceptance toward online learning by the end of the quarter. Furthermore, if students have the necessary technical resources to support e-Learning, then this may positively impact their learning experience. For this likely reason, Al-Balas and colleagues have emphasized the necessity of collaboration with technology services to provide students with valuable support for e-learning.³⁷ In the online course, technical assistance was provided to students by the university’s information technology department throughout the duration of the quarter. By the end of the quarter, students strongly supported the continuance of online education (Table 3). Several approaches to classroom learning during the pandemic have been proposed in the health care literature. The flipped classroom model,³⁸ online interactive lectures,³⁹ blend of traditional and online learning,³⁷ and asynchronous and synchronous remote delivery¹⁸ have been suggested. Flexibility afforded to students was also an important element noted in continuing to provide online education.⁴⁰

One of the objectives of this study was to compare learning outcomes from the online cohort with those of the in-class face-to-face cohort. While the extent of variability in the midterm and final examination grades for the 2019 cohort was similar (IQR, 7 vs 8), the variability in the midterm and final examination grades for the 2020 cohort was considerable (IQR, 4.35 vs 11.3) (Figure 1). The large variability in the final grade for the online cohort is attributed to two students scoring below 50% on the examination. Although the difference in outcomes was not significant (p = .075), there was an overall decrease in final examination performance by the 2020 online learning cohort compared to performance by the 2019 in-person learning cohort [78.2 (11.7) vs. 84.6 (5.58)] (Figure 1). More studies are warranted to conclusively determine whether online learning hinders student performance. Reports within the pharmacy literature depict mixed results in student performance when online learning was compared to traditional classroom learning prior to the pandemic. For instance, Al-Dahir and colleagues noted students performed better in a face-to-face problem-based module than in an online module.¹⁴ In contrast, Porter and colleagues found no differences in student performance between online and face-to-face delivery in an elective course.¹² In the present study, no significant differences were observed in student performance in the course across two cohorts. This implies that online learning did not negatively impact student learning performance in this communications course. As the two cohorts were similar in age and possessed identical credentials prior to entering the program, we believe they represent similar student characteristics and learning potential. The findings from the present study concur with the observations reported by He and colleagues in a systematic review of health care education on remote learning.⁴¹ Their analysis did not conclude significant differences in effectiveness between online and traditional face-to-face learning. Studies in other health care professions reported that students can gain knowledge via e-Learning or face-to-face instruction; however, the level of student reception for e-Learning was mixed.^42-44

There are several factors which are exclusive to this study in comparison to other investigations of online vs face-to-face learning. As proposed by Shalka, factors associated with uncommon events, such as those brought on by the COVID-19 pandemic, can increase stress in students.⁴⁵ Students may have also encountered a host of unprecedented psychological, social, and logistical problems in transitioning to online learning. Additionally, they may have had challenges maintaining concentration in learning because of prevalent safety concerns for themselves and others.⁴⁶ All these unique challenges set apart the students included in this research study from those in other studies of online learning conducted outside the context of a pandemic.

This study has some limitations. First, all evaluations were conducted in a single course in the program at one school. Nonetheless, since the communications course is required for first year students, their perceptions and learning outcomes could be extrapolated to other required courses delivered in the curriculum during the pandemic. This is because other required courses and their respective assessments were also administered remotely using the same teleconferencing technology and proctoring system as that for the communications course. Second, technical difficulties, if any, related to Proctorio may also have influenced students’ reception to the online method of instruction. Third, anxiety reported by some students in the pre-course survey may have had other causes besides online learning. As mentioned earlier, this was their first quarter in the program, and it is commonplace for students to be nervous at the beginning of a new program irrespective of the delivery platform. Fourth, the positive change in student reception toward online learning reported at the end of the quarter may have been influenced by experiences gathered outside of the communications course. Notably, the 2020 cohort was enrolled simultaneously in five other pharmacy courses, all delivered online, that may have had a positive effect in the changes in their perception of online learning. Finally, the class size was relatively small compared to the student cohorts in most other pharmacy schools and colleges, and represents a narrow student demographic.

CONCLUSION

First year PharmD students were partly prepared for online learning at the outset of a required communications course, with the remainder of their adjustment to online learning occurring as the quarter progressed. Student learning performance for the online cohort did not differ significantly from that of the in-class face-to-face cohort the year before. Online learning existed prior to the COVID-19 pandemic in health care professions and will likely remain after the crisis is over. Moving past the pandemic, educators and leadership at pharmacy schools and colleges may reassuringly continue to encourage and develop online instruction in their didactic curricula. The findings from this study may not only pertain to course delivery in the current pandemic, but also could be extrapolated to future e-Learning investigations. It remains to be determined how courses containing laboratory components and practice-based skills can be adapted for online learning. Future studies should also be directed at gathering faculty perception data including the challenges faced in providing online instruction to health care students.