Impact of Hybrid Delivery of Education on Student Academic Performance and the Student Experience

Abstract

Objectives

To compare student academic performance and the student experience in the first-year doctor of pharmacy (PharmD) program between the main and newly opened satellite campuses of the University of Maryland.

Methods

Student performance indicators including graded assessments, course averages, cumulative first-year grade point average (GPA), and introductory pharmacy practice experience (IPPE) evaluations were analyzed retrospectively. Student experience indicators were obtained via an online survey instrument and included involvement in student organizations; time-budgeting practices; and stress levels and their perceived effect on performance.

Results

Graded assessments, course averages, GPA, and IPPE evaluations were indistinguishable between campuses. Students' time allocation was not different between campuses, except for time spent attending class and watching lecture videos. There was no difference between students' stress levels at each campus.

Conclusions

The implementation of a satellite campus to expand pharmacy education yielded academic performance and student engagement comparable to those from traditional delivery methods.

INTRODUCTION

As of January 2008, over 5,700 chain community pharmacist positions remained unfilled in the United States.¹ This already large shortfall in the supply of community pharmacists is expected to rise, while the number of prescriptions dispensed is estimated to increase during the same time period. By 2020, the country will be deficient by an estimated 150,000 pharmacists.² To respond to this burgeoning demand, the University of Maryland School of Pharmacy made plans to increase its student population of 120 per professional year. Since the school had reached classroom capacity, initial plans were to build additional pharmacy-education space to accommodate more students at the main campus in Baltimore. However, this approach presented 2 road blocks: lack of appropriate funding to build the additional space and a 7-year lag time (including time for construction) before the expansion would result in an increased number of graduates. This approach would not address the immediate shortage; therefore, alternative measures were needed. The school decided to explore implementation of a satellite campus, which would increase enrollment by 33% if 40 additional students were enrolled per class for the first through fourth years (P1-P4).³ In August 2007, the first cohort of 40 students matriculated at the satellite campus in Rockville, MD. The opening of the satellite campus was met with keen interest by the Accreditation Council for Pharmacy Education (ACPE), which noted in its On-Site Evaluation Team Report that overall comparability between the programs should be maintained, particularly in the areas of curricular delivery and outcomes, and student services and satisfaction with these services. Thus, the objectives of this study were to compare student academic performance and student experiences between the main and satellite campuses.

METHODS

Background on Implemented Technology

Development of the new distance program involved many technological revisions. In order to minimize the need to hire a significant number of new faculty members at the satellite campus (3.5 faculty members and 1 staff person were located at the satellite campus during the first year), a hybrid delivery of the curriculum was chosen, utilizing both synchronous (videoconferencing) and asynchronous (recorded lectures available over the Internet) delivery of content, as well as live faculty-led small-group activities, office hours, and laboratories.

Asynchronous technology.

All required didactic material was delivered live and recorded at the main campus using Mediasite (Sonic Foundry, Madison, WI) as the streaming rich media platform. This material was subsequently posted to a secure server and made available to students at both campuses. To prepare for the asynchronous delivery of content to the satellite campus, the school outfitted 3 lecture rooms and a control room at the main campus, giving it the capability to record in 2 rooms simultaneously. To allow students at the satellite campus to plan time for content review, lectures were made available online no later than 6:00 pm on the day they were recorded.

Synchronous technology.

Live (synchronous) video conferences using Polycom (Polycom, Pleasanton, CA) and Tandberg (Tandberg, New York, NY) videoconferencing systems were used to facilitate student government meetings, elective courses, and class review sessions. Two classrooms at the main campus were outfitted for synchronous video conferencing between campuses. At the satellite campus, 5 classrooms were equipped with this technology. All video conferencing classrooms had the capacity to broadcast content using 1 or more cameras to capture students and instructor(s), microphones to record student communication, and a display of the remote facility or the content being broadcast.

Once connectivity between the campuses was established, training sessions were scheduled for faculty members and student leaders to orient them to the video conferencing equipment on the main campus. In-depth training was required for all full-time faculty members; guest instructors were briefed by faculty and staff members on the use of this delivery system and assisted with special arrangements. This provided individuals the opportunity to become comfortable with the broadcasting equipment at the main campus in order to deliver high-quality recorded content to the satellite campus. Faculty members and student leaders had the opportunity to visit the satellite campus to view the rooms available for synchronous activities. For each synchronous activity, at least 1 Internet technology (IT) staff member was present in the classroom on each campus to ensure technology was working properly.

Student Organizations

In order to encourage participation of satellite campus students in student organizations, an organizational gala was held at the main campus during orientation to bring together students from all years and promote professional and social organizations offered by the school. Satellite campus student participation in student organizations was achieved in a variety of ways, depending on student need and the regulations of the organizations. Many organizations held their regular meetings using synchronous technology. On occasion, however, organizations had nonsynchronous meetings but alternated the location of the meetings between the 2 campuses.

On their own initiative, students at the satellite campus created a new student organization, the DC Metro Student Pharmacist Association (DSPA), which would be open to students on both campuses but housed at the satellite campus. The students developed a constitution, elected officers, and received acceptance through the School of Pharmacy Student Government Association.

Student Academic Performance

Student performance indicators included graded assessments (examinations and quizzes for the 2007-2008 academic year), final course averages, cumulative first-year GPA, and IPPE evaluations completed by the preceptor evaluating the student.

Didactic assessments.

For each required course in the first-year curriculum, identical examinations and quizzes were administered throughout each semester simultaneously on both campuses and graded by the faculty members in Baltimore. During the first year, 43 examinations and quizzes were administered at each campus. Overall differences between campuses were tested for using repeated-measures ANOVA. Determination of differences between individual assessments was done using a 2-tailed independent-samples t test. Final averages for course-specific data (which included graded assignments such as homework, research papers, etc) were analyzed using one-way ANOVA. Cumulative grade point averages at the end of the first academic year were compared using an independent-samples t test. All assessment-related analyses were performed with SPSS, Version 16.0 (SPSS, Inc, Chicago, IL). Normality assumptions were met for 36 of the 43 data sets; the remaining 7 exhibited nonhomogeneous variance structure (Levene's test for homogeneity of variance, p < 0.05). For these cases, an approximate test of equality of means was performed.⁴ Corrections for multiple comparisons were not made, as our goal was to maximize our ability to determine whether academic performance differed between campuses.

Evaluation of IPPEs.

All P1 students in the 2007-2008 academic year took part in 3 IPPEs: a 1-week experience in the fall semester, a 1-week experience in the spring semester, and a 1-day experience in the spring. Typically, these experience occur in the community pharmacy and institutional (ie, inpatient hospital pharmacy) setting. At the end of each IPPE, students were evaluated by their preceptor. Parameters for the evaluation included: compliance with Health Insurance Portability and Accountability Act (HIPAA); demonstration of skills necessary to dispense medications and process medication orders; demonstration of decision-making skills; knowledge of generic and brand names and therapeutic classes for the top 50 prescription medications; and professionalism and behavior indicators. All 160 evaluations were tabulated and analyzed for differences between campuses. IPPE evaluations were analyzed using independent-samples t test.

The Student Experience

As part of the ACPE accreditation process and our own efforts to understand the student experience, including their concerns and points of view, a survey was conducted of all students at both campuses in April 2008. Administered via SurveyMonkey (Portland, OR, www.surveymonkey.com), the survey covered a broad range of topics, including time allocation, technical support services, professionalism and the school experience, curriculum, experiential learning, and academic and administrative support. Survey responses were based on a 4-point Likert scale. This report specifically analyzed involvement in student organizations; time-budgeting practices (hours per week spent attending class, watching lecture videos, studying, participating in school-sponsored activities, and working); and stress levels and their perceived effect on student performance.

To analyze the survey responses, we first determined whether the 2 campus populations represented a single homogeneous group by performing a nonparametric linear regression using DISTLM, version 5 (University of Auckland, Auckland, New Zealand) followed by a permutation-based ANOVA model that included terms for campus site, gender, and a site-by-gender interaction. We then tested for correlations between campus site and responses to specific survey questions with a chi-square analysis using SPSS.

RESULTS

Student Performance

Student academic performance, based on graded examinations and quizzes, was generally equivalent between campuses (main campus: 84.5 ± 1.0; satellite campus: 84.2 ± 1.5). The mean test score difference between campuses was 0.26%, which was not significantly different (p = 0.83). Of 43 individual examinations, only 1 was significantly different (p = 0.019). After Bonferroni adjustment, this value was nonsignificant. Means of individual course grades also did not differ significantly in any of the 9 required P1 courses (Table 1). Overall cumulative grade point averages between the 2 campuses were also closely matched (main campus: 3.32 ± 0.05; satellite campus: 3.22 ± 0.06; p = 0.242).

Table 1.

Comparison of Main and Satellite Students' Grades for all First-Year Required Courses

For all 3 IPPEs, students received uniformly high evaluation scores, and the average scores between campuses were not significantly different for any of the IPPEs (p = 0.837, 0.372, and 0.830, for scores on the first, second, and third IPPE, respectively).

Student Experience

To better understand the student perspective, we analyzed results of a student satisfaction survey that was developed internally and administered in April 2008. Respondents numbered 103 (of 122) and 29 (of 39) from the main and satellite campuses, respectively, for an overall response rate of 81% (slightly skewed toward the main campus).

Approximately 25% of students participated in student organizations. In the first year, 13 of 39 (33.3%) students at the satellite campus participated in 1 or more student organization(s) based at the main campus. Of the 13 students at the satellite campus, 7 pledged a fraternity at the main campus with the understanding that they would come to the main campus for chapter meetings and events. In its first year, 38 of the 39 students became members of DSPA, illustrating their sense of community and desire to serve the region in which they lived and went to school.

As part of the student satisfaction survey, students reported how much time they spent on the following activities: attending live lectures; watching lecture videos; other course work outside of class (eg, group projects); school-sponsored activities; and outside employment. Time categories of 0-5 hours per week, 5-10 hours per week, 10-20 hours per week, and 20-30 hours per week were used. Chi-square analysis showed no significant differences in time allocation, with the exception of time spent attending lectures vs. time spent viewing recorded lectures (Table 2). Time spent on student activities, as well as the number of organizations students were members of and active in, were not significantly different (Table 3).

Table 3.

Participation in Student Organizations Between Main and Satellite Campuses

ACPE standards require that school's assessments “include measurement of perceived stress in faculty, staff, and students, and evaluate the potential for a negative impact on programmatic outcomes and morale.”⁵ To that end, the student satisfaction survey asked students about the level of stress they were experiencing as well as how it affected their academic performance. Responses did not vary significantly between campuses (P = 0.265 and 0.865, respectively), with a modality reporting “constant but manageable” stress levels (77% and 75% at the main and satellite campus) and that the stress level “does not affect my performance or morale” (36% and 39%).

DISCUSSION

The ongoing shortage of pharmacists nationwide has generated the need to increase pharmacy training opportunities. A number of pharmacy schools have begun using a distance-education model, allowing for greater geographic coverage, use of cutting-edge technologies, and maximal use of existing facilities. Student performance in distance-based settings, which typically incorporate technologies that are relatively new to pharmacy programs, has been inconsistent. In some cases, distance campus-based students tend to achieve better grades than their counterparts.⁶ Others have reported significantly lower scores among distance-learning students.⁷ Still others have reported no difference in academic performance.⁸ In other educational fields, no consistent differences have been found in academic performance between traditional and distance education students.⁹ This inconsistency in student performance results underscores the need for continued research into the performance levels of students in contrasting academic settings and further investigation into the causes and consequences of student performance differences.

The University of Maryland attempted to standardize as much as possible the learning experience for its students at the main and satellite campuses. Thus, we implemented a hybrid model in which students at both campuses were able to take advantage of distance-learning technologies and provided ample opportunity for students at both campuses to interact with faculty members. This approach appears to have been successful thus far, as student performance is similar between campuses.

However, a number of factors may generate differences among student groups, such as age, prior work experience, and marital status. Because of the flexibility of the schedule, we anticipate that over the next several years the satellite campus may house students who are older (on average) than their main campus counterparts, as well as students who tend to work more hours outside of school than students on the main campus, which may in turn lead to differences in students allocation of time between campuses. Older students are potentially more mature and possess the time management skills needed to be successful in a distance program. However, if these students are also working more than their main-campus counterparts, they are likely minimizing their participation in nonmandatory activities on campus, which could affect their acquisition of other essential skills learned throughout pharmacy school (outside of academia), such as communication and interpersonal skills, professionalism, and teamwork. During the first year of the distance-education program, our school noticed a trend toward decreased attendance in nonmandatory activities (eg, case discussions and review sessions) at the satellite campus and in lectures at the main campus. How this will affect academic performance in the future is yet to be seen. Close monitoring of both academic indicators and performance on advanced pharmacy practice experiences (APPEs) will be needed.

CONCLUSION

Student academic performance was not significantly different between the main and satellite campuses at the University of Maryland School of Pharmacy. Although students from each campus allocated their time differently (eg, watching lectures online vs. attending class), this was expected given the format of curriculum delivery. Time spent on student activities and membership in student organizations was similar. Moving forward, soliciting continuous feedback from students and faculty members from both campuses will be a must with appropriate modifications made as necessary. At the University of Maryland School of Pharmacy, hybrid delivery of education has thus far been successful in providing learning experiences comparable to those provided in traditional pharmacy education settings.