Web-based Multimedia Vignettes in Advanced Community Pharmacy Practice Experiences

Abstract

Objectives

To evaluate the effectiveness of Web-based multimedia vignettes on complex drug administration techniques to augment the training of pharmacy students in advanced community pharmacy practice experiences.

Design

During the orientation for a community APPE, students were randomly assigned to either a study group or control group After they began their APPE, students in the study group were given an Internet address to access multimedia vignettes which they were required to watch to augment their training and standardize their counseling of patients in the use of inhalers and ear and eye drops.

Assessment

A 12-item questionnaire was administered to students in both groups at the orientation and again on the last day of the APPE to evaluate their knowledge of counseling patients in the use of inhalers and ear and eye drops. The control group did not experience any improvement in their counseling knowledge of the research topics during their month-long experience. Students in the intervention group scored higher on their postintervention test than students in the control group (p < 0.001).

Conclusions

Student learning outcomes from experiential training can be improved through the use of Web-based multimedia instructional vignettes.

INTRODUCTION

High volumes of ear and eye drops and dry powder and metered-dose inhalers are prescribed and dispensed annually.¹ Because of the sheer number of prescriptions written annually for these medications and the concomitant importance of their proper administration, the verifiable and consistent training of pharmacy students in administration and patient counseling techniques for these medications is essential. Studies of eye and ear drop administration^2,3 and inhaler and dry powder administration techniques⁴ suggest that using Web-based educational modules to support the training of pharmacy students completing APPEs in diverse locations may be effective.

This study was an opportunity to identify ways in which technology could be used to improve the quality and consistency of experiential training in the College of Pharmacy at the University of Arkansas for Medical Sciences. Highly accessible interactive Web-based multimedia training vignettes presented by experts have the potential to improve student training and subsequent patient care. Within health education, a meta-analysis of Internet-based learning compared with no intervention demonstrated the usefulness of this technology.⁵ Computer-aided learning (CAL) has been used for teaching aseptic technique,⁶ case history taking⁷, communications,⁸ mentorship of preceptors,⁹ biotechnology,¹⁰ pharmaceutics,¹¹ pharmacy administration,¹² critical-thinking skills,¹³ literature evaluation,¹⁴ continuing education,¹⁵ teaching the top 200 prescribed drugs,¹⁶ pharmacokinetics,¹⁷ and pharmacology.¹⁸ CAL is also effectively used in pharmacy practice for both economic and availability purposes.¹⁹ Given the success of other applications of this technology, the authors anticipated that a similar delivery system would be helpful in addressing needs related to the training of fourth-year pharmacy students in community APPEs. These students are located across the state at several volunteer training sites. Further, the instruction that these students receive from preceptors depends on a number of factors that vary from store to store and change from day to day, including the assertiveness of the students, the amount and quality of interactions between the student and preceptor, the preceptor's/pharmacist's knowledge²⁰ and teaching skills, and the availability of quality training time secondary to managing a heavy prescription volume. Using computer-based multimedia instructional vignettes featuring experts in the field to teach methods and techniques for complex medication administration in a consistent manner to students in experiential settings holds promise. Other colleges may be interested in the success of our experience to assess, convey, use, or collaborate on similar Web-based training initiatives.

The objectives of this study were: (1) to evaluate the adequacy of the first 3 years of the pharmacy curriculum in teaching pharmacy students about complex administration techniques for inhalers and eye and ear drops; (2) to evaluate the effectiveness of a series of Web-based learning vignettes to augment students' advanced community practice experiences, resulting in improved learning outcomes (test scores); and (3) to gather preliminary evidence on the potential for implementing similar Web-based, multimedia tools as part of the APPE curriculum.

DESIGN

Development and Production of Vignettes

After identifying likely opportunities for successful computer-aided instruction (medication administration techniques), we determined the type and format of computer-delivered training needed. The study team agreed that any tools developed would have to address several critical issues. The broad geographic dispersion of the students (statewide) would require near-universal accessibility by Web-enabled computers. Integration of audio, video, and text instruction to accommodate and reinforce a range of learning styles was also important. A multimedia approach would allow the user the flexibility to use the tool in various settings, eg, using only the written and video portions of the vignette at times when the audio portion would be disruptive. To be an effective onsite resource, “on demand” delivery also was crucial so that students could access the instruction at convenient times and on an as-needed basis. Further, by virtue of the ubiquitous nature of the Internet, this resource material would be readily available for the students during their practice experiences.

Multimedia vignettes were developed for the following 5 topics: dry powder inhaler administration; metered-dose inhaler administration; eye-drop administration technique; ear-drop administration technique for adults; and ear-drop administration technique for infants. Software used to produce the vignettes included PowerPoint for slide show development, Impatica, version 3.0.1(Impatica Inc., Ottawa, Ontario, Canada) for compression, and Impatica OnCue for Internet delivery. Impatica OnCue allows for synchronization of Impaticized Microsoft PowerPoint, highly compressed AVI (audio video interleave) digital video and audio files, with simultaneous synchronized scrolling text (similar to the delivery of lyrics in a karaoke system). Additional features included a searchable text function and a control panel with dynamic indexing and navigation for replays of individual synchronized slides. The modules allowed viewers to select a band width delivery and corresponding degree of compression to match their Internet connection speed. The vignettes were developed prior to the study initiation and stored on one of the college's Web servers, as well as on CDs and DVDs for computer-based delivery. The training clips were 1:31 to 3:51 minutes long, which is considered ideal for Web-based “on demand” training. These vignettes were reviewed and assessed for content validity (eg, accuracy, completeness) by local faculty experts responsible for instruction of these topics within the curriculum and revised and edited based on their comments and suggestions.^1,20

Development of the Test Instrument

Questions to assess students' knowledge of dry powder and metered-dose inhaler administration techniques and eye-drop administration techniques were developed. The test instrument was reviewed for content validity by faculty members with expertise in the subject areas, and revisions were made based on their suggestions.

Before the study began, a beta test was conducted using a few faculty members to test the instructions, treatment manipulations, and computer projected questionnaire. Feedback was used to modify the test instrument and improve parts of the study design.

Implementation of Vignettes

Subjects were 79 fourth-year students completing their community APPE (a degree requirement). During online orientation for the community APPE, students were asked to participate in a study assessing their knowledge of administration techniques for inhalers and eye and ear drops. All students agreed to participate and completed a 12-item preintervention test. No further information about the study was provided at that time. Students were then randomly assigned to a pharmacy. Half of the practice sites were selected to incorporate the vignettes both as required viewing and as a resource available to students (students at these sites were assigned to the study group), while the other half of the sites did not make the vignettes available or inform their students about them at all (students at these sites were assigned to the control group). At the beginning of the second series of APPEs (month 2), the status of the participating sites was reversed so that all those sites that used the vignettes during month 1 did not use them in month 2. This was intended to ensure the influence of outstanding APPE preceptors at each of the sites did not influence the study results.

The 1-month APPE consisted of students practicing at a community pharmacy under the guidance of a UAMS faculty member off site (via e-mail communication) and a preceptor onsite. During the APPE, students completed specific assignments developed to expose them to critical aspects of patient care in a community pharmacy. At specific points during the month-long APPE, one of the UAMS coordinators would send an e-mail to the students in the study group with a link to a specific vignette and instructions for viewing it and applying the new knowledge. The purpose of the vignettes was twofold: to augment and extend prior learning and to maintain faculty involvement in the learning process of these widely dispersed students. On the last day of the APPE, the students returned to campus for debriefing, as well as presentations of projects and demonstration of skills learned. All participants also were asked to complete a postintervention survey at this time.

This study was reviewed and approved by UAMS Institutional Review Board (IRB).

EVALUATION AND ASSESSMENT

All students enrolled in the course (79) agreed to participate in the study and all completed their community APPE and were included in the analysis. Students who completed a second community APPE as an elective in month 2 were not included in the study the second time. Means and standard deviations were calculated for each scale and each item. Changes in test scores were compared between students who were required to watch the vignettes and had access to them as resources and students who did not.

The 12-question preintervention and postintervention tests were scored, with 1 point given for each correct response so that the highest possible score was 12. Test scores were analyzed using independent t tests at the 0.001 level of significance. Preintervention test results for study and control groups were comparable, with average scores of 6.1 and 6.4, respectively. A two-tailed t test revealed no significant difference (p > 0.05) among the averages of the 2 groups at the preintervention stage. After the month-long APPE, however, the average test scores of those who used the learning vignettes increased notably over those of the control group. The average score of students in the control group was 6.6 while the study group's average score was 8.9, a difference of 2.3 points on a 12-point scale (p < 0.001).

DISCUSSION

The first objective of this study was to evaluate the information that students acquired during the first 3 years of the pharmacy curriculum or through pharmacy work experiences outside of the PharmD program. Students often work part-time in pharmacies prior to or after entering the PharmD program, but the quality of the experience and training they receive in these positions varies widely and is difficult to assess. Low scores on the preintervention test suggests that the didactic portion of the pharmacy curriculum and external work experiences alone do not result in a full understanding of the complex techniques surrounding the correct use of inhalers and administration of ear and eye drops. Low scores would also add weight to the argument that on-demand, Web-based vignettes could play a useful role in preparing students to practice pharmacy/pharmaceutical care.

The second objective was to evaluate the effectiveness of using computer-aided learning to augment and extend the teaching by faculty members overseeing these advanced community practice experiences. The vignettes were effective in increasing student knowledge of administration techniques. To determine whether this objective was met, the status quo of learning (without vignettes) was observed. There was not a significant difference in test scores between pretest and posttest in these control groups. However, increases in the students' administration counseling skills secondary to the added resource of computer-aided instruction within an experiential drug store setting were observed. The authors demonstrated improvement between the study and control students' pretest and posttest knowledge in most administration technique questions studied.

The third objective was to gather preliminary evidence on the efficacy of making similar Web-based, multimedia tools a part of the community APPE curriculum. Two faculty coordinators oversee student learning in these multiple APPE settings across the state. These APPE faculty coordinators were interviewed for their perceptions on the ability and promise of Web-based technology/multimedia to provide consistent and accessible training to APPE students. Two of the faculty coordinators who collaborated on this study considered the vignettes an effective way to augment students' experiential training. Specific comments included: “the vignettes allowed us to provide a level of quality and standardization for experiences across the state that had not been possible before,” “student learning was consistent regardless of the site and preceptor variations,” “patients ultimately benefited through the increased knowledge of the students,” and “as a faculty preceptor I believe the vignettes are a extremely valuable to our students,” saying that the vignettes provided a standardized learning experience that would minimize differences in preceptors' training abilities, knowledge base, availability to students, and interest in teaching students. Also, beyond the educational benefits of the vignettes, they enhanced the students' sense of being connected to the college, reducing their sense of isolation and anxiety. These outcomes are especially important to the college, since it is challenging to stay connected and in communication with students during their APPEs. Although not intended to be part of the original study, anecdotal feedback during student debriefing included positive comments: “excellent resource” and “helpful to have an advanced instructor resource that we can use at the actual clerkship site.”

The use of multimedia computer presentations to efficiently serve as a primary or “booster” resource to augment student knowledge is an important area of study. With the efficiency of these vignettes, the authors demonstrated significant opportunities to use Internet-based multimedia resources in a number of scenarios. Although the authors do not advocate the use of these vignettes to replace traditional instruction, vignettes have the potential to extend instruction and require further study. With students missing class, losing concentration in the classroom settings, and simply not understanding the topic the first time around, these vignettes could serve as a resource for specific topics within traditional instruction. Further, these vignettes by nature of being located on the Internet are available to students with Internet access during their practice experiences. Alternatively, the vignettes can be placed on a CD or DVD for the student to carry to their practice sites.

Colleges of pharmacy recognize the importance of teaching computer skills and technological advancements in pharmacy practice. In fact, technology education is an accreditation skill required by the Accreditation Council for Pharmacy Education (ACPE).²¹ The use of computers to teach general or specialized practice skills varies by college.²² Although ACPE recognizes the importance of technological training of students for practice, colleges of pharmacy have been slow to develop technological advancements in pharmacy instruction. The dearth of various forms of computer-aided learning in pharmacy curriculums leaves considerable room for development, deployment, and study. Perhaps only through collaborative efforts between colleges of pharmacy or via sufficient grants will a compendium of quality computer-based training materials be available for incorporation throughout our curriculums. Whether the perceived academic value of CAL by faculty members is a perception or reality should not delay the study of the effectiveness of CAL in traditional classroom or experiential instruction but rather should spur research into the topic.

The effectiveness of using ultra-short multimedia vignettes can also be studied for use in training other health professionals or for the direct training of patients. The community pharmacist is the most accessible and trusted source of patient/consumer information on medicines, prescriptions, medical devices, and drug administration techniques. The relationship that the medical consumer develops with his or her pharmacist is typically one imbued with great trust, and is one of the most valuable relationships in the health/medical continuum of care. The limits to taking full advantage of this relationship to significantly and positively improve public health, however, is the vast diversity in both the pharmacy settings and individual pharmacists in practice. Too little is known about the knowledge base of practicing pharmacists with respect to administration techniques, and even less about their ability to communicate effectively, and the time and opportunity to counsel patients in the course of routine interaction. With more study on both, appropriate educational resources and tools (and the appropriate methods to deliver them) could be developed. Because of the limits of pharmacist-delivered instructions, on-demand vignettes may have considerable potential for augmenting pharmacist counseling.

Although the average score of students in the control group and the study group was significantly different by 2.3 points (p < 0.001) and therefore generalizable to the larger student population (see graph one), there were some limitations to the study. One large group of students were studied with a specified set of APPE sites. The study design only allowed less than a month between exposure to the Web-based learning vignettes and the postintervention test. A study period longer than 1 month showing the effectiveness of the videos would be ideal but not practical within the constraints of the typical experiential year. Another limitation is that the preintervention and postintervention tests assessed knowledge acquisition and not administration skills. Due to the extensive content covered during the debriefing day, there was only time to administer the knowledge postintervention test covering this material. Ideally, an objectively structured clinical examination (OSCE) of learned skills would have been administered. In addition to skills testing, an OSCE would also evaluate the students' ability to communicate the newly acquired knowledge.

SUMMARY

The effectiveness of using CAL to efficiently and effectively serve as an adjunct to instructional training methods in the community APPEs was studied. Statistical demonstration of increased counseling knowledge was established. Additionally, the college-based community APPE coordinators' perceived the value of CAL to maintain a connectedness to the students completing APPEs across the state is important. With variable learning experiences between experiential sites, tools that provide standardization of learning among sites and experiences encountered by all students is of significant importance.

The study findings indicate the potential of ultra-short Internet-based vignettes in training students over a short period of time in real world practice settings. Other vignettes could be developed and studied for a variety of skill-based topics (eg, various physical assessment procedures, or other specialized patient counseling techniques such as birth control devices, insulin injection, and glucometers) that would be useful in experiential training.