Evaluation of an Injection Training and Certification Program for Pharmacy Students

Abstract

Objective. To evaluate an injection training and certification program for third-year (P3) pharmacy students, and to measure the impact of students’ administration of immunizations at an influenza clinic on their knowledge, skills, and competence in immunization.

Design. A repeated measures design was used to assess students’ injection skills across the injection training and certification program and the influenza clinic. A repeated measures design was also used to evaluate students’ self-reported knowledge, experience, and confidence.

Assessment. Postcertification and during influenza clinic comparisons showed significant improvement in students’ knowledge, experience, and confidence after taking part in the influenza clinic. University staff members and students indicated in a survey that they were satisfied with the clinic services provided by pharmacy students.

Conclusion. The injection training and certification program and the university influenza clinic were effective in enhancing and fostering student skills development.

INTRODUCTION

Five provinces (British Columbia, Alberta, Ontario, New Brunswick, and Nova Scotia) in Canada have granted pharmacists the authority to administer vaccines, and only pharmacists in Alberta can administer medications by injection. More and more pharmacists are incorporating this expanded scope into their practice.¹ However, only the University of Alberta and the University of British Columbia incorporate injection training into the pharmacy curriculum. The University of Alberta offers it as an elective program. As of 2009, approximately 38% of colleges and schools of pharmacy in the United States provided immunization education and training to pharmacy students as part of their core curricula, and other institutions offered such training as an elective course.² Bain and colleagues proposed that the low immunization rates in the United States could be mitigated by mandating immunization training and education as part of the core curricula in US colleges and schools of pharmacy.² In Alberta, the immunization rates are suboptimal,³ and Bain and colleagues’ call to action parallels the initiatives being undertaken by Canadian pharmacists to acquire injection training to provide vaccinations to the community at large.

There are few published works on the assessment of pharmacists or pharmacy students’ skills, knowledge, or confidence to administer medications by injection. In 2007, Turner and colleagues conducted a retrospective preceptor and student-based survey evaluating their experiences during a second-year vaccination course and community-based immunization clinics.⁴ Certified pharmacist preceptors rated the value of students administering immunizations 9.2 on a 10-point Likert scale (1=no value and 10=great value). In 2010, Pelly and colleagues conducted an assessment of the immunization education in Canadian nursing, medical, and pharmacy programs. They found wide variation in curricula between the different professions and between programs of the same discipline.⁵

In April 2010, the Alberta College of Pharmacists (ACP), a regulatory body governing pharmacists, pharmacy technicians, and pharmacies in Alberta, approved the University of Alberta Faculty of Pharmacy and Pharmaceutical Sciences’ proposed training and certification program to administer medications by injection. To enhance the knowledge, competence, and level of confidence of pharmacy students’ injection skills, the Faculty of Pharmacy and Pharmaceutical Sciences collaborated with the University Health Centre to include pharmacy students in the university’s annual influenza campaign. This learning opportunity was designed to move beyond theoretical instruction about injection and immunization and provide students hands-on experience in a clinical environment. Initial anecdotal pharmacy student feedback indicated that the program enhanced students’ understanding of how to effectively perform injections while providing an important service to the university community.⁶ While this initial evidence suggested the injection training program was effective, a more formal evaluation process was required before verifying these findings. To date, only 1 study has evaluated pharmacy students’ injection skills, and it was only through reflection of perceived improvement in skills by students and preceptors.⁷

Our primary objectives of the study were to evaluate the University of Alberta injection training and certification program and the impact that students’ participation in the University Health Centre Influenza Clinic had on their injection skills, knowledge, experience, and confidence. The secondary outcome was to evaluate patient satisfaction from the service provided by pharmacy students at the influenza clinic.

DESIGN

All P3 pharmacy students at our institution are trained to administer medications by injection. They are also offered the opportunity to sign up for injection certification in the second term of their third year as the certification component is not mandatory. All students who are certified to administer medications by injection are permitted to participate in the University Health Centre Influenza Clinic. Students who were certified in April 2012 and/or students who participated at the fall 2012 influenza clinic were invited to participate in the study. One hundred sixteen of 124 third-year pharmacy students received their injection certification in April 2012. Because not all pharmacy students at the University of Alberta were from Alberta and some provinces did not reciprocate the injection certificate from Alberta, some out-of-province students may have opted out of the certification. Eighty-six students volunteered to participate in the annual influenza clinic in fall 2012.

All staff members and students at the University of Alberta were offered free influenza vaccines at the University Health Centre Influenza Clinic in October and November 2012. Both pharmacy and nursing students had participated in the influenza clinic annually since 2010. In 2012, 3,699 staff members and students received an influenza vaccine. Those who received a vaccine from a pharmacy student were invited to participate in a satisfaction survey.

The University of Alberta Research and Ethics Board reviewed and approved the study in February 2012. Informed consent was obtained from all pharmacy student participants. To maintain anonymity, the digits of their address and last 4 digits of their cell phone number were used to match students to influenza clinic data.

EVALUATION AND ASSESSMENT

The impact of the injection training and influenza clinic participation on pharmacy students’ injection skills was assessed using a repeated measures design. In April 2012, P3 pharmacy students were provided with a 2-hour training session during their skills laboratory a week prior to certification. During the certification, students administered 3 supervised injections, 2 intramuscular and 1 subcutaneous, into the arms of a fellow classmate. Pharmacists approved as injection trainers by ACP used a newly developed survey instrument to rate students’ injection skills in 5 areas: vaccine preparation, aseptic technique, injection technique, disposal of sharps, and aftercare. These skills are the required competencies established by ACP to certify pharmacists to administer injections. Focusing on these skills contributed to the content validity of the tool (ie, relevance and representativeness).

Injection skills were rated in each of the skill categories according to the level of independence demonstrated by the students using a scale where 0=dependent, 1=assisted, 2=supervised, and 3=independent. The pharmacist raters were trained how to use the marking criteria to ensure consistency.

Students returned in November 2012 to participate in the University Health Centre Annual Influenza Campaign. The same set of skills of a random subset of students who took part in the injection certification and who participated in the influenza clinic were reassessed by at least 2 trained and randomly assigned pharmacist raters after the students had provided 1, 5, 10, 15, and 20 injections. Based on these data, the survey instrument demonstrated strong inter-rater reliability, with 68% of ratings in perfect agreement and 98% of ratings within 1 point on the score scale. Trends in the mean scores of student skills in each area were evaluated for signs of change over the duration of the program.

Table 1 shows the demographic characteristics of students who received certification and participated at the influenza clinic. The subset of matched cases appeared to be similar to the set of students who began the study at certification. Given that the subset of students did not differ from the certification sample on an unmeasured confounding variable, findings from the analysis of the smaller subset should generalize back to the larger sample of students at certification.

Table 1.

Demographics of Students Who Received Injection Certification and at Students Who Participated in the Influenza Clinic

Table 2 shows the mean rating for each injection skill measured at each injection milestone. Mean scores across all skills decreased between certification and the start of the clinic. During the clinic, skill scores increased from the first to the 20^th injection. This pattern of increase after each successive milestone (18 of 20 possible successive means showed an increase) was significant as indicated by a 2-tailed sign test (p=0.004).

Table 2.

Mean Injection Skills Scores as Rated by Pharmacist Trainers After April 2012 Certification Through Milestone Injections During an Influenza Clinic^a

The impact of the injection skills training and the influenza clinic on self-reported knowledge, experience, and confidence was also assessed using a repeated measures design with 2 instances of measurement. Students rated their level of knowledge, experience, and confidence after certification and again after their participation in the influenza clinic. Specifically, students rated their level of knowledge, experience, and confidence for 10 skills using a scale from 1 to 5 where 1=low and 5=high. Scores were calculated for 5 categories: communication (average of scores for 3 questions on the tool), obtaining consent, setting up an influenza clinic within a pharmacy, aseptic technique, and landmarking technique. The effect of the influenza clinic on student self-reported knowledge, experience, and skills was calculated by conducting dependent samples t tests of the difference between certification and influenza clinic mean scores.

Immediately following certification and prior to the participation of the influenza clinic, students’ self-reported ratings were close to the highest possible rating (ie, 5) in knowledge for obtaining consent (4.6) and landmarking (4.3), as well as in experience with aseptic technique (Table 3). Taking part in the influenza clinic did not result in significant changes in these skills.

Table 3.

Matched Pharmacy Student Self-Reported Outcome Means Precertification and Post Influenza Clinic^d

Alternatively, students’ self-reported knowledge of communicating effectively, setting up an influenza clinic, and aseptic technique increased significantly over the duration of the study (Table 2). Similarly, self-reported experience with communicating effectively, obtaining consent, setting up an influenza clinic, and landmarking increased significantly. Finally, self-reported confidence with all 5 skills increased significantly between certification and the completion of participating in the clinic.

Cross-sectional data from the patients served by the pharmacy students at the influenza clinic were used to evaluate the public perception and satisfaction. Pharmacy and nursing students administered 3,699 doses of the influenza vaccine in the 2012 influenza clinic. However, only 1,314 patients completed the pharmacy student satisfaction survey. Most of these patients (71%) were between the ages of 17 and 30 years, and 56% were female. Thirty-seven percent were not aware that pharmacists in Alberta were authorized to administer drugs by injection. When asked to rate the quality of service provided by the pharmacy students on a scale from 1 to 5 (1=very poor, 5=excellent), 99% (1302/1314) felt the service was very good or excellent, and 97% (1275/1314) agreed or strongly agreed they were willing to receive vaccines from a pharmacist in the future.

DISCUSSION

The influenza clinic had a positive impact on the students’ self-reported knowledge, experience, and confidence in administering vaccinations. However, some of the students’ skills were lost between the time of certification and the start of the influenza clinic. There is some empirical evidence to suggest that across the 5 injection skills assessed, participating in the clinic resulted in improved skills after repetitive administrations. The influenza clinic provided students with more practice administering vaccines to patients in a practical setting, which they otherwise may not have been able to gain until after graduation. The findings in the study provided evidence that, in general, the flu clinic experience improved student injection skills.

The influenza clinic failed to increase students’ knowledge of most skills as indicated by the nonsignificant differences between certification and influenza clinic scores. However, there was a significant improvement in experience and confidence ratings for the same skills. Finally, the results from the patient satisfaction survey were similar to the previous year in that patients were generally very satisfied.⁸

While the findings from the study support the inference of a positive effect of participating in an influenza clinic on skills and self-reported knowledge, experience, and confidence, some limitations were acknowledged. Our results are not expected to generalize beyond groups of students who have similar characteristics to the ones who took part in this study. Nonetheless, the pharmacy students in our study were not expected to differ dramatically from most pharmacy students in the final year of their bachelor of science in pharmacy degree. Therefore, our findings likely apply to most programs that offer similar types of opportunities for injection skills training.

Even though our survey instrument has not been fully validated. It has achieved some level of content validity because of how it was developed (the skills assessed were based on the ACP list of injection competencies).

With respect to the methods employed, pharmacist raters were not blinded to the number of injections students had completed. As a result, there is the potential bias that pharmacist raters expected students’ injection skills to improve with practice. An improved design would be to blind raters to the number of injections students administered to eliminate potential bias.

The study used a repeated measures quasi-experimental design that facilitated the observation of self-reported and pharmacists-rated skills over time. Because this design does not involve a control group, alternative explanations for the reported results may exist. That said, it seems highly unlikely that a variable other than experience gained while taking part in the influenza clinic is responsible for the observed increases.

SUMMARY

The injection training and certification program at the University of Alberta was effective in preparing pharmacy students to administer drugs by injection. Furthermore, the University Health Centre Annual Influenza Campaign was successful in having a positive impact on students’ knowledge, experience, and confidence in administering drugs by injection. Overall, patients who received a vaccine from a pharmacy student at the clinic were satisfied with the service and were willing to receive a vaccine from a pharmacist in the future.