Abstract
Background and purpose
Due to the COVID-19 pandemic, students at the University of Waterloo in spring 2020 enrolled in the injections training curriculum were unable to complete the practical assessment component under the usual in-person model. Therefore, an alternative assessment strategy needed to be adopted to meet these curricular outcomes.
Educational activity and setting
To allow students to complete their training and apply this skill during co-operative work placements in fall 2020, pharmacist supervisors (preceptors) who were authorized to administer injections were asked to evaluate the practical assessment on behalf of the university. Students were mailed supplies to use for practice and assessment, and preceptors were provided the grading rubric and a copy of the didactic training materials for their reference. To obtain feedback on the process and identify areas for improvement, students and supervisors were invited to complete a brief survey containing both Likert scale and open-ended questions upon completion of the assessment.
Findings
By fall 2020 term end, 69 of 121 students successfully completed the practical assessment component at a workplace. Survey responses indicated that, despite some challenges accommodating the assessment within a busy pharmacy's existing workflow and identifying volunteers to receive the injections, the modified assessment was well received.
Summary
Supervisors can be effective adjuncts to in-class instruction and assessment of injection technique. Even when initial assessments can take place at the university, providing supervisors with access to training materials and rubrics can reinforce these skills for students immediately prior to their implementation into practice.
Keywords: Vaccination, Student, Pharmacy, Assessment
Background and purpose
In Ontario, Canada, pharmacists, pharmacy students, and interns with authorization to administer injections are permitted to administer immunizations to patients two years and older against influenza, and patients five years and older against 13 other vaccine-preventable diseases, including pneumococcal disease and herpes zoster.1 , 2 Authorization to administer injections is dependent on the successful completion of a recognized injections training program that addresses the 14 Immunization Competencies for Health Professionals specified by the Public Health Agency of Canada, as well as an additional competency specified by the National Association of Pharmacy Regulatory Authorities.3 , 4 Uptake of pharmacist-administered immunizations has been high, with Ontario pharmacists administering over 1.4 million influenza vaccinations during the 2019–2020 influenza season (personal correspondence, Ontario Pharmacists Association). Across Canada, pharmacies are the leading location for patients to receive the influenza vaccine, administering 40% of all influenza vaccines in the 2019–2020 influenza season.5
At the University of Waterloo, pharmacy students complete a year-round co-operative education curriculum. In their second and third years, students alternate between academic terms and experiential education work terms across a variety of practice settings.6 Because students complete their second work term in the fall semester, coinciding with the peak influenza vaccination period of October and November,5 , 6 students complete their training on the administration of injections just prior (in the spring term of their second year). This benefits their training by allowing them the opportunity to participate in administering influenza immunizations the following term (while on placement) and increases immunization capacity for the community pharmacies where they are completing their work terms. The injection training curriculum consists of both lectures and online self-study modules. To receive their authorization to administer injections, students must pass a written examination as well as a practical assessment of their injection technique. This practical assessment consists of the administration of a subcutaneous (SC) and an intramuscular (IM) injection of normal saline on a peer, and is evaluated by an injections-trained pharmacist for draw-up technique, landmarking, and proper administration. While multiple attempts are allowed, this has not historically been required for students to be successful.
On 11 March 2020, the World Health Organization declared a COVID-19 pandemic.7 Shortly after, the University of Waterloo School of Pharmacy (SOP) halted all in-person instruction, transitioning students to online learning for the remainder of the winter term and for all of the spring term (in which the injections training would usually be completed). This caused concern for our community pharmacy work term sites who were anticipating significantly increased interest from the public for influenza immunization and were depending on student participation to meet patient demand. As students could no longer complete the practical assessment of their injection training in person at the university, alternatives to complete this assessment were explored. While remote assessment by video was considered, there were concerns that video assessment may be difficult to perform due to limited visibility of the arm during the injections. Ultimately, faculty opted to have students' experiential education supervisors (also known as preceptors) serve as their in-person assessors, as fall term work terms were still proceeding under COVID-19 precautions.
Correspondence with other schools of pharmacy (SOP) across Canada indicated that no other programs were considering a similar approach, and a literature search was unable to identify any other instances of experiential education supervisor-led assessments of injection technique for credentialing purposes. While it is recognized that direct and indirect supervision of students administering injections would be an expected component of experiential education, students have generally completed all aspects of their formal injections training curriculum prior to this supervision while on placement. The aim of this article is to describe our program modification and feedback from students and supervisors, should other programs wish to adopt a similar model for offsite practical assessment as part of their injections training curriculum.
Educational activity and setting
Various faculty and staff members were involved in the transition of the injections training practical assessment to a workplace-based model, including the faculty responsible for coordinating the injections training program, the experiential education team, the associate director of curriculum, and the school's programmatic assessment officer. It was also confirmed with the Ontario pharmacy regulatory body responsible for licensing that this model of assessment would be recognized.
Since not all students were completing placements in settings where such an assessment could take place (e.g., those within the pharmaceutical industry, professional associations, or research positions), participation in the modified assessment was optional. Students who were unable to complete their practical assessment at a workplace would be offered the opportunity to complete it at the institution once in-person instruction was permitted. A dedicated email account was created specifically for communications related to injections training, which was monitored by administrative staff and faculty. This email was used to inform students of the option to complete their practical assessment at their workplace. Interested students were also asked to have their supervisor contact the SOP at that email address to express an interest in serving as an assessor and confirm that they held the qualification to do so (i.e. being a practicing pharmacist with current authorization to administer injections). Having a dedicated email address for this correspondence streamlined the process, allowing us to more easily track student/pharmacist pairs who were participating in the workplace-based assessment, provide digital materials required to complete the assessment, and issue certificates of completion to students.
Supervisors (preceptors) planning to serve as assessors were provided, via email, a document containing the contents of the students' online module on administration of vaccines. This module included instruction on the pre-injection patient assessment, injection preparation (e.g., selection of the correct needle and syringe, reconstitution), SC and IM injection technique, safe disposal of sharps and other supplies, and post-injection patient monitoring. The intent of providing this information to supervisors was two-fold: (1) To serve as a refresher to the supervisor on injection administration technique for their own use; and (2) To ensure that supervisors were aware of what students were taught in order to address any questions and perform the assessment in a way that aligned with the university's expectations. Supervisors were also provided a copy of the grading rubric used by the program (eAppendix 1), and were asked to use this rubric in their assessment. At the bottom of this rubric an area was added for the supervisor to add their name and license number so that the university could confirm they held current authorization to administer injections using the public register of pharmacists in their province of practice.
All students were mailed a bag of supplies (contents provided in Table 1 ) to be used for the assessment. Instructions on the use of the supplies and the performance of the practical assessment are provided in Table 2 . Supplies were provided to allow students to practice on an inanimate object prior to doing their assessment injections. Students and assessors were also provided with a waiver of liability document to be reviewed and signed by the individual receiving the injections from the student during the assessment (eAppendix 2).
Table 1.
List of supplies provided to students for practice injections and practical assessment.
| Supplies | Quantity |
|---|---|
| 3 mL Luer-Lok syringes | 4 |
| 5/8-in. 25-gauge safety needle | 2 |
| 1-in. 25-gauge safety needle | 2 |
| 10 mL vials 0.9% NaCl (isotonic saline preservative free) | 2 |
| Alcohol swabs | 4 |
| Cotton balls | 2 |
| Adhesive bandages | 2 |
mL = milliliter; NaCl = sodium chloride.
Table 2.
Instructions for practice and completion of practical assessment.
|
IM = intramuscular; kg = kilogram; mL = milliliter; PPE = personal protective equipment; SC = subcutaneous.
As this was the first instance, to our knowledge, of experiential education supervisors (preceptors) being equipped to serve as injection technique assessors for the purposes of injection training credentialing, a research component for quality improvement purposes was embedded. Ethics approval was obtained from the Office of Research Ethics at the University of Waterloo to survey students and supervisors on their experience with this model and obtain feedback on areas for improvement should a similar approach be used again. An invitation to participate in the study and a link to access the survey, hosted by Qualtrics (Qualtrics, LLC) was added to the assessment rubric.
The questionnaire consisted of a mix of Likert and open-ended questions (eAppendix 3), and was designed to be brief to encourage completion in a busy workplace environment. The Likert scale question responses were coded as 1 = strongly disagree, 2 = somewhat disagree, 3 = neither agree nor disagree, 4 = somewhat agree, and 5 = strongly agree. Students and supervisors completed different questionnaires to capture their specific perspective as either learners or assessors; however, the intent of questions remained consistent across both versions to allow for comparison of responses. Questions examined the adequacy of supplies provided to students, students' preparedness to complete the practical, adequacy of communication received from the university on expectations for completing the assessment, barriers that facilitators faced when performing the assessment in practice, and benefits and limitations of offering the assessment under this adapted model. Completion of the questionnaire by students and supervisors was voluntary and did not impact students' academic status in any way. Means and SDs were calculated for Likert questions, with a two-sample t-test performed to examine differences between student and supervisor responses using IBM SPSS 2020 (IBM, Corp.). Open-ended question responses were grouped into key themes by two researchers. The research team was also interested in capturing the proportion of students who were able to complete their practical assessment under this model as a proxy for feasibility.
Findings
As of the end of fall 2020, 69 of 121 students in the cohort successfully completed their practical assessment in a workplace environment. Information was not collected from those who did not complete the practical about their reason for not completing it (i.e. lack of interest or unavailability of a suitable practice site or assessor). To our knowledge, none of the 69 students completing the assessment were unsuccessful on their first attempt. Of the 138 individuals with the potential to complete the questionnaire (69 students and their supervisors), responses to the survey were received from 31 students and 18 supervisors for response rates of 44.9% and 26.1%, respectively. Of these responses, 43 indicated their place of practice as community pharmacy, 2 as hospital, and 1 each as outpatient pharmacy, specialty pharmacy, and endocrinology group.
Responses to the 3 Likert scale questions are summarized in Table 3 . Overall, each item averaged near the “somewhat agree” level of the scale, with students rating each item lower than supervisors, and a statistically significant difference noted for the item on the adequacy of injection training material and supplies provided. Open-ended question responses indicated that students wished to be provided with additional needles and syringes for practice as well as an injection pad, and often sought additional videos online demonstrating proper injection technique prior to completing the assessment. One student noted that it would be helpful to use a more realistic material: “A material to use that mimics flesh/muscle. We used fruit but it did not feel like the real thing.”
Table 3.
Responses to Likert scale questions on experiences related to completing a workplace-based injection practical assessment.
| Item | Student responses, n = 31 Mean (SD) | Supervisor responses, n = 18 Mean (SD) | P for difference between groups |
|---|---|---|---|
| The injection training material and supplies helped me/my student complete the injection practical. | 3.97 (1.54) | 4.83 (0.38) | .02 |
| I/my student felt prepared and ready to complete the injection practical. | 4.03 (1.14) | 4.61 (0.78) | .06 |
| The School of Pharmacy clearly communicated expectations and instructions related to the injection training. | 4.35 (0.98) | 4.56 (1.15) | .50 |
Students' level of preparedness appeared to vary markedly, potentially impacted by the level of instruction provided by the supervisor. As described by one student: “I felt very prepared, but mostly because I practiced a lot, and the pharmacists I worked with were extremely helpful (i.e. to answer questions, correct my technique, helpful tips, allowed me to practice in a more realistic setting - I had an injection pad and the pharmacist even strapped it to their arm to emulate an actual situation).”
Other students would have preferred a longer time period to prepare: “I was decently comfortable doing the practical but would have appreciated at least a couple other days of introduction and injections practice. With the supplies given this was not really feasible so I had to settle for doing my first injections on fruit to a person within a couple hours. This did not faze me too much, but I think other students would benefit from as much prior practice as they could get.” Additional key themes identified from the open-ended responses are summarized below:
Flexibility and convenience
Both students and supervisors appreciated the flexibility of being able to complete the assessment at their preferred time. Both students and supervisors felt this modified assessment was a suitable alternative to an in-person assessment at the university, citing that it provided:
-
•“Freedom to achieve certification at any time and convenience with practicing as well.”(Student)
-
•“Easily accessible during a period when they cannot be in school.”(Supervisor)
Challenges with identifying individuals to receive injection
Both students and supervisors commented that it was challenging to identify a person to whom the student could administer the injection, with some ultimately injecting their supervisor which appeared to either increase or alleviate nervousness for the student.
-
•“Difficult to find a volunteer as individuals were nervous about being injected by a student.”(Student)
-
•“It is a little bit more anxiety-provoking for students to practice on live “patients“ instead of their classmates.”(Supervisor)
-
•“At first very intimidating, especially doing the injection on my supervisor. He was very nice about it, encouraging, and reassured me I wouldn't hurt him.”(Student)
Additional learning opportunities associated with completing assessment within an active practice setting
Students recognized the benefits of having the opportunity to ask questions of their supervisor before and during the assessment, and to observe the supervisor administering injections in practice before completing their practical assessment.
-
•“The pharmacist gave me personal tips and tricks which did not necessarily exactly follow the steps listed [on the rubric], but her own methods in order to protect herself from being hurt/accidentally poked by the needle.”(Student)
-
•“Since I have done multiple injections (over 1000) it was good to be able to share practical knowledge and real-life experiences of what to expect.”(Supervisor)
Workflow challenges and potential for inconsistency in instruction across practice sites
Numerous students and supervisors commented on the challenges with integrating the assessment within the existing workflow of a busy pharmacy. Additionally, concerns were raised about varying quality of supervision that may be offered across workplaces.
-
•“Lack of staffing made it difficult to do a full review with the student of the training material. Being the only pharmacist on duty it was difficult to give full attention to the student.”(Supervisor)
-
•“[My assessment] was informal and disorganized, which heightened nerves and anxiety for getting certified.”(Student)
Comparison with completing the assessment in class
Students expressed differing preferences for completing the assessment in class vs. in the workplace. Some felt less nervous completing the assessment at the practice setting as it seemed less formal, while others would have appreciated the consistency and additional practice opportunities offered by having dedicated time to perform the assessment in class.
-
•“It was a much more relaxed environment, and my preceptor/supervisor was very supportive and thorough. It relieved the stress of perhaps injecting my friends and having someone assess it mark by mark.”(Student)
-
•“I practiced a lot on fruit at home but still felt really nervous before the practical assessment with my supervisor. Having an official training day at the school sounds like something that would have helped with my confidence.”(Student)
Discussion
This article reports on the utilization of workplace-based injection assessments for the purpose of credentialing students to administer injections. Overall, we were pleased with the outcome of our first offering of the program under this modified approach, and have concluded that many workplaces are capable of, and willing to, formally assess students' injection technique. Over half of the cohort successfully completed the assessment, with comments from students and supervisors generally positive and indicating that the assessment was “simple and quick” and provided a suitable alternative to in-class assessment. Informal feedback later in the fall term indicated that pharmacists were appreciative of having additional injection-certified staff to meet patient demand and that students felt empowered as they had an important skill to contribute during a busy influenza season, receiving significant real-world immunization practice. Future iterations of this training will examine the benefits of providing students with additional practice supplies and demonstration videos, and allowing students to complete this assessment immediately following the completion of the didactic components of the program and the written examination.
Despite some challenges integrating the assessment into a pharmacy's workflow, the completion of this assessment at a workplace offered advantages over an in-class assessment. Namely, students valued the opportunity to ask questions and learn from the practice experience of their supervisor and to immediately apply this new skill in practice. As expected, students' comfort and confidence with administering their first injections varied widely, consistent with what has been observed in previous years during in-person practical assessments. This appeared to be magnified by their perceived level of support and their relationship with their supervisor. Indeed, some students indicated a preference for being assessed by their supervisor, while others felt nervous being assessed by (or even administering the injections to) a pharmacist who would be evaluating their experiential education placement overall.
To our knowledge, no students were unsuccessful with their initial attempt at administering the injections; however, only completed assessment forms were returned to the university, so we cannot confirm whether additional injections were required for students to meet all items on the assessment rubric. We must also acknowledge response rates of 44.9% and 26.1% among students and supervisors, respectively; therefore, we cannot rule out response bias that may impact the generalizability of the results across all students and supervisors who completed this practice-based assessment. Additionally, as we did not require video evidence of the injections performed for the purpose of the assessment, we are unable to confirm if we would have assessed the injections similarly; however, no supervisors commented that the rubric was unclear in any way or required modification to meet their needs. Other programs considering this model may wish to also ask for video documentation of the assessment if they wish to confirm the quality of students' injection technique.
Should a similar model be adopted for subsequent cohorts of students, it would be beneficial to provide students with additional supplies for practice, including an injection pad if feasible. Videos from faculty demonstrating all steps in injection preparation and administration would also help to increase students' confidence by clearly demonstrating the university's expectations for proper administration. While students reported accessing videos via YouTube (Google, LLC) and other websites, our experience has been that few videos online demonstrate landmarking as recommended in our program, and some also demonstrate aspiration during IM injections, which is no longer recommended.8 Finally, it was our intention to have the supervisor serve as only an observer and assessor for the injection, and not be the individual also being injected; however, survey responses indicate that in some situations the supervisor received the assessment injections. While it may be possible for the supervisor to adequately observe and evaluate an IM injection, administration of a SC injection on the back of the arm would not be visible to the injector. For future assessments, it will be made clear in the instructions that the assessor should be observing injections being administered to another individual so they can fully observe all aspects of the injection.
Due to the COVID-19 pandemic, students' didactic learning on administration of vaccines occurred and was tested in the spring semester, while the option to complete the injections practical assessment was delayed until fall. Indeed, students did comment on the disconnect in timing between learning about injection administration and giving their first injections on a person as a disadvantage. Should this model be repeated, we will aim to make the option of a practice-based assessment available to students who work at eligible practice sites (e.g., through part-time employment) immediately after successful completion of the didactic training and written examination. Recognizing that not all students will have this option available, others will be able to complete the assessment during fall co-operative placements or at the school (as soon as in-person instruction is available).
To our knowledge, this is the first published report of the practical assessment of injection training performed by practicing pharmacists rather than by the faculty who provided the didactic instruction. We suggest that pharmacy programs consider a similar model if in-class assessment is not possible in a timely manner, or if reassessment of student technique is desired before administering injections in practice.
Summary
Workplace-based assessment for the purpose of credentialing students to administer injections in practice is feasible and well-received by both students and their experiential education supervisors. Faculty should consider providing students with ample supplies to perform multiple practice injections on an injection pad (if available) and demonstration videos to best prepare students to successfully administer their first injections.
The following are the supplementary data related to this article.
eAppendix 1 Grading rubric for injection practical assessment.
eAppendix 2 Waiver of liability agreement.
eAppendix 3 Injection training assessment questions.
Disclosure(s)
None.
Declaration of competing interest
None.
Acknowledgements
The authors wish to acknowledge Allison Tario, Cynthia Richard, Robin Andrade, and Anthony Miller for their assistance with implementing this assessment.
References
- 1.Pharmacy Act. Government of Ontario; 2022. https://www.ontario.ca/laws/regulation/940202 Accessed 12 January 2022. [Google Scholar]
- 2.Expanded scope of practice. Ontario College of Pharmacists; 2022. https://www.ocpinfo.com/practice-education/expanded-scope-of-practice/ Accessed 12 January 2022. [Google Scholar]
- 3.Immunization competencies for health professionals. Public Health Agency of Canada; November 2008. https://www.phac-aspc.gc.ca/im/pdf/ichp-cips-eng.pdf Accessed 12 January 2022. [Google Scholar]
- 4.Supplemental competencies on injection for Canadian pharmacists. National Association of Pharmacy Regulatory Authorities; November 2012. https://napra.ca/sites/default/files/2017-09/Supplemental_Competencies_on_Injection_for_Canadian_Pharmacists2012.pdf Accessed 12 January 2022. [Google Scholar]
- 5.Seasonal influenza vaccination coverage in Canada, 2019-2020. Public Health Agency of Canada; 12 November 2020. https://www.canada.ca/en/public-health/services/immunization-vaccines/vaccination-coverage/2019-2020-seasonal-influenza-flu-vaccine-coverage.html Accessed 17 January 2022. [Google Scholar]
- 6.Doctor of pharmacy (PharmD) curricular grid Rx2018 – Rx2021. University of Waterloo School of Pharmacy; 19 July 2017. https://uwaterloo.ca/pharmacy/sites/ca.pharmacy/files/uploads/files/pharmd_curricular_grid_-_current_rx2018-rx2021_-_updated_july_19_2017.pdf Accessed 12 January 2022. [Google Scholar]
- 7.WHO Director-General's opening remarks at the media briefing on COVID-19. World Health Organization; 11 March 2020. https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19---11-march-2020 Accessed 12 January 2022. [Google Scholar]
- 8.General best practice guidelines for immunization: best practices guidance of the Advisory Committee on Immunization Practices (ACIP) – vaccine administration. Centers for Disease Control and Prevention; 21 June 2019. https://www.cdc.gov/vaccines/hcp/acip-recs/general-recs/administration.html Accessed 12 January 2022. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eAppendix 1 Grading rubric for injection practical assessment.
eAppendix 2 Waiver of liability agreement.
eAppendix 3 Injection training assessment questions.