Piloting The CARD™ System for education of students about vaccination: Does it improve the vaccination experience at school?

Abstract

Objective

Many students are fearful of vaccine injection-associated pain. In prior research, we created Knowledge Translation (KT) tools to address school vaccinations and associated pain, fear, and fainting. The objectives of this pilot implementation project were to determine the acceptability and impact of these KT tools on student knowledge, attitudes, and perceptions of their vaccination experience.

Methods

Pre–post mixed methods design. Students in an independent school in the Greater Toronto Area, Ontario, participated in two separate focus groups before and after school vaccinations. In both sessions, they independently completed a knowledge and attitudes survey, reviewed three KT tools (two videos and one pamphlet) and then repeated the knowledge and attitudes survey. They provided structured and qualitative feedback about the KT tools and described the impact of the education on the vaccination experience.

Results

Altogether, 11 grade 7 students participated. Knowledge scores were higher post-tool review compared to baseline in the first focus group. There was no significant difference in fear scores and attitudes about getting vaccinated. Qualitative feedback was categorized into two themes: intervention characteristics and characteristics of the school environment. Students reported the KT tools helped them to prepare for vaccination. They used the information on vaccination day to reduce their own fear and pain and to assist peers. They believed all students should view the KT tools. Students reported that teachers and nurses did not do enough to make vaccinations a positive experience. For example, they did not provide a private setting as an option for vaccination and prevented them from using some coping strategies recommended in the KT tools.

Discussion

This study provides preliminary evidence of the acceptability and positive impact of the KT tools on students’ vaccination experiences. Future research is recommended that involves inclusion of all students and adults in the KT intervention.

Many students undergoing vaccinations at school are fearful of vaccine injection-associated pain (1). Evidence-based interventions are available (2); yet, they are under-utilized in clinical practice (1). Public health authorities in Canada and in many other countries routinely provide vaccination-related education to the public, including education about vaccinations at school (3). Current education products focus on factual information about the diseases and vaccines; however, this may not be suitable for students who are the primary clients of school vaccination services and would like to learn ways they can make vaccinations more comfortable (4). Creating effective educational tools for students should ideally involve their input to ensure they are relevant (5). Incorporating the needs and preferences of students in vaccination education also supports currently espoused models of education (6) and health care delivery (e.g., patient-centred care) (7), including vaccination (8), that call for their participation.

We undertook a program of research to address the identified knowledge to care gap (9). We sought the input of students to identify their specific vaccine educational needs and preferred methods of education (1,4,10) We then developed three key Knowledge Translation (KT) tools to meet their educational needs and preferences within a larger multifaceted KT intervention called The CARD™ System (C-Comfort, A-Ask, R-Relax, D-Distract) that targets all stakeholders involved in school vaccinations—students, nurses, school staff, and parents. CARD™ acts as a framework for planning and delivering vaccinations that is student-centred and promotes coping (11). We collected preliminary evidence of the acceptability and effectiveness of the three key KT tools with respect to knowledge acquisition, and impact on attitudes about pain and fear (12). In the present study, we pilot tested the implementation of these three KT tools with students prior to and after their first experience with school vaccination clinics. Using a pre–post mixed methods design, the objectives were to assess acceptability to students and examine the impact of the KT tools on student knowledge, attitudes, and perceptions of their vaccination experience. In a subsequent study in this program of research, we implemented the multifaceted KT intervention (i.e., CARD™) more broadly, including more KT tools and all stakeholder groups (i.e., students, public health staff, school staff, parents), and evaluated impact on student symptoms and process outcomes in a controlled clinical trial (13,14).

METHODS

Participants and setting

Students in grade 7 who did not have prior experience with school-based vaccinations attending a large, urban, independent school (Greater Toronto Area, Canada), were eligible for participation. Students received invitations to participate by the school Vice Principal. Informed consent was obtained and all students and their parents signed a consent form. The study was approved by the Research Ethics Board of the University of Toronto.

Procedures and measures

Students attended two separate 1-hour sessions facilitated by one of the researchers (AT) in one of the school’s classrooms. These sessions included both quantitative methodology (surveys) and qualitative methodology (oral and unstructured written feedback). The first session was held 6 days prior to the school’s vaccination clinic and the follow up session was held 13 days afterwards. These times were chosen to coincide with the usual time when students are educated about vaccination by public health nurses and to allow students to recall the event afterwards. The same researcher also attended the vaccination clinic as an observer.

Prevaccination day (session 1)

At the beginning of the first session, students independently completed a knowledge test which included 10 yes/no questions about the effectiveness of different strategies to reduce pain, fear and fainting (see Appendix 2 of overview article in this series for the knowledge test questions) (11). The questionnaire was modified from a prior tool incorporating the content from KT tools in the prior project, and has evidence of construct validity. In addition, students answered the questions: ‘How afraid are you of vaccination needles?’ on a scale of 0 (no fear) to 10 (worst possible fear); and ‘Do you think you should get vaccinations?’ on a five-point Likert scale (1=yes, 2=maybe, 3=don’t know, 4=maybe no, and 5=no).

Students then viewed three KT tools: two videos and one pamphlet. The first video (https://youtu.be/z57vTpb19wQ) (duration, 4 minutes) instructs students about vaccinations. It includes information about what a vaccine is and how it works, side effects of vaccines and the process for school-based vaccinations, including consent and information about what will happen on the day of vaccination. The second video (https://youtu.be/c41HvgEKQSk) (duration, 7 minutes) instructs students in the CARD™ mnemonic with vignettes of children undergoing vaccination using the included strategies. A companion pamphlet to the second video provides students with examples of strategies that are included for each of the letters of CARD™ and fill-in-the-blank spaces for students to record strategies they plan to use for their upcoming vaccinations (see Figure 2 of overview article in this series for the pamphlet) (11). Students provided both structured and unstructured written feedback on acceptability of the KT tools (see Appendix 1 of overview article in this series for the questions); responses were dichotomized for the purposes of analysis (11). They then repeated the knowledge test. Students provided information about age and sex.

Vaccination day

One of the researchers attended the school on the vaccination clinic day as an observer and took notes about the vaccination process. The researcher did not interact with the students and stayed out of view at the back of the room.

Postvaccination day (session 2)

The same study procedures used in the prevaccination day were repeated in session 2, in exactly the same way. Students additionally provided oral and written feedback on the relevance and impact of the education on their experiences with vaccination (student perceptions), and the number of vaccines received.

Sample size and statistical analysis

The target sample size was set at 8 to 40 participants based on feasibility. Quantitative data (e.g., acceptability, knowledge) were analyzed using descriptive statistics (i.e., central tendency and variability). The number of correct knowledge test answers was summed out of 10 for analysis. Knowledge test scores and fear levels were compared across all four time-points (prevaccination baseline, prevaccination post-tool review, postvaccination baseline, postvaccination post-tool review) using a Friedman test. A-priori pairwise comparisons were made using Wilcoxon signed ranks test between: 1) prevaccination baseline and prevaccination post-tool review and 2) prevaccination baseline and postvaccination baseline. Student opinion about whether they should get vaccinated was compared across all four time-points using Cochran’s Q test; data were dichotomized for this analysis (i.e., ‘yes’ versus any other response, as this was deemed to be the clinically important outcome by the project team). A-priori pairwise comparisons were made using McNemar test for the same time epochs as above (i.e., prevaccination baseline and prevaccination post-tool review, and prevaccination baseline and postvaccination baseline). SPSS (version 24) was used to analyze the data. The significance level was set at <0.05.

The audiotaped discussion was transcribed verbatim. Qualitative analyses were performed using directed content analysis (15) with the Consolidated Framework for Implementation Research (CFIR) as the guiding framework (16). Three authors participated in the analysis; one determined the coding themes and the other two double-checked themes against the transcripts. Disagreements were resolved by discussion and consensus. Field notes were used to triangulate the data.

RESULTS

The study was conducted in November, 2017. Altogether, 11 students participated; all students were 12 years old and 8 were female. One female student was absent from school the day of the second session; hence data were available for 10 students in the second session. Out of nine students reporting on the number of vaccines they received at school, three reported getting three vaccines; five got two vaccines; and one got one vaccine.

Quantitative data

Feedback regarding acceptability of the KT tools is shown in Table 1. The majority of the participants reported they understood the information in the videos and pamphlets, and that the amount of information was just right. Knowledge test scores are displayed in Table 2. Session 1 (prevaccination) knowledge scores were higher post-tool review compared to baseline. There was no significant difference in fear scores and attitudes about getting vaccinated (Table 3).

Table 1.

Structured feedback for Knowledge Translation (KT) tools

	Prevaccination session (n=11)	Postvaccination session (n=10)
Video 1 (Information about getting a vaccination at school)
Understood most or all information	(n=9) 9 (100)	(n=8) 8 (100)
Amount of information “just right”	(n=10) 7 (70)	(n=9) 5 (56)
Video 2 (Information about The CARD™ System)
Understood most or all information	(n=9) 9 (100)	(n=8) 6 (75)
Amount of information “just right”	(n=10) 7 (70)	(n=8) 6 (75)
Student pamphlet (Information about The CARD™ System)
Understood most or all information	(n=7) 7 (100)	(n=7) 6 (86)
Amount of information “just right”	(n=7) 6 (86)	(n=7) 6 (86)

Values are frequency (percent).

Table 2.

Knowledge test scores

Time of test	Median (range)*
Prevaccination session (n=11)
Baseline	6 (4–8)a,b
Post-tool review	7 (5–9)a
Postvaccination session (n=10)
Baseline	7 (3–10)b
Post-tool review	7 (4–10)

Values are median (range) range of scores 0–10.

*Friedman test: P=0.131.

Wilcoxon signed rank test for a-priori pairwise comparisons: ^aprevaccination baseline vs. prevaccination post-tool, P=0.046, ^bprevaccination baseline vs. postvaccination baseline, P=0.105.

Table 3.

Fear of needles and attitude about need for vaccination

	Prevaccination session		Postvaccination session
	Baseline (n=11)	Post-tool review (n=11)	Baseline (n=10)	Post-tool review (n=10)
How afraid are you of getting vaccination needles? Pick a number from 0 to 10, where 0 is no fear and 10 is worst possible fear.
Fear score (range, 0–10)*	5 (1–9)a,b	4 (1–9)a	2 (0–8)b	2 (0–8)
Do you think you should get vaccinations?
Number of ‘Yes’ responses**	7 (64)c,d	9 (82)c	9 (100)d,†	10 (100)

Values are median (range) or frequency (percent).

*Friedman test: P=0.166.

^**Cochran’s Q test: P=0.066.

^†n=9.

Wilcoxon signed rank test for a-priori pairwise comparisons: ^aprevaccination baseline vs. prevaccination post-tool, P=0.157, ^bprevaccination baseline vs. postvaccination baseline, P=0.091.

McNemar test for a-priori pairwise comparisons: ^cprevaccination baseline vs. prevaccination post-tool, P=0.50, ^dprevaccination baseline vs. postvaccination baseline, P=0.25.

Qualitative data

Feedback during both sessions was broadly categorized into two domains of CFIR (16): 1) intervention characteristics and 2) inner setting (i.e., characteristics of the environment/organization in which the interventions are employed). These themes are described in detail below with supporting student quotations. The students are identified by their subject number.

Intervention characteristics

Students in the prevaccination session reported increased knowledge from the education included in the videos and pamphlet. #11: “(I learned) why we should get vaccines.” #6: “I learned how to cope with vaccination pain and fear.” Other students stated they were already aware of most of the information. All students agreed the KT tools helped prepare them for vaccinations. Students in both the pre- and postvaccination sessions advised showing the educational tools to all students prior to school vaccinations and providing a review on the day of the vaccination clinic as a reminder. #7: “Like maybe as a preparation procedure we can like get everybody together, play these videos…”

Students appreciated observing vignettes of actual students undergoing vaccinations. #1: “I really feel like it’s important for people to see this so they know there are different reactions to this and it’s okay to feel whatever you’re feeling but also that it’s not something that you have to be afraid of and there’s a reason that you’re doing that [the vaccines].” Students also talked about the amount of information that is suitable for different students; specifically, they noted that some students may be reassured by more information and others may have more fear with too much information. One student with a high level of needle fear reported the education was ineffective for alleviating their fear. #8: “So I really hate needles… if someone has the same fears as me they probably don’t want to see them.”

Students generalized the utility of CARD™ to other situations. #6: “I think the information, like the CARDs, should be shared with everybody because even if they’re not going to use it for vaccines specifically, this coping with fear and like stress and that sort of stuff is helpful for everybody. At some point in your life, you’re going to use this sort of a strategy and I think it’s important for people to get to know.”

Several students in the prevaccination session requested more information about the specific vaccines that they were getting. In the postvaccination session, one student reported confusion about the vaccines that were needed. #2: “But then it turned out that I had gotten (vaccine 1) and (vaccine 2)… And I would prefer if I had been given more information on my vaccines so I could plan ahead. That would have been good.” Some students in the postvaccination session commented on gaps or discrepant information between the educational tools and their actual experiences on vaccination day and suggested more information be included. For example, there was concern about the potential under-depiction of pain. One student who was afraid of needles reported that the needle was more painful than portrayed. #10: “…too many people saying it didn’t hurt because it did.” Another student reported that the postinjection pain was more significant than the video suggested. #2: “…Even a week afterwards it still hurt a lot.” Other students reported that the video shows vaccinations in small classrooms which differed from the setting in their school. #4: “In the video it showed that the kids were getting the vaccinations in like a room with just the nurses but we got our vaccinations in the auditorium in front of our whole class.”

Inner setting

In the prevaccination session, students discussed the importance of peer interactions. #5: “I think it’s really important for peers to talk to each other because they are feeling similar things most of the time.” Students also acknowledged, however, that usual peer communication regarding school vaccinations can be problematic. #6: “…the tone is always negative. Nobody seems comfortable or likes talking about this and like they’re always like ‘I’m so scared, I don’t want this to happen’ and that sort of thing.” Students presented ideas for how to facilitate helpful interactions regarding vaccinations, including teachers playing a supportive role and using the KT tools. #5: “…And so it’s important for teachers to actually spur discussion about it because, I think, without the teacher encouragement, students aren’t going to really have that will power to talk about it.” #7: “As vaccination day gets closer and closer, vaccination starts springing up in conversations a lot more and especially with my friends, I can, uh, I will be scared and most of my friends will be as well so we can talk about the benefits of the vaccination which will help us calm down a lot more because it springs up more often.” Students did not see a role for nurses to support students as they are not familiar with them; rather, parents, teachers, or peers were preferred. #4: “Um, so I don’t think the nurses need to be involved exactly but I think they should know that kids may need to bring a friend in to be more comforted, cause if they deny them the ability to bring a friend in, they may be more traumatized…”

In the postvaccination session, students reiterated the need for all students to receive the education. #11: “I think that maybe if other people like our classmates got this information it might be a lot more helpful. Like it would help them do stuff and like they can conquer it the whole time.” Students reported they used their knowledge to help uneducated peers on vaccination day. #6: “…we had the advantage of like getting people in the loop about what was going to happen. Our information allowed us to not only help ourselves but help everybody around us. And so like I think it’s really valuable to get this certain information.” They also reported feeling empowered by the education. #1: “I felt I had to be braver so I wouldn’t mess up in front of everybody and like start freaking out and that actually helped me kind of concentrate and not freak out…”

Students indicated that adults should also receive the information. #6: “Even if we showed these same videos…so they are also aware of what’s going on it would definitely help.” In part, this may have been due to their perception that adults did not do enough to make vaccinations a pleasant experience for them. #2: “The nurse didn’t help at all…We weren’t allowed to take our bags with us so I couldn’t read and another thing was that um, when I asked questions, the nurses didn’t reply…” #4: “The teacher wasn’t letting people bring friends up with them. I think eventually someone explained it to her and I was allowed to bring a friend and that really helped me.” Students were critical of information which did not match their experiences. #10: “When I asked (the nurse) like how much it will hurt, he answered ‘just like a mosquito bite’ which is misleading because I don’t feel mosquito bites but I felt the needle.” In this context, students did what they could with and without the support of adults. #2: “…So I did use some of the things but some of the things just didn’t work.”

Students reported that beyond simply receiving the information, adults should take a more active role in trying to improve the experience and offer specific strategies to assist with student coping. Students specifically mentioned privacy as an intervention that was not offered on vaccination day that they wanted but for which they felt they could not advocate. #2: “It should have been offered because some people were just too scared to ask for privacy even if they had wanted it. And I have a friend who was like ‘Oh, I want privacy, but I’m just too scared to ask’ and I think it should’ve been offered. Like do you want privacy or are you ok?”

Students reported how the flow of the clinic raised anxiety among students. #4: “The wait time was really long so like you build up anxiety as you waited. And I think the teachers should know about that sort of thing…” The clinic was also disruptive to the school routine. #1: “Since everybody was at different times, everyone was in a different position in class then…it affected uh, the next few periods because a lot of people were at different stages.” Students reported feeling unable to concentrate when returning to class and that perhaps the regular curriculum could be interrupted temporarily. #1: “I personally was feeling really just tired and like my arm was really hurting right after the vaccinations and I felt I just spent a lot of time concentrating and then we have to go to class and everything kept on happening so a lot of people got behind or felt ill. And I don’t think it was fair to people who had this. I think it would’ve been better if we had an hour or something to just sit and watch a movie.” Students also commented on the lack of availability of water or food. #6: “I myself was sick that day. And like after my needle I felt like I was going to puke and felt like really bad. And like I asked for water and nobody had water…And I think that it’s important that we’re given something…”

Students suggested being able to comment on the vaccination experience. #1: “It also would’ve been nice if we were given a survey right after the needles to take back and then hand it in. And then it could’ve been about how you felt it was, (what) worked for you, and how you felt you were treated and then also it might help the people who were giving out the vaccinations…”

Observer field notes from vaccination day

The field notes of the observer independently noted that vaccinations were provided in a large auditorium by three vaccinating nurses at the front. Entire grade 7 classes were present with their teachers; in total, approximately 100 students were vaccinated. Students were seated in auditorium chairs with classmates overlooking vaccinations and individual classes were called up in sequence. Students were vaccinated in view of peers and teachers. Teachers maintained crowd control (kept students together, quieted them down), directed them to injecting nurses, and supported fearful students. Friends were initially not allowed to sit with classmates to provide support; but then allowed later on into the clinic. Postvaccination, students sat in chairs for several minutes before returning to class.

DISCUSSION

This is the first study to implement student-centred KT tools about school vaccinations in preparation for school vaccination clinics. We demonstrated preliminary evidence of acceptability and effectiveness with respect to impact on student knowledge and perceptions about improving the vaccination experience. Specifically, the majority of students reported that they understood the information presented in the tools and that the quantity was appropriate. There was an increase in knowledge about effective interventions. Students reported they used the information included in the KT tools to plan coping strategies on vaccination day; however, their choices were not always supported by immunizing nurses or teachers. They also used their knowledge to support peers who had not been included in the education. Recommendations were made to tailor or supplement the education with information about the unique circumstances of vaccination clinics within each school and to include adults in the education so that they can better support students.

We employed a feasible method of implementation for the KT tools that could be applied in any school setting—students were simply shown the information at school. Student feedback suggested that this student-focused approach, while helpful, is insufficient in and of itself. The education did not entirely match their experiences with vaccination and they had some questions that were not covered by the KT tools. Together, these results point to the need for involvement of teachers and/or nurses to help tailor the information and supplement the information to address specific student questions.

In addition, although students generally felt they benefited from the education, they were largely dissatisfied with how teachers and nurses behaved on vaccination day. Students perceived that teachers and nurses did not do enough to help make vaccinations a positive experience. Students reported that teachers should facilitate knowledge sharing, learning, and use of strategies to reduce fear and pain. Similarly, nurses should support students with their chosen coping strategies.

We targeted only students for education using the KT tools in order to determine the impact when using the simplest implementation approach. We used these results to inform a subsequent project, whereby we fully implemented the multifaceted KT intervention (The CARD™ System). The results demonstrated a significant benefit for students in terms of knowledge and symptoms (e.g., reduced fear), as well as satisfaction of other stakeholders (13,14).

These results are also consistent with the findings of a prior focus group study whereby students reported they should be prepared for vaccinations ahead of time and that adults should support their efforts to reduce pain (4). Being more supportive of student requests has the potential to build more positive client-health provider relationships (i.e., improve trust). Building trust in vaccines, in the immunization process, and in the program are important elements in reducing vaccine hesitancy (17).

There are several strengths of this study. First, we targeted the KT tools to students, the primary clients for school-based vaccination services, empowering and engaging them to participate in their health care. Second, we collected student feedback at two time-points relative to vaccination: before and afterward. Within each time-point, we further collected information before and after review of our KT tools. This allowed us to identify potential gaps and feedback that would not be available had we not circled back to students and used such a comprehensive approach. Although we did not attempt to verify student vaccination experiences, the observer field notes captured some of the same events that students spoke about (e.g., lack of ability to bring friends for support; lack of privacy for vaccination), thereby providing triangulation of reported experiences.

There are several limitations. First, the small sample size limited our ability to demonstrate statistical significance for quantitative outcomes. Qualitative feedback, however, provided support for effectiveness of the KT tools in terms of learning and improving the vaccination experience. While knowledge scores rose during the prevaccination time-point after tool review, scores did not continue to rise significantly after that. It may be that students focus on specific interventions in the KT tools that are of interest, familiar, or personally relevant (i.e., that they would use themselves). Further research is needed to explore student perceptions of the specific interventions included in the KT tools. Second, we used a before and after design and cannot rule out the impact of other factors on the outcomes. Third, we included grade 7 students in one Toronto school, and therefore, may not have captured the breadth of views about the KT tools for students across different grade levels eligible for school vaccinations across the country. The current study is unable to determine whether educational needs and/or impact of the education varies according to sex/gender and school context. Finally, qualitative coding utilized existing domains of CFIR (16), which were originally developed to capture the perceptions of implementers not patients. We recommend that future studies incorporate inductive coding and extend on the CFIR domains to record patient perceptions and implementation success.

In summary, we documented preliminary acceptability and effectiveness of student-targeted KT tools about vaccination and pain, fear, and fainting mitigation. The results provide support for future research that examines the impact of full implementation of this multifaceted KT intervention to better prepare students for school vaccinations, including supplementation and/or tailoring for the specific school vaccination setting and inclusion of adults. Using a student-centred approach to vaccination delivery has the potential to significantly improve student symptoms, attitudes, and behaviours related to vaccination.

Funding: This project was funded by the Canadian Institutes of Health Research Knowledge to Action Grant (KAL-147564).

Potential conflicts of interest: AT reports Section 9 Trademark No. 924835 for CARD. LMB reports that Immunize Canada received grants from Pfizer Canada, Merck Canada, GSK Canada, Seqirus Canada and Sanofi Pasteur outside the submitted work. There are no other disclosures. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.

Pain Pain Go Away Team:

Srdjana Filipovic PhD², Christine Halpert RN MA⁸, Jocelyn Cortes RN⁹, Melanie Badali PhD¹⁰, Kate Robson¹¹, Leslie Alderman RN¹², Christene deVlaming-Kot RN MHSc¹², Angela Alfieri-Maiolo RN MPHN¹², M. Mustafa Hirji MD MPH¹², Tori McDowall RN¹²

²The Hospital for Sick Children, Toronto, Ontario; ⁸British Columbia Centre for Disease Control; ⁹Ontario Ministry of Health and Long-Term Care; ¹⁰AnxietyBC, Vancouver, British Columbia; ¹¹Canadian Family Advisory Network; ¹²Niagara Region Public Health & Emergency Services, Thorold, Ontario