The CARD™ System for improving the vaccination experience at school: Results of a small-scale implementation project on student symptoms

Abstract

Background

Many students are afraid of receiving vaccinations at school. We implemented a novel, multifaceted knowledge translation intervention incorporating evidence-based vaccination coping strategies—The CARD™ System (C-Comfort, A-Ask, R-Relax, D-Distract)—and evaluated impact on student attitudes, knowledge, coping strategies used, and symptoms during school-based vaccinations.

Methods

Mixed methods. Ten schools participated in a controlled clinical trial: five experimental and five control. Experimental School (ES) students completed a knowledge and attitudes survey during an in-class CARD™ educational session prior to school vaccinations and selected coping strategies for upcoming vaccinations. Control School (CS) students received the usual vaccine education lesson, which did not include information about or selection of coping strategies. At all schools and during both vaccination clinic visits (fall and spring), injecting nurses recorded specific coping strategies used, and students independently rated their fear, pain, and dizziness during vaccinations. Focus groups were conducted at five schools after all clinics were completed (three ES, two CS).

Results

ES students had higher knowledge (P<0.001), less fear (P=0.03), and greater willingness to be vaccinated (P=0.001) after the in-class education session. Students rated the education as understandable, sufficient, useful, and that it prepared them for vaccinations. During school vaccination clinics, ES students selected more coping interventions than CS students. There were fewer students with high levels of fear (P=0.008) and dizziness (P=0.04) in the ES group. In round 2, fewer students (P=0.02) in the ES group returned to the clinic postvaccination because they were feeling unwell. ES students participating in focus groups scored higher on their knowledge test (P<0.001) compared with CS students and reported learning and benefitting from CARD™.

Discussion

This small-scale implementation study provides preliminary evidence of the effectiveness of CARD™ in improving vaccination experiences for students at school. Future research is recommended that examines CARD™ in different settings to confirm these results.

Vaccines are considered integral to prevention and control of infectious diseases (1). All Canadian youth are offered free vaccination via school-based programs. Many of them can have negative experiences or refuse vaccines due to fear of injection-associated pain (2). Public health institutions routinely provide vaccine-related education; however, it emphasizes information about diseases and vaccines. Students have expressed a desire to learn coping techniques for improving the school vaccination experience (3,4). Although evidence-based interventions are available (5), they are not widely implemented (2). Knowledge Translation (KT) strategies that facilitate uptake of interventions have not been adapted for the school vaccination setting.

A program of research was initiated to address this knowledge-to-care gap (6,7). Using feedback from students and other stakeholders involved in school-based vaccinations (nurses, school staff, parents) and published literature, we developed a multifaceted KT intervention called The CARD™ System (C – Comfort, A – Ask, R – Relax, D – Distract)—herein called CARD™—to try to improve the vaccination experience for the students (3,4,8). CARD™ provides a framework for planning and delivering school vaccinations that is student-centred and promotes coping. Preliminary research demonstrates that key tools of CARD™ are acceptable and relevant to the needs of students and other stakeholders (9,10). However, its effects when fully implemented in the school setting have not been examined. We therefore planned a small-scale controlled implementation study that examined the impact of CARD™ on different vaccine delivery program outcomes. In this article, we report on the effectiveness on improving students’ knowledge, attitudes, use of pain, fear and fainting mitigation strategies, and symptoms (pain, fear, and dizziness) during vaccination. Separately, we report on other program delivery outcomes (11).

METHODS

Design

The study utilized both quantitative and qualitative components, including a controlled clinical trial and focus group interviews in a subsample of participants.

Participants and setting

Participants were grade 7 students from ten schools serviced by Niagara Region Public Health in Niagara Region, Ontario. Five schools acted as the experimental schools and five acted as the control schools. The schools were selected with the input of the school board and matched on size and socio-economic status of residents in the neighbourhood.

Procedures and measures

Two vaccine clinics were held in both experimental and control schools; one in the fall (round 1) and one in the spring (round 2). Students were typically given one dose of both hepatitis B and human papilloma virus vaccine in each clinic. Quadrivalent conjugated meningococcal vaccine was provided at one of the two clinics.

Public health personnel involved in the school vaccination program included nurses assigned to each school (school liaison nurses) and vaccine clinic nurses (charge nurses and injecting nurses). The school nurses plan the fall clinics, including; confirming spaces, delivering in-class education to students about vaccination, distributing consents to students, and organizing returned consents in preparation for the clinic. The liaison nurses are present at all fall clinics and are involved in triaging students, supervising waiting students, and acting as supports for students during the clinic.

Charge nurses and injecting nurses are present at both clinics. Charge nurses ensure the clinic room space is arranged appropriately, support injecting nurses during the clinic, and oversee clinic statistics and consent forms. Charge nurses work with liaison nurses to make sure all students are seen and that there is good flow with school activities. Charge nurses assume responsibility for organizing consents, triaging and supervising students when the liaison nurses are not present. Vaccine clinic nurses are primarily responsible for vaccination administration and documentation activities but can also be involved in making telephone calls (to parents), clinic flow and triaging. Additional details regarding the preparation and delivery of vaccinations for the two groups is described below.

Prevaccination in-class student lesson

Experimental schools

Prior to the first set of school vaccination clinics in the fall of 2017, an in-class education lesson lasting approximately 80 minutes (i.e., two class periods) was delivered by a school liaison nurse. The lesson consisted of baseline questionnaires, a multimedia presentation, and postpresentation questionnaires. At baseline, students independently completed a knowledge test (10 yes/no questions about the effectiveness of various strategies to reduce pain, fear, and fainting) (7), and then answered questions regarding their level of fear of vaccination needles (11-point scale, from 0 to 10), and willingness to get vaccinations (five-point Likert scale, from yes to no (1=yes, 2=maybe, 3=don’t know, 4=maybe no, 5=no).

The nurse then delivered a multimedia presentation consisting of information about: a) the specific diseases being protected against with the vaccines being offered (PowerPoint slides), b) how vaccines work, possible side effects, and school-based vaccination procedures (CARD™ video 1: https://youtu.be/z57vTpb19wQ); and c) instructions on coping with pain, fear and fainting during vaccination (CARD™ video 2: https://youtu.be/c41HvgEKQSk).

After the presentation, the nurse distributed a pamphlet to students that summarized CARD™ (7). The pamphlet included a list of strategies for each of the letters of CARD™ and blank spaces for students to list the strategies they wanted to use for their upcoming vaccinations. Students recorded their preferences, and they were reviewed by the nurse and lead author for the purposes of planning the vaccination day (see below: Organization and preparation of vaccination clinics). If there was time, the nurse discussed hypothetical scenarios with students and the CARDs that could be played to mitigate the concerns presented in the scenarios. The same knowledge and attitudes questionnaires were then administered. An additional survey about students’ opinions of the education session was then completed, with questions about level of understanding, amount of information, usefulness of information, and student level of preparedness for vaccination (7). Scores were dichotomized for analysis.

At the end of the education session, the nurse distributed and reviewed the vaccination consent package for students to bring home, which consisted of a pamphlet with information about the diseases and vaccines being offered to students, a CARD™ pamphlet for parents (7), and the vaccination consent form that was to be completed and returned to the school before the first school vaccination clinic date. After the session, the nurse reviewed all the CARD™ pamphlets (approximately 10 minutes, depending on the number of students) to record special requests (e.g., privacy) for the clinic. The nurse provided teachers with a poster about CARD™ for the classroom.

Control schools

In the fall of 2017, nurses delivered an in-class education lesson lasting approximately 40 minutes (i.e., 1 period). This included a 15-minute slide presentation with information about vaccine-preventable diseases and vaccines being offered to prevent them, and how to prepare for vaccination (e.g., wearing appropriate clothing, eating breakfast). There was some overlap with the slide presentation in the experimental schools. After the presentation, there was a brief question and answer period. The school nurse distributed and reviewed the vaccination consent package, which included the same information as the intervention school package except it excluded the parent pamphlet about CARD™. Students did not complete knowledge or attitudes questionnaires, nor select strategies for coping.

Both experimental and control school liaison nurses visited their respective schools a few days before the fall clinic date to collect and organize returned consent forms. For experimental schools only, the nurse briefly reviewed CARD™ with individual students who were absent on the day of the in-class education lesson (approximately 5 minutes, depending on the number of students involved). They also reviewed any scenarios that were not discussed during the lesson because of time constraints with the entire class (approximately 5 to 10 minutes).

In all experimental schools, the liaison nurse contacted the principals prior to round 2 clinics to remind them of upcoming clinics and asked them to provide reminders to school staff, students and parents via usual school communication methods (e.g., school announcements, classroom announcements, electronic messages). Nurses then ensured that these reminders were provided. In control schools, the charge nurse contacted the principals to remind them of upcoming clinics. The principals may or may not have communicated this information with school staff, students, and families.

Organization and preparation of vaccination clinics

Experimental schools

The experimental school liaison nurses were present at both school vaccine clinics. For both rounds, the charge nurse and injecting nurses attempted to minimize visual cues that could increase fear when setting up clinic spaces. A separate waiting area was set-up for students waiting for vaccination outside the clinic room (i.e., outside the school library). Dividers were placed on the tables so that equipment (needles, syringes) was not easily visible to students. Over time, additional measures were taken, including covering the clinic windows to prevent viewing of clinic activities from the outside, setting up chairs so that students faced opposite to other students and nurses, providing more space between the clinic tables, and making the clinic space inaccessible to individuals not associated with the clinic. If there were two ways to access the main clinic space, then one served as an entrance and the other an exit so that students did not cross paths. A separate office in the library served as the privacy room for students who selected this option. In addition, a distraction toolkit was provided for each clinic workstation, consisting of bubble pens, fidget spinners, and pipe cleaners.

Before the vaccination clinic began, the liaison nurse briefly reviewed CARD™ strategies with the students and introduced the injecting nurses to the entire class (approximately 5 minutes). The classroom teacher distributed the students’ CARD™ pamphlets. Liaison nurses coordinated the order in which the students were called from class according to their selected coping strategies on their CARD™ pamphlets. Changes could be made by students on the day of vaccination. Students who identified as highly fearful, seeking privacy, or requesting a specific friend for support were triaged. Liaison nurses assisted with managing crowd control and distracting students waiting for their turn to be vaccinated.

Control schools

In control schools, liaison nurses were present during round 1 clinics only. The charge nurse assumed the role of the liaison nurse in round 2. Students all waited in the same room and could see peers being vaccinated. There was no attempt to modify the tables and student positioning to prevent students from seeing each other or equipment. There was no private space. Students were called out of classrooms in alphabetical order.

For experimental and control groups, a few students were taken out of class at a time; this corresponded roughly with the number of injecting nurses (between 2 and 5). In round 1 clinics, the first group of students were accompanied by the liaison nurse, with subsequent groups of students going to and from the clinic unsupervised. In round 2, the charge nurse led the first group of students down to the clinic if the liaison nurse was not present. In experimental schools, students could be left unattended while waiting outside the clinic. They were allowed to use electronic devices or other distractions while waiting.

Vaccination process

Experimental schools

When it was a student’s turn for vaccination, the student was directed to the injecting nurse by the charge nurse. The nurse introduced herself, carried out a medical history, and asked students about their level of fear. In the first round of school clinics, injecting nurses inquired about student fear using a dichotomous scale (yes/no) prior to vaccination. In the second round, this question was changed to a continuous scale (0 to 3) to improve sensitivity of the assessment. Injecting nurses then asked the student ‘what CARDs he or she would like to play’ during vaccination and accommodated their requests. Nurses invited students to use any of the items from their distraction toolkit if students wanted a distraction and did not have one. Some teachers sent students to the clinic with the CARD™ pamphlets in round 1; however, these were retained by the classroom teacher in round 2 as they were determined not to be necessary (i.e., students did not use them).

Control schools

Injecting nurses followed the same approach as in the experimental schools except that they did not inquire about student level of fear and did not ask students about the strategies they wanted to use. Some nurses brought their own distraction items (e.g., stickers); however, this was not standardized. Injecting nurses engaged in dialogue during the vaccination process to try to distract students. They directed students to inhale while the nurse counted down from three and then to exhale when the nurse administered the vaccine. Occasionally, injecting nurses instructed students to look away.

In both experimental and control schools, immediately after being vaccinated, students were sent to a work station in the clinic room to independently complete a questionnaire where they rated their pain, fear, and dizziness during vaccination on an 11-point scale from 0 to 10 (8). Injecting nurses recorded the coping strategies used for pain, fear and fainting on a checklist (8). The number of students that returned to the clinic because they were feeling unwell was recorded for each group.

For a subsample of checklists, the lead author conducted reliability checks on the data recorded by the injecting nurses to confirm nurses’ transcription accuracy for coping strategies used by students. Thirty-one checklists were completed in duplicate in each round and the per cent agreement on the individual items was 100%.

Nurse training and measures taken to prevent contamination

Ten nurses were trained in CARD™ prior to its implementation in experimental schools (two liaison nurses and eight injecting and charge nurses) via an educational workshop (8,10). They were provided with policies and procedures, CARD™ resources (videos, pamphlets) and supported throughout the study by department managers, an internal study champion, and external content experts (via the study champion and department managers). Part-way through the year, a new liaison nurse was trained in CARD™ by the study champion to take over the responsibilities for another liaison nurse who left the program. Nurses trained in CARD™ did not deliver education or vaccinations to control schools and control school nurses did not deliver education or vaccinations to experimental schools. Nurses trained in CARD™ did not discuss CARD™ with nurses not trained in CARD™. Control nurses followed usual practices.

Focus groups interviews

Within 2 weeks of the completion of round 2 vaccination clinics, invitations were sent by school principals to students, parents, and school staff of participating schools to participate in focus group interviews to share their experiences with school vaccinations. In this study, we report on the data for students; the data for other groups are reported separately (10). Focus group interviews consisted of quantitative and qualitative components, including knowledge and fear questionnaires (same tool as for the in-class education session in experimental schools) and a facilitated focus group interview led by the second author using a semi-structured interview guide. In experimental schools, students reflected on their vaccination experience at school in round 1 and 2 clinics, and on CARD™. In control schools, students reflected on their vaccination experience in round 1 and 2 clinics. In both experimental and control schools, students were asked for their opinions as to how to improve school vaccinations.

The study received ethical approval from the University of Toronto Health Sciences Research Ethics Board and Niagara Catholic District Board. Informed consent was obtained from all focus group interview participants. Consent was waived for in-class education and clinic data to allow for collection of population level data.

Sample size and analytic strategy

The number of schools selected was based on feasibility. With 120 students per group, we could detect a 50% reduction in the rate of students with high levels of fear (score > 6 on a scale that ranges from 0 to 10) if the rates were 30% in the control group and 15% in the experimental group rate with a power of 80% and alpha=0.05 (SamplePower™).

For the experimental group, student acceptability of the in-class education (yes/no) was summarized descriptively. Pre–post-education knowledge test scores and attitudes were analyzed using a paired t-test. Use of specific coping strategies (yes/no) was compared between groups using Chi-squared test. Student symptoms (i.e., fear, pain, and dizziness scores) were compared using repeated measures analysis of variance using Proc Mixed adjusting for sex and number of injections, with clustering (schools) accounted for by fitting a random effect. We examined group, time and group x time interactions (i.e., whether the effect of treatment depends on time). Outcome data were dichotomized into yes/no using a cut-off of > 6 out of 10 (this is the usual threshold for severe pain and was applied to fear and dizziness as it was deemed to be clinically significant to stakeholders), then data were analyzed using logistic regression using Genmod and Glimmix, adjusting for sex and number of injections. Clustering was accounted for by fitting a random effect. Group, time, and group × time interactions were similarly examined. The number of students returning to the clinic after vaccination because of postvaccination symptoms was compared using Chi-squared test. Focus group knowledge test scores and fear levels were compared between groups using a t-test. All analyses were conducted using SAS v.9.4 and SPSS v.24; the significance level was 0.05.

A maximum of one focus group per school was conducted with a target sample size of 3 to 12 participants. The focus groups were audio recorded, transcribed and analyzed by three individuals that identified themes together. Using directed content analysis, themes were categorized according to the domains of the Consolidated Framework for Implementation Research (CFIR) (12).

RESULTS

The study was conducted from June, 2017 to June, 2018. Altogether, 163 and 160 students were in grade 7 in the five participating experimental and five control schools, respectively. A total of 124 and 123 students, respectively, were vaccinated at school in round 1 in the experimental and control schools, respectively. One student in the control school had special needs and was vaccinated with the assistance of an Educational Aid. For round 2, there were 111 and 112 vaccinated students, respectively. There were no differences in the characteristics of students in each group (Table 1). Five focus group interviews were conducted with students in three experimental schools and two control schools, including a total of 23 students (13 in the experimental group and 10 in the control group) (see Table 2 for participant characteristics).

Table 1.

Demographics of students vaccinated in round 1 and round 2 clinics*

	Experimental (CARD™)	Control	P-value**
Clinic visit 1 (round 1)	(n=124)	(n=123)
No. of females	51 (41)	60 (49)	0.23
Mean number of injections	2.5 (0.6)	2.6 (0.6)	0.37
Clinic visit 2 (round 2)	(n=111)	(n=112)
No. of females	47 (42)	59 (53)	0.12
Mean number of injections	1.6 (0.5)	1.7 (0.5)	0.14

*Values are frequency (percent) or mean (standard deviation).

**Chi-squared test or T-test.

Table 2.

Demographics of students participating in focus group interviews*

	Experimental (CARD™) (n=13)	Control (n=10)	P-value**
No. of Females	5 (39)	6 (60)	0.31
Mean Age in years	12.4 (0.5)	12.4 (0.5)	0.94

*Values are frequency (percent) or mean (standard deviation).

**Chi-squared test or T-test.

Quantitative data

Table 3 displays pre- and posteducation knowledge and attitudes of students in the experimental group. Knowledge scores were higher posteducation (P<0.001). Student level of fear was lower (P=0.03). There was an increase in willingness to be vaccinated (P=0.001).

Table 3.

In-class education pre–post-knowledge, fear, and willingness to be vaccinated scores for students in the experimental (CARD™) Group (n=142)*

	Pre-education	Posteducation	P-value**
Knowledgea	6.1 (2.3)	6.9 (2.5)	<0.001
Fear Levela	4.4 (3.6)	4.1 (3.6)	0.03
Willingness to be vaccinatedb	1.7 (1.1)	1.5 (1.0)	0.001

*Values are mean (standard deviation).

**Paired t-test.

^aValues for knowledge and fear range from 0 (none) to 10 (maximum).

^bValues for willingness to be vaccinated range from 1 (yes) to 5 (no).

Table 4 displays student feedback from students in the experimental group regarding the in-class education. The majority of students reported that: a) they understood the information, b) the amount was just right, c) the information was useful, and d) they felt prepared for vaccination.

Table 4.

Student attitudes about in-class education in the experimental (CARD™) group (n=141)

	Frequency (percent)
Understood all or most of the information	134 (95)
Amount of information “just right”	113 (80)
Information useful	116 (82)
Feel well prepared or over-prepared for vaccination	117 (83)

Table 5 displays the frequency of use of different coping strategies during vaccination. There were a significantly greater number of students in the experimental group that used the following strategies at round 1 and 2 clinics (P≤0.03 for all analyses): external distraction aid, friend as a support person, privacy, deep breathing. The frequency of verbal distraction was higher in the experimental group in round 2 clinics (P=0.02). There was no difference between groups in the utilization of topical anaesthetics or lying down during the procedure.

Table 5.

Coping strategies used by students in round 1 and round 2 clinics*

	Experimental (CARD™)	Control	P-value**
Clinic visit 1 (round 1)	(n=124)	(n=123)
Verbal distraction	120 (97)	119 (97)	0.99
External distraction device/object	74 (60)	11 (9)	<0.001
Deep breathing	103 (82)	88 (72)	0.03
Friend present	58 (47)	4 (3)	<0.001
Topical anaesthetic	0 (0)	1 (1)	0.32
Privacy	34 (27)	1 (0.8)	<0.001
Lying down	2 (2)	2 (2)	0.99
Clinic visit 2 (round 2)	(n=111)	(n=112)
Verbal distraction	106 (95)	97 (87)	0.02
External distraction device/object	60 (54)	3 (3)	<0.001
Deep breathing	86 (77)	67 (60)	0.005
Friend present	53 (48)	4 (4)	<0.001
Topical anaesthetic	0 (0)	0 (0)	N/A
Privacy	22 (20)	0 (0)	<0.001
Lying down	3 (3)	2 (2)	0.64

*Values are frequency (percent), as documented in the injecting nurse’s checklist (yes/no).

**Chi-squared test.

Mean student fear, pain, and dizziness scores are displayed in Table 6. Fear scores showed a group × time interaction (P=0.04). Scores were lower for the experimental group at time 2 (P=0.02), however, there were no differences between groups at time 1 (P=0.17). Pain scores showed a group × time interaction (P=0.004). There was no evidence of a difference, however, between groups at time 1 (P=0.80) or time 2 (P=0.22). Dizziness scores showed no significant effects of group (P=0.07), time (P=0.85) or group × time (P=0.96).

Table 6.

Mean fear, pain, and dizziness scores in students undergoing vaccination in round 1 and round 2 clinics*

	Experimental (CARD™)	Control
Clinic visit 1 (round 1)	(n=124)	(n=122)
Feara	3.8 (3.0)	4.6 (3.2)
Paina	3.5 (2.2)	3.4 (2.0)
Dizzinessa	0.8 (1.7)	1.3 (2.5)
Clinic visit 2 (round 2)	(n=111)	(n=111)
Feara	2.7 (2.9)	4.3 (3.0)
Paina	2.9 (2.4)	3.7 (2.1)
Dizzinessa	0.6 (1.4)	1.2 (2.0)

*Values are mean (standard deviation).

^aValues for fear, pain, and dizziness range from 0 (none) to 10 (maximum).

ProcMixed repeated measures analysis of variance results:

Fear: There was a significant group × time interaction (P=0.04); fear scores were lower in the experimental group at time 2 (P=0.02) but not at time 1 (P=0.17).

Pain: There was a significant group × time interaction (P=0.004); however, there was no evidence of a difference between groups at either time 1 (P=0.80) or time 2 (P=0.22).

Dizziness: There was no evidence of group (P=0.07) or time (P=0.85) effects, and no group × time interaction (P=0.96).

Dichotomized (high/low) fear, pain, and dizziness scores are displayed in Table 7. Fear showed a significant group effect (P=0.008); fear was lower in the experimental group (OR=0.47; 95% confidence interval [CI] 0.27 to 0.82). There was no significant effect of time (P=0.39) or group × time (P=0.69). Pain showed no evidence of group (P=0.87), or time (P=0.43) effects, and no group × time interaction (P=0.67). Dizziness showed a significant group effect (P=0.04); dizziness was lower in the experimental group (OR 0.26; 95% CI 0.07 to 0.91). There was no evidence of time (P=0.28) effects, or a group × time interaction (P=0.71).

Table 7.

Frequency of high fear, pain, and dizziness scores in students undergoing vaccination in round 1 and round 2 clinics*

	Experimental (CARD™)	Control
Clinic visit 1 (round 1)	(n=124)	(n=122)
Feara	24 (19)	38 (31)
Paina	13 (10.5)	11 (9)
Dizzinessa	4 (3)	12 (10)
Clinic visit 2 (round 2)	(n=111)	(n=111)
Feara	17 (15)	33 (30)
Paina	10 (10)	11 (10)
Dizzinessa	1 (1)	5 (5)

*Values are frequency (percent) of high scores.

^aValues for fear, pain, and dizziness range from 0 (none) to 10 (maximum); scores dichotomized into high (>6 out of 10) or low (0–6).

Glimmix logistic regression results:

Fear: There was a significant effect of group (P=0.008); fear was lower in the experimental group (OR = 0.47; 95% CI 0.27–0.82). There was no evidence of time (P=0.39) effects, and no group × time interaction (P=0.69).

Pain: There was no evidence of group (P=0.87), or time (P=0.43) effects, and no group × time interaction (P=0.67).

Dizziness: There was a significant effect of group (P=0.04); dizziness was lower in the experimental group (OR 0.26; 95% CI 0.07–0.91). There was no evidence of time (P=0.28) effects, or group x time interaction (P=0.71).

Six students in the control group and 1 in the experimental group returned to the clinic because they were feeling unwell in the first round of clinics (P=0.054). In the second round, it was 8 and 1, respectively (P=0.02). No students fainted in either clinic 1 or clinic 2.

In focus groups, students in the experimental group scored higher on the knowledge test compared to control group students (P<0.001) (Table 8).

Table 8.

Knowledge and fear level in focus group participants*

	Experimental (CARD™) (n=13)	Control (n=10)	P-value**
Knowledgea	9.1 (0.9)	6.5 (1.5)	<0.001
Fear Levela	4.3 (3.7)	4.6 (4.3)	0.84

*Values are mean (standard deviation).

^aValues for knowledge and fear range from 0 (none) to 10 (maximum).

**T-test.

Qualitative data

Student responses from the focus groups were categorized into two domains of CFIR: 1) intervention characteristics; and 2) characteristics of individuals. The themes that emerged from the focus groups are described below, with example quotations. The students are identified by study group (ES – experimental school, or CS – control school) and participant number.

Intervention characteristics (The CARD™ System)

Education session: General

Experimental school students reported increased vaccine-related knowledge after the prevaccination education session. Students felt that CARD™ helped prepare them for vaccinations. ES4: “The CARD strategy definitely helped me with learning how to distract myself and different ways to calm myself and relax myself.” ES3: “I agree with everybody here. I think that it did help me and you should continue to do that because it comforted me a lot more knowing what was gonna happen and what I could do.”

Some students in the control schools expressed a need for information on ways to cope with vaccinations. CS1: “I think for people who get nervous or dizzy over needles, I think it’s good to have tactics or things you could do to make it better, probably it would help a lot.”

Even though the experimental school focus groups were conducted 7 months after their educational sessions, students recalled the components of the CARD™ acronym. ES11: “The C stood for comfort, the A stood for ask and then the R stood for relax and D is distract.” ES3: “I know [CARD]. And I know my strategies that work best for me and it helps me.”

Conversely, students in the control schools had difficulty remembering the brief, standard lesson. “Facilitator: Do you remember the lesson you had at school with the vaccine nurse in the fall where she came and talked to you a little bit about vaccination? … CS2: “No.” CS3: “I don’t remember it at all.”

Education session: Videos

The video portion of the educational lesson was especially appreciated by students in experimental schools. Students found it helpful to know what to expect in the vaccination process. ES1: “…nothing was really a surprise. The videos were almost spot on with everything that was gonna happen so they helped out a lot.” Students in control schools expressed a desire for a prevaccination video that would prepare them. CS6: “[Having] a video. Like a visual representation of them explaining how they feel and how it felt and how you can, like, coping strategies.” Experimental school students found the educational videos to be comprehensive. ES11: “I think it was good. Like they covered everything and made sure we… knew what we were doing… showing you how they were actually gonna do it. They were trying to count down the steps to how they’re gonna do it to make you prepared.”

Reminders

Students from experimental schools stated that the nurses’ reminders of vaccination day approaching and a brief review of CARD™ were helpful. ES6: “When the time was closer to the vaccination the nurses came in again just to remind us that there—it was coming close. So I did, I kept it in my head that—things to do when I do have to have the needle. And strategies to work with it.”

Relative advantage: Choosing to be vaccinated at the school clinic over the doctor’s office

Experimental school students found the nurses provided a more comforting experience than the doctor’s office. ES11: “… it’s just like the manners of the different nurses. Like the nurses at school, they’re more caring and like, comforting. But the nurses at the doctors’ offices are just like trying to get through everybody…” Some students pointed out that doctors’ offices do not often provide distraction techniques, unlike their schools, which adopted CARD™. ES6: “… But I was never given the strategies that I was for [CARD] because at the doctors they don’t give as much before, they just give you the needle.”

Class introductions

Experimental school students appreciated the way the injecting nurses were introduced to them in their classroom before the clinic started, rather than meeting the injecting nurses for the first time as they sat down just before their injection in the clinic. ES6: “I liked how the nurses came in the classroom because I saw like who was doing it, like the people around, like they’re professional. Like to remind us that we’re getting them done to help us and benefit us...”

Relative advantage: Compared to students vaccinated at school in the past

Students at the experimental schools valued the role of CARD™ in improving their vaccination experience at school. ES6: “… some people in the past grades didn’t have [CARD]… my grade got it, but if now that that we have this in our head, that we have these ideas, we won’t have to feel the pain of the other… we have strategies now to work with it. But before they didn’t, so they might have to do the vaccinations without any help. That’s why this CARD System worked for me.”

Clinic process

Some experimental school students commented on being able to be vaccinated with a friend. ES1: “…I brought my friend with me, so that made it better in the sense that I had someone with me.” Several students in control schools expressed that they believed fearful students should be able to go to the clinic with someone they were friends with, as opposed to following alphabetical order. CS10: “… it’s in alphabetical order and unless you’re friends with the person that’s beside you in the alphabet, you’re kinda just with someone like that you know that’s in your class but you’re not very close with.”

Experimental school students could not see students being vaccinated because they were waiting out in the hallway. In control schools, students waited in the same room (i.e., library) where the vaccinations were given. CS1: “They had chairs against the wall and then 5 people would wait there and then they’d go and call 5 more people.” A control school student described seeing others in the library while he was waiting for his turn, which increased nervousness. CS6: “… everyone was looking around and started getting nervous cause they were seeing the nurses preparing the needles and the injections and everybody getting it and making those faces…”

Privacy

Students in experimental schools appreciated having the option of privacy. ES1: “… I had the private room as well, that made it better… it was actually way more helpful…” Some control school students would have preferred vaccination in private. CS1: “The first time I was with one of my friends, and he started crying… he got really nervous watching all of us… I prefer not being in a room with people because I don’t really like people watching me …” CS6: “I think privacy is the best thing because that way it shows that the nurses care about the students and how they’re feeling and all that.” Privacy was not a preference for other control students. CS3: “To see other people go through it and still be calm gives me courage to stay calm and go through it without going crazy.”

Distractions

Many experimental school students liked having the option to use external distraction aids as part of their school vaccination experience. ES4: “… I like how there was a lot of distractions that they provided us with. There [were] different toys you could play with and we were allowed to have our electronics with us.” When asked if the distraction aids helped this student, the participant answered, “Yeah, definitely.” ES5: “Well I brought my friend and there was a lot of fidget toys there… and I just like distracted myself and I looked away and I was like focusing on my friend and I was fidgeting.”

Experimental school students appreciated that school staff allowed them to bring their electronic devices to school for distraction purposes. ES1: “We usually bring our phones to school and we put them away ‘cause we’re not really supposed to be on them. But like for this occasion I think like our teacher understood like all of us were nervous and if we have our phones there it would take away our nervousness.” Students felt that they are old enough to be trusted to use their devices responsibly. ES3: “I think [we] should be trusted enough by grade 7 to know just to go on games and they shouldn’t be going on anything else.”

The students at experimental schools found having a ‘buddy’ present at the vaccination served as a distraction and helped to relax students. ES10: “He just distracted me. We just talked.” ES11: “My buddy helped me. He let me blow bubbles in his face and then he was trying to catch them so like it made me laugh and relax.” In discussing the qualities of a good buddy, students mentioned knowing their buddy well and enjoying their buddy’s company. ES11: “They’ve been around for me like my whole life so I know them and they’re nice to be around.”

The control school students acknowledged that some students would benefit from having a friend with them when vaccinated. CS1: “My friend who was like crying, I think it would’ve been better if he had one of his friends because he went in with one of the girls- he didn’t have anyone- company I guess who could’ve maybe made him feel better about it.”

CARD™ strategy flexibility

Some experimental school students mentioned they liked the flexibility of changing strategies between clinic 1 and clinic 2, picking the ones that would be most beneficial. ES11: “the first time I just did the distract CARD and I was just talking to my buddy.” ES12: “but the second time I used the R and the D CARD. I was distracted by my buddy.” ES9: “and we were blowing bubbles at each other, so it was fun and got my mind off the needle.”

Characteristics of individuals

Knowledge and beliefs: Disclosing fear and preferred vaccine accommodations

Students in both experimental and control schools expressed that their willingness to identify their fear of vaccines varied between students. ES1: “… Everyone has a different like take on that. So it’s like 50/50—half of them may have said like, “oh like I’m really scared” and some of them may have just like kept it to themselves…”

Some control school students suggested that it would be a good idea if students could indicate they were fearful. CS5: “… there should be an option on that paper like is there like, um, like is there any way like we can help them not be scared or like any way, you know. If he needs privacy while doing it, should be an option on the form that says that.” CS6: “there should be an option on the piece of paper cause that way they can look at the paper and they know, oh here, she needs the privacy.”

Self-efficacy: Being a supportive buddy

The experimental school students felt that with the CARD™ strategies, they were empowered to help their peers and younger students. ES10: “So if you know what people need you can help, you can help your buddy out by doing the specific tasks and such like helping them like distract them or just keeping their eyes away from the needles.” ES7: “Yeah, I’ll probably like pass it down. So like some of them … have a needle when [they’re] younger.”

Self-efficacy Using CARD™

Some students in experimental schools mentioned that CARD™ prepared them for school vaccinations because they knew about coping strategies. ES3: “… [CARD] comforted me a lot more knowing what was gonna happen and what I could do.” CARD™ also helped them be less afraid. ES3: “In past experiences I was really afraid so that’s why I was afraid when I first got my vaccines at school. But now I’m not as afraid of needles because of the strategies I was taught.” The students in the experimental schools said that they planned to use CARD™ in the future, in other settings. ES12: “I’ll try to take what we learned here and bring it to the doctor’s office… tell them like tell us to relax, and if they have any distractions, [use] them.”

Nurse role

Students in the experimental schools appreciated the nurses focusing on the students’ needs and wishes during the vaccination process. ES6: “Well the nurses that were doing it really helped because they, even before, even after all the lessons that we learned about it, they still asked when we were getting it, “what can we do?” Like, “at this time, you’re getting it now, what should—what do you want now?” Like, “We’ve asked you before, but it’s—now this is the time, so what do you want us to do to help you?” In contrast, some students at the control schools expressed that nurses controlled the vaccine process. CS2: “[The nurse] said, ‘I’m going to pinch your arm so you don’t feel it as much, so it doesn’t hurt as much.” Facilitator: ‘Did she know you wanted her to help you?’ CS2: “No. No. She just told me she was going to do that.”

Teacher role

Some students appreciated teachers reminding them about the CARD™ coping strategies before the vaccinations. ES2: “Our teacher reviewed the CARD™ system and she also answered questions that some of the kids had…” In control schools, some students considered their teachers to be removed from the vaccination day process. CS1: “[The teacher] kind of just sent us down…I’m not sure how much he actually knew...” Experimental school students described expecting more involvement of their teachers. ES11: “I think our teacher should be aware with our feelings. Like be more involved as a teacher… if you like have the watery eyes they’ll ask and like our teacher, he’ll go until you like start crying and then he’ll ask you if you’re okay.”

Parent role

Students at experimental schools mentioned that some of their parents were familiar with and supported CARD™. ES1: “They thought it was like a good thing for us to have and they wanted me to use it if I was nervous and actually on that day I wasn’t really that nervous cause I know that I had like the CARDs there...” ES6: “Yeah they did [look at the CARD parent pamphlet]. They mostly looked at the strategies to do when there was a needle so, um, yeah they just reminded me of what to do and mostly distraction.” Other parents of students in experimental schools seemed to be less involved. ES10: “I didn’t talk to my parents that much about the vaccine. They asked me how it was and did it hurt. I told them it was good and no, it didn’t hurt.” Some control school students mentioned they had limited discussions about vaccination with their parents. CS3: “I gave them the form to sign and we didn’t talk about it because my parents know like there’s not much more that I need to know about it.”

Some children who are highly fearful of needles reported not being vaccinated at school. ES13: “Well it wasn’t really my decision. It was my parents’ decision ‘cause I’m really afraid of needles and my parents didn’t trust public health at all ‘cause they didn’t know what would happen.”

DISCUSSION

This is the first study to implement a student-centred multifaceted KT intervention (The CARD™ System) in the school vaccination setting and evaluate its effects on student-important outcomes. We demonstrated a positive impact of CARD™ on student knowledge, attitudes, use of coping strategies and some vaccination symptoms. Students reported they understood the information and that it prepared them for vaccination.

These results are consistent with the feedback obtained from adult stakeholders (i.e., public health staff, school staff, and parents) in the same study, described in detail in a separate manuscript in this series (11). Briefly, nurses and school staff reported that students in CARD™ schools were prepared for vaccination and had less fear. They similarly reported distractions, friends, and privacy as the most frequently used coping strategies (11). All stakeholder groups recommended continuing CARD™ after the study.

Since CARD™ is a multifaceted KT intervention, it is likely that multiple components of the intervention led to the observed benefits. Based on the results of student knowledge and attitudes testing and qualitative feedback, the in-class education lesson appears to have been an integral component. The mnemonic aspect of CARD™ likely facilitated student learning (13), especially in light of the level of anxiety associated with the subject matter (14). Use of evidence-based interventions during vaccination would have contributed to implementation success as they have previously been shown to reduce symptoms. The context for implementation is also expected to have contributed to the success of the intervention, including the willingness of the public health unit and school board involved to undertake the project and their commitment to its success. Students reported being impressed by the accommodations made for them and the caring attitudes displayed by adults. These are important factors in the development of trust in the health care providers and in the vaccination program (15).

The results are consistent with prior studies whereby we demonstrated acceptability and knowledge acquisition in students who viewed CARD™ resources (9,10). In one of these studies, however, students reported that nurses and teachers did not do enough to make vaccinations at school a positive experience (10). In that study, however, students alone (i.e., not adults) were educated about CARD™ and no changes were made to the school vaccination program by the school or public health unit.

Prior research shows that one of the most common concerns students have about getting vaccinated is needle pain (16). The broader literature supports providing procedural and coping information for patients undergoing different medical procedures (2,17). Education is hypothesized to prepare individuals in multiple ways. First, it provides knowledge about the procedure. This can reduce fear of the unknown. Next, it allows individuals to plan coping strategies. This can assist in establishing feelings of trust, control and promoting self-efficacy for coping (17). The results of the present study support education of students to reduce their fear and promote coping. We previously demonstrated a reduction in student fear about vaccination after CARD™ education (9). Separately, reduced fear of needles and increased willingness for future vaccination was demonstrated in a Japanese study of school-age children given a pain-related training session (18).

In our preliminary work leading up to this trial, students reported wanting to use external distraction devices (e.g., cell phones) during vaccination (8). We therefore ensured that distraction items were permissible for use during the vaccination clinics. We found that external distraction devices were the most frequent student-selected coping strategy in experimental group students. Students confirmed their preferences for external distractions in the focus group interviews, expressing their wish to be trusted by adults to use their electronic devices responsibly.

Another frequently utilized student-selected coping strategy was having a friend present during vaccination. Students similarly mentioned wanting a buddy to accompany them in our prior studies (8,10). Given that school vaccinations are often the first medical encounter involving a needle procedure that students experience without the presence of a parent, it is perhaps unsurprising that many of them would prefer to have a support person present. Concerns have been raised about the effectiveness of peers as supports by nurses, school staff and parents. There were some instances in the present study whereby nurses reported that students were ineffective in this role and they intervened (11). However, this was not raised as a common or significant issue. More effective education of students was recommended to improve their efficacy in this role. Incorporating a case scenario about supporting a buddy in the in-class education lesson may be one way of enabling peer supports to be more effective. It is important to note that an added benefit of having students serve as support persons is that it might lead to a concomitant diminution in the number of adults required to serve in this role.

Being vaccinated in private was another popular student-selected coping strategy. Once again, students advocated for privacy in our prior work leading up to this trial (8). Vaccinating in public spaces has been reported to be problematic (16). Despite this, privacy is usually only made available for students being vaccinated at school that are required to disrobe for vaccination. Interestingly, some experimental group students mentioned wanting partial privacy, meaning more separation between clinic tables. A positive feature of the included experimental schools was the presence of a separate office within the main clinic space (i.e., library). This layout may not be available at certain schools and other suitable spaces would need to be identified (e.g., health room).

One limitation is that this was a small study, involving only one public health unit, 10 schools, and grade 7 students, limiting the ability to generalize the findings. Second, the study was not randomized and there is the potential for selection bias. Third, the study was not blinded and there is the potential for performance bias. Nurses in the control group, however, were not aware of the intervention, therefore the risk of bias is low. Contamination is unlikely as control nurses were unaware of CARD™ and CARD™ resources were inaccessible to them. Also, nurses trained in CARD™ did not attend control schools. There are several strengths of the study. First, participants (i.e., students) were blinded, which minimized outcome detection bias. Second, we included all vaccinated students, minimizing the potential for attrition bias and improving generalizability. Third, we used a rigorous and comprehensive approach to data collection, evaluating CARD™ using both qualitative and quantitative methods. Our qualitative analysis utilized a deductive approach guided by the CFIR domains. In future studies, we recommend adding an inductive approach to capture the specific feedback of individuals not involved in intervention implementation (i.e., patients), since their feedback was not originally considered in the current domains of CFIR.

In summary, this study provides preliminary evidence of the effectiveness of The CARD™ System in improving the school vaccination experience. Together with the other data from this program of research (8–11), Niagara Region Public Health decided to implement CARD™ across the entire school vaccination program as of September, 2018. To further support public health and school staff, an additional training video was developed specifically for them (https://youtu.be/FXj6ELi4BVg). Additional research is recommended to further explore this novel KT intervention for procedural pain and fear management, inside and outside of the school setting. Before undertaking this work, modification of CARD™ resources may be necessary to ensure that they are suitable for diverse medical contexts.

Funding: This project was funded by the Canadian Institutes of Health Research Knowledge to Action Grant (KAL-147564).

Potential conflicts of interest: AT reports Section 9 Trademark No. 924835 for CARD. LMB reports that Immunize Canada received grants from Pfizer Canada, Merck Canada, GSK Canada, Seqirus Canada and Sanofi Pasteur outside the submitted work. There are no other disclosures. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.

Pain Pain Go Away Team: Lucie M Bucci BA⁶, Christine Halpert RN MA⁷, Anthony N. T. Ilersich¹, Angelo L. T. Ilersich¹, Cathryn Schmidt², Evelyn Wilson MAEd BPHE⁸, Jocelyn Cortes RN⁹, Srdjana Filipovic PhD², Melanie Badali PhD¹⁰, Kate Robson¹¹, M. Mustafa Hirji MD MPH³

¹Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, Ontario; ²The Hospital for Sick Children, Toronto, Ontario; ³Niagara Region Public Health & Emergency Services, Thorold, Ontario ⁶Immunize Canada, Ottawa, Ontario; ⁷British Columbia Centre for Disease Control; ⁸Ontario Institute for Studies in Education, University of Toronto; ⁹Ontario Ministry of Health and Long-Term Care; ¹⁰AnxietyBC, Vancouver, British Columbia; ¹¹Canadian Family Advisory Network