The CARD™ System for improving the vaccination experience at school: Results of a small-scale implementation project on program delivery

Abstract

Objective

We conducted a small-scale implementation study that integrated The CARD™ System (C-Comfort, A-Ask, R-Relax, D-Distract)—a multifaceted knowledge translation intervention designed to improve the vaccination experience at school—within the school vaccination program.

Methods

Mixed methods design, including a controlled clinical trial and focus group interviews. The experimental group included five schools whereby CARD™ was implemented. The control group included five schools whereby no changes were made. Focus groups were held at the end of the school year. For nurses trained in CARD™, additional focus groups were held at CARD™ training and after the first round of vaccine clinic visits. Outcomes included vaccination rate at school and adult stakeholder perceptions.

Results

Altogether, 323 students attended study schools. Fifty-five nurses, school staff, and parents participated in 15 focus groups. The school vaccination rate did not differ (P>0.05) between groups for round 1 clinics (76% versus 77%) or round 2 clinics (68% versus 70%). Participants reported acceptability, appropriateness, feasibility, and satisfaction with CARD™. Experimental group nurses were able to integrate CARD™ within usual activities, including clinic planning, student education, and clinic-day set-up and student vaccinations. Students in experimental schools were described by nurses and school staff as more prepared and less fearful during vaccinations. Nurses reported that CARD™ built on their practice; they had higher confidence in their ability to assess pain and fear and higher satisfaction with their ability to manage it. Nurses also reported improved collaboration with students and with each other. All stakeholder groups recommended continuing CARD™. Some additional time was required, primarily related to clinic planning activities and data collection for study purposes.

Conclusion

CARD™ is a promising new approach for improving the delivery of vaccinations at school. Exploration of approaches to increase parental reach and monitoring of vaccination uptake rate over time are recommended.

School-based vaccinations are a convenient and efficient way of delivering vaccinations to large numbers of youth (1). Stakeholders involved in school vaccinations, however, identify several concerns with this method of vaccination delivery, including student fear and fainting, which may negatively impact the vaccination experience and contribute to vaccination hesitancy (2–7). More work has been recommended to optimize vaccination delivery in this setting. We recently developed a Clinical Practice Guideline (CPG) that includes interventions to minimize pain, fear, and fainting during vaccination (8). Knowledge translation (KT) tools to help clinicians tailor these recommendations for school-based vaccinations were not included due to a lack of research in this setting. We undertook a program of research to address this evidence-to-care gap (9). With input from all relevant stakeholders, we developed a multifaceted KT intervention called The CARD™ System (C-Comfort, A-Ask, R-Relax, D-Distract); herein called CARD™ (10–13). CARD™ is an overarching approach to vaccination delivery that incorporates the CPG recommendations (8) into activities undertaken ahead of time and on the day of vaccination to promote student-centred care and coping during vaccination. By integrating CARD™ into the school vaccination program, there is the potential to improve the vaccination experience for students and onlookers alike due to a reduction in student symptoms (fear, pain, and fainting). More positive experiences with vaccination may, in turn, lead more students to choose to be vaccinated at school and in general.

In this paper, we report the results of a small-scale implementation project that integrated The CARD™ System within the school vaccination program and assessed program evaluation outcomes with respect to stakeholder perceptions of the vaccination process and school-based vaccination rates. Separately, we report on student-centred outcomes, including fear, pain, dizziness (precursor of fainting), and satisfaction (14).

METHODS

Design

This study utilized both quantitative and qualitative methods, including a controlled clinical trial and focus group interviews in a subsample of participants of the trial.

Setting and population

The setting was the geographical region of Niagara, located in southern Ontario, Canada, which receives vaccination services by Niagara Region Public Health (NRPH). The study population included NRPH public health nurses providing services to publicly funded Catholic elementary schools participating in the study, and school staff and parents from the same schools. Separately, we report on grade 7 students receiving those vaccination services (14). Five schools served as experimental schools and five other schools served as controls. Schools were chosen in collaboration with the school board and were separated by geographic location and matched on size (number of students in school) and socioeconomic status of the families in the neighbourhood.

Procedures

For both experimental and control schools, the vaccines being offered were the same: meningococcal vaccine, hepatitis B vaccine and human papilloma virus vaccine. Both hepatitis B and human papilloma virus vaccine required two doses for completion of the series; hence, there were two clinics held at each school (one in the fall-round 1 and one in the spring-round 2). Up to three injections were given to students in each clinic. One injector was present for each child and multiple vaccine injections were given sequentially.

Public health personnel involved in the study included school liaison nurses, charge nurses, and injecting nurses. Some of their activities were altered for the experimental group. We briefly describe their usual activities and elaborate on changes that were made for the experimental group (see also our companion manuscript reporting on student outcomes) (14). School liaison nurses are responsible for planning the first school vaccination clinic. This includes confirming spaces, educating students, and organizing consent forms. Liaison nurses also attend the first clinic and are involved in triaging students, supervising waiting students, and supporting students that are highly fearful. Charge nurses and injecting nurses are present at both clinics. Charge nurses oversee the clinics and are responsible for student flow, clinic set-up and documentation of clinic statistics. Injecting nurses administer vaccines. All nurses work together to support one another and ensure the clinic runs smoothly. For round 2 clinics, the liaison nurses are not usually present, and the charge nurse assumes the role of the liaison nurse and contacts the school principal prior to the clinic to confirm the date. During the clinic, the charge nurse is solely responsible for triaging and supervising students, in addition to other duties.

Clinic planning

In the experimental group, liaison nurses undertook the following vaccination clinic planning activities: 1) securing appropriate clinic space (e.g., library); 2) confirming the clinic spaces are available and that principals have made the relevant individuals (students, parents, and staff) aware of upcoming vaccination days; 3) educating students about CARD™; and 4) having students select their preferences for coping strategies using the student CARD™ pamphlet.

For experimental and control groups, a liaison nurse initiated the planning of school clinics with principals at the end of the prior school year (i.e., June) whereby the clinic dates were secured. In experimental schools, nurses secured their preferred space for the clinic (i.e., school library) and a private office within the library that could be used to accommodate privacy. Principals also committed to allowing students to use distraction devices. In addition, in experimental schools, all clinics were scheduled for the morning. Control school clinics could occur in the morning or afternoon.

Approximately 2–4 weeks prior to the first school vaccine clinics, the school liaison nurse in both groups visited grade 7 classes to disseminate the vaccine consent forms and deliver a lesson about vaccination. In control schools, the liaison nurse delivered the usual in-class education lesson (one period, approximately 40 minutes), which consisted of a slide presentation focused on disease and vaccine information. In experimental schools, the lesson time was doubled (i.e., two periods, approximately 80 minutes). Students received a shorter slide presentation about vaccine-preventable diseases and vaccines being offered to prevent them and students were additionally educated about CARD™. Students viewed two videos (one general video about vaccination: https://youtu.be/z57vTpb19wQ; and one video about coping with fear, pain, and fainting: https://youtu.be/c41HvgEKQSk) and then filled out their own preferences for coping strategies they wanted to use during vaccination on a student CARD™ pamphlet (available in ref. 10). Students also filled in knowledge and attitudes surveys, and, time permitting, reviewed selected written scenarios. Liaison nurses reviewed strategies students selected on the pamphlet, identified students with special requests (e.g., privacy), and noted these requests for upcoming vaccination clinics.

Liaison nurses in both groups disseminated vaccination consent forms to students with instructions to take them home, sign them with their parents, and then return them back to the school. In the experimental group, an extra pamphlet about CARD™ directed to parents (10) was included in the take-home package. Experimental group liaison nurses also disseminated a pamphlet about CARD™ directed to school staff (10) in the mailboxes of all school staff.

Approximately 1 or 2 days prior to the first vaccination clinic, liaison nurses in both groups returned to the schools to review returned consent forms in preparation of the clinic. Nurses in the experimental schools additionally reviewed CARD™ with students that were absent on the lesson day, confirmed student requests (including recording names of friends selected to act in a supportive role for specific students), and answered any additional questions students had.

In experimental schools, the liaison nurses contacted school principals prior to the second clinic to remind them of the clinic and ensured that they sent out reminders to school staff, parents and students using their usual communication methods (e.g., electronic, verbal). As per usual practice, charge nurses in the control group assumed the role of liaison nurses for round 2 clinics, including clinic planning whereby they contacted school principals to remind them of upcoming clinics.

Clinic day

In the experimental group, all nurses present at school clinics—both fall and spring—applied CARD™ to their activities. This included: 1) setting-up clinic spaces to minimize fear (e.g., external waiting area, blocking view of equipment and peers getting vaccinated); 2) visiting classrooms before the clinic to introduce themselves to the class and remind students about CARD™; 3) identifying and prioritizing order of vaccination of students according to their requests; and 4) using CARD™ during vaccination, including assessing student fear prior to vaccination and accommodating student-selected coping strategies during vaccination. These activities were not systematically applied in the control schools. Nurses in control schools routinely utilized verbal distraction and directed students on using specific coping strategies (i.e., relaxing arm, deep breathing, looking away) during vaccinations.

Nurses in experimental and control schools documented aspects of the appointment for the purposes of the study (e.g., student use of coping strategies/interventions) using a checklist. Separately, students recorded their symptoms (pain, fear, and dizziness) on a survey.

For both experimental and control schools, students attended vaccination clinics in small groups; however, in control groups, they were taken out of class in alphabetical order. Liaison nurses were present at both vaccination clinics in experimental schools and assisted with organization and flow of students and student supervision. Liaison nurses in the control group were only present at the first clinic. Liaison nurses and charge nurses in control schools did not arrange for privacy or other accommodations; they did, however, try to respond to specific requests.

Nurse training

Liaison school nurses and injecting clinic and charge nurses delivering services to the experimental schools were educated about CARD™ during a half-day workshop (with lunch provided) where they reviewed all of the resources and accompanying policy and process changes made to accommodate implementation (see also overview manuscript in this series for list of topics and resources) (10). Nurses were given a resource binder with key resources and supplies (e.g., fidget spinner, bubble pen). (N.B.: results of qualitative feedback obtained from nurses about CARD™ tools, and changes in their knowledge and attitudes during the education are reported in a separate manuscript in this series) (12).

Over the study period, 11 nurses (3 liaison school nurses, 7 injecting and charge nurses, 1 liaison and injecting nurse) were involved in delivering care at the experimental schools and 20 at the control schools (5 school nurses, 15 injecting and charge nurses). No nurses trained in CARD™ delivered services to the control schools and no nurses in the control schools delivered services to the experimental schools. Nurses educated about CARD™ did not discuss it with nurses that were not and CARD™ implementation resources were not accessible to nurses in control schools. Managers oversaw this process and nurses’ activities.

Outcomes

We evaluated implementation outcomes identified as important by stakeholders, including perceptions of the intervention (acceptability, appropriateness, feasibility, and satisfaction), and success of program delivery (school-based vaccination rate). Outcome data were collected using surveys, researcher field notes, and focus group interviews. In both experimental and control groups, injecting nurses recorded vaccinations given and features of the student–nurse interaction during vaccination (e.g., coping strategies used, including presence of a nurse or school staff as a support person [yes/no]; interaction duration compared to normal [less, same, more]; and satisfaction with interaction compared to normal [less, same, more]) on a checklist used in prior studies that was modified for the present study (10). A researcher tracked clinic features, including room set-up, number of injecting nurses present, and duration of clinic (from arrival to departure) (10).

A subsample of parents, school staff, and nurses from the experimental and control groups participated in separate focus group interviews. School staff and parents were invited to participate via the school principal; nurses were invited via a public health manager. School staff and parents from each study group participated in focus group interviews at the participating schools. Nurses from each group participated in interviews at public health headquarters. Control school nurses, school staff, and parent participants took part in one focus group interview after the second clinic (round 2) visit. Experimental school nurses participated in focus groups at three timepoints: 1) during CARD™ training (August, 2017); 2) after the first round of school vaccination clinic visits (December, 2017); and 3) after the second round of clinics (June, 2018). As previously mentioned, part of the results from the first nurse focus group (during CARD™ training) are reported in a separate manuscript (12). We purposefully planned multiple focus groups for experimental group nurses to allow for reflection and evaluation of the progress and experience with CARD™ implementation and planning how to refine implementation further.

Focus group interviews consisted of semi-structured interview questions related to positive and negative experiences with school vaccinations and opportunities for improvement. Nurses and school staff in the experimental group additionally answered questions about the impact of CARD™ on the vaccination experience and implementation considerations. All participants provided their level of agreement with attitude statements (five-point likert scale: 1-strongly agree, 2- agree, 3- neutral, 4- disagree, 5- strongly disagree), with nurses in the experimental group responding to additional statements about CARD™. School staff and parents answered a knowledge test with questions about the effectiveness of different interventions to reduce pain, fear, and fainting during vaccination (range, 0 to 10) (10). At the end of the control school staff and parent interviews, participants were given a summary of CARD™, shown the CARD™ pamphlets (10), and asked to provide their comments. These methods and tools were informed from our prior research (11–13), and incorporated aspects from the Consolidated Framework for Implementation Research (CFIR) (15).

The study received ethical approval from the University of Toronto Health Sciences Research Ethics Board and the Niagara Catholic District School Board. Informed consent was obtained from all focus group interview participants. Consent was waived for clinic data (e.g., field notes, checklists, vaccination uptake) to allow for collection of population-level data.

Sample size and analytic strategy

The sample size for the clinical trial was based on student symptoms: 120 students were needed per group to detect a 50% reduction in the rate of students with high levels of fear (control group rate, 30%; experimental group rate, 15%) with a power of 80% and alpha=0.05 (SamplePower™)—the results for these outcomes are reported separately (14). School vaccination rate was a secondary outcome of the trial; there were no a-priori sample size calculations performed for this outcome.

Quantitative data were summarized using descriptive statistics (i.e., central tendency and variability). School-based vaccination rate and adult support person utilization rate were compared between groups using Chi square test. The number of injections administered was compared using T-test. Perceptions of the duration of interactions and satisfaction with interaction during vaccination were compared using Mann Whitney U test. Results were analyzed for clinic visit 1 and 2 separately. Focus group attitudes and knowledge test scores were compared between groups using T-test or Repeated Measures Analysis of Variance (RM ANOVA), as appropriate. The statistical program SPSS (Version 24) was used to analyze the data. A P-value of <0.05 was considered significant.

For focus group interviews, we recruited 3 to 12 participants per session. Audiotapes from the interviews were transcribed verbatim and directed content analysis (16) was used to analyze the data, guided by CFIR (15). Altogether, three individuals coded the data into themes using NVivo software. Disagreements were resolved by consensus. The field notes and responses to attitudes questions were reviewed to examine parsimony with identified themes.

RESULTS

The study was conducted between June, 2017 and June, 2018. Altogether, 163 and 160 students were in grade 7 in the five participating experimental and five control schools, respectively. Fifteen stakeholder focus group interviews were conducted, including 55 participants (see Table 1 for characteristics); 9 in the experimental school group (3 nurse, 5 school staff, and 1 parent); and 6 in the control school group (1 nurse, 3 school staff, and 2 parent). One teacher and three parents that planned to participate were absent on the day of the focus group.

Table 1.

Demographic characteristics of participants

	Age in years	Female sex
Public health nurses (n=19)a	39 (12)*	19 (100)
School staff (n=27)b	44 (8)**	18 (67)
Parents (n=9)	45 (4)***	9 (82)

Values are mean (standard deviation) or frequency (percent).

^an=14 injecting and charge nurses; n=4 school nurses; n=1 both.

^bn=13 teachers, n=7 principals, n=4 resource teachers, n=2 vice principals, n=1 administrative staff.

*n=18; ^**n=26; ^***n=8.

Table 2 displays vaccination rate at school, number of injections per student, personnel and time requirements, and nurse satisfaction levels, as reported by nurses in clinic visit 1 and 2. There were no differences (P>0.05) between groups in school vaccination rate or number of injections. There was higher satisfaction reported during round 1 clinics in the experimental group nurses compared to the control group (P=0.001).

Table 2.

Vaccination rate, personnel and time requirements, and nurse satisfaction

	Experimental (CARD™) (n=163)	Control (n=160)	P-value*
School clinic visit 1 (round 1)
Number of students vaccinated at school	124 (76.1)	123 (76.9)	0.87
Mean number of injections per student	2.5 (0.6)	2.6 (0.6)	0.37
Number of students vaccinated requiring adult supporta	7 (5.6)	13 (10.6)	0.16
Median time involved in visit (reported relative to normal: 0=less, 1=same, 2=more)	1 (0–2)	1 (0–2)	0.19
Median nurse satisfaction with interaction (reported relative to normal: 0=less, 1=same, 2=more)	1 (0–2)	1 (0–2)	0.001b
School clinic visit 2 (round 2)
Number of students vaccinated at school	111 (68.1)	112 (70.0)	0.71
Mean number of injections per student	1.6 (0.5)	1.7 (0.5)	0.14
Number of students vaccinated requiring adult supporta	3 (2.7)	6 (5.4)	0.31
Median time involved in visit (reported relative to normal: 0=less, 1=same, 2=more)	1 (0–2)	1 (0–2)	0.08
Median nurse satisfaction with interaction (reported relative to normal: 0=less, 1=same, 2=more)	1 (0–2)	1 (0–2)	0.63

Values are frequency (percent), mean (standard deviation) or median (range) data from nurse checklist.

*Categorical data analyzed using Chi square test; ordinal data analyzed using Mann Whitney U test; continuous data analyzed using T-test.

^aFrequency of presence of an adult (nurse or school staff) to provide additional support during vaccination.

^bSatisfaction was higher for the experimental (CARD™) group compared to the control group.

Table 3 displays the responses of experimental group nurses and control nurses to attitude statements about school vaccinations after clinic 2. Experimental group nurses reported more positive perceptions (P<0.05 for all comparisons) for the following: 1) confidence in their ability to assess fear and pain, 2) satisfaction with how they manage pain and fear, 3) coordination among school liaison and injecting and charge nurses, 4) collaborations with nurses, 5) collaborations with students, 6) level of staffing, 7) regular provision of input at staff meetings, and 8) value of input at staff meetings.

Table 3.

Attitudes among public health nurses after school clinic visit 2 (round 2)

	Experimental (CARD™) (n=7)	Control (n=8)	P-value*
I believe it is important to treat students’ pain and fear during vaccinations	1.0 (0.0)	1.1 (0.4)	0.37
Students should be given information about how to make vaccinations less painful and frightening	1.0 (0.0)	1.3 (0.7)	0.37
Parents should be given information about how to make vaccinations less painful and frightening	1.0 (0.0)	1.3 (0.7)	0.37
Teachers should be given information about how to make vaccinations less painful and frightening	1.0 (0.0)	1.3 (0.7)	0.37
I am confident in my ability to screen for fear and pain in students during vaccinations	1.4 (0.5)	1.6 (0.5)	0.48
I am confident in my ability to reduce pain and fear in students during vaccinations	1.3 (0.5)	2.0 (0.0)	0.001
I believe pain and fear are frequently experienced by students during vaccinations	1.7 (0.5)	2.4 (0.9)	0.11
I believe pain and fear during vaccination can have a negative effect on students	1.3 (0.5)	1.6 (0.5)	0.22
I am satisfied with how children who are fearful of vaccinations are currently identified in my unit	1.7 (1.0)	2.4 (1.0)	0.19
I believe that dealing with fearful students during vaccinations is too time consuming	4.4 (0.5)	4.0 (0.8)	0.23
I am satisfied with how parents of very fearful students are informed about their options for getting their children vaccinated at school or in other health care settings	3.1 (0.9)	2.8 (1.0)	0.45
I believe that very fearful students should be vaccinated outside the school setting	2.1 (1.2)	2.5 (0.8)	0.50
I am satisfied with how student pain and fear during vaccinations are currently documented in my unit	2.6 (1.1)	2.6 (0.7)	0.91
I am satisfied with how I currently manage pain and fear in students during vaccinations	1.3 (0.5)	2.5 (0.5)	0.001
I am willing to try new ways to reduce pain and fear in students during vaccinations	1.0 (0.0)	1.4 (0.5)	0.08
The school and VPD** nurses in my unit work together as a well-coordinated team	1.0 (0.0)	1.5 (0.5)	0.03
I experience good collaboration with other school and VPD** nurses	1.0 (0.0)	1.9 (0.6)	0.003
I experience good collaboration with school staff	2.4 (0.5)	2.6 (0.7)	0.57
I experience good collaboration with students	1.4 (0.5)	2.1 (0.4)	0.01
Communication breakdowns that lead to suboptimal delivery of care are uncommon in my unit	2.4 (1.5)	2.9 (0.8)	0.48
I can easily speak up if I perceive a problem with patient care in my unit	1.1 (0.4)	1.8 (0.7)	0.06
Disagreements in my unit are resolved appropriately	1.7 (0.5)	2.0 (0.5)	0.30
It is easy for nurses in my unit to ask questions when they do not understand something	1.1 (0.4)	1.6 (0.5)	0.06
The levels of staffing in my unit are sufficient to handle the number of students	1.4 (0.5)	2.4 (0.7)	0.02
Meetings are regularly performed to discuss work processes in my unit	1.7 (0.5)	2.1 (0.8)	0.28
I value meetings in my unit	1.3 (0.5)	1.5 (0.5)	0.44
I regularly provide input during meetings in my unit	1.4 (0.8)	2.4 (0.7)	0.03
My input is well received during meetings in my unit	1.3 (0.5)	2.4 (0.7)	0.006

Values range from 1 to 5 on a five-point likert scale: 1-strongly agree, 2- agree, 3- neutral, 4- disagree, 5- strongly disagree; values are mean (standard deviation).

*T-test; ** VPD- vaccine preventable disease; includes injecting and charge nurses.

Table 4 displays experimental group nurses’ attitudes about CARD™ over time (time 1- at training, time 2- after clinic 1, time 3- after clinic 2). Attitudes about CARD™ were very positive and remained positive over time except in one aspect—the time required for completing the clinic checklist (i.e., part of the data collection required for the study); nurses stated that documentation was too time consuming.

Table 4.

Attitudes about The CARD™ System among experimental group public health nurses over time

	Focus Group 1*	Focus Group 2*	Focus Group 3*
	After CARD™ Education (n=10)	After Round 1 Clinics (n=10)	After Round 2 Clinics (n=7)
I understand the individual components of the CARD system	1.3 (0.5)	1.5 (0.5)	1.1 (0.4)
The CARD system is aligned with our organizational goals	1.2 (0.4)	1.5 (0.5)	1.1 (0.4)
I believe that the CARD system improves the student experience during vaccinations	1.4 (0.5)	1.4 (0.5)	1.1 (0.4)
I am confident in my ability to use the CARD system	1.4 (0.5)	1.7 (0.7)	1.0 (0.0)
I am willing to try all components of the CARD system	1.1 (0.3)	1.5 (0.5)	1.0 (0.0)
I believe the CARD system is being used in my unit**	4.0 (0.7)	1.8 (0.6)	1.6 (0.8)
I believe the documentation involved in the CARD system is too time consuming**	1.4 (0.7)	3.5 (0.7)	4.1 (0.9)
I have the support I need from other personnel to implement the CARD system	1.6 (0.7)	1.7 (0.7)	1.3 (0.5)
Management supports my daily efforts in implementing the CARD system	1.4 (0.5)	1.4 (0.7)	1.6 (0.8)
I would recommend the CARD system to reduce pain and fear during vaccinations	1.6 (0.7)a	1.3 (0.5)	1.3 (0.5)
I think it is realistic to continue to use the CARD system in our setting	n/a	1.4 (0.5)	1.3 (0.5)
I am likely to continue to use the CARD system in the future	n/a	1.4 (0.5)	1.3 (0.5)

Values range from 1 to 5 on a five-point likert scale: 1-strongly agree, 2- agree, 3- neutral, 4- disagree, 5- strongly disagree; values are mean (standard deviation).

*Repeated measures analysis of variance.

**P<0.05.

^an=1 missing.

Table 5 displays results for knowledge tests and attitudes of school staff and parents in experimental and control groups. There was no evidence of a difference between groups.

Table 5.

Attitudes and knowledge about pain and fear among school staff and parents

	Experimental (CARD™)			Control
	Overall† (n=19)	School Staff (n=16)	Parents (n=3)	Overall† (n=17)	School Staff (n=11)	Parents (n=6)
Knowledge*
Knowledge score	8.7 (1.2)	8.7 (1.3)	9.0 (1.0)	8.6 (1.5)	8.7 (1.3)	8.5 (1.8)
Attitudes**
Vaccine injections cause the same amount of pain in everybody	4.2 (0.4)	4.3 (0.4)	4.0 (0)	4.0 (0.6)	4.0 (0.8)	4.0 (0)
I am satisfied with how (my) child(ren)’s pain has been managed during vaccine injections	1.8 (0.7)b	1.6 (0.5)b	2.7 (1.2)	1.6 (0.5)	1.6 (0.5)	1.5 (0.5)
We don’t need to do anything about students’ pain during vaccine injections at school because pain is a normal part of the procedure	3.8 (1.1)	3.8 (1.2)	4.0 (0)	3.4 (1.1)	3.6 (1.0)	2.8 (1.2)
Doctors and nurses should help make vaccine injections at school less painful for children	2.2 (0.9)	2.3 (0.9)	1.3 (0.6)	2.5 (0.9)	2.4 (0.8)	2.8 (1.2)
Parents and students should be given information about how to make vaccine injections at school less painful for students	2.0 (0.9)	2.1 (1.0)	1.7 (0.6)	1.9 (0.6)	1.9 (0.7)	2.0 (0)
Teachers should be given information about how to make vaccine injections at school less painful for students	2.0 (0.9)	2.1 (0.9)	1.7 (0.6)	2.4 (0.7)	2.3 (0.6)	2.7 (0.8)
Vaccine injections cause the same amount of fear in everybody.	4.2 (0.7)	4.3 (0.8)	4.0 (0)	4.5 (0.5)a	4.5 (0.5)	4.4 (0.5)a
I am satisfied with how (my) child(ren)’s fear has been managed during vaccine injections.	1.8 (0.5)c	1.8 (0.6)c	2.0 (0)	1.9 (0.3)	1.9 (0.3)	1.8 (0.4)
We don’t need to do anything about students’ fear during vaccine injections at school because pain is a normal part of the procedure	3.9 (0.9)	3.9 (1.0)	4.0 (0)	3.6 (1.1)	3.7 (1.1)	3.5 (1.2)
Doctors and nurses should help make vaccine injections at school less frightening for children	2.1 (0.9)	2.1 (1.0)	1.7 (0.6)	2.2 (0.8)	2.2 (0.9)	2.2 (0.8)
Parents and students should be given information about how to make vaccine injections at school less frightening for students	1.8 (1.0)	1.9 (1.0)	1.3 (0.6)	1.9 (0.5)	1.9 (0.5)	1.8 (0.4)
Teachers should be given information about how to make vaccine injections at school less frightening for students	1.9 (0.9)	2.0 (1.0)	1.7 (0.6)	2.2 (0.7)	2.2 (0.8)	2.3 (0.5)

*Knowledge levels range from 0 to 10 for 10 yes/no questions; values are mean (standard deviation).

**Attitude levels range from 1–5 on a 5-point Likert scale: 1=strongly agree, 2=agree, 3=no opinion, 4=disagree, 5=strongly disagree; values are mean (standard deviation).

^†T-test: P>0.05 for all comparisons between groups.

^an=1 missing; ^b n=2 missing, ^c n=3 missing.

Focus group discussions identified aspects of acceptability, appropriateness, feasibility, and satisfaction with CARD™ within all CFIR domains (intervention characteristics; characteristics of individuals; outer setting; inner setting; and process) (15). There was substantial overlap; therefore, the results were condensed into four CFIR domains to reduce redundancy: 1) intervention characteristics, 2) characteristics of individuals, 3) outer setting, and 4) inner setting. Each domain is described below, with selected quotes provided as examples in Supplementary Appendices 1 to 4, respectively. The perceptions of participants in the experimental groups are presented first and where divergent and/or deemed significant, perceptions of participants in the control group are presented separately. Individuals are identified by group (experimental or control), stakeholder group (nurse, school staff, parent), and participant number.

Intervention characteristics

Design quality and packaging

Nurse, school staff, and parent participants approved of CARD™. They remarked on the appeal of the language and recommended that CARD™ resources be made widely available. The program itself was also felt to act as a distracting agent for students.

In the schools where CARD™ was implemented, school administrators reported that the method used by the liaison nurse to plan the clinics was effective and helped them to prepare. One administrator recommended providing a checklist specifically for the schools, to help them with triaging of the items that needed to be addressed and to keep track of their status.

The in-class student videos were praised by experimental group nurses and school staff; both commented that they were comprehensive. Participants commented on the ability of the student CARD™ pamphlet to engage students and to help them to prepare for vaccination. They stated students knew what they wanted and were able to make changes to their choices on vaccination day. Some recommendations were made to modify the pamphlet for use in different contexts. Examples included using a checkbox of options for older students, using pictures for younger students, using a digital platform, incorporating interactive activities, and making a wallet-size laminated card for students.

Nurses used the information from student responses on the student CARD™ pamphlet on clinic day. Teachers mentioned that it was not possible to ensure that student requests made on the student CARD™ pamphlet were private (e.g., students could become aware if their peers chose to have friends). This was not, however, reported to be associated with any adverse sequelae as student choices were treated discretely and there appeared to be no special or negative attention given by either nurses or peers to the choices made by students with respect to privacy or the presence of a support person. Students reviewed CARD™ in the class prior to vaccinations but did not require the physical copy of the pamphlet when attending the clinic.

Parents liked the parent CARD™ pamphlet; however, not all parents in experimental schools remembered receiving it. One of the two experimental school parent participants whose children were eligible for vaccinations at school remembered receiving the parent CARD™ pamphlet. That parent discussed it with her daughter, who she described as fearful of needles. It was suggested that the information from the pamphlet be integrated into a pre-existing parent vaccine pamphlet already being disseminated by public health with the consent form rather than be disseminated as an additional stand-alone pamphlet.

CARD™ education was felt to be effective for school staff who were present and involved in the in-class lesson for their students. Despite dissemination of a CARD™ pamphlet directed to school staff in the mailboxes of all school staff, those that were not intimately involved with the lesson (e.g., the French teacher) were felt to be largely unaware of the information, which, in turn, could negatively affect the clinic.

Experimental group nurses and school staff both believed that the nurse introductions to the students and review of CARD™ on clinic day prior to the initiation of the clinic were helpful for preparing the students and reducing their anxiety. Nurses believed that asking students explicitly about their fear prior to vaccination was particularly helpful in their interactions with the students. In addition, nurses attempted to minimize visual cues during the clinic. Nurses reported limited effectiveness of the table divider in preventing students from looking at vaccine equipment.

Experimental group nurses and school staff reported that distraction, bringing a friend and privacy were the most common interventions selected by the students. Students who were not getting vaccinated were also involved in the clinic by acting as supports for those being vaccinated. Privacy was accommodated and discretely delivered in the clinic.

Nurses reported that the CARD™ educational workshop was effective. Nurses stated they refreshed their memory by reviewing the material just before initiation of the clinics. A new educational video was suggested that included nurses modeling student–nurse interactions with CARD™.

Adaptability

Experimental school nurses described trialing and tailoring interventions to achieve the desired results. This included providing clearer instructions to students after the first in-class lesson to ensure they were selecting their own preferences for coping strategies to use during vaccination rather than filling-in general answers on the student CARD™ pamphlet. In one school where students were prohibited from using personal devices because of a school policy, school devices (e.g., iPads) were allowed.

Nurses also made incremental changes to ‘fine-tune’ intervention implementation over time. For instance, the clinic set-up was continually adjusted to minimize visual cues as much as possible, beginning with having a separate waiting area outside the clinic space and using a table divider to obscure equipment, to: applying covers on clinic windows, physically separating clinic tables in the main clinic space more, seating students opposite-facing to each other and nurses, and using separate clinic entrances and exits.

Nurses reviewed CARD™ during the class introductions and also made more notes after clinic visit 1 to help inform clinic visit 2. Nurses used CARD™ to help coach students during interactions, including finding appropriate options if initial student requests were ineffective. Nurses also altered the tool used to assess baseline student fear between the first and second clinic from a dichotomous (yes/no) to a continuous (0 to 3) measure to improve sensitivity.

Relative advantage

All participant groups commented on the advantages of CARD™ over the status quo. Parents stated that teaching students about CARD™ at school was beneficial because students might not otherwise receive the education at home. Nurses and school staff described CARD™ school clinics as ‘less chaotic’, ‘calmer’, ‘smooth’, and ‘student-led’. Nurses believed assessing student symptoms explicitly allowed them to tailor their approach. They also believed they identified students that might go on to faint in the future. Nurses spoke about terminating interactions with students sooner if the student was not comfortable getting vaccinated in order to avoid a negative experience. Nurses in non-CARD™ schools reported more stress in themselves.

Costs of the intervention

Experimental school nurses reported being supported by internal CARD™ champions, including both a liaison school nurse and an injecting nurse. These individuals were, in turn, supported by an external individual to the organization (lead investigator).

There was more time involved in preparing for vaccination clinics because of additional topics to be discussed with principals during the clinical planning consultation and a longer in-class student education lesson (about 1.5 hours).

Nurses in experimental and control schools advocated for the presence of the school liaison nurse at both clinics, not just the first one. School staff also preferred to have the liaison nurse present during the clinics because she was a known face. One experimental nurse, however, changed her mind after the second round of clinics because she felt she did not have enough to do on clinic day. This nurse acted as the charge nurse and roles overlap with those of the liaison school nurse.

Data collection forms used (i.e., nurse checklist of strategies/interventions used and student postvaccination symptom survey) and debriefs held on clinic days also added some time to the clinics (about half an hour) that are not part of routine practice. Nurses suggested periodic use of these data collection procedures as a check-in of the effectiveness of interventions.

Control group feedback

Participants unfamiliar with CARD™ made suggestions for improving the vaccination experience that were compatible with CARD™ interventions.

Characteristics of individuals

Knowledge, beliefs, and attitudes about the intervention

All participating parents and school staff were supportive of immunization and providing vaccinations in the school setting. Parents stated that they have an important role to play in obtaining student buy-in and preparing students for a successful vaccination experience.

All stakeholders remarked on alignment of CARD™ with their roles and relationships. Nurses believed, however, that not all school staff were fully supportive of the program. Nurses and school staff believed that CARD™ prepared the students, reduced their symptoms (e.g., fear, dizziness) and made vaccination a more positive experience. Nurses remarked that the effectiveness of CARD™ improved with each clinic. They also believed that the positive experience would transfer to younger students, who were aware of what’s happening in their school. Nurses did not expect students with a true needle phobia to be vaccinated at school, with or without CARD™.

All participant groups believed CARD™ could improve student ability to cope, not only on clinic day, but at future vaccinations and other stressful situations. They also valued students having a role in supporting their fellow students. Parents believed that making student choices integrated with the vaccination delivery process was important for ensuring its success, rather than just having knowledgeable students self-advocate.

Nurses in the experimental group felt that with CARD™, students were active participants in their health care and that they did not have to control the appointment. They changed some of their behaviours to be able to deliver patient-centred care. One nurse believed that certain students could have a formal role in the clinic as the designated ‘support’ person for other students.

Self-efficacy

Nurses in the experimental schools believed that they were successful implementing CARD™. They stated that CARD™ validated and formalized what they already knew was effective and built on their practice. Nurses reported that CARD™ helped them to understand students’ needs better. One nurse remarked that she believed having the private room enabled her to successfully vaccinate some students that she would not have otherwise been able to vaccinate. Nurses also reported being more satisfied with their role and that they were more confident in their ability to advocate for an appropriate clinic space from school staff.

School staff agreed that CARD™ helped them to support students. Principals reported being less involved on clinic day when compared to prior years, which reflected improved student coping. Teachers agreed they could deliver the education about CARD™ to students. Some teachers mentioned that additional resources or training might be needed. In general, teachers were not comfortable with instructing on vaccine-specific information.

Parents believed that they are effective supports for their children during vaccinations. Parents tailored preparation of their children according to the child’s individual needs; more discussion and coaching was given to more fearful and vocal students. Parents that did not perceive that their involvement was needed did not engage in extensive discussions with their children and as a result, may have been unaware of their child’s feelings.

Individual stage of change

All stakeholder groups agreed that The CARD™ System should be continued. Some nurse and school staff participants expressed strong views about ‘not being able to go back’ to usual practices after practicing CARD™. Nurses spoke about instant and gradual buy-in of CARD™. Nurses were confident that improvements were a result of CARD™ whereas school staff were less convinced. Some school staff recommended continual monitoring to ensure benefits were sustained.

Control group feedback

Control group nurses spoke about parents having an important role and ensuring they were educated about how to support their children. Nurses were generally confident in their abilities and believed that they were doing their best given the challenges. They believed their interactions with students were unpredictable and that they had no control over aspects of the clinic. Navigating challenging interactions with students was perceived to be integral to their role.

Nurses and school staff commonly described the clinic atmosphere as a product of the personalities of the students that was beyond their control. They had difficulty discriminating between true fear and overly dramatic behaviours in students, and they referred to some students as immature. Nurses in the control schools did not speak about students as active participants in vaccination. They described dealing with student fear in the moment rather than being proactive. They engaged students if there were problems, and when they did, they did not find students to be particularly helpful. Education of students about coping was not viewed as a priority.

Nurses were concerned that adverse experiences could negatively impact students. Negative interactions also negatively impacted the energy level and satisfaction of nurses.

Injecting nurses in control schools recommended being able to see the in-class lesson given by the school liaison nurse to know better what the students were taught. Injecting nurses reported being formally trained in technical skills related to vaccination and competence in patient-centredness (i.e., soft skills) was acquired over time through shadowing and experience. If clinic nurses noticed students were nervous, they would send them to more senior nurses.

Outer setting

Patient needs

School staff and nurses asserted that addressing student anxiety about vaccination was meeting student needs and demonstrated to students that adults cared about them. Vaccination at school was believed to be a big step for students as it was potentially their first medical encounter without the presence of a parent as a support person and CARD™ was believed to contribute to a successful medical encounter. CARD™ strategies were also believed to be useful for other stressful events and situations that students have to deal with. Asking students specifically about fear was believed to engage them in their care and to address their concern head-on (i.e., talking openly about ‘the elephant in the room’).

Policy and incentives

Nurses reported that policies were required to ensure that CARD™ was consistently applied in all schools. There was agreement among nurse participants in experimental and control groups that more work was needed to ensure best practices could be realized in all schools; this might require involvement of the school superintendent and school principals.

Control school feedback

While collaboration was reported between school staff and nurses in control schools, there appeared to be less understanding and accommodation of the needs of students for both of them.

Inner setting

Tension for change

Experimental group nurses advocated for expansion of CARD™ implementation across NRPH to realize a wider impact on the quality of the school vaccination program. While the present project was perceived to lead to more collaborative relationships between CARD™-designated school staff and public health nurses, nurses reported that more education of school staff was needed, particularly of teachers, to achieve optimal results. In addition, staff turnover in both stakeholder groups was felt to pose an ongoing barrier to optimal collaboration that needs to be mitigated using different techniques (e.g., yearly education).

Compatibility

CARD™ was perceived as compatible with norms, values, and the usual work systems by all participant groups. Many of the interventions in CARD™ had previously been tried in control and experimental schools, but were not routine (e.g., vaccinating fearful students first, allowing friends to serve as supports to students getting vaccinated, allowing students to use external distraction devices during vaccination). Privacy was believed to be easily accommodated in most schools.

Control school feedback

School staff and nurses’ feedback from control schools identified student symptoms and safety as concerns during vaccination clinics. Timing of the clinic relative to the school schedule was discussed. School personnel were concerned that vaccinations that took place over the lunch hour interfered with students’ ability to eat and that postimmunization events could be missed as there was suboptimal supervision of students in the playground at that time. Nurses, on the other hand, were more concerned with afternoon vaccinations because they might have to rush to finish before students were dismissed.

DISCUSSION

While school-based vaccinations are a convenient and efficient way of delivering vaccinations to school-age children, they are often associated with a negative experience for students and adults alike due to student concerns about needle-associated pain (2–4). Fear of pain can contribute to vaccination hesitancy and lead some students and parents to refuse school vaccinations (5–7). Evidence-based strategies exist to mitigate fear and pain during vaccination (8); yet, there has been no systematic approach to integrating these strategies into practice. We developed a multifaceted KT intervention to address this knowledge-to-care gap called The CARD™ System (10). CARD™ incorporates all that is known about mitigating vaccination pain, fear and fainting into the vaccination delivery process. In this study, we implemented CARD™ in five schools in the Niagara region and compared vaccination delivery outcomes with five schools where we made no changes. We evaluated feedback from adult stakeholders, including public health nurses, school staff, and parents. We demonstrated acceptability, appropriateness, feasibility, and satisfaction with CARD™. We demonstrated no difference in the school vaccination rate in experimental versus control schools.

With respect to acceptability, nurses in the CARD™ schools reported integrating CARD™ in their practice. The benefits of CARD™ were observable to nurses and school staff—both reported that students were calmer and less fearful. Nurses continually refined and expanded interventions over time to optimize their effects. Participants from all stakeholder groups recommended continuing CARD™ after the study. Nurses reported CARD™ should be expanded across the entire school vaccination program.

With respect to appropriateness, many of the interventions within CARD™ (e.g., allowing students to have a friend support person, using an external distraction) had been tried before and were believed to help students; however, they were not explicitly or systematically offered. CARD™ provided a framework and process for making these interventions available to all students. CARD™ was described as giving a voice to students, which was considered important to all stakeholder groups, and integral to delivering patient-centred care. Nurses and teachers both reported a better understanding of the needs of students. Nurses believed that asking students explicitly about their fear, which was new to their practice, was a fundamental component. Parents also reported that making student choice a part of the procedure (i.e., asking students explicitly about the ‘CARDs they wanted to play’) rather than simply providing students with education about coping strategies was important for their self-efficacy as students were invited to select their own preferences rather than having to self-advocate.

CARD™ fit in with usual program activities—only minor adjustments were made to clinic planning and the day of vaccination. The primary resource involved in implementation was personnel time. Managers were involved in creating policies and procedures and reviewing implementation progress. School liaison and injecting nurses were involved in carrying out CARD™-related activities, including data collection for the study. Altogether, approximately 1.5 hours was added to preclinic planning (securing space/accommodations and student teaching) by the liaison nurses, and half an hour for the clinic nurses (data collection and debriefing postvaccination). The presence of the liaison nurse at round 2 clinics, which is not customary, added approximately 2 hours per clinic. A liaison and injecting nurse served as resources to staff and answered specific questions; discussions about CARD™ were also integrated in regular staff meetings for individuals involved in its implementation. Finally, a one-time training session was required for all front-line public health staff (4 hours). Other than personnel time, colour pamphlets were printed for all students, parents, and school staff and some re-usable supplies were purchased for the clinics (e.g., table dividers, fidget spinners, bubble pens). Finally, tools (e.g., videos) were developed by the study team which incurred a one-time cost. Many costs are likely to diminish with full implementation of the program and increasing familiarity by all stakeholders.

With respect to satisfaction, nurses reported being more satisfied with their role because of CARD™. Nurses reported higher confidence in their ability to assess and manage fear and pain and improved collaboration with students and with each other. They viewed students as active participants and were focused on coaching them through the procedure. They reported more enjoyable interactions with the students. Finally, nurses also reported higher confidence in their ability to advocate for an appropriate clinic space as a result of their experience with implementing CARD™. It has to be determined whether this higher satisfaction with their role translates in a higher profile for nursing staff and lower employee turnover rate.

The results of this study are consistent with our quantitative analysis of student-reported symptoms (reported separately) (14), whereby we demonstrated fewer students with high levels of fear and dizziness during vaccination in CARD™ schools compared with non-CARD™ schools. The most frequently self-selected coping strategies were distraction, friends, and privacy—which are consistent with the accounts of nurses and school staff in the present study.

Because we simultaneously altered many parts of the vaccination delivery process and continually refined our methods, we cannot discern the critical factors leading to the observed changes. Based on the feedback obtained from the different stakeholders, the observed success is a result of the combination of the specific interventions used as well as the context. With respect to the interventions, the educational tools (i.e., videos, pamphlets) were considered effective. With respect to context, the principles of CARD™ were compatible with organizational values, beliefs, and work processes. Collaboration among the stakeholders facilitated acquisition of appropriate clinic spaces and execution of clinic procedures. We know from our preliminary work with CARD™ leading up to this trial that education of students alone leads to limited benefit because of the failure of adults to integrate the child’s preferences into the vaccination process (13).

It is important to note that while policies and procedures were provided to guide CARD™ implementation, nurse training did not include specific guidance on interactions with students. Nurses had to develop their own approach for integrating CARD™ into student interactions. Nurses admittedly needed to ‘undo’ some common behaviours that were inconsistent with student-centred care, such as instructing all students to look away from the needle during injection. In addition, except for creating an external waiting area and placing dividers (i.e., tabletop poster) and a distraction toolkit on the workstation tables, there was minimal guidance provided to nurses on ways to set-up the clinic to minimize visual cues that might increase student fear (e.g., medical equipment, students getting vaccinated). Nurses continually added-on ways to minimize visual cues further, including; physically separating clinic stations, and positioning students opposite-facing to nurses and each other.

With input from representatives from all stakeholder groups, we incorporated study learnings into a new educational video for training and support of nurses and school staff (https://youtu.be/FXj6ELi4BVg). This video briefly describes the CARD™ framework and how to apply the principles of this approach, including vignettes of nurse–student interactions during vaccination. Providing this information reduces the learning curve for nurses and school staff just beginning to use CARD™ and reduces the amount of time (and cost) related to ‘shadowing’ or ‘on the job’ training.

With respect to school vaccination rate, while not positively impacted by CARD™, it is important to note that it was not negatively impacted by CARD™. These results demonstrate that openly acknowledging and addressing student concerns about fear and pain during vaccination does not lead to more students being afraid and not getting vaccinated.

The results suggest that more work is needed with respect to education of school staff and parents about CARD™. School staff, particularly those not involved in the in-class education, were reported to be inconsistent in their ability to support students. In addition, not all parents remembered getting the information. Parents, while not physically present, are important players behind the scenes, helping their children make decisions about vaccination and coaching them if they are afraid. School staff act as surrogate parents for students when they are at school. Improving parent and school staff education may lead to parents and teachers having a bigger role in supporting students, which in turn, can increase implementation success. It may also have a benefit on the decision to be vaccinated at school. Showing CARD™ videos to all school staff and providing website links for CARD™ resources to parents using usual school–parent communication methods could be examined in future studies. For parents, this could be done ahead of time, as decisions to be vaccinated at school may occur before students become eligible for school-based vaccinations. Knowing that vaccinations are being delivered using a student-centred approach may make parents and children more comfortable with getting vaccinated at school. Having parents knowledgeable about CARD™ can also help them advocate for and achieve improved pain management practices for their children across different medical settings where they may be undergoing painful medical procedures (e.g., doctor’s office, hospital).

There are several strengths. First, nurses in the control group were blinded to the intervention, which minimized the likelihood that they would alter vaccination delivery services. Second, we included data for all vaccinated students, minimizing the potential for attrition bias. Third, we obtained detailed feedback from all the stakeholders involved, increasing the likelihood that all the relevant perspectives were obtained. Finally, we used a rigorous and comprehensive approach to data collection, evaluating CARD™ using both qualitative and quantitative methods. Triangulation of the data was demonstrated among the different stakeholder groups and data analysis techniques, adding credibility to findings. In future studies, we recommend inductive as well as deductive qualitative analysis approaches to account for the fact that CFIR domains were not originally developed to incorporate the perspectives of non-implementers (e.g., parents) (15).

One limitation is that we included a single public health unit and a relatively small number of schools (small to middle size) limiting the generalizability of the results. Focus group interviews were conducted with a relatively small number of parents and those that participated may differ in important ways from those that did not. Additional research is needed to explore barriers and facilitators to CARD™ implementation across diverse school vaccination contexts and include the perspectives of more parents. Second, the study was not randomized and therefore susceptible to selection bias. Third, nurses in both groups filled in checklists of interventions used during vaccination. It is possible that this promoted self-reflection and evaluation of practices which improved practices in both groups. Given that no policies or processes were provided to facilitate practice changes for control group nurses, it is unlikely that there was a significant change in their usual activities. Fourth, we did not gather information about vaccinations not delivered at school and nonschool-based vaccinations (e.g., influenza virus vaccine) and cannot comment on any differences between experimental and control groups. Our primary outcome was the school vaccination experience and hence, the school vaccination rate was included as a secondary outcome. Additional research is needed to explore the impact of CARD™ on patterns of vaccination uptake over time and more broadly; the benefits of CARD™ may accrue with increased knowledge and experience with the approach among the various stakeholders. Finally, our results represent the combination of efforts of a multidisciplinary and multisectoral team. Future work is recommended to explore harnassing professionals across health care sectors to expand this program beyond the school vaccination setting. Inter-disciplinary education is recommended to extend this learning to students in Medicine, Nursing, Public Health, Pharmacy, and Teaching programs. Future work is also recommended that explores the utility of CARD™ for coping with stressful situations outside of the medical context.

In conclusion, CARD™ was successfully integrated within the vaccination process and aligned well with the roles and responsibilities of the different stakeholders. As a result of this study and other related work in this program of research (10–14), NRPH expanded implementation of CARD™ across the entire school vaccination program in the fall of 2018, including about 150 schools without commensurate changes to staffing levels. The addition of the school liaison nurses to clinics visit 2, however, was not adopted due to insufficient evidence of need. Changes in practices were presented to school board administrators via meetings and executive summaries, and to the public via the organization’s website.

Funding: This project was funded by the Canadian Institutes of Health Research Knowledge to Action Grant (KAL-147564).

Potential conflicts of interest: AT reports Section 9 Trademark No. 924835 for CARD. LMB reports that Immunize Canada received grants from Pfizer Canada, Merck Canada, GSK Canada, Seqirus Canada and Sanofi Pasteur outside the submitted work. There are no other disclosures. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.

Pain Pain Go Away Team: Horace Wong MSc¹, Lucie M. Bucci BA⁶, Christine Halpert RN MA⁷, Evelyn Wilson MAEd BPHE⁸, Jocelyn Cortes RN⁹, Anthony N. T. Ilersich¹, Angelo L. T. Ilersich¹, Cathryn Schmidt², Srdjana Filipovic PhD², M. Mustafa Hirji MD MPH³, Kate Robson¹⁰, Melanie Badali PhD¹¹

¹Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, Ontario; ²The Hospital for Sick Children, Toronto, Ontario; ³Niagara Region Public Health & Emergency Services, Thorold, Ontario; ⁶Immunize Canada, Ottawa, Ontario; ⁷British Columbia Centre for Disease Control; ⁸Ontario Institute for Studies in Education, University of Toronto; ⁹Ontario Ministry of Health and Long-Term Care; ¹⁰Canadian Family Advisory Network; ¹¹AnxietyBC, Vancouver, British Columbia