Pragmatic cluster randomised control trial using Vaxcards as an age-appropriate tool to incentivise and educate school students about vaccination

Abstract

Objective

This trial aimed to determine if return rates of consent forms for vaccination could be improved when Vaxcards were offered as an incentive to school children.

Setting

Nineteen schools in South East Melbourne participated.

Interventions

Students in the experimental arm received a pack of Vaxcards when they returned their government consent form.

Outcome measures

Return of ‘yes’ consent forms for vaccination as part of a local government council vaccine programme was the primary outcome of this trial. Return rates were compared between the intervention and control schools and with historical return rates.

Results

Secondary school students (N=3087) from 19 schools participated. Compared with historical returns, a small global reduction in ‘yes’ responses to consent forms of −4.21% in human papilloma virus consent ‘yes’ responses and −4.69% for diphtheria, tetanus and pertussis was observed across all schools. No difference between the experimental and control groups was observed.

Conclusions

Low ‘yes’ consent rates and reduction in consent rates between 2018 and 2019 for all groups are concerning. This finding highlights the need for behaviour change interventions across all groups to increase vaccine confidence. Lack of effect of incentivisation with Vaxcards in this study may have been due to the timing of receiving the cards (after the decision to vaccinate had been made, not before) and the limited intensity of the intervention. Optimising the timing and the intensity of exposure to Vaxcards could improve the outcome.

Trial registration number

ACTRN12618001753246.

Strengths and limitations of this study.

• This trial was conducted in a real-world, pragmatic setting.

• Behaviour change interventions are complex in their nature due to the ways in which behaviours develop in different contexts for different individuals.

• Reward of Vaxcards for returning a consent card regardless of response may mean the incentive to return a form consenting ‘yes’ was diluted, impacting the main outcome measured.

• It may not have been able to effectively control for bias in this sample.

• There may have been an unrecorded data-reporting lag by the council if they were still waiting for consent cards to come in retrospectively after following up students within schools.

Background

Vaccines are a safe and efficacious preventive measure for many illnesses. Despite a long history and evidence of safety and effectiveness, vaccination rates are variable, depending on geography, socioeconomic status and confidence in vaccination.^1–4 Education, incentives for vaccination and engagement with those who are hesitant to vaccinate are critical areas to investigate in order to increase vaccination rates.⁴

Vaccination rates vary globally but dip below targeted goals for vaccine coverage in many advanced economies including Australia, where vaccination coverage is around 90%–94%.⁵ Within Australia, there is variation in vaccine coverage between states. Meanwhile, within states there can be substantial regional variation.⁵ Growing understanding from social network analysis shows clustering of vaccine refusal and lowering herd immunity, potentially providing focal points for outbreaks.^6–8

Incentivising vaccination is a common practice in population health programmes.^9–12 It has been shown that monetary and non-monetary incentives improve vaccination uptake by up to three times.¹³ A Cochrane review of strategies to improve vaccine uptake in adolescents showed health education, class-based school vaccine strategy, multicomponent provider interventions and targeting parents and financial incentives may all improve uptake.¹⁴

Recent government programmes within Australia implemented in 2001, such as ‘no jab, no play’ and ‘no jab, no pay’, involve withholding childcare or welfare payments from parents of unvaccinated children. The aim with these programmes is to deter vaccination avoidance by withholding financial support to families eligible for these schemes.¹⁵ This strict approach appears to increase catch-up vaccine status, especially in lower socioeconomic groups,¹⁶ but the full implication on longer term vaccine trust and confidence is not understood. Other research has suggested it is unclear whether this punitive approach is effective.^{15 17} Of children aged 1–6 years, 3% are affected by registered or presumptive (unregistered) vaccination objection, which suggests that the overall impact of vaccination objection on vaccination rates has remained largely unchanged.¹⁷

Large vaccination programmes, such as in schools, rely on simple systems to provide informed consent to participate in the programmes. Ethically, individuals must understand the risks and benefits of vaccination and organisations must gain consent before invasive procedures like intramuscular injections.¹⁸ The informed consent process can be a barrier to participation in these programmes and result in missed opportunities for vaccination.¹⁹ Students in the state of Victoria, Australia, are provided a consent card prior to vaccination which their parents must sign and return in order to receive the human papilloma virus (HPV) and diphtheria, tetanus and pertussis (DtPa) vaccines around the age of 12 at their school. This occurs in the first year of their secondary schooling. Depending on the local region, students are provided this form from 1 to 6 months in advance, determined by their school. This means sometimes they can transition from primary to secondary school during the time period in which they are required to return the consent form. Many forms are lost, forgotten or deprioritised during this transition period and there is little incentive for the student or the parent to return the form other than the benefit of receiving the vaccination.¹⁹ Some vaccine programmes monetarily incentivise schools or parents to attain minimum rates of consent form return; however, this school-level incentivisation has limited impact on target vaccination levels.²⁰

The school vaccination programme is a target area for interventions that can help increase vaccination rates. However, there is no consensus as to what interventions are most effective to incentivise and educate about vaccination in adolescents. A Cochrane review called for a more understanding of adolescent-specific hesitancy and targeted interventions that are class-based, multimodal, use appropriate incentives and involve health education delivery.¹⁴ There is other evidence that game-based interventions can be a successful modality for behaviour change, when they are carefully designed for the right context and consider the right mechanism of action.²¹ All of these are potentially modifiable by Vaxcards—a collectable, educational table-top card game.

Collectables and gamification are educational tools that can help children engage with learning, generate discussion and provide incentive to engage with the content being delivered.^{21 22} This medium of education increases motivation and engagement.²² The collectable card game ‘Vaxcards’ has cards with characters based on diseases that children are vaccinated against. Vaxcards was launched after a successful crowdfunding campaign in 2016, and its viability as a stand-alone game with educational quality is shown by being listed as a staff pick new games award on Kickstarter and by being selected in the National Serious games working group.²³ Within the health community it has attracted significant interest, being the topic of a top shared and read article on the Bill & Melinda Gates Foundation-supported GAVI website and the focus of a feature article in The Lancet.²⁴ The authors write: ‘Vaxcards appear to be an innovative card game for children, but beneath that they may have the potential to overcome some of the behavioral barriers when incorporated with existing vaccination programs’.

The objective of the present trial is to test the use of Vaxcards as an ethical, non-monetary incentive to support school vaccination programmes for secondary school students. It will determine if the return of consent form for vaccination improves when the card game is offered as an incentive. We hypothesise that students in schools that were incentivised to return the vaccination consent form will show improved vaccination consent form return rates.

Methods

The study will be reported using Consort guidelines and extensions for cluster and pragmatic trials.²⁵

Trial design

This is a pragmatic, cluster randomised controlled trial²⁶ involving secondary schools within a large local government area in the outer South East Melbourne, Australia. Block randomisation was used to allocate participating schools to one of two groups forming the experimental and the control group.

Participants and setting

The participating local government area is in southeast Victoria, Australia, on the fringes of the state capital city of Melbourne. It encompasses a diverse cultural population of high and low socioeconomic status families, and is one of the highest growth areas in the state. The vaccination consent rate within the catchment schools in 2018 varied from 64.6% to 91.3%, which are below the WHO and Australian government target coverage of 95%. Children aged 12–13 years in this area are receiving the Australian schedule vaccination for HPV and DtPa booster. This is also the target age for collectable card games. We were not able to collect specific individual-level data on the exact breakdown of age/gender/socioeconomic status of the individual students due to our ethics agreement with data collection from the council and governmental department of education, and we relied on council-reported immunisation rates at the school level only.

Control arm

The control arm took part in the normal processes of the school-based vaccination programme. In this arm, parents of children received information about the vaccination programme during term 4 of the preceding school year as usual. They were asked to sign and return a consent form to the school before the vaccination programme occurs early in term 1 of the next school year. The local government council records the return rates of consent forms for vaccination.

Experimental arm

Students underwent the normal government vaccination process as above. At the time of consent form distribution, children and their parent/caregiver were provided a handout advising them that children who return the consent form will be given a ‘basic pack’ of the card game Vaxcards. This form contained an explanatory statement about the study and offered the parent or the carer a chance to decline participation of their child in the study or to contact the research team for further information.

The school staff member responsible for coordinating the government vaccination programme provided one basic pack of Vaxcards to children who returned a vaccination consent form in the intervention schools in February 2019. Consistent with the pragmatic nature of the trial, the school determined which staff member was responsible for this. The card pack was handed to each student who returned his or her vaccination consent form, regardless of response or consent to vaccination. This was done to not exclude non-consenting students from the intervention because they could change their consent status any time prior to vaccination.

Proposed intervention

Vaxcards packs contain 13 disease character cards that represent the diseases vaccinated against during the routine childhood immunisation schedule in Australia (measles, mumps, rubella, diphtheria, tetanus, pertussis, HPV, rotavirus, haemophilus B, hepatitis B, meningococcal, pneumococcal, varicella). Each character is designed to anthropomorphise the disease, with traits of symptoms of the disease and information or ‘powers’ that reflect the microbiology of the disease, the vector or mode of transmission, and information on global incidence and mortality which reflect how powerful the character is within the game. Each player collects their own set of disease characters and exchanges addition, subtraction and multiplier game mechanics to influence a sliding scale of ‘hit points’. The game is designed so disease characters maintain their scientific names and encourage the use of terminology and symptomatology among players. The gameplay is light-hearted in nature and non-violent or threatening.

Outcomes

The primary outcome was consent to vaccination based on the returned council consent forms, routinely collected by the local government councils.

Procedure for randomisation and blinding

A statistician, blinded to the school characteristics, conducted the randomisation and the allocation sequence was performed by assigning clusters to interventions. It was not possible to blind participants to the intervention; however, they were blinded to the information about the existence of a control/experimental group until after vaccination. Block randomisation determined the allocation of participating schools to one of two groups forming the experimental and the control group. To ensure balanced proportions of these school characteristics in each cluster in the test and control groups, we stratified the randomisation by school based on the number of year 7 student enrolments (‘less than 100’ and ‘more than 100’) and consent return rate (<90% and ≥90%). After randomisation and allocation to groups, the lead investigator, who was not blinded, consented and recruited schools to participate. The statistician remained blinded.

Initially, two schools included in the randomisation process were not identified as specialised schools for children with intellectual disability (ID 25 and ID 12). Once identified, these two schools were removed from the original randomisation allocation (as both were randomly allocated to the control group) because a likely confounder was student type. Instead of excluding these two schools, reallocation occurred after the creation of another stratification factor (special needs) of one school to the test group and the other to the control. For pragmatic reasons, the school scheduled to begin the vaccination programme later in the year than the other school was assigned to the test group, as this gave the researchers more time to introduce the intervention. The control school was the other school.

Sample size and pretrial power calculation

Within the 31 schools in the local government council, 25 were participating in the 2019 local government vaccination programme. Of these, in 2018, there were 12 schools in this council area with enrolments of less than 100 year 7 students aged 12–13 and 11 schools with 100 or more. There were eight schools with a historical consent return rate of less than 90%.

Of the 31 eligible schools, 6 were not participating in the council vaccination programme and were excluded (figure 1). One school had already returned the council consent forms and was excluded. This school (from the experimental group) was replaced by another school from the same strata, randomly selected from the control group. Of the remaining 24 schools, 19 agreed to participate in the trial and 7 were randomised to the experimental arm (n=965 students) and 12 to the control arm (n=2122 students).

Figure 1.

PRISMA chart showing the flow of participants through the randomised control trial. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

A prestudy power calculation was conducted assuming 23 clusters of school involvement. The study had ample power (>95%) to detect a change in proportion of 5%. The power calculation was done using Stata V.16 statistical software for a stepped-wedge trial with 23 clusters defined at the level of the school. The primary outcome measure was return of the consent form from 120 students per school, with significance level set at 0.05, intraclass correlation coefficient within schools of 0.3, approximately half the schools receiving the intervention (ie, steps=1) and with data examined at two time points (baseline: returns in 2018; year 1: returns in 2019).

Data collection methods and instruments used

The primary outcome data were de-identified, routinely reported local government council data on consent form returns which were provided to the researchers by the local government council. Return of consent cards and the number consenting to vaccination were reported to the researchers for analysis.

Patient and public involvement

No patients were involved in the study. Stakeholders of teachers and council vaccination services were involved in the design of the trial to best fit in the existing vaccine schedule without disrupting workflow of the current consent card collection and vaccination process.

Data analysis

The main outcome was the change in consent rate (HPV, DtPa, both) for each school, which was calculated by comparing the council-collected vaccine consent rates from the baseline in 2018 with rates from the trial year of 2019.

We used linear regression to investigate each of these three outcomes with three bivariate independent variables: 2018 school students in year 7 (>100, <100), previous 2018 consent form return rates (<90%, >90%) and intervention group allocation (test, control).

We used Stata V.16 for the regression analyses.

Participant flow

Several schools declined the invitation to be involved in the trial. The most common reasons included not participating in the council vaccine programme (six), too busy or had other commitments (two), ‘too sensitive’ research topic (three), and requesting the investigators to seek an additional approval from a Catholic school’s research ethics committee which was not attained within the timeframe of the study (one).

Results

The trial involved a total of 3087 secondary school students in 19 school clusters. Consent forms were returned from 2754 students. Of those returned 2081 were marked ‘yes’ comprising 75.6% of all returned HPV forms (range between schools=50.3% and 90.6%) and 2113 were marked ‘yes’ comprising 76.7% of all returned DtPa forms (range between schools=54.6% and 92.2%) in 2019. There were 327 outstanding consent cards that were not returned according to council data.

With regard to primary outcome, there was so significant difference in vaccine consent rates between the control and experimental groups.

Subgroup analysis demonstrated a significant improvement in consent to vaccination between 2018 and 2019 for students from small schools (<100 students in the year level). The combined increase in returned ‘yes’ forms for both vaccines in these schools was 4.49% (10.99% better than larger schools, with a coefficient of −0.11 for school size on vaccination rate change; p=0.04, CI −0.22 to −0.01). There was no significant difference in the subgroup analysis of prior vaccination rates in schools (see table 1).

Table 1.

Analysis of change in vaccination consent rate between the intervention and the substrata

	% change Control	Difference (%) Intervention	Coefficient (%)	P value	95% CI
HPV
Intervention	−2.05	−2.74	0.69	0.37	−0.05 (−0.16 to 0.07)
	<100 students	100+ students
Size of school	3.98	−6.33	10.31	0.06	−0.11 (−0.22 to 0.01)
	Low consent school	High consent school
Previous consent rate	0.44	−6.65	7.10	0.28	−0.06 (−0.17 to 0.05)
DtPa
Intervention group	−2.35	−1.79	−0.57	0.43	−0.04 (−0.15 to 0.07)
	<100 students	100+ students
Size of school	5.00	−6.67	11.68	0.03	−0.12 (−0.23 to −0.01)
	Low consent school	High consent school
Previous consent rate	0.89	−6.89	7.78	0.2	−0.06 (−0.17 to 0.04)
Combined DtPa+HPV
Intervention group	−2.20	−2.26	0.06	0.39	−0.05 (−0.15 to 0.06)
	<100 students	100+ students
Size of school	4.49	−6.50	10.99	0.04	−0.11 (−0.22 to −0.01)
	Low consent school	High consent school
Previous consent rate	0.67	−6.77	7.44	0.22	−0.1 (−0.17 to 0.04)

DtPa, diphtheria, tetanus and pertussis vaccine; HPV, human papilloma virus vaccine.

Compared with the historical comparison, there was a mean global reduction in ‘yes’ responses in the returned consent forms across all schools of −4.21% for HPV and −4.69% for DaPa. The average ‘yes’ response (ie, a positive response consenting to participation in vaccination) across all schools for the previous year (2018) was 79.77% for HPV (range 56.6%–90.72%) and 81.42% for DaPa (range 61.54%–95.88%). There was no statistically significant difference between the change in the proportion of returned consent forms with a consent to vaccination between the experimental and control groups (see table 2).

Table 2.

Combined intervention and control school consent rate changes

	2018 average consent (%)	2019 average consent (%)	Difference (%)
HPV
Consent all schools	79.7	75.6	−4.2
Range	56.6–90.7	50.3–90.6	−21.5 to 34.0
DtPa
Consent all schools	81.4	76.7	−4.7
Range	61.5–95.9	54.5–92.2	−17.3 to 29.9
Combined HPV+DtPa
Consent all schools	80.6	75.9	−4.75
Range	59.4–93.3	52.4–91.4	−19.4 to 32.0

DtPa, diphtheria, tetanus and pertussis; HPV, human papilloma virus.

There was considerable intraschool variation in the proportion responding ‘yes’ between 2018 and 2019. ‘Yes’ consent for HPV forms ranged from −21.5% to +34.02% and ‘yes’ consent for DaPa ranged from −17.27% to +29.92%.

Discussion

There was no significant difference in the consent rates between the experimental and control groups and this likely reflects the complexity of vaccine confidence interventions and the challenges of behaviour change that requires multimodal interventions. One major finding from these data is the low consent rates and the global reduction in consent rates between 2018 and 2019 for both the control and experimental groups. Consent for vaccination is far below the target range of 95% specified by the local government council area in which this trial was undertaken. This highlights the need for interventions to increase these rates and prevent further vaccine hesitancy in the setting of public school vaccination programmes.

In order to improve vaccination rates towards the target level of >95%, we must improve the low consent rates of these students. Consent is required by these school vaccination programmes in order to vaccinate the children, so without targeting the barriers to consenting we will not improve the actual rates of vaccinated individuals in the student population.

Vaxcards can be considered a desirable and ‘ethical’, non-monetary incentive to influence behaviour change directed towards adolescents being vaccinated. Collectables and gamification are important educational tools that can help children engage with learning, generate discussion and provide an incentive to engage with the content being delivered.²² The theoretical underpinnings of Vaxcards as an intervention are multifaceted. This is represented in our logic model of the theory of change (figure 2). Initially, the use of receiving Vaxcards as an incentive to return consent forms acts, from a behavioural standpoint, as a reward. As we previously mentioned, rewards have shown good effect in increasing vaccine uptake, but there have been no tangible take-home interventions directly designed for this age group. Second, Vaxcards acts as a social tool for students, parents, peers and teachers to interact and lower the barrier to discussing topics around vaccination and diseases. Third, it uses these educational points throughout the gameplay to increase knowledge of infectious diseases in the content of the game, increasing salience of their risks and the benefit of preventive vaccination against them. This requires the buy-in of the government or organisations to distribute Vaxcards alongside vaccination programmes and school stakeholders to also deliver and engage with the tool. The timely delivery and playing of the game should theoretically result in increased vaccine consent and uptake, conversations around vaccines and knowledge of the diseases, and increase vaccine confidence. The intervention drew on the principles of the use of incentives and took a pragmatic approach adding to existing council and school strategies to improve consent return rates, but was unable to impact return rates in this trial.

Figure 2.

Logic model (or theory of change) for Vaxcards.

The first limitation to improving consent is reducing the logistical barriers to returning the consent card. The card is provided in some schools the year before the vaccination occurs. This means that some students receive the cards in their last year of primary school, to bring home and have their parent’s sign, only to attempt returning the card to the child’s school when the new school year begins. Many students change schools during these 2 years and there is a risk of lost to follow-up when changing school systems.

Smaller schools did show a significant improvement in consent for vaccination, irrespective of previous vaccination consent levels. One explanation may be that smaller schools are better placed to communicate health promotion activities to students and parents given the individual concerns vaccination can generate. It could explain why larger schools are having trouble improving vaccine consent given the large difference between the 1-year time difference and the concurrent emergence of hesitancy in the community.

There is also an element of timing in the provision of the incentive and the intensity of the intervention. Perhaps exposure to the intervention earlier will lead to more of an impact. For example, receiving the Vaxcards with the educational packet from the council instead of as a reward for returning consent forms will enable a chance of impacting vaccine hesitancy to those who are most likely to require it, who never returned consent cards and were therefore not exposed to the intervention. It is also likely that the intensity was not enough. In this study we do not know how the participants used the cards, whether they engaged with the information in the cards or had an opportunity to clarify their understanding to learn more about them. Introduction or integration of the game in science or health classes could reinforce the educational aspects and impact on vaccine uptake, as evidenced in the Cochrane review of adolescent vaccine interventions.¹⁴

In a pragmatic sense it is unlikely a single intervention alone will largely change complex paradigm like vaccine hesitancy. In future trials, we would like to investigate the intervention in combination with other incentives and educational materials to determine if a multifaceted approach can shift consent rates.

Strengths and limitations

The outcomes of this trial were impacted by the unexpectedly large deviation in vaccination consent between the clusters of schools. The sample size was based in part on an estimated effect size, which may have been too optimistic. A larger study considering all these things is needed to more definitively determine the efficacy of Vaxcards as a stand-alone intervention when delivered as an ethical incentive for vaccination. We also do not know the vaccine outcome of children who did not return a consent form, declined school vaccine or ticked ‘had elsewhere’. It is possible these students did indeed get vaccinated outside the school programme and the ‘consent’ rates do not infer the true vaccine status of the group. Lastly, all students who returned consent cards were given a pack of Vaxcards, regardless of response. This decision was made pragmatically by the schools to not discriminate based on responses and to include all students. It also aligned with the incentives of the school, which are measured in total consent card return rate, not consent ‘yes’ return rates. This is an interesting point to possibly consider as a target to increase consent ‘yes’ rates, to change school performance indicators to align with public health outcomes rather than the return of forms regardless of outcome. Nevertheless, using Vaxcards as a reward for returning a consent card regardless of response may mean the incentive to return a form consenting ‘yes’ was diluted, impacting the main outcome measured.

A further limitation of this study is the insufficient cluster size required for statistical assessment. Also, it may not have been able to effectively control for bias in this study and so there are lessons for future studies of Vaxcards and of vaccine hesitancy in schools for a larger trial. There may have been an unrecorded data-reporting lag by the council if they were still waiting for consent cards to come in retrospectively after following up students within schools.

This trial was conducted in a real-world, pragmatic setting. Behaviour change interventions are complex in their nature due to the ways in which behaviours develop in different contexts for different individuals.^{27 28} The design of this trial may also have dilution the effect if the intervention efficacy that may impact the results. For this reason, a much larger trial involving more clusters to account for this dilution effect would be suitable to further assess the intervention.

Conclusions

Vaxcards is a novel intervention that addresses many recommendations made in the recent Cochrane review of effective interventions to improve vaccine uptake in adolescents.¹⁴ These elements include being an ethical incentive that can be incorporated into other health education and health promotion initiatives as part of multicomponent approaches to support vaccine uptake among school-aged children. This potential requires further investigation in order to assess its impact on vaccine uptake and vaccine confidence. Optimising the timing and the intensity of the intervention as part of a multimodal approach will be required to significantly shift hesitancy and improve consent rates and ultimately uptake of vaccination.

Data availability statement

Data are available upon reasonable request. Data available upon request.

Ethics statements

Patient consent for publication

Not required.

Ethics approval

The study was approved by the Monash University Research and Ethics Committee (Project ID 22340).