To address parental vaccination concerns and increase vaccine acceptance, we conducted an RCT to measure the effectiveness of a Web-based tailored messaging intervention.
Abstract
BACKGROUND:
To increase vaccine acceptance, we created a Web-based the “Vaccines and Your Baby” intervention (VAYB) that provided new parents with vaccine information messages tailored to vaccine beliefs and values. We evaluated the effectiveness of the VAYB by comparing timely uptake of infant vaccines to an untailored version of the intervention (UT) or usual care intervention (UC) only.
METHODS:
Between April 2016 and June 2019, we conducted a randomized clinical trial. Pregnant women and new parents were randomly assigned to the VAYB, UT, or UC arms. In the VAYB and UT arms, participants were exposed to interventions at 4 time points from pregnancy until their child was 15 months of age. The primary outcome was up-to-date status for recommended vaccines from birth to 200 days of age. A modified intent-to-treat analysis was conducted. Data were analyzed with logistic regression to generate odds ratios (ORs) and 95% confidence intervals (CIs).
RESULTS:
We enrolled 824 participants (276 VAYB, 274 UT, 274 UC), 143 (17.4%) of whom were lost to follow-up. The up-to-date rates in the VAYB, UT, and UC arms were 91.44%, 92.86%, and 92.31%, respectively. Infants in the VAYB arm were not more likely to be up to date than infants in the UC arm (OR = 0.89; 95% CI, 0.45–1.76) or in the UT arm (OR = 0.82; 95% CI, 0.42–1.63). The odds of being up to date did not differ between UT and UC arms (OR = 1.08; 95% CI, 0.54–2.18).
CONCLUSIONS:
Delivering Web-based vaccine messages tailored to parents’ vaccine attitudes and values did not positively impact the timely uptake of infant vaccines.
What’s Known on This Subject:
Many parents have concerns about vaccination for their children, leading some to delay or decline administration. Tailored messaging provides individualized messages to patients and positively impacted a wide variety of health behaviors. However, results are limited for childhood vaccinations.
What This Study Adds:
This randomized clinical trial tested the effectiveness of a tailored messaging intervention to increase the timely uptake of childhood vaccinations. Results indicate it was not an effective approach to address vaccine acceptance among new parents in an integrated health system.
Universal vaccination has led to the eradication or control of numerous potentially fatal infectious diseases.1 Despite this success, ∼1 in 10 parents choose to refuse or delay vaccines for their children,2–4 a behavior responsible for a number of vaccine preventable disease outbreaks across the United States.5
Although a large majority of parents choose to vaccinate their children, many parents report having concerns about childhood vaccines.6 Parents who are hesitant to vaccinate their children tend to question the safety and effectiveness of vaccines, despite the preponderance of evidence demonstrating that the benefits of vaccination greatly outweigh the risks. Vaccine-hesitant parents have expressed specific concerns that children receive too many vaccines in single well-child visits, as well as worries that vaccine ingredients, such as aluminum and formaldehyde, are toxic.7 Interventions designed to address such concerns and increase vaccine acceptance have been focused primarily on correcting a knowledge deficit, with limited success.7–9
Knowledge about the benefits and risks of vaccines is just one of several factors that affects vaccination decisions. Emotion, trust, and values also play important roles.10–12 Intervention strategies should therefore attempt to address the multiple determinants of vaccination decisions. One promising approach is message tailoring, which involves crafting messages to address each person’s unique attitudes, beliefs, experiences, knowledge, and values.13 Compared to nontailored information, tailored information is more likely to be read and remembered, thus rendering individuals more receptive to persuasion.14,15 Although message tailoring has been shown to improve a wide range of health behaviors, there are limited data on its impact on childhood vaccination behaviors.16,17 Moreover, the capacity to create and deliver tailored vaccine messages to parents in busy clinical settings may be limited, suggesting that methods to provide parents with the information in nonclinical settings are needed.
To improve vaccine acceptance, we designed and built the Web-based tailored messaging “Vaccines and Your Baby” intervention (VAYB).18 For this report, we evaluated the effectiveness of the VAYB with a randomized clinical trial (RCT) in which timely uptake of infant vaccines was compared across 3 study arms.
Methods
Study Overview
Between April 2016 and June 2019, we conducted a single-site RCT to evaluate the impact of the VAYB on timely childhood vaccination among infants of women recruited during the last trimester of pregnancy. The protocol for the study has been published previously but is summarized below.19
Participants were randomly assigned to 1 of 3 study arms: the VAYB, an untailored version of the intervention (UT), or usual care (UC) only. The intervention was administered at 4 time points across the follow-up: the last trimester of pregnancy or child age <2 months, at child age 4 to 6 months, at child age 10 to 12 months, and at child age 13 to 15 months. Our primary outcome was up-to-date status for vaccines administered from birth to 200 days of age. We hypothesized that infants of participants in the VAYB arm would be more likely to be up to date than infants of participants in the UT or UC arms. This study was approved by the Institutional Review Boards of Kaiser Permanente Colorado (KPCO), University of Colorado, and the University of Michigan.
Study Setting, Participants, and Random Assignment
Participants were recruited from KPCO, a nonprofit integrated health care system that serves ∼5000 pregnant women and 140 000 children across the Denver and Boulder region per year.
Recruitment was conducted in 6-week waves between April 2016 and December 2017. Potential participants were pregnant women identified by using the KPCO electronic health record (EHR). Women had to be in the third trimester of pregnancy, ≥18 years of age, English speaking, and enrolled in the KPCO health plan at the time of recruitment. Pregnant women were excluded if they had a diagnosis of fetal demise, miscarriage, congenital anomaly in the pregnancy, or high-risk maternal condition. Eligible women received a combination of mail, e-mail, and phone calls that directed them to a study registration Web site where they could consent into the study. The recruited woman or their partner could enroll in the study. Study participants were able to enroll from the last trimester of pregnancy to when their child was <2 months of age.
After consent, participants were directed to the study Web site where they created a username and password. They were then administered a validated preintervention survey to assess demographics, vaccination values, intentions to vaccinate, and vaccine attitudes and concerns.19,20 Vaccine hesitancy was measured with the Parent Attitudes About Childhood Vaccines short (PACV-short), a validated 5-item screening instrument used to assess hesitancy on a scale of 0 to 10.21 Participants scoring ≥5 were classified as “vaccine hesitant,” and those scoring <5 were “nonhesitant.” To ensure balance across study arms, random assignment was conducted within the 2 strata of vaccine hesitancy.
We conducted stratified random assignment with a permuted block technique. Participants were assigned to the VAYB, UT, and UC study arms by using a random allocation ratio of 1:1:1. Random assignment was conducted by using the SAS/STAT procedure PROC PLAN (SAS Institute, Inc, Cary, NC). Study participants were not blinded to the assigned study arm. Study investigators and statisticians were blinded to the study arm assignment for analysis.
Interventions
Conceptual Model
The intervention was based on a hybrid of the theory of planned behavior22 and the values-attitude-behavior model,23 combined with approaches from motivational interviewing and persuasive messaging.24,25 The values-attitude-behavior model complements the theory of planned behavior by stating that personal values affect attitudes. In contrast to attitudes and beliefs, values are thought to remain stable over time.26,27 The objective of the intervention was to influence vaccine attitudes and intentions through messages that address each participant’s vaccination attitudes and concerns, framed according to personal values (eg, acknowledging their “right to choose” or desire to “keep their child safe”). We crafted tailored messages that used empathetic language and emphasized autonomy to affirm individual values.28
Tailored Intervention
The Web-based tailored intervention was developed by using an iterative, user-driven approach that included surveys, one-on-one interviews, and usability testing. The details of this process have been described previously.18 Informational content for the intervention was derived from peer-reviewed sources and online materials provided by the Centers for Disease Control and Prevention and the American Academy of Pediatrics.
The messages conveying the information were tailored to each participants’ intention to vaccinate, personal attitudes about vaccination, vaccination values, and the child’s nickname, sex, and age. These data were collected from the preintervention survey, which activated an embedded algorithm to deliver the tailored messaging (Supplemental Information). The details for how the parent vaccine values survey was developed and evaluated have been described elsewhere.20 After participants completed the preintervention survey, they were automatically directed to the VAYB Web site, which was personalized on the basis of their survey responses. Information on the Web site was arranged across 9 clickable tiles. The top 3 tiles were prominently labeled “Just for You” and contained the most highly tailored content that was based on the participants’ vaccination values and top 3 vaccination concerns. The remaining content was lightly tailored on the basis of the participants’ other, less pressing concerns identified by their survey responses. The lightly tailored content did not incorporate vaccination values.
The intervention and surveys were administered again when the child was age 4 to 6, 10 to 12, and 13 to 15 months. The Web site was retailored and refreshed at each time point on the basis of the updated survey responses. Tailoring on attitudes was updated at all 3 follow-up time points, whereas tailoring on values was updated only at the 10- to 12-month time point.
UT
An untailored version of the VAYB Web site was created to isolate the effect of the tailoring. This version (UT) had the same design and factual information as the VAYB Web site, but it was not personalized to the participants’ survey responses, and the content did not change across the time points.
UC
Participants in all 3 study arms were eligible to receive standard pediatric preventive care at KPCO. This consists of scheduled 20-minute well-child visits at 2, 4, 6, and 12 months of age, with an option for a 9-month visit.29 Recommended childhood immunizations are administered at these health supervision visits, and it is standard practice at KPCO to offer parents Vaccine Information Statements relevant to that visit.30
Participants in the UT and UC arms were administered the same surveys at the 4 intervention time points as those received by participants in the VAYB arm.
Outcome
The primary outcome was immunization status assessed over the first 200 days of age to cover a majority of the routinely administered infant vaccines and to minimize loss to follow-up. Immunization data were extracted from the KPCO EHR for the following 6 vaccines recommended by the Advisory Committee on Immunization Practices: hepatitis B, rotavirus, diphtheria-tetanus-acellular pertussis, Haemophilus influenzae type b, pneumococcal conjugate, and inactivated poliovirus. Immunization status was quantified by using days undervaccinated, a continuous metric used to measure the differences between the ages at which the vaccine doses were actually administered and ages at which the doses should have been administered according to the Advisory Committee on Immunization Practices recommendations.2 For these 6 vaccines, days undervaccinated can range from 0 to 648 days.
Days undervaccinated was analyzed as a dichotomous and continuous variable. For the dichotomized measure, infants with 0 cumulative days undervaccinated were considered up to date, and infants with ≥1 days undervaccinated were considered not up to date.
Two secondary outcomes were also assessed. First, we examined up-to-date status for the measles-mumps-rubella (MMR) and varicella vaccines among infants with at least 489 days of follow-up, which is when days undervaccinated for the first doses of these vaccines begin to accrue. Second, we repeated the primary analysis excluding the hepatitis B vaccine from the assessment of up-to-date status. This latter analysis was conducted because the hepatitis B birth dose tends to be administered in the inpatient rather than outpatient setting.
Statistical Methods
The trial was powered to detect a difference in proportion of up-to-date status between 6.8% and 9.4% between the 3 study arms. This required a sample size of 477 participants for a difference of 9.4% (odds ratio [OR] = 3) and 1002 participants for a difference of 6.8% (OR = 2), assuming a baseline vaccine hesitancy of 15%, a 1:1:1 allocation ratio, 80% statistical power, and 2-sided α = .05.
We conducted a modified intent-to-treat analysis that adhered to study arm assignment but excluded infants if their vaccination data could not be completely ascertained over the 200-day follow-up. Infants were excluded if they died, were not enrolled in KPCO, enrolled in KPCO after 60 days of age, disenrolled from KPCO before the end of follow-up, or were not receiving primary care at KPCO.
For the primary analysis, up-to-date status and days undervaccinated were assessed from birth to 200 days of age. Up-to-date status was analyzed with logistic regression to estimate ORs and 95% confidence intervals (CIs). For days undervaccinated, we used a nonparametric analysis and rank transformation approach because the distribution of days undervaccinated was highly skewed to the right. Days undervaccinated were first ranked in increasing order of magnitude for all infants independent of study arm assignment, and then the mean ranks were compared across the study arms by using a 1-way analysis of variance. As a preplanned subgroup analysis, the data were stratified by baseline vaccine hesitancy status. As a post hoc subgroup analysis, up-to-date status was assessed across the study arms among participants with baseline PACV-short scores between 4 and 7. Participants with these scores may represent vaccine “fence-sitters,” defined as parents who are willing to vaccinate but also have concerns about vaccines.31,32
For the secondary analyses, logistic regression was used to assess up-to-date status for the MMR and varicella vaccines among infants with at least 489 days of follow-up and to assess up-to-date status excluding the hepatitis B vaccine. Analyses were conducted with SAS software 9.4.
Results
Study Participants and Baseline Characteristics
In the KPCO cohort, there were 4929 pregnant women who were potentially eligible for the study (Fig 1). After exclusions, there were 824 participants enrolled in the study, 795 (96.5%) of whom were recruited during pregnancy, and 29 (3.5%) were recruited when their child was <2 months of age (Table 1). According to the 1:1:1 randomization ratio, 276 were randomly assigned to the VAYB arm, 274 were randomly assigned to the UT arm, and 274 were randomly assigned to the UC arm (Fig 1). Baseline characteristics were evenly distributed across study arms (Table 1). For the entire study population, the mean age at enrollment was 32 years, 85% were white, 12% were Hispanic, and 85.3% were college-educated. At baseline, 14.3% of the participants were categorized as being vaccine hesitant, with a mean hesitancy score of 2.25 (SD = 2.21).
FIGURE 1.
Consolidated Standards of Reporting Trials (CONSORT) study flow diagram.
TABLE 1.
Baseline Characteristics of Enrolled Members, by Study Arm (N = 824)
| Characteristica | Total (n = 824) | VAYB (n = 276) | UT (n = 274) | UC (n = 274) |
|---|---|---|---|---|
| Enrolled during pregnancy, n (%)b | 795 (96.48) | 268 (97.10) | 267 (97.45) | 260 (94.89) |
| Female sex, n (%)b | 821 (99.64) | 274 (99.28) | 273 (99.64) | 274 (100) |
| Parent’s age at enrollment, y, mean (SD) | 31.99 (4.37) | 31.96 (4.49) | 32.20 (4.22) | 31.81 (4.41) |
| Race, n (%)b | ||||
| White | 703 (85.32) | 233 (84.42) | 233 (85.04) | 237 (86.50) |
| Black | 8 (0.97) | 3 (1.09) | 4 (1.46) | 1 (0.36) |
| Asian American or Pacific Islander | 32 (3.88) | 14 (5.07) | 10 (3.65) | 8 (2.92) |
| Multiracial or other | 65 (7.89) | 21 (7.61) | 23 (8.39) | 21 (7.66) |
| Nonresponse | 16 (1.94) | 5 (1.81) | 4 (1.46) | 7 (2.55) |
| Ethnicity, n (%)b | ||||
| Hispanic | 100 (12.14) | 39 (14.13) | 31 (11.31) | 30 (10.95) |
| Non-Hispanic | 714 (86.65) | 232 (84.06) | 241 (87.96) | 241 (87.96) |
| Nonresponse | 10 (1.21) | 5 (1.81) | 2 (0.73) | 3 (1.09) |
| No. children, n (%)b | ||||
| Pregnant with first child | 155 (18.81) | 52 (18.84) | 54 (19.71) | 49 (17.88) |
| One child | 503 (61.04) | 167 (60.51) | 164 (59.85) | 172 (62.77) |
| Two children | 111 (13.47) | 41 (14.86) | 36 (13.14) | 34 (12.41) |
| Three or more children | 55 (6.67) | 16 (5.80) | 20 (7.30) | 19 (6.93) |
| Household income, n (%)b | ||||
| <$40 000 | 59 (7.16) | 21 (7.61) | 18 (6.57) | 20 (7.30) |
| $40 000–$80 000 | 222 (26.94) | 84 (30.43) | 66 (24.09) | 72 (26.28) |
| $81 000–$120 000 | 299 (36.29) | 95 (34.42) | 105 (38.32) | 99 (36.13) |
| $121 000–$150 000 | 83 (10.07) | 26 (9.42) | 27 (9.85) | 30 (10.95) |
| >$150 000 | 129 (15.66) | 38 (13.77) | 49 (17.88) | 42 (15.33) |
| Nonresponse | 32 (3.88) | 12 (4.35) | 9 (3.28) | 11 (4.01) |
| Employment, n (%)b | ||||
| Employed full-time | 567 (68.81) | 188 (68.12) | 187 (68.25) | 192 (70.07) |
| Employed part-time | 112 (13.59) | 41 (14.86) | 40 (14.60) | 31 (11.31) |
| Unemployed | 13 (1.58) | 7 (2.54) | 2 (0.73) | 4 (1.46) |
| Stay-at-home parent | 123 (14.93) | 35 (12.68) | 42 (15.33) | 46 (16.79) |
| Student | 8 (0.97) | 5 (1.81) | 2 (0.73) | 1 (0.36) |
| Nonresponse | 1 (0.12) | 0 (0) | 1 (0.36) | 0 (0) |
| Marital status, n (%)b | ||||
| Married | 716 (86.89) | 236 (85.51) | 239 (87.23) | 241 (87.96) |
| Separated | 3 (0.36) | 2 (0.72) | 1 (0.36) | 0 (0) |
| Divorced | 7 (0.85) | 2 (0.72) | 1 (0.36) | 4 (1.46) |
| Not married | 13 (1.58) | 3 (1.09) | 7 (2.55) | 3 (1.09) |
| Single | 14 (1.70) | 3 (1.09) | 6 (2.19) | 5 (1.82) |
| Living with a partner | 71 (8.62) | 30 (10.87) | 20 (7.30) | 21 (7.66) |
| Education, n (%)b | ||||
| Grade school | 6 (0.73) | 1 (0.36) | 2 (0.73) | 3 (1.09) |
| High school | 19 (2.31) | 9 (3.26) | 6 (2.19) | 4 (1.46) |
| Some college | 92 (11.17) | 35 (12.68) | 28 (10.22) | 29 (10.58) |
| College | 323 (39.20) | 105 (38.04) | 103 (37.59) | 115 (41.97) |
| Graduate school | 380 (46.12) | 125 (45.29) | 134 (48.91) | 121 (44.16) |
| Nonresponse | 4 (0.49) | 1 (0.36) | 1 (0.36) | 2 (0.73) |
| Hesitancy by PACV-short scale,c mean (SD) | 2.25 (2.21) | 2.21 (2.36) | 2.25 (2.10) | 2.29 (2.18) |
| Hesitant by PACV-short scale, n (%)b,c, d | 118 (14.32) | 39 (14.13) | 39 (14.23) | 40 (14.60) |
No statistically significant differences in characteristics across study arms.
Percentages represent column percentages.
PACV-short scale is a validated 5-question screening tool assessing vaccine hesitancy on a scale from 0 to 10; higher scores indicate a higher degree of hesitancy.
Participants scoring ≥5 (on a scale from 0 to 10) on the PACV-short scale were classified as vaccine hesitant.
After random assignment, incomplete vaccination data led to 143 (17.4%) exclusions (Fig 1). These exclusions were considered loss to follow-up, which ranged from 13.1% to 19.6% across the study arms (P = .08). The mean vaccine hesitancy scores of the participants lost to follow-up did not differ significantly across the study arms (P = .92).
Effectiveness
The proportions of infants who were up to date at age 200 days were 91.4%, 92.9%, and 92.3% for the VAYB, UT, and UC arms, respectively. The 95% CIs for the ORs comparing up-to-date status between study arms all included 1 and were not statistically significant (Table 2). Infants in the VAYB were not more likely to be up to date than infants in the UC arm (OR = 0.89; 95% CI, 0.45–1.76), nor were they more likely to be up to date than infants in the UT arm (OR = 0.82; 95% CI, 0.42–1.63). Similarly, the odds of being up to date did not differ between the UT and UC arms (OR = 1.08; 95% CI, 0.54–2.18). In the overall KPCO cohort of potentially eligible participants who were not enrolled in the study (n = 3366), the proportion of infants who were up to date during the study period was 85.6%.
TABLE 2.
Proportion of Infants Up to Date for Vaccination Status and OR Estimates for Up-To-Date Vaccination Status Between Study Arms: All Infants in the Study (N = 681)
| Study Arm | Infants Up to Date, n (%) | Study Arm Comparisons | OR for Up-to-Date Vaccination Status (95% CI) | P |
|---|---|---|---|---|
| VAYB (n = 222) | 203 (91.44) | VAYB versus UC | 0.89 (0.45–1.76) | .74 |
| UT (n = 238) | 221 (92.86) | UT versus UC | 1.08 (0.54–2.18) | .82 |
| UC (n = 221) | 204 (92.31) | VAYB versus UT | 0.82 (0.42–1.63) | .57 |
In the subanalysis stratified by vaccine hesitancy status, infants of vaccine-hesitant parents in the VAYB arm were less likely to be up to date than infants of vaccine-hesitant parents in the UT arm (OR = 0.28; 95% CI, 0.08–0.98) (Table 3). The odds of being up to date did not differ between the VAYB and UC arms (OR = 0.67; 95% CI, 0.23–1.96) or between the UT and UC arms (OR = 2.42; 95% CI, 0.65–9.01). Among infants of parents who were not vaccine-hesitant, the odds of being up-to-date did not differ between the study arms in any of the comparisons (Table 4). In the post hoc subgroup analysis of potential fence-sitters, the OR for up-to-date status comparing the VAYB to UC arm was not statistically significant (OR = 0.69; 95% CI, 0.16–3.01).
TABLE 3.
Proportion of Infants Up to Date for Vaccination Status and OR Estimates for Up-To-Date Vaccination Status Between Study Arms: When Parents Were Vaccine Hesitant at Baseline (n = 98)
| Study Arm | Infants Up to Date, n (%) | Study Arm Comparisons | OR for Up-to-Date Vaccination Status (95% CI) | P |
|---|---|---|---|---|
| VAYB (n = 33) | 22 (66.67) | VAYB versus UC | 0.67 (0.23–1.96) | .46 |
| UT (n = 33) | 29 (87.88) | UT versus UC | 2.42 (0.65–9.01) | .19 |
| UC (n = 32) | 24 (75.0) | VAYB versus UT | 0.28 (0.08–0.98) | .05 |
TABLE 4.
Proportion of Infants Up to Date for Vaccination Status and OR Estimates for Up-To-Date Vaccination Status Between Study Arms: When Parents Were Not Vaccine Hesitant at Baseline (n = 583)
| Study Arm | Infants Up to Date, n (%) | Study Arm Comparisons | OR for Up-to-Date Vaccination Status (95% CI) | P |
|---|---|---|---|---|
| VAYB (n = 189) | 181 (95.77) | VAYB versus UC | 1.13 (0.43–3.0) | .80 |
| UT (n = 205) | 192 (93.66) | UT versus UC | 0.74 (0.31–1.77) | .50 |
| UC (n = 189) | 180 (95.24) | VAYB versus UT | 1.53 (0.62–3.78) | .35 |
For the secondary analysis of MMR and varicella vaccination status at age 489 days, the ORs for up-to-date status comparing the VAYB to UC arm were not statistically significant (ORMMR = 1.18; 95% CI, 0.49–2.84; ORvaricella = 1.06; 95% CI, 0.41–2.74) (Tables 5 and 6). For the analysis excluding the hepatitis B vaccine, the OR for up-to-date status comparing the VAYB to UC arm was not statistically significant (OR = 0.94; 95% CI, 0.47–1.88).
TABLE 5.
Proportion of Infants Up to Date on the First Dose of MMR Vaccine and OR Estimates for Up-To-Date MMR Vaccination Status Between Study Arms (n = 590)
| Study Arm | Infants Up to Date, n (%) | Study Arm Comparisons | OR for Up-to-Date Vaccination Status (95% CI) | P |
|---|---|---|---|---|
| VAYB (n = 194) | 184 (94.85) | VAYB versus UC | 1.18 (0.49–2.84) | .72 |
| UT (n = 213) | 204 (95.77) | UT versus UC | 1.45 (0.59–3.58) | .42 |
| UC (n = 183) | 172 (93.99) | VAYB versus UT | 0.81 (0.32–2.04) | .66 |
TABLE 6.
Proportion of Infants Up to Date on the First Dose of Varicella Vaccine and OR Estimates for Up-To-Date Varicella Vaccination Status Between Study Arms:(n = 590)
| Study Arm | Infants Up to Date, n (%) | Study Arm Comparisons | OR for Up-to-Date Vaccination Status (95% CI) | P |
|---|---|---|---|---|
| VAYB (n = 194) | 185 (95.36) | VAYB versus UC | 1.06 (0.41–2.74) | .90 |
| UT (n = 213) | 202 (94.84) | UT versus UC | 0.95 (0.39–2.35) | .91 |
| UC (n = 183) | 174 (95.08) | VAYB versus UT | 1.12 (0.45–2.76) | .81 |
The mean rank scores for days undervaccinated were 344.3, 338.7, and 340.2 for the VAYB, UT, and UC arms, respectively. These mean rank scores did not differ significantly for any of the study arm comparisons. P values for the VAYB versus UC, UT versus UC, and VAYB versus UT arm comparisons were 0.64, 0.87, and 0.52, respectively (Table 7). In the subgroup analysis stratified by vaccine hesitancy status, the mean rank scores did not differ for 5 of the 6 study arm comparisons (Tables 8 and 9). The only significant difference was between infants of vaccine-hesitant parents in the VAYB and the UT arms is infants of vaccine-hesitant parents in the VAYB arm had significantly more days undervaccinated than those of infants of hesitant parents in the UT arm (mean rank difference = 11.06; P = .03) (Table 8).
TABLE 7.
Days Undervaccinated, Mean Ranks for Days Undervaccinated, and Difference in the Mean Ranks Between Study Arms: All Infants in the Study (N = 681)
| Study Arm | Days Undervaccinated Percentiles | Mean Ranksa | Study Arm Comparisons | Difference in Mean Ranks | Pb | ||
|---|---|---|---|---|---|---|---|
| 5th | 50th | 95th | |||||
| VAYB (n = 222) | 0 | 0 | 230 | 344.27 | VAYB versus UC | 4.10 | .64 |
| UT (n = 238) | 0 | 0 | 110 | 338.72 | UT versus UC | −1.44 | .87 |
| UC (n = 221) | 0 | 0 | 46 | 340.17 | VAYB versus UT | 5.54 | .52 |
Obtained by ranking all observations in increasing order of magnitude of days undervaccinated and calculating the mean.
Obtained by using 1-way analysis of variance on the ranks for days undervaccinated.
TABLE 8.
Days Undervaccinated, Mean Ranks for Days Undervaccinated, and Difference in the Mean Ranks Between Study Arms: Infants of Parents Who Were Vaccine Hesitant at Baseline (n = 98)
| Study Arm | Days Undervaccinated Percentiles | Mean Ranksa | Study Arm Comparisons | Difference in Mean Ranks | Pb | ||
|---|---|---|---|---|---|---|---|
| 5th | 50th | 95th | |||||
| VAYB (n = 33) | 0 | 0 | 642 | 54.82 | VAYB versus UC | 4.88 | .35 |
| UT (n = 33) | 0 | 0 | 367 | 43.76 | UT versus UC | −6.18 | .23 |
| UC (n = 32) | 0 | 0 | 642 | 49.94 | VAYB versus UT | 11.06 | .03 |
Obtained by ranking all observations in increasing order of magnitude of days undervaccinated and calculating the mean.
Obtained by using 1-way analysis of variance on the ranks for days undervaccinated.
TABLE 9.
Days Undervaccinated, Mean Ranks for Days Undervaccinated, and Difference in the Mean Ranks Between Study Arms: Infants of Parents Not Vaccine Hesitant at Baseline (n = 583)
| Study Arm | Days Undervaccinated Percentiles | Mean Ranksa | Study Arm Comparisons | Difference in Mean Ranks | Pb | ||
|---|---|---|---|---|---|---|---|
| 5th | 50th | 95th | |||||
| VAYB (n = 189) | 0 | 0 | 0 | 289.58 | VAYB versus UC | −0.98 | .88 |
| UT (n = 205) | 0 | 0 | 107 | 295.55 | UT versus UC | 4.99 | .44 |
| UC (n = 189) | 0 | 0 | 0 | 290.57 | VAYB versus UT | −5.97 | .36 |
Obtained by ranking all observations in increasing order of magnitude of days undervaccinated and calculating the mean.
Obtained by using 1-way analysis of variance on the ranks for days undervaccinated.
Discussion
In this RCT, provision of a Web-based tailored messaging intervention to expectant and new parents did not have a positive effect on the timely uptake of infant immunizations. Infants of parents in the VAYB arm were not more likely to be up to date for routinely administered vaccines or have fewer days undervaccinated over the first 200 days of life than infants of parents in the UT or UC study arms. These results do not support our hypothesis that the behaviors of parents can be influenced by carefully crafted messages tailored to their vaccination attitudes and values.
In several studies, authors have examined the impact of tailored messaging on vaccine attitudes, intentions, and behaviors related to human papillomavirus and influenza vaccination.33–38 They tended to focus on Web-based formats or text messaging to deliver the tailored messages, and outcomes were assessed in adolescent girls, parents of adolescent girls, or adults. The results varied, with only 1 RCT revealing a positive effect on both initiation and completion of human papillomavirus vaccination series in adolescent girls.33 However, it is difficult to relate the VAYB results to these investigations because the study populations, intervention contents, and vaccination outcomes were considerably different. Perhaps most importantly, the other investigations were focused on 1 specific vaccine, whereas the VAYB attempted to address concerns across multiple vaccines.
This study had many strengths. The VAYB, survey instruments, and data collection tools were all developed by using iterative, theory- and user-driven approaches led by a multidisciplinary team.18 The effectiveness of the intervention was evaluated with a rigorous randomized design, and the study population was recruited during pregnancy, representing a key time period for infant vaccination decisions.39,40 In addition, rather than relying on recall, vaccination outcomes were extracted from the EHR, which captures vaccine administrations with a high degree of accuracy.41,42
The subgroup analysis produced an unexpected result: among vaccine-hesitant participants, the UT arm was significantly more effective than the VAYB arm. Infants in the VAYB arm were significantly less likely to be up to date and had more days undervaccinated than infants in the UT arm. It is possible that our inferences based on the participants’ vaccine values and attitudes overreached and led to messages that were less relevant, thus undermining the credibility of the tailoring.43 In addition, we conducted 9 comparisons in the analysis without a correction for multiple comparisons, suggesting that this subgroup association could be significant by chance alone. Although we did not plan for a Bonferroni multiple comparison test a priori, such a correction would have rendered this association nonsignificant at the α < .05 level.
Although tailored messaging has been shown to be effective at eliciting health behavior change, the magnitude of effect sizes from randomized studies have been small.15 In integrated health systems such as KPCO, detecting modest effect sizes for infant vaccine acceptance is challenging because vaccination rates in these settings tend to be high.44 In our intervention trial, >90% of the infants across all 3 intervention arms were up to date, and 14% of the study participants were classified as vaccine hesitant at baseline. Although this rate of vaccine hesitancy aligns with other published estimates,4,45,46 it is possible that our intervention could have been effective if it had been implemented in other health care environments with lower infant vaccination rates and higher rates of vaccine hesitancy.
This study also had limitations. The trial was conducted in a single integrated health plan in Colorado, which may have affected the trial’s generalizability. Infants in the trial were more likely to be up to date than infants in the entire KPCO population during the study period (92.2% vs 85.6%), which may have also affected the generalizability of the trial. Overall, the loss to follow-up was 17.4%, largely attributed to parents who disenrolled from KPCO after their child was born. Although this may have also impacted the generalizability of the trial, the loss to follow-up did not differ significantly across the study arms. In addition, the rate of vaccine hesitancy among those lost to follow-up was similar across the arms, suggesting that these exclusions did not affect the internal validity of the results.
We designed the intervention to tailor messages on both vaccine values and attitudes. Although this is a potential strength of the intervention, it also possible that for some participants, the tailored content was on target for one construct but off target for the other construct. Such discrepant effects could explain the null results observed in the primary analysis. However, this could not be formally evaluated because we did not design the trial to evaluate the effects of the 2 types of tailoring separately.
Conclusions
This RCT suggests that Web-based tailored messaging does not positively impact parental immunization behaviors during infancy. Given that vaccine hesitancy remains a significant public health issue,47 researchers should continue to develop and test communication approaches to reduce parental vaccination concerns and improve the timely uptake of infant immunizations.
Acknowledgments
We acknowledge Courtney Kraus and Kathy Gleason who recruited participants and collected data for the trial.
Glossary
- CI
confidence interval
- EHR
electronic health record
- KPCO
Kaiser Permanente Colorado
- MMR
measles-mumps-rubella
- OR
odds ratio
- PACV-short
Parent Attitudes About Childhood Vaccines short
- RCT
randomized clinical trial
- UC
usual care intervention
- UT
untailored intervention
- VAYB
Vaccines and Your Baby intervention
Footnotes
Dr Glanz conceptualized and designed the study and drafted the initial manuscript; Dr Wagner conceptualized the study, designed the data collection protocols and instruments, and managed data acquisition; Dr Narwaney conceptualized the study, conducted the analyses, and interpreted the data; Ms Pyrzanowski conceptualized the study design; Mr Sevick contributed expertise to the analytic plan and conducted data interpretation; Drs Kwan and Resnicow conceptualized and designed the intervention and interpreted the data; Dr Dempsey conceptualized and designed the study and interpreted the data; and all authors reviewed and revised the manuscript, approved the final manuscript as submitted, and agree to be accountable for all aspects of the work.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
Deidentified participant data will not be made available.
This trial has been registered at www.clinicaltrials.gov (identifier: NCT02665013).
FINANCIAL DISCLOSURE: Dr Dempsey serves on the advisory boards for Merck, Sanofi, and Pfizer and has worked as a consultant for Pfizer. None of these companies played a role in this research, and Dr Dempsey does not receive research funding from these groups; the other authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Funded by the National Institutes of Health, Eunice Kennedy Shriver National Institute of Child Health and Human Development (R01 HD079457). Funded by the National Institutes of Health (NIH).
References
- 1.Centers for Disease Control and Prevention Ten great public health achievements--United States, 2001–2010. MMWR Morbidity and Mortality Weekly Report. 2011;60(19):619–623 [PubMed] [Google Scholar]
- 2.Glanz JM, Newcomer SR, Narwaney KJ, et al. A population-based cohort study of undervaccination in 8 managed care organizations across the United States. JAMA Pediatr. 2013;167(3):274–281 [DOI] [PubMed] [Google Scholar]
- 3.Dempsey AF, Schaffer S, Singer D, Butchart A, Davis M, Freed GL. Alternative vaccination schedule preferences among parents of young children. Pediatrics. 2011;128(5):848–856 [DOI] [PubMed] [Google Scholar]
- 4.Kempe A, Saville AW, Albertin C, et al. Parental hesitancy about routine childhood and influenza vaccinations: a national survey. Pediatrics. 2020;146(1):e20193852. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Phadke VK, Bednarczyk RA, Salmon DA, Omer SB. Association between vaccine refusal and vaccine-preventable diseases in the United States: a review of measles and pertussis. JAMA. 2016;315(11):1149–1158 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Kennedy A, Lavail K, Nowak G, Basket M, Landry S. Confidence about vaccines in the United States: understanding parents’ perceptions. Health Aff (Millwood). 2011;30(6):1151–1159 [DOI] [PubMed] [Google Scholar]
- 7.Salmon DA, Dudley MZ, Glanz JM, Omer SB. Vaccine hesitancy: causes, consequences, and a call to action. Vaccine. 2015;33(suppl 4):D66–D71 [DOI] [PubMed] [Google Scholar]
- 8.Sadaf A, Richards JL, Glanz J, Salmon DA, Omer SB. A systematic review of interventions for reducing parental vaccine refusal and vaccine hesitancy. Vaccine. 2013;31(40):4293–4304 [DOI] [PubMed] [Google Scholar]
- 9.Jarrett C, Wilson R, O’Leary M, Eckersberger E, Larson HJ; SAGE Working Group on Vaccine Hesitancy . Strategies for addressing vaccine hesitancy - a systematic review. Vaccine. 2015;33(34):4180–4190 [DOI] [PubMed] [Google Scholar]
- 10.MacDonald NE; SAGE Working Group on Vaccine Hesitancy . Vaccine hesitancy: definition, scope and determinants. Vaccine. 2015;33(34):4161–4164 [DOI] [PubMed] [Google Scholar]
- 11.Amin AB, Bednarczyk RA, Ray CE, et al. Association of moral values with vaccine hesitancy. Nat Hum Behav. 2017;1(12):873–880 [DOI] [PubMed] [Google Scholar]
- 12.Witteman HO. Addressing vaccine hesitancy with values. Pediatrics. 2015;136(2):215–217 [DOI] [PubMed] [Google Scholar]
- 13.Hawkins RP, Kreuter M, Resnicow K, Fishbein M, Dijkstra A. Understanding tailoring in communicating about health. Health Educ Res. 2008;23(3):454–466 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Skinner CS, Strecher VJ, Hospers H. Physicians’ recommendations for mammography: do tailored messages make a difference? Am J Public Health. 1994;84(1):43–49 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Campbell MK, DeVellis BM, Strecher VJ, Ammerman AS, DeVellis RF, Sandler RS. Improving dietary behavior: the effectiveness of tailored messages in primary care settings. Am J Public Health. 1994;84(5):783–787 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Noar SM, Benac CN, Harris MS. Does tailoring matter? Meta-analytic review of tailored print health behavior change interventions. Psychol Bull. 2007;133(4):673–693 [DOI] [PubMed] [Google Scholar]
- 17.Lustria MLA, Cortese J, Noar SM, Glueckauf RL. Computer-tailored health interventions delivered over the Web: review and analysis of key components. Patient Educ Couns. 2009;74(2):156–173 [DOI] [PubMed] [Google Scholar]
- 18.Dempsey A, Kwan BM, Wagner NM, et al. A values-tailored web-based intervention for new mothers to increase infant vaccine uptake: development and qualitative study. J Med Internet Res. 2020;22(3):e15800. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Dempsey AF, Wagner N, Narwaney K, et al. ‘Reducing Delays In Vaccination’ (REDIVAC) trial: a protocol for a randomised controlled trial of a web-based, individually tailored, educational intervention to improve timeliness of infant vaccination. BMJ Open. 2019;9(5):e027968. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Cataldi JR, Sevick C, Pyrzanowski J, et al. Addressing personal parental values in decisions about childhood vaccination: measure development. Vaccine. 2019;37(38):5688–5697 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Oladejo O, Allen K, Amin A, Frew PM, Bednarczyk RA, Omer SB. Comparative analysis of the Parent Attitudes about Childhood Vaccines (PACV) short scale and the five categories of vaccine acceptance identified by Gust et al. Vaccine. 2016;34(41):4964–4968 [DOI] [PubMed] [Google Scholar]
- 22.Ajzen I. The theory of planned behavior. Organ Behav Hum Decis Process. 1991;50(2):179–211 [Google Scholar]
- 23.Homer PM, Kahle LR. A structural equation test of the value-attitude-behavior hierarchy. J Pers Soc Psychol. 1988;54(4):638 [Google Scholar]
- 24.Rollnick S, Miller WR. What is motivational interviewing? Behav Cogn Psychother. 1995;23(4):325–334 [DOI] [PubMed] [Google Scholar]
- 25.Sweeney AM, Moyer A. Self-affirmation and responses to health messages: a meta-analysis on intentions and behavior. Health Psychol. 2015;34(2):149–159 [DOI] [PubMed] [Google Scholar]
- 26.Konty MA, Dunham CC. Differences in value and attitude change over the life course. Sociol Spectr. 1997;17(2):177–197 [Google Scholar]
- 27.Schwartz SH. Are there universal aspects in the structure and contents of human values? J Soc Issues. 1994;50(4):19–45 [Google Scholar]
- 28.Millner W, Rollnick S. Motivational Interviewing: Preparing People to Change Addictive Behaviors. New York, NY: Guilford Press; 1991 [Google Scholar]
- 29.Hagan JF Jr., Shaw JS, Duncan PM, eds.. Bright Futures: Guidelines for Health Supervision of Infants, Children, and Adolescents, 3rd ed Elk Grove Village, IL: American Academy of Pediatrics; 2008 [Google Scholar]
- 30.Centers for Disease Control and Prevention. Vaccine information statements (VISs): current VISs. 2019. Available at: https://www.cdc.gov/vaccines/hcp/vis/current-vis.html. Accessed December 11, 2019
- 31.Leask J. Target the fence-sitters. Nature. 2011;473(7348):443–445 [DOI] [PubMed] [Google Scholar]
- 32.Gust D, Brown C, Sheedy K, Hibbs B, Weaver D, Nowak G. Immunization attitudes and beliefs among parents: beyond a dichotomous perspective. Am J Health Behav. 2005;29(1):81–92 [DOI] [PubMed] [Google Scholar]
- 33.Dempsey AF, Pyrznawoski J, Lockhart S, et al. Effect of a health care professional communication training intervention on adolescent human papillomavirus vaccination: a cluster randomized clinical trial. JAMA Pediatr. 2018;172(5):e180016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Jordan ET, Bushar JA, Kendrick JS, Johnson P, Wang J. Encouraging influenza vaccination among Text4baby pregnant women and mothers. Am J Prev Med. 2015;49(4):563–572 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.DiClemente RJ, Murray CC, Graham T, Still J. Overcoming barriers to HPV vaccination: a randomized clinical trial of a culturally-tailored, media intervention among African American girls. Hum Vaccin Immunother. 2015;11(12):2883–2894 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Gerend MA, Shepherd MA, Lustria ML. Increasing human papillomavirus vaccine acceptability by tailoring messages to young adult women’s perceived barriers. Sex Transm Dis. 2013;40(5):401–405 [DOI] [PubMed] [Google Scholar]
- 37.Pot M, Paulussen TG, Ruiter RA, et al. Effectiveness of a web-based tailored intervention with virtual assistants promoting the acceptability of HPV vaccination among mothers of invited girls: randomized controlled trial. J Med Internet Res. 2017;19(9):e312. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Bennett AT, Patel DA, Carlos RC, et al. Human papillomavirus vaccine uptake after a tailored, online educational intervention for female university students: a randomized controlled trial. J Womens Health (Larchmt). 2015;24(11):950–957 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Glanz JM, Wagner NM, Narwaney KJ, et al. A mixed methods study of parental vaccine decision making and parent-provider trust. Acad Pediatr. 2013;13(5):481–488 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Lieu TA, Zikmund-Fisher BJ, Chou C, Ray GT, Wittenberg E. Parents’ perspectives on how to improve the childhood vaccination process. Clin Pediatr (Phila). 2017;56(3):238–246 [DOI] [PubMed] [Google Scholar]
- 41.Mullooly J, Drew L, DeStefano F, et al.; Vaccine Safety DataLink Team . Quality of HMO vaccination databases used to monitor childhood vaccine safety. Am J Epidemiol. 1999;149(2):186–194 [DOI] [PubMed] [Google Scholar]
- 42.Daley MF, Glanz JM, Newcomer SR, et al. Assessing misclassification of vaccination status: implications for studies of the safety of the childhood immunization schedule. Vaccine. 2017;35(15):1873–1878 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Kreuter MW, Oswald DL, Bull FC, Clark EM. Are tailored health education materials always more effective than non-tailored materials? Health Educ Res. 2000;15(3):305–315 [DOI] [PubMed] [Google Scholar]
- 44.Baggs J, Gee J, Lewis E, et al. The Vaccine Safety Datalink: a model for monitoring immunization safety. Pediatrics. 2011;127(suppl 1):S45–S53 [DOI] [PubMed] [Google Scholar]
- 45.Opel DJ, Taylor JA, Zhou C, Catz S, Myaing M, Mangione-Smith R. The relationship between Parent Attitudes About Childhood Vaccines survey scores and future child immunization status: a validation study. JAMA Pediatr. 2013;167(11):1065–1071 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Henrikson NB, Anderson ML, Opel DJ, Dunn J, Marcuse EK, Grossman DC. Longitudinal trends in vaccine hesitancy in a cohort of mothers surveyed in Washington State, 2013–2015. Public Health Rep. 2017;132(4):451–454 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Patel M, Lee AD, Clemmons NS, et al. National update on measles cases and outbreaks - United States, January 1–October 1, 2019. MMWR Morb Mortal Wkly Rep. 2019;68(40):893–896 [DOI] [PMC free article] [PubMed] [Google Scholar]