Immunization effects of a communication intervention to promote preteen HPV vaccination in primary care practices

Abstract

Objectives

HPV vaccination at the recommended ages of 11–12 is highly effective yet has stalled well below the goal of 80% of the population. We evaluated a statewide practice-based communication intervention (tools: brochures, posters, online training for providers and resources for parents, video game for preteens) to persuade parents, preteens and providers to vaccinate against HPV. The 9-month intervention started May 1, 2015.

Methods

We compared vaccine initiation and completion rates over three 9-month periods (baseline, intervention, post-intervention) between practices enrolled in the intervention and a comparable comparison group. All practices reported to the North Carolina Immunization Registry (NCIR) and had at least 100 11- and 12-year-olds who had not completed the HPV vaccine series. Of 175 eligible practices, the 14 intervention practices included 19,398 individuals and the 161 comparison practices included 127,896 individuals. An extended Cox model was used to test the intervention effect.

Results

The intervention had a significant effect on both initiation and completion during the intervention and post-intervention periods; the estimated hazard ratio (HR) for initiation was 1.17 (p = .004) during the intervention and 1.11 (p = .005) post-intervention. Likewise, completion during the intervention period was 17% higher in intervention practices, after controlling for baseline differences. This effect increased in the post-intervention period to 30% higher (p = .03).

Conclusions

Individuals in the intervention practices were 17% more likely to initiate and complete HPV vaccination than in the comparison practices during the intervention period and the effect was sustained post-intervention. This intervention is promising for increasing rates of HPV vaccination at ages 11–12.

1. Introduction

A vaccine to prevent the acquisition of human papillomavirus (HPV) types that cause genital warts and cancers was recommended by the Advisory Committee on Immunization Practices (ACIP) for routine use in females in 2006 and in males in 2011 [1,2]. More than ten years later, uptake of the vaccine has stalled well below the goal of 80% of the population [3], with only 50% of females and 38% of males ages 13–17 in the United States having completed the 3-dose series in 2016 [4]. In October 2016, ACIP recommended a 2-dose series if the HPV vaccine was initiated before the age of 15 [5]. Preteens can receive HPV vaccine at the same clinical visit as Tdap and meningococcal vaccines yet often do not [4]. The result is that a highly effective medical innovation to prevent HPV-related disease goes unused in a large segment of the population [3].

Many parents may not consent to HPV vaccination because of concern about safety, side effects and possibly encouraging early sexual activity [6]. Many providers do not stress early vaccination as most effective [7–9] Other clinical care setting interventions to promote HPV vaccination have focused on (1) stressing cancer prevention [10], (2) bundling recommendation of HPV vaccination with other adolescent vaccinations (Tdap and meningococcal) [11], (3) training providers to issue a strong, presumptive recommendation [9], (4) assessing the effect of “an announcement” vs a “conversation” with parents [12], and (5) helping with quality control efforts in the clinic through medical record searches and reminders, text messages and training [13].

With extensive input from parents, preteens and providers [14,15], we developed tools to encourage communication and persuade parents and providers to choose HPV vaccination at the routine recommended ages of 11–12. These tools include print and online materials discussing the risk of HPV infection and more immediate consequences of HPV-related disease (e.g., genital warts) and involving the preteen in the decision making process [16–18], a strength of this particular intervention and a gap in previous interventions. To evaluate our main intervention outcome, we analyzed data from the North Carolina Immunization Registry (NCIR) to compare pre-intervention immunization initiation and completion of 11–12 year olds with post-intervention immunizations.

2. Methods

2.1. Study design

The Protect Them study tests the effectiveness of a practice-based communication intervention to promote preteen vaccination against HPV. Intermediate outcomes include reported discussion about HPV and HPV vaccination among provider, parent and preteen. Vaccination outcomes include initiation and completion of the HPV series.

To ensure statistical power, we recruited practices that reported to the NCIR and had at least 100 11- and 12-year-olds who had not yet completed the HPV series. Practices were enrolled through recruitment in random order with a goal, based on statistical power, of 16 practices per study wave. Eligible practices that declined or who were not contacted served as a comparison group. The study was approved by the University’s Institutional Review Board.

2.2. Setting

We divided North Carolina into three approximately equal geographic regions and conducted the study in three waves from 2015 to 2017. Multiple waves allowed for adjustment due to secular trends, e.g., change in vaccination recommendations to 2 doses before the age of 15 and Food and Drug Administration (FDA) license of a 9-valent vaccine. This paper describes the results of Wave 1 and includes eligible practices in the central region of NC.

2.3. Intervention description

The Protect Them study fosters conversations about HPV vaccination and the prevention of STIs and cancer; the focal groups are providers, parents and preteens and the setting is the clinic. Research staff recruited the practices in random order by first sending a fax about the study, then following up by phone for a response. We approached 62 practices to enroll a sample of 14 practices for Wave 1. We asked practices to 1) commit 50% of their providers to online training about vaccine epidemiology, communicating with parents and preteens about HPV and HPV vaccine, and systems level supports; 2) display posters and brochures in English and Spanish with the headline “1 in 2 people will get HPV, which causes genital warts and cancer” for 9 months in their clinics; and 3) screen parent/preteen dyads for a smaller nested study (data not reported here) to test the acceptability and efficacy of an original video game, Land of Secret Gardens. The game uses growing a healthy garden as a metaphor for a healthy body protected against HPV. The intervention lasted 9 months to allow time for vaccination initiation and completion of the series by at least a portion of preteens. Incentives included $2000 (practice), $100 (provider) and $100 in gift cards (dyads in the nested study) to complete pre and post surveys and use the materials. Process and outcome evaluations of these components on intermediate outcomes (e.g., changes in knowledge, beliefs, attitudes, communication) are ongoing.

2.4. Sample

There were 175 eligible practices for inclusion in Wave 1. From February 2015 to May 2015, sixty-two practices were contacted to yield 14 practices for the study (enrollment rate of 22%). The other 161 practices served as a comparison group. We contacted 24 practices at least once, 32 at least two or more times and visited six in person. Most frequent reasons given for not enrolling were: no time, practice undergoing change and understaffed.

In the intervention group, data collection included surveys of practices and providers pre and post intervention (not reported here). For both the intervention and comparison groups, NCIR vaccination data were collected over a 27-month period (9-months pre-intervention, 9-months during intervention, and 9-months post-intervention, from Aug 1, 2014–Oct 31, 2016) and were provided in a de-identified dataset to the study team from the state health department.

Individuals were included in this analysis if they had not completed the three-dose HPV series before the start of the study period. For the majority of our statistical models, we included participants who were 11–13 years old (our targeted age range for vaccination) any time during the study period. Because of the length of the study, this captured a set of individuals who ranged in age from 9 to 14 years old at the start of the 9-month intervention. In a follow-up analysis of whether the intervention had a differential effect according to age, we expanded the sample to include all participants who were 9–14 at the start of the intervention, regardless of whether they were 11–13 at any time during the study period.

2.5. Statistical analysis

The NCIR registers child and adolescent vaccines given in North Carolina and tracks each child and adolescent with a unique ID number that translates across practices. The registry includes information about vaccines given in NC (date, vaccine type, and provider) as well as historical vaccine information from medical records. In addition, the registry includes child and adolescent date of birth and race/ethnicity. The NCIR also contains limited information about the socioeconomic status of each child by indicating whether the child is eligible for a publicly funded vaccine (Medicaid, Uninsured, Underinsured, or membership, e.g., Indian Health Service). We use the eligibility and race/ethnicity variables in our analyses as a way of capturing differences among practices.

As we did in a previous study using NCIR data [17], we built time-to-event models with time-varying predictors to account for intervention status. The two vaccination outcomes of interest were initiation of the HPV series and completion of the three-dose HPV series. For each outcome, we used the Cox models to test whether intervention practices experienced increased vaccination during the 9-month intervention period and whether there were sustained improvements during the 9-month post-intervention period.

The model is an extended Cox model with both time-independent and time-varying predictors [19]. The model includes a time-independent indicator of whether an individual is in an intervention practice, X₁. The first time-varying predictor, X₂(t), captures the start of a practice’s participation in the intervention. An additional time-varying covariate, X₃(t), is used to model the incremental effect of the post-intervention period. The model predictors control for a systematic baseline difference in vaccination hazard. The model parameters are β₁, β₂, and β₃ for X₁, X₂(t), and X₃(t) respectively. The parameter β₁ control for baseline (pre-intervention) differences between intervention vs. comparison practices, important due to the non-randomized allocation to intervention. β₂ is the intervention effect and β₂ + β₃ is the sustained effect of the intervention beyond the active intervention period. A test of β₃ indicates whether the effect was different post-intervention vs. during the intervention.

Additionally, we explore sex and age as potential modifiers of the intervention effect. Effect modification is tested by adding interactions between the potential effect modifier and the during-intervention and post-intervention incremental effects. Stratified models are used to estimate the hazard ratio for each subgroup.

In all models, a frailty parameter (random intercept) accounts for between-practice variability (i.e., within-practice correlation). Models were fit using PROC PHREG in SAS v.9.3.

2.6. Sensitivity analyses

After fitting the models as planned, two additional models were fit as sensitivity analyses to explore the robustness of the intervention effect.

2.7. Limitations of the NCIR data

There are several limitations to the use of the NCIR registry for research purposes. First, a child is considered to be a patient of a practice if his/her last registered vaccine is with that practice and the patient has not been coded as inactive by the practice. This is error-prone, as children may have moved to a different practice (or even out-of-state), and the first practice may not have notified the registry. While this might attenuate the vaccination estimates, it is unlikely to cause significant bias to the comparison of the two groups. Second, the race/ethnicity/gender data are reported from the practice and may contain errors.

3. Results

3.1. Characteristics of sample

The primary sample consisted of 147,294 individuals in NCIR who were listed as active patients in the practices eligible to participate in the Protect Them intervention, were ages 11–13 at any time during the 27-month study period, and had not completed the HPV vaccine series by the start of the study period. Of these, 19,398 were in intervention practices and 127,896 in comparison practices. There were 128,295 who had neither completed nor initiated the HPV series; 18,999 had initiated but not yet completed the series by the start of the study period. Table 1 lists patients’ demographic characteristics by intervention status. The NCIR demographic data has a relatively large number of missing values on race, ethnicity, gender and insurance status and are included as “Unknown”. Aggregate demographic characteristics of the 14 intervention practices and the 161 comparison practices are presented in Table 2. Overall, the intervention practices had a slightly higher percentage of patients with private insurance and a lower percentage of Hispanic individuals in the sample, but the most noteworthy difference was the number of individuals in the sample. Intervention practices were larger, with an average of 1386 individuals vs. 795 individuals in comparison practices.

Table 1.

Characteristics of individuals who had not completed the HPV vaccine by the start of the intervention (N = 147,294).

		Comparison Group N (%)	Intervention Group N(%)
Age at start of active intervention (9 months into study period)	9	12585 (9.84)	1819 (9.38)
	10	24929 (19.49)	3913 (20.17)
	11	26086 (20.4)	4025 (20.75)
	12	25388 (19.85)	3754 (19.35)
	13	22714 (17.76)	3359 (17.32)
	14	16194 (12.66)	2528 (13.03)
Race	American Indian or Alaska Native	4109 (3.21)	236 (1.22)
	Asian	2993 (2.34)	298 (1.54)
	Black or African-American	31799 (24.86)	5892 (30.37)
	Native Hawaiian or Other Pacific Islander	227 (0.18)	61 (0.31)
	Other Race	2827 (2.21)	137 (0.71)
	Unknown	21775 (17.03)	4230 (21.81)
	White	64166 (50.17)	8544 (44.05)
Ethnicity	Hispanic or Latino	5891 (4.61)	502 (2.59)
	Not Hispanic Or Latino	34803 (27.21)	5348 (27.57)
	Unknown	87202 (68.18)	13548 (69.84)
Insurance status	Blank	10206 (7.98)	1672 (8.62)
	Not Insured/Underinsured	10277 (8.04)	1889 (9.74)
	Private	45373 (35.48)	8190 (42.22)
	Public	62040 (48.51)	7647 (39.42)
Sex	F	59276 (46.35)	8897 (45.87)
	M	63036 (49.29)	9370 (48.3)
	Unknown	5584 (4.37)	1131 (5.83)

Table 2.

Practice Characteristics.

	Intervention (N = 14)			Comparison (N = 161)
	Mean	SD	Range	Mean	SD	Range
Number of individuals in the sample	1385.6	1911.6	144–6407	794.4	789.1	75–4190
Percent white race	48.1	2.3	44.4–52.9	48.6	3.1	35.7–61.4
Percent female	62	21.6	27.3–89.5	63.5	20.2	15.3–95
Percent Hispanic ethnicity	8.8	4.9	1.7–18.8	16.8	19.9	0–84.2
Percent with private insurance	41.6	19.8	3.5–74.3	37.7	25.3	0–94.4

3.2. Intervention effects on HPV vaccination

Vaccination outcomes were initiation and completion of the HPV series. Kaplan-Meier curves were created to show the proportion remaining unvaccinated (uncompleted) over time (Fig. 1). While informative, these curves include all individuals and do not account for within-practice correlation. As such, they may unduly reflect the outcomes from large practices.

Fig. 1.

Kaplan-Meier Curve of initiation and completion in intervention vs. comparison.

The extended Cox model was used to test the intervention effect (Table 3). The intervention had a significant effect on both initiation and completion during the intervention, and the effect was also apparent post-intervention. The estimated hazard ratio (HR) for initiation was 1.17 during the intervention, therefore after controlling for baseline practice differences, individuals in the intervention practices were 17% more likely to initiate vaccination than in the comparison practices during the intervention period. The HR was 1.11 (p = .005) in the post-intervention period, indicating an 11% increase over baseline, which was a significantly smaller effect than during the intervention period (p = .004). Likewise, the completion rate during the intervention period was 18% higher in intervention practices (HR = 1.18), after controlling for baseline differences. This effect actually significantly increased in the post-intervention period, to 31% higher (HR = 1.31, p-value for change = .03). This trend for initiation and completion reflects the six-month lag between initiation and completion of the three-dose series; the larger increase in completion in the post-intervention period would reflect the completion of series that were initiated during the intervention period.

Table 3.

Cox models for intervention effect on initiation and completion of the HPV series.

	Intervention effect during the intervention		Intervention effect post-intervention		p for during vs. post
Outcome	Hazard ratio (95% CI)	p	Hazard ratio (95% CI)	p
Primary analysis
Initiation	1.17 (1.1, 1.26)	<.0001	1.11 (1.03, 1.19)	.005	.004
Completion	1.18 (1.07, 1.31)	.001	1.31 (1.19, 1.44)	<.0001	.02
Sensitivity analysis excluding large practices (adjusted models only)
Initiation	1.09 (0.97, 1.23)	.13	1.10 (0.98, 1.23)	.10	.91
Completion	1.27 (1.09, 1.47)	.002	1.25 (1.08, 1.45)	.003	.88
Sensitivity analysis excluding refusals (adjusted models only)
Initiation	1.17 (1.08, 1.26)	<.0001	1.10 (1.02, 1.19)	.0009	.11
Completion	1.19 (1.07, 1.32)	.001	1.27 (1.15, 1.41)	<.0001	.14

Note: All models adjusted for age, eligibility (insurance status), sex, and race/ethnicity. Unadjusted models (not shown) produced nearly identical results.

3.3. Effect modification by sex and age

To explore effect modification, we fit models for initiation stratified by sex and age (see Table 4). For stratification by age, we used three age groups: 9–10, 11–12, and 13–14 at the start of the intervention (note that this is a wider age range than in the previous models). Though statistically significant in both sexes, the intervention effect tended to be higher in males than females. To explore further, we re-ran the adjusted primary models in Table 3 with the addition of an interaction term between sex and the intervention effect. For initiation, there was no differential intervention effect between males or females either during (p = .09) or post-intervention (p = .20). For completion, the interaction was significant both during the intervention (p = .01) and post-intervention (p = .04), indicating that the intervention effects were higher in males.

Table 4.

Stratified models to explore effect modification by sex and age

		Intervention effect during the intervention		Intervention effect post-intervention
Outcome	Group	Hazard Ratio	p	Hazard Ratio	p
Initiation	Male	1.22 (1.13, 1.32)	<.0001	1.13 (1.04, 1.22)	<.0001
	Female	1.15 (1.06, 1.24)	0.0007	1.08 (0.99, 1.17)	0.008
	≤10	0.95 (0.73,1.23)	0.68	0.73 (0.57, 0.95)	0.02
	11–12	1.14 (1.04, 1.25)	0.004	1.07 (0.97, 1.18)	0.19
	≥13	1.20 (1.05, 1.36)	0.007	1.18 (1.03, 1.36)	0.02
Completion	Male	1.37 (1.23, 1.52)	<.0001	1.29 (1.15, 1.44)	<.0001
	Female	1.21 (1.09, 1.35)	0.0003	1.16 (1.04, 1.3)	0.008
	≤10	1.43 (0.80, 2.56)	0.23	1.43 (0.84, 2.42)	0.19
	11–12	1.13 (0.97, 1.32)	0.11	1.30 (1.12, 1.5)	0.0004
	≥13	1.17 (1.02, 1.34)	0.03	1.11 (0.96, 1.29)	0.14

Note: Models were adjusted for race, insurance status, sex (in model stratified by age) and age (in model stratified by sex.

Tests of effect modification by age using interaction terms indicated that the intervention effect on both initiation and completion varied across age groups both during- and post-intervention (p = .02 and p < .0001 respectively for initiation, p = .01 and p < .0001 for completion). Specific hazard ratios are given in Table 4. In the 11–12-year-olds, who are at the recommended age for vaccination, the intervention had a positive effect on both completion and initiation, though this effect was only statistically significant during the intervention for initiation, and post-intervention for completion. In the teens, the intervention had a positive impact on initiation during both periods and on completion during the intervention. There appeared to be no intervention benefit in the 9–10-year-olds and there was some evidence that intervention practices were less likely to initiate vaccination in the younger children. These stratified analyses should be interpreted with caution. In the younger children, power may be limited because less than 5% completed the vaccine during the 27-month period. The teens had about half the initiation rate of the 11–12-year-olds; presumably those who were most likely to get vaccinated did so when they were 11–12.

3.4. Sensitivity analyses with size of practice and study refusal

Of the eight practices that contributed 3000 or more individuals to the dataset, three (38%) were in the intervention group, which is an overrepresentation considering that only 8% of all practices were in this group. Based on this, we re-ran the primary analysis after excluding these eight practices. The resulting sample contained 115,518 individuals, 78% of the original sample. The hazard ratios are similar to those seen in the entire sample (see Table 3). However, the intervention no longer shows a significant effect on initiation (p = .13 during, .10 post-intervention), and the accompanying hazard ratios are smaller.

To further understand the implications of removing these large practices, we looked at Kaplan-Meier curves for completion within each of the enrolled practices (Fig. 2). There is substantial between-practice variation and a single practice with very low completion rates. This practice, contributing over 6500 individuals to the dataset, was a publicly-funded health department that outsourced its primary care. Its exclusion explains the slightly increased hazard ratios in the sensitivity analysis.

Fig. 2.

Kaplan-Meier Curve of completion in intervention practices.

A second sensitivity analysis in Table 3 excluded refusals from the comparison group. In this analysis, there were significant intervention effects on both outcomes during-and post-intervention, with hazard ratios nearly identical to those in the primary analysis. There was, however, no longer a significant difference between the during- and post-intervention periods.

4. Discussion

The Protect Them study shows promise for practice-based communication strategies—including age-appropriate discussion of HPV as a preventable STI-- to be effective in promoting preteen HPV vaccination. This is important because additional vaccines to prevent STIs (e.g. herpes and chlamydia) are in development and they will not prevent cancer. Our analysis of vaccination outcomes in Wave 1 showed that preteens in the intervention practices were 17% more likely to initiate and to complete vaccination than those in the comparison practices. Additional evaluation of provider practices and choices by parents and preteens needs to occur to round out the estimation of intervention effects from our tools.

Approaches to boost HPV vaccination at ages 11–12 have been conducted with some success, including from our two earlier social marketing campaigns [14,17,18,20]. Clinician education, clinical practice quality improvement strategies, patient reminder/recall, communication campaigns, and stakeholder engagement have been designed to help providers and practices increase their HPV immunization rates [21]. By comparison and based on extensive formative research, we developed an approach to HPV vaccination decision-making that includes preteens in age appropriate discussion about HPV and its transmission through sexual activity. What makes our tools particularly strong is the considerable input from parents and preteens and providers who serve them [14,15,17,22]. Our two previous county-based social marketing campaigns in North Carolina showed a boost in preteen HPV immunizations in the campaign counties compared to immunizations in non-intervention counties [17,18]. For a more sustained communication effort that potentially could be adopted in clinical settings, we developed and evaluated these tools for a communication intervention to use in primary care practices.

Our study has several limitations. A major limitation is that although the practices were randomly ordered in the sample frame, the practices were not randomly allocated to study condition. The intervention practices had higher HPV vaccination uptake at baseline and may have been more motivated to administer the vaccine even without the intervention. Other efforts in North Carolina to boost preteen HPV vaccination may have influenced our practices even though they were not simultaneously enrolled at the time. Finally, the immunization registry does not have a perfect count, as patients may move to another practice and not be deleted from their former practice.

5. Conclusion

The objective of our study was to conduct and measure a practice-based intervention with communication tools to motivate preteen HPV vaccination through provider/parent/preteen conversations about HPV as a sexually transmitted infection preventable by vaccination. We hypothesized that HPV vaccination initiation and completion would be higher in intervention practices than comparison practices. With the continuing gaps in HPV vaccination among preteens, we recommend using and measuring the impact of these tools in other settings.