Improving Adolescent HPV Vaccination in a Randomized Controlled Cluster Trial Using the 4 Pillars^™ Practice Transformation Program

Abstract

Objective

Uptake of meningococcal vaccine (MCV) and tetanus, diphtheria and pertussis (Tdap) vaccine among adolescents has approached Healthy People 2020 goals,[1] but human papillomavirus (HPV) vaccination has not. This study evaluated an intervention using the 4 Pillars^™ Practice Transformation Program to increase HPV, MCV and Tdap uptake among adolescents in primary care practices.

Methods

Practices with at least 50 patients 11–17 years old with estimated vaccination rates less than national goals, were assigned to intervention (n=11) and control (n=11) groups in a randomized controlled cluster trial; 9 intervention and 11 control sites completed the study. The baseline and active study periods were 7/1/2013–6/30/2014 and 7/1/2014–3/31/2015, respectively. Vaccination and demographic data for patients who had a visit in both study periods were derived from de-identified EMR extractions. Primary outcomes were vaccination rates and percentage point (PP) changes. Data were analyzed in 2015–16.

Results

Among the cohort of 10,861 adolescent patients, 38% were 11–13 years old; 50% were female; 18% were non-white; and 64% were commercially insured. Average baseline HPV initiation rates were 52.5% for intervention and 61.8% for control groups. After 9 months, the intervention sites increased HPV initiation 10.2 PP compared with 7.3 PP in control sites (P<0.001); HPV series completion rates did not differ between groups. Implementation of >10 strategies to improve rates significantly increased the likelihood of HPV series initiation (OR=2.06, 95% CI=1.43, 2.96).

Conclusions

Using >10 strategies from the 4 Pillars^™ Practice Transformation Program is effective for increasing HPV series initiation among adolescents.

INTRODUCTION

Although the number of vaccines and complexity of the adolescent vaccination schedule are considerably less than for younger children, achieving optimal uptake for all recommended vaccines among adolescents has proven to be challenging [1]. As of 2014, national human papilloma virus vaccine (HPV) initiation and 3-dose rates for adolescents 13–17 years of age were 60.0% and 39.7% for females [2], and 41.7% and 21.6%, respectively for males [2]. In contrast, the tetanus-diphtheria-acellular pertussis (Tdap) vaccination rate for 13–17 year olds was 87.6% and the meningococcal vaccine (MCV) vaccination rate was 79.3% [2]. One explanation for the differences in rates for these vaccines, all of which are recommended to be given (or started) at around age 11 to 12 years [3–5], is the state-level school entry requirement for MCV and Tdap. At least 24 states mandate MCV [6] and 40 states mandate Tdap [7] receipt for children generally around age 12 years. Conversely, receipt of HPV is voluntary in most states.

Reasons cited for low HPV vaccination rates among adolescents are many. For example, contacts with the health care system decrease as children reach adolescence [8]. Moreover, one study found that most teens (72% of females, 79% of males) would need three visits to complete the HPV vaccine series within one year of initiation at a preventive care visit [9]. Hesitation and delay of HPV vaccination by parents have also been attributed to the relationship between HPV infection and sexual activity, leading to concerns that HPV vaccination will increase promiscuity [10–12]; Other reasons include fears about side effects, adverse events [11, 13]; or “uncertainty” about the vaccine such as insufficient information or lack of perceived community norms to vaccinate [10]; low perceived risk of a child becoming infected or having HPV disease [14]; and lack of provider recommendation [12]. Reported reasons for lack of completion of the series among females are not knowing about subsequent doses, being too busy [15], and, at some time points, having public insurance [16].

Providers may hesitate to broach HPV vaccination with parents for a variety of reasons including beliefs that offering HPV vaccine with other vaccines will not increase parental acceptance, a significant percentage of parents will defer regardless of physician recommendation, and their own concerns about waning immunity [17].

The multifactorial nature of this issue calls for multi-strategy interventions. The 4 Pillars^™ Practice Transformation Program (4 Pillars^™ Program) is an evidence-based, step-by-step guide for improving immunizations in primary care practices with strategies addressing a variety of barriers to vaccination. The purpose of this report is to describe changes in adolescent HPV vaccinations after a 9-month intervention in 20 pediatric and family medicine practices in the Pittsburgh metropolitan area using a randomized controlled cluster trial.

METHODS

This trial took place during 2013–2014 (baseline) and 2014–2015 (active study period), and was approved by the Institutional Review Board of the University of Pittsburgh. Consent was implied by agreement by the lead physician to participate in the study.

Sample Size and Sites

Optimal Design software (University of Michigan, Version 1.77. 2006) was used to calculate sample size, for a 2-level cluster randomized trial with a binary outcome – vaccination status during the intervention period. We sought a 10% absolute increase in vaccination rate. To achieve 80% power with an alpha of 0.05, and assuming a probability of vaccination in the control group to be 40% and in the intervention group to be 50% and an intracluster correlation (ICC) of 0.20, we estimated that a sample size of 18 clusters or sites with 250 patients per cluster was required. Primary care family medicine (FM) and pediatric (Peds) practices from two practice-based research networks (PBRN) in Pittsburgh (FM Pittnet and Peds PittNet) and a clinical network in Southwestern Pennsylvania (Community Medicine, Inc.) were solicited for participation based on preliminary estimates of patient volume and vaccination rates. When 22 sites agreed to participate, solicitation ceased. All sites used a common electronic medical record (EMR), EpicCare.

Cluster Randomization

This intervention took place at the practice level. Therefore, cluster randomization which allocates practices, rather than individual patients to the intervention arms [18], was used. Some practices had more than one site; each site was considered to be a cluster. However, with one exception, related sites were randomized as a block. Eligibility requirements included having an adolescent practice of at least 50 patients, estimated vaccination rates for at least one adolescent vaccine (HPV, Tdap, MCV) less than national goals [1] and a willingness to make office changes to increase vaccination rates. The number of practices with ≥250 patients and suboptimal vaccination rates was smaller than anticipated. Twenty-two practices were stratified by location (urban, suburban or rural) and by discipline (pediatrics or family medicine) then randomized by the data analyst, into the intervention (11) or control group (11) within strata. Control sites were informed by the research team that their intervention would take place the following year and were not contacted again until the end of the active study period when the primary contact for each practice was asked to complete a survey about strategies being used to promote adolescent vaccination.

4 Pillars^™ Practice Transformation Program

The 4 Pillars^™ Practice Transformation Program is founded on four key, evidence-based [19, 20] domains: Pillar 1 - Convenient vaccination services; Pillar 2 – Communication with patients about the importance of immunization and the availability of vaccines; Pillar 3 - Enhanced office systems to facilitate immunization; Pillar 4 - Motivation through an office immunization champion (Champion). Supplemental Table 1 describes some of the strategies contained in the 4 Pillars^™ Program. The Program website includes background on the importance of protecting patients against vaccine-preventable diseases, barriers to increasing vaccination from both provider and patient perspectives and strategies to eliminate those barriers. Intervention sites were expected to implement strategies from each of the 4 pillars, but were encouraged to use as many strategies as possible and appropriate for the practice to maximize their impact on vaccination rates.

The web-based 4 Pillars^™ Program also included a practice transformation dashboard (PTD) that was developed from previous work [21] that established an empirically-based implementation framework that includes systematic uptake, establishment, and maintenance of research findings into routine practice. The core components of a PTD include: staff selection and training on the specific evidence-based practices, expert consultation and coaching of staff and administration, program evaluation to assess and provide feedback, facilitative administrative supports to ensure data are used to focus and inform decision making, and systems interventions. The PTD included an at-a-glance summary of the practice’s unique information and program status such as a listing of the selected intervention strategies and a task list of incomplete intervention activities sorted by suggested due date.

Interventions

The intervention was designed using Diffusion of Innovations theory [22], and included the 4 Pillars^™ Program, provider education, and one-on-one coaching of a Champion for each site. One of the investigators (MPN) visited each intervention site to introduce the study and the 4 Pillars^™ Program and to work with staff to develop practice-specific ideas for implementing strategies from each of the 4 pillars. Each site identified a Champion who was responsible for logging into the 4 Pillars^™ Program website to register the practice, select strategies and access practice improvement resources. Other roles for the Champion included promoting implementation of chosen strategies, working to motivate the staff and participating in biweekly telephone coaching with a research liaison.

Data collection

De-identified demographic, office visit and vaccination data were derived from EMR data extractions performed by the UPMC Center for Assistance in Research using the eRecord. A longitudinal data base was created with only those patients who were 11–17 years (date of birth between 1/1/1995 and 4/1/2003) and who had a visit during both the baseline and intervention periods. Champions or a member of the office’s leadership team in both the Intervention and Control sites completed a survey to report what strategies their practices were employing to improve adolescent vaccination rates at the end of the active study period. The number of strategies was totaled for each practice for inclusion in regression analyses.

Statistical analyses

Descriptive analyses were performed for patient demographic characteristics including, age (11–13 years old and 14–17 years old), sex, race (white, non-white), and health insurance type (commercial vs. public, other). The analytical periods were baseline: 7/1/2013–6/30/2014 and intervention: 7/1/2014–3/31/2015. Proportions were reported for categorical variables and means and standard deviations were reported for continuous variables. Because HPV is given in a 3-dose series, preferably at 0, 1–2 and 6 months, vaccination rates were measured in two ways – series initiation and series completion. The primary outcome measures were the practice-level cumulative HPV series initiation and completion rates reported at the end of the baseline and the active study periods. Cumulative MCV and Tdap vaccination rates were also measured for these time periods for comparison with HPV vaccination rates. Chi-square tests were performed to test for differences in cumulative vaccination rates at baseline and post intervention.

To determine which factors were related to vaccination rates, while accounting for the clustered nature of the data, hierarchical Cox proportional hazard models with the robust sandwich estimate were fitted to examine the effect of intervention on HPV series initiation and completion rates, taking account of heterogeneity in demographic characteristics (including age, sex, race, health insurance type, intervention group and number of strategies used; this analysis applied only to patients who were unvaccinated at baseline).

In addition, generalized estimating equations (GEE) were used to examine the association between total number of strategies used at each clinical site on HPV series initiation and completion rates, adjusting for demographic characteristics (including age, sex, and race and health insurance type). Sensitivity analyses were used to identify the optimal discriminating cutoff point and generate a binary variable for the number of strategies that was related to HPV series initiation and completion. Intervention group and number of strategies were not included in the same model because they were significantly associated with each other. In addition, intervention sites were divided into low implementers (≤10 strategies) and high implementers (>10 strategies) based on the number of strategies reported at the end of the active study period. Post hoc comparisons among low implementers and high implementers using the Chi square test with the Bonferroni correction were conducted to assess the effect of the intervention. Otherwise, statistical significance for two-sided tests was set at a type I error (alpha) equal to 0.05. All analytical procedures were performed using SAS® 9.3.

Results

Two practices that were randomized to the intervention group dropped out of the study. Baseline demographic characteristics by practice and intervention group are presented in Table 1. Participating practices ranged widely in the number of adolescent patients (53–1,597), the percent of 11–13 year old patients (18.1%–47.7%), non-white patients (1.6%–98.1%) and commercially insured patients (11.3%–84%). Intervention and control groups differed significantly in the proportion of 11–13 year olds, proportion of non-white patients and proportion of commercially insured patients (P<0.001) with the control group having fewer young teens, more non-white and publicly insured patients than the intervention group.

Table 1.

Baseline demographic and practice characteristics, by site, by intervention group and overall

Site	N	11–13 years old, %	Female, %	Non-white, %	Commercial Health Insurance, %
Intervention sites
A	1263	42.9	49.4	4.8	82.6
C	390	36.9	51.5	67.7	21.5
D	1815	36.0	51.0	2.4	74.3
E	133	30.1	60.2	51.9	11.3
F	987	47.7	52.1	3.5	71.1
G	136	31.6	47.8	11.0	55.9
H	99	36.4	51.5	37.4	49.5
I	66	34.9	56.1	18.2	31.8
J	53	34.0	49.1	98.1	18.8
Intervention total	4942	39.9	49.6	12.0	62.2
Control sites
L	1597	35.2	49.2	1.6	69.6
M	704	39.5	48.4	1.3	77.7
N	933	44.0	50.3	82.4	23.4
O	143	35.7	52.5	46.1	16.1
P	86	24.4	61.6	2.3	38.4
Q	160	18.1	50.6	5.0	60.6
R	117	28.2	51.3	5.1	67.5
S	103	32.0	44.7	3.9	64.1
T	913	36.6	48.2	23.4	71.9
U	921	39.5	49.5	9.0	84.0
V	242	36.0	53.3	83.5	32.6
Control total	5919	37.2*	51.1	23.4*	67.8*
Overall	10861	38.4	50.3	18.2	64.7

^*P <0.001 for differences between intervention and control groups

Cumulative HPV vaccination series initiation rates (Table 2) and vaccination series completion rates (Table 3) were analyzed by age group (11–13 year olds and 14–17 year olds) and overall. At baseline, HPV vaccination series initiation rates were significantly lower in the intervention group than in the control group for both age groups combined (52.5% intervention sites vs. 61.8% control sites; P<0.001), and for each age group. Increases in series initiation, as measured by percentage point (PP) differences between baseline and the end of the active study period for 11–13 year olds and for 14–17 year olds were significant (P<0.001), and were significantly larger in the intervention group (10.2 PP) than the control group (7.3 PP; P<0.001) overall and for both age groups (13.7 PP intervention group vs. 10.1 PP control group in 11–13 year olds; P<0.001 and 7.8 intervention group vs. 5.7 control group in 14–17 year olds; P<0.001; ICC=0.38). At baseline, HPV vaccination series completion rates were also significantly lower in the intervention group than the control group overall and for each age group (Table 3). At the end of the active study period, PP increases in series completion did not differ between intervention (12.8 PP) and control (12.7 PP) groups; thus, the intervention group’s rates at the end of the active study period did not surpass those of the control group. In the intervention group, the changes in PP for series completion from baseline to the end of the active study period for 11–13 year olds and for 14–17 year olds were significant (P<0.001; ICC=0.34).

Table 2.

HPV series initiation among children 11–17 years old overall and by age group

Site	11–13 years			14–17 years			Total 11–17 years
		% Vaccinated			% Vaccinated			% Vaccinated
	Total N	Baseline (7/1/2013– 6/30/2014)	Intervention (7/1/2014– 3/31/2015)	Total N	Baseline (7/1/2013– 6/30/2014)	Intervention (7/1/2014– 3/31/2015)	Total N	Baseline (7/1/2013– 6/30/2014)	Intervention (7/1/2014– 3/31/2015)	Number of Intervention Strategies
Intervention sites
C	144	63.1	72.9	246	75.6	82.1	390	71.0	78.7	7
D	654	28.5	42.5	1161	56.9	66.4	1815	46.7	57.7	8
F	471	22.9	38.0	516	62.4	69.1	987	43.5	54.3	9
A	542	44.4	56.8	721	66.1	72.3	1263	56.8	65.7	9
I	23	47.8	52.1	43	60.4	65.1	66	56.0	60.6	10
G	43	48.8	69.7	93	66.6	75.2	136	61.0	73.5	12
J	18	77.7	100.0	35	100.0	100.0	53	92.4	100.0	13
H	36	41.6	58.3	63	84.1	90.4	99	68.6	78.7	14
E	40	60.0	80.0	93	68.8	80.6	133	66.1	80.4	15
Intervention total	1971	36.1*	49.8	2971	63.4**	71.2	4942	52.5*	62.7	7–15
Percentage point difference between baseline and intervention periods			13.7†,‡			7.8†,‡			10.2†
Control sites
P	21	47.6	57.1	65	63.0	70.7	86	59.3	67.4	1
R	33	39.3	45.4	84	44.0	58.3	117	42.7	54.7	3
N	410	77.5	88.2	523	85.2	89.4	933	81.8	88.9	2
S	33	27.2	30.3	70	48.5	54.2	103	41.7	46.6	4
Q	29	37.9	41.3	131	41.2	49.6	160	40.6	48.1	4
T	334	44.3	58.3	579	73.7	80.6	913	62.9	72.5	7
O	51	64.7	82.3	92	81.5	86.9	143	75.5	85.3	7
V	87	50.5	63.2	155	65.8	68.3	242	60.3	66.5	8
M	278	44.2	54.3	426	57.7	63.6	704	52.4	59.9	9
U	364	51.0	57.9	557	65.8	70.1	921	60.0	65.3	10
L	562	48.3	57.6	1035	64.1	69.7	1597	58.6	65.4	10
Control total	2202	52.9*	63.0	3717	67.0**	72.7	5919	61.8*	69.1	1–10
Percentage point difference between baseline and intervention periods			10.1†			5.7†			7.3†

Note: HPV vaccination rates are cumulative as of the end of each period.

^*P<0.001 for difference between intervention and control groups at baseline, by Chi-square test.

^**P<0.01 for difference between intervention and control groups at baseline, by Chi-square test.

^†P<0.001 difference between intervention and control groups in percentage point increases during intervention, by Chi-square test.

^‡P<0.001 for PP difference from baseline to intervention between 11–13 year olds and 14–17 year olds, by Chi-square test.

Table 3.

HPV series completion among children 11–17 years old

Site	11–13 years			14–17 years			Total
		% Vaccinated			% Vaccinated			% Vaccinated
	Total N	Baseline (7/1/2013– 6/30/2014)	Intervention (7/1/2014– 3/31/2015)	Total N	Baseline (7/1/2013– 6/30/2014)	Intervention (7/1/2014– 3/31/2015)	Total N	Baseline (7/1/2013– 6/30/2014)	Intervention (7/1/2014– 3/31/2015)
Intervention sites
A	542	14.0	28.5	721	43.5	55.7	1263	30.8	44.1
C	144	11.1	36.1	246	50.0	59.3	390	35.6	50.7
D	654	12.6	26.2	1161	39.2	51.4	1815	29.6	42.3
E	40	15.0	27.5	93	39.7	48.3	133	32.3	42.1
F	471	10.1	22.5	516	46.1	58.1	987	28.9	41.1
G	43	11.6	37.2	93	49.4	61.2	136	37.5	53.6
H	36	5.5	19.4	63	55.5	69.8	99	37.3	51.5
I	23	17.3	34.7	43	48.8	55.8	66	37.8	48.4
J	18	44.4	66.6	35	82.8	85.7	53	69.8	79.2
Intervention total	1971	12.5*	27.3	2971	43.7**	55.3	4942	31.3**	44.1
Percentage point difference between baseline and intervention periods			14.8†			11.6†			12.8
Control sites
L	562	22.9	37.0	1035	45.6	55.6	1597	37.6	49.0
M	278	17.9	28.7	426	33.5	45.5	704	27.4	38.9
N	410	19.0	43.4	523	57.3	73.9	933	40.5	60.5
O	51	35.2	52.9	92	59.7	69.5	143	51.0	63.6
P	21	28.5	42.8	65	52.3	60.0	86	46.5	55.8
Q	29	10.3	17.2	131	25.1	30.5	160	22.5	28.1
R	33	15.1	27.2	84	25.0	35.7	117	22.2	33.3
S	33	3.0	12.1	70	32.8	37.1	103	23.3	29.1
T	334	17.0	31.7	579	48.3	62.8	913	36.9	51.4
U	364	28.5	44.7	557	57.8	62.8	921	46.2	55.7
V	87	16.0	27.5	155	41.9	50.9	242	32.6	42.5
Control total	2202	21.1*	36.9	3717	47.0**	57.8	5919	37.3**	50.0
Percentage point difference between baseline and intervention periods			15.8			10.8			12.7

Note: HPV vaccination rates are cumulative as of the end of each period.

^*P<0.001 for difference between intervention and control groups at baseline, by Chi-square test.

^**P<0.01 for difference between intervention and control groups at baseline, by Chi-square test.

^†P=0.001 for PP difference from baseline to intervention between 11–13 year olds and 14–17 year olds, by Chi-square test.

Note: No differences observed between intervention and control groups in percentage point increases during intervention for HPV series completion, by Chi-square test.

As of 2014, HPV vaccination rates among males still lagged behind those of females [2]. Thus, HPV vaccination series initiation and completion rates were also examined by sex and are shown in Table 4. HPV vaccination rates for females exceeded those of males at baseline and at the end of the active study period, in both the intervention and control groups. Significant PP improvements were observed in both males and females from baseline to the end of the active study period and increases in male groups were generally larger than female groups, but the intervention did not eliminate disparities in rates between males and females.

Table 4.

HPV series initiation and series completion among children 11–17 years old, by sex

Site	Males					Females
		Initiation		Completion			Initiation		Completion
		% Vaccinated					% Vaccinated
	Total N	Baseline (7/1/2013– 6/30/2014)	Intervention (7/1/2014– 3/31/2015)	Baseline (7/1/2013– 6/30/2014)	Intervention (7/1/2014– 3/31/2015)	Total N	Baseline (7/1/2013– 6/30/2014)	Intervention (7/1/2014– 3/31/2015)	Baseline (7/1/2013– 6/30/2014)	Intervention (7/1/2014– 3/31/2015)
Intervention sites
A	639	50.5	60.0	25.6	39.1	624	63.3	71.4	36.2	49.1
C	189	68.7	78.3	28.0	44.9	201	73.1	79.1	42.7	56.2
D	890	39.3	50.1	20.3	33.9	925	53.8	65.1	38.7	50.4
E	53	58.4	75.4	20.7	28.3	80	71.2	83.7	40.0	51.2
F	473	41.2	52.2	24.3	38.9	514	45.7	56.2	33.2	43.1
G	71	52.1	70.4	22.5	46.4	65	70.7	76.9	53.8	61.5
H	48	72.9	79.1	31.2	52.0	51	64.7	78.4	43.1	50.9%
I	29	48.2	55.1	41.3	48.2	37	62.1	64.8	35.1	48.6
J	27	88.8	100.0	66.6	74.0	26	96.1	100.0	73.0	84.6
Intervention total	2419	47.0*	57.7**	24.1*	38.3**	2523	57.8*	67.5**	38.1*	49.7**
Percentage point difference between baseline and intervention periods			10.7		14.2†			9.7		11.6†
Control sites
L	812	51.9	59.6	29.0	41.7	785	65.4	71.5	46.4	56.6
M	363	48.7	57.8	23.4	35.5	341	56.3	62.1	31.6	42.5
N	464	77.3	86.2	35.5	55.6	469	86.3	91.6	45.4	65.4
O	68	67.6	80.8	35.2	48.5	75	82.6	89.3	65.3	77.3
P	33	48.4	63.6	33.3	48.4	53	66.0	69.8	54.7	60.3
Q	79	27.8	36.7	12.6	18.9	81	53.0	59.2	32.0	37.0
R	57	22.8	42.1	8.7%	19.2	60	61.6	66.6	35.0	46.6
S	57	35.0	40.3	15.7	26.3	46	50.0	54.3	32.6	32.6
T	473	58.3	69.5	28.7	43.7	440	67.9	75.6	45.6	59.7
U	465	56.5	61.9	40.4	50.9	456	63.5	68.8	52.1	60.5
V	113	49.5	60.1	16.8	30.0	129	69.7	72.0	46.5	53.4
Control total	2984	55.9*	64.7**	29.7*	43.3**	2935	67.8*	73.6**	45.1*	56.8**
Percentage point difference between baseline and intervention periods			8.8‡		13.6§		5.8‡			11.7§

Note: HPV vaccination rates are cumulative as of the end of each period.

^*P<0.001 for difference between males and females at baseline for series initiation and series completion, by Chi square test.

^**P<0.001 for difference between males and females at end of intervention for series initiation and series completion, by Chi square test.

No difference in percentage point increases between males and females in the intervention group for series initiation, by Chi square test.

^†P<0.01 for percentage point increases between males and females in the intervention group for series completion, by Chi square test.

^‡P<0.001 for percentage point increases between males and females in the control group for series initiation, by Chi square test.

^§P<0.05 for percentage point increases between males and females in the control group for series completion, by Chi square test.

By way of comparison, MCV and Tdap vaccination rates are shown in Supplemental Table 2. Although the 4 Pillars^™ Program addressed all adolescent vaccines, neither vaccine was targeted for intervention because they are both required for school admission in this state [6] [7]. The majority of sites (17/20) had baseline MCV vaccination rates for 11–13 year olds above 70%. Baseline Tdap vaccination rates for 11–17 year olds were similarly high, with only one site below 70%. Non-significant increases in MCV rates were observed in the active study period in both intervention and control groups, but PP increases in Tdap rates were significantly higher for intervention sites (4.5 PP) than control sites (3.9 PP; P<0.05).

The results of the regression analyses for HPV initiation and completion are shown in Table 5. Although the intervention (Model 1) was not significantly related to either HPV vaccination series initiation or completion, this model indicated that HPV vaccination series initiation was associated with being non-white and publicly insured. There was no difference in the likelihood of vaccine initiation between older and younger adolescents nor between males and females. This model indicated that HPV vaccination series completion was associated with being an older adolescent, non-white and female.

Table 5.

Results of Cox proportional hazards regression including intervention arm (Model 1) and of generalized estimating equations including implementation group (Model 2) on HPV series initiation and series completion

	Outcome variables
Independent sariable	HPV series initiation		HPV series completion
Model 1 Intervention vs. Control	Hazards Ratio (95% Confidence Interval)	P value	Hazards Ratio (95% Confidence Interval)	P value
Age: 11–13 years, ref. = 14–17 years	1.11 (0.89–1.39)	0.344	0.68 (0.57–0.83)	<0.001
Race: White, ref. = non-white	0.61 (0.47–0.80)	<0.001	0.74 (0.61–0.90)	0.002
Sex: Female, ref. = male	1.08 (0.99–1.17)	0.057	1.17 (1.10–1.24)	<0.001
Insurance: Commercial, ref. = public insurance	0.84 (0.72–0.97)	0.018	0.88 (0.76–1.01)	0.074
Group: Intervention, ref. = control	0.95 (0.77–1.2)	0.583	0.94 (0.82–1.08)	0.370
Model 2 High Implementers (>10 strategies) vs. Low Implementers (≤10 strategies)	Odds Ratio (95% Confidence Interval)	P value	Odds Ratio (95% Confidence Interval)	P value
Age: 11–13 years, ref. = 14–17 years	1.18 (0.98–1.36)	0.061	0.81 (0.71–0.92)	0.005
Race: White, ref. = non-white	0.52 (0.42–0.63)	<0.001	0.65 (0.56–0.77)	<0.001
Sex: Female, ref. = male	1.03 (0.90–1.21)	0.521	1.06 (0.94–1.19)	0.242
Insurance: Commercial, ref. = public insurance	0.81 (0.70–0.95)	0.013	0.83 (0.73–0.94)	0.014
Strategies: >10, ref. ≤10	2.08 (1.43–3.01)	<0.001	0.95 (0.68–1.31)	0.811

Post-hoc Analyses on Number of Strategies Implemented

Because of the possibility that some control sites began to focus efforts on improving HPV vaccination uptake by virtue of being included in the study, analyses to determine the effect of the level of uptake of the intervention were conducted. The number of strategies varied from 1 to 15 across all sites. Sensitivity analyses indicated that 10 strategies was an appropriate cut-off level. Four intervention sites used more than 10 strategies (high implementers), five intervention sites and all control sites used 10 or fewer strategies (low implementers). For HPV series initiation, high implementers had significantly higher PP increases (11.9) than low implementers (8.5; P<0.005). For HPV series completion there were no significant differences in PP differences between low and high implementers. The strategies most frequently indicated on the survey (used by ≥50% of practices) were vaccination fliers and posters, EMR or written provider reminders, staff in-service about the effort, walk-in vaccination visits, patient reminders through personal telephone calls, and an office immunization champion. We do not know what other specific strategies the immunization champions in each site used to motivate the staff.

A second regression model that included the number of strategies dichotomized into ≤10 and >10, but did not include intervention group is shown in the bottom of Table 5. In this model, higher likelihood of HPV series initiation was associated with non-white race, having public insurance and being in a practice that implemented more than ten strategies to increase adolescent HPV vaccination uptake. Higher likelihood of HPV vaccination series completion was associated with being older (14–17 years), non-white and on public insurance; whereas the number of strategies did not have an effect.

Discussion

Since ACIP announced the recommendations that both girls (2007) [23] and later, boys (2011) [24] should receive HPV vaccine to prevent HPV infection, research on potential strategies to increase vaccine uptake has abounded. Strategies that have been studied include: education programs for parents [25] and patients [26–29], patient reminder/recall systems using post cards [29], phone calls [28, 30], and text messaging [31]; social marketing [32]; provider education [26, 30, 32]; EMR alerts [26, 30]; incentives [33]; and audit and feedback [26, 33]. A 2015 review found that practice- and community-based interventions can result in improved HPV vaccination rates [34]. Furthermore, a review of barriers to HPV vaccination found that recommendation by a health care professional is one of two most influential stages where improvement to HPV vaccination could occur [35].

In this randomized controlled cluster trial, the intervention primarily focused on behavior change at the primary care practice level. Practices could choose from among many of these previously studied strategies that were combined into the 4 Pillars^™ Practice Transformation Program, and tailor their implementation to fit their practice’s unique setting, population and culture. Guided by the 4 Pillars^™ Program, the intervention group increased HPV series initiation 10 PP to 63% compared with a 7 PP increase in the control group. Preliminary data from the following year that evaluated maintenance of the 4 Pillars^™ Program, suggests an increase in HPV series completion (unpublished data, personal communication, CJ Lin 2016). Patients in practices using more than ten strategies were more likely to have initiated the HPV vaccine series by the end of the active study period.

This intervention was equally effective for males and females for HPV series initiation, as there was no significant difference in PP increases between males and females in the intervention group. Furthermore, the PP increases for males for HPV vaccination series completion were significantly larger than for females, reducing the sex difference in overall completion rates. This intervention may be a way to promote vaccination for adolescent males without specifically targeting male patients. This finding of reduced disparities as a result of a practice-based intervention has been observed in a previous study that reduced racial disparities in children’s influenza vaccination without specifically targeting minority children [36].

Similar to national data [2], low income (as evidenced by non-commercial health insurance), and non-white race were significantly related to higher odds of HPV series initiation and non-white race was significantly related to higher odds of HPV series completion.

MCV rates increased by approximately 10 PP among 11–13 year olds in both intervention and control groups and Tdap increased significantly (4.5 PP) to 97% in the intervention group compared with a 3.6 PP (P<0.05) increase to 97% for Tdap in the control group. We anticipated that MCV and Tdap rates would not differ substantially between intervention and control groups due to existing school entry requirements in our locale.

Simultaneous administration of MCV and HPV vaccines has been suggested as a means to reduce missed opportunities to initiate HPV vaccination [37], and simultaneous HPV vaccination with other adolescent vaccines is an important strategy in the 4 Pillars^™ Program. Other research has indicated that missed opportunities are an important cause of HPV series completion failure [38]. Physician recommendation at every visit can increase HPV vaccination by encouraging HPV initiation which is associated with completion, and by reducing missed opportunities to complete the HPV series [35, 39].

Previous research has found that provider interventions were significantly more effective compared with parent/patient interventions [30], no intervention controls [33], or parent/patient educational programs in general [40]. Moreover, there is some evidence that provider interventions are more effective for improving HPV series initiation, and parent/patient interventions such as recall reminders or education are more effective for improving HPV series completion [26, 28]. In this study that focused on activities performed by or at the provider’s office, overall increases in HPV series initiation were larger in intervention than control groups and in groups that utilized more than 10 strategies. These efforts did not carryover to HPV series completion, likely due to the short time frame of the trial.

Increases in vaccination rates in control practices

The increases in vaccination in the control practices may be attributed to several factors. First, Hawthorne and publicity effects likely occurred because the practices had agreed to participate in an intervention study and, furthermore, this was discussed publicly at a PBRN meeting devoted to this study as well as a PBRN steering committee meeting. Second, transference, a.k.a., bleed, of the intervention to control practices within networks likely occurred. This phenomenon is well-described in cluster trial interventions [41, 42]. Third, secular trends may have contributed; however, national HPV vaccination rates are increasing at a lower rate than in this study. Percentage point increases in rates of receipt of ≥1 HPV dose among girls 13–17 years old were 3.5 from 2012 to 2013 and 3.3 from 2013–2014 and among boys 13–17 years old were 13.8 from 2012–2013 and 8.1 from 2013–2014[43] [2].

Fidelity and Number of Strategies Implemented

Practices in this study were supported in the use of the 4 Pillars^™ Program following the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) model [44]. For example, improving patient notification about needed vaccines and increasing convenience of vaccinations (Reach); implementing standing order protocols (Effectiveness); creating an immunization champion role (Adoption); supporting the use of site-specific immunization strategies via conference calls and an online dashboard to track progress (Implementation); and motivating staff by sharing progress towards goals via progress charts (Maintenance) were all possible strategies that could be used by sites to improve adolescent vaccination. Fidelity to the intervention may be analyzed by assessing the number of interventions or strategies implemented.

The likelihood of HPV vaccination was examined accounting for demographic characteristics and the number of strategies used. Practices would need to implement more than 10 strategies to double the odds ratio for HPV series initiation; whereas, the number of strategies did not change the likelihood of HPV series completion over the 9-month active study period. These findings, as well as the variable degree of success of different strategies from previous research, suggest a need to use both a substantial number of intervention strategies and combination of provider- and parent/patient-focused strategies that are specific to the individual practice. While the majority of the strategies in the 4 Pillars^™ Program are provider-focused, there are many strategies directed at patient/parent education and communication, especially in Pillar 2 – Patient Communication. Furthermore, the 4 Pillars^™ Program encourages practices to test the effectiveness of strategies in Plan-Do-Study-Act cycles, modify their approach if necessary and create strategies to optimize their efforts. Thus the 4 Pillars^™ Program is a practical, adaptable mechanism for guiding practice improvement change.

Strengths and Limitations

This is one of the few randomized controlled cluster trials to test a combination of strategies to improve adolescent HPV vaccine uptake in primary care. It benefits from a large sample of patients and a diversity of settings and patient populations, increasing its generalizability.

Several limitations should be noted. First, although conducted appropriately, randomization of the practices did not result in equal distribution of baseline vaccination rates, patient race, or insurance coverage, which makes interpretation more difficult. Baseline rates were not available to us at the outset of the study for the cohort of patients with at least one visit during both baseline and intervention. Randomization was based on type of practice, location and the researchers’ general knowledge of the patient populations. As it turned out, of the two family medicine residency sites and two pediatric residency sites that were paired by specialty, the larger of each pair was randomized to the control group. These practices tend to care for large proportions of disadvantaged and minority children. Second, we cannot exclude an influence of a ceiling effect, although given the moderate HPV vaccination rates, we feel that this is unlikely. Third, the modest length (nine months) of the intervention limited its ability to observe differences in HPV series completion (which requires at least six months from start to finish) between intervention and control groups. The intervention was terminated at this point so that the control group could receive the intervention, as promised at recruitment. Fourth, two intervention sites dropped out; one was due to EMR issues (conversion from paper to electronic records occurred later than anticipated). The other site was one of a pair of sister offices, one of which chose not to participate after the group initially agreed. Finally, although the average practice size was 543, which is more than double the size used in the power calculations, some smaller sites were included.

Conclusions

In a randomized controlled cluster trial, HPV series initiation increased significantly more in intervention than control primary care practices. Using more than 10 strategies from the 4 Pillars^™ Practice Transformation Program was effective for increasing HPV series initiation among adolescents. Thus, achieving meaningful changes in adolescent vaccination rates requires a concerted effort by primary care practices across several intervention domains.