Maintenance of Increased Childhood Influenza Vaccination Rates 1 Year After an Intervention in Primary Care Practices

Abstract

Objective:

Influenza vaccination rates among some groups of children remain below the Healthy People 2020 goal of 70%. Multistrategy interventions to increase childhood influenza vaccination have not been evaluated recently.

Methods:

Twenty pediatric and family medicine practices were randomly assigned to receive the intervention in either year 1 or year 2. This study focuses on influenza vaccine uptake in the 10 year 1 intervention sites during intervention and the following maintenance year. The intervention included the 4 Pillars Immunization Toolkit—a practice improvement toolkit, early delivery of donated vaccine for disadvantaged children, staff education, and feedback on progress. During the maintenance year, practices were not assisted or contacted, except to complete follow-up surveys. Student’s t tests assessed vaccine uptake of children aged 6 months to 18 years, and multilevel regression modeling in repeated measures determined variables related to the likelihood of vaccination.

Results:

Influenza vaccine uptake increased 12.4 percentage points (PP; P < .01) during active intervention and uptake was sustained (+0.4 PP; P >. 05) during maintenance, for an average change of 12.7 PP over all sites, increasing from 42.2% at baseline to 54.9% (P < .001) during maintenance. In regression modeling that controlled for age, race, and insurance, likelihood of vaccination was greater during intervention than baseline (odds ratio 1.47; 95% confidence interval 1.44–1.50; P < .001) and greater during maintenance than baseline (odds ratio 1.50; 95% confidence interval 1.47–1.54; P < .001).

Conclusions:

In primary care practices, a multistrategy intervention that included the 4 Pillars Immunization Toolkit, early delivery of vaccine, and feedback was associated with significant improvements in childhood influenza vaccination rates that were maintained 1 year after active intervention.

The national childhood influenza vaccination rate has increased significantly since the recommendation for universal childhood vaccination in 2008.¹ Among all children 6 months to 17 years of age, the percentage vaccinated was 24% in 2008–2009²; 44% in 2009–2010³; 43% in 2010–2011⁴; 52% in 2011–2012⁵; 57% in 2012–2013.⁶ Although vaccination rates among children aged 6 to 23 months have exceeded5 the Healthy People 2020 goal of 70%,⁷ secular trends indicate an overall slowing in the rate of increase. Moreover, rates among children aged 13 to 17 years remain below 50%^5,8 and rates reported from individual practices and regional studies are well below goals for certain demographic groups, including older children, racial minorities, and those without health insurance.^9,10 These disparities suggest a need for interventions that raise rates among all groups of children.

Few studies have been published about interventions that were specifically designed to increase childhood influenza vaccination following the recommendation for universal influenza vaccination for children aged ≥6 months. Of 4 studies identified, 3 were limited to specific demographic groups (low-income^11,12 or high-risk children¹³) and limited the type of intervention strategy being tested (community-centered education,¹¹ mailed reminders,¹³ and text message reminders¹²). Only our study was a multistrategy intervention among children across the socioeconomic and age spectrum; year 1 results of this study have been published.10 Each of these studies reported significant increases in influenza vaccination rates as a result of the intervention or interventions; however, none has measured whether the rates were maintained after the intervention period ended.

The present study evaluated the effect of a single-season, multistrategy intervention program to raise influenza vaccination rates among children aged 6 months to 18 years in primary care practices and maintain them over an additional year. This report describes the 2-year experience of the practices randomized to the year 1 intervention.

Methods

This trial covered 3 influenza seasons; 2010–2011 was the baseline year, 2011–2012 (year 1) was the active intervention year, and 2012–2013 (year 2) was the maintenance year, in a repeated-measures design. The study was approved by the University of Pittsburgh institutional review board. The Clinical Trial Registry Name/Number are “From Innovation to Solutions: Childhood Influenza”/NCT01664793.

Sample Size Calculation and Site Selection

Optimal Design software, version 1.77 (University of Michigan, 2006) was used to calculate the sample size for a cluster randomized trial seeking a 10% to 15% absolute increase in vaccination rate and a minimum practice size of 100 to 200 pediatric patients. Twenty clusters¹⁴ were necessary to achieve 80% power with an alpha of 0.05. To be eligible, each site must have had a patient population of at least 200 children aged 6 months through 18 years, access to vaccination data via an electronic medical record (EMR), and willingness to implement the intervention. Primary care pediatric and family medicine practices from 2 University of Pittsburgh practice-based research networks (http://www.pedspittnet.pitt.edu/; http://www.familymedicine.pitt.edu/content.asp?id=2353) and 1 clinical network were solicited until 20 sites agreed to participate.

Participating sites were stratified by location—inner city (urban practices with primarily disadvantaged children), urban, suburban, and rural—and by discipline (pediatrics vs family medicine), then randomized into the year 1 or year 2 intervention. All consort criteria for a randomized cluster trial¹⁴ were met.¹⁰

Interventions

The intervention was designed using the Diffusion of Innovations theory¹⁵ and included the 4 Pillars Immunization Toolkit (http://www.pittvax.pitt.edu/child-flu-toolkit), provider education, feedback on influenza vaccines provided, and early delivery of donated vaccines for disadvantaged children to ensure that vaccine was available contemporaneously for commercially insured and Vaccines for Children–supported children. The intervention has been described in detail,¹⁰ as have the results for the first year of intervention. Briefly, the 4 Pillars Immunization Toolkit includes background on the importance of protecting children against influenza, barriers to increasing influenza vaccination from both provider and parent/patient perspectives, and strategies to eliminate those barriers. Practices were expected to implement strategies from each of the 4 pillars, which were developed from 4 key evidence-based^16,17 strategies: pillar 1—convenient vaccination services; pillar 2—notification of patients about the importance of immunization and the availability of vaccines; pillar 3—enhanced office systems to facilitate immunization; and pillar 4—motivation through an office immunization champion. A summary of the intervention strategies, including the 4 Pillars, is included in Online Appendix Table 1. Intervention sites were not assisted or contacted during year 2 except to complete a follow-up survey.

Data Collection

Demographic, office visit, and influenza vaccination data were derived from EMR data extractions 3 months after each influenza season. The Center for Assistance in Research Using the Electronic Record (CARe) served as the honest broker to retrieve deidentified data from the EMR. Office visit codes were those that would capture preventive visits, counseling visits, and consult visits that took place between July 1, 2010 and February 28, 2011; July 1, 2011 and February 29, 2012; and July 1, 2012 and February 28, 2013. Influenza vaccination procedure codes for the same time periods were used. Data for children from participating practices also included race, sex, age 6 months to 18 years, and insurance type. A child was considered to be an active patient of the practice and was included in the data set if he or she had a visit between July 1 and February 28/29 for each year of the study, chosen to coincide with each year’s influenza vaccination season because the vast majority of influenza vaccines are provided during these months. Each year, the denominator included all active patients aged 6 months to 18 years, and the numerator was the number of those children who had received at least 1 dose of influenza vaccine.

To measure the degree of implementation¹⁸ and maintenance of strategies, 2 individuals from each site (the lead physician and nurse) were asked to complete a survey that assessed strategy use at the end of each intervention year (1 = yes, 0 = no). For each strategy listed, the responses from each site were averaged and summed across all strategies and divided by 19 in year 1 and 17 in year 2, to provide a percentage. (Early delivery of donated vaccines and provider education did not occur in the maintenance year.)

Statistical Analyses

Data from the EMR extraction were validated by verifying that data were within the requested parameters for site, visit dates, and patient age. Site-specific influenza vaccination rates were calculated for baseline, active intervention, and maintenance years. Paired t tests were used for between-year comparisons of vaccination rates within each site and by t tests for overall vaccination rates. The Cochran-Armitage trend test was used to examine trends in vaccination rates over the 3 study periods. Comparison of the average percentage of strategies implemented for each intervention arm was made using a t test. Multilevel generalized estimating equation modeling, which accounts for the clustered nature of the data—that is, patients are clustered within practices—was conducted using influenza vaccination status as the binary outcome variable. The independent variables included patient level age groups (<2 years, 2–8 years, 9–18 years), race (white vs nonwhite), insurance type (public/self-insured vs commercial), and year (baseline, active intervention, maintenance). Two-way comparisons for each 2- or 3-level independent variable were included in the analysis—for example, <2 years vs 2 to 8 years, <2 years vs 9 to 18 years and 2 to 8 years vs 9 to 18 years. Statistical significance for 2-sided tests was set at α = 0.05. All analyses were performed by SAS/STAT software, version 9.3 (SAS Institute, Cary, NC, USA).

Results

The focus of this analysis is the sustained effect of the 1-year intervention over 2 years; therefore, only the results of the year 1 intervention group are presented here. Results from the year 2 intervention sites can be found in the Online Appendix. Two family medicine and 8 pediatric practices were randomized to the year 1 intervention arm, with the number of children per site ranging from 523 to 7189 (Table 1). There were no significant differences in the overall distributions of age, sex, race, or insurance status between intervention arms, signifying successful randomization. The sample represented a range of patient demographic distributions. Children were evenly divided between boys and girls, approximately 30% of children were nonwhite, 40% were self-insured or publicly insured, 12% were <2 years of age, 45% were 2 to 8 years of age, and 43% were 9 to 18 years of age (Table 1).

Table 1.

Baseline Characteristics of Sites and Patients

Site	n	Discipline*	Location	Race		Insurance		Female, %	Age Group
				White, %	Nonwhite, %	Other, %†	Commercial, %		<2 y, %	2–8 y, %	9–18 y, %
1	537	FM	Suburban	86.0	14.0	22.5	77.5	51.2	3.2	25.7	71.1
2	1357	FM	Inner city	15.4	84.6	68.2	31.8	53.2	9.9	41.5	48.6
3	523	Ped	Urban	39.6	60.4	79.9	20.1	50.1	17.2	44.0	38.8
4	2009	Ped	Inner city	17.1	82.9	79.8	20.2	48.9	19.4	50.9	29.7
5	5913	Ped	Rural	95.6	4.4	32.6	67.4	49.8	10.3	43.3	46.5
6	3886	Ped	Suburban	94.0	6.0	30.4	69.6	50.0	14.2	55.9	29.9
7	3959	Ped	Suburban	89.4	10.6	31.0	69.0	48.4	12.0	49.6	38.4
8	7189	Ped	Suburban	93.5	6.5	24.3	75.7	48.8	11.1	43.6	45.4
9	6047	Ped	Urban	71.8	28.2	22.4	77.6	49.2	12.1	44.9	43.0
10	4114	Ped	Suburban	95.0	5.0	12.2	87.8	50.0	10.8	51.9	37.3
Site average, %				69.7	30.3	40.4	59.6	50.0	12.0	45.1	42.9

^*FM indicates family medicine practice; Ped, pediatric practice.

^†Other indicates self-pay or publicly insured.

Table 2 shows the percentage of children vaccinated in each site for each year of the study. The average percentage vaccinated at baseline was 42.2%. During the active intervention period, influenza vaccine uptake increased 12.4 percentage points (PP; P < .01). Eight of 10 sites significantly increased influenza vaccine uptake. During the maintenance year, average influenza vaccine uptake was sustained (increase from active intervention and maintenance = 0.4 PP; P >.5) resulting from significant improvements in influenza vaccination in 3 sites, no change in 6 sites, and a significant decrease in 1 site (Table 2). Over the 2 years, average influenza vaccine uptake across sites increased by 12.7 PP to 54.9%. When children in all sites were combined, 61.1% of approximately 45,000 children were vaccinated in the second year of the study, increasing significantly from 50.3% at baseline.

Table 2.

Influenza Vaccination Rates During Baseline (July 2010–February 2011), Active Intervention (July 2011–February 2012), and Maintenance (July 2012–February 2013) Years

Site	Baseline Overall (n = 35,534)		Active Intervention Overall (n =42,168)		Maintenance Overall (n = 44,923)		Immediate Intervention Effect	Overall (2 Year) Intervention Effect	Sustained Intervention Effect
	No. of Patients	Percentage Vaccinated (Column a)	No. of Patients	Percentage Vaccinated (Column b)	No. of Patients	Percentage Vaccinated (Column c)	Difference, % (b – a)	Difference, % (c – a)	Difference, % (c – b)
1	537	14.0	603	26.0	478	24.7	12.1*	10.7*	−1.4
2	1357	24.2	1370	43.0	1305	39.9	18.8*	15.8*	−3.0
3	523	28.5	1735	48.2	1885	45.5	19.8*	17.0*	−2.8
4	2009	36.8	6787	65.2	7037	62.1	28.4*	25.4*	−3.0†
5	5913	42.3	5815	52.9	6013	55.1	10.7*	12.9*	2.2**
6	3886	42.1	4174	54.9	4658	56.9	12.8*	14.8*	2.0
7	3959	49.9	4138	60.9	4637	62.8	11.0*	12.9*	1.9
8	7189	54.0	7346	61.1	7745	65.4	7.0*	11.3*	4.3†
9	6047	63.2	5866	64.4	6214	67.0	1.2	3.8*	2.6*
10	4114	67.4	4334	69.3	4951	70.3	1.9	2.9*	1.0
Site average		42.2		54.6		54.9	12.4*	12.7†	0.3
All children combined		50.3		59.7		61.1	9.4*	10.8†	1.4

^*P < .01 for difference in vaccination rates between seasons by paired t tests for within-site comparisons and by t tests for overall comparison.

^**P < .05 for difference in vaccination rates between seasons by paired t tests for within-site comparisons.

^†P < .001 for difference in vaccination rates between seasons by paired t tests for within-site comparisons and by t tests for overall comparison.

Vaccination uptake for each year by demographic group is shown in Table 3. Across age groups, vaccination uptake was highest among the youngest children (6 to 23 months) and lower in older age groups. Nonwhite and self-insured or publicly insured children had consistently lower vaccination uptake than white or commercially insured children. Although racial disparities in vaccination persisted across years, the differences between whites and nonwhites decreased over time. Higher vaccine uptake was observed among males in both the intervention and maintenance years. Using the Cochran-Armitage test for trend, significant increases in vaccination were observed in all groups except children <2 years old, whose rates were above Healthy People 2020 goals at baseline.

Table 3.

Influenza Vaccination by Age, Sex, Race, and Health Insurance During Baseline, Active Intervention, and Maintenance Years^*

	Vaccinated, n (%)
Characteristic	Baseline (n = 35,534)	Active Intervention (n = 42,168)	Maintenance (n = 44,923)	P**
Age group
<2 y	3,124 (73.7)	3,882 (77.4)	4,758 (72.9)	.110
2–8 y	9,094 (54.7)	12,670 (64.3)	13,107 (65.6)	<.001
9–18 y	5,663 (38.6)†	8,614 (49.4)†	9,586 (52.0)†	<.001
Sex
Female	8,759 (49.8)	12,263 (59.1)	13,474 (61.0)	<.001
Male	9,122 (50.8)	12,903 (60.2)‡	13,977 (61.2)	<.001
Race
Nonwhite	2,497 (38.4)	6,435 (58.4)	6,518 (56.4)	<.001
White	15,384 (53.0)†	18,731 (60.1)§	20,933 (62.7)†	<.001
Health insurance
Self- or publicly insured	4,439 (40.3)	9,417 (58.2)	9,903 (56.5)	<.001
Commercially insured	13,442 (54.8)†	15,749 (60.6)†	17,548 (64.1)†	<.001
Overall	17,881 (50.3)	25,166 (59.7)	27,451 (61.1)	<.001

^*Baseline indicates July 2010 to February 2011; active intervention, July 2011 to February 2012; and maintenance, July 2012 to February 2013.

^**P value for difference across the 3 time periods by Cochran-Armitage trend test.

^†P < .001 within intervention year across demographic groups, by chi-square test.

^‡P < .05 within intervention year across demographic groups, by chi-square test.

^§P < .01 within intervention year across demographic groups, by chi-square test.

During active intervention, sites reported using an average 14.7 out of 19 toolkit and/or intervention strategies (77.2%; range, 71% to 89%). During the maintenance year, sites reported using an average 11.7 out of 17 toolkit strategies (69%; range, 47% to 97%), indicating moderate retention of the practice change intervention techniques.

Results of the regression analysis accounting for age group, race, insurance type, and year are shown in Table 4. The percentage of strategies used and sex were not related to likelihood of vaccination and were excluded from the model. The likelihood of vaccination was higher in younger age groups than in older age groups, in white children than in nonwhite children, and those who were commercially insured compared to self-insured or publicly insured. The likelihood of vaccination in both the active intervention year and the maintenance year was approximately 50% higher compared with baseline, indicating a significant increase in vaccine uptake when adjusting for demographic factors. The likelihood of vaccination in the maintenance year compared with the active intervention year was approximately 2% higher, indicating that the sites were able to sustain vaccination levels during the maintenance year.

Table 4.

Likelihood of Child Receiving Influenza Vaccine in Gener-alized Estimating Equation (GEE) Modeling^*

Independent Variable	Odds Ratio (95% Confidence Interval)	P
2–8 y, reference = <2 y	0.54 (0.52–0.56)	<.001
9–18 y, reference = <2 y	0.31 (0.30–0.32)	<.001
9–18 y, reference = 2–8 y	0.57 (0.56–0.59)	<.001
White race, reference = nonwhite race	1.15(1.11–1.19)	<.001
Commercial insurance, reference = self-insurance and public insurance	1.33 (1.29–1.37)	<.001
Active intervention, reference = baseline	1.47 (1.44–1.50)	<.001
Maintenance, reference = active intervention	1.02 (1.00–1.05)	.024
Maintenance, reference = baseline	1.50 (1.47–1.54)	<.001

^*GEE regression model includes influenza vaccination status as the binary outcome variable and age groups, race, insurance type, and year as the independent variables.

Discussion

National survey data reveal that childhood influenza vaccination rates have increased steadily since the universal influenza vaccination recommendations for children older than 6 months were adopted in 2008, with the largest increases and highest rates observed among the youngest children (6 to 23 months of age).^2–6 These data suggest that primary care practices are effectively vaccinating children <2 years old, who are seen frequently for well-child and immunization visits. Conversely, as children get older, the number of visits needed to receive other recommended vaccines declines, as do influenza vaccination rates. Although these data suggest that strategies for reaching older children and adolescents are needed, few randomized trials to increase childhood influenza vaccination have been published since 2008.

The 4 Pillars Immunization Toolkit combines evidence-based strategies and techniques^16,17 for increasing immunizations and is designed to assist practices with their quality improvement processes. The toolkit, along with community education, in-service training, and early delivery of donated vaccine for disadvantaged children, helped 8 of 10 practices make significant improvements in their childhood influenza vaccination uptake after 1 year (mean increase = 12.4 PP, from 42.2% to 54.6%). At the end of the maintenance year, influenza vaccination was sustained at a level significantly higher than at baseline despite receiving no additional support to maintain the practice improvement. Although the practices no longer had early delivery of vaccine supplies or the direct support of the research team, they were able to sustain the changes they had made, presumably because the strategies outlined in the 4 Pillars Immunization Toolkit can be individualized to suit the practice’s structure and culture. The regression analysis that accounted for age, race, insurance type, and year confirmed the success of the intervention and its ability to be maintained for a year after active intervention.

Moreover, significant improvements from baseline were observed in older children, nonwhite children, and self-insured or publicly insured children—groups whose vaccination rates are typically more intractable. The differences we observed between boys and girls were not observed in previous reports of this study¹⁰ or in national estimates⁵ but have been reported by others.¹⁹

Previous research has indicated that a one-size-fits-all approach to practice change is less than ideal²⁰ and that sustainability of practice change has been attributed to using a tailored intervention²¹ and the presence of a practice champion.²² The 4 Pillars Immunization Toolkit offers a set of integrated strategies in 4 categories to allow customization by the individual practice. This ability to apply the toolkit’s strategic approaches in an individualized manner likely contributed to the long-term success of the intervention. We found at the end of the maintenance year that practices were still using 69% of the toolkit strategies.

Stange et al²¹ reported increased immunization rates after an intensive 9-month intervention that were maintained at 24 months using feedback mailed to practices every 6 months. In the current study, feedback on influenza doses provided was given weekly during the active intervention year but was discontinued in the maintenance year. Hence, we are unable to evaluate the effect of this strategy on long-term maintenance.

Failure to implement or maintain practice improvement projects often results because the changes required do not account for the many competing demands in primary care. These competing demands include a growing list of screenings and preventive care measures, time and resource constraints of the practice, treatment priorities, patient fears and questions, perceived patient financial constraints, and the dynamic nature of a patient visit.^23,24 Thus, interventions that are designed to function within the context of other competing demands, such as standing order policies that place responsibility for assessing/vaccinating without an individual physician order (pillar 3, enhanced vaccination systems) are likely to be more successful and sustainable.

Strengths and Limitations

This pre/post trial of 10 primary care practices serving over 40,000 children aged 6 months to 18 years reports on 2 seasons of influenza vaccination after a single season of practice improvement intervention. Although it is not possible to sustain some of the interventions such as, early delivery of vaccine supplies, externally generated feedback, or facilitation by a research team, the toolkit strategies can be customized to support long-term sustainability within the practice. The study’s limitations include the fact that children younger than 9 years who received at least 1 dose of vaccine were counted in the numerator as vaccinated, as we were unable to determine which children were first-time vaccinees and required 2 doses. It is possible that the observed changes in vaccine uptake reflected secular trends. We observed significant increases in the year 2 group during their control year.¹⁰ However, significant reductions in age disparities support the intervention’s effectiveness (eg, the observed rates in 9- to 18-year-olds are higher than those reported in national survey data). The time period for calculating vaccination rates chosen for this study is shorter than that typically used by national studies. We chose July 1 to February 28/29 for several reasons. First, some influenza vaccine, such as live attenuated influenza vaccine, has a short shelf life and typically expires long before the beginning of the next vaccination season; second, new commercial vaccine orders usually begin to arrive in practices in August, with Vaccines for Children influenza vaccine arriving later²⁵; and finally, most influenza vaccines are provided by the end of December. Furthermore, our estimates are based on EMR data, not self-report. We believe these rates present a more realistic view of primary care–administered influenza vaccine.

Conclusion

The 4 Pillars Immunization Toolkit is an effective, evidence-based guide to assist primary care practices with increasing childhood influenza vaccination rates. Improved vaccination levels were maintained during the following season when practices sustained their use of the toolkit strategies, reinforcing the value of the individualized selection of practice change strategies.

What’s New.

A multistrategy intervention including a practice improvement toolkit, provider education, early delivery of donated vaccines, and feedback on progress was successful for increasing and maintaining childhood influenza vaccination rates over 2 years in primary care practices.