Cluster Randomized Trial of a Toolkit and Early Vaccine Delivery to Improve Childhood Influenza Vaccination Rates in Primary Care

Abstract

Purpose

To increase childhood influenza vaccination rates using a toolkit and early vaccine delivery in a randomized cluster trial.

Methods

Twenty primary care practices treating children (range for n=536-8,183) were randomly assigned to Intervention and Control arms to test the effectiveness of an evidence-based practice improvement toolkit (4 Pillars Toolkit) and early vaccine supplies for use among disadvantaged children on influenza vaccination rates among children 6 months-18 years. Follow-up staff meetings and surveys were used to assess use and acceptability of the intervention strategies in the Intervention arm. Rates for the 2010-2011 and 2011-2012 influenza seasons were compared. Two-level generalized linear mixed modeling was used to evaluate outcomes.

Results

Overall increases in influenza vaccination rates were significantly greater in the Intervention arm (7.9 percentage points) compared with the Control arm (4.4 percentage points; P<0.034). These rate changes represent 4522 additional doses in the Intervention arm vs. 1,390 additional doses in the Control arm. This effect of the intervention was observed despite the fact that rates increased significantly in both arms - 8/10 Intervention (P<0.001) and 7/10 Control sites (P-values 0.04 to <0.001). Rates in two Intervention sites with pre-intervention vaccination rates >58% did not significantly increase. In regression analyses, a child's likelihood of being vaccinated was significantly higher with: younger age, white race (Odds ratio [OR]=1.29; 95% confidence interval [CI]=1.23-1.34), having commercial insurance (OR=1.30; 95%CI=1.25-1.35), higher pre-intervention practice vaccination rate (OR=1.25; 95%CI=1.16-1.34), and being in the Intervention arm (OR=1.23; 95%CI=1.01-1.50). Early delivery of influenza vaccine was rated by Intervention practices as an effective strategy for raising rates.

Conclusions

Implementation of a multi-strategy toolkit and early vaccine supplies can significantly improve influenza vaccination rates among children in primary care practices but the effect may be less pronounced in practices with moderate to high existing vaccination rates.

Introduction

Despite the 2008 Advisory Committee on Immunization Practices recommendation that all children over the age of 6 months receive an annual influenza vaccine [1], national vaccination uptake in the United States remains substantially below desired levels of 70% [2], averaging 51.5% An array of [3]. An array of evidence-based interventions to improve childhood influenza vaccine uptake exists [4-7]. While significant gains have been reported, no single intervention has raised rates sufficiently; rather, the evidence suggests the need for a combination of strategies. The Community Preventive Services Task Force (Task Force) [8] recommended using two or more of three strategic approaches in preference to using several techniques within a single strategic approach. They are: 1) enhancing access to vaccination services; 2) increasing demand among patients; and 3) provider- and system-based interventions such as reminders, modified office flow, standing order programs (SOPs) and electronic immunization tracking.

Based on Task Force recommendations [8] and previous research in adult primary care practices [9], we modified an adult immunization toolkit to create the 4 Pillars Toolkit for Increasing Childhood Influenza Immunization (4 Pillars Toolkit) in primary care practices serving children. A practice-based, cluster randomized trial was conducted using the 4 Pillars Toolkit and early delivery of vaccine supplies for Vaccines for Children (VFC)-eligible children. This report describes: 1) the intervention that included the 4 Pillars Toolkit; 2) resultant changes in influenza vaccination rates; 3) the individual and practice level characteristics that affected influenza vaccination from two-level generalized linear mixed modeling; and 4) recommendations for policy and practice.

Methods

This trial took place during the 2011-2012 influenza season and was approved by the University of Pittsburgh Institutional Review Board.

Sample Size and Sites

Optimal Design software (University of Michigan, Version 1.77. 2006) was used to calculate sample size, for a randomized trial seeking a 10-15% absolute increase in vaccination rate, and a minimum practice size of 100-200 pediatric patients. A sample size of 20 clusters (10 Intervention and 10 Control practices) was determined necessary to achieve 80% power with an alpha of 0.05. Primary care pediatric and family medicine practices from two practice-based research networks (http://www.pedspittnet.pitt.edu/; http://www.familymedicine.pitt.edu/content.asp?id=2353) and one clinical network in Southwestern Pennsylvania were solicited for participation. When 20 sites agreed to participate, solicitation ceased. All sites were part of the UPMC Health System and used a common electronic medical record (EMR), EpicCare, with the exception of one practice with two offices that used a different EMR system (Allscripts Professional).

Cluster Randomization

Cluster randomization allocates clinical practices rather than individuals to the intervention arms [10]; hence, each practice or office was considered as a cluster. To be eligible, the office must have had a patient population of at least 200 children ages 6 months through 18 years, access to vaccination data via an EMR and willingness to make office changes to increase influenza vaccination rates. Participating practices were stratified by location – inner city (urban practices with primarily disadvantaged children), urban, suburban and rural and by discipline (pediatrics vs. family medicine). The practices were than randomized into the Intervention or Control arms within strata with the two offices of one rural practice assigned one to each arm. Practices randomized to the Control arm were informed that their intervention would take place the following season and were not contacted again until the end of the influenza season.

Interventions

The intervention was designed using Diffusion of Innovations theory [11], and included the 4 Pillars Toolkit, provider education, and vaccine supply interventions which are described in Table 1. One of the investigators (MPN) visited each Intervention site before the beginning of the influenza season, and following a standard procedure, introduced the study and the package of interventions at a staff meeting and worked with staff to develop practice-specific ideas for implementing the toolkit. Each Intervention practice received ≤200 doses of donated vaccine for Vaccines for Children (VFC) eligible children until practices received their VFC supplies allowing sites to vaccinate disadvantaged children as early as commercially insured whose supplies typically become available sooner. The intervention was conducted from September 2011 through March 2012.

Table 1. Intervention strategies used to increase childhood influenza vaccination rates and post intervention effectiveness ratings from Intervention sites.

Strategy		Number of sites using strategy	Effectiveness score* Range = 0-100	Debrief session score** Range = 0-100
	4 Pillars Toolkit
Pillar 1	Convenient vaccination services
Convenient influenza vaccination		10	73.8	46.2
Description: Practices were encouraged to reduce access barriers by offering convenient influenza vaccination services such as after-hours vaccine clinics, walk-in vaccination, dedicated vaccination stations, designated vaccination only hours and vaccination offered at any non-febrile illness visit.
Pillar 2	Notification of parents/patients about the importance of influenza vaccination and the availability of vaccine
Office posters		10	76.7	100
Description: The research team downloaded influenza vaccination posters from the CDC website and provided at least enough to post in each practice's exam rooms. Offices were encouraged to create their own posters and fliers to reminder patient, parents and providers.
Patient reminders		5	20.6	Not rated
Description: Practices were strongly encouraged to notify all parents/patients of the availability of influenza vaccine, date and time of any influenza vaccination clinics, and physician recommendations to be vaccinated. Practices could use any appropriate means, e.g., autodialed phone calls, emails, text messages, “on-hold” messages, fliers, social media, verbal reminders at check-in, etc.
Autodialed phone messages		9	70.0	72.6
Description: The research team worked with the practices to send one or more autodialed message in September, October and December to the entire patient population or a subset (e.g., those still not vaccinated by a certain date) of each practice.
Pillar 3	Enhanced office systems to facilitate influenza immunization
Physician prompts		10	90.7	69.3
Description: The electronic medical record (EMR) was programmed to generate a physician prompt called a best practice alert (BPA) which would appear on the computer screen whenever an unvaccinated child was being seen.
Vaccination as part of vital signs(Immunizations given as part of vital signs)		6	31.5	39.6
Description: Practices were to make influenza vaccination routine by having clinical support staff assess immunization status as part of the process of rooming patients and recording vital signs, and by incorporating SOPs into the practice.
Standing Order Protocols (SOPs)		7	58.6	29.7
Description: Staff would inform the parent, obtain consent, give the Vaccine Information Sheet and prepare the vaccine, and when feasible, vaccinate eligible children without the need for a specific physician's order.
Pillar 4	Motivation through an office immunization champion
Immunization champion (IC)		10	67.7	Not rated
Description: The IC in each practice was an enthusiastic motivator who used her or his time and energy to provide feedback, encourage competition and energize the staff to keep up the efforts throughout the season. The IC shared weekly graphs sent by the research team (see descriptions below under provider and community interventions)depicting the number of vaccines given and missed opportunities to vaccinate and used them as a basis to discuss ways to improve or sustain efforts.
	Provider and community interventions
Pre-intervention visits (Staff/Provider education in-service on toolkit)		10	86.6	100
Description: Investigators visited intervention practice to introduce the study and toolkit at a staff meeting and brainstorm ideas for implementing the 4 pillars in practical and meaningful ways for each practice.
Mid-season refresher (Staff/Provider education online)		9	45.0	Not rated
Description: Staff was offered the opportunity to view a short online slide presentation mid-influenza season, and answer a short survey for which they received participant payment.
Weekly reports (Feedback from investigators)		10	84.2	66.0
Description: Based on the previous year's total number of influenza vaccines given, each practice was given an initial goal of 25% increase over the previous year. This goal was divided into 16 weeks and graphed. Weekly counts of vaccines given were plotted on line graphs comparing actual values with the goal and were emailed to the Immunization Champion (IC) from September through mid-December.
Weekly reports (Comparison of progress with other practices)		10	72.9	66.0
Description: ICs also received a bar graph showing their practices' progress compared with the other intervention sites.
Videos		--	Not rated	59.4
Description: The investigators developed two videos based on focus group findings to encourage teens and younger children to be vaccinated. The teen video was advertised in practices using fliers with a headline to catch teens' attention and a QR code for direct linking to smart phones. A second video was a 30 second public service announcement, produced in collaboration with the county health department and a local television station, and featured a local child celebrity. It played on intervention practices' waiting room electronic message boards and aired 280 times on TV from early September through March 2012.
Community outreach		--	Not rated	Not rated
Description: The research team conducted community outreach, primarily in disadvantaged communities, to reach groups with traditionally low vaccination rates, visiting places of worship, community centers and social service agencies, distributing fliers and talking with people gathered there.
	Vaccine supply/policy interventions
Early delivery of Vaccines for Children (VFC) and donated influenza vaccine		10	94.2	100
Description: Selective early delivery of VFC influenza vaccine to Intervention practices and the delivery of 50-750 doses (distributed proportionally to the size of the practice) of donated influenza vaccine for administration to VFC children. Practices were encouraged to extend the vaccination season by vaccinating as soon as supplies arrived until the end of February.
Borrowing of commercial vaccine for VFC patients		8	40.9	39.6
Description: Investigators received permission for practices to borrow commercial supplies of vaccine to administer to VFC patients until VFC supplies arrived.

Note: NA= not asked.

^*Effectiveness score: Average rating by Intervention arm sites for effectiveness of strategy for raising influenza vaccination rates, effectiveness range= 1-100 with 0=did not use.

^**Debrief session score: Overall assessment of Intervention arm sites' staff on techniques. 0=did not use; 1=not effective; 2=moderately effective; 3=very effective. Average score from all practices then multiplied by 33 to adjust to 0=100 range.

Toolkit

The 4 Pillars Toolkit was based on four evidence-based [8, 12] key strategies: Pillar 1 – Convenient vaccination services; Pillar 2 - Notification of patients about the importance of immunization and the availability of vaccines; Pillar 3 - Enhanced office systems to facilitate immunization; Pillar 4 - Motivation through an office immunization champion. Table 1 describes the strategies used in more detail. The 4 Pillars Toolkit includes background on the importance of protecting children against influenza, barriers to increasing influenza vaccination from both provider and parent/patient perspectives and strategies to eliminate those barriers. Practices were expected to implement strategies from each of the 4 pillars.

Data collection

At the end of the influenza season, all Intervention sites were revisited by an investigator who used a discussion guide to get feedback from the staff on which strategies they used and how effective they believed them to be, in order to assess fidelity of the intervention [13]. Notes were summarized and coded into a 4-point scale (0=did not use, 1=not effective, 2= somewhat effective, 3=very effective). In addition, two individuals from each intervention site (head nurse or office manager and lead physician) scored the effectiveness of each study-specific strategy on a scale of 1-100, assigning a zero if their practice did not use the strategy. The scores for each question were averaged across both respondents for each practice. Sites also reported approximate date of receipt of VFC vaccines; months were converted into their corresponding numbers (i.e., September = 9) with the first half of the month (if given) assigned a 0.0 and dates in the second half of the month assigned a 0.5 and dates were averaged for each arm.

De-identified demographic, office visit and influenza vaccination data were derived from EMR data extractions performed by the UPMC Center for Assistance in Research using the eRecord and from a similar data extraction from the EMR by staff of the non-UPMC sites following the 2011-2012 influenza season.

Statistical analyses

Descriptive analyses were performed for patient demographic characteristics (age, sex, race, and health insurance). Chi-square tests were used to examine whether children's characteristics differed between the Intervention and Control arms. Site-specific influenza vaccination rates were calculated for the pre-intervention and intervention years. The denominator was defined as the number of children who had been seen at least once (indicates being an active patient) during 3/1/2010 – 2/28/2011 for the pre-intervention year and 3/1/2011 – 2/29/2012 for the invention year. The numerator was defined as the number of children who had received at least one dose of influenza vaccine during each influenza season (8/1/2010 – 2/28/2011 for the pre-intervention year and 8/1/2011 – 2/29/2012 for the intervention year). Chi-square tests were used to compare vaccination rates in each arm and for each year. Number of doses given was the count of all doses of influenza vaccine given to eligible children between 8/1/2011 and 2/29/2012.

To determine which factors were related to childhood influenza vaccination rates while accounting for the clustered nature of the data, two-level generalized linear modeling was conducted using influenza vaccination status as a binary outcome variable using SAS® 9.3. Patient level variables that were significantly different across arms (age, race, and health insurance) were included in regression analyses. Initially, the practice level independent variables were pre-intervention vaccination rate, intervention arm, number of strategies used to increase vaccination and effectiveness score for individual strategies. Strategies selected for regression analyses were those only available to the Intervention arm (e.g. early delivery of vaccine); Control sites for those strategies were assigned scores of zero. Correlations among all strategy effectiveness scores were tested using correlation coefficients. All independent variables were tested to determine co-linearity removing those with a variance inflation factor (VIF) >10 [14, 15]. A random intercept model with variance components covariance structure was chosen as the final model based on the lowest value of Akaike information criterion. Statistical significance of two-sided tests was set at type I error (alpha) equal to 0.05.

Results

Demographics

Each arm contained two family medicine and 8 pediatric practices, 1 rural and 2 urban practices, but differed in the number of inner city and suburban practices (Table 2). During the pre-intervention year, the Intervention and Control arms did not differ by percent female patients, but Intervention practices overall had a greater proportion of non-white, commercially insured, and younger children than Control practices (P<0.001). The number of eligible children ranged from 536 to 8,183.

Table 2. Demographic characteristics of practices and patients during the pre-intervention season (2010-2011).

Site	N of children	Type of practice*	Location	Race		Insurance		Female (%)	Age Mean (SD)
				White (%)	Non-white (%)	Public/ Self-pay/ Uninsured/(%)	Commercial (%)
Intervention sites
1	536	FM	Suburban	86.0	14.0	22.6	77.4	51.1	11.4 (5.1)
2	1,670	FM	Urban	14.8	85.2	68.1	31.9	52.3	8.8 (5.8)
3	1,083	Peds	Inner city	39.2	60.8	79.3	20.7	48.9	7.1 (5.1)
4	4,317	Peds	Inner city	16.4	83.6	80.5	19.5	49.3	6.0 (4.7)
5	6,780	Peds	Rural	94.0	6.0	33.1	66.9	49.6	8.3 (5.3)
6	4,424	Peds	Suburban	93.1	6.9	30.6	69.4	50.1	6.4 (4.4)
7	4,541	Peds	Suburban	88.7	11.3	31.0	69.0	48.4	7.4 (4.9)
8	8,183	Peds	Suburban	93.1	6.9	23.8	76.2	48.6	8.2 (5.3)
9	7,040	Peds	Urban	71.0	29.0	22.4	77.6	49.0	7.9 (5.3)
10	4,719	Peds	Suburban	94.0	6.0	12.4	87.6	49.8	7.3 (4.8)
Control sites
11	1,276	FM	Inner city	38.6	61.4	87.8	12.2	55.1	9.4 (6.1)
12	3,107	Peds	Suburban	73.1	26.9	72.9	27.1	48.9	8.7 (5.3)
13	5,810	Peds	Suburban	72.7	27.3	65.7	34.3	49.3	8.8 (5.5)
14	549	FM	Suburban	94.0	6.0	27.0	73.0	50.8	11.0 (5.6)
15	2,702	Peds	Rural	95.5	4.5	22.2	77.8	47.0	7.9 (5.2)
16	5,653	Peds	Urban	63.8	36.2	35.8	64.2	47.6	7.5 (4.9)
17	6,264	Peds	Suburban	86.6	13.4	16.2	83.8	48.9	8.3 (5.1)
18	4,876	Peds	Suburban	93.4	6.6	12.4	87.6	49.3	8.6 (5.1)
19	3,234	Peds	Suburban	91.2	8.8	9.2	90.8	48.2	8.6 (5.0)
20	4,835	Peds	Urban	68.5	31.5	29.4	70.6	48.4	7.1 (5.2)
Control sites, overall N=38,306				77.5	22.5 †	35.6	64.4 †	48.8	8.2 (5.2) ‡
Intervention sites, overall N=43,293				77.2	22.8	34.0	66.0	49.3	7.6 (5.2)

^*Type: FM=Family Medicine practice; Peds=Pediatric practice.

SD=Standard Deviation

^†P<.001 for difference between Intervention and Control arms by Chi square test.

^‡P<.001 for difference between Intervention and Control arms by Wilcoxon test due to non-normal distribution of ages.

Vaccination

Overall pre-intervention influenza vaccination rates were similar in the Intervention (46.0%) and the Control arms (45.7%; P=.373, Table 3). In the intervention season, the rate in the Intervention arm (53.8%) was significantly greater than that for the Control arm (50.1%; P<0.001), with an average pre-intervention to intervention change in vaccination rate of 7.9 percentage points (PP) for the Intervention arm and 4.4 PP for the Control arm (P=0.034). Influenza vaccination rates increased significantly in eight of ten Intervention practices (P<0.001) with absolute differences ranging from 0.6 PP to 21.5 PP, and in seven of ten Control sites (P values=0.04 to <0.001) with differences ranging from -3.2 PP to 9.4 PP. The two Intervention practices that did not significantly increase their vaccination rates were those with pre-intervention rates >58%. Omitting the practices with pre-intervention rates >58% resulted in an average pre-intervention to intervention change in rates of 12.1 PP in the Intervention arm and 4.6 PP in the Control arm (P=.005 for the difference).

Table 3. Influenza Vaccination Rates in Intervention and Control Sites for the Pre-intervention (2010-11) and Intervention (2011-12) Seasons.

Site	Children (n)	Preintervention season (2010-11) %	Children (n)	Intervention season (2011-12) %	Absolute difference (percentage point)	P value*
Intervention sites
1	536	14.0	712	23.0	9.0	<0.001
2	1,670	21.0	1,661	37.5	16.5	<0.001
3	1,083	26.1	2,123	40.6	14.5	<0.001
4	4,317	35.5	7,925	57.0	21.5	<0.001
5	6,780	39.0	6,743	48.0	9.0	<0.001
6	4,424	39.4	4,821	50.8	11.4	<0.001
7	4,541	45.4	4,748	54.7	9.3	<0.001
8	8,183	50.3	8,376	56.2	5.8	<0.001
9	7,040	58.2	6,942	58.8	0.6	0.49
10	4,719	63.6	4,988	64.5	0.9	0.37
Overall	43,293	46.0	49,039	53.9	7.9†	<0.001
Control sites
11	1,276	14.7	1328	24.2	9.4	<0.001
12	3,107	31.3	2,864	40.7	9.4	<0.001
13	5,810	31.8	5,656	36.4	4.6	<0.001
14	549	32.2	578	29.1	-3.2	0.25
15	2,702	42.8	2,841	51.4	8.6	<0.001
16	5,653	43.7	5,559	49.1	5.4	<0.001
17	6,264	52.2	6,457	56.1	3.9	<0.001
18	4,876	53.4	4,942	55.4	2.0	0.04
19	3,234	54.6	3,358	55.8	1.2	0.32
20	4,835	62.9	5,043	63.3	0.4	0.65
Overall	38,306	45.7	38,626	50.1	4.4†	<0.001

^*For difference in vaccination rates between pre-intervention and intervention seasons.

^†Difference between Intervention and Control arms P<0.034.

Among all Intervention sites 4,522 more doses were given in the intervention year over the previous year for a total of 29,863 doses, whereas among all Control sites in the same season, total doses increased by 1,390 to 22,088. On average, Intervention practices received VFC supplies approximately 1 month earlier (mid-August) than Control practices (mid-September). Some Control sites received VFC influenza vaccine as late as October and November.

Intervention

The average effectiveness scores from the surveys and the debrief sessions for the intervention strategies and the number of Intervention sites using them are shown in Table 1. The strategies rated as most effective by practice leadership were early delivery of influenza vaccines donated by a vaccine manufacturer that could be used for VFC children (94.2); electronic physician prompts (90.7); pre-intervention in-service visits (86.6); weekly feedback on rates from the investigators to the immunization champion (84.2); posters (76.7) and express vaccination services (73.8). These results were generally similar to the ratings given by the staff at the follow-up meetings in which 7 practices reported using physician prompts and express vaccine clinics and 10 practices reported using early delivery of vaccine, provider in-service meetings and posters.

Using effectiveness scores, regression analyses were conducted to examine which of the intervention strategies influenced likelihood of vaccination among children in the Intervention practices. Out of 14 strategies, six had a significant impact on likelihood of vaccination. They were preseason in-service meetings (OR=1.03; 95% CI=1.00-1.05; P=0.038); early delivery of influenza vaccine (OR=1.03; 95%CI=1.00-1.05; P=0. 021); borrowing commercial vaccine for VFC children (OR=1.05; 95%CI=1.02-1.08; P=.002); feedback on immunization rates from the research team to the immunization champion (OR=1.03; 95%CI=1.01-1.06; P=0.010); comparisons of the practices' progress to one another (OR=1.04; 95%CI=1.01-1.06; P=.006); and feedback on immunization rates from the immunization champion to the staff (OR=1.05; 95%CI=1.02-1.07; P<.001). These ORs indicate that for every 10 point increase in a strategy's effectiveness score, the odds of vaccination increased by 3%-5%. Co-linearity among these strategies precluded their inclusion in further regression analyses.

In final regression analyses, (Table 4) younger children, white children (OR=1.29; 95%CI=1.23-1.34) and commercially insured (OR=1.30; 95%CI=1.25-1.35) children were more likely to be vaccinated than their older (OR=0.91; 95%CI=0.90-0.91), non-white and publicly insured counterparts. Furthermore, children in practices with higher pre-intervention vaccination rates (OR=1.25; 95%CI=1.16-1.34) and those in Intervention practices (OR=1.23; 95%CI=1.23-1.50) were significantly more likely to be vaccinated; the latter finding indicates the positive effect of the intervention while controlling for baseline rate.

Table 4. Patient and practice level variables related to vaccination status in two-level generalized linear mixed modeling.

Variable	Odds Ratio (95% CI)	P value
Patient level variables
Age	0.91 (0.90-0.91)	<0.001
White race (ref. = non-white)	1.29 (1.23-1.34)	<0.001
Commercial health insurance (ref. = public/self-pay/uninsured)	1.30 (1.25-1.35)	<0.001
Practice level variables
Pre-intervention vaccination rate (unit=10% increase)	1.25 (1.16-1.34)	<0.001
Intervention (ref. = Control)	1.23 (1.01-1.50)	<0.05

Discussion

This study employed provider and patient education, early access to vaccine for low income children and an immunization practice improvement toolkit to raise childhood influenza vaccination rates in pediatric and family medicine practices. These interventions were presented to practices as a package which could be adapted to fit the structure and culture of individual sites. Both Intervention and Control arms significantly increased vaccination rates overall; however the absolute change in rate in the Intervention arm was significantly higher. The observed change in rate in the Control arm may be due to community interventions, secular increases in national rates, or simply because the practices had agreed to participate in the study [16]. The intervention was effective despite the larger practice sizes and the increase in patients in the Intervention sites (Table 3), both of which can inhibit practice change. The final vaccination rate in the Intervention arm (53.8%) is somewhat higher than previous studies among high risk children which reported post intervention rates centering around 30% but reaching as high as 62% [4, 5, 7, 17-23]. Studies of all children 6 months to 18 years of age or healthy infants only, are fewer in number, but reported changes in rates among infants ranged from 20 PP to 34 PP [7, 19, 24], with one intervention study reporting an overall intervention rate of 44% [6].

Practices with pre-intervention vaccination rates above 58% did not significantly improve rates as a result of the intervention, indicating a possible threshold effect. We speculate that practices with a high pre-intervention rate viewed themselves as already doing all that was feasible to vaccinate against influenza. Few studies have reported overall vaccination rates above 50%, with one observational study [25] reporting a maximum of 60% among 118 pediatric and family practices across the country. Thus, it may be difficult to achieve the 70% national goal by relying solely on primary care practices to vaccinate. Perhaps expansion to other venues such as school-based influenza vaccination clinics [26] or for admission to child care [27] are the best means to reach children who are not receiving influenza vaccine from their doctors.

The effect of age on vaccination rates seems to be consistent across studies including the present study, with younger children more likely to be vaccinated than older children [3, 25, 28]. Medicaid-insured children have been reported as more likely to be vaccinated than privately insured or uninsured children at community health centers [29], but were less likely to be vaccinated in the present study, which included some community health centers and may be due to later delivery of VFC vaccines to Control sites. Differences in influenza vaccination rates across racial groups vary, with no differences reported between black and Latino low income children [30], higher rates among Asian and Hispanic children than among white children in community health centers [29], higher rates among white children than black children in inner-city practices [19, 31] and in this study. These differences may be attributed to the demographic differences of the source population and the types of health centers studied.

In this study, vaccination was encouraged as soon as vaccine arrived and continued past December when influenza vaccination typically tapers off. Intervention practices received donated influenza vaccine supplies to be used for non-insured and VFC children, received preferential early delivery of VFC influenza vaccine through arrangements made with the Pennsylvania Department of Health and also were given permission to borrow commercial supplies to vaccinate VFC children if needed for adequate supply. One barrier that may prevent practices from vaccinating as many children as possible is the typical delay in delivery of VFC influenza vaccine supplies relative to commercial supplies [32]. VFC-eligible children who visit the practice before supplies arrive often leave unvaccinated and may not return later in the season to be vaccinated. Although the difference in delivery dates is decreasing, studies have reported that VFC vaccines arrive 2-4 weeks later than commercial supplies, which results in lower two dose compliance rates [32, 33]. Intervention practices rated early delivery of vaccine supplies as the most effective strategy available to them during the intervention. The timing of the distribution of VFC influenza vaccine to providers is determined by individual state immunization programs based on their receipt of vaccine from federal depots and their program priorities. Hence, early distribution of all VFC vaccine is not always possible; however, early distribution of a portion of VFC vaccine early in the season has the potential to increase vaccination rates among VFC-eligible children.

Motivational efforts by the immunization champion were also rated as effective by Intervention practices. With a long vaccination season (up to six months), the efforts of the immunization champion to motivate the office staff are an important element of a successful vaccination program. Recent studies have not reported on a pre-influenza season staff educational session, motivation, or an immunization champion as essential parts of an influenza vaccination improvement package, but they are relatively low cost and easy strategies to implement [12, 34]. Influenza vaccination of children is cost-saving in the US [35], provided that vaccine costs <$20-25 [36]. Furthermore, a variety of quality improvement recommendations are specifying that primary care practices increase immunization rates [37], and in some cases are being financially rewarded for improvements. These benefits should outweigh the potential cost of educational programs and monitoring rates.

These findings suggest that efforts to improve influenza vaccination by practices should include: offering vaccine as early as possible, assigning an immunization champion, educating the staff about vaccination procedures, and providing regular feedback to providers and staff about the practice's vaccination rates and progress towards its goals. The ability to improve childhood influenza vaccination rates may depend upon the demographic distribution of the practice's patient population, its current vaccination rate and its overall efforts to achieve better coverage. If there is a threshold effect for office-based interventions, other types of immunization programs (e.g., school based), may be necessary to reach national vaccination goals. From a policy perspective, contemporaneous early delivery of commercial and VFC influenza vaccines and/or the ability to use supplies on hand, enables practices to serve all children equally, and not require some of them to return to the practice to be vaccinated at a later date.

Strengths and Limitations

To date, this study is the only published randomized cluster trial to examine both patient- and practice level characteristics, including an evidence-based intervention, on childhood influenza vaccination rates. Previous studies have not used the randomized cluster trial and few have focused the intervention on the entire span of childhood. This study was limited by the facts that the rural sites randomly assigned to each arm were two offices of the same practice and that the community educational outreach and/or the knowledge that they were in a study may have led to increases in rates in the Control arm practice, thereby reducing the observed differences between arms. Further, vaccination rates may have been underestimated because vaccines given outside the practice may or may not have been captured from other sources.

Conclusions

A multi-strategy toolkit and provision of early vaccine can significantly improve vaccination rates over secular trends, except in practices with high pre-intervention coverage. Improving access to influenza vaccine by early delivery of vaccine supplies, so that opportunities to vaccinate all children are available early in the season, allows practices to vaccinate more children by extending the timeline of vaccine availability. This toolkit of evidence-based strategies can be implemented by an immunization champion in a variety of primary care practices.

Figure 1. Randomization Scheme.

Abbreviations

95% CI

Confidence interval (95%)

Task Force

Community Preventive Services Task Force

EMR

Electronic medical record

OR

Odds ratio

SOPs

Standing order programs

VFC

Vaccines for Children

VIF

Variance inflation factor

Appendix

Two-level generalized linear mixed modeling

Patient-level Model

$$\text{Logistic}({\text{vaccination status}}_{\mathrm{i}\mathrm{j}}(\text{yes vs}.\text{no}))={\mathrm{b}}_{0\mathrm{j}}+{\mathrm{b}}_{1\mathrm{j}}({\text{age group}}_{\mathrm{i}\mathrm{j}})+{\mathrm{b}}_{2\mathrm{j}}({\text{race}}_{\mathrm{i}\mathrm{j}})+{\mathrm{b}}_{3\mathrm{j}}({\text{insurance}}_{\mathrm{i}\mathrm{j}})+{\mathrm{\varepsilon }}_{\mathrm{i}\mathrm{j}}$$

Practice-Level Model

$$\begin{array}{c}{\mathrm{b}}_{0\mathrm{j}}={\mathrm{\beta }}_{00}+{\mathrm{\beta }}_{01}({\text{pre}-\text{intervention rate}}_{\mathrm{j}})+{\mathrm{\beta }}_{02}({\text{intervention}}_{\mathrm{j}}(\text{yes vs}.\text{no}))+{\mathrm{u}}_{0\mathrm{j}}\\ {\mathrm{b}}_{1\mathrm{j}}={\mathrm{\beta }}_{10}\hfill \\ {\mathrm{b}}_{2\mathrm{j}}={\mathrm{\beta }}_{20}\hfill \\ {\mathrm{b}}_{3\mathrm{j}}={\mathrm{\beta }}_{30}\hfill \\ {\mathrm{b}}_{4\mathrm{j}}={\mathrm{\beta }}_{40}\hfill \end{array}$$

Mixed Model

$$\text{Logistic}({\text{vaccination status}}_{\mathrm{i}\mathrm{j}})={\mathrm{\beta }}_{00}+{\mathrm{\beta }}_{01}({\text{pre}-\text{intervention rate}}_{\mathrm{j}})+{\mathrm{\beta }}_{02}({\text{intervention}}_{\mathrm{j}}(\text{yes vs}.\text{no}))+{\mathrm{\beta }}_{10}({\text{age group}}_{\mathrm{i}\mathrm{j}})+{\mathrm{\beta }}_{20}({\text{race}}_{\mathrm{i}\mathrm{j}})+{\mathrm{\beta }}_{30}({\text{insurance}}_{\mathrm{i}\mathrm{j}})+{\mathrm{u}}_{0\mathrm{j}}+{\mathrm{\varepsilon }}_{\mathrm{i}\mathrm{j}}$$