Abstract
BACKGROUND
Quality improvement in primary care has focused on improving adult immunization.
OBJECTIVES
Test the effectiveness of a step-by step, evidence-based guide, the 4 Pillars™ Immunization Toolkit, to increase adult pneumococcal vaccination.
DESIGN
Randomized controlled cluster trial (RCCT) in Year 1 (6/1/2013–5/31/2014) and a pre-post study in Year 2 (6/1/2014–1/31/2015) with data analyzed in 2016. Baseline year was 6/1/2012–5/31/2013. Demographic and vaccination data were derived from de-identified EMR extractions.
SETTING
25 primary care practices stratified by city (Houston, Pittsburgh), location (rural, urban, suburban) and type (family medicine, internal medicine), randomized to receive the intervention in Year 1 (n=13) or Year 2 (n=12).
PARTICIPANTS
A cohort of 18,107 patients ≥65 years at baseline with a mean age of 74.2 years; 60.7% were women, 16.5% were non-white and 15.7% were Hispanic.
INTERVENTION
The Toolkit, provider education, and one-on-one coaching of practice-based immunization champions. Outcome measures were 23-valent pneumococcal polysaccharide vaccine (PPSV) and pneumococcal conjugate vaccine (PCV) rates and percentage point (PP) changes.
RESULTS
In the RCCT, all intervention and control groups had significantly higher PPSV vaccination rates with average increases ranging from 6.5–8.7 PP (P<0.01). The intervention was not related to higher likelihood of PPSV vaccination. In the Year 2 pre-post study, the likelihood of PPSV and PCV vaccination was significantly higher in the active intervention sites than the maintenance sites in Pittsburgh, but not in Houston.
CONCLUSION
In a randomized controlled cluster trial, both intervention and control groups increased PPSV among adults ≥65 years. In a pre-post study, private primary care practices using the 4 Pillars™ Immunization Toolkit significantly improved PPSV and PCV uptake compared with practices that were in the maintenance phase of the study.
Keywords: Immunization, adults, pneumococcal polysaccharide vaccine, pneumococcal conjugate vaccine, Tdap vaccine
INTRODUCTION
Adult immunizations have been garnering increased attention as an important area for quality improvement for several reasons. Firstly, Healthy People 2020 goals for vaccines given to adults have not been met. In 2013, United States’ (U.S.) pneumococcal vaccination uptake was 21.2% among those 19–64 years with high risk conditions,1 compared with a goal of 60%,2 and 59.7% among those ≥65 years,1 compared with a goal of 90%.2 Secondly, the National Quality Strategy (NQS) established under the Affordable Care Act has set a long term goal to “promote healthy living and well-being through receipt of effective clinical preventive services across the lifespan in clinic and community settings.”3 The Center for Medicare and Medicaid Services has also made reporting of influenza and pneumococcal vaccines one requirement for providers to avoid negative payment adjustments.4 Clearly, adult immunizations are viewed as an integral part of quality care.
Provider barriers to adult vaccination include economic barriers arising from Medicare coverage that varies among vaccines, inconvenient vaccine storage in some medical offices, urgent and chronic medical conditions competing with time for prevention efforts, uncertainty about the patient’s vaccination status, missed immunization opportunities, and lack of patient and provider reminders.5–10 Patient-related barriers include not knowing that a vaccine was recommended, not believing that the clinician recommended the vaccine, and fear of vaccine side effects.11–14 Facilitators of vaccination include standing order programs (SOPs), prompts from the electronic medical record, use of teamwork, and longer time with the physician.15
Based on a comprehensive literature review, The Community Preventive Services Task Force found that improving vaccination uptake requires behavior changes at the system, provider, support staff, and patient levels.16 Sustainable change requires a coordinated, multipronged, adaptable approach; hence, the need for a practice improvement toolkit that can support change among diverse practice cultures is evident. The 4 Pillars™ Immunization Toolkit (Toolkit) is the product of years of health services research on the barriers to and facilitators of adult immunizations from the provider and patient perspectives,17,18 national and local surveys,5,6,19 and pilot studies on a toolkit to increase use of standing orders for adult vaccines.20
We undertook a 2-year study of 25 primary care practices to test the effectiveness of the Toolkit for increasing uptake of pneumococcal polysaccharide vaccine (PPSV) and in year 2, the newly recommended 13-valent pneumococcal conjugate vaccine (PCV) among adults ages 65 and older. The purpose of this study was to report the effect of the intervention on pneumococcal vaccination rates, percentage point differences and likelihood of pneumococcal vaccination. The data are presented in two ways. The randomized controlled cluster trial (RCCT) analysis compared changes in vaccine uptake in the intervention and control groups at the end of Year 1. In Year 2, controls were crossed over into active intervention and the Year 1 intervention groups became maintenance groups. Year 2 data were analyzed using a pre-post design in which changes in vaccine uptake were measured from the end of Year 1 to the end of the Year 2 intervention.
METHODS
The baseline year was 6/1/2012–5/31/2013. The RCCT took place during Year 1 (6/1/2013–5/31-2014) and the pre-post study took place during Year 2 (6/1/2014–1/31/2015). The new CDC recommendations for PCV were published and payment discussions with the Centers for Medicare and Medicaid Services occurred in the fall of 2014. This trial was approved by the Institutional Review Boards of the University of Pittsburgh, Baylor College of Medicine and the Harris Health System.
Sample Size and Sites
Optimal Design software (University of Michigan, Version 1.77. 2006) was used to calculate sample size for a RCCT seeking a 10–15% absolute increase in vaccination rate, and a minimum practice size of 100 patients. A sample size of 20 clusters or sites (10 Intervention and 10 Control practices) was determined to be necessary to achieve 80% power with an alpha of 0.05. Eligible primary care family medicine (FM) and internal medicine (IM) practices from a practice-based research network (PBRN) in Pittsburgh (FM Pittnet), a clinical network in Southwestern Pennsylvania (Community Medicine, Inc.) and a PBRN of safety net practices in Houston (SPUR-Net) were solicited for participation. When 25 sites agreed to participate, solicitation ceased. All sites used a common electronic medical record (EMR), EpicCare within their respective health systems.
Cluster Randomization
Cluster randomization allocates clinical practices rather than individuals to the intervention arms.21 Some practices had more than one site; thus each site was considered as a cluster. Eligibility requirements included having at least 100 patients ≥18 years of age, preliminary baseline vaccination rates for at least one adult vaccine (influenza, pneumococcal, Tdap) <50% and a willingness to make office changes to increase vaccination rates. Participating practices were stratified first by city (Pittsburgh or Houston), location (urban, suburban or rural), and discipline (internal or family medicine). The practices were then randomized into the Year 1 intervention or Year 2 intervention within strata (Figure 1). Year 2 intervention sites (controls) were informed that their intervention would take place during Year 2 of the study and were not contacted again until the next year.
Figure 1.
Randomization Scheme
Toolkit
The 4 Pillars™ Immunization Toolkit is founded on four evidence-based16,22 key strategies: Pillar 1 - Convenient vaccination services; Pillar 2 – Communication with patients about the importance of immunization and the availability of vaccines; Pillar 3 - Enhanced office systems to facilitate immunization; Pillar 4 - Motivation through an office immunization champion (IC). Supplemental Table 1 describes some of the strategies contained in the Toolkit. The Toolkit includes background on the importance of protecting patients against vaccine-preventable diseases, barriers to increasing vaccination from both provider and patient perspectives and strategies to eliminate those barriers. Practices were expected to implement strategies from each of the 4 pillars.
In Year 1, the Toolkit was a printed and bound document, supplemented by a web-based practice transformation dashboard. The dashboard was developed from the work of Fixsen et al.23 who established an empirically-based implementation framework that includes systematic uptake, establishment, and maintenance of research findings into routine practice. The core components include: staff selection and training on the specific evidence-based practices, expert consultation and coaching of staff and administration, program evaluation to assess and provide feedback, facilitative administrative supports to ensure data are used to focus and inform decision making, and systems interventions.
Once the practice was registered, any staff member could log into the dashboard. The IC was responsible for registering the practice and its staff members, and identifying strategies that the practice would implement. The Toolkit provided step-by-step guidance for implementing the strategies, and the dashboard showed the practices’ progress through the change process. Practices could monitor their progress on graphs that reported biweekly numbers of vaccines given. In Year 2, the Toolkit was fully digitized to a website (4pillarstoolkit.pitt.edu) that incorporated all of the paper-based background information, as well as the dashboard capabilities.
Interventions
The intervention was designed using Diffusion of Innovations theory,24 and included the Toolkit, provider education, and one-on-one coaching of the immunization champion for each practice. One of two investigators (AEB, MPN) visited each intervention site to introduce the study and the Toolkit, and to work with staff to develop practice-specific ideas for implementing Toolkit strategies. Each practice identified an IC who was responsible for updating the practice transformation dashboard (Year 1) as intervention strategies were employed, and in Year 2 use the web-based Toolkit to guide strategy implementation and record progress in the dashboard, assisted by the research liaison as needed. Other tasks for the IC included participating in the biweekly telephone-call with a research liaison for coaching, ensuring that chosen strategies were being implemented and working to maintain motivation of the staff.
The Toolkit was updated with new pneumococcal vaccination recommendations during Year 2 at the end of September 2014, within a few days of their publication and each practice’s leadership team was directly notified by email of the changes to the pneumococcal recommendations. In November 2014, additional information was sent to the Pittsburgh practices about local insurance coverage for PCV vaccines. Information on CMS coverage for PCV was sent to all practices in early January 2015.
Data collection
De-identified demographic (date of birth, sex, race or ethnicity, health insurance coverage), office visit (dates) and vaccination data (vaccines given and dates) were derived from EMR data extractions performed by the UPMC Center for Assistance in Research using the eRecord and from a similar data extraction from the EMR by staff of the SPUR-NET for the Texas sites. Data extractions took place at the end of each year of the study. A longitudinal data base was created with only those patients who were ≥65 years at baseline and who had a visit each year during the three-year study period, creating a cohort of individuals who would have been patients of the practice during their practice’s respective baseline (6/1/2012–5/31-2013), control or intervention (Year 1; 6/30/2013–5/31/2014), and intervention or maintenance years (Year 2; 6/1/2014–1/31/2015).
Statistical analyses
Descriptive analyses were performed for patient demographic characteristics (age, sex, race, and ethnicity). Health insurance was not included in the regression analyses because all insurances would have covered pneumococcal vaccine in those aged ≥65 years. Because of significant differences in patient populations, size and structure of the practices in Houston and Pittsburgh, these sites were grouped separately for analysis. Age was used as a continuous variable. Race and ethnicity were recorded differently in each city. In Pittsburgh, with few Hispanic patients, ethnicity was rarely recorded; hence patients were grouped by race into white and non-white with blacks and Hispanics assigned to the non-white group and only race data are presented and used in analysis. In Houston, with few non-Hispanic patients, race was rarely recorded; hence only ethnicity (Hispanic and non-Hispanic) are presented and used in analysis.
The two pneumococcal vaccines (PPSV and PCV) would typically be administered once during the 3-year project period (PCV was a new recommendation late in Year 2). The analytical periods were baseline: 6/1/2012–5/31/2013; Year 1: 6/1/2013–5/31/2014; and Year 2: 6/1/2014–1/31/2015. Proportions were reported for categorical variables and means and standard deviations were reported for continuous variables. The primary outcome measures were the cumulative PPSV and PCV vaccination rates reported at baseline, Year 1 and Year 2. Chi-square tests were performed to test for differences in cumulative vaccination rates at different time points.
Year 1 RCCT analyses
To determine which factors were related to PPSV vaccination while accounting for the clustered nature of the data, Cox proportional hazard models with the robust sandwich estimate were fitted, taking account of heterogeneity in demographic characteristics (including age, sex, and race/ethnicity). Because of the lack of variability in the number of strategies used by practices and the correlation of strategies with intervention group, only intervention group was included in the models.
Year 2 Pre-post analyses
At the end of Year 1, practices were offered the opportunity to continue active intervention during Year 2. Four practices opted to do so. At the same time, the Year 1 control sites began the intervention. For the pre-post analyses, the four practices in Pittsburgh that continued the intervention in Year 2 were combined with the Year 1 control sites and were referred to as the active intervention group. The six practices that did not actively participate in Year 2 were referred to as the maintenance group. In Houston, the Year 1 intervention sites were referred to as the maintenance group and the Year 1 control sites that received the intervention in Year 2 were referred to as the active intervention group. Cox proportional hazard models with the robust sandwich estimate were again fitted, taking account of heterogeneity in demographic characteristics (including age, sex, and race/ethnicity) to determine which factors were related to PPSV and PCV uptake. Statistical significance of two-sided tests was set at a type I error (alpha) equal to 0.05. All analytical procedures were performed using SAS® 9.3.
RESULTS
Twenty-four sites completed the intervention; the demographic and other characteristics for patients ≥65 years are shown in Table 1. The large differences between practices in Houston and Pittsburgh supported the separate analyses of sites by city. Houston sites were larger practices with higher reported proportions of Hispanic patients, higher proportions of female patients, and non-commercially insured patients than the Pittsburgh sites.
Table 1.
Demographic and practice characteristics for patients ≥65 years of age at baseline
| Site | N | Age, mean (SD) | Female, % | White, % | Non-white, % | Hispanic, % | Health insurance status
|
||
|---|---|---|---|---|---|---|---|---|---|
| Medicaid,* % | Commercial, % | Medicare, % | |||||||
| Pittsburgh | |||||||||
| Intervention sites | |||||||||
| B | 266 | 77.1 (8.4) | 72.9 | 64.3 | 35.7 | 0.8 | 1.1 | 33.5 | 65.4 |
| C | 898 | 76.5 (8.3) | 61.4 | 99.8 | 0.2 | 0.0 | 1.0 | 44.4 | 54.8 |
| D | 1933 | 76.5 (7.9) | 55.6 | 99.6 | 0.4 | 0.1 | 0.5 | 46.7 | 52.8 |
| E | 455 | 73.9 (7.3) | 57.6 | 94.5 | 5.5 | 0.2 | 1.3 | 48.1 | 50.6 |
| G | 265 | 75.8 (7.5) | 54.7 | 83.8 | 16.2 | 0.0 | 1.5 | 37.0 | 61.5 |
| H | 175 | 76.8 (8.3) | 61.1 | 66.9 | 33.1 | 0.0 | 6.3 | 32.6 | 61.1 |
| F** | 1353 | 74.9 (7.6) | 55.1 | 97.7 | 2.3 | 0.3 | 1.9 | 42.8 | 55.3 |
| J** | 297 | 77.5 (8.5) | 53.2 | 90.2 | 9.8 | 0.3 | 2.7 | 52.5 | 44.8 |
| K** | 113 | 75.4 (8.0) | 71.7 | 99.1 | 0.9 | 0.0 | 0 | 55.8 | 44.2 |
| M** | 340 | 76.7 (7.9) | 55.3 | 98.5 | 1.5 | 0.3 | 2.1 | 45.6 | 52.4 |
|
| |||||||||
| Total | 6095 | 75.9 (7.9) | 57.5 | 95.1 | 5.5 | 0.2 | 1.4 | 44.6 | 54.0 |
|
| |||||||||
| Control sites | |||||||||
| N | 960 | 76.2 (7.6) | 59.6 | 99.0 | 1.0 | 0.1 | 1.2 | 60.4 | 38.4 |
| O | 1445 | 74.3 (7.3) | 54.3 | 99.2 | 0.8 | 0.1 | 2.1 | 38.8 | 59.1 |
| R | 907 | 73.3 (6.8) | 52.8 | 98.8 | 1.2 | 0.0 | 2.0 | 39.2 | 58.8 |
| S | 128 | 73.7 (7.5) | 64.8 | 68.8 | 31.2 | 0.0 | 3.1 | 32.0 | 64.8 |
| U | 966 | 74.8 (7.1) | 61.1 | 95.5 | 4.5 | 0.4 | 2.4 | 30.9 | 66.7 |
| W | 202 | 80.6 (8.7) | 73.8 | 91.6 | 8.4 | 1.0 | 2.5 | 42.1 | 55.4 |
| X | 227 | 73.0 (6.9) | 42.3 | 96.9 | 3.1 | 0.0 | 1.3 | 37.9 | 60.8 |
| Y | 1387 | 74.7 (7.4) | 59.3 | 97.7 | 2.3 | 0.1 | 1.6 | 39.7 | 58.8 |
|
| |||||||||
| Total | 6223 | 74.8 (7.4) | 57.5 | 94.2 | 5.8 | 0.1 | 1.9 | 41.1 | 57.0 |
|
| |||||||||
| Houston | |||||||||
| Intervention sites | |||||||||
| A | 769 | 72.7 (7.1) | 66.6 | 6.5 | 93.5 | 67.2 | 44.5 | 1.6 | 54.0 |
| I | 1544 | 72.2 (6.8) | 69.8 | 3.2 | 96.8 | 16.3 | 40.7 | .6 | 58.7 |
| L | 697 | 70.8 (6.3) | 67.7 | 17.4 | 27.4 | 72.6 | 74.8 | 0.3 | 25.0 |
|
| |||||||||
| Total | 3010 | 72.1 (6.8) | 68.5 | 7.3 | 92.7 | 42.4 | 49.6 | 0.8 | 49.7 |
|
| |||||||||
| Control sites | |||||||||
| P | 926 | 71.7 (6.7) | 66.2 | 5.7 | 23.6 | 76.4 | 64.3 | 1.2 | 34.6 |
| T | 668 | 71.3 (6.4) | 66.8 | 12.0 | 49.1 | 50.9 | 69.6 | 0.2 | 30.2 |
| V | 1185 | 71.1 (6.1) | 65.9 | 6.1 | 57.0 | 43.0 | 78.8 | 0.6 | 20.6 |
|
| |||||||||
| Total | 2779 | 71.3 (6.4) | 66.2 | 7.4 | 92.6 | 56.0 | 71.8 | 0.7 | 27.6 |
| All groups | 18107 | 74.2 (7.6) | 60.7 | 67.8 | 16.5 | 15.7 | 20.4 | 29.4 | 50.3 |
Also includes Other/self-pay/indigent/charity care
These sites participated in the intervention for 2 years
Year 1 RCCT study
Cumulative PPSV vaccination rates for all patients ≥65 years at each site and by intervention group at Baseline and Year 1 are shown in Table 2. Individual practice baseline rates for PPSV ranged from a high of 80.9% to a low of 31.0% with average baseline rates of 71.4% and 67.7% for the intervention groups in Pittsburgh and Houston, respectively.
Table 2.
Cumulative pneumococcal polysaccharide vaccine (PPSV) vaccination rates at baseline and the end of the RCCT Year 1 intervention, among patients age ≥ 65 years.
| Year 1 – Randomized Controlled Cluster Trial
| ||||||||
|---|---|---|---|---|---|---|---|---|
| % Vaccinated with PPSV
|
% Vaccinated with PPSV
|
|||||||
| Intervention Site | Total N | End of Baseline 5/31/2013 | End of Year 1 5/31/2014 | Control Site | Total N | End of Baseline 5/31/2013 | End of Year 1 5/31/2014 | |
| Pittsburgh | ||||||||
|
| ||||||||
| B | 266 | 60.2 | 63.2 | N | 960 | 80.1 | 84.0 | |
| C | 898 | 79.4 | 84.7 | O | 1445 | 51.1 | 60.1 | |
| D | 1933 | 73.4 | 80.4 | R | 907 | 62.8 | 68.4 | |
| E | 455 | 69.0 | 78.2 | S | 128 | 68.8 | 75.0 | |
| F | 1353 | 78.9 | 85.2 | U | 967 | 79.7 | 84.1 | |
| G | 265 | 56.6 | 64.2 | W | 202 | 74.7 | 81.7 | |
| H | 175 | 57.1 | 60.6 | X | 227 | 79.7 | 87.7 | |
| J | 297 | 39.4 | 45.1 | Y | 1387 | 69.4 | 78.1 | |
| K | 113 | 31.0 | 40.7 | – | ||||
| M | 340 | 80.9 | 88.0 | – | ||||
|
| ||||||||
| Intervention total | 6095 | 71.4 | 77.9* | Control total | 6223 | 71.5 | 78.1* | |
|
| ||||||||
| Total percentage point difference from baseline to Year 1 | 6.5** | 6.6** | ||||||
|
| ||||||||
| Houston | ||||||||
|
| ||||||||
| A | 769 | 75.8 | 83.6 | P | 926 | 79.5 | 83.9 | |
| I | 1544 | 58.9 | 69.6 | T | 668 | 71.9 | 79.6 | |
| L | 697 | 78.5 | 83.5 | V | 1185 | 73.5 | 80.9 | |
|
| ||||||||
| Intervention total | 3010 | 67.7 | 76.4* | Control total | 2779 | 75.1 | 81.6* | |
| Total percentage point difference from baseline to Year 1 | 8.7† | 6.5† | ||||||
|
| ||||||||
| All groups | 18718 | 71.4 | 78.3* | |||||
| Total percentage point difference from baseline to Year 1 | 6.9 | |||||||
P<0.001 for difference between Baseline and Year 1
P=0.84 for difference between Intervention and Control groups in Pittsburgh
P<0.001 for difference between Intervention and Control groups in Houston
At the end of the intervention year, PPSV rates increased significantly in both intervention and control groups in both cities. Increases ranged from 6.5 to 8.7 PP (P<0.001 by Chi square test for baseline to Year 1 rates). The differences in PP changes between intervention and control groups were significant for Houston sites (P<0.001), but not for Pittsburgh sites (P=0.84).
Regression analyses that accounted for the clustered nature of the data as well as race/ethnicity, age, and sex, were conducted to determine the effect of the intervention on vaccination rates in the RCCT (Table 3). The likelihood of PPSV vaccination in Year 1 was not related to the intervention but was significantly related to younger age (P<0.001) in the Pittsburgh sites. Similarly, in Houston sites, younger age and non-Hispanic ethnicity, but not the intervention, were related to a higher likelihood of PPSV vaccination.
Table 3.
Cox proportional hazard models to detect intervention effect on time to vaccination with pneumococcal polysaccharide vaccine for patients ages ≥ 65 years, adjusting for covariates
| Year 1 - Randomized Controlled Cluster Trial Analysis 6/1/2013 – 5/31/2014 | ||||
|---|---|---|---|---|
|
|
|
|||
| Hazard Ratio (95% CI) | P Value | Hazard Ratio (95% CI) | P Value | |
| Pittsburgh sites | Houston sites | |||
| Intervention, ref. = Control | 1.03 (0.78–1.37) | 0.830 | 1.33 (0.94–1.88) | 0.110 |
| Female, ref. = Male | 0.99 (0.87–1.13) | 0.900 | 1.05 (0.90–1.21) | 0.560 |
| Age, years | 0.92 (0.90–0.95) | <0.001 | 0.95 (0.93–0.97) | <0.001 |
| White race, ref. = Non-white | 0.88 (0.61–1.26) | 0.520 | – | – |
| Non-Hispanic ethnicity, ref. = Hispanic | – | – | 1.23 (1.03–1.48) | 0.030 |
Year 2 Pre-Post Study
At the end of the pre-post study comparing the Year 2 active intervention sites and the maintenance sites, individual site PPSV rates ranged from a low of 43.4% to a high of 94.7% (data not shown). Percentage point differences from Year 1 to Year 2 in the maintenance and active intervention sites are shown in Table 4. Active intervention and maintenance groups in both cities significantly improved PPSV vaccination rates from Year 1 to Year 2 (P<0.001). By the end of Year 2, 79% of practices (19/24) had PPSV rates at or above 70% and 58% of practices (14/24) had PPSV rates at or above 80% (data not shown).
Table 4.
Percentage point (PP) changes from Year 1 to Year 2 for cumulative pneumococcal polysaccharide vaccine (PPSV) and pneumococcal conjugate vaccine (PCV) vaccination rates, among patients age ≥ 65 years.
| Year 2 – Pre-Post Study
| |||||||
|---|---|---|---|---|---|---|---|
| Difference in % Vaccinated with PPSV from Year 1 to Year 2 | Difference in % Vaccinated with PCV from Year 1 to Year 2 | ||||||
|
| |||||||
| Maintenance Sites | PP* | Active Intervention Sites | PP* | Maintenance Sites | PP* | Active Intervention Sites | PP* |
| Pittsburgh | |||||||
|
| |||||||
| B | 0.7 | N | 1.7 | B | 0.7 | N | 3.5 |
| C | 2.0 | O | 6.9 | C | 0.4 | O | 2.7 |
| D | 2.6 | R | 6.1 | D | 0.6 | R | 3.1 |
| E | 1.4 | S | 1.6 | E | 2.1 | S | 7.1 |
| G | 6.0 | U | 2.2 | G | 1.1 | U | 29.7 |
| H | 3.4 | W | 1.5 | H | 1.1 | W | 1.0 |
| X | 7.0 | X | 10.1 | ||||
| Y | 7.2 | Y | 5.9 | ||||
| F | 5.2 | F | 21.9 | ||||
| J | 8.1 | J | 15.2 | ||||
| K | 2.7 | K | 0.8 | ||||
| M | 0.8 | M | 33.2 | ||||
|
| |||||||
| Total Maintenance | 2.4** | Total Active Intervention | 5.2** | Total Maintenance | 0.8** | Total Active Intervention | 11.5** |
|
| |||||||
| Houston | |||||||
|
| |||||||
| Maintenance sites | PP* | Active intervention sites | PP* | Maintenance sites | PP* | Active intervention sites | PP* |
|
| |||||||
| A | 7.2 | P | 3.9 | A | 0.1 | P | 0 |
| I | 7.5 | T | 4.5 | I | 3.6 | T | 11.0 |
| L | 3.6 | V | 5.3 | L | 0.3 | V | 0.7 |
|
| |||||||
| Total Maintenance | 6.5† | Total Active Intervention | 4.6† | Total Maintenance | 1.9† | Total Active Intervention | 2.9† |
Note: Year 1=6/1/2013–5/31/2014 and Year 2=6/1/2014–1/31/2015.
PP= Percentage point difference between Year 1 and Year 2.
P<0.001 for PP difference between Maintenance and Active Intervention groups.
P<0.01 for PP difference between Maintenance and Active Intervention groups.
PP differences in PCV vaccination rates for individual practices at the end of Year 2 are also shown in Table 4. Overall, the increase in PCV rates among active intervention sites was significantly greater than among the maintenance sites (P<0.001 for Pittsburgh sites and P<0.01 for Houston sites). Figure 2 shows PPSV and PCV vaccination rates for the active intervention and maintenance groups from baseline to the end of follow-up.
Figure 2. a and b. Cumulative Pneumococcal Polysaccharide Vaccine (PPSV) Vaccination and Pneumococcal Conjugate Vaccine (PCV) Rates for Adults ≥65 Years at the End of Baseline and Year 2 by Intervention Group and City.
*P<0.001 for percentage point (PP) difference from Baseline to Year 2 between Active Intervention and Maintenance groups by Chi-square test
The likelihood of receipt of PPSV at the end of the pre-post study in Pittsburgh was significantly higher for the active intervention group, for males and for younger patients than for the maintenance group, females and older patients (P<0.05; Table 5). In the Houston sites there was no effect of the intervention on likelihood of PPSV receipt, but males and non-Hispanic patients were more likely to receive PPSV than female and Hispanic patients (P<0.001).
Table 5.
Cox proportional hazard models to detect intervention effect on time to vaccination for pneumococcal polysaccharide vaccine (PPSV) and for pneumococcal conjugate vaccine (PCV) for patients age ≥ 65 years, adjusting for covariates
| Year 2 - Pre-Post Study 6/1/2014 – 1/31/2015
| ||||
|---|---|---|---|---|
| Hazard Ratio (95% CI) | P value | Hazard Ratio (95% CI) | P value | |
| Pneumococcal Polysaccharide Vaccine | Pittsburgh | Houston | ||
| Active intervention, ref. = Maintenance | 1.78 (1.20–2.63) | 0.004 | 0.79 (0.56–1.09) | 0.150 |
| Female, ref. = Male | 0.80 (0.69–0.92) | 0.002 | 0.84 (0.73–0.96) | <0.01 |
| Age, years | 0.94 (0.93–0.96) | <0.001 | 0.98 (0.96–1.01) | 0.150 |
| White race, ref. = Non-white | 1.36 (0.82–2.24) | 0.240 | – | – |
| Non-Hispanic ethnicity, ref. = Hispanic | – | – | 0.64 (0.52–0.78) | <0.001 |
| Pneumococcal Conjugate Vaccine | Pittsburgh | Houston | ||
| Active intervention, ref. = Maintenance | 14.69 (6.46–33.39) | <0.001 | 1.67 (0.27–10.47) | 0.580 |
| Female, ref. = Male | 0.89 (0.79–1.02) | 0.090 | 0.90 (0.65–1.24) | 0.530 |
| Age, years | 1.01 (1.00–1.02) | 0.030 | 1.03 (1.01–1.05) | <0.001 |
| White race, ref. = Non-white | 0.95 (0.54–1.64) | 0.840 | – | – |
| Non-Hispanic ethnicity, ref. = Hispanic | – | – | 2.00 (1.10–3.61) | 0.020 |
The effect of the intervention on the recently introduced PCV was marked; those in the active intervention group were 14.7 times more likely to receive PCV than the maintenance group. Age was related to likelihood of PCV vaccination but in this case it was the older patients who were more likely to receive the vaccine. In the Houston sites, there was no effect of the intervention on the likelihood of PCV receipt, although non-Hispanic patients were twice as likely to receive PCV as Hispanic patients (P=0.02) and older patients were slightly more likely to receive PCV (P<0.001).
DISCUSSION
National coverage for PPSV among adults ≥65 years of age was 59.7% in 2013, a level similar to recent years.1,25 In this study at baseline, many but not all, sites reported rates above that level, likely reflecting a segment of the population with access to primary care. Over the course of the RCCT, PPSV uptake increased significantly in both intervention and control groups and increased measurably in the majority of individual practices. During Year 2, the active intervention group increased average PPSV rates by 4.6 to 5.2 percentage points while the maintenance groups continued to increase their average PPSV rates (2.4–6.5 percentage points). These increases are considerably higher than recent secular trends. In previous research, we have reported that pediatric practices seem to reach a plateau in influenza vaccination rates at about 58%.26 This study shows that even practices with relatively high baseline vaccination rates can increase the proportion of their patients who receive PPSV, with a directed effort. Such increases will help practices achieve quality improvement goals as have been set forth by organizations such as the Centers for Medicare and Medicaid Services4 and the National Quality Strategy.3
In both the RCCT and pre-post study, younger age was associated with higher likelihood of PPSV vaccination suggesting that practices targeted patients for PPSV vaccination as soon as they became 65 years old and this may be a successful strategy. There is no obvious explanation for why women were less likely to be vaccinated in the pre-post study, nor why non-Hispanic patients were more likely to receive PPSV in the RCCT but not in the pre-post study. However, these findings suggest that special efforts to market adult vaccines to population subgroups may be required in this setting.
Adoption and implementation, which are components of the RE-AIM evaluation framework,27,24 varied by setting according to reports from the research assistants, likely reflecting a variety of factors. Anecdotally, usability of the dashboard was a reported problem in Year 1, which was improved for Year 2. One practice served a religious sect that is hesitant about vaccination, another practice which serves a large immigrant community closed one of its two offices during Year 2, and another site was late in adoption of the intervention in Year 2.
In December 2011, FDA licensed PCV for use among adults ages 50 and older28 and the following June, ACIP recommended its use for adults ≥19 years of age with certain high risk and immunocompromising conditions.29 In September 2014, CDC recommended PCV for all adults ≥65 years of age to be given in series with PPSV.30 Hence, PCV was available and recommended for use among high risk patients, but was not age-based for adults at the time that the study began in 2013, and there are currently no national comparison data for PCV uptake. We report significantly higher likelihood of PCV vaccination in the active intervention groups compared with the maintenance groups, indicating a significant effect of the intervention on vaccination rates.
In late 2014, there was some confusion reported by the participating practices as to whether health insurance plans would cover administration of both vaccines and this may have delayed their uptake. This experience speaks to the need for easy-to-use resources such as the 4 Pillars™ Immunization Toolkit to help physicians and clinical staffs understand the implications of changing vaccine recommendations and reimbursement policies. Practices have been shown to improve implementation of interventions when they are well supported through a facilitator such as an IC,31,32 and specifically, to increase pneumococcal vaccination using clinician education and financial incentives.33
Strengths and Limitations
The strengths of this study are its randomized design, the large number and diversity of patients, diverse practice settings including safety net clinics, randomized design, and two intervention years of vaccination reporting. Limitations are that during the Year 1 intervention, delivery of the EMR data was delayed, preventing the research team from providing feedback to the sites in both cities about their progress. The CDC’s change in recommendations regarding PCV late in Year 2 with the related confusion about Medicare coverage interfered with implementation. Given the increase in immunization rates in the control arm in Year 1, either intervention transference among study arms occurred in sites within the same network, a Hawthorne effect, or secular trends occurred; nationally, PPSV rates have been relatively flat, arguing against national secular trends.1
Conclusions
In a randomized controlled cluster trial, both intervention and control groups increased PPSV among adults ≥65 years. In a pre-post study, small to medium sized private primary care practices using the 4 Pillars™ Immunization Toolkit significantly improved PPSV and PCV uptake compared with similar practices that were in the maintenance phase of the study. In large, safety net practices the intervention did not result in significant improvements in either PPSV or PCV uptake. The Toolkit may need enhancements to address the needs of this type of practice.
Supplementary Material
Supplemental Table 1. Intervention Strategies Used to Increase Adult Vaccination Rates from the 4 Pillars™ Immunization Toolkit.
Acknowledgments
Funding Source: This investigation was supported by a grant (U01 IP000662) from the Centers for Disease Control and Prevention. The views expressed herein are those of those authors and not those of the Centers for Disease Control and Prevention. The project described was also supported by the National Institutes of Health through Grant Numbers UL1 RR024153 and UL1TR000005.
Sponsor’s Role:
CDC project officers F. Ahmed and S. Patel both offered guidance on design and analysis and edited the manuscript. NIH had no role.
Conflict of Interest Disclosures
| Elements of Financial/Personal Conflicts | Author 1 Richard K. Zimmerman |
*Author 2 Mary Patricia Nowalk |
Author 3 Chyongchiou J Lin |
All other authors no conflicts |
||||
|---|---|---|---|---|---|---|---|---|
| Yes | No | Yes | No | Yes | No | Yes | No | |
| Employment or Affiliation | X | X | X | X | ||||
| Grants/Funds | X | X | X | X | ||||
| Honoraria | X | X | X | X | ||||
| Speaker Forum | X | X | X | X | ||||
| Consultant | X | X | X | X | ||||
| Stocks | X | X | X | X | ||||
| Royalties | X | X | X | X | ||||
| Expert Testimony | X | X | X | X | ||||
| Board Member | X | X | X | X | ||||
| Patents | X | X | X | X | ||||
| Personal Relationship | X | X | X | X | ||||
For “yes” x mark(s): give brief explanation below:
Drs. Zimmerman and Lin have a research grant from Sanofi Pasteur, Inc. Drs. Zimmerman, Nowalk and Lin have research grants from Merck & Co, Inc., and Pfizer, Inc. The other authors have no conflicts of interest to disclose.
Author Contributions
Richard K. Zimmerman developed the toolkit, designed the study, advised on data analysis and edited the manuscript.
Anthony E. Brown coordinated the study in the Houston sites and edited the manuscript.
Valory N. Pavlik advised on study design and analysis and edited the manuscript.
Krissy K. Moehling executed the study in the Pittsburgh sites and edited the manuscript.
Jonathan M. Raviotta created the Toolkit website and edited the manuscript.
Chyongchiou J. Lin analyzed the data and edited the manuscript.
Song Zhang analyzed the data and edited the manuscript.
Mary Hawk designed and executed the qualitative data collection, advised on the design of the dashboard, and edited the manuscript.
Shakala Kyle provided technical assistance and edited the manuscript.
Suchita Patel advised on the data analysis and edited the manuscript.
Faruque Ahmed advised on the study design, data analysis and edited the manuscript.
Mary Patricia Nowalk designed the Toolkit, executed the study, advised on data analysis and drafted the manuscript.
Literature Cited
- 1.Williams WW, Lu PJ, O’Halloran A, et al. Vaccination Coverage Among Adults, Excluding Influenza Vaccination - United States, 2013. Morb Mortal Wkly Report (MMWR) 2015 Feb 6;64(4):95–102. [PMC free article] [PubMed] [Google Scholar]
- 2.U.S. Department of Health and Human Services. Healthy People 2020: Immunization and Infectious Diseases. 2013 http://www.healthypeople.gov/2020/topicsobjectives2020/overview.aspx?topicid=23. Accessed January 18, 2012.
- 3.Agency for Healthcare Research and Quality. National Quality Strategy: Overview. 2015 [Slide Set] http://www.ahrq.gov/workingforquality/nqs/overview.htm. Accessed August 20, 2015.
- 4.Centers for Medicare & Medicaid Services. Physician Quality Reporting System. 2015 https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/PQRS/. Accessed February 22, 2016.
- 5.Zimmerman RK, Nowalk MP, Bardella IJ, et al. Physician and practice factors related to influenza vaccination among the elderly. Am J Prev Med. 2004 Jan;26(1):1–10. doi: 10.1016/j.amepre.2003.09.020. [DOI] [PubMed] [Google Scholar]
- 6.Zimmerman RK, Santibanez TA, Janosky JE, et al. What affects influenza vaccination rates among older patients? An analysis from inner-city, suburban, rural, and Veterans Affairs practices. Am J Med. 2003 Jan;114(1):31–38. doi: 10.1016/s0002-9343(02)01421-3. [DOI] [PubMed] [Google Scholar]
- 7.Zimmerman RK, Santibanez TA, Fine MJ, et al. Barriers and facilitators of pneumococcal vaccination among the elderly. Vaccine. 2003 Mar 28;21(13–14):1510–1517. doi: 10.1016/s0264-410x(02)00698-9. [DOI] [PubMed] [Google Scholar]
- 8.Santibanez TA, Zimmerman RK, Nowalk MP, Jewell IK, Bardella IJ. Physician attitudes and beliefs associated with patient pneumococcal polysaccharide vaccination status. Ann Fam Med. 2004 Jan-Feb;2(1):41–48. doi: 10.1370/afm.53. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Nowalk MP, Zimmerman RK, Cleary SM, Bruehlman RD. Missed opportunities to vaccinate older adults in primary care. J Am Board Fam Pract. 2005 Jan-Feb;18(1):20–27. doi: 10.3122/jabfm.18.1.20. [DOI] [PubMed] [Google Scholar]
- 10.Nowalk MP, Zimmerman RK, Feghali J. Missed opportunities for adult immunization in diverse primary care office settings. Vaccine. 2004 Sep 3;22(25–26):3457–3463. doi: 10.1016/j.vaccine.2004.02.022. [DOI] [PubMed] [Google Scholar]
- 11.Santibanez TA, Nowalk MP, Zimmerman RK, et al. Knowledge and beliefs about influenza, pneumococcal disease, and immunizations among the elderly. J Am Geriatr Soc. 2002;50:1711–1716. doi: 10.1046/j.1532-5415.2002.50466.x. [DOI] [PubMed] [Google Scholar]
- 12.Zimmerman RK, Nowalk MP, Raymund M, et al. Tailored interventions to increase influenza vaccination in neighborhood health centers serving the disadvantaged. Am J Public Health. 2003 Oct;93(10):1699–1705. doi: 10.2105/ajph.93.10.1699. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Zimmerman RK, Santibanez TA, Fine MJ, et al. Barriers and facilitators of pneumococcal vaccination among the elderly. Vaccine. 2003 Mar 28;21(13–14):1510–1517. doi: 10.1016/s0264-410x(02)00698-9. [DOI] [PubMed] [Google Scholar]
- 14.Zimmerman RK, Mieczkowski TA, Wilson SA. Immunization Rates and Beliefs among Elderly Patients of Inner-City Neighborhood Health Centers. Health Promotion Practice. 2002;3(2):197–206. [Google Scholar]
- 15.Belshe RB, Ambrose CS, Yi T. Safety and efficacy of live attenuated influenza vaccine in children 2–7 years of age. Vaccine. 2008 Sep 12;26(Suppl 4):D10–16. doi: 10.1016/j.vaccine.2008.06.083. [DOI] [PubMed] [Google Scholar]
- 16.Task Force on Community Preventive Services. Guide to Community Preventive Services. 2012 http://www.thecommunityguide.org/index.html. Accessed January 18, 2013.
- 17.Nowalk MP, Zimmerman RK, Shen S, Jewell IK, Raymund M. Barriers to pneumococcal and influenza vaccination in older community-dwelling adults (2000–2001) J Am Geriatr Soc. 2004 Jan;52(1):25–30. doi: 10.1111/j.1532-5415.2004.52006.x. [DOI] [PubMed] [Google Scholar]
- 18.Santibanez TA, Nowalk MP, Zimmerman RK, et al. Knowledge and beliefs about influenza, pneumococcal disease, and immunizations among older people. J Am Geriatr Soc. 2002 Oct;50(10):1711–1716. doi: 10.1046/j.1532-5415.2002.50466.x. [DOI] [PubMed] [Google Scholar]
- 19.Zimmerman RK, Albert SM, Nowalk MP, Yonas MA, Ahmed F. Use of standing orders for adult influenza vaccination a national survey of primary care physicians. Am J Prev Med. 2011 Feb;40(2):144–148. doi: 10.1016/j.amepre.2010.10.027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Ball SW, Walker DK, Donahue SMA, et al. Influenza Vaccination Coverage Among Health-Care Personnel-2011-12 Influenza Season, United States (Reprinted from MMWR vol, 38, pg 753, 2012) J Am Med Assoc. 2012 Nov 21;308(19):1964–1966. [Google Scholar]
- 21.Campbell MK, Piaggio G, Elbourne DR, DG A. Consort 2010 statement: extension to cluster randomised trials. BMJ (Clinical research ed) 2012;345:e5661. doi: 10.1136/bmj.e5661. [DOI] [PubMed] [Google Scholar]
- 22.Melinkovich P, Hammer A, Staudenmaier A, Berg M. Improving pediatric immunization rates in a safety-net delivery system. Jt Comm J Qual Patient Saf. 2007 Apr;33(4):205–210. doi: 10.1016/s1553-7250(07)33024-9. [DOI] [PubMed] [Google Scholar]
- 23.Fixsen D, Naoom S, Blase K, Friedman R, Wallace F. Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231); 2005. [Google Scholar]
- 24.Saltelli A, Tarantola S, Campolongo F, Ratto M. Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models. New York, NY: Willey; 2004. [Google Scholar]
- 25.Williams WW, Lu PJ, O’Halloran A, et al. Noninfluenza Vaccination Coverage Among Adults - United States, 2012. Morb Mortal Wkly Report (MMWR) 2014 Feb 7;63(5):95–102. [PMC free article] [PubMed] [Google Scholar]
- 26.Polisena J, Chen Y, Manuel D. The proportion of influenza vaccination in Ontario, Canada in 2007/2008 compared with other provinces. Vaccine. 2012 Mar 2;30(11):1981–1985. doi: 10.1016/j.vaccine.2012.01.009. [DOI] [PubMed] [Google Scholar]
- 27.Jilcott S, Ammerman A, Sommers J, Glasgow RE. Applying the RE-AIM framework to assess the public health impact of policy change. Ann Behav Med. 2007 Oct;34(2):105–114. doi: 10.1007/BF02872666. [DOI] [PubMed] [Google Scholar]
- 28.Food and Drug Administration. FDA expands use of Prevnar 13 vaccine for people ages 50 and older. 2011 http://www.fda.gov/newsevents/newsroom/pressannouncements/ucm285431.htm. Accessed February 22, 2016.
- 29.Centers for Disease Control and Prevention. Use of 13-Valent Pneumococcal Conjugate Vaccine and 23-Valent Pneumococcal Polysaccharide Vaccine for Adults With Immunocompromising Conditions: Recommendations of the Advisory Committee on Immunization Practices (ACIP) Morb Mortal Wkly Report (MMWR) 2012 Oct 12;61(40):816–819. [PubMed] [Google Scholar]
- 30.Tomczyk S, Bennett NM, Stoecker C, et al. Use of 13-Valent Pneumococcal Conjugate Vaccine and 23-Valent Pneumococcal Polysaccharide Vaccine Among Adults Aged >= 65 Years: Recommendations of the Advisory Committee on Immunization Practices (ACIP) Morb Mortal Wkly Report (MMWR) 2014 Sep 19;63(37):822–825. [PMC free article] [PubMed] [Google Scholar]
- 31.Baskerville NB, Liddy C, Hogg W. Systematic Review and Meta-Analysis of Practice Facilitation Within Primary Care Settings. Ann Fam Med. 2012 Jan-Feb;10(1):63–74. doi: 10.1370/afm.1312. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Taylor EF, Machta RM, Meyers DS, Genevro J, Peikes DN. Enhancing the Primary Care Team to Provide Redesigned Care: The Roles of Practice Facilitators and Care Managers. Ann Fam Med. 2013 Jan-Feb;11(1):80–83. doi: 10.1370/afm.1462. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Lau D, Hu J, Majumdar SR, Storie DA, Rees SE, Johnson JA. Interventions to Improve Influenza and Pneumococcal Vaccination Rates Among Community-Dwelling Adults: A Systematic Review and Meta-Analysis. Ann Fam Med. 2012 Nov-Dec;10(6):538–546. doi: 10.1370/afm.1405. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental Table 1. Intervention Strategies Used to Increase Adult Vaccination Rates from the 4 Pillars™ Immunization Toolkit.