Trends in Influenza Vaccine Uptake and Severe Influenza-Related Outcomes at Kaiser Permanente Southern California, 2007-2017

Abstract

Introduction:

Major efforts to increase influenza vaccine uptake among Kaiser Permanente Southern California (KPSC) members have been undertaken in recent years. However, whether these improvements translate to a decline in severe influenza-related outcomes has not been examined. We aimed to understand the impact of the influenza vaccination program at KPSC by examining influenza vaccine uptake and 3 severe influenza-related outcomes.

Methods:

We conducted an ecologic trend analysis to understand influenza vaccine uptake and influenza-related hospitalization, intensive care unit (ICU) admission, and mortality for each influenza season (2007-2017). The same cohort was followed from the influenza season to the noninfluenza season immediately afterward while using the noninfluenza season as the comparison group. We also assessed the within-season correlation between influenza vaccine uptake and influenza-related outcomes.

Results:

Influenza vaccine uptake rose from 23.9% to 45.5%, and all 3 influenza-related outcome rates declined (hospitalization: 35.4-26.8/10,000 patients; ICU: 5.9-5.2/10,000 patients; and mortality: 3.4-2.3/10,000 patients). Influenza vaccine uptake was negatively correlated with hospitalization (−0.32, p < 0.001) and mortality (−0.29, p = 0.001). However, once we adjusted for the noninfluenza season, the results of the correlation analysis were no longer statistically significant.

Conclusion:

Although we could not establish a statistically significant inverse relationship between influenza vaccination and severe influenza-related outcomes over the study period, our findings indicate an overall decline in influenza-related outcomes over the study period, suggesting improvements in both preventive and acute care quality at KPSC.

INTRODUCTION

Routine annual influenza vaccination is recommended for the prevention of influenza among all persons aged ≥ 6 months who do not have contraindications.¹ Attention focused on improving influenza vaccine uptake across the United States has been relatively effective, with the Centers for Disease Control and Prevention reporting that the percentage of vaccinated individuals increased from 43% in the 2010-2011 season to 49% in the 2018-2019 season.² Efforts to increase vaccination rates are undertaken with the assumption that they will translate to decreases in influenza-related hospitalizations and death.

Different study designs have been used to assess influenza vaccine effectiveness (VE), but achieving a truly accurate assessment is difficult. Although randomized, placebo-control studies are considered the ideal study design for evaluating vaccine protection and minimizing bias, they are typically conducted prelicensure for influenza vaccine rather than for postlicensure annual assessments of VE.^3,4 Observational cohort studies are perhaps the most common approach used to assess influenza VE, but the findings have been rather controversial. Researchers have detected substantial selection biases, resulting in an overestimation of vaccine benefits.^5-9 For example, a study conducted among elderly subjects reported that influenza vaccine reduced all-cause mortality by more than 50%¹⁰; however, this number was subsequently questioned by Simonsen et al,⁹ who found it difficult to reconcile that the vaccination could prevent more than 10-fold the number of deaths attributed to the virus. Some observational cohort studies have used laboratory-confirmed influenza cases to evaluate influenza-related deaths,^11,12 but they are likely to provide underestimates of VE.¹³

Another approach is a test-negative study, a special type of case-control study in which all individuals presenting with an influenza-like illness (ILI) are tested for influenza; cases are those who test positive, and comparison subjects are those who test negative.^14,15 Although this study design reduces confounding due to differences in health care seeking behavior, data are sparser on the use of test-negative studies to capture severe influenza-related outcomes.⁴

Given the limitations of the aforementioned designs, ecologic trend studies provide another option for assessing the relationship between influenza vaccination and outcomes for large study populations. Although trend studies do not take into account individual vaccination status, they are also not vulnerable to selection bias and can provide valuable information on group-level effects.⁹ Because influenza is a seasonal infection, the true protective effect of vaccination should be limited to the influenza season. Analyses comparing outcome rates in the influenza season versus the noninfluenza season assume that the rate difference can be attributed to influenza^16,17; thus, the purpose of comparing season and nonseason rates is to adjust for secular confounding. Despite their limitations, ecologic trend studies have proven valuable in epidemiological research and for establishing important medical guidelines.¹⁸

To understand the potential impact of the vaccination program at Kaiser Permanente Southern California (KPSC), we examined the association between influenza vaccination and severe influenza-related outcomes. The availability of many years of data from KPSC's extensive electronic health records (EHR) and diverse patient population provide a solid footing for using an ecologic trend study to evaluate associations.

METHODS

Study Setting

We assessed influenza vaccination rates and 3 outcomes (influenza-related hospitalization, intensive care unit [ICU] admission, and death) from 2007 to 2017 using data from KPSC, which serves more than 4.6 million members across 15 medical centers. KPSC maintains comprehensive EHR, which include information on vaccines administration, diagnosis and procedure codes, encounter records, pharmacy data, and demographics.

Subjects

In this study, the annual sample of study subjects was formed based on the number of KPSC health plan members who were actively enrolled on September 1 of each year.

Measures

Exposure

Influenza vaccine uptake was the main exposure measure. Vaccination uptake was determined using records of influenza vaccines administered at a KPSC or a partner facility as well as from patient-reported receipt of the vaccine outside the system. The annual influenza vaccination period was defined as September 1 of each year to April 30 of the following year.

Outcomes

We assessed the rates of 3 influenza-related events: hospitalizations, ICU admissions, and deaths based on pneumonia and influenza (P&I) ICD diagnosis codes or underlying cause of death codes (ICD-9: 480-488, ICD-10: J09-J18) in both the influenza season (October-May) and noninfluenza season (the following June-September). We used place of service codes (the location where medical care was provided) and event type information to identify hospitalizations and department specialty information to identify ICU admissions. Influenza-related deaths were identified using multiple cause-of-death data (P&I ICD codes) from the state of California and supplemented with information from EHR.¹⁹ Patients with multiple encounters for 1 outcome measure (eg, more than 1 hospitalization) in each season were only counted once. Patients with more than 1 outcome measure (eg, hospitalization followed by a transfer to the ICU) were counted once for each.

Potential Confounders

We identified age category (0-4 years, 5-17 years, 18-49 years, 50-64 years, and ≥ 65 years), sex, race/ethnicity (Hispanic, non-Hispanic Asian, non-Hispanic Black, non-Hispanic White, and Multiple/Other/Unknown race/ethnicity), and place of service (ie, medical center) as potential confounders and adjusted for them in the multivariable analysis.

Analysis

Ecologic Trend Analysis

Influenza vaccine uptake and the 3 outcomes (influenza-related hospitalization, ICU admission, and death) were evaluated during each influenza season from 2007-2008 through 2016-2017. Overall rates were age-, sex-, and race/ethnicity-standardized (direct standardization) using the population from the final season (2016-2017) to account for demographic changes over the study period. Standardized rates and rates stratified by age group, sex, and race/ethnicity were plotted. We used χ² tests to evaluate group total rate differences for demographic variables. Total rates and 95% confidence intervals (CIs) were estimated using the binomial proportions exact method. Annual percentage change and 95% CIs were estimated using Poisson regression models adjusting for age, sex, race/ethnicity, and medical center. The same cohort was followed from the influenza season to the noninfluenza season occurring immediately afterward, using noninfluenza season data as the comparison group.

Within-Season Correlation

In addition to the ecologic trend analysis, we assessed the correlation between influenza vaccine uptake and influenza-related outcome rates in the same season and medical center. The correlation was weighted by the population size of each medical center. We compared the correlation in the influenza season versus the noninfluenza season. The noninfluenza season was used to remove the baseline differences in the underlying characteristics of the population in various medical centers. p Values were calculated for the Pearson's correlation coefficients. All analyses were conducted using SAS version 9.4 (SAS Institute Inc, Cary, NC).

Institutional Review Board Approval

Ethical approval for this study was obtained from the Institutional Review Board of KPSC. Waivers for informed consent and HIPAA authorization were granted by the Institutional Review Board.

RESULTS

General Characteristics of the Study Population

KPSC membership steadily increased over the study period, from 3.2 million in the 2007-2008 season to 4.1 million in the 2016-2017 season (Table 1). The percentage of members age ≥ 65 years increased by approximately 3.6% (from 10.9% to 14.5%), and the percentage of female subjects remained consistent (51.4%-51.5%). There were notable shifts in race/ethnicity over the study period, with Hispanic membership increasing from 26.8% to 36.6% and the Multiple/Other/Unknown group decreasing from 21.5% to 12.0%. The substantial changes in Multiple/Other/Unknown race/ethnicity were mainly due to a KPSC initiative implemented in 2011 to collect self-reported race/ethnicity to meet stipulations of meaningful use introduced as part of the Affordable Care Act.

Table 1.

Demographic characteristics of study population in Years 1, 5 and 10, 2007-2017, Kaiser Permanente Southern California

	Year 1a		Year 5		Year 10
	Season 2007-2008		Season 2011-2012		Season 2016-2017
	n	%	n	%	n	%
Ageb
0-4 years	146,309	4.6	152,342	4.5	171,323	4.2
5-17 years	663,634	20.9	668,357	19.7	712,292	17.5
18-49 years	1,390,351	43.8	1,463,049	43.1	1,792,994	44.0
50-64 years	624,385	19.7	692,448	20.4	808,648	19.8
≥65 years	346,539	10.9	420,877	12.4	588,851	14.5
Sex
Female	1,629,829	51.4	1,757,241	51.7	2,098,389	51.5
Male	1,541,389	48.6	1,639,832	48.3	1,975,719	48.5
Race/ethnicity
Hispanic	850,279	26.8	1,036,975	31.9	1,492,413	36.6
Multiple/Other/Unknown	681,045	21.5	467,246	14.4	490,279	12.0
Non-Hispanic Asian	223,993	7.1	270,308	8.3	386,146	9.5
Non-Hispanic Black	292,081	9.2	299,728	9.2	334,773	8.2
Non-Hispanic White	1,123,820	35.4	1,171,652	36.1	1,370,497	33.6
Total	3,171,218	100.0	3,397,073	100.0	4,074,108	100.0

^a.Cohort for each influenza season was defined as KPSC members on September 1st of each year.

^b.Subjects’ age for each influenza season was calculated on September 1st of each year.

Influenza Vaccine Uptake

There was a notable increase in influenza vaccine uptake over the 10 influenza seasons (Figure 1 and see Table S1 in the Supplemental Material), from 23.9% in 2007-2008 to 45.5% in 2016-2017. The total rate of influenza vaccine uptake for the 10 seasons was 37.4% (95% CI = 37.4-37.4%). The unadjusted and adjusted annual percent change was 6.1% (95% CI = 6.1-6.1%) and 5.4% (95% CI = 5.4-5.4%) (Table 2), respectively.

Figure 1.

Influenza vaccine uptake, 2007-2017§, Kaiser Permanente Southern California. § Influenza vaccination period defined as September-April.*Overall rates were standardized for age, sex, and race/ethnicity using data from the last season (2016-17).

Table 2.

Adjusted annual percent change of influenza vaccination and influenza-related outcomes, 2007-2017, Kaiser Permanente Southern California

	Influenza season (October-May)		Noninfluenza season (June-September)
	Adjusted annual changea (%)	95% CI	Adjusted annual changea (%)	95% CI
Influenza vaccination	5.4	5.4 to 5.4
Influenza-related hospitalization	−4.7	−4.9 to −4.5	−5.0	−5.3 to −4.7
Influenza-related ICU admission	−3.1	−3.6 to −2.6	−4.1	−4.9 to −3.4
Influenza-related death	−5.5	−6.2 to −4.8	−5.7	−6.9 to −4.6

^a.Adjusted for age, sex, race/ethnicity, and medical center.

ICU = intensive care unit.

Influenza vaccine uptake improved in all subgroups (see Figures S1 and S2 and Table S1 in the Supplemental Material) during the study period. Members age ≥ 65 years consistently had the highest uptake (10-season total rate = 69.5%, 95% CI = 69.5-69.6%), and members age 18-49 years had the lowest uptake (25.3%, 95% CI = (25.3% [25.3%, 25.3%]); p-value <0.001 for five age groups. The total vaccination uptake rate was 40.4% (95% CI = 40.4-40.4%) among women and 34.2% (95% CI = 34.2-34.2%) among men (p < 0.001). Non-Hispanic Asian members had the highest mean vaccination rate (48.8%, 95% CI = 48.8-48.9%), and non-Hispanic White members ranked second (42.5%, 95% CI = 42.5-42.5%), followed by Hispanic members (36.7%, 95% CI = 36.7-36.8%) and non-Hispanic Black members (30.8%, 95% CI = 30.7-30.8%). Members in the Multiple/Other/Unknown race/ethnicity category had the lowest uptake (23.1%, 95% CI = 23.1-23.1%; p < 0.001).

Outcomes

Influenza-Related Hospitalization

Influenza Season

The rate of influenza-related hospitalization during the influenza season decreased over the study period, from 35.4/10,000 patients in 2007-2008 to 26.8/10,000 patients in 2016-2017 (Figure 2 and see Table S2 in the Supplemental Material). The unadjusted annual percent change was −2.8 % (95% CI = −3.0 to −2.6%), and the adjusted annual percent change was −4.7% (95% CI = −4.9 to −4.5%) (Table 2). Members age ≥ 65 years had the highest rate of influenza-related hospitalization during all influenza seasons (total rate = 153.0, 95% CI = 151.9-154.1/10,000 patients), whereas those age 5 to 17 years had the lowest (5.1, 95% CI = 4.9-5.3/10,000 patients; p < 0.001). Men had a higher mean influenza-related hospitalization rate during the influenza season (33.0, 95% CI = 32.7-33.3/10,000 patients) compared with women (29.6, 95% CI = 29.3-29.8/10,000 patients; p < 0.001).

Figure 2.

Rates of influenza-related outcomes per 10,000 patients, 2007-2017, Kaiser Permanente Southern California.

Differences in influenza-related hospitalization rates were also observed by race/ethnicity (see Figure S3 and Table S2 in the Supplemental Material). Non-Hispanic White members had the highest rate (50.0, 95% CI = 49.6-50.4/10,000 patients), and members in the Multiple/Other/Unknown category had the lowest (12.9, 95% CI = 12.6-13.3/10,000 patients; p < 0.001).

Noninfluenza Season

During the noninfluenza seasons, the influenza-related hospitalization rate also declined during the study period, shifting from 12.5/10,000 patients (2007-2008) to 10.4/10,000 patients (2016-2017). The unadjusted annual percent change was −3.1% (95% CI = −3.4 to −2.7%). and the adjusted annual percent change was −5.0% (95% CI = −5.3 to −4.7%) (Table 2).

Influenza-Related ICU Admissions

Influenza Season

The rate of influenza-related ICU admissions during the influenza season decreased over the study period, from 5.9 per 10,000 patients in the 2007-2008 season to 5.2 per 10,000 patients in the 2016-2017 season (Figure 2 and see Table S3 in the Supplemental Material). The unadjusted annual percent change was −1.2% (95% CI = −1.7 to −0.7%), and the adjusted annual percent change was −3.1% (95% CI = −3.6 to −2.6%) (Table 2).

Members age ≥ 65 years had the highest mean influenza-related ICU admission rate during the influenza season (25.8, 95% CI = 25.3-26.3/10,000 patients), whereas members age 5 to 17 years had the lowest rate (0.9, 95% CI = 0.8 to 0.9/10,000 patients; p < 0.001). Men had a higher influenza-related ICU admission rate during the influenza season (6.2, 95% CI = 6.1-6.3/10,000 patients) compared with women (4.8, 95% CI, 4.7-4.9/10,000 patients; p < 0.001).

The influenza-related ICU admission rate varied by race/ethnicity (see Figure S3 and Table S3 in the Supplemental Material). During the study period, non-Hispanic Black members had the highest rate (9.1, 95% CI = 8.7-9.4/10,000 patients), and members in the Multiple/Other/Unknown race/ethnicity category again had the lowest rate (2.0, 95% CI = 1.9-2.1/10,000 patients; p < 0.001).

Noninfluenza Season

In the noninfluenza season, the influenza-related ICU admission rate also decreased during the study period, from 2.3/10,000 patients in the 2007-2008 season to 2.0/10,000 patients in the 2016-2017 season. The unadjusted annual percent change was −2.3% (95% CI = −3.0 to −1.5%), and the adjusted annual percent change was −4.1% (95% CI = −4.9 to −3.4%) (Table 2).

Influenza-Related Death

Influenza Season

Rates of influenza-related deaths occurring within the influenza season (Figure 2 and see Table S4 in the Supplemental Material) decreased over the study period, from 3.4 per 10,000 patients (2007-2008) to 2.3/10,000 patients (2016-2017). The unadjusted annual percent change was −3.3% (95% CI = −4.0 to −2.6%), and the adjusted annual percent change was −5.5% (95% CI = −6.2 to −4.8%) (Table 2).

Members age ≥ 65 years had the highest mean influenza-related death rate during the influenza season (16.9, 95% CI = 16.5-17.2/10,000 patients), whereas members age 5 to 17 years had the lowest rate (0.04, 95% CI = 0.03-0.06/10,000 patients; p < 0.001). Men had a slightly higher influenza-related death rate during the influenza season (2.8, 95% CI = 2.7-2.9/10,000 patients) than women (2.4, 95% CI = 2.3-2.4/10,000 patients; p < 0.001).

Differences in influenza-related mortality during the influenza season were also observed by race/ethnicity (see Figure S3 and Table S4 in the Supplemental Material). We found that non-Hispanic White members had the highest death rate (4.8, 95% CI = 4.6-4.9/10,000 patients), whereas Hispanic members had the lowest (0.8, 95% CI = 0.8-0.9/10,000 patients; p < 0.001).

NonInfluenza Season

The rates of influenza-related deaths in the noninfluenza season decreased over the study period, from 1.1/10,000 patients (2007-2008) to 0.8/10,000 patients (2016-2017). The unadjusted annual percent change was −3.5% (95% CI = −4.6 to −2.4%), and the adjusted annual percent change was −5.7% (95% CI = −6.9 to −4.6%) (Table 2).

Correlation Between Influenza Vaccine Uptake and Influenza-Related Outcomes

Table 3 shows the weighted correlation between influenza vaccine uptake and each influenza-related outcome in both the influenza season and noninfluenza season as well as the influenza-related outcomes adjusted for the noninfluenza season rates. Without controlling for the noninfluenza season, influenza vaccine uptake was negatively correlated with hospitalization (−0.32; p < 0.001) and mortality (−0.29; p < 0.001), but these associations were no longer significant after adjustment.

Table 3.

Correlation between influenza vaccine uptake and influenza-related outcomes, 2007-2017, Kaiser Permanente Southern California

Influenza-related outcomes	Influenza vaccine uptake weighted correlation (%)a	p valueb
Influenza season ratesc
Hospitalization	−0.323	< 0.001
ICU admission	−0.145	0.089
Death	−0.285	0.001
Noninfluenza season ratesc
Hospitalization	−0.381	< 0.001
ICU admission	−0.230	0.006
Death	−0.205	0.015
Adjusted ratesd
Hospitalization	−0.012	0.891
ICU admission	0.133	0.118
Death	−0.095	0.263

^a.Weight = subject count in each medical center in each season.

^b.p value calculated using Pearson’s correlation.

^c.Influenza season defined as October-May; noninfluenza season defined as the June-September immediately following the influenza season.

^d.Correlation between influenza vaccine uptake and influenza-related outcomes, adjusted for the following noninfluenza season rate.

ICU = intensive care unit.

DISCUSSION

Overall, influenza vaccine uptake rose steadily over the study period, suggesting that targeted efforts to improve coverage at KPSC have been successful. At the same time, we found declining rates of severe outcomes attributed to ILI, with the exception of an uptick during the H1N1 pandemic in 2009-2010. Similarly, our within-season correlation analysis showed a significant inverse association between vaccination and severe influenza-related outcomes. Despite these encouraging findings, once we adjusted for the noninfluenza season, the results of the correlation analysis were no longer statistically significant. Although general improvements in care (whether concomitant or through specific interventions) may explain the decrease in severe outcomes, they could be completely confounded and difficult to tease out at the ecologic level.

Although a number of observational studies have reported powerful protective effects of influenza vaccine against severe outcomes,^11,12 our findings are aligned with other trend analyses. An ecologic trend study of 14 European nations identified a significant inverse relationship between influenza vaccination and ILI incidence in only 1 country.²⁰ Another study used a cyclical regression model with over 30 years of US data to examine influenza vaccination and mortality (identified with P&I cause-of-death codes) and could not find a significant association.⁸ An ecologic trend study used P&I codes from death certificates and reported no decline in mortality despite increasing rates of influenza vaccination coverage.²¹

Despite some similar findings, not all ecologic studies have reported nonsignificant results. A Brazilian study using 14 years of data observed a negative correlation (−0.001; p < 0.001) between vaccination coverage and hospitalization rates in the elderly population.²² However, this study did not provide a comparison group to adjust for secular trends in care improvement over time. An ecologic trend study using all-cause mortality data identified an inverse relationship between influenza vaccination and death.²³ It may be that the selected outcome measure contributes to discrepancies found among studies with similar designs. Studies evaluating influenza-related deaths typically use either P&I cause-of-death or all-cause mortality data. Due to the broader classification of the latter, estimates obtained with this approach appear to overestimate influenza deaths.²⁴

For studies investigating severe outcomes other than mortality, laboratory confirmation with reverse transcription-polymerase chain reaction testing is considered the gold standard.²⁵ However, obtaining population-wide estimates with this approach is infeasible because influenza is not routinely verified with this method. Additionally, the onset of secondary complications is often responsible for influenza-related hospitalization or death, but the virus may have cleared by that point.^11,26 Given these limitations, ICD codes are often used instead, but there are challenges with this as well. Using the parent category of P&I codes, influenza cases have reportedly been overestimated,²⁷ and using codes for influenza only may result in underestimates. Furthermore, the nonspecific clinical presentation of influenza likely corresponds with nonspecific diagnosis coding, which can lead to inaccurate conclusions about the benefits of vaccination.²⁸

In addition to the complexities noted above, some specific limitations of our study should be considered when interpreting the results. Because ecologic trend studies use aggregated group-level data, exposure, confounders, and outcomes are based on population averages. As such, analyses were not done on individual-level data, and we cannot be certain if vaccination exposure preceded our examined outcomes; thus, causal inferences cannot be made. Although influenza vaccination rates steadily improved at KPSC over the study period from 23.9% to 45.5%, it is possible that our vaccination uptake was too low to detect significant correlations. Other studies have found that higher influenza vaccination coverage was associated with decreased rates of hospitalization and mortality.^29,30 The association between vaccination and influenza-related outcomes can also vary from year to year based on the severity of the circulating influenza virus subtypes and potential mismatch between the vaccine contents and circulating viruses. In addition, because the activity of influenza virus and ILI may be influenced by weather, the true influenza seasons in Southern California could be different than September to April.³¹ Furthermore, there was potential misclassification because the exact start and end of the influenza season each year may have differed from October and May, which could affect comparisons between the influenza season and noninfluenza season. However, sensitivity analyses conducted using a narrower period for the influenza season (November-April) showed that our analysis estimates were robust to changes in the definition of influenza season.

There are some strengths of our study to note as well, related to both the study design and setting. First, an advantage of using an ecologic study design is that it is relatively quick to conduct, allowing data to be incorporated as soon as it becomes available. In addition, this type of analysis provides a high-level overview of trends, which can be helpful in making future projections. We also were able to link information from KPSC’s EHR with death data from California.

Although we could not establish a statistically significant inverse relationship between influenza vaccination and severe influenza-related outcomes over the study period, our findings suggest improvements in both preventive and acute care quality at KPSC. This is aligned with a previous investigation of mortality trends, which reported declining rates of all-cause mortality among KPSC members from 2001 to 2016.¹⁹ At the same time, our study highlights some of the challenges and tradeoffs of examining the relationship between influenza vaccination and severe outcomes. These tradeoffs suggest that no single approach is likely to provide a definitive estimate of effectiveness and that truth may best be determined through the triangulation of multiple approaches, while taking the necessary factors into account.