Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Jan 1.
Published in final edited form as: Am J Kidney Dis. 2019 Aug 1;75(1):72–83. doi: 10.1053/j.ajkd.2019.05.018

Comparative Effectiveness of High-Dose Versus Standard-Dose Influenza Vaccine Among Patients Treated by Maintenance Hemodialysis

Anne M Butler 1,2, J Bradley Layton 3, Vikas R Dharnidharka 4, John M Sahrmann 1, Marissa J Seamans 5, David J Weber 6, Leah J McGrath 7
PMCID: PMC6926162  NIHMSID: NIHMS1536423  PMID: 31378646

Abstract

Rationale & Objective:

Studies among patients on maintenance dialysis suggest that standard-dose influenza vaccine (SDV) may not prevent influenza-related outcomes. Little is known about the comparative effectiveness of SDV versus high-dose influenza vaccine (HDV) in this population.

Study Design:

Cohort study using data from the United States Renal Data System.

Setting & Participants:

507,552 adults undergoing in-center maintenance hemodialysis between the 2010/11 and 2014/15 influenza seasons.

Exposures:

SDV and HDV.

Outcomes:

All-cause mortality, hospitalization due to influenza or pneumonia, and influenza-like illness (ILI) during the influenza season.

Analytic Approach:

Patients were eligible for inclusion in multiple yearly cohorts, thus our unit of analysis was the influenza patient-season. To examine the relationship between vaccine dose and effectiveness outcomes, we estimated risk differences and risk ratios using propensity score weighting of Kaplan-Meier functions, accounting for a wide range of patient- and facility-level characteristics. For non-mortality outcomes, we used competing risk methods to account for the high mortality rate in the dialysis population.

Results:

Within 225,215 influenza patient-seasons among adults ≥65 years, 97.4% received SDV and 2.6% received HDV. We observed similar risk estimates for HDV and SDV recipients for mortality (risk difference, −0.08%; 95% CI, −0.85% to 0.80%), hospitalization due to influenza or pneumonia (risk difference, 0.15%; 95% CI, −0.69% to 0.93%), and ILI (risk difference, 0.00%; 95% CI, −1.50% to 1.08%). Our findings were similar among adults <65, as well as within subgroups defined by influenza season, age group, dialysis vintage, month of influenza vaccination, and vaccine valence.

Limitations:

Residual confounding and outcome misclassification.

Conclusions:

The HDV does not appear to provide additional protection beyond the SDV against all-cause mortality or influenza-related outcomes for adults undergoing hemodialysis. The additional cost and side-effects associated with HDV should be considered when offering this vaccine. Future studies of HDV and other influenza vaccine strategies are warranted.

Keywords: Comparative effectiveness, end-stage renal disease (ESRD), hemodialysis, high-dose influenza vaccine, hospitalization, immunization, influenza, mortality, standard-dose influenza vaccine, vaccine effectiveness, influenza-like illness (ILI), hospitalization, vaccination, infection, renal dialysis

Introduction

Influenza causes substantial morbidity and mortality in patients with end-stage renal disease (ESRD). Patients with ESRD have an impaired innate and adaptive immune system, including defects in complement activation and B- and T-cell function,13 which contribute to increased risks for severe complications from influenza or influenza-related hospitalizations, mortality, and healthcare costs compared to the general population.47 The Centers for Disease Control and Prevention (CDC) has long recommended that patients with ESRD receive annual influenza vaccination.8,9

Currently, several types of influenza vaccines are available in the United States, including the standard-dose and high-dose influenza vaccines (HDV). The trivalent—and more recently quadrivalent—inactivated, standard-dose, seasonal influenza vaccines have been administered as standard practice in dialysis clinics. In 2009, the United States Food & Drug Administration licensed a trivalent HDV for use among adults ≥65 years to induce higher antibody responses and provide better protection from influenza than the standard-dose vaccine (SDV).10 The HDV contains the same three strains as SDV but more antigen (60 versus 15 μg per strain) than standard vaccines. The use of HDV among patients on maintenance hemodialysis has increased slowly over time.11

An accumulating body of literature demonstrates that the HDV is more effective than SDVs in preventing influenza-related medical encounters, hospitalizations, and mortality in the general population of adults ≥65 years.1218 Yet, some studies in healthy, older adults have not demonstrated a benefit of HDV.19,20 In the dialysis population, a recent observational study by Miskulin et al. reported that HDV was more effective than SDV in preventing all-cause hospitalization during the 2016/17 influenza season but not the 2015/16 season.21 However, this study was limited by small sample size, adjustment for a small set of potential confounders, and the absence of influenza-related outcomes, warranting additional examination of the effectiveness of the HDV in the dialysis population.

We sought to compare the clinical effectiveness of HDV vs. SDV in preventing all-cause mortality, hospitalization due to influenza or pneumonia, and influenza-like illness (ILI) among patients with chronic kidney failure undergoing maintenance hemodialysis. As the healthy-user effect and frailty are well-documented challenges of observational studies of influenza vaccine effectiveness in populations with poor health, we used robust methods to account for possible confounding. We also used competing risk methods to account for the high mortality rate in the dialysis population.

Methods

Data Source

We utilized the United States Renal Data System (USRDS) to identify patients with chronic kidney disease receiving in-center maintenance hemodialysis during the years 2009–2015. USRDS is a national registry of patients with ESRD who are eligible for Medicare coverage from the Centers for Medicare and Medicaid Services (CMS), and it contains routinely-reported clinical information from dialysis centers, as well as Medicare claims for inpatient, outpatient, and physician diagnoses and procedures.22

Study Design and Patients

We constructed yearly cohorts for five individual influenza seasons: 2010/11, 2011/12, 2012/13, 2013/14, 2014/15. The index dates were anchored on the date of influenza vaccination (Figure 1), which was required to be administered prior to the start date of the influenza season. Eligible patients included adults ≥18 years with ESRD and Medicare as a primary insurance payer who had started hemodialysis at least 9 months prior to the index date, to allow for a 3-month Medicare enrollment period prior to the 6- month baseline period. We further required continuous hemodialysis receipt for the 3 months immediately prior to vaccination. Baseline covariates were ascertained during the 6-month baseline period prior to the index date. The follow-up period began on the start date of the influenza season. The start and end dates of each influenza season were defined using national influenza surveillance data from the CDC.2327 Specifically, the start and end of each season were defined as the midpoint of the first week during which >10% and <10%, respectively, of national culture isolates were positive for influenza (Table S1). The primary analysis was conducted in adults ≥65 years; a secondary analysis was conducted in adults <65 years. Patients could be eligible for inclusion in multiple yearly cohorts.

Figure 1.

Figure 1.

Study design schematic of inclusion criteria and follow-up time for each yearly cohort of adult patients undergoing maintenance hemodialysis. The index date was anchored on the date of influenza vaccination, which was required to be administered prior to the start date of the influenza season. Follow-up began on the start date of the influenza season.

Exposure Assessment

Influenza vaccines were classified as HDV, SDV, or unknown (Table S2), based on outpatient, physician, and inpatient procedure coding in billing claims for influenza vaccine or influenza vaccine administration. We searched for Current Procedural Terminology (CPT), Healthcare Common Procedure Coding System (HCPCS), and International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes assigned between August 1 and the end of each influenza season. We assumed that multiple codes within a 7-day period were duplicate codes for the same vaccination, and therefore, two codes for the same vaccine dose were collapsed, whereas two codes for a known and unknown dose were categorized as the known dose. Codes within 7 days for different known doses were considered separate doses. Codes spaced by greater than 7 days were also considered separate doses.

Outcome Assessment

We examined three primary clinical outcomes: all-cause mortality, the first occurrence of hospitalization for influenza or pneumonia, and the first occurrence of ILI. All-cause mortality was assessed using the CMS ESRD Death Notification Form. As laboratory-confirmed influenza status is not available in the claims data, we employed two levels of claims-based influenza definitions from diagnosis coding: 1) hospitalization for influenza or pneumonia; and 2) inpatient or outpatient influenza-like illness (ILI) (see Table S3 for code lists). We performed a sensitivity analysis of ILI with a more narrow definition, as done previously.28,29

Covariates

Our analyses included a wide range of covariates identified during the 6-month baseline period (Figure 1, Table S4). Covariates included demographic characteristics (e.g., age, sex, race, dual-eligible for Medicaid, region, year), clinical characteristics (e.g., cause of ESRD, years on dialysis), dialysis facility characteristics (e.g., affiliation, type, profit status, size), and several comorbidities and procedures captured using ICD-9-CM and HCPCS codes. To account for potential confounding by frailty30,31 and the healthy-user bias,32,33 we also included covariates on preventive health services (e.g., other vaccinations, health screenings), healthcare utilization (e.g., skilled nursing days, hospital days), and frailty indicators (e.g., oxygen use, mobility aids). We categorized the timing of vaccine administration as vaccination in August or September versus October through the start of influenza season.

Statistical Analysis

Patients could be eligible for inclusion in multiple yearly cohorts, thus our unit of analysis was the influenza patient-season. To examine the relationship between vaccine dose and effectiveness outcomes, we estimated crude and weighted cumulative risk functions using Kaplan-Meier methods, accounting for the competing risk of death (for the non-mortality outcomes of ILI and hospitalization due to influenza or pneumonia).3436 We addressed potential confounding due to differences in observed covariates between HDV and SDV recipients by using propensity score weighting methods. Specifically, we applied standardized mortality ratio (SMR) weights to the Kaplan-Meier estimator. We estimated the predicted probability of receiving HDV, or the propensity score (ps), using logistic regression with a priori selected potential confounders as predictors (Table S4). Age was modeled using restricted quadratic splines with four knots. SMR weights for SDV recipients were calculated as ps/(1-ps); HDV recipients remained unweighted.37 We plotted the standardized mean differences of baseline covariates in the unweighted and SMR-weighted populations to determine whether weighting the population reduced imbalances of observed covariates and made the treatment groups more exchangeable.38 We stratified models by influenza season, valence, age, dialysis vintage (in years), and timing of influenza vaccination; weights were created separately for each subgroup analysis. Covariate balance was assessed separately for each subgroup analysis. Patients receiving influenza vaccine prior to the beginning of the influenza season were followed from the first day of the influenza season until the first occurrence of the outcome, a competing risk event (i.e., death for non-mortality outcomes), or a censoring event (i.e., disenrollment from Medicare Parts A and B, kidney transplant, switch to peritoneal dialysis, loss to follow-up, receipt of a subsequent influenza vaccine, or end of influenza season). We calculated risk differences and risk ratios using the cumulative risk function estimates at the end of each influenza season. A nonparametric-based bootstrap was used to estimate 95% confidence intervals (CIs) for daily risk differences and risk ratios between groups. Additionally, we conducted bias analyses to estimate the strength of unmeasured confounders that would be required to mask associations between vaccine type and each primary outcome after adjusting for measured covariates.39 This study using deidentified data was considered exempt from human subjects review by the Institutional Review Board at Washington University. Analyses were performed with SAS version 9.4 (SAS Institute, Cary, NC) and R Statistical Software 3.3.40

Results

After applying inclusion and exclusion criteria, we identified 255,281 eligible adult patients who contributed 507,552 unique influenza patient-seasons (Figure S1). In the primary analysis of adults ≥65 years (n = 225,215 influenza patient-seasons), SDV made up 97.4% (n=219,439) of the eligible vaccinations, and HDV was administered in the remaining 2.6% (n=5,776). All HDV were trivalent, whereas 76.7% of SDV was trivalent and 23.3% was quadrivalent. Tables 1 and S5 present characteristics of the study population stratified by vaccine dose. The mean age was slightly higher among HDV recipients than SDV recipients (75.8 vs. 74.6 years). HDV administration was less common among patients who were black or other race, dual-eligible for Medicaid, on dialysis for ≥3 years, or residing outside of the Midwest. HDV recipients had a higher prevalence of several comorbidities including ischemic heart disease, chronic obstructive pulmonary disease, cancer, liver disease. HDV and SDV recipients had a similar prevalence of frailty indicators. HDV recipients were more likely to receive preventive health care such as diabetic eye exams, lipid testing, and cancer screenings. Some characteristics of the HDV recipients changed over time (e.g., dialysis vintage; Table S6). The distribution of observed covariates was well-balanced after propensity-score weighting, as indicated by standardized mean differences less than 10% in the weighted population;41 Figure S2 presents these results for the primary analysis.

Table 1.

Characteristics of the 225,215 unique influenza patient-seasons among US adults ≥65 years undergoing maintenance hemodialysis, by influenza vaccine dose.

Characteristic HDV
n = 5,776 (2.6%)
SDV
n = 219,439 (97.4%)
Mean age, y 75.8 ± 6.9 74.6 ± 7.0
Male sex 3,064 (53.0%) 111,827 (51.0%)
Race
 White 4,378 (75.8%) 134,917 (61.5%)
 Black 1,125 (19.5%) 71,408 (32.5%)
 Other 273 (4.7%) 13,114 (6.0%)
Cause of chronic kidney failure
 Diabetes 2,524 (43.7%) 103,702 (47.3%)
 Hypertension 1,898 (32.9%) 71,973 (32.8%)
 GN 443 (7.7%) 14,708 (6.7%)
 Other 911 (15.8%) 29,056 (13.2%)
Region
 Northeast 928 (16.1%) 37,869 (17.3%)
 South 1,959 (33.9%) 97,158 (44.3%)
 West 801 (13.9%) 39,280 (17.9%)
 Midwest 2,088 (36.1%) 45,132 (20.6%)
Dual-eligible for Medicaid 1,338 (23.2%) 76,492 (34.9%)
Dialysis vintage
 < 1 year 420 (7.3%) 11,394 (5.2%)
 1–2 years 2,512 (43.5%) 76,237 (34.7%)
 3–4 years 1,348 (23.3%) 54,441 (24.8%)
 5–9 years 1,172 (20.3%) 58,560 (26.7%)
 ≥ 10 years 324 (5.6%) 18,807 (8.6%)
Hospitalized in last month (weeks)
 0 days 5,391 (93.3%) 205,549 (93.7%)
 1–6 days 257 (4.4%) 9,358 (4.3%)
 ≥ 7 days 128 (2.2%) 4,532 (2.1%)
Skilled nursing facility in last month 133 (2.3%) 5,911 (2.7%)
Comorbidities
 Cancer 895 (15.5%) 25,393 (11.6%)
 COPD 898 (15.5%) 29,177 (13.3%)
 Diabetes 3,666 (63.5%) 141,026 (64.3%)
 Hypertension 3,896 (67.5%) 139,110 (63.4%)
 Ischemic heart disease 3,613 (62.6%) 122,555 (55.8%)
 Liver disease 441 (7.6%) 10,511 (4.8%)
Frailty indicators
 Ambulance / life support 1,328 (23.0%) 55,171 (25.1%)
 Difficulty walking 714 (12.4%) 28,605 (13.0%)
 Mobility aidsa 526 (9.1%) 22,011 (10.0%)
 Skin ulcer (decubitus) 575 (10.0%) 19,792 (9.0%)
 Use of oxygen 664 (11.5%) 22,370 (10.2%)
 Weakness 632 (10.9%) 26,796 (12.2%)
Screening tests/Prevention
 HbA1C test 3,155 (54.6%) 129,109 (58.8%)
 Cancer screening 841 (14.6%) 30,543 (13.9%)
 Diabetic eye exam 2,177 (37.7%) 71,798 (32.7%)
 Hepatitis B vaccine/titre 1,288 (22.3%) 42,217 (19.2%)
 Lipid test 1,991 (34.5%) 66,996 (30.5%)
 Pneumococcal vaccine 321 (5.6%) 10,974 (5.0%)

Abbreviations: COPD, chronic obstructive pulmonary disease; HDV, high-dose vaccine; SD, standard deviation; SDV, standard-dose vaccine; HbA1c, hemoglobin A1c.

a

Defined as use of walker, wheelchair, or modified bathroom equipment.

The counts of outcome events, censoring events, competing risk events, and person-time at risk are presented for each outcome in Table 2. Among SDV recipients, the crude risks of death and hospitalization for pneumonia or influenza were similar (8.7% and 7.6%, respectively); in contrast, the risk of ILI was more than three times higher (28.1%). For each outcome, we observed similar weighted risks for HDV and SDV recipients throughout the influenza season, accounting for the competing risk of death for non-mortality outcomes (Fig 2). In the weighted analyses, we observed similar associations between vaccine dose and risk of mortality (risk difference, −0.08%; 95% CI, −0.85% to 0.80%), hospitalization due to influenza or pneumonia (risk difference, 0.15%; 95% CI, −0.69% to 0.93%), and ILI (risk difference, 0.00%; 95% CI, −1.50% to 1.08%). Results did not change appreciably in the analysis with the restricted ILI definition (Table S7). Estimates on the relative scale followed similar patterns (Table 2).

Table 2.

Estimates of the preventive effect of the high-dose vaccine (HDV) vs. standard-dose vaccine (SDV) on mortality, hospitalization due to influenza or pneumonia, and influenza-like illness measured at the end of each influenza season, accounting for the competing risk of death (for non-mortality outcomes), among adults ≥65 years.

Vaccine Dose Exposure Outcome Events Censoring Events Competing Risk Events (Deaths) Person-time, y Crude cumulative Risk, % (95% CI) Risk Difference, % (95% CI) Risk Ratio (95% CI)
Crude Weighted Crude Weighted
MORTALITY
 SDV 15,999 203,440 NA 85,068 8.7 (8.6, 8.9) Reference Reference Reference Reference
 HDV 457 5,319 NA 2,270 9.3 (8.5, 10.2) 0.58 (−0.22, 1.33) −0.08 (−0.85, 0.80) 1.07 (0.97, 1.15) 0.99 (0.91, 1.08)
HOSPITALIZATION DUE TO INFLUENZA/PNEUMONIA
 SDV 14,441 192,011 12,987 82,544 7.6 (7.5, 7.8) Reference Reference Reference Reference
 HDV 463 4,943 370 2,191 9.2 (8.4, 10.0) 1.51 (0.66, 2.28) 0.15 (−0.69, 0.93) 1.20 (1.09, 1.30) 1.02 (0.92, 1.10)
INFLUENZALIKE ILLNESS
 SDV 54,711 157,639 7,089 74,565 28.1 (27.9, 28.3) Reference Reference Reference Reference
 HDV 1,559 4,012 205 1,967 30.0 (28.9, 31.2) 1.90 (0.54, 3.02) 0.00 (−1.50, 1.08) 1.07 (1.02, 1.11) 1.00 (0.95, 1.04)

Abbreviations: CI, confidence interval; HDV, high-dose vaccine; NA, not applicable; SDV, standard-dose vaccine.

a

Risk differences (and risk ratios) were calculated as the difference (ratio) between the SDV and HDV cumulative risk functions.

Figure 2.

Figure 2.

Cumulative risk and 95% confidence interval estimates of mortality, hospitalization for pneumonia or influenza, and influenza-like illness, accounting for the competing risk of death (for non-mortality outcomes), pooled across five influenza seasons (2010/11, 2011/12, 2012/13, 2013/14, 2014/2015). Cumulative risk was estimated using standardized mortality ratio-weighted Kaplan-Meier functions. The scales of the y-axes differ by outcome.

Within subgroups, the risks of mortality, hospitalization for pneumonia or influenza, and influenza-like illness were generally similar between HDV and SDV recipients throughout the influenza season, apart from a few notable exceptions (Figs 35, Tables S8S10). First, we observed a higher risk of hospitalization among HDV versus SDV recipients in the 2010/11 season (risk difference, 2.85%; 95% CI, 0.59% to 5.86%), but not for any other seasons. Also, for both the hospitalization and ILI outcomes, we observed higher risk among HDV versus SDV recipients who were 65–74 years, lower risk among those who were 75–84 years, and no difference in risk between those <65 or ≥85 years.

Figure 3.

Figure 3.

The risk of mortality among patients who received the high-dose versus standard-dose influenza vaccines, by subgroup. Risk differences were calculated as the difference between weighted cumulative risk functions, with bootstrap confidence intervals. Analyses were performed in adults ≥65 years, with the exception of the age stratified analyses. Abbreviations: CI, confidence interval; HDV, high-dose vaccine; SDV, standard-dose vaccine.

Figure 5.

Figure 5.

The risk of influenza-like illness among patients who received the high-dose versus standard-dose influenza vaccines, by subgroup. Risk differences were calculated as the difference between weighted cumulative risk functions, accounting for the competing risk of death, with bootstrap confidence intervals. Analyses were performed in adults ≥65 years, with the exception of the age stratified analyses. Abbreviations: CI, confidence interval; HDV, high-dose vaccine; SDV, standard-dose vaccine.

In the bias analyses, we quantified the possibility that our observed null associations may be explained by an unmeasured confounding variable. We observed that the exposure–confounder relative risk and the confounder–outcome relative risk must have been at least as large as 1.4, 1.5, and 1.3 for mortality, hospitalization due to influenza or pneumonia, and ILI, respectively, to have shifted the corresponding weighted upper confidence limit across the null (Tables S1113).

Discussion

We conducted a large comparative study of the effectiveness of the high-dose versus standard-dose influenza vaccines among adults with chronic kidney failure undergoing maintenance hemodialysis in the US. In the overall population, we observed similar risks of all-cause mortality, hospitalization due to influenza or pneumonia, and ILI among patients who received HDV compared with those who received SDV. Our findings were generally consistent across subgroups of influenza season, age group, years on dialysis, timing of influenza vaccination, and valence. We observed differences between HDV and SDV in a few subgroups, but the patterns were not consistent across influenza seasons or age groups. Associations between a potential unmeasured confounder and our three outcomes would have required associations that are stronger than our measured confounder-outcome associations to have masked preventive effects.

Consideration of the results of comparative effectiveness studies of preventive therapies requires careful thought about the tradeoffs of benefits versus costs. Due to the substantial influenza burden in the dialysis population, even small improvements in vaccine effectiveness could be sufficient for recommending a specific type of annual influenza vaccination in this population. However, increased risks of side effects and increased costs must also be considered when weighing the cost-benefit of implementing a new vaccination strategy. Our results suggest that the HDV does not provide additional benefit beyond the SDV on a population-level. Additionally, the excess cost of the HDV is about $33 per dose,42 incurring an additional cost burden of $16.9 million under the assumption that all 511,000 patients on dialysis would be vaccinated with HDV.43 HDV is also more reactogenic, causing more frequent side effects, although most of these adverse events are minor (e.g., injection-site reactions, systemic adverse events, and gastrointestinal events).12,19,4448 Safety data on the HDV is limited to the healthy adult population; a comparison of the safety of HDV versus SDV in the ESRD population is needed.

Previous observational studies have demonstrated that SDVs are ineffective or minimally effective at reducing all-cause mortality or influenza-related hospitalizations in the ESRD population,29,49 despite reasonably robust antibody responses regarded as protective in 30% to 90% of vaccine recipients.5055 Our work extends these findings by clarifying that the HDV does not provide additional protection beyond the SDV against all-cause mortality or influenza-related outcomes, even though the HDV elicits greater antibody responses than SDV in patients with ESRD.56 These observations highlight the importance of post-licensure clinical studies that extend beyond immunogenicity-related outcomes to incorporate clinical outcomes.

Our conclusions differ somewhat from an observational study using data from a not-for-profit dialysis provider in the US.21 Similar to our study, Miskulin et al. reported similar rates of all-cause mortality among HDV versus SDV recipients. In contrast with our study, Miskulin et al. reported lower rates of all-cause hospitalization among HDV versus SDV recipients, though their finding was restricted to the 2016/2017 influenza season and not the 2015/2016 season. The 95% confidence intervals for the preventive effect of HDV (hazard ratio, 0.93; 95% CI, 0.86–1.00) included the null value, whereas the effect estimate moved down and away from the null in the subgroup analysis restricted to patients aged ≥65 years (hazard ratio, 0.88; 95% CI, 0.79–0.97). Other stratified analyses suggested similar protection among HDV and SDV recipients. Our results are not directly comparable with the previous study given several differences. Specifically, Miskulin et al. included later seasons (2015/2016 and 2016/17), patients undergoing peritoneal dialysis, and a smaller sample size (N=19,157). Furthermore, the previous study did not incorporate influenza-related outcomes nor did it use competing risks methodology to account for the high mortality in the dialysis population.

Perhaps most importantly, the analysis by Miskulin et al. accounted for a limited set of potential confounders and therefore their effect estimates may be subject to residual confounding bias. In contrast, our study accounted for potential confounding by frailty30,31 and the healthy-user bias32,33 through adjustment for a comprehensive set of individual-level and facility-level potential confounders. It is well-established that observational studies of the effectiveness of preventive interventions such as influenza vaccination in elderly or sick populations can be highly subject to confounding by frailty and healthcare-seeking behavior, as those who receive the treatment generally have higher anticipation of benefit by providers (e.g., longer predicted life expectancies), or stronger personal motivation to receive care (e.g. better access to healthcare, higher socioeconomic status, less frailty).49,5760 Often, this confounding is difficult to control and results in artificially protective adjusted effect estimates due to residual confounding. We were concerned that this phenomenon might also be present when comparing HDV -- a newer, more expensive, specialized vaccine — to the standard vaccine. However, that is not what we observed. Instead, older individuals receiving HDV tended to be sicker, and the crude effect estimate was above the null, suggesting that the confounding was not due to the healthy-user phenomenon. This is supported by our previous work, where we demonstrated that comparisons of HDV to SDV among adult patients with ESRD are less subject to confounding by the healthy user bias than comparisons of vaccinated vs unvaccinated persons.61 However, despite our attempts, the possibility of unmeasured confounding remains.

Our findings should be interpreted within the context of several factors that affect vaccine effectiveness, including influenza virus circulation, serologic vaccine match, and seasonal influenza severity.62 Our study period included a range of seasonal influenza severity (one low severity, two moderate, two high seasons) as measured in the overall population of older adults in the US.63 These severity metrics correlated with the observed seasonal risk of clinical outcomes in our analyses. Also, influenza A (H3N2) viruses – which are associated with lower vaccine effectiveness - were the predominant strains during all influenza seasons in our study, with the exception of 2013/14.2327 Compared to influenza A (H1N1) or influenza B viruses, H3N2 viruses have more frequently undergone antigenic change resulting in differences between the virus components of the influenza vaccine and circulating influenza viruses.64 However, the serologic vaccine match – the proportion of circulating influenza strains that matched the vaccine - was relatively high during three of the four seasons with H3N2 as the predominant strain (i.e., 97%, 80%, 86%, and 40% for seasons 2010/11, 2011/12, 2012/13, and 2014/15, respectively),2327 indicating similarity between the influenza viruses in the vaccine and circulating in the community.

Our results are subject to several limitations. First, our observational study design did not involve randomization of the exposure and therefore our effect estimates are potentially subject to confounding by unobserved differences between exposure groups. However, we attempted to control confounding through restriction of the study population (i.e., by vaccination status and age) as well as analytic adjustment for a rich set of covariates. It is noteworthy that we did not account for individual dialysis facilities within our regression models; however, our analysis accounted for dialysis center-level characteristics – several of which were strong predictors of HDV receipt.11 Second, our analysis only accounted for baseline characteristics measured prior to vaccination. However, this is supported by a previous study of influenza vaccine effectiveness among patients with ESRD that demonstrated that accounting for time-varying confounders of health status measured from both clinical and claims data did not reduce bias.49 Third, our study design required survival until 9 months after dialysis initiation; therefore, our results may not generalize to all incident patients. Fourth, in the absence of laboratory-confirmed influenza outcomes, we used claims-based definitions. Thus, outcome misclassification is a possibility, though we used sensitivity analyses to explore the impact of the definitions on our findings. Fifth, some subgroup analyses may have had limited ability to detect a true difference between HDV and SDV given small numbers of HDV recipients. Sixth, the start date of follow-up was based on national influenza data, and therefore ignores geographic variation in the timing of influenza seasons. Lastly, our study is primarily based on administrative billing claims data which is collected for administrative and reimbursement purposes, rather than for clinical research. Thus, we were not able to account for certain important clinical factors unavailable in claims data (e.g., BMI, vascular access, laboratory measurements) that may result in confounding bias; and some of our measures may be subject to misclassification, such as vaccination ascertainment if patients paid out-of-pocket. However, we expect this misclassification to be unlikely, as patients receiving regular in-center hemodialysis have very frequent healthcare encounters in dialysis clinics, and influenza vaccination is offered to Medicare beneficiaries without a co-pay. Importantly, our use of this data is also a strength because it captures information on a wide variety of health-related diagnoses, procedures, use of durable medical equipment, frailty markers, hospitalizations, and death.

In summary, our large comparative study failed to demonstrate that HDV has superior effectiveness compared to SDV for preventing all-cause mortality and influenza-related outcomes among patients receiving maintenance hemodialysis. Given the findings of our population-level study, along with the substantially higher cost and side-effect profile of HDV compared to SDV, it appears that HDV should not conclusively be considered the standard of care at the present time for influenza immunization of patients treated by maintenance hemodialysis. The findings of our population-level study should not be interpreted to discourage influenza vaccination in the dialysis population. Rather, dialysis patients should continue to receive annual influenza immunization per CDC guidelines.8 In addition, future studies of alternative strategies (e.g., booster doses) and alternative vaccine production technologies (e.g., adjuvanted or cell-based vaccines) are warranted, as there remains a need for improved influenza prevention efforts in this population.

Supplementary Material

1

Figure 4.

Figure 4.

The risk of hospitalization due to influenza / pneumonia among patients who received the high-dose versus standard-dose influenza vaccines, by subgroup. Risk differences were calculated as the difference between weighted cumulative risk functions, accounting for the competing risk of death, with bootstrap confidence intervals. Analyses were performed in adults ≥65 years, with the exception of the age stratified analyses. Abbreviations: CI, confidence interval; HDV, high-dose vaccine; SDV, standard-dose vaccine.

Acknowledgements:

The authors wish to acknowledge Amber Salter from Washington University for her biostatistical advice. The authors also wish to acknowledge Carrie O’Neil from Washington University for her editorial assistance.

Support: This project was supported by the US National Institute of Allergy and Infectious Diseases (NIAID), National Institutes of Health (NIH), through grant award number 1R21AI138385. AMB is supported by a grant from the National Center for Advancing Translational Sciences (NCATS), NIH under award number KL2 TR002346. MJS is supported by a grant from the National Institute on Drug Abuse (NIDA), NIH under award number T32DA007292. Data programming for this study was conducted by the Center for Administrative Data Research, which is supported in part by the Washington University Institute of Clinical and Translational Sciences grant UL1 TR002345 from the National Center for Advancing Translational Sciences (NCATS) of the National Institutes of Health (NIH), Grant Number R24 HS19455 through the Agency for Healthcare Research and Quality (AHRQ). The study sponsors did not have any role in study design; collection, analysis, and interpretation of data; writing the report; or the decision to submit the report for publication.

Financial Disclosure: AMB has received investigator-initiated support from Amgen. JBL is an employee of RTI International, an independent, non-profit research organization which performs research on behalf of government and commercial clients, including pharmaceutical companies. LJM is an employee of NoviSci, a data sciences company, and previously was an employee of RTI Health Solutions. The remaining authors declare that they have no relevant financial interests.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Publisher's Disclaimer: Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. The data reported here have been supplied by the United States Renal Data System (USRDS). The interpretation and reporting of these data are the responsibility of the authors and in no way should be seen as an official policy or interpretation of the U.S. government.

Peer Review: Received _______. Evaluated by 2 external peer reviewers, with direct editorial input from a Statistics/Methods Editor, an Associate Editor, and the Editor-in-Chief. Accepted in revised form May 23, 2019.

References

  • 1.Descamps-Latscha B The immune system in end-stage renal disease. Curr Opin Nephrol Hypertens. 1993;2(6):883–891. [DOI] [PubMed] [Google Scholar]
  • 2.Hauser AB, Stinghen AE, Kato S, et al. Characteristics and causes of immune dysfunction related to uremia and dialysis. Perit Dial Int. 2008;28 Suppl 3:S183–187. [PubMed] [Google Scholar]
  • 3.Kato S, Chmielewski M, Honda H, et al. Aspects of immune dysfunction in end-stage renal disease. Clin J Am Soc Nephrol. 2008;3(5):1526–1533. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Allon M, Depner TA, Radeva M, et al. Impact of dialysis dose and membrane on infection-related hospitalization and death: results of the HEMO Study. J Am Soc Nephrol. 2003;14(7):1863–1870. [DOI] [PubMed] [Google Scholar]
  • 5.Chavers BM, Solid CA, Gilbertson DT, Collins AJ. Infection-related hospitalization rates in pediatric versus adult patients with end-stage renal disease in the United States. J Am Soc Nephrol. 2007;18(3):952–959. [DOI] [PubMed] [Google Scholar]
  • 6.Naqvi SB, Collins AJ. Infectious complications in chronic kidney disease. Adv Chronic Kidney Dis. 2006;13(3):199–204. [DOI] [PubMed] [Google Scholar]
  • 7.Sarnak MJ, Jaber BL. Mortality caused by sepsis in patients with end-stage renal disease compared with the general population. Kidney Int. 2000;58(4):1758–1764. [DOI] [PubMed] [Google Scholar]
  • 8.Grohskopf LA, Sokolow LZ, Broder KR, Walter EB, Fry AM, Jernigan DB. Prevention and Control of Seasonal Influenza with Vaccines: Recommendations of the Advisory Committee on Immunization Practices-United States, 2018–19 Influenza Season. MMWR Recomm Rep. 2018;67(3):1–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Eickhoff TC. Immunization against influenza: rationale and recommendations. J Infect Dis. 1971;123(4):446–454. [DOI] [PubMed] [Google Scholar]
  • 10.Licensure of a high-dose inactivated influenza vaccine for persons aged >or=65 years (Fluzone High-Dose) and guidance for use - United States, 2010. MMWR Morb Mortal Wkly Rep. 2010;59(16):485–486. [PubMed] [Google Scholar]
  • 11.McGrath LJ, Layton JB, Krueger WS, Kshirsagar AV, Butler AM. High-dose influenza vaccine use among patients receiving hemodialysis in the United States, 2010–2013. Vaccine. 2018;36(41):6087–6094. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Wilkinson K, Wei Y, Szwajcer A, et al. Efficacy and safety of high-dose influenza vaccine in elderly adults: A systematic review and meta-analysis. Vaccine. 2017;35(21):2775–2780. [DOI] [PubMed] [Google Scholar]
  • 13.DiazGranados CA, Dunning AJ, Kimmel M, et al. Efficacy of high-dose versus standard-dose influenza vaccine in older adults. N Engl J Med. 2014;371(7):635–645. [DOI] [PubMed] [Google Scholar]
  • 14.DiazGranados CA, Dunning AJ, Robertson CA, Talbot HK, Landolfi V, Greenberg DP. Efficacy and immunogenicity of high-dose influenza vaccine in older adults by age, comorbidities, and frailty. Vaccine. 2015;33(36):4565–4571. [DOI] [PubMed] [Google Scholar]
  • 15.Izurieta HS, Thadani N, Shay DK, et al. Comparative effectiveness of high-dose versus standard-dose influenza vaccines in US residents aged 65 years and older from 2012 to 2013 using Medicare data: a retrospective cohort analysis. Lancet Infect Dis. 2015;15(3):293–300. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Shay DK, Chillarige Y, Kelman J, et al. Comparative Effectiveness of High-Dose Versus Standard-Dose Influenza Vaccines Among US Medicare Beneficiaries in Preventing Postinfluenza Deaths During 2012–2013 and 2013–2014. J Infect Dis. 2017;215(4):510–517. [DOI] [PubMed] [Google Scholar]
  • 17.Richardson DM, Medvedeva EL, Roberts CB, Linkin DR. Comparative effectiveness of high-dose versus standard-dose influenza vaccination in community-dwelling veterans. Clin Infect Dis. 2015;61(2):171–176. [DOI] [PubMed] [Google Scholar]
  • 18.Young-Xu Y, Van Aalst R, Mahmud SM, et al. Relative Vaccine Effectiveness of High-Dose Versus Standard-Dose Influenza Vaccines Among Veterans Health Administration Patients. J Infect Dis. 2018;217(11):1718–1727. [DOI] [PubMed] [Google Scholar]
  • 19.DiazGranados CA, Dunning AJ, Jordanov E, Landolfi V, Denis M, Talbot HK. High-dose trivalent influenza vaccine compared to standard dose vaccine in elderly adults: safety, immunogenicity and relative efficacy during the 2009–2010 season. Vaccine. 2013;31(6):861–866. [DOI] [PubMed] [Google Scholar]
  • 20.DiazGranados CA, Robertson CA, Talbot HK, Landolfi V, Dunning AJ, Greenberg DP. Prevention of serious events in adults 65 years of age or older: A comparison between high-dose and standard-dose inactivated influenza vaccines. Vaccine. 2015;33(38):4988–4993. [DOI] [PubMed] [Google Scholar]
  • 21.Miskulin DC, Weiner DE, Tighiouart H, et al. High-Dose Seasonal Influenza Vaccine in Patients Undergoing Dialysis. Clin J Am Soc Nephrol. 2018;13(11):1703–1711. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Saran R, Li Y, Robinson B, et al. US Renal Data System 2015 annual data report: epidemiology of kidney disease in the United States. Am J Kidney Dis. 2016;67(3)(suppl 1):S1–S390. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Update: influenza activity--United States, 2010–11 season, and composition of the 2011–12 influenza vaccine. MMWR Morb Mortal Wkly Rep. 2011;60(21):705–712. [PubMed] [Google Scholar]
  • 24.Update: influenza activity - United States, 2011–12 season and composition of the 2012–13 influenza vaccine. MMWR Morb Mortal Wkly Rep. 2012;61(22):414–420. [PubMed] [Google Scholar]
  • 25.Influenza activity--United States, 2012–13 season and composition of the 2013–14 influenza vaccine. MMWR Morb Mortal Wkly Rep. 2013;62(23):473–479. [PMC free article] [PubMed] [Google Scholar]
  • 26.Epperson S, Blanton L, Kniss K, et al. Influenza activity - United States, 2013–14 season and composition of the 2014–15 influenza vaccines. MMWR Morb Mortal Wkly Rep. 2014;63(22):483–490. [PMC free article] [PubMed] [Google Scholar]
  • 27.Appiah GD, Blanton L, D’Mello T, et al. Influenza activity - United States, 2014–15 season and composition of the 2015–16 influenza vaccine. MMWR Morb Mortal Wkly Rep. 2015;64(21):583–590. [PMC free article] [PubMed] [Google Scholar]
  • 28.Lindsay L, Jackson LA, Savitz DA, et al. Community influenza activity and risk of acute influenza-like illness episodes among healthy unvaccinated pregnant and postpartum women. Am J Epidemiol. 2006;163(9):838–848. [DOI] [PubMed] [Google Scholar]
  • 29.McGrath LJ, Kshirsagar AV, Cole SR, et al. Influenza vaccine effectiveness in patients on hemodialysis: an analysis of a natural experiment. Arch Intern Med. 2012;172(7):548–554. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Cuthbertson CC, Kucharska-Newton A, Faurot KR, et al. Controlling for Frailty in Pharmacoepidemiologic Studies of Older Adults: Validation of an Existing Medicare Claims-based Algorithm. Epidemiology. 2018;29(4):556–561. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Faurot KR, Jonsson Funk M, Pate V, et al. Using claims data to predict dependency in activities of daily living as a proxy for frailty. Pharmacoepidemiol Drug Saf. 2015;24(1):59–66. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Brookhart MA, Patrick AR, Dormuth C, et al. Adherence to lipid-lowering therapy and the use of preventive health services: an investigation of the healthy user effect. Am J Epidemiol. 2007;166(3):348–354. [DOI] [PubMed] [Google Scholar]
  • 33.Shrank WH, Patrick AR, Brookhart MA. Healthy user and related biases in observational studies of preventive interventions: a primer for physicians. J Gen Intern Med. 2011;26(5):546–550. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Lau B, Cole SR, Gange SJ. Competing risk regression models for epidemiologic data. Am J Epidemiol. 2009;170(2):244–256. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Fine J, Gray R. A proportional hazards model for the subdistribution of a competing risk. J Am Stat Assoc. 1999;94(446):496–509. [Google Scholar]
  • 36.Cole SR, Hernan MA. Adjusted survival curves with inverse probability weights. Comput Methods Programs Biomed. 2004;75(1):45–49. [DOI] [PubMed] [Google Scholar]
  • 37.Brookhart MA, Wyss R, Layton JB, Sturmer T. Propensity score methods for confounding control in nonexperimental research. Circ Cardiovasc Qual Outcomes. 2013;6(5):604–611. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Yang D, Dalton JE. A unified approach to measuring the effect size between two groups using SAS®. 2012. [Google Scholar]
  • 39.Ding P, VanderWeele TJ. Sensitivity Analysis Without Assumptions. Epidemiology. 2016;27(3):368–377. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.R Core Team (2014). R: A language and environment for statistical computing, Vienna, Austria: URL http://www.R-project.org/ [Google Scholar]
  • 41.Austin PC. Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Stat Med. 2009;28(25):3083–3107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Centers for Medicare & Medicaid Services. Seasonal Influenza Vaccines Pricing https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Part-B-Drugs/McrPartBDrugAvgSalesPrice/VaccinesPricing.html. Accessed December 12, 2018. [Google Scholar]
  • 43.Saran R, Robinson B, Abbott KC, et al. US Renal Data System 2018 Annual Data Report: epidemiology of kidney disease in the United States. Am J Kidney Dis. 2019;73(3)(suppl 1):SVII-+. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Falsey AR, Treanor JJ, Tornieporth N, Capellan J, Gorse GJ. Randomized, double-blind controlled phase 3 trial comparing the immunogenicity of high-dose and standard-dose influenza vaccine in adults 65 years of age and older. J Infect Dis. 2009;200(2):172–180. [DOI] [PubMed] [Google Scholar]
  • 45.DiazGranados CA, Dunning AJ, Robertson CA, Talbot HK, Landolfi V, Greenberg DP. Effect of Previous-Year Vaccination on the Efficacy, Immunogenicity, and Safety of High-Dose Inactivated Influenza Vaccine in Older Adults. Clin Infect Dis. 2016;62(9):1092–1099. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.DiazGranados CA, Saway W, Gouaux J, et al. Safety and immunogenicity of high-dose trivalent inactivated influenza vaccine in adults 50–64 years of age. Vaccine. 2015;33(51):7188–7193. [DOI] [PubMed] [Google Scholar]
  • 47.Kaka AS, Filice GA, Myllenbeck S, Nichol KL. Comparison of Side Effects of the 2015–2016 High-Dose, Inactivated, Trivalent Influenza Vaccine and Standard Dose, Inactivated, Trivalent Influenza Vaccine in Adults >/=65 Years. Open forum infectious diseases. 2017;4(1):ofx001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Moro PL, Arana J, Cano M, et al. Postlicensure safety surveillance for high-dose trivalent inactivated influenza vaccine in the Vaccine Adverse Event Reporting System, 1 July 2010–31 December 2010. Clin Infect Dis. 2012;54(11):1608–1614. [DOI] [PubMed] [Google Scholar]
  • 49.McGrath LJ, Ellis AR, Brookhart MA. Controlling Time-Dependent Confounding by Health Status and Frailty: Restriction Versus Statistical Adjustment. Am J Epidemiol. 2015;182(1):17–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Vogtlander NP, Brown A, Valentijn RM, Rimmelzwaan GF, Osterhaus AD. Impaired response rates, but satisfying protection rates to influenza vaccination in dialysis patients. Vaccine. 2004;22(17–18):2199–2201. [DOI] [PubMed] [Google Scholar]
  • 51.Scharpe J, Peetermans WE, Vanwalleghem J, et al. Immunogenicity of a standard trivalent influenza vaccine in patients on long-term hemodialysis: an open-label trial. Am J Kidney Dis. 2009;54(1):77–85. [DOI] [PubMed] [Google Scholar]
  • 52.Kunisaki KM, Janoff EN. Influenza in immunosuppressed populations: a review of infection frequency, morbidity, mortality, and vaccine responses. Lancet Infect Dis. 2009;9(8):493–504. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Brydak LB, Roszkowska-Blaim M, Machala M, Leszczynska B, Sieniawska M. Antibody response to influenza immunization in two consecutive epidemic seasons in patients with renal diseases. Vaccine. 2000;18(28):3280–3286. [DOI] [PubMed] [Google Scholar]
  • 54.Evison J, Farese S, Seitz M, Uehlinger DE, Furrer H, Muhlemann K. Randomized, double-blind comparative trial of subunit and virosomal influenza vaccines for immunocompromised patients. Clin Infect Dis. 2009;48(10):1402–1412. [DOI] [PubMed] [Google Scholar]
  • 55.Broeders NE, Hombrouck A, Lemy A, et al. Influenza A/H1N1 vaccine in patients treated by kidney transplant or dialysis: a cohort study. Clin J Am Soc Nephrol. 2011;6(11):2573–2578. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Dikow R, Eckerle I, Ksoll-Rudek D, et al. Immunogenicity and efficacy in hemodialysis patients of an AS03(A)-adjuvanted vaccine for 2009 pandemic influenza A(H1N1): a nonrandomized trial. Am J Kidney Dis. 2011;57(5):716–723. [DOI] [PubMed] [Google Scholar]
  • 57.Jackson LA. Benefits of examining influenza vaccine associations outside of influenza season. Am J Respir Crit Care Med. 2008;178(5):439–440. [DOI] [PubMed] [Google Scholar]
  • 58.Jackson LA, Jackson ML, Nelson JC, Neuzil KM, Weiss NS. Evidence of bias in estimates of influenza vaccine effectiveness in seniors. Int J Epidemiol. 2006;35(2):337–344. [DOI] [PubMed] [Google Scholar]
  • 59.Jackson LA, Nelson JC, Benson P, et al. Functional status is a confounder of the association of influenza vaccine and risk of all cause mortality in seniors. Int J Epidemiol. 2006;35(2):345–352. [DOI] [PubMed] [Google Scholar]
  • 60.Jackson ML, Yu O, Nelson JC, et al. Further evidence for bias in observational studies of influenza vaccine effectiveness: the 2009 influenza A(H1N1) pandemic. Am J Epidemiol. 2013;178(8):1327–1336. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Butler AM, Layton JB, Krueger WS, Kshirsagar AV, LJ M. Assessing Residual Bias in Estimating Influenza Vaccine Effectiveness: Comparison of High-dose versus Standard Dose Vaccines. Med Care. 2019;57(1):73–78. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Centers for Disease Control and Prevention. Vaccine Effectiveness - How Well Does the Flu Vaccine Work? https://www.cdc.gov/flu/about/qa/vaccineeffect.htm. Accessed December 12, 2018.
  • 63.Biggerstaff M, Kniss K, Jernigan DB, et al. Systematic Assessment of Multiple Routine and Near Real-Time Indicators to Classify the Severity of Influenza Seasons and Pandemics in the United States, 2003–2004 Through 2015–2016. Am J Epidemiol. 2018;187(5):1040–1050. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Allen JD, Ross TM. H3N2 influenza viruses in humans: Viral mechanisms, evolution, and evaluation. Hum Vaccin Immunother. 2018;14(8):1840–1847. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1

RESOURCES