Depletion-of-susceptibles Bias in Analyses of Intra-season Waning of Influenza Vaccine Effectiveness

Abstract

Bias arises in studies of waning vaccine effectiveness when higher-risk individuals are depleted from the at-risk population at different rates between study groups. We examined how this bias arises and how to avoid it. A reanalysis of data from California confirmed a finding of intra-season waning of influenza vaccine effectiveness.

A potential bias in studies of intra-season waning of influenza vaccine effectiveness (VE) arises from the depletion of “susceptibles.” This bias arises when relatively high-risk individuals—those susceptible to illness—are depleted from the population still at risk, and the amount of depletion differs between the groups or times being compared [1–5].

A recent study of vaccinated persons in Northern California found the odds of influenza increased by 16% every 28 days following an individual’s vaccination [6], implying that VE wanes with time since vaccination. A commentary [7] and subsequent Letter to the Editor [8] suggested this finding could have been biased by differential depletion of susceptibles.

We examin how depletion-of-susceptibles bias can arise and how it can be avoided. Although this bias can arise in studies of many types of exposures and outcomes, we focus on studies of waning influenza VE. We reexamined the data from Northern California and show why the apparent waning of VE there cannot be attributed to a depletion-of-susceptibles bias.

HOW DEPLETION-OF-SUSCEPTIBLES BIAS ARISES

In VE studies, this bias can arise when: (1) the vaccine is effective but not 100% effective; (2) a subset of the population is more susceptible to infection or being identified as infected than the rest of the population (“susceptibles”); and (3) persons drop out of the risk pool once they are infected, either because infection confers immunity or the research question only pertains to a first infection. “Susceptibility” summarizes all person-level factors—other than vaccination status—that increase a person’s risk of infection. We treat susceptibility as dichotomous for simplicity, but the bias arises in the same way if susceptibility is complex and multidimensional.

Once influenza begins to circulate, susceptibles in the unvaccinated group are more likely than susceptibles in the vaccinated group to get infected and drop out of the risk pool (because vaccinated susceptibles have some protection from the vaccine). As the season progresses, the proportion of the unvaccinated who are susceptible shrinks faster than does the proportion of the vaccinees who are susceptible. Thus, the rate of influenza in the unvaccinated pool becomes closer to the rate in the vaccinated pool. Unless we adjust for susceptibility [4], the differential depletion of susceptibles can make VE appear to wane when there is no real waning. Adjustment for susceptibility is a way to avoid depletion-of-susceptibles bias, but is not feasible if aspects of susceptibility are unknown or unmeasured.

Another way to avoid this bias is available to studies of influenza vaccination, because influenza mainly circulates during only part of the year—typically mid-December to March—after most influenza vaccinations have been given. Until the influenza virus starts to circulate, no individual can drop out of the risk pool due to infection. Consequently, there is no depletion of susceptibles (and no differential depletion of susceptibles) prior to the start of the influenza season. Below, we contrast 2 hypothetical randomized controlled trials (RCTs), 1 of which is designed to avoid depletion-of-susceptibles bias by exploiting variation across vaccinees in the amount of time there is for vaccine protection to wear off (wane) before influenza starts circulating.

HYPOTHETICAL RANDOMIZED CONTROLLED TRIALS

Consider 2 hypothetical RCTs (Figure 1). Both are conducted in settings where influenza circulates from mid-December through March and susceptibility is unmeasured; both minimize other types of biases by virtue of randomization, complete follow-up, and valid outcome ascertainment.

Figure 1.

Hypothetical randomized controlled trials illustrating approaches to estimated waning of influenza vaccine effectiveness. Abbreviation: RCT, randomized controlled trial; RR, relative risk; VE, vaccine effectiveness.

Trial 1 Randomizes Vaccination Status

From August through November, enrollees consent to randomization to be vaccinated or unvaccinated. After influenza begins to circulate, RCT1 estimates the rate ratio (RR; incidence among vaccinated, divided by incidence among unvaccinated) each week and tests whether VE wanes using a test of the trend in the RR.

Trial 2 Randomizes Vaccination Date

In July, enrollees consent to be vaccinated during a randomly assigned week from August through November. RCT2 estimates a relative RR (rRR) that compares the incidence among the more remotely vaccinated to the incidence among the more recently vaccinated (where “remotely” refers to vaccinations given earlier: for example, in August rather than in November).

What Trial 1 Would Find

First consider what RCT1 would find if VE is 75% and does not wane at all: where, regardless of susceptibility, every vaccinee not yet infected has a 75% lower risk of infection every day than if they were unvaccinated. Susceptibles in the unvaccinated arm would get influenza and drop out 4 times as fast as susceptibles in the vaccinated arm. As the epidemic progresses, measured VE would decline, but RCT1 would not be able to distinguish whether the decrease is due to a waning of VE or a depletion of susceptibles.

Now consider a scenario where VE really does wane: where VE peaks soon after vaccination and wanes steadily after that. Just as in the no-waning scenario, the unvaccinated arm loses more of its susceptibles than does the vaccinated arm. In both scenarios, depletion reduces the denominator (incidence in the unvaccinated) of the RR more than the numerator (incidence in the vaccinated); thus, waning is overestimated.

What Trial 2 Would Find

If RCT2 is done in a no-waning scenario, the depletion of susceptibles is not differential and not a source of bias, because the August and September vaccinees would always be just as protected as the October and November vaccinees. RCT2 would find that influenza incidence during each week of the season is unrelated to the time since vaccination, and would correctly conclude that VE does not wane.

Now let’s consider what happens in RCT2 in a scenario where there is real waning of VE, such that some of the vaccine’s protection has worn off in the August vaccinees by mid-December, when influenza starts circulating. Right from the start of influenza season, incidence rates would be higher in August vaccinees than in November vaccinees. From this, we would correctly infer that VE wanes. As the cumulative incidence of influenza increases during the season, the less-protected August vaccinees would get more depleted of their susceptibles than would the better-protected November vaccinees. In this scenario, the depletion of RCT2’s susceptibles would reduce the numerator (incidence in August vaccinees) of the rRR more than the denominator (incidence in November vaccinees), with the result being that waning would tend to be underestimated.

Thus, in a scenario where VE really wanes, RCT2 would permit an unbiased estimate of waning early in the epidemic (so long as the cumulative incidence is low enough that depletion is negligible). Later in the epidemic, as the cumulative incidence increases, RCT2 would tend to understate waning rather than exaggerate it. (See Supplementary Tables 1 and 2 for numeric examples of these hypothetical RCTs.)

REANALYSIS OF VACCINE EFFECTIVENESS WANING IN PRIOR STUDY

The study from Northern California [6]—like some other recent studies of waning VE [9–14] —used observational data in a way that emulated RCT2. Like RCT2, this study included only vaccinees, and the influenza risk was examined in relation to the time since vaccination. As described above, if all persons in the analysis had been vaccinated before influenza began to circulate, any waning of VE in this study design could not have been exaggerated by depletion of susceptibles. But the inclusion of persons vaccinated during influenza season could have biased the analysis toward an overestimation of waning. Therefore, we excluded the 9% of the study population vaccinated after 1 December (earlier than the start of any influenza season in the study), and then reanalyzed the data. Our reanalysis yielded results similar to those in the published report: the risk of influenza was related to the time since vaccination (P < .0001); and the odds of influenza among vaccinees increased by 18%, on average, for each additional 28 days after vaccination (odds ratio = 1.18, 95% confidence interval 1.12–1.23).

We note 3 caveats regarding our discussion of how to avoid depletion-of-susceptibles bias. First, a vaccinee-only study can only estimate a change in relative VE, rather than VE. An unvaccinated reference group could be added to both RCT2 and an observational, vaccinee-only study, but doing so would pose challenges: it would be unethical for RCT2 to withhold vaccination and, in an observational study, vaccinated persons are likely to differ from unvaccinated persons in difficult-to-measure ways that are even greater than those between early and late vaccinees. Second, for some research questions, the depletion of susceptibles might be a relevant part of the target effect that we want to estimate, rather than a source of bias, but it is a source of bias when our research question pertains to the optimal time for vaccination. Third, while the emulation of RCT2 can protect an observational study from a depletion-of-susceptibles bias, it could still be vulnerable to other sources of bias.

CONCLUSION

Differential depletion of susceptibles can make VE appear to wane during the influenza season even if there is no waning. A reanalysis of recent data from California avoided this bias and confirmed a previous finding of intra-season waning of influenza VE. The bias was avoided by focusing on people who were vaccinated before influenza was circulating and then examining their influenza risk in relation to the time since vaccination. If there is no true waning of VE, this study design is not vulnerable to depletion-of-susceptibles bias. If there is true waning, this study design might underestimate the amount of waning, but would not exaggerate it.

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Notes

Disclaimer. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of General Medical Sciences or the National Institutes of Health. The sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or the decision to submit the manuscript for publication.

Financial support. This work was supported by the National Institute of Allergy and Infectious Diseases, National Institutes of Health (grant #1R01AI107721-01); the National Institute of General Medical Sciences (cooperative agreement U54GM088558), and the Permanente Medical Group.

Potential conflicts of interest. G. T. R. reports research support from Pfizer. N. P. K. reports research support from Sanofi Pasteur, GlaxoSmithKline, Protein Science, MedImmune, Pfizer, Merck, and Dynavax. M. L. reports support from Merck, Pfizer, Antigen Discovery, and Affinivax. All other authors report no potential conflicts. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.