Effects of Confounding Bias in Coronavirus Disease 2019 (COVID-19) and Influenza Vaccine Effectiveness Test-Negative Designs Due to Correlated Influenza and COVID-19 Vaccination Behaviors

Abstract

Background

The test-negative design is commonly used to estimate influenza and coronavirus disease 2019 (COVID-19) vaccine effectiveness (VE). In these studies, correlated COVID-19 and influenza vaccine behaviors may introduce a confounding bias where controls are included with the other vaccine-preventable acute respiratory illness (ARI). We quantified the impact of this bias on VE estimates in studies where this bias is not addressed.

Methods

We simulated study populations under varying vaccination probabilities, COVID-19 VE, influenza VE, and proportions of controls included with the other vaccine-preventable ARI. Mean bias was calculated as the difference between estimated and true VE. Absolute mean bias in VE estimates was classified as low (<10%), moderate (10% to <20%), and high (≥20%).

Results

Where vaccination probabilities are positively correlated, COVID-19 and influenza VE test-negative studies with influenza and severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) ARI controls, respectively, underestimate VE. For COVID-19 VE studies, mean bias was low for all scenarios where influenza represented ≤25% of controls. For influenza VE studies, mean bias was low for all scenarios where SARS-CoV-2 represented ≤10% of controls. Although bias was driven by the conditional probability of vaccination, low VE of the vaccine of interest and high VE of the confounding vaccine increase its magnitude.

Conclusions

Where a low percentage of controls is included with the other vaccine-preventable ARI, bias in COVID-19 and influenza VE estimates is low. However, influenza VE estimates are likely more susceptible to bias. Researchers should consider potential bias and its implications in their respective study settings to make informed methodological decisions in test-negative VE studies.

Correlated influenza and coronavirus disease 2019 (COVID-19) vaccination behaviors can bias influenza and COVID-19 vaccine effectiveness (VE) estimates from test-negative studies. Bias depends upon the proportion of controls with the other vaccine-preventable disease, correlation between vaccines, and true VE of both vaccines.

Phase 4 observational studies are essential to examine the direct effects of vaccination in a real-world setting. Due to its relative simplicity, the test-negative study design is the predominant observational design to estimate vaccine effectiveness (VE) for influenza [1], and increasingly, coronavirus disease 2019 (COVID-19) [2–4]. In these studies, test-negative participants are persons who seek healthcare for an acute respiratory illness (ARI) and are tested for the disease of interest. Participants who test positive are classified as “cases,” while participants who test negative are “controls.” VE is estimated using the formula (1 – odds ratio) × 100, where the odds ratio (OR) compares the vaccination odds between cases and controls [5].

Similar to other case-control studies, controls in the test-negative design are used as a proxy to estimate the true vaccination odds in the source population of cases [5, 6]. Since COVID-19 and influenza VE test-negative designs select ARI controls who are negative for the disease of interest, a foundational design assumption is that the risks of alternative causes of ARI are independent of exposure status (ie, vaccination) [7]. Where this assumption is violated, VE estimates are biased unless independence is established by deconfounding.

To date, work examining this assumption in influenza test-negative VE studies has focused on direct, biological mechanisms by which influenza and/or vaccination influences the risk of alternate ARI causes [8, 9]. However, a relationship between influenza vaccination and alternate ARI need not be causally related to violate this assumption; violation can also occur due to a relationship established by an indirect, confounding pathway [10].

Recent systematic reviews and surveys among healthcare workers and the general population have demonstrated a positive correlation between influenza and COVID-19 vaccination probabilities, with persons receiving an influenza vaccination 3 times more likely to accept COVID-19 vaccination [11–14]. Because of this relationship, the risks of influenza and severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) are no longer independent of the vaccination probabilities for the other vaccine-preventable ARI (ie, COVID-19 and influenza vaccination, respectively). Therefore, where test-negative controls in either influenza or COVID-19 VE studies include persons with the other vaccine-preventable ARI, vaccination for these diseases acts as a confounder. Where this confounder is unaccounted for, the fundamental assumption of exposure independence in control selection is violated, leading to biased VE estimates.

Since SARS-CoV-2 and influenza are likely to co-circulate in upcoming influenza seasons and test-negative VE results may be used to inform vaccination policies, it is important to understand the scope and magnitude of this confounding bias on COVID-19 and influenza VE estimates. Here, we aim to contribute to this knowledge by examining its theoretical basis and deconfounding methods to remove bias, and quantifying bias in COVID-19 and influenza VE estimates using simulations, where this bias is not otherwise addressed.

METHODS

Theoretical Basis

In Figure 1A, we examine the theoretical basis for bias from inclusion of controls with a nonindependent exposure in COVID-19 or influenza VE test-negative designs. In the directed acyclic graph, COVID-19 vaccination ($V_{covid}$) 
and influenza vaccination ($V_{flu}$) are related to a common ancestor we refer to as an individual’s motivation to seek vaccination ($M$). Here, (M) represents a set of unobserved variables, including beliefs and acceptance of vaccines; external vaccination pressures that influence the uptake of both vaccines, such as vaccine mandates or policies; and perceived vulnerability/risks of vaccine-preventable diseases to oneself or vulnerable contacts. Through this common ancestor, a correlation is established between influenza and COVID-19 vaccination probabilities through the pathway $V_{flu}\leftarrow (M)\to V_{covid}$, which extends to their causal descendants, including infection ($I$), creating the confounding pathways (i) $V_{flu}\leftarrow \left(M\right)\to I_{SARS-CoV-2}$, violating the assumption of exposure independence in influenza VE test-negative designs with SARS-CoV-2 controls; and (ii) $V_{covid}\leftarrow \left(M\right)\to I_{flu}$, 
violating this assumption in COVID-19 VE test-negative designs with influenza controls.

Figure 1.

A, Simplified directed acyclic graph illustrating the relationship between coronavirus disease 2019 (COVID-19) and influenza vaccination probabilities. Vaccination motivation $M$is a common ancestor of influenza vaccine uptake $V_{flu}$ and COVID-19 vaccine uptake $V_{covid}$. The parentheses indicate that $M$is unmeasured. Through $M$a forked, confounding pathway exists linking $V_{covid}$ to medically attended influenza acute respiratory illness (ARI) $I_{flu}$ ($V_{covid}\leftarrow (M)\to I_{flu})$, 
and $V_{flu}$ to medically attended severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) ARI $I_{SARS-CoV-2}$ ($V_{flu}\leftarrow (M)\to I_{SARS-CoV-2})$. B, Adjustment in the statistical model for $V_{flu}$ closes the confounding pathway from $V_{covid}\leftarrow (M)\to I_{flu}$ in COVID-19 vaccine effectiveness (VE) test-negative studies that include influenza controls. C, Similarly, adjustment for $V_{covid}$ in an influenza VE test-negative study that includes SARS-CoV-2 controls closes the confounding pathway from $V_{flu}\leftarrow (M)\to I_{SARS-CoV-2}$.

In Figure 2, we demonstrate how these relationships lead to bias in VE estimates. Where test-negative controls with the alternate, vaccine-preventable ARI are included, unvaccinated persons are overrepresented among controls, leading to a higher $\mathrm{O}\mathrm{R}$ and lower VE estimates.

Figure 2.

Exploration of bias in coronavirus disease 2019 (COVID-19) test-negative vaccine effectiveness (VE) studies that include influenza controls (A) and influenza test-negative VE studies that include severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) controls (B). Under the 2 assumptions that (i) influenza and COVID-19 vaccinations are protective against their respective diseases, and (ii) influenza and COVID-19 vaccination behaviors are positively correlated, inclusion of controls with the other vaccine-preventable acute respiratory illness will overrepresent unvaccinated controls ($d$). Overrepresentation of ($d$) increases the odds ratio (OR) comparing vaccination odds between cases and controls, and underestimates true VE given the formula VE = (1 – OR) × 100.

Mitigation of Bias in VE Estimates

Given this causal structure, we propose 2 options to mitigate confounding bias in COVID-19 and influenza test-negative designs, where co-circulation of SARS-CoV-2 and influenza is present: (i) deconfounding in the analysis, or (ii) deconfounding in the study design. Alternatively, a third option is to ignore bias if it is small and not meaningful to VE estimates. We address options (i) and (ii) in the following section; in subsequent sections, we quantify bias using simulations to understand the implications of option (iii).

(i) Deconfounding in the analysis: In Figure 1B, we demonstrate how exposure independence can be restored in a COVID-19 VE study by statistical adjustment or stratification for influenza vaccination. Similarly, in Figure 1C, we demonstrate how statistical adjustment for COVID-19 vaccination restores the validity of this assumption in influenza VE studies. Although these mechanisms can recover unbiased estimates of VE, they may interfere with Wacholder and colleagues’ “efficiency principle” in case-control designs [15]. Where $V_{covid}$ and $V_{flu}$ are highly correlated, statistical adjustment for the confounder $V_{covid}$ in an influenza VE study or $V_{flu}$ in a COVID-19 VE study also reduces conditional variability of the exposure in the strata of the confounder [15]. Therefore, although adjustment removes bias, precision of VE estimates may be reduced [15]. This may be of particular concern in studies that explore subpopulation VE or waning VE, which require additional statistical power.

(ii) Deconfounding in the design: Deconfounding may also be achieved in the design via study restriction. To avoid violation of the efficiency principle here [15], ample controls must exist who are independent of the exposure probability. In a test-negative design, restriction may be implemented by excluding influenza controls from COVID-19 test-negative VE designs or SARS-CoV-2 controls from influenza test-negative VE designs. Practically, these are achieved by testing and only enrolling controls who test negative for both diseases. Another alternative restriction method is to enroll controls who test positive for a different ARI cause, presumed independent of the exposure probability.

Simulations to Quantify Bias in VE Estimates

Yet, a third option is to ignore VE bias if it is anticipated to be small and not meaningful (ie, option iii). However, it is important to understand the magnitude of confounding bias to make this determination. We used simulated adult populations where COVID-19 and influenza vaccination were positively correlated to estimate mean bias in COVID-19 and influenza VE estimates. While simulations were performed for each disease separately, the following input parameters were included in both analyses:

Cov _flu = influenza vaccination coverage

Cov _covid = COVID-19 vaccination coverage

IP _flu = incidence proportion (risk) of medically attended influenza ARI in the unvaccinated population during the study period

IP _covid = incidence proportion (risk) of medically attended COVID-19 ARI in the unvaccinated population during the study period

VE _flu = true influenza VE to prevent medically attended influenza ARI

VE _covid = true COVID-19 VE to prevent medically attended COVID-19 ARI

P _controls = proportion of ARI controls who represent the alternate vaccine-preventable ARI

For all simulations, we assumed Cov_flu = 55% and Cov_covid = 70%, which approximates 2020–2021 influenza vaccination coverage and 2-dose COVID-19 vaccine coverage among US adults at the time of our analyses [16, 17]. We also assumed $I{P}_{flu}=5\mathrm{\%}$ and $I{P}_{covid}=5\mathrm{\%}$, based upon previous simulations investigating bias in test-negative designs [1]. We examined 
3 scenarios for $V{E}_{flu}$of 40%, 50%, and 60% against medically attended influenza ARI, consistent with $V{E}_{flu}$ data from recent influenza seasons [18]. Furthermore, for $V{E}_{covid}$, we examined 3 scenarios of 40%, 65%, and 90% effectiveness against medically attended SARS-CoV-2 ARI to encompass recent 2-dose COVID-19 VE estimates in Delta- and Omicron-predominant periods [4, 19–23]. For each scenario, we examined a range for $P_{controls}$ of 0, 0.1, 0.25, 0.5, 0.75, 0.9, and 1.0 to explore the maximum bias and underlying shape of the bias curve.

For COVID-19 VE studies, we provide several examples of bias estimates assuming$P_{controls}=0.3$; this level was selected considering the proportion of influenza cases identified among ARI participants in historical influenza test-negative VE studies, which used similar ARI definitions as recent COVID-19 studies [4, 24, 25].

We also included an input variable representing the conditional probability of vaccination given vaccination with the other ARI vaccine. Specifically, $R{R}_{Covidvx|fluvx}$ was used to simulate bias in COVID-19 VE studies, and $R{R}_{fluvx|covidvx}$ was used to simulate bias in influenza VE studies. These variables were defined as the following:

$R{R}_{covidvx|fluvx}=$ risk ratio comparing the uptake (risk) of COVID-19 vaccine between persons who did and did not receive influenza vaccination

$R{R}_{fluvx|covidvx}=$ risk ratio comparing the uptake (risk) of flu vaccine between persons who did and did not receive a COVID-19 vaccination

We used a range of input values for $R{R}_{covidvx|fluvx}$ of 1.5, 2.0, and 3.0; a range of 2.0, 5.0, and 8.0 was used for $R{R}_{fluvx|covidvx}$. 
With the exception of $R{R}_{covidvx|fluvx}=3.0,$ estimates were based on conditional probabilities of influenza and COVID-19 vaccination from a recent, nationally representative survey of US adults sponsored by the Centers for Disease Control and Prevention (Supplementary Appendix 1) [14]. A value of $R{R}_{covidvx|fluvx}=3.0$ was selected to supplement survey data because it represented the upper limit for this value based on input values of $Co{v}_{flu}$= 55% and $Co{v}_{covid}$ = 70%. Since we assumed $Co{v}_{flu}$ was lower than $Co{v}_{covid}$, a reasonable upper limit of $R{R}_{fluvx|covidvx}$ could not be estimated. Table 1 contains information on all input values for simulations.

Table 1.

Input Parameter Values for Simulations

Parameter	Description	Values	Reference(s)
$Co{v}_{flu}$	Flu vaccination coverage	55%	[16]
$Co{v}_{covid}$	COVID-19 vaccination coverage	70%	[17]
$I{P}_{flu}$	Incidence proportion of medically attended influenza ARI among persons unvaccinated for influenza	5%	Consistent with previous test-negative design simulations [1]
$I{P}_{covid}$	Incidence proportion of medically attended COVID-19 ARI among persons unvaccinated for COVID-19	5%	Consistent with previous test-negative design simulations [1]
$V{E}_{flu}$	Influenza vaccine effectiveness	40%, 50%, 60%	[18]
$V{E}_{covid}$	COVID-19 vaccine effectiveness	40%, 65%, 90%	[4, 19–23]
$P_{controls}$	Proportion of controls who represent the alternate vaccine-preventable ARI	0, 0.1, 0.25, 0.5, 0.75, 0.9, 1.0	NA—full range to examine maximum bias and shape of bias curve
$R{R}_{covidvx|fluvx}$	Risk ratio comparing the uptake of COVID-19 vaccination among persons receiving and not receiving influenza vaccination	1.5, 2.0, 3.0	Estimated based on [14], except for 3.0 (max value based on coverage estimates)
$R{R}_{fluvx|covidvx}$	Risk ratio comparing the uptake of influenza vaccination among persons receiving and not receiving COVID-19 vaccination	2.0, 5.0, 8.0	Estimated based on [14]

Where multiple values are specified, populations were simulated for each value separately.

Abbreviations: ARI, acute respiratory illness; COVID-19, coronavirus disease 2019; NA, not applicable.

Simulated populations of COVID-19 and influenza test-negative studies were created using 3 sequential steps that differed slightly for each disease. Specifically, to explore bias in COVID-19 VE test-negative designs, we simulated the (i) marginal probabilities of COVID-19 and influenza vaccine uptake in the source population, given $Co{v}_{flu}$, $Co{v}_{covid}$, and $R{R}_{covidvx|fluvx}$; 
(ii) odds of COVID-19 vaccination among SARS-CoV-2 cases, given $I{P}_{covid}$, $V{E}_{covid}$, and $Co{v}_{covidvx}$; and (iii) odds of COVID-19 vaccination among influenza controls, given the marginal probabilities from step (i), $I{P}_{flu}$, $V{E}_{flu}$, and $P_{controls}$. To explore bias in influenza VE test-negative designs, we modified the 3 steps to simulation of the (i) marginal probabilities of COVID-19 and influenza vaccination in the source population, given $Co{v}_{flu}$, $Co{v}_{covid}$, and $R{R}_{fluvx|covidvx}$; (ii) odds of influenza vaccination among influenza cases, given $I{P}_{flu}$, $V{E}_{flu}$, 
and $Co{v}_{fluvx}$; and (iii) odds of influenza vaccination among SARS-CoV-2 controls, given the marginal probabilities simulated in step (i), $I{P}_{covid}$, $V{E}_{covid}$, and $P_{controls}$.

For each scenario, we performed 10 000 simulations with a population of 200 000 subjects. Based on $I{P}_{covid}$, this starting population approximated the number of cases in COVID-19 test-negative VE studies [4, 20]. From the simulated populations we estimated $\widehat{VE}$ as (1 – OR), where OR compares the vaccination odds among cases (step ii) and controls (step iii). Mean bias in VE test-negative studies was estimated as the difference of $(\widehat{VE}-VE)$, or the difference between observed and true VE; 95% confidence intervals (CIs) were calculated as the 2.5th and 97.5th quantiles of the simulated data. We used a priori thresholds to classify absolute mean bias of <10% as low, 10% to <20% as moderate, and ≥20% as high. For all parameters with a range of plausible input values, separate populations were simulated for each scenario.

To examine bias associated with only the inclusion of controls with the other vaccine-preventable ARI, we ignored other sources of bias arising from misclassification, unmeasured confounding, and selection bias. Similarly, we did not consider situations of influenza and SARS-CoV-2 coinfection. All analyses were conducted using RStudio with R version 4.1.0 (R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

Bias in COVID-19 VE Estimates

Figure 3 examines mean bias in COVID-19 VE estimates under varying levels of $R{R}_{covidvx|fluvx}$,$V{E}_{flu}$, $V{E}_{covid}$, and $P_{controls}$. In all scenarios, inclusion of influenza controls underestimated true COVID-19 VE. In general, there was greater bias in VE estimates with increasing $R{R}_{covidvx|fluvx}$, lower $V{E}_{covid},$ higher $V{E}_{flu}$, and increasing influenza $P_{controls}$. As an example, in a study with $P_{controls}=0.3$, $V{E}_{covid}=40\mathrm{\%}$, and $V{E}_{flu}=60\mathrm{\%}$, where $R{R}_{covidvx|fluvx}$ was changed from 3.0 to 1.5, ${\widehat{VE}}_{covid}$ was 9.7% (95% CI: 5.5%–14.7%) and 4.7% (95% CI: .9%–8.8%) lower, respectively, than $V{E}_{covid}$, with ${\widehat{VE}}_{covid}$ of 30.3% (95% CI: 25.3%–34.5%) and 35.3% (95% CI: 31.2%–39.1%), respectively. Similarly, using the above scenario where $R{R}_{covidvx|fluvx}=3.0$ and $V{E}_{covid}$ was modified from 40% to 90%, ${\widehat{VE}}_{covid}$ was only 1.7% (95% CI: .7%–2.7%) lower than $V{E}_{covid}$, or 88.3% (95% CI: 87.3%–89.3%). In all scenarios, where $P_{controls}$ with influenza represented ≤25% of the control populations, mean bias was low. Bias was also low for all scenarios where $V{E}_{covid}=90\mathrm{\%}$. Moderate to high bias in COVID-19 VE estimates was observed in some scenarios where influenza $P_{controls}$ approached 50%.

Figure 3.

Mean bias and 95% confidence intervals in coronavirus disease 2019 vaccine effectiveness estimates derived from a test-negative study with influenza controls under varying scenarios of $R{R}_{covidvx|fluvx}$, $V{E}_{flu}$, $V{E}_{covid}$, and $P_{controls}$. Abbreviations: COVID-19, coronavirus disease 2019; RR, risk ratio; VE, vaccine effectiveness.

Bias in Influenza VE Estimates

In all scenarios, inclusion of SARS-CoV-2 controls underestimated influenza VE (Figure 4). Patterns of bias in influenza VE estimates were similar to those in COVID-19 studies, where greater bias was observed with lower values of $V{E}_{flu}$ (ie, the vaccine of interest), higher $V{E}_{covid}$ (ie, the confounding vaccination), increasing $R{R}_{fluvx|covidvx}$, and increasing $P_{controls}$. While bias was low for all scenarios with ≤10% of SARS-CoV-2 $P_{controls}$, there was moderate bias in some scenarios where SARS-CoV-2 $P_{controls}$ approached 25%. High bias was observed for several scenarios with 50% $P_{controls}$of SARS-CoV-2, and nearly all scenarios had moderate to high bias with ≥75% $P_{controls}$ of SARS-CoV-2.

Figure 4.

Mean bias and 95% confidence intervals in influenza vaccine effectiveness estimates derived from a test-negative study with severe acute respiratory syndrome coronavirus 2 controls under varying scenarios of $R{R}_{fluvx|covidvx}$, $V{E}_{flu}$, $V{E}_{covid}$, and $P_{controls}$. Abbreviations: COVID-19, coronavirus disease 2019; RR, risk ratio; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2; VE, vaccine effectiveness.

DISCUSSION

In this article, we provide the theoretical basis and quantification of confounding bias in COVID-19 and influenza VE test-negative designs related to the inclusion of influenza and SARS-CoV-2 controls, respectively. While positive correlation in the uptake of influenza and COVID-19 vaccination consistently led to VE underestimation, there was minimal bias in scenarios with low percentages of controls with the other vaccine-preventable ARI. Specifically, bias in COVID-19 VE estimates was low for scenarios with ≤25% of influenza controls and for influenza VE in scenarios with ≤10% of SARS-CoV-2 controls. Where controls with the other vaccine-preventable ARI exceeded these levels, moderate to high bias in VE estimates can occur.

Although confounding in influenza and COVID-19 VE estimates is driven by correlated vaccine behaviors, we found that the magnitude of bias was highly dependent upon true VE of the vaccine of interest and the confounding vaccination. In general, where true VE was high for the vaccine of interest and low for the confounding vaccine, there was less bias in VE estimates. These principles may be generalized to other vaccine-preventable ARI etiologies as well. For example, inclusion of ARI controls with vaccine-preventable serotypes of Streptococcus pneumoniae may similarly bias COVID-19 and influenza test-negative VE estimates, where probabilities of pneumococcal vaccination and the vaccine of interest are correlated.

In general, we found greater bias in influenza test-negative VE studies in comparison with COVID-19. This is because, in most scenarios examined, true influenza VE was lower than true COVID-19 VE [4, 18–22], which exacerbates bias arising from a greater conditional probability of vaccination. Particularly in a COVID-19 pandemic setting, where ARI controls may be more likely to have SARS-CoV-2, these relationships suggest researchers should consider deconfounding to avoid meaningful bias in influenza test-negative VE estimates.

While our findings may be viewed as reassuring regarding COVID-19 test-negative VE studies, these results are subject to several important limitations. First, we caution that COVID-19 test-negative VE study populations likely include subgroups for whom the conditional probability of vaccination differs. Where this occurs, bias in VE estimates also varies. For example, older persons or persons who are at higher risk of severe disease may have a higher conditional probability of vaccination in comparison with younger or healthy persons. Using the parameters we specified in our results, if we assume a conditional probability of vaccination of 3.0 among older persons vs 1.5 among younger persons, COVID-19 VE would be estimated as only 30.3% (95% CI: 25.3%–34.5%) among older populations vs 35.3% (95% CI: 31.2%–39.1%) among younger persons, when true VE for both populations is 40%. Similarly, regional variation in COVID-19 and influenza vaccination coverage arising from health inequities or common attitudes toward vaccinations may also affect the conditional probability of vaccination [16, 17]. Additionally, even where the conditional probability of vaccination is similar, we found that bias varies by true VE. In the case of COVID-19, where true VE likely varies by vaccine product, SARS-CoV-2 strain, or time since vaccination [4, 20, 23], bias in VE estimates will also differ. For example, if strain-specific $V{E}_{covid}$ is 90% vs 40%, using the parameters we applied in our results section, COVID-19 VE would be estimated as 88.3% (95% CI: 87.3%–89.3%) for a strain where true VE is 90% vs only 30.3% (95% CI: 25.3%–34.5%) for a strain where true VE is 40%. Similarly, bias in COVID-19 VE estimates may also differ by outcome, such as symptomatic disease vs hospitalization. Collectively, these examples highlight the importance of deconfounding to promote comparability in VE estimates, even in situations where bias is low.

It is important to acknowledge a deconfounding requirement may impact the feasibility of a test-negative study, that is, a design commonly implemented using administrative data [2, 4]. A challenge of deconfounding in the analysis is that it requires measurement of both vaccinations, which may not be reliably recorded in a vaccine registry. Furthermore, deconfounding by study design requires additional costs associated with testing for other pathogen(s), unless this testing is routinely performed. This may not be possible in current limited-resource settings or in the future, if other causes of ARI, such as respiratory syncytial virus, become vaccine preventable. Furthermore, since VE may differ due to health inequities, the systematic exclusion of these settings is problematic. However, where additional efforts are made to implement deconfounding, an advantage is that influenza and COVID-19 test-negative VE studies can be run in parallel without much additional effort.

In conclusion, our work suggests low bias in VE estimates derived from COVID-19 and influenza test-negative studies with influenza and SARS-CoV-2 controls, respectively, in situations where these controls represent a low proportion of total test-negative controls. Nonetheless, we encourage researchers to consider this potential bias and its implications. Where researchers determine that bias is not meaningful and do not undertake deconfounding, adequate justification should be provided to promote critical interpretation and confidence in study results.

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Notes

Financial support. This work was supported by a new faculty startup award from the Albany College of Pharmacy and Health Sciences.

Potential conflicts of interest. M. K. D. has received subaward grant funding from the US National Institutes of Health and the St Luke’s Wood River Foundation for unrelated research. All other authors report no potential conflicts of interest.

All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.