Influence function methods to assess the effectiveness of influenza vaccine with survey data

Abstract

Objective

To examine a robust relative risk (RR) estimation for survey data analysis with ideal inferential properties under various model assumptions.

Data sources

We employed secondary data from the Household Component of the 2000–2016 US Medical Expenditure Panel Survey (MEPS).

Study design

We investigate a broad range of data‐balancing techniques by implementing influence function (IF) methods, which allows us to easily estimate the variability for the RR estimates in the complex survey setting. We conduct a simulation study of seasonal influenza vaccine effectiveness to evaluate these approaches and discuss techniques that show robust inferential performance across model assumptions.

Data collection/Extraction methods

Demographic information, vaccine status, and self‐administered questionnaire surveys were obtained from the longitudinal data files. We linked this information with medical condition files and medical event to extract the disease type and associated expenditures for each medical visit. We excluded individuals who were 18 years or younger at the beginning of each panel.

Principal findings

Under various model assumptions, the IF methods show robust inferential performance when the data‐balancing procedures are incorporated. Once IF methods and data‐balancing techniques are implemented, contingency table‐based RR estimation yields a comparable result to the generalized linear model approach. We demonstrate the applicability of the proposed methods for complex survey data using 2000–2016 MEPS data. When employing these methods, we find a significant, negative association between vaccine effectiveness (VE) estimates and influenza‐incurred expenditures.

Conclusions

We describe and demonstrate a robust method for RR estimation and relevant inferences for influenza vaccine effectiveness using MEPS data. The proposed method is flexible and can be extended to weighted data for survey data analysis. Hence, these methods have great potential for health services research, especially when data are nonexperimental and imbalanced.

What is already known on this topic

• There is no unified approach to estimate the variance of relative risk estimators that incorporate matching weights in the survey data analysis setting.

What this study adds

• Influence function methods incorporate sampling and matched‐data weights to evaluate relative risk estimator variability in survey data analysis and show accurate variance estimation for both simple and more complicated models.

• National‐level publicly available survey datasets are feasible sources to evaluate the public health impact of the influenza vaccine; we discuss procedures for accurate estimation and inference using such datasets.

• Implementing various data‐balancing techniques, we identify robust methods for vaccine effectiveness estimation and relevant inference with procedural discussions for handling unbalanced data, and provide guidelines for implementing the approach with data‐balancing procedures.

1. INTRODUCTION

Seasonal influenza epidemics occur regularly each winter, causing approximately 500,000 deaths per year worldwide. ¹ Routine annual vaccination is recommended for older adults and other high‐risk groups with influenza complications, as influenza strains in seasonal epidemics vary from year to year. ² , ³ Vaccination is also recommended in other groups, such as school children, to reduce the general population infection rate. ⁴

However, estimating the overall effectiveness of the influenza vaccine is challenging due to the nature of the disease, validity of effectiveness measures, and reliance upon narrow study populations. The annual effectiveness of influenza vaccination varies considerably for different virus subtypes. ⁵ , ⁶ , ⁷ , ⁸ Preliminary estimates may be based on pharmaceutical data, although some vaccines are licensed based on randomized clinical trials that use antibody response to the vaccine as measured in the laboratory, rather than influenza infection among people who were vaccinated. ⁹ Due to the impracticality of randomized controlled trials, observational studies in the form of a test‐negative design are also used to estimate the relative risk (RR), where the test‐negative design limits to a certain group of people such as those with acute respiratory illness at ambulatory care settings. These limitations call into question the effectiveness measured through such studies.

Given these concerns, estimating influenza vaccine effectiveness from large national‐level survey data such as Medical Expenditure Panel Survey (MEPS) data can offer additional insights regarding the protective role that influenza vaccination plays in the general population. Few studies employ reproducible and robust methods for estimating influenza vaccine effectiveness (VE) in the context of complex survey design, as such surveys are not specifically designed to identify a causal effect. Current extant literature does not provide a comprehensive comparison of methodologies such as different matching methods with various confounding factors (data‐balancing method), statistical methods to be used (e.g., generalized linear models), weighting methods, or combinations of these approaches. We initially consider only observed confounding factors for the proposed methods. We later discuss the effect of unobserved confounding factors.

Our aim is to discuss reliable strategies for RR estimation to address influenza VE as well as robust variance estimation based on publicly available data. We note that VE is formally expressed as 1‐RR. ¹⁰ RR is a ratio of the probability of an event in the treatment group versus that in the control group. It is commonly used to measure the effect of treatment or exposure on an outcome in health‐related cohort studies, ¹¹ and it is easier to interpret than an odds ratio (OR). ¹² , ¹³ For the methodology, we develop an inferential technique for effectiveness measures based on the influence function (IF) and examine this method in the context of various model assumptions and data‐balancing techniques. IF is a classical method to evaluate the variability of a composite statistic in complex surveys. In this study, we show that IF methods provide accurate variance estimation for models of differing levels of complexity and incorporate sampling and matched‐data weights—a necessary step for survey data analysis. Using this method, we provide RR estimates for the influenza vaccine using 2000–2016 MEPS data and compare these estimates with results from the US Centers for Disease Control and Prevention (CDC). In addition, we investigate the influenza‐incurred expenditure estimates for each panel and identify a significant negative association between the estimated VE and the corresponding expenditures.

2. METHODS

2.1. Data source and extraction

We use data from the Household Component of the 2000–2016 MEPS‐HC. ¹⁴ The MEPS is designed as an ongoing survey to permit annual estimates of healthcare utilization, expenditures, insurance coverage, and sources of payment for the US civilian noninstitutionalized population. ¹⁵ The MEPS‐HC collects data longitudinally with overlapping panels in which any given sample panel has been interviewed a total of five times (five rounds) over a two‐year period. Each year, MEPS‐HC data are collected and recorded with a series of files. There are three major types of files used in our study: (1) Household Component Full‐Year Files including longitudinal data files and medical conditions files, which contain person‐level data covering the calendar year; (2) Household Component Event Files, which is a set of eight event‐level files consisting of medical events for persons; (3) Appendix to MEPS event file, which links condition files with event files. For this study, we first use the variables CONDIDX and DUPERSID to link the medical conditions file with the Appendix file. We concatenate all event files, and sum the medical expenditure for each event from all sources and then link the concatenated file with the Appendix file through DUPERSID and EVNTIDX. Since each person could have multiple event records, we create an indicator variable to show the influenza status with a corresponding summation of expenditure on influenza. Finally, we link this merged file to the longitudinal data file using DUPERSID. We include individuals 18 years or older at the beginning of each panel with influenza vaccine information and at least one record in the event files. We did not seek Institutional Review Board approval for this study because the MEPS data is de‐identified and publicly available.

2.2. Study measures

The longitudinal data files in MEPS‐HC consist of panel surveys. Each panel survey consists of five rounds (R1 to R5) for two consecutive years. Influenza vaccination is recorded once a year in R3 and R5. We employ two different influenza vaccination (case) definitions that reflect different levels of inclusivity, that is, Case condition I: receiving the influenza vaccine in the first year or else control; Case condition II: receiving influenza in the first year and second year or else control. In addition, four different time range definitions are considered: at least one influenza incident in R3 (range I), in R3 and R4 (range II), in R3, R4, and R5 (range III), and across all rounds (range IV). We note that R3 mostly covers the influenza season (fourth quarter of the first year, first quarter of the second year), but some information could be collected during R4 instead of R3 due to the overlapping study design.

For the covariates, we primarily rely on covariates collected through questionnaires in the longitudinal data file. We identify demographic and socioeconomic covariates that may affect influenza vaccination status: age, race, gender, educational attainment, employment status, and insurance type. As health status may affect influenza vaccine receipt, we also include the following covariates: perceived physical health status, perceived mental health status, body mass index (BMI), and presence of chronic conditions for which influenza vaccination is recommended. ¹⁶ The chronic health conditions include diabetes, asthma, emphysema, chronic bronchitis, cancer, and heart disease (stroke included). Overall, we identify 13 relevant covariates for panel 5–11 (years 2000–2007) and 2 additional covariates (cancer and chronic bronchitis) for panel 12–20 (years 2007–2016). The relevance of these covariates is well discussed in the literature (Table S1). Also, for a detailed categorization of these variables and significant effect on vaccine receipt using the MEPS data, see Table S2. We test the multicollinearity of these covariates in predicting vaccine receipt defined by both Case conditions. All generalized variance inflation factors (GVIFs) are less than 1.7 after adjusting for the dimension of the confidence ellipsoid (i.e., GVIF^1/2df ), well below the rule of thumb of 4. ¹⁷ A total of 4.78% of observations are excluded due to missing data in covariates. We examine the missing pattern of data, and the results show that it is independent of the vaccine receipt status. In this paper, we provide the analysis based on both the 13 covariates and the 15 covariates (for the data from 2007).

2.3. Nonparametric variance estimation

The influence function method is a linearization technique that is conventionally used in survey sampling to simplify variance estimation of a statistic with a complicated structure. ¹⁸ , ¹⁹ , ²⁰ The IF quantifies the effect of an individual value on the target parameter; thus, we can incorporate individual weights (e.g., matching weights) into IFs for matched data or survey data and obtain variability of a statistic without relying on a parametric distribution assumption.

Suppose that $T$ is a quantity of interest and ${\text{IF}}_i\left(T\right)\left(i=1,\dots ,n\right)$ is the corresponding IF for each observation with the sample size $n$. Once the IFs are obtained, variance estimation is possible in the form of $\mathit{Var}\left(\overline{\text{IF}\left(T\right)}\right)$, where $\overline{\text{IF}\left(T\right)}$ indicates the sample mean of IFs where parameters in ${\text{IF}}_i\left(T\right)$ are replaced by their sample estimates. ²¹ When the statistic of interest is a joint function of other quantities (say $f\left(T\right)$), the IF is obtained using the functional delta method ²² as

$$\text{IF}\left(f\left(T\right)\right)=\partial f\left(T\right)/\partial T\cdot \text{IF}\left(T\right),$$ (1)

where $f$ is a differentiable function defined on the space of values for $T$.

For the RR, let us define $X$ and $Y$ as the binary exposure (e.g., 1: vaccination vs. 0: non‐vaccination) and the binary outcome (e.g., 1: influenza infection vs. 0: no influenza infection), respectively. The RR estimator for the $2\times 2$ contingency has a form $R_C=\overline{A_1}\overline{B_2}/\overline{A_2}\overline{B_1}$, where $\overline{A_1},\overline{A_2},\overline{B_1}$ and $\overline{B_2}$ are the sample means of $A_{1i}=I\left(X_i=1,Y_i=1\right)$, $A_{2i}=I\left(X_i=1\right)$, $B_{1i}=I\left(X_i=0,Y_i=1\right)$, $B_{2i}=I\left(X_i=0\right)$, $i=1,\dots ,n$, respectively and $I$ is the indicator function. Then, the IF of $R_C$ for ith observation is evaluated in the following form:

$$\text{IF}\left(R_{Ci}\right)=\frac{\overline{A_1}\overline{B_2}}{\overline{B_1}\overline{A_2}}\left(\frac{A_{1i}}{\overline{A_1}}+\frac{B_{2i}}{\overline{B_2}}-\frac{B_{1i}}{\overline{B_1}}-\frac{A_{2i}}{\overline{A_2}}\right).$$ (2)

A detailed derivation for Equation (2) is given in Appendix (A1) and (A2).

Now, consider a generalized linear model with X, Y, and additional covariates $Z_1,\dots ,Z_k.$ For RR estimators based on the generalized linear model, the IF needs to be derived based on an implicit function, such as the likelihood functions. We consider three commonly used approaches for RR estimation: logistic regression, log‐binomial regression, and probit regression. Detailed derivations of IFs of the RR based on logistic regression are found in Appendix A. The IFs for other models are similarly obtained. For logistic regression, the RR estimator and the estimated IF have the following forms:

$$\begin{array}{l}{R}_L=\frac{1+e^{-\left({\widehat{\beta }}_0+{\widehat{\beta }}_2{Z}_1+\cdots +{\widehat{\beta }}_{k+1}{Z}_k\right)}}{1+e^{-\left({\widehat{\beta }}_0+{\widehat{\beta }}_1+{\widehat{\beta }}_2{Z}_1+\cdots +{\widehat{\beta }}_{k+1}{Z}_k\right)}},\text{IF}{\left(R_L\right)}_i=\left[\frac{e^{-\left({\widehat{\beta }}_0+{\widehat{\beta }}_1\right)}-e^{-{\widehat{\beta }}_0}}{{\left(1+e^{-\left({\widehat{\beta }}_0+{\widehat{\beta }}_1\right)}\right)}^2}\frac{e^{-\left({\widehat{\beta }}_0+{\widehat{\beta }}_1\right)}\left(1+e^{-{\widehat{\beta }}_0}\right)}{{\left(1+e^{-\left({\widehat{\beta }}_0+{\widehat{\beta }}_1\right)}\right)}^2}\right]{\text{IF}}_L{\left(\widehat{\mathbf{\beta }}\right)}_i,\\ \end{array}$$ (3)

where ${\text{IF}}_L{\left(\widehat{\mathbf{\beta }}\right)}_i\equiv {\left(-\frac{1}{n}{\sum }_{i=1}^n{\mathbf{v}}_i{\mathbf{v}}_i^{\prime }{\mu }_L\left({\mathbf{v}}_i^{\prime }\widehat{\mathbf{\beta }}\right)\left(1-{\mu }_L\left({\mathbf{v}}_i^{\prime }\widehat{\mathbf{\beta }}\right)\right)\right)}^{-1}{\mathbf{v}}_i\left(y_i-{\mu }_L\left({\mathbf{v}}_i^{\prime }\widehat{\mathbf{\beta }}\right)\right)$, $\widehat{\mathbf{\beta }}$ and ${\mathbf{v}}_i$ are vectors of coefficient estimates and all covariates and ${\mu }_L\left({\mathbf{v}}_i^{\prime }\widehat{\mathbf{\beta }}\right)=\exp \left({\mathbf{v}}_i^{\prime }\widehat{\mathbf{\beta }}\right){\left(1+\exp \left({\mathbf{v}}_i^{\prime }\widehat{\mathbf{\beta }}\right)\right)}^{-1}$. The RR estimator and the estimated IF for the log‐binomial regression are

$$R_B=e^{{\widehat{\beta }}_1},\text{IF}{\left(R_B\right)}_i=e^{{\widehat{\beta }}_1}{\text{IF}}_B{\left(\widehat{\mathbf{\beta }}\right)}_i,$$ (4)

where ${\text{IF}}_B{\left(\widehat{\mathbf{\beta }}\right)}_i\equiv {\left(-\frac{1}{n}{\sum }_{i=1}^n{\mathbf{v}}_i{\mathbf{v}}_i^{\prime }{\mu }_L\left({\mathbf{v}}_i^{\prime }\widehat{\mathbf{\beta }}\right)\right)}^{-1}{\mathbf{v}}_i\left(y_i-{\mu }_B\left({\mathbf{v}}_i^{\prime }\widehat{\mathbf{\beta }}\right)\right)$ and ${\mu }_B\left({\mathbf{v}}_i^{\prime }\widehat{\mathbf{\beta }}\right)=\exp \left({\mathbf{v}}_i^{\prime }\widehat{\mathbf{\beta }}\right)$. The RR estimator for the probit regression and the estimated IF are

$$\begin{array}{l}{R}_P=\frac{\mathrm{\Phi }\left({\widehat{\beta }}_0+{\widehat{\beta }}_2{Z}_1+\cdots +{\widehat{\beta }}_{k+1}{Z}_k\right)}{\mathrm{\Phi }\left({\widehat{\beta }}_0+{\widehat{\beta }}_1+{\widehat{\beta }}_2{Z}_1+\cdots +{\widehat{\beta }}_{k+1}{Z}_k\right)},\\ \text{IF}{\left(R_P\right)}_i=\left[\frac{\phi \left({\widehat{\beta }}_0+{\widehat{\beta }}_1\right)\mathrm{\Phi }\left({\widehat{\beta }}_0\right)-\phi \left({\widehat{\beta }}_0\right)\mathrm{\Phi }\left({\widehat{\beta }}_0+{\widehat{\beta }}_1\right)}{{\left(\mathrm{\Phi }\left({\widehat{\beta }}_0\right)\right)}^2}\frac{\phi \left({\widehat{\beta }}_0+{\widehat{\beta }}_1\right)}{{\left(\mathrm{\Phi }\left({\widehat{\beta }}_0\right)\right)}^2}\right]{\text{IF}}_P{\left(\widehat{\mathbf{\beta }}\right)}_i,\end{array}$$ (5)

where $\mathrm{\Phi }$ and $\phi$ are the standard normal cumulative distribution function and density function, respectively, and ${\text{IF}}_P{\left(\widehat{\mathbf{\beta }}\right)}_i\equiv {\left(-\frac{1}{n}{\sum }_{i=1}^n{\mathbf{v}}_i{\mathbf{v}}_i^{\prime }\phi \left({\mathbf{v}}_i^{\prime }\widehat{\mathbf{\beta }}\right)\right)}^{-1}{\mathbf{v}}_i\left(y_i-\mathrm{\Phi }\left({\mathbf{v}}_i^{\prime }\widehat{\mathbf{\beta }}\right)\right)$.

We construct the confidence intervals for the RR estimates based on the Wald confidence interval. ²³

2.4. Data‐balancing techniques

Data balancing is a procedure in which the treatment assignment can be independent of the potential outcomes conditioned on the pretreatment variables. ²⁴ To determine the best performing techniques for matched‐data analysis using IFs, we use the well‐known propensity score using logistic regression. Relevant to propensity score balancing and related data analysis, major techniques that we consider are as follows:

1. Propensity score methods: We consider the propensity score many‐to‐one nearest neighbor matching ²⁴ , ²⁵ (M) and the stabilized propensity score weighting ²⁶ (SW). We selected two as the maximum number to match according to the literature. ²⁷ We note that increasing the number may deflate the variability of the estimate but also inflate the bias simultaneously, resulting in no overall gain in terms of the mean square error.

2. Covariates used for model fitting: After applying the propensity score method, we consider two models, a model with only the treatment variable (no acronym used) and a full model with treatment variable and all covariates used in propensity score estimation (F).

3. Weight adjustment: Weight adjustment ²⁸ (A) after matching is also considered due to the risk of introducing additional bias into the estimates due to more than one matching.

Figure S1 describes various combinations of techniques that we used for logistic regression (Logit). These different techniques are also applied to the log‐binomial regression (Logb) and the probit regression (Probit), producing a total of 18 different analytical combinations as well as additional four analytical techniques for the contingency table method (CT). We note that, as an alternative to nearest neighbor matching, caliper matching ²⁹ can be used. In our simulation, caliper matching appears not to outperform nearest neighbor matching if the true model is not logistic; thus, it is not considered in this study, as we assume that the distribution is unknown.

We package the implementation of data‐balancing methods with the corresponding IF functions in R (version: 4.0.2) as the source code available through the GitHub website (https://github.com/Melindatian/MEPS-data-analysis).

2.5. Simulation scheme

We investigate the inferential performance of various data‐balancing techniques using an extensive Monte Carlo study (1000 simulations per scenario). In the simulation, the treatment variable $X$ is confounded by up to five covariates $\left(k=1,\dots ,5\right)$ in the form of the logistic model, where the covariates include a binary covariate at default and other continuous covariates. Specifically, $X$ is generated from the Bernoulli distribution $\left(\text{Bernoulli}\left(p_X\right)\right)$, where $p_X$ is defined as $\text{logit}{p}_X={\alpha }_0+{\sum }_{i=1}^k{Z}_i{\alpha }_i,k=1,\dots ,5$ with the coefficients $\left({\alpha }_0,\dots ,{\alpha }_5\right)=\left(-0.3,1.6,1.8,0.6,0.5,-1.8\right)$ describing various associations between the treatment variable and the varying number of covariates. Also, we independently generate $Z_1$ from $\text{Bernoulli}\left(0.4\right)$ and $Z_i,i=1,\dots ,5$ from the standard normal distribution. The outcome variable $Y$ is generated by logistic model and log‐binomial model, the two common regression models used for evaluating the treatment effectiveness. For logistic model, we generate $Y$ from $\text{Bernoulli}\left(p_Y\right)$, where $p_Y$ is defined as $\text{logit}{p}_Y={\beta }_0+X{\beta }_1+{\sum }_{j=2}^{k+1}{Z}_j{\beta }_j,k=1,\dots ,5$ with the coefficients $\left({\beta }_0,\dots ,{\beta }_6\right)=\left(-1,2,-0.2,-0.1,0.1,0.4,-0.3\right)$, describing the varying number of covariates confounded with the treatment variable $X$. Also, for log‐binomial model, we generate $Y$ from $\text{Bernoulli}\left(p_Y\right)$, where $p_Y$ is defined as $\log p_Y={\beta }_0+X{\beta }_1+{\sum }_{j=2}^{k+1}{Z}_j{\beta }_j,k=1,\dots ,5$. The coefficients $\left({\beta }_0,{\beta }_1\right)$ are given as $\left(-1,0.5\right)$ _, and the coefficients for the varying number of covariates in ${\sum }_{j=2}^{k+1}{Z}_j{\beta }_j$ are given as $\left(-0.63\right),\left(-1.23,-0.09\right),\left(-1.25,-0.16,-0.02\right),\left(-1.25,-0.16,-0.02,-0.05\right)$, and $\left(0.21,-0.01,-0.12-0.13,-0.01\right)$ for $k=1,\dots ,5$, respectively, making sure that $p_Y$ is a positive value. The propensity score models for $X$ include all the covariates. Under each model, we compare the 22 methods introduced previously (Tables 1 and 2).

TABLE 1.

Simulated “true” variance of RR estimates (SV) versus mean of estimated variance using the IF (IFV) under the data generated by the logistic model (k is the number of covariates used)

	k = 1		k = 2		k = 3		k = 4		k = 5
	SV	IFV	SV	IFV	SV	IFV	SV	IFV	SV	IFV
Logit	0.054	0.052	0.059	0.055	0.057	0.052	0.080	0.072	0.075	0.073
LogitM‐F	0.073	0.070	0.086	0.084	0.088	0.079	0.133	0.127	0.152	0.146
LogitM	0.073	0.071	0.084	0.084	0.082	0.078	0.110	0.111	0.123	0.121
LogitMA	0.088	0.079	0.114	0.106	0.098	0.085	0.150	0.152	0.143	0.147
LogitSW‐F	0.055	0.052	0.061	0.057	0.059	0.053	0.109	0.096	0.135	0.115
LogitSW	0.051	0.049	0.055	0.052	0.053	0.051	0.110	0.105	0.143	0.137
Logb	0.053	0.049	0.057	0.053	0.056	0.052	0.089	0.090	0.087	0.103
LogbM‐F	0.073	0.070	0.084	0.083	0.083	0.077	0.109	0.104	0.121	0.113
LogbM	0.073	0.071	0.084	0.084	0.082	0.078	0.110	0.111	0.123	0.121
LogbMA	0.088	0.079	0.114	0.106	0.098	0.085	0.150	0.152	0.143	0.147
LogbSW‐F	0.051	0.048	0.057	0.051	0.055	0.050	0.153	0.089	0.195	0.112
LogbSW	0.051	0.049	0.055	0.052	0.053	0.051	0.110	0.105	0.143	0.137
CT	0.052	0.049	0.054	0.052	0.053	0.051	0.089	0.084	0.091	0.093
CTM	0.073	0.071	0.084	0.084	0.082	0.078	0.110	0.111	0.123	0.121
CTMA	0.088	0.079	0.114	0.106	0.098	0.085	0.150	0.152	0.143	0.147
CTSW	0.051	0.049	0.055	0.052	0.053	0.051	0.110	0.105	0.143	0.137
Probit	0.054	0.052	0.059	0.055	0.057	0.052	0.079	0.071	0.074	0.072
ProbitM‐F	0.072	0.070	0.085	0.084	0.087	0.078	0.131	0.125	0.147	0.142
ProbitM	0.073	0.071	0.084	0.084	0.082	0.078	0.110	0.111	0.123	0.121
ProbitMA	0.088	0.079	0.114	0.106	0.098	0.085	0.150	0.152	0.143	0.147
ProbitSW‐F	0.055	0.053	0.062	0.057	0.059	0.053	0.108	0.094	0.143	0.115
ProbitSW	0.051	0.049	0.055	0.052	0.053	0.051	0.110	0.105	0.143	0.137

Note: The abbreviations in the first column are combinations of following abbreviations: Logit (logistic), Logb (log‐binomial), CT (contingency table), Probit (probit), M (propensity score matching), F (full covariates adjusted; without F indicates no covariate adjustment), A (weight adjustment), and SW (stabilized propensity score weighting).

TABLE 2.

Coverage rates (%) and the widths of confidence intervals (95% confidence level) for data simulated from Logit and Log‐binomial model, respectively

True model	Fitted model	k = 1	k = 2	k = 3	k = 4	k = 5
Logistic	Logit (true model)	94.9 (0.894)	94.9 (0.922)	95.1 (0.896)	93.6 (1.050)	95.3 (1.056)
	LogitM‐F	95.0 (1.036)	94.4 (1.137)	94.3 (1.099)	94.1 (1.399)	95.9 (1.499)
	LogitM	95.8 (1.045)	96.6 (1.137)	95.7 (1.095)	95.8 (1.306)	96.6 (1.365)
	LogitMA	94.2 (1.103)	95.3 (1.279)	95.1 (1.145)	88.1 (1.528)	86.7 (1.502)
	LogitSW‐F	95.1 (0.897)	94.3 (0.936)	94.1 (0.906)	93.7 (1.214)	93.0 (1.330)
	LogitSW	93.4 (0.866)	94.2 (0.896)	94.2 (0.884)	94.6 (1.272)	94.1 (1.453)
	Logb	93.5 (0.871)	94.0 (0.903)	93.2 (0.894)	88.7 (1.173)	80.6 (1.257)
	LogbM‐F	95.4 (1.040)	96.0 (1.128)	95.4 (1.085)	92.7 (1.266)	92.9 (1.320)
	LogbM	95.8 (1.045)	96.6 (1.137)	95.7 (1.095)	95.8 (1.306)	96.6 (1.365)
	LogbMA	94.2 (1.103)	95.3 (1.279)	95.1 (1.145)	88.1 (1.528)	86.7 (1.502)
	LogbSW‐F	93.5 (0.863)	94.0 (0.889)	94.2 (0.878)	81.3 (1.172)	76.4 (1.313)
	LogbSW	93.4 (0.866)	94.2 (0.896)	94.2 (0.884)	94.6 (1.272)	94.1 (1.453)
	CT	89.7 (0.867)	86.1 (0.896)	84.8 (0.889)	17.3 (1.135)	8.0 (1.195)
	CTM	95.8 (1.045)	96.6 (1.137)	95.7 (1.095)	95.8 (1.306)	96.6 (1.365)
	CTMA	94.2 (1.103)	95.3 (1.279)	95.1 (1.145)	88.1 (1.528)	86.7 (1.502)
	CTSW	93.4 (0.866)	94.2 (0.896)	94.2 (0.884)	94.6 (1.272)	94.1 (1.453)
Log‐binomial	Logb (true model)	94.9 (0.518)	95.0 (0.559)	95.5 (0.555)	95.5 (0.646)	95.8 (0.578)
	LogbM‐F	95.1 (0.581)	94.6 (0.672)	94.6 (0.643)	95.9 (0.796)	94.3 (0.751)
	LogbM	95.7 (0.615)	96.7 (0.755)	96.9 (0.709)	96.2 (0.836)	92.6 (0.737)
	LogbMA	93.8 (0.611)	94.9 (0.786)	95.3 (0.713)	92.7 (1.021)	94.2 (0.932)
	LogbSW‐F	95.4 (0.530)	94.9 (0.569)	94.4 (0.563)	92.4 (0.762)	89.9 (0.738)
	LogbSW	95.5 (0.546)	96.8 (0.618)	97.0 (0.619)	95.4 (0.938)	91.8 (0.741)
	CT	74.1 (0.584)	29.6 (0.718)	41.0 (0.694)	86.5 (0.634)	48.8 (0.410)
	CTM	95.7 (0.615)	96.7 (0.755)	96.9 (0.709)	96.2 (0.836)	92.6 (0.737)
	CTMA	93.8 (0.611)	94.9 (0.786)	95.3 (0.713)	92.7 (1.021)	94.2 (0.932)
	CTSW	95.5 (0.546)	96.8 (0.618)	97.0 (0.619)	95.4 (0.938)	91.8 (0.741)
	Logit	88.6 (0.457)	88.0 (0.485)	91.6 (0.487)	90.5 (0.585)	92.1 (0.558)
	LogitM‐F	91.0 (0.523)	88.5 (0.597)	92.2 (0.596)	92.0 (0.750)	92.9 (0.724)
	LogitM	95.7 (0.615)	96.7 (0.755)	96.9 (0.709)	96.2 (0.836)	92.6 (0.737)
	LogitMA	93.8 (0.611)	94.9 (0.786)	95.3 (0.713)	92.7 (1.021)	94.2 (0.932)
	LogitSW‐F	86.4 (0.459)	86.9 (0.493)	90.9 (0.495)	90.2 (0.720)	87.8 (0.729)
	LogitSW	95.5 (0.546)	96.8 (0.618)	97.0 (0.619)	95.4 (0.938)	91.8 (0.741)

Note: The abbreviations in the first column are combinations of following abbreviations: Logit (logistic), Logb (log‐binomial), CT (contingency table), M (propensity score matching), F (full covariates adjusted; without F indicates no covariate adjustment), A (weight adjustment), SW (stabilized propensity score weighting).

For an additional scenario, we consider incomplete models to show the robustness of proposed methods for real‐world data analysis, where not all covariates are used for matching, model fitting, and weighting adjustment; that is, the un‐confoundedness assumption is invalid. The full model is constructed following the same process as in the simulation scheme previously discussed. Then, we remove the covariates one by one in the model for RR estimations, pretending that some covariates are not identified. We compare the performance of all 22 methods based on coverage rates with the width of confidence intervals, and the results are shown in Table 3.

TABLE 3.

Coverage rates (%) and the widths of confidence intervals (95% confidence level) under incomplete covariates are used. The data are generated by logistic regression with the full covariates

	Used covariates = 5	Used covariates = 4	Used covariates = 3	Used covariates = 2	Used covariates = 1
Logit	91.5 (1.176)	89.2 (1.176)	88.9 (1.176)	33.2 (1.300)	16.8 (1.358)
LogitM‐F	94.8 (1.568)	95.5 (1.568)	93.7 (1.616)	45.9 (1.568)	25.8 (1.467)
LogitM	96.5 (1.300)	97.2 (1.300)	95.8 (1.358)	54.3 (1.358)	33.8 (1.300)
LogitMA	93.2 (1.663)	94.5 (1.616)	94.4 (1.663)	92.4 (1.663)	71.3 (1.616)
LogitSW‐F	92.5 (1.321)	89.4 (1.277)	88.7 (1.283)	37.8 (1.318)	18.2 (1.355)
LogitSW	93.9 (1.260)	96.3 (1.177)	96.3 (1.183)	66.5 (1.211)	37.2 (1.224)
Logb	83.2 (1.176)	80.0 (1.240)	78.3 (1.176)	42.6 (1.240)	30.4 (1.240)
LogbM‐F	96.5 (1.240)	96.2 (1.300)	95.9 (1.300)	62.9 (1.358)	38.8 (1.300)
LogbM	96.5 (1.300)	97.2 (1.300)	95.8 (1.358)	54.3 (1.358)	33.8 (1.300)
LogbMA	93.2 (1.663)	94.5 (1.616)	94.4 (1.663)	92.4 (1.663)	71.3 (1.616)
LogbSW‐F	89.2 (1.224)	94.2 (1.175)	94.7 (1.181)	67.8 (1.202)	37.7 (1.222)
LogbSW	93.9 (1.260)	96.3 (1.177)	96.3 (1.183)	66.5 (1.211)	37.2 (1.224)
CT	23.6 (1.240)	23.6 (1.240)	23.6 (1.240)	23.6 (1.240)	23.6 (1.240)
CTM	96.5 (1.300)	97.2 (1.300)	95.8 (1.358)	54.3 (1.358)	33.8 (1.300)
CTMA	93.2 (1.663)	94.5 (1.616)	94.4 (1.663)	92.4 (1.663)	71.3 (1.616)
CTSW	93.9 (1.260)	96.3 (1.177)	96.3 (1.183)	66.5 (1.211)	37.2 (1.224)
Probit	92.7 (1.176)	90.4 (1.176)	90.1 (1.176)	35.1 (1.300)	17.4 (1.358)
ProbitM‐F	96.4 (1.518)	96.0 (1.568)	94.0 (1.616)	47.4 (1.568)	26.1 (1.467)
ProbitM	96.5 (1.300)	97.2 (1.300)	95.8 (1.358)	54.3 (1.358)	33.8 (1.300)
ProbitMA	93.2 (1.663)	94.5 (1.616)	94.4 (1.663)	92.4 (1.663)	71.3 (1.616)
ProbitSW‐F	94.1 (1.309)	91.9 (1.269)	91.2 (1.277)	40.5 (1.315)	18.5 (1.352)
ProbitSW	93.9 (1.260)	96.3 (1.177)	96.3 (1.183)	66.5 (1.211)	37.2 (1.224)

Note: The abbreviations in the first column are combinations of following abbreviations: Logit (logistic), Logb (log‐binomial), CT (contingency table), Probit (probit) M (propensity score matching), F (full covariates adjusted; without F indicates no covariate adjustment), A (weight adjustment), SW (stabilized propensity score weighting).

3. RESULTS

3.1. Simulation result

Overall, the IF approach produces accurate variance estimations across the various data‐balancing methods and models (Table 1). The coverage rates of the confidence intervals based on the IF method are shown in Table 2, where the value $k$ indicates the number of covariates used in the true model. The probit model results are found in Table S3, which show similar properties to logistic regression. We note that the top entry under each true model in Table 2 is the correct model specification with the full covariates, assuring the coverage rate close to the nominal level. Without matching, relatively unsatisfactory coverage rates are observed in the cases of the wrong model assumption even with all correct confounding covariates incorporated (e.g., approaches with “Logit”’ acronyms under log‐binomial as the true model). On the other hand, most of the matching and analytical methods show satisfactory coverage rates, including the CT method (CTM, CTMA, CTSW). Both M and SW balancing methods yield overall satisfactory results in terms of coverage rates; however, SW provides narrower confidence intervals most of the time (Table 2) and the smallest bias (Figure S2), regardless of the model used. For using full covariates (F) or including only the treatment variable (no acronym), there are no differences if a true model is used; however, a wrong model may cause poor performance when we include full covariates (LogbM‐F and LogbSW‐F under Logistic; LogitM‐F, and LogitSW‐F under Log‐binomial).

For incomplete models (Table 3), as expected, the coverage rate does not reach the nominal level and decreases more as the number of covariates decreases. However, applying data propensity score methods, the coverage rate is improved by M, MA, and SW methods for all models, demonstrating the benefit of data‐balancing under incomplete models. When the number of unidentified covariates is relatively small (e.g., three or four covariates), the SW method shows the best performance overall. When most covariates are unidentified (e.g., one covariate), the MA method performs best at the expense of wider confidence interval widths. As shown with the complete model (Table 2), there is no advantage of incorporating full covariates in the model for RR estimation in general.

In summary, after balancing, models with only the treatment variable as a covariate produce satisfactory inference as to whether we use the correct model or not. A simpler approach, such as the contingency Table (CT) method, performs well in comparison with other parametric models. Among all propensity score methods, the stabilized propensity score weighting (SW) method is most robust in terms of coverage rates, the width of the confidence interval, and the bias.

3.2. RR estimation using the MEPS data

Based on the original data (without balancing), we carry out our initial estimations of the proportions of influenza vaccination, overall influenza infection, and influenza infection among vaccinated people (Figure 1A). All these analyses incorporate the MEPS survey design. Across the different definitions (Case conditions I and II) and ranges (ranges I to IV), influenza vaccination rates have steadily increased over the study period (black solid line), as shown in Figure 1A. Case condition I (vaccinated in the first year only) shows a higher vaccination rate compared to Case condition II (vaccinated both years). For influenza infection rates, they first decrease and then steadily increase in both the overall group (gray solid line) and the vaccinated group (gray dashed line). Range IV has the largest estimates among different ranges since it counts events with all follow‐up periods. While the different definitions of cases and ranges give rise to different proportion estimates in terms of magnitudes, they produce overall similar trend patterns over the years, and thus, it is clear that they would not lead to vastly different comparative evaluations of VE for the case and the control. Compared among these lines in Figure 1A, the trend of influenza infection rate does not always negatively change with the positive trend of flu vaccine rate (e.g., the years 2000–2001 and 2011–2012). Overall, we observe slightly lower influenza infection rates among the vaccinated group compared to the unvaccinated group, with the exception of years 2014–2015.

FIGURE 1.

(A) (top panel): The overall proportion of flu vaccination (black solid line with black tick marks on the left y‐axis), the overall proportion of flu infection (gray dashed line with gray tick marks on the right y‐axis) and the proportion of flu infection among vaccinated people (gray solid line) for Case condition I (vaccine year 1; left panels) and Case condition II (vaccine year 1 and 2; right panels) and ranges I–IV (from the top row to bottom row) based on MEPS panels 5 (starting in 2000)–20 (starting in 2015), United States, 2000–2016. (B) (bottom panel): The RR estimates (log‐transformed) of influenza infection between case (vaccinated) and control (not vaccinated) under Case conditions I (vaccine year 1) and II (vaccine years 1 and 2) and ranges I–IV for MEPS panels 5 (starting in 2000)–20 (starting in 2015), United States, 2000–2016. The dashed lined with triangle points is the RR estimate using the original data. The solid line with round points is the RR estimates by the CTSW model. The filled points indicate a significant difference from RR = 1 (significance level = 0.05) for original and balanced data, respectively. MEPS, medical expenditure panel survey; RR, relative risk

Figure S3 shows the proportions of influenza vaccination among various subgroups of 15 covariates previously mentioned. While the general trend of influenza vaccination is increasing for all subgroups of 15 covariates, there are substantial differences among subgroups (also see Table S2 for the detailed demographic summary). Since VE can vary in different subgroups, ³⁰ balancing these covariates is required for fair and accurate estimation of VE.

Based on our simulation study observations, we choose the CTSW method to investigate the RR for the eight different definition combinations (Figure 1B). We estimate the propensity score based on a complex survey design. We incorporate the survey weights by constructing a new weight from the multiplication of the propensity score weights and survey weights. ³¹ Data balancing is carried out using the 13 covariates throughout the study period, and the two additional covariates are added for the data from the year 2007, as indicated in the study measures section. We include the analyses using the 13 covariates consistently throughout the study period in Figures S4 and S5, which provide analogous results and conclusions that we describe here. Compared with the original data (dashed line) in Figure 1B, RR estimates using the balanced data (solid line) are consistently more neutral (close to 1) and less significant most of the time, indicating the bias correction ability of the balancing method. In general, RR estimates tend to be less than 1 (i.e., $\log \left(1\right)$ or 0 in Figure 1B) for both the original and balanced data showing vaccine effectiveness. There are a few years with RR estimates greater than 1, although these are statistically insignificant at the level of 5% (e.g., years 2001, 2009, and 2014). Overall, analysis using the balanced data reinforces the observations made using the original data.

In addition, we compare our results to information available from the CDC website ³⁰ (Figure 2). In the following discussion, we choose the definitions of Case condition II and range I, which are the most comparable to CDC data. We observe that the CDC's VE estimates lie in the range of our estimated confidence intervals for the years 2004–2009 and 2013–2016 (Figure 2A). Overall MEPS VE trend is lower and not significantly associated with the CDC's VE trend (Kendall's tau = −0.07, p‐value = 0.731). We also examine the association between VE estimates and influenza‐related mean expenditure using the MEPS data (Figure 2B and 2C). The expenditure is adjusted for inflation ³² , ³³ , ³⁴ , ³⁵ and estimations are carried out using the balanced data. In Figure 2B, influenza‐related expenditure is generally stable over the years. In the comparison of group difference (treatment subtracted by control) in mean expenditures (Figure 2C) and VE estimates, there is a significant negative association at the level of 5% (Kendall's tau = −0.38, p‐value = 0.041). This indicates that the vaccinated group spends less on influenza‐related expenses than the not‐vaccinated group.

FIGURE 2.

(A) (top plot): VE (1‐RR) estimates and 95% confidence intervals using MEPS data (solid line), United States, 2000–2016 and VE from CDC (dashed line), United States, 2004–2016; (B) (middle plot): The mean flu‐incurred expenditure per person for overall (solid line), vaccinated (dotted line), and not‐vaccinated groups (dashed line), United States, 2000–2016; (C) (bottom plot): The mean expenditure difference between the treatment and control groups (dashed line, right‐side y‐axis) and VE estimate using MEPS data (solid line, left‐side y‐axis), United States, 2000–2016. All analysis based on Case conditions II (vaccine year 1 and 2) and ranges I for MEPS panels 5 (starting in 2000)–20 (starting in 2015). CDC, centers for disease control and prevention; MEPS, medical expenditure panel survey; RR, relative risk; VE, vaccine effectiveness

4. DISCUSSION

National‐level publicly available survey datasets are feasible sources to evaluate the public health impact of the influenza vaccine. In this article, we discussed procedures for accurate estimation and inference using such datasets. We described a robust variance estimation method based on the IF of RR estimators in several model settings, which can incorporate the complex survey design for the inference of RR. Through simulation studies, we showed that with the appropriately selected data‐balancing method, a simple model such as a contingency table can offer comparable inferential performance to more complicated models.

Our analysis often lowered VE estimates after data balancing. This may be due to including more high‐risk groups with compromised vaccine responsiveness ³⁶ , ³⁷ in our analysis. Our results also showed lower VE estimates compared with VE estimates from the CDC. The discrepancies between our results and the CDC records may be due to several factors. First, the CDC results were based on the test‐negative design study,⁹ whereas MEPS were developed for creating a nationally representative sample. It was acknowledged that the test‐negative design could potentially create selection bias confounding of healthcare‐seeking behavior. ³⁸ , ³⁹ The sample size of the CDC studies ranged from 346 to 8436, whereas the sample size of the MEPS data used in this study was around 10,000 with weights that can show the national‐level population effect. Also, we controlled for the effects of demographic and healthcare‐seeking confounding factors. Several studies have shown that influenza VE varies in different subgroups. ⁴⁰ The recommendation from the CDC ⁴¹ of influenza vaccine for high‐risk groups may also increase vaccination proportion in these groups, and thus, data balancing for VE estimate may be required. It is also notable that the CDC study calculated VE as $\mathrm{VE}=1-\text{OR}$, ⁴² , ⁴³ different from their recommendation in the tutorial. ¹⁰

Our results suggested that high VE may be associated with a low mean expenditure of the influenza vaccination group compared to the no influenza vaccination group based on MEPS data (Figure 2). This novel finding using the MEPS data indicated that the receipt of influenza vaccine may produce a positive outcome in the form of lower healthcare expenditures during periods of high VE. Previous research partially supported this finding. ⁴⁴ , ⁴⁵ Shireman et al. ⁴⁴ concluded that the use of high‐dose trivalent influenza vaccine reduced total healthcare expenditures for residents in the nursing home. Dabestani et al. ⁴⁶ claimed that the influenza vaccine was cost‐effective in certain population groups in the United States in their systematic review.

There were some limitations in our study due to utilizing MEPS data. First, the MEPS recorded influenza at known medical events. As a result, if a person has influenza without visiting medical facilities or purchasing medicines using his/her insurance, there would be no record. Or, an individual could receive a false negative test for influenza, which could be more likely during the peak of flu season. ⁴⁷ Second, due to the high variation of the influenza virus, several common subtypes were spreading in the human population and thus required different influenza vaccinations. The influenza VE can vary by type and subtype. ⁴⁸ However, MEPS data only recorded the general influenza vaccination and disease status without detailed vaccine information, so we cannot investigate VE by type and subtype. Finally, according to the CDC, adults with neurological conditions, blood disorders, kidney disorders, liver disorders, and weakened immune systems other than cancer (e.g., HIV) were also a high priority for influenza vaccine receipt. ⁴¹ This detailed information was not available in the form of questionnaires in the longitudinal data. However, our study results suggested that including these additional disease statuses may further dilute the effectiveness of the vaccine, while not substantial (e.g., Figure 1B and Figure S4).

Finally, we would like to emphasize that our results did not suggest that the annual influenza vaccine was ineffective. Although the RR for vaccinated individuals was higher in some years when compared to the unvaccinated, this result was not statistically significant and therefore may simply be occurring by chance. In addition, we measured VE based on whether an individual was infected with influenza, rather than whether they were hospitalized or died from the disease. Prior studies have found that vaccine receipt was associated with lower likelihood of ICU admission, ⁴⁹ a shorter ICU length of stay, and lower influenza‐related mortality ⁵⁰ among adults hospitalized with influenza. Thus, as suggested by our expenditure results and the existing literature, it is likely the case that vaccinated individuals experienced less adverse outcomes due to influenza infection than those who did not receive a vaccine.

In summary, we provided procedures for accurate estimation of VE using the survey data and a robust method to estimate its variability. We applied our methods to estimate VE using MEPS data and investigated the impact of the influenza vaccine on the general population in terms of influenza infection and related medical expenses. The discussed analytical procedure has great potential for the analysis of other nonexperimental survey data as well.

Supporting information

Data S1. Supporting Information.

Figure S1. The diagram for six methods (at the bottom row) under logistic regression. Analytic model* indicates a basic model to obtain RR, For example, the logistics regression (Logit), log‐binomial regression (Logb), probit regression (Probit), and contingency table (CT). RR, relative risk.

Figure S2. Boxplot of the relative risk estimates based on data simulated from the logistic model (top) and the log‐binomial model (bottom) with six balancing methods (no balancing, M‐F, M, MA, SW‐F, SW). M, propensity score matching; F, full covariates adjusted; without F indicates no covariate adjustment; A, weight adjustment; SW, stabilized propensity score weighting.

Figure S3. The subgroup rates of the 15 covariates comparing control and case (Case condition II, vaccine years 1 and 2) for MEPS panels 5 (starting in 2000)–20 (starting in 2015), United States, 2000‐2016. Equation Section (Next) Due to the MEPS data design, the race group “Amer. Indian, AK Native, or mult. races” does not exist for Panel 5. Chronic bronchitis status and cancer status are available after panel 12 (starting from 2007).

AK Native, Alaska Native; Amer. Indian, American Indian; MEPS, medical expenditure panel survey; mult. races, multiple races.

Figure S4. The RR estimates (log‐transformed) of influenza infection between case (vaccinated) and control (not vaccinated) under Case conditions I (vaccine year 1) and II (vaccine years 1 and 2) and ranges I–IV for MEPS panels 5–20 (United States, 2000–2016) with only 13 variables (variables chronic bronchitis status and cancer status are removed in the analysis). The dashed lined with triangle points is the RR estimate using the original data. The solid line with round points is the RR estimates by the CTSW model. The filled points indicate a significant difference from RR = 1 (significance level = 0.05) for original and balanced data, respectively. RR, relative risk.

Figure S5. A (top plot): VE (1‐RR) estimates and 95% confidence intervals using MEPS data (solid line), United States, 2000–2016 and VE from CDC (dashed line), United States, 2004–2016; B (middle plot): The mean flu‐incurred mean expenditure per person for overall (solid line), vaccinated (dotted line), and not‐vaccinated groups (dashed line), United States, 2000–2016; C (bottom plot): The mean expenditure difference between the treatment and control groups (dashed line, right‐side y‐axis) and VE estimate using MEPS data (solid line, left‐side y‐axis), United States, 2000–2016. All analysis based on Case conditions II (vaccine years 1 and 2) and ranges I for MEPS panels 5 (starting in 2000)—20 (starting in 2015) with only 13 variables (variables chronic bronchitis status and cancer status are removed in the analysis). RR, relative risk, VE, vaccine effectiveness.

APPENDIX A.

A.1. Derivation of the IF in (2)

Let $s_i\left(i=1,\dots ,q\right)$ denote a scalar quantity and $S={\prod }_{i=1}^q{s}_i$. Based on Equation (1), we have $\text{IF}\left(\log S\right)=\frac{\partial \log S}{\partial S}\text{IF}\left(S\right)$, which is equivalent to $\text{IF}\left({\sum }_{i=1}^q\log s_i\right)=\frac{\text{IF}\left(S\right)}{S}$. ¹⁸ This gives rise to

$$\text{IF}\left(S\right)=S\left(\sum _{i=1}^q\frac{\text{IF}\left(s_i\right)}{s_i}\right).$$ (A1)

The RR is a function of $\overline{A_1},\overline{A_2},\overline{B_1},\overline{B_2}$, that is, $\mathrm{RR}=f\left(\overline{A_1},\overline{A_2},\overline{B_1},\overline{B_2}\right)$. Based on (A1), we evaluate the IF of RR as

$$\text{IF}\left({\mathrm{RR}}_i\right)=\frac{\overline{A_1}\overline{B_2}}{\overline{A_2}\overline{B_1}}\left(\frac{\text{IF}\left(\overline{A_1}\right)}{A_1}+\frac{\text{IF}\left(\overline{B_2}\right)}{B_2}-\frac{\text{IF}\left(\overline{B_1}\right)}{B_1}-\frac{\text{IF}\left(\overline{A_2}\right)}{A_2}\right)=\frac{\overline{A_1}\overline{B_2}}{\overline{B_1}\overline{A_2}}\left(\frac{A_{1i}}{\overline{A_1}}+\frac{B_{2i}}{\overline{B_2}}-\frac{B_{1i}}{\overline{B_1}}-\frac{A_{2i}}{\overline{A_2}}\right).$$ (A2)

A.2. Derivation of the IF in (3)

Suppose that a function to maximize is $G\left(\mathbf{\beta }\right)$ (e.g., the log‐likelihood function to estimate $\mathbf{\beta }$), that requires to solve $E\left[g\left(\mathbf{\beta }\right)\right]=0$, where $g\left(\mathbf{\beta }\right)$ is the derivative of $G\left(\mathbf{\beta }\right)$ with respect to $\mathbf{\beta }$. Following the standard approach of IF for an implicit function, ¹⁸ , ⁵¹ the distribution contaminated by $\mathrm{ϵ}$ produces an equation $\left(1-\mathrm{ϵ}\right)E\left[g\left(\mathbf{\beta }\right)\right]+\mathrm{ϵ}g\left(\mathbf{\beta }\right)=0$. Taking the derivative of the equation with respect to $\mathrm{ϵ}$ produces the equation: $-E\left[g\left(\mathbf{\beta }\right)\right]+\left(1-\mathrm{ϵ}\right)E\left[{▽}_{\mathbf{\beta }}g\left(\mathbf{\beta }\right)\right]\frac{\partial \mathbf{\beta }}{\partial ϵ}=-g\left(\mathbf{\beta }\right)-ϵ{▽}_{\mathbf{\beta }}g\left(\mathbf{\beta }\right)\frac{\partial \mathbf{\beta }}{\partial ϵ}$ (${\nabla }_{\mathbf{\beta }}$ is the vector derivative with respect to $\mathbf{\beta }$). Letting $\mathrm{ϵ}=0$ and solving for $\partial \mathbf{\beta }/\partial ϵ$ gives the IF of $\mathbf{\beta }$ in the following:

$$\text{IF}\left(\mathbf{\beta }\right)=E{\left[-{▽}_{\beta }g\left(\mathbf{\beta }\right)\right]}^{-1}g\left(\mathbf{\beta }\right).$$ (A3)

For logistic regression, the maximum‐likelihood estimator (MLE) of model coefficient $\mathbf{\beta }$ satisfies

$$g\left(\widehat{\mathbf{\beta }}\right)=\sum _{i=1}^n{\mathbf{v}}_i\left(y_i-\mu \left({\mathbf{v}}_i^{\prime }\widehat{\mathbf{\beta }}\right)\right)=0,$$ (A4)

where $y_i$ and ${\mathbf{v}}_i$ are the binary outcome and the vector of all covariates, respectively, and ${\mu }_L\left({\mathbf{v}}_i^{\prime }\widehat{\mathbf{\beta }}\right)=\exp \left({\mathbf{v}}_i^{\prime }\widehat{\mathbf{\beta }}\right){\left(1+\exp \left({\mathbf{v}}_i^{\prime }\widehat{\mathbf{\beta }}\right)\right)}^{-1}$. Using (A3), we have ${\text{IF}}_L{\left(\widehat{\mathbf{\beta }}\right)}_i={\left(-\frac{1}{n}{\sum }_{i=1}^n{\mathbf{v}}_i{\mathbf{v}}_i^{\prime }{\mu }_L\left({\mathbf{v}}_i^{\prime }\widehat{\mathbf{\beta }}\right)\left(1-{\mu }_L\left({\mathbf{v}}_i^{\prime }\widehat{\mathbf{\beta }}\right)\right)\right)}^{-1}{\mathbf{v}}_i\left(y_i-{\mu }_L\left({\mathbf{v}}_i^{\prime }\widehat{\mathbf{\beta }}\right)\right)$. Now, using (1), where $f\left(T\right)=R_L=\frac{1+e^{-\left({\beta }_0+{\beta }_2{Z}_1+\cdots +{\beta }_{k+1}{Z}_k\right)}}{1+e^{-\left({\beta }_0+{\beta }_1+{\beta }_2{Z}_1+\cdots +{\beta }_{k+1}{Z}_k\right)}}$, we obtain $\text{IF}{\left(R_L\right)}_i$ in (3).

Tian M, Yu J, Lillvis DF, Vexler A. Influence function methods to assess the effectiveness of influenza vaccine with survey data. Health Serv Res. 2022;57(1):200‐211. doi: 10.1111/1475-6773.13895