Identification of the Optimal Treatment Regimen in the Presence of Missing Covariates

Summary

Covariates associated with treatment-effect heterogeneity can potentially be used to make personalized treatment recommendations towards best clinical outcomes. Methods for treatment-selection rule development that directly maximize treatment-selection benefits have attracted much interest in recent years, due to the robustness of these methods to outcome modeling. In practice, the task of treatment-selection rule development can be further complicated by missingness in data. Here we consider the identification of optimal treatment-selection rules for a binary disease outcome when measurements of an important covariate from study participants are partly missing. Under the missing at random assumption, we develop a robust estimator of treatment-selection rules under the direct-optimization paradigm. This estimator targets the maximum selection benefits to the population under correct specification of at least one mechanism from each of the two sets — missing data or conditional covariate distribution, and treatment assignment or disease outcome model. We evaluate and compare performance of the proposed estimator with alternative direct-optimization estimators through extensive simulation studies. We demonstrate the application of the proposed method through a real data example from an Alzheimer’s disease study for developing covariate combinations to guide the treatment of Alzheimer’s disease.

1 ∣. INTRODUCTION

In clinical settings, heterogeneity may exist among individual responses to treatment or prevention of a targeted disease. For example, in HIV vaccine research, a vaccine’s protective effect can vary with host genetic factors such as Fc-receptor genotype¹. In trials of potential Alzheimer’s disease treatments, response to treatment can also be affected by demographics or lab-based baseline tests². These demographic variables, genetic factors, and/or biomarkers associated with treatment-effect heterogeneity, if identified, can be used to recommend treatment to individuals who will benefit from it and avoid unnecessary treatment (and potentially even harmful treatment effects) of others.

Traditionally, the development of treatment-selection rules relies on modeling the disease outcome as a function of treatment and covariates. The active treatment is subsequently recommended to those whose predicted disease risk is reduced if receiving treatment relative to if left untreated^3,4,5,6,7. While risk modeling approaches have the advantage of providing individual risk estimates, corresponding treatment-selection performance relies critically on proper modeling of disease risk and can be suboptimal when the risk model is misspecified. In recent years, an alternative framework that directly optimizes the treatment-selection benefit to the population has gained much attention and has shown promise in personalized medicine with performance robust to risk modeling. For example, Zhang et al⁸ proposed finding marker combinations that optimize the empirical or double-robust estimates of population mean outcome within a pre-specified parametric class. Zhang et al⁹ phrased the problem into a weighted optimization problem using a zero-one loss for all estimates of the population mean of potential outcomes. Additional cost due to treatment such as side effect and monetary cost can similarly be incorporated into this weighted optimization problem¹⁰. Other authors have proposed alternative optimization algorithms by approximating the 0-1 loss in the estimated objective function with surrogate losses such as exponential loss, hinge loss, or ramp loss^10,11,12,13. These methods were all developed assuming availability of data representative of the target population.

In practice, however, covariates for developing treatment-selection rules might be partly missing, with the likelihood of missingness depending on study design or other random mechanisms. Limited research to date has been devoted to accommodating missing data in developing treatment-selection rules, especially under the direct optimization framework. One relevant missing data problem was presented in Shortreed et al¹⁴, in which the authors investigated a problem setting of comparing sequential dynamic treatment regimens in the sequential multiple assignment randomized trials, where multiple imputation methods were used to handle missing data when estimating mean response in each treatment regimen using regression based methods. In this paper, we consider the identification of covariate-based optimal treatment-selection rules under the direct optimization framework when an important covariate measurement is partly missing from study participants. Under a missing at random (MAR) mechanism, to effectively account for missingness of the covariate, we consider an augmented inverse probability weighting strategy that combines the modeling of the covariate missing mechanism with the modeling of the covariate distribution to achieve a double-robust estimation of the targeted objective function, which is then optimized to derive the optimal treatment-selection rules. The idea of augmented inverse probability weighting strategy to robustly handle missing data was originally proposed by Robins et al¹⁵ and has been previously utilized to evaluate causal exposure effects in the presence of missing exposure¹⁶, confounders, or disease status¹⁷. In the field of treatment selection, it has also been adopted in the direct optimization paradigm to accommodate missing potential outcomes if assigned a particular treatment⁹. However, there has not been any research about its use in this context, i.e. when covariates for developing optimal treatment-regimens are subject to missingness. Here we utilize this strategy to handle both missingness of covariates and missingness of potential outcomes. We develop a robust estimator for treatment-selection rules that targets the maximum selection benefits to the population under correct specification of at least one mechanism from each of the two sets — missing data probability or conditional covariate distribution, and treatment assignment or disease outcome model. The estimator can handle not only missingness of a single covariate but also missingness of a set of covariates that are missing together in study participants. Through extensive simulation studies, we demonstrate advantages of the proposed estimator compared to alternative direct-optimization estimators that rely on correct modeling of missing data probability and alternative estimators that simply discard data with missing covariate measurements. We demonstrate the application of the proposed method through a real data example from an Alzheimer’s disease study for developing covariate-based rules to guide the treatment of Alzheimer’s disease.

2 ∣. METHODS

2.1 ∣. Problem Setting

We consider a problem setting for identifying optimal treatment-selection rules for an unfavorable binary clinical outcome, which we refer to as disease, although the developed methods work for continuous outcomes as well. Suppose in a cohort of individuals or a randomized trial, we have information available for a binary disease outcome Y, with values 1 and 0 indicating disease and no disease, respectively, and information for a binary treatment indicator T, with values 1 and 0 standing for treated and untreated, respectively. We also let Y(1) and Y(0) indicate the potential outcome of Y if receiving or not receiving the treatment, respectively.

Assume there is a vector of covariates X with dimension p, based on which we are interested in developing a treatment-selection rule. Let A(X) be a function of X that indicates the covariate-based treatment-selection rule, which recommends to treat those individuals with covariates X = x such that A(x) = 1 and not to treat individuals with A(x) = 0. In this paper we focus on identifying an optimal rule based on thresholding of a linear combination of X, i.e.

$$A(X)=I\{f(X)>0\}\text{with}f(X)={\xi }_0+{\xi }_1^{T}X$$ (1)

that minimizes the disease rate in the population if the treatment-selection rule is applied to the population, namely E[Y{A(X)}], where Y{A(X)} = Y(1)I {A(X) = 1} + Y(0)I {A(X) = 0} is the potential outcome that would be observed if a randomly chosen subject from the population were to be assigned treatment according to A(X). But note the proposed methods also apply in general when f(X) is a nonlinear combination of X.

Among the set of covariates X, suppose there is an important covariate X₂ that is partially missing, while the rest of the covariates, denoted as X₁, are fully observed among all individuals. Examples of X₁ are demographic variables that are easily obtained from every individual in a study, whereas X₂ can be a biomarker that is expensive to measure or a lab test that is expensive to perform, with its missingness caused by sampling design or some other random mechanism such that the probability of observing X₂ depends on other covariates, treatment status, and/or the clinical outcome. Let R be the binary indicator of observing X₂. The data available are then n i.i.d. copies of O_i = (Y_i, T_i, X_1i, R_i, X_2i R_i), where i = 1, …, n. Next we describe the assumptions and models used in our methods of deriving an optimal treatment-selection rule.

2.2 ∣. Assumptions and Models

We make the following assumptions.

(A1) Stable unit treatment value (SUTVA)¹⁸ and consistency: Y(0) and Y(1) of one subject are independent of the treatment assignments of other subjects, and given the treatment a subject actually received, a subject’s potential outcomes equal the observed outcomes;

(A2) Ignorable treatment assignment conditional on covariates, or no unmeasured confounding: T ⊥ Y(t)∣X, t = 0, 1;

(A3) Positivity assumption for treatment: 0 < P(T = 1∣X) < 1 with probability 1;

(A4) Missing at random assumption with respect to X₂: R ⊥ X₂∣X₁, Y, T;

(A5) Positivity assumption with respect to observation of X₂: P(R = 1∣X₁, Y, T) > 0 with probability 1;

(A6) Non-exceptional law: P{f(x) = 0} = 0 for all x that occur with a positive probability.

When there is no missingness in X₂, assumptions (A1), (A2) and (A3) allow identifiability of the objective function E[Y{A(X)}], which we want to minimize when deriving optimal treatment-selection rules. Specifically, as previously pointed out^8,9, Y{A(X)} is observed and equals to observed Y when A(X) = T and is otherwise unobserved. Let C = I{A(X) = T} be the indicator of observing Y{A(X)}, then P(C = 1∣Y{A(X)}, X) = P(C = 1∣X). Based on an inverse probability weighting (IPW) idea¹⁹ to handle missingness in Y{A(X)}, we have

$$E[Y\{A(X)\}]=E\left[\frac{Y\{A(X)\}C}{P(C=1\mid X)}\right]=E\left[\frac{TA(X)Y}{P(T=1\mid X)}+\frac{(1-T)\{1-A(X)\}Y}{P(T=0\mid X)}\right].$$

In the presence of missing X₂, assumptions (A4) and (A5), together with (A1)-(A3), allow identifiability of E[Y{A(X)}] based on the observed data, given that

$$E[Y\{A(X)\}]=E\left\{\frac{R}{P(R=1\mid X_1,Y,T)}\left[\frac{TA(X)Y}{P(T=1\mid X)}+\frac{(1-T)\{1-A(X)\}Y}{P(T=0\mid X)}\right]\right\},$$

again with IPW to address missingness in X₂. Assumption (A6) ensures the existence of a unique minimizer of E[Y{A(X)}] to avoid non-regular estimators of treatment-selection rules²⁰.

Next we list the working models that are utilized in the methods we propose.

1. The propensity score (PS) model, which is a model of the treatment probability conditional on covariates P(T = 1∣X; α), with α the corresponding parameter;

2. The outcome regression (OR) model, which is a model of the outcome Y conditional on covariates and treatment P(Y = 1∣X, T; β), with β the corresponding parameter;

3. The missing data (MD) model, which is a model for observing X₂ conditional on other covariates, treatment, and outcome, P(R = 1∣X₁, T, Y; γ), with γ the corresponding parameter;

4. The calibration (CL) model, which is a model for distribution of X₂ conditional on other covariates, treatment, and outcome, P(X₂∣X₁, T, Y; η), with η the corresponding parameter.

Next, we consider methods for deriving robust treatment-selection rules under the paradigm of direct optimization of the targeted objective function, which is the disease rate in the population if applying the treatment-selection rule. We start by reviewing this paradigm when there is no missingness in covariates.

2.3 ∣. Review of Direct Optimization Framework Without Covariate Missingness

Towards the objective of minimizing E[Y{A(X)}], the direct-optimization approaches^8,11 pre-specify a class of rules like in (1), derive a consistent estimate of E[Y{A(X)}] as a function of A(X), and then estimate the optimal rule as the minimizer of this consistent estimate of E[Y{A(X)}]. Two of the most commonly used forms for W are derived from the inverse-probability weighed (IPW) and the augmented inverse-probability weighted (AIPW) estimators of E[Y{A(X)}] for addressing missing potential outcomes^8,9. As mentioned in section 2.2, the estimation of E[Y{A(X)}] can be viewed as a missing data problem with respect to Y{A(X)}, where Y{(A(x)} is observed and equals Y only among subjects with A(x) = T and is otherwise missing. The expectation E[Y{A(X)}] can thus be estimated using an IPW approach where the contribution of each individual in the study is weighted by the probability of observing Y{A(X)}, i.e. P{T = A(X)∣X}. The consistency of the IPW estimator of E[Y{A(X)}] requires correct specification of the model P(T∣X), i.e. the PS model. An IPW-type estimator of E[Y{A(X)}]^8,9 can be constructed as

$$\tilde{E}[Y\{A(X)\}]=\frac{1}{n}\sum _{i=1}^n\frac{Y_i{T}_{i}A(X_i)}{P(T=1\mid X_i;\widehat{\alpha })}+\frac{Y_i(1-T_i)\{1-A(X_i)\}}{P(T=0\mid X_i;\widehat{\alpha })},$$ (2)

with P(T = t∣X; $\widehat{\alpha }$) the estimate of P(T = t∣X; α). Researchers have also proposed minimization of a double-robust AIPW estimator for E[Y{A(X)}]^8,9, which augments the IPW estimator (2) with an extra term that involves modeling of the outcome using the OR model:

$$\begin{gathered} \widehat{E}[Y\{A(X)\}] \\ =\tilde{E}[Y\{A(X)\}]-\frac{1}{n}\sum _{i=1}^n\frac{I\{T_i=A(X_i)\}-P\{T_i=A(X_i)\mid X_i\}}{P\{T_i=A(X_i)\mid X_i\}}[{\mathit{Risk}}_0(X_i,\widehat{\beta })\{1-A(X_i)\}+{\mathit{Risk}}_1(X_i,\widehat{\beta })A(X_i)], \end{gathered}$$ (3)

where $Ris{k}_0(X,\widehat{\beta })$ and $Ris{k}_1(X,\widehat{\beta })$ are estimates of Risk₀(X, β) = P(Y = 1∣X, T = 0; β) and Risk₁(X, β) = P(Y = 1∣X, T = 1; β). The AIPW estimator of E[Y{A(X)}] is consistent if either the PS or the OR model is correctly specified^8,9.

The IPW and AIPW estimators of treatment-selection rules can be obtained by minimizing corresponding IPW and AIPW estimators of E[Y{A(X)}] in (2) and (3). Zhang et al⁸ adopted grid search and genetic algorithms for optimization. Zhang et al⁹ and Zhao et al¹¹ reformulated the optimization problem as a classification problem. Equivalently, as shown in Huang and Fong¹⁰, solving for f(X) in (1) (or minimization of (2) and (3)) can be framed as the problem of minimizing a weighted sum of 0-1 loss, i.e.,

$$\widehat{f}(X)={\mathrm{argmin}}_f\frac{1}{n}\sum _{i=1}^n{W}_{i}I\{f(X_i)\le 0\},$$ (4)

where W is a subject-specific weight that can be computed from the observed data. In this paper, we follow the convention in Huang and Fong¹⁰ by identifying treatment-selection rule as a problem of minimizing a weighted sum of 0-1 loss. This belongs to a general optimization problem that has also been studied in other problem settings such as the optimization of the area under the receiver operating characteristics curve²¹. As shown in Supplementary Appendix A, finding f(X) to minimize the IPW estimator of E[Y{A(X)}] is equivalent to finding solution in (4) with

$$W_i=-\left\{\frac{T_i{Y}_i}{P(T=1\mid X_i;\widehat{\alpha })}-\frac{(1-T_i)Y_i}{P(T=0\mid X_i;\widehat{\alpha })}\right\}.$$ (5)

In this paper, we refer to this W as the IPW weight. In other work, it has also been referred to as the outcome weight¹¹. Finding f(x) to minimize the AIPW estimator of E[Y{A(X)}] is equivalent to finding the solution of (4) with

$$\begin{gathered} W_i=-\frac{T_i{Y}_i}{P(T=1\mid X_i;\widehat{\alpha })}+\frac{(1-T_i)Y_i}{P(T=0\mid X_i;\widehat{\alpha })} \\ +\frac{T_i-P(T=1\mid X_i;\widehat{\alpha })}{P(T=1\mid X_i,\widehat{\alpha })}{\mathit{Risk}}_1(X_i;\widehat{\beta })+\frac{T_i-P(T=1\mid X_i;\widehat{\alpha })}{1-P(T=1\mid X_i,\widehat{\alpha })}{\mathit{Risk}}_0(X_i;\widehat{\beta }). \end{gathered}$$ (6)

In this paper, we refer to this weight as the AIPW weight.

Various algorithms could be used to solve the minimization problem in (4). For example, grid search could be performed when the dimension of covariates is small. When the number of covariates gets larger, more computationally efficient algorithms are needed. For example, Zhang et al⁸ and Jiang et al²² used the genetic algorithm as implemented in the R rgenoud package²³ to solve the optimization problem. Other authors have used algorithms that approximate the 0-1 loss with a surrogate loss, such as the hinge loss (as in the support vector machine), the ramp loss, or the binomial deviance function^10,11,12,24. As will be presented later, for numerical studies in this paper, we use grid search when combining two covariates and use the weighted logistic regression when several covariates are combined. The weighted logistic regression is a surrogate-loss based algorithm: it approximates the misclassification error with a binomial deviance loss function²⁵.

2.4 ∣. Direct Minimization Methods in the Presence of a Missing Covariate

For this paper we consider the settings where a covariate X₂, potentially important for building a treatment-selection rule, is partially missing. Again we want to construct a consistent estimate of E[Y{A(X)}] such that the treatment-selection rule A(X) can be estimated by minimizing this estimated objective function. Equivalently, since the minimization problem can be phrased as minimization of the sum of the weighted 0-1 loss W ∑ I(f(X) ≤ 0)/n as in (4), our goal is to recover the sum of weighted loss in the pseudo-population that does not have missingness in X₂ and use this quantity as the targeted objective function for minimization. The computation of the IPW or AIPW weights W described earlier in (5) and (6) for settings without missing X₂ requires estimation of the propensity score (PS) model parameter α and/or the outcome regression (OR) model parameter β; in the presence of missing X₂, one would also need to estimate the missing data (MD) model and/or the calibration (CL) model. Next, we describe how these parameters will be computed in the presence of missing covariates, and then propose our estimators of optimal treatment-selection rules.

2.4.1 ∣. Estimation of parameters in various Models in the Presence of a Missing Covariate

First, consider the estimation of the parameter γ in the MD model. It can be achieved by maximizing the likelihood of observing X₂, i.e.,

$$L_{\gamma }=\prod _{i=1}^{n}P(R=1\mid X_{1i},Y_i,T_i;\gamma {)}^{R_i}\{1-P(R=1\mid X_{1i},Y_i,T_i;\gamma ){\}}^{1-R_i}$$

or solving the corresponding score equation. A natural choice for the MD model P(R = 1∣X₁, Y, T; γ) is the logistic regression model. We let $\widehat{\gamma }$ be the corresponding maximum likelihood estimator (MLE) of γ.

Next, we consider the estimation of the parameter α in the PS model. When there is no missing data, α can be estimated by maximizing the likelihood of observed treatment,

$$L_{\alpha }=\prod _{i=1}^{n}P(T=1\mid X_i;\alpha {)}^{T_i}\{1-P(T=1\mid X_i;\alpha ){\}}^{1-T_i},$$

or equivalently, estimated as the solution of the corresponding score equation. Again, a natural choice for the PS model P(T = 1∣X; α) is the logistic regression model. When there is missingness in X₂, a natural strategy to estimate α is to maximize a weighted likelihood, which weights the contribution of each participant to the likelihood by the inverse of the probability of observing X₂, i.e.,

$$W{L}_{\alpha }=\prod _{i=1}^{n}P(T=1\mid X_i;\alpha {)}^{v_i{T}_i{R}_i}\{1-P(T=1\mid X_i;\alpha ){\}}^{v_i(1-T_i)R_i},$$

where $v_i=1∕P(R=1\mid X_{1i},Y_i,T_i;\widehat{\gamma })$. That is, we find α by solving the inverse probability weighted (IPW) score equation

$$\sum _{i=1}^n\frac{R_i}{P(R=1\mid X_{1i},Y_i,T_i;\widehat{\gamma })}{U}_{\alpha i}\equiv 0,$$ (7)

where

$$U_{\alpha i}=\frac{\partial \log L_{\alpha i}}{\partial \alpha }=T_i\frac{\partial P(T=1\mid X_i;\alpha )∕\partial \alpha }{P(T=1\mid X_i;\alpha )}-(1-T_i)\frac{\partial P(T=1\mid X_i;\alpha )∕\partial \alpha }{1-P(T=1\mid X_i;\alpha )}.$$

We call the corresponding α estimator ${\widehat{\alpha }}_{IPW}$. Note that under the correctly specified PS model, ${\widehat{\alpha }}_{IPW}$ will lead to a consistent estimator of P(T = 1∣X) only if the MD model is also correctly specified.

One can consider further improving the robustness of the α estimate by augmenting the IPW-score equation (7) with an additional term that models the distribution of X₂ conditional on X₁, Y, and T, i.e. the calibration (CL) model, following ideas discussed in earlier work^15,16,26. That is, we solve the following augmented inverse-probability weighted score equation:

$$\sum _{i=1}^n\left[\frac{R_i}{P(R=1\mid X_{1i},Y_i,T_i;\widehat{\gamma })}{U}_{\alpha i}-\left\{\frac{R_i}{P(R=1\mid X_{1i},Y_i,T_i;\widehat{\gamma })}-1\right\}\widehat{E}(U_{\alpha i}\mid X_{1i},Y_i,T_i)\right]\equiv 0.$$ (8)

Under the MAR assumption of X₂, P(X₂∣X₁, Y, T) = P(X₂∣X₁, Y, T, R = 1). Thus, one can obtain $\widehat{E}(U_{\alpha }\mid X_1,Y,T)=E(U_{\alpha }\mid X_1,Y,T;\widehat{\eta })$ using samples with X₂ observed. For discrete X₁, P(X₂∣X₁, Y, T) can be estimated empirically; when X₁ includes continuous components, we adopt the semiparametric location-scale model²⁷ to estimate P(X₂∣X₁, Y, T), as will be shown later in Section 2.4.2. Note that with the correctly specified PS model, (8) is asymptotically unbiased if either the MD or the CL model is correctly specified. We use ${\widehat{\alpha }}_{AIPW}$ to denote the corresponding solution of (8).

Now we consider the estimation of the parameter β in the OR model. When there is no missing data, β can be estimated by maximizing the likelihood of Y conditional on covariates and treatment,

$$L_{\beta }=\prod _{i=1}^{n}P(Y=1\mid X_i,T_i;\beta {)}^{Y_i}\{1-P(Y=1\mid X_i,T_i;\beta ){\}}^{1-Y_i},$$

or equivalently, estimated as the solution of the corresponding score equation. Again, a natural model for P(Y = 1∣X, T; β) is the logistic regression model. When there is missingness in X₂, again one can estimate β by maximizing the weighted likelihood,

$$W{L}_{\beta }=\prod _{i=1}^{n}P(Y=1\mid X_i,T_i,\beta {)}^{v_i{Y}_i{R}_i}\{1-P(Y=1\mid X_i,T_i,\beta ){\}}^{v_i(1-Y_i)R_i},$$

or equivalently solve the inverse probability weighted (IPW) score equation

$$\sum _{i=1}^n\frac{R_i}{P(R=1\mid X_{1i},Y_i,T_i;\widehat{\gamma })}{U}_{\beta i}\equiv 0,$$ (9)

with

$$U_{\beta i}=\frac{\partial \log L_{\beta i}}{\partial \beta }=Y_i\frac{\partial P(Y=1\mid X_i,T_i;\beta )∕\partial \beta }{P(Y=1\mid X_i,T_i;\beta )}-(1-Y_i)\frac{\partial P(Y=1\mid X_i,T_i;\beta )∕\partial \beta }{1-P(Y=1\mid X_i,T_i;\beta )}.$$

We call the corresponding β estimator ${\widehat{\beta }}_{IPW}$. Note that under the correctly specified OR model, ${\widehat{\beta }}_{IPW}$ will lead to a consistent estimator of P(Y = 1∣X, T) only if the MD model is also correctly specified.

Similar to the estimation of α, one can further improve the robustness of the β estimate by augmenting the IPW-score equation (9) with an extra term that relies on the CL model. That is, we solve the following augmented inverse-probability weighted score equation:

$$\sum _{i=1}^n\left[\frac{R_i}{P(R=1\mid X_{1i},Y_i,T_i;\widehat{\gamma })}{U}_{\beta i}-\left\{\frac{R_i}{\widehat{P}(R=1\mid X_{1i},Y_i,T_i;\widehat{\gamma })}-1\right\}\widehat{E}(U_{\beta i}\mid X_{1i},Y_i,T_i)\right]\equiv 0.$$ (10)

With the correctly specified OR model, (10) is asymptotically unbiased if either the MD or the CL model is also correctly specified. We use ${\widehat{\beta }}_{AIPW}$ to denote the corresponding solution of (10).

2.4.2 ∣. Identification of the Optimal Treatment Regimen in the Presence of a Missing Covariate

Now we consider estimation of the optimal treatment-selection rule under MAR of X₂, where the missingness in X₂ is handled using either the IPW or AIPW technique.

(I). Using IPW to account for missingness in X₂

First, to extend the minimization problem in (4) to the missing covariate scenario, it is natural to adopt the inverse probability weighted strategy to account for missingness in X₂ by weighting the contribution of each individual to the targeted objective function to be minimized by the inverse of the probability of observing X₂. That is, for each type of individual-specific weight W as specified earlier in (5) and (6), we compute

$$\frac{1}{n}\sum _{i=1}^n\frac{R_i}{P(R=1\mid X_{1i},Y_i,T_i;\widehat{\gamma })}{W}_{i}I\{f(X_i)\le 0\}.$$ (11)

Note that W is a function of the parameter estimates in the PS and/or OR model. Allowing somewhat loose use of notation, let W_∞ indicate the corresponding W when the estimators of the PS and OR models are replaced by their probability limits. It is straightforward to show that (11) converges to E[W_∞I{f(X) ≤ 0}] if the MD model is correctly specified. When computing the individual-specific weight W, we can use either ${\widehat{\alpha }}_{IPW}$ or ${\widehat{\alpha }}_{AIPW}$ for estimation of the IPW-weight W in (5) and use either $\{{\widehat{\alpha }}_{IPW},{\widehat{\beta }}_{IPW}\}$ or $\{{\widehat{\alpha }}_{AIPW},{\widehat{\beta }}_{AIPW}\}$ for estimation of the AIPW weight W in (6). We then derive the optimal treatment-selection rule by minimizing the objective function (11) with respect to f.

Let $\widehat{\xi }=\{{\widehat{\xi }}_0,{\widehat{\xi }}_1\}$ be the minimizer of (11) for $f(X)={\xi }_0+{\xi }_1^{T}X$. When the MD model is correctly specified, the convergence of $\widehat{\xi }$ to the minimizer of $E\left[W_{\infty }I({\xi }_0+{\xi }_1^{T}X)\le 0\right]$ can be shown under regularity conditions. A sketch of the proof is included in Web Supplementary Appendix B. We refer to the corresponding estimator of the treatment-selection rule as IPW(MD)-IPW(PS) if the IPW weight W in (5) is used in (11) and as IPW(MD)-AIPW(PS) if the AIPW weight W in (6) is used in (11). As mentioned in section 2.3, we have argmin_fE[W_∞I{f(X) ≤ 0} = argmin_fE[Y{A(X)}] under either one of the following two conditions: 1) the IPW weight W in (5) with consistent α estimate is used in (11) (i.e. if ${\widehat{\alpha }}_{IPW}$ is used and MD and PS models are correctly specified, or if ${\widehat{\alpha }}_{AIPW}$ is used and “(MD or CL) and PS" models are correctly specified); or 2) the AIPW weight W in (6) is used with consistent α or β estimate (i.e. if $\{{\widehat{\alpha }}_{IPW},{\widehat{\beta }}_{IPW}\}$ is used and “MD and (PS or OR)" models are correctly specified, or if $\{{\widehat{\alpha }}_{AIPW},{\widehat{\beta }}_{AIPW}\}$ is used and “(MD or CL) and (PS or OR)" models are correctly specified). As a result, as summarized in Table 1, in the presence of missing X₂, the IPW(MD)-IPW(PS) estimator of the treatment-selection rule converges to the optimal linear rule that targets the minimization of population disease burden when both the MD and PS models are correctly specified; the IPW(MD)-AIPW(PS) estimator of the treatment-selection rule converges to the optimal linear rule when both the MD model and one of the PS and OR models are correctly specified.

TABLE 1.

Correct model specifications required for robust performance of treatment-selection rules for various direct optimization methods.

MD – PS	α	β	Correct Models Required
IPW(MD)–IPW(PS)	${\widehat{\alpha }}_{IPW}$		MD and PS
	${\widehat{\alpha }}_{AIPW}$		MD and PS
IPW(MD)–AIPW(PS)	${\widehat{\alpha }}_{IPW}$	${\widehat{\beta }}_{IPW}$	MD and (PS or OR)
	${\widehat{\alpha }}_{AIPW}$	${\widehat{\beta }}_{AIPW}$	MD and (PS or OR)
AIPW(MD)–IPW(PS)	${\widehat{\alpha }}_{AIPW}$		(MD or CL) and PS
AIPW(MD)–AIPW(PS)	${\widehat{\alpha }}_{AIPW}$	${\widehat{\beta }}_{AIPW}$	(MD or CL) and (PS or OR)

(II). Using AIPW to account for missingness in X₂

We next propose an estimator of the optimal treatment-selection rule with increased robustness in the presence of a missing covariate. Let q(X, Y, T) = W(X, Y, T)I{f(X) ≤ 0}. We propose to find the optimal f(X) by minimizing an augmented IPW estimate of E(q):

$$\begin{gathered} \widehat{E}(q)=\frac{1}{n}\sum _{i=1}^n\left[\frac{R_i}{P(R=1\mid X_{1i},Y_i,T_i;\widehat{\gamma })}q(X_i,Y_i,T_i)-\left\{\frac{R_i}{P(R=1\mid X_{1i},Y_i,T_i;\widehat{\gamma })}-1\right\}\widehat{E}\{q(X,Y,T)\mid X_{1i},Y_i,T_i\}\right] \\ =\frac{1}{n}\sum _{i=1}^n\left[\frac{R_i}{P(R=1\mid X_{1i},Y_i,T_i;\widehat{\gamma })}{W}_{i}I\{f(X_i)\le 0\}\right] \\ -\frac{1}{n}\sum _{i=1}^n\left(\left\{\frac{R_i}{P(R=1\mid X_{1i},Y_i,T_i;\widehat{\gamma })}-1\right\}\widehat{E}[WI\{f(X)\le 0\}\mid X_{1i},Y_i,T_i]\right), \end{gathered}$$ (12)

where $\widehat{E}\left\{q(X,Y,T)\mid X_1,Y,T\right\}$ is the estimate of the average of q(X, Y, T) conditional on the CL model of X₂∣X₁, Y, T. Note that $\widehat{E}(q)$ is consistent for E{W_∞I{f(X) ≤ 0}} if either the MD or the CL model is correctly specified. In particular, when the MD model is correctly specified,

$$E[R∕P(R=1\mid X_1,Y,T;\widehat{\gamma })]\to 1,$$

the second term in (12) is asymptotically negligible by a conditioning argument and the first term converges to E[W_∞I{f(X) ≤ 0}]. When the CL model is correctly specified, let γ_∞ be the product limit of $\widehat{\gamma }$, we have

$$\begin{gathered} \widehat{E}(q)=E\left[\frac{R}{P(R=1\mid X_1,Y,T;{\gamma }_{\infty })}q(X,Y,T)-\left\{\frac{R_i}{P(R=1\mid X_1,Y,T;{\gamma }_{\infty })}-1\right\}E\{q(X,Y,T)\mid X_1,Y,T\}\right]+o_p(1) \\ =E\left(\frac{R}{P(R=1\mid X_1,Y,T;{\gamma }_{\infty })}E[q(X,Y,T)-E\{q(X,Y,T)\mid X_1,Y,T\}\mid X_1,Y,T,R]\right) \\ +E[E\{q(X,Y,T)\mid X_1,Y,T\}]+o_p(1) \\ E[W_{\infty }I\{f(X)\le 0\}]+o_p(1). \end{gathered}$$

For maximum robustness, here we consider W to be the AIPW weights (6) and refer to the corresponding treatment-selection rule estimator as AIPW(MD)-AIPW(PS) (Table 1). It is important that one uses ${\widehat{\alpha }}_{AIPW}$ and ${\widehat{\beta }}_{AIPW}$ in computing W to ensure the robustness of W in the sense of argmin_fE[W_∞I{f(X) ≤ 0}] = argmin_fE[Y{A(X)}] when either the MD or CL model is correctly specified and either the PS or OR model is correctly specified. Consequently, the objective function (12) that AIPW(MD)-AIPW(PS) maximizes converges to a limit that is equal up to an additive constant (i.e., something that does not depend on f) to the population disease burden that can be achieved using the optimal f(X), when either the MD or CL model and either the PS or OR model are correctly specified.

As a comparative procedure, note that one can also construct an AIPW(MD)-IPW(PS) estimator with ${\widehat{\alpha }}_{AIPW}$ that utilizes AIPW to account for missinginess in X₂ and IPW to account for missingness in potential outcomes. Using similar argument as above, it is straightforward to see that the robust performance of this estimator requires correct specification of either the MD or the CL model and the PS model (Table 1).

The AIPW(MD)-AIPW(PS) estimator of the treatment-selection rule can again be solved by minimizing a weighted sum of the 0-1 loss function using an augmented dataset as follows. First, suppose a semiparametric location-scale model is used for estimating the CL model (e.g. when X₁ includes continuous component). Then our estimator can be derived based on the following steps.

• Step I. Estimate MD model parameter $\widehat{\gamma }$ as described in Section 2.4.1.

• Step II. Estimate distribution of X₂∣X₁, Y, T with a semiparametric location-scale model. That is, assume the cumulative distribution function (CDF) of X₂ conditional on X₁, Y, T is

  $$F(X_2\mid X_1=x_1,Y=y,T=t)=F_0\left(\frac{X_2-\mu (x_1,y,t)}{\sigma (x_1,y,t)}\right),$$

  with F₀ a baseline CDF, $\mu (X_1,Y,T)={\eta }_1^T{V}_1$, and $\log \{\sigma (X_1,Y,T)\}={\eta }_2^T{V}_2$, with V₁ and V₂ some specified functionals of (X₁, Y, T), e.g. (1, X₁, Y, T) or with X₁ replaced by its B-spline basis. Call the set with X₂ observed (or R = 1) the validation set $\mathcal{S}$. Let $m={\sum }_{i=1}^n{R}_i$ be the number of observations in $\mathcal{S}$. The estimators ${\widehat{\eta }}_1$ and ${\widehat{\eta }}_2$ are computed as the solutions to the following estimating equations

  $$\begin{gathered} \sum _{j\in \mathcal{S}}{R}_j{V}_{1j}(X_{2j}-{\eta }_1^T{V}_{1j})∕\text{exp}(2{\eta }_2^T{V}_{2j})=0 \\ \sum _{j\in \mathcal{S}}{R}_j{V}_{2j}\left\{(X_{2j}-{\eta }_1^T{V}_{1j}{)}^2-\text{exp}(2{\eta }_2^T{V}_{2j})\right\}∕\text{exp}(2{\eta }_2^T{V}_{2j})=0. \end{gathered}$$

  We obtain residuals e₁,… ,e_m, where $e_j=\left(X_{2j}-{\widehat{\eta }}_1^T{V}_{1j}\right)∕\text{exp}({\widehat{\eta }}_2^T{V}_{2j})$ for $j\in \mathcal{S}$.

• Step III. Estimate ${\widehat{\alpha }}_{AIPW}$ and ${\widehat{\beta }}_{AIPW}$ as described in Section 2.4.1 based on (8) and (10).

• Step IV. Construct the augmented dataset

  • Each observation i with R_i = 1 contributes W_i/P(R = 1∣X_1i, Y_i, T_i; $\widehat{\gamma }$)I {f(X_i ≤ 0)}
  • Each observation i is augmented m times with the j^th imputed new observation $X_{2ij}^{\star }=e_j\times \widehat{\sigma }(X_{1i},Y_i,T_i)+\widehat{\mu }(X_{1i},Y_i,T_i)$ for $j\in \mathcal{S}$, and contributes

    $$-\frac{1}{m}\left\{\frac{R_i}{P(R=1\mid X_{1i},Y_i,T_i;\widehat{\gamma })}-1\right\}{W}_{ij}^{\star }I\left\{f(X_{ij}^{\star })\le 0\right\},$$

    where $W_{ij}^{\star }$ is the (6) weights computed based on $X_{2ij}^{\star }$, X_1i, Y_i and T_i.

• Step V. Minimize the weighted sum of 0-1 loss based on the augmented dataset. That is, we solve for

$$\begin{gathered} f(x)={\mathrm{argmin}}_f\frac{1}{n}\sum _{i=1}^n[\frac{R_i{W}_i}{P(R=1\mid X_{1i},Y_i,T_i;\widehat{\gamma })}I\{f(X_i)\le 0\}\phantom{]} \\ -\left\{\frac{R_i}{P(R=1\mid X_{1i},Y_i,T_i;\widehat{\gamma })}-1\right\}\phantom{[}\frac{1}{m}\sum _{j\in \mathcal{S}}{W}_{ij}^{\star }I\{f(X_{ij}{)}^{\star }\le 0\}]. \end{gathered}$$

When X₁ is a discrete variable, we can use a similar algorithm with an empirical estimate of the CL model with the following modifications regarding Steps II and IV.

• Step II. Estimate the distribution of X₂∣X₁, Y, T empirically.

• Step IV. Construct the augmented dataset.

  • Each observation i with R = 1 contributes W_i/P(R = 1∣X_1i, Y_i, T_i; $\widehat{\gamma }$)I {f(X_i, ≤ 0)}
  • Call the set with X₂ observed the validation set. Let ${\mathcal{S}}_i$, be the validation set with X₁ = X_1i, Y = Y_i, T = T_i. Let m_i be the number of observations in ${\mathcal{S}}_i$. Each observation i is augmented m_i times with the j^th imputed new observation having $X_{2ij}^{\star }$ equal to the the j^th X₂ value in ${\mathcal{S}}_i$, and each augmented observation contributes

    $$-\frac{1}{m}\left\{\frac{R_i}{P(R=1\mid X_{1i},Y_i,T_i;\widehat{\gamma })}-1\right\}{W}_{ij}^{\star }I\{f(X_{ij}^{\star })\le 0\},$$

    where $W_{ij}^{\star }$ is the (6) weights computed based on $X_{2ij}^{\star }$, X_1i, Y_i and T_i.

2.5 ∣. Direct Minimization Methods when a Set of Covariates are Missing together

Previously we have proposed treatment-selection estimators in the presence of a missing covariate, but the methodology can be easily extended to allow for the missingness of a set of covariates when those covariates are observed or missing together from study participants. For example, the covariates of consideration might be several lab assays measured from blood that are missing together if a study participant does not contribute specimen for measurement. With a slight abuse of notation, let X₂ = {X₂₁, …, X_2K} be K-dimensional covariates that are always missing together and R = 1, 0 be the indicator of whether X₂ is measured. Corresponding CL model requires estimation of the joint distribution of {X₂₁,…, X_2K} conditional on X₁, Y, and T. The procedure described in Section 2.4.2 for estimating AIPW(MD)-AIPW(PS) can be extended by fitting semiparametric location-scale model of each individual covariate X_2k, k = 1,…, K conditional on X₁, Y, T in Step II. This leads to the estimators of the location and scale parameters for each component of X₂, as well as a K-dimensional vector of residuals from each participant in the validation set that captures the correlation structure among {X₂₁,…, X_2K}²⁸. These model estimators will then be used to impute new observations of X₂ (of dimension K) in Step IV (Details are presented in Supplementary Appendix C).

3 ∣. SIMULATION STUDIES

In this section, we conduct simulation studies to evaluate the performance of our proposed robust estimator of optimal treatment-selection rules in the presence of a missing covariate. We simulate $(X_1^{\ast },X_2)$ from a bivariate normal distribution with mean 0 and variance 1 each, with a correlation of 0.2. Let X₁ be the covariate without missingness and X₂ be the covariate with missingness. We consider both settings where X₁ is discrete $(X_1=I[X_1^{\ast }>0])$ and settings where X₁ is continuous $(X_1=X_1^{\ast })$. We simulate treatment T conditional on X₁, X₂ from a true PS model P₀(T = 1∣X₁, X₂), then simulate the clinical endpoint Y conditional on X₁, X₂ and T from a true OR model P₀(Y = 1∣X₁, X₂, T). Finally, we simulate the indicator R for observing X₂ from a true MD model P₀(R = 1∣X₁, Y, T).

We consider direct optimization methods to estimate an optimal linear treatment-selection rule of the form A(X) = I(ξ₀ + ξ₁X₁ + ξ₂X₂ > 0) that minimizes a weighted sum of zero-one loss. To derive our estimators we adopt the following working models:

• MD: logitP(R = 1∣X₁, Y, T) = γ₀ + γ₁X₁ + γ₂Y + γ₃T

• PS: $\log \text{it}P(T=1\mid X)={\alpha }_0+{\alpha }_1^{T}X$

• OR: $\log \text{it}P(Y=1\mid X,T)={\beta }_0+{\beta }_1^{T}X+{\beta }_{2}T+{\beta }_3^{T}XT$

• CL: for the calibration model, we estimate the distribution of X₂ conditional on X₁, Y, T empirically when X₁ is discrete and use a semiparametric location-scale model when X₂ is continuous, where the mean of X₂ is modeled as a function of the linear effect of Y and T and the natural cubic spline basis of X₁, with three knots at the 25%, 50%, and 75% percentiles.

We investigate different simulation settings allowing working models for the MD, PS, or OR models to be misspecified. In particular, the settings where working models for the MD, PS, and OR models are correctly specified have 1) logitP₀(R = 1∣X₁, T, Y) = 1.8 – 0.5X₁ for discrete or continuous X₁; 2) logitP₀(T = 1∣X₁, X₂) = α₀ + α₁X₁ + α₂X₂ with α₀, α₁, α₂ equal to {0, 0.2, −0.2} for discrete X₁ and equal to {−1, 1, 0} for continuous X₁; and 3) logitP₀(Y = 1∣X₁, X₂, T) = β₀ + β₁X₁+ β₂X₂ + β₃T + β₄X₁T + β₅X₂ T, with {β₀, β₁, β₂, β₃, β₄, β₅} equal to {0, −1, −1, 0.1, −2, −2} for discrete X₁ and equal to {0, −1, −1, 0.5, −2, −2} for continuous X₁. For settings with misspecification of the MD model, we consider logitP₀(R = 1∣X₁, T, Y) = 1.8 – 0.5X₁ – T – Y – 2TY; for misspecification of the PS model, we consider $\log \text{it}{P}_0(T=1\mid X_1^{\ast },X_2)={\alpha }_0+{\alpha }_1\mid X_1^{\ast }-X_2\mid$ with {α₀, α₁} equal to {0, 0.2} for discrete X₁ and equal to {−1, 1} for continuous X₁; for misspecification of the OR model, we consider $\log \text{it}{P}_0(Y=1\mid X_1,X_2,T)={\beta }_0+{\beta }_1{X}_1+{\beta }_2{X}_2+{\beta }_3{X}_2^2+{\beta }_{4}T+{\beta }_5{X}_{1}T+{\beta }_6{X}_{2}T$, with (β₀, β₁, β₂, β₃, β₄, β₅, β₆} equal to {0, −1, −1, −0.5, 0.1, −2, −2} for discrete X₁ and equal to {0, −1, −1, −0.5, 0.5, −2, −2} for continuous X₁.

We study treatment-selection performance of our proposed estimator that utilizes AIPW weights to account for both missinginess in X₂ and missingness in potential outcomes Y{A(X)}, with ${\widehat{\alpha }}_{AIPW}$ and ${\widehat{\beta }}_{AIPW}$, namely AIPW(MD)-AIPW(PS). We also analyze the performance of the other estimators listed in Table 1: estimators that use IPW to account for missingness in X₂ and use IPW/AIPW weights to account for missingness in potential outcomes based on either {${\widehat{\alpha }}_{IPW}$ and ${\widehat{\beta }}_{IPW}$} or {${\widehat{\alpha }}_{AIPW}$ and ${\widehat{\beta }}_{AIPW}$}, which we refer to as IPW(MD)-IPW(PS) and IPW(MD)-AIPW(PS), respectively, as well as the AIPW(MD)-IPW(PS) estimator that uses AIPW to account for missingness in X₂ and uses IPW to account for missingness in potential outcomes based on {${\widehat{\alpha }}_{AIPW}$ and ${\widehat{\beta }}_{AIPW}$}. For comparison, we also present naive approaches that discard missing data in X₂ and apply direct optimization methods to complete cases only, which use either IPW or AIPW to account for missing potential outcomes. We refer to these complete-case approaches as CC(MD)-IPW(PS) and CC(MD)-AIPW(PS), respectively. Grid search algorithm is used to solve the minimization problem in each setting. We conduct 5,000 Monte-Carlo simulations in each setting. In each simulation, treatment-selection rules $\widehat{A}(X)$ are estimated from a training data of size 300, 600, or 1,000, and then the corresponding $E\left[Y\{\widehat{A}(X)\}\right]$ are estimated as the average of ${\sum }_{i=1}^n\left[\widehat{A}(X_i)\left\{P_0(Y=1\mid X_{i}T=1)\right\}+\left\{1-\widehat{A}(X_i)\right\}\left\{P_0(Y=1\mid X_i,T=0)\right\}\right]∕n$ over 100 independently simulated test data sets of size 5,000 each. The performance of a particular method for deriving the treatment-selection rule is then estimated as the average of $E\left[Y\{\widehat{A}(X)\}\right]$ estimates over 5,000 Monte-Carlo simulations. In our simulation settings, the disease prevalence among treated individuals is smaller than among untreated individuals, so the best treatment-selection strategy in the absence of any covariate is to treat everyone. In Tables 2 and 3 and supplementary Tables 1-4, we present the performance of each method as the reduction in disease rate using the estimated optimal treatment-selection rule compared to the strategy of treating all, i.e. $100\times \left\{E\{Y(1)\}-E\left[Y\{\widehat{A}(X)\}\right]\right\}$. As a reference, we also present the reduction in disease rate using the true optimal linear rule.

TABLE 2.

Performance (Mean (SE)) of various direct optimization treatment-selection rule estimators (based on X₁ and X₂) in settings with discrete X₁ (n = 300).

			$100\times \left(E\{Y(1)\}-E\left[Y\{\widehat{A}(X)\}\right]\right)$
Model	α	β	OR Correct	OR Misspecified
Optimal Linear			5.839	9.066
Both PS and MD are Correct
CC(MD)–IPW(PS)			4.318 (0.019)	7.633 (0.019)
CC(MD)–AIPW(PS)			4.836 (0.013)	7.997 (0.014)
IPW(MD)–IPW(PS)	${\widehat{\alpha }}_{IPW}$		4.318 (0.019)	7.629 (0.019)
	${\widehat{\alpha }}_{AIPW}$		4.266 (0.020)	7.598 (0.020)
IPW(MD)–AIPW(PS)	${\widehat{\alpha }}_{IPW}$	${\widehat{\beta }}_{IPW}$	4.839 (0.013)	8.003 (0.013)
	${\widehat{\alpha }}_{AIPW}$	${\widehat{\beta }}_{AIPW}$	4.839 (0.013)	8.003 (0.013)
AIPW(MD)–IPW(PS)	${\widehat{\alpha }}_{AIPW}$		4.364 (0.019)	7.685 (0.018)
AIPW(MD)–AIPW(PS)	${\widehat{\alpha }}_{AIPW}$	${\widehat{\beta }}_{AIPW}$	4.873 (0.013)	8.034 (0.013)
PS Misspecified, MD Correct
CC(MD)–IPW(PS)			3.976 (0.023)	7.369 (0.022)
CC(MD)–AIPW(PS)			4.834 (0.013)	7.967 (0.014)
IPW(MD)–IPW(PS)	${\widehat{\alpha }}_{IPW}$		3.962 (0.023)	7.355 (0.022)
	${\widehat{\alpha }}_{AIPW}$		3.920 (0.023)	7.330 (0.023)
IPW(MD)–AIPW(PS)	${\widehat{\alpha }}_{IPW}$	${\widehat{\beta }}_{IPW}$	4.837 (0.013)	7.967 (0.014)
	${\widehat{\alpha }}_{AIPW}$	${\widehat{\beta }}_{AIPW}$	4.835 (0.013)	7.971 (0.014)
AIPW(MD)–IPW(PS)	${\widehat{\alpha }}_{AIPW}$		4.014 (0.013)	7.420 (0.021)
AIPW(MD)–AIPW(PS)	${\widehat{\alpha }}_{AIPW}$	${\widehat{\beta }}_{AIPW}$	4.878 (0.013)	8.019 (0.013)
MD Misspecified, PS Correct
CC(MD)–IPW(PS)			1.132 (0.018)	3.223 (0.032)
CC(MD)–AIPW(PS)			3.094 (0.022)	5.554 (0.029)
IPW(MD)–IPW(PS)	${\widehat{\alpha }}_{IPW}$		2.432 (0.025)	4.987 (0.035)
	${\widehat{\alpha }}_{AIPW}$		1.069 (0.019)	3.149 (0.034)
IPW(MD)–AIPW(PS)	${\widehat{\alpha }}_{IPW}$	${\widehat{\beta }}_{IPW}$	3.881 (0.021)	6.679 (0.026)
	${\widehat{\alpha }}_{AIPW}$	${\widehat{\beta }}_{AIPW}$	3.900 (0.021)	6.671 (0.025)
AIPW(MD)–IPW(PS)	${\widehat{\alpha }}_{AIPW}$		3.626 (0.023)	6.590 (0.025)
AIPW(MD)–AIPW(PS)	${\widehat{\alpha }}_{AIPW}$	${\widehat{\alpha }}_{AIPW}$	4.432 (0.017)	7.513 (0.019)
Both PS and MD Misspecified
CC(MD)–IPW(PS)			0.968 (0.016)	2.877 (0.030)
CC(MD)–AIPW(PS)			3.058 (0.022)	5.473 (0.029)
IPW(MD)–IPW(PS)	${\widehat{\alpha }}_{IPW}$		2.109 (0.025)	4.518 (0.035)
	${\widehat{\alpha }}_{AIPW}$		0.962 (0.017)	2.920 (0.032)
IPW(MD)–AIPW(PS)	${\widehat{\alpha }}_{IPW}$	${\widehat{\beta }}_{IPW}$	3.852 (0.021)	6.589 (0.026)
	${\widehat{\alpha }}_{AIPW}$	${\widehat{\beta }}_{AIPW}$	3.885 (0.021)	6.620 (0.026)
AIPW(MD)–IPW(PS)	${\widehat{\alpha }}_{AIPW}$		3.440 (0.024)	6.391 (0.029)
AIPW(MD)–AIPW(PS)	${\widehat{\alpha }}_{AIPW}$	${\widehat{\beta }}_{AIPW}$	4.416 (0.017)	7.467 (0.019)

TABLE 3.

Performance of various direct optimization treatment-selection rule estimators (based on X₁ and X₂) in settings with continuous X₁ (n = 300).

			$100\times \left(E\{Y(1)\}-E\left[Y\{\widehat{A}(X)\}\right]\right)$
Model	α	β	OR Correct	OR Misspecified
Optimal Linear			5.522	6.808
Both PS and MD Correct
CC(MD)–IPW(PS)			2.753 (0.029)	3.940 (0.030)
CC(MD)–AIPW(PS)			4.486 (0.014)	5.802 (0.013)
IPW(MD)–IPW(PS)	${\widehat{\alpha }}_{IPW}$		2.889 (0.028)	4.091 (0.030)
	${\widehat{\alpha }}_{AIPW}$		2.873 (0.029)	4.089 (0.030)
IPW(MD)–AIPW(PS)	${\widehat{\alpha }}_{IPW}$	${\widehat{\beta }}_{IPW}$	4.494 (0.013)	5.803 (0.013)
	${\widehat{\alpha }}_{AIPW}$	${\widehat{\beta }}_{AIPW}$	4.500 (0.013)	5.801 (0.013)
AIPW(MD)–IPW(PS)	${\widehat{\alpha }}_{AIPW}$		2.947 (0.028)	4.182 (0.029)
AIPW(MD)–AIPW(PS)	${\widehat{\alpha }}_{AIPW}$	${\widehat{\beta }}_{AIPW}$	4.515 (0.013)	5.818 (0.013)
PS Misspecified, MD Correct
CC(MD)–IPW(PS)			2.900 (0.031)	4.152 (0.032)
CC(MD)–AIPW(PS)			4.537 (0.013)	5.809 (0.013)
IPW(MD)–IPW(PS)	${\widehat{\alpha }}_{IPW}$		2.954 (0.030)	4.212 (0.031)
	${\widehat{\alpha }}_{AIPW}$		2.962 (0.030)	4.222 (0.031)
IPW(MD)–AIPW(PS)	${\widehat{\alpha }}_{IPW}$	${\widehat{\beta }}_{IPW}$	4.543 (0.013)	5.820 (0.013)
	${\widehat{\alpha }}_{AIPW}$	${\widehat{\beta }}_{AIPW}$	4.544 (0.013)	5.824 (0.013)
AIPW(MD)–IPW(PS)	${\widehat{\alpha }}_{AIPW}$		3.016 (0.030)	4.306 (0.031)
AIPW(MD)–AIPW(PS)	${\widehat{\alpha }}_{AIPW}$	${\widehat{\beta }}_{AIPW}$	4.566 (0.012)	5.842 (0.012)
MD Misspecified, PS Correct
CC(MD)–IPW(PS)			0.427 (0.015)	0.957 (0.019)
CC(MD)–IPW(PS)			2.801 (0.026)	3.677 (0.030)
IPW(MD)–IPW(PS)	${\widehat{\alpha }}_{IPW}$		0.843 (0.023)	1.486 (0.026)
	${\widehat{\alpha }}_{AIPW}$		0.452 (0.018)	1.006 (0.022)
IPW(MD)–AIPW(PS)	${\widehat{\alpha }}_{IPW}$	${\widehat{\beta }}_{IPW}$	3.093 (0.028)	3.999 (0.032)
	${\widehat{\alpha }}_{AIPW}$	${\widehat{\beta }}_{AIPW}$	3.949 (0.019)	5.133 (0.019)
AIPW(MD)–IPW(PS)	${\widehat{\alpha }}_{AIPW}$		2.361 (0.031)	3.503 (0.032)
AIPW(MD)–AIPW(PS)	${\widehat{\alpha }}_{AIPW}$	${\widehat{\beta }}_{AIPW}$	4.148 (0.018)	5.433 (0.017)
Both PS and MD Misspecified
CC(MD)–IPW(PS)			0.601 (0.014)	1.208 (0.017)
CC(MD)–AIPW(PS)			3.098 (0.020)	3.925 (0.023)
IPW(MD)–IPW(PS)	${\widehat{\alpha }}_{IPW}$		1.319 (0.024)	2.074 (0.026)
	${\widehat{\alpha }}_{AIPW}$		0.484 (0.018)	1.188 (0.020)
IPW(MD)–AIPW(PS)	${\widehat{\alpha }}_{IPW}$	${\widehat{\beta }}_{IPW}$	3.628 (0.021)	4.570 (0.023)
	${\widehat{\alpha }}_{AIPW}$	${\widehat{\beta }}_{AIPW}$	3.786 (0.020)	4.728 (0.022)
AIPW(MD)–IPW(PS)	${\widehat{\alpha }}_{AIPW}$		2.495 (0.030)	3.616 (0.031)
AIPW(MD)–AIPW(PS)	${\widehat{\alpha }}_{AIPW}$	${\widehat{\beta }}_{AIPW}$	4.039 (0.022)	5.336 (0.019)

Table 2 shows the performance of various estimators for discrete X₁ with training data size n = 300. Here for our proposed AIPW(MD)-AIPW(PS) estimator, the distribution of X₂∣X₁, Y, T is estimated empirically within each stratum defined by X₁, Y, T and thus the estimate of the CL model is consistent. We compare performance between different estimators under various settings with some misspecification in the PS, OR, or MD model. First, comparing between methods that use IPW/AIPW to account for covariate missingness and methods based on complete cases, depending on the actual model settings, the former methods always have better or comparable performance than the latter, suggesting that one should always adjust for missingness in covariates using proper methods (such as IPW or AIPW) in deriving the optimal treatment-regimen. Second, comparing among various methods that use a weighting technique to account for missingness in covariates, we observe the following: 1) Overall, the proposed AIPW(MD)-AIPW(PS) estimator achieves the best performance among all the direct-optimization estimators; 2) Using AIPW to account for missingness in potential outcomes is generally more advantageous compared to using IPW. That is, IPW(MD)-AIPW(PS) estimators outperform IPW(MD)-IPW(PS) estimators, and AIPW(MD)-AIPW(PS) outperforms AIPW(MD)-IPW(PS). The difference is more distinguished when PS model is misspecified. 3) When the MD model is correctly specified, using IPW weights to account for missingness in X₂ leads to similar performance as using AIPW weights. That is, performance is similar between IPW(MD)-AIPW(PS) and AIPW(MD)-AIPW(PS) estimators, and between IPW(MD)-IPW(PS) and AIPW(MD)-IPW(PS) estimators, regardless of whether or not the PS and OR models are correctly specified. When the MD model is misspecified, however, the proposed AIPW(MD)-AIPW(PS) estimator outperforms the IPW(MD)-AIPW(PS) estimators; similarly the AIPW(MD)-IPW(PS) estimator outperforms the IPW(MD)-IPW(PS) estimators. Between the two IPW(MD)-AIPW(PS) estimators, using ${\widehat{\alpha }}_{AIPW}$ and ${\widehat{\beta }}_{AIPW}$ have small advantages over using ${\widehat{\alpha }}_{IPW}$ and ${\widehat{\beta }}_{IPW}$. Similar pattern is observed for training data sample size n=600 or 1000 (Supplementary Tables 1 and 2).

Table 3 shows the performance of various direct-optimization estimators for continuous X₁ for training data size n=300. Corresponding results for n=600 and 1000 are presented in Supplementary Tables 3-4. Here for our proposed AIPW(MD)-AIPW(PS) estimator, the distribution of X₂∣X₁, Y, T is estimated using a semiparametric location-scale model where the location parameter is modeled as a flexible function of X₁, Y and T to approximate its true distribution. The patterns of relative performance among the various estimators are similar to what we observe in Table 2. The proposed estimator again has best performance across all different scenarios.

In Supplementary Appendix D, we also present an additional simulation setting for deriving optimal treatment-selection rules based on four covariates. A discrete covariate X₁ is observed among all participants whereas covariates X₂, X₃, X₄ are observed together only among a subset of participants. We use the weighted logistic regression¹² to minimize the weighted sum of 0-1 loss ${\sum }_{i=1}^n{W}_{i}I\{f(X_i)\le 0\}∕n$ given that it can be translated into an equivalent problem of minimizing a weighted sum of classification error ${\sum }_{i=1}^n\mid W_i\mid I\left[\text{sgn}(W_i)\ne \text{sgn}\{f(X_i)\}\right]∕n$ (see Supplementary Appendix D for details). Supplementary Tables 5-7 present performance of various direct-optimization estimators for settings with four covariates to derive the optimal treatment-selection rule. The joint distribution of X₂, X₃, X₄∣X₁, Y, T is again estimated using a semiparametric location-scale model. The same pattern as before can be seen for comparing various estimators, with the proposed AIPW(MD)-AIPW(PS) outperforms other alternatives.

4 ∣. DATA EXAMPLE

In this section we demonstrate the performance of our proposed estimator of treatment-selection rules using data from an intervention trial for Alzheimer’s disease, the Clinical Antipsychotic Trials of Intervention Effectiveness-Alzheimer’s Disease (CATIE-AD) trial²⁹. CATIE-AD was a double-blinded randomized clinical study to assess the effectiveness of atypical antipsychotic drugs in patients with AlzheimerâĂŹs disease. In Phase 1 of the study, subjects were randomly assigned to receive olanzapine, quetiapine, resperdone, or placebo in a 2:2:2:3 ratio. After the first two weeks, physicians could discontinue the treatment and move a patient to the next phase if the patient’s response was not satisfactory. One of the primary outcomes in Phase 1 was response rate at 12 weeks. Patients who were still in Phase 1 and with at least minimal improvement on the Clinical Global Impression of Change (CGIC) scale at 12 weeks were considered responding to the treatment.

We apply various estimators of treatment-selection rules using phase-1 data from the olanzapine (n = 99) and placebo (n = 139) arms to maximize the rate of responding to treatment at 12 weeks. In this data example a positive response to treatment is desired and so is a larger $E\left[Y\left\{\widehat{A}(X)\right\}\right]$ value. The list of baseline covariates we consider for deriving the treatment-selection rule includes serum chloride concentration (X₂ with 3.4% missing), and a few fully observed covariates (X₁) including age, gender, race, number of medicines taken, and Brief Psychiatric Rating Scale score. For various direct-optimization estimators, we adopt ordinary logistic regression models with the main effects for each predictor as working models for the MD, PS, and OR model. For the CL model, we fit a semiparametric location-scale model with the mean of log(chloride) a function of main effects for other covariates, response, and treatment status, where the natural cubic spline basis of continuous covariates including age, number of medicines taken, and Brief Psychiatric Rating Scale score are adopted. Supplementary Table 8 presents the estimated treatment-selection rules based on various direct optimization approaches. The cross-validated (CV) performance of various treatment-selection rule estimators are presented in Table 4. To compute the CV-based performance estimate, the data are randomly split into five folds stratified on the treatment and response status, including four folds for training and the remaining one fold for testing. Based on each random training subset, a treatment-selection rule $\widehat{A}(X)$ is estimated; the estimated rule is then applied to the random test subset to construct an AIPW(MD)-AIPW(PS) estimate of $E\left[Y\left\{\widehat{A}(X)\right\}\right]$ (based on working models fitted using the test set). The above process (randomly splitting data, estimating optimal rules using training set and its performance in test set) is repeated 100 times with the average $E\left[Y\left\{\widehat{A}(X)\right\}\right]$ computed. Percentile bootstrap confidence intervals are provided for these measures. In the bootstrap procedure, 1,000 resamplings are performed stratified on the treatment and response status; the procedure of computing the cross-validated $E\left[Y\left\{\widehat{A}(X)\right\}\right]$ estimates is then applied to each resampled data. For comparison, we also present the corresponding $E\left[Y\left\{\widehat{A}(X)\right\}\right]$ estimate for the strategies of treating none and treating all. Without relying on additional covariates, treating all leads to larger estimates $(\widehat{E}\{Y(1)\}=0.313)$ compared to treating none $(\widehat{E}\{Y(0)\}=0.183)$. With additional covariates, all estimators of treatment-selection rules lead to larger or comparable CV estimates of treatment response with respect to $E\left[Y\left\{\widehat{A}(X)\right\}\right]$ compared to treating all. Meanwhile, all covariate-based treatment-selection rules only recommend ~ 63 – 67% individuals to receive the active treatment, which is desirable given the cost and potential side effects of the treatment. Among the weighted estimators that account for missingness in serum chloride concentrations, the CV-based estimated treatment-selection performance in our proposed AIPW-AIPW estimator is larger compared to other estimators. The estimated performance for naive treatment-selection rule estimators based on complete cases is slightly smaller compared to the weighted estimators.

TABLE 4.

Treatment-selection rule performance (95% bootstrap CI) for the CATIE-AD data example

Model	α	β	$E\left[Y\{\widehat{A}(X)\}\right]$*	% Treat*
None Treat			0.185 (0.111, 0.259)	0
All Treat			0.316 (0.211, 0.421)	100
Raw Data
CC(MD)-IPW(PS)			0.315 (0.213, 0.478)	63.7 (44.7, 77.0)
CC(MD)-AIPW(PS)			0.325 (0.212, 0.476)	63.8 (49.4, 81.9)
IPW(MD)-IPW(PS)	${\widehat{\alpha }}_{IPW}$		0.317 (0.212, 0.468)	64.5 (45.1, 77.8)
	${\widehat{\alpha }}_{AIPW}$		0.315 (0.212, 0.469)	64.2 (44.7, 77.1)
IPW(MD)-AIPW(PS)	${\widehat{\alpha }}_{IPW}$	${\widehat{\beta }}_{IPW}$	0.325 (0.210, 0.466)	66.7 (48.9, 82.0)
	${\widehat{\alpha }}_{AIPW}$	${\widehat{\beta }}_{AIPW}$	0.327 (0.211, 0.472)	66.8 (48.7, 82.1)
AIPW(MD)-IPW(PS)	${\widehat{\alpha }}_{AIPW}$		0.317 (0.214, 0.47.6)	63.9 (45.0, 77.6)
AIPW(MD)-AIPW(PS)	${\widehat{\alpha }}_{AIPW}$	${\widehat{\beta }}_{AIPW}$	0.329 (0.211, 0.482)	65.9 (48.8, 82.1)
Extra missingness in the chloride measurement
CC(MD)-IPW(PS)			0.280 (0.182, 0.431)	43.7 (23.0, 63.6)
CC(MD)-AIPW(PS)			0.281 (0.184, 0.440)	44.7 (20.9, 69.8)
IPW(MD)-IPW(PS)	${\widehat{\alpha }}_{IPW}$		0.265 (0.183, 0.429)	38.8 (23.9, 62.1)
	${\widehat{\alpha }}_{AIPW}$		0.266 (0.186, 0.428)	39.4 (25.4, 63.2)
IPW(MD)-AIPW(PS)	${\widehat{\alpha }}_{IPW}$	${\widehat{\beta }}_{IPW}$	0.260 (0.185, 0.444)	37.6 (22.1, 66.6)
	${\widehat{\alpha }}_{AIPW}$	${\widehat{\beta }}_{AIPW}$	0.274 (0.185, 0.441)	38.9 (24.2, 65.1)
AIPW(MD)-IPW(PS)	${\widehat{\alpha }}_{AIPW}$		0.241 (0.183, 0.423)	29.3 (19.7, 60.7)
AIPW(MD)-AIPW(PS)	${\widehat{\alpha }}_{AIPW}$	${\widehat{\beta }}_{AIPW}$	0.325 (0.211, 0.466)	68.5 (47.4, 81.2)

^*: performance estimated based on 100 random cross-validation with a 4:1 training-test split

To view the impact of misspecification of the MD model, among the study participants who received the active treatment (T = 1) or who did not respond positively to treatment (Y = 0), we randomly make half of chloride measurements to be missing such that the probability of missing chloride is related to an interaction term between Y and T, which is not included in the MD working model. In this way, missing rate of chloride in the data is ~46% and the working model for MD is now misspecified. With this extra missingness applied to training subset, smaller CV-based $E\left[Y\left\{\widehat{A}(X)\right\}\right]$ estimates compared to the strategy of treating all were observed among complete-case estimators, IPW(MD)-IPW(PS), IPW(MD)-AIPW(PS), or AIPW(MD)-IPW(PS) estimators. In contrast, the proposed AIPW(MD)-AIPW(PS) estimator has the largest CV-based performance estimate among all.

5 ∣. CONCLUDING REMARKS

In this paper, we considered the derivation of optimal treatment-selection rules when an important covariate is partially missing. Building on the direct-optimization framework, we proposed an estimator that utilizes augmented inverse probability weighting to accommodate both missingness in a covariate and missingness in potential outcomes. The estimator has more robust performance than alternative estimators in the sense that it maximizes the desired criterion function when one of the missing data model/calibration model is correctly specified and one of the propensity score model/outcome regression model is correctly specified.

This estimator requires specification of four working models PS, OR, MD, and the CL models. Note that the CL model can deduce from PS, OR, and P(X₂∣X₁):

$$P(X_2\mid X_1,T,Y)=\frac{P(X_2,T,Y\mid X_1)}{P(T,Y\mid X_1)}=\frac{P(Y\mid X_1,X_2,T)P(T\mid X_1,X_2)P(X_2\mid X_1)}{\int P(Y\mid X_1,X_2,T)P(T\mid X_1,X_2)P(X_2\mid X_1)d{X}_2}.$$ (13)

It is thus possible that some PS, OR, and CL model cannot be simultaneously true. Therefore in practice we recommend to adopt a flexible CL model. In particular, for discrete X₁, one could use empirical estimate for P(X₂∣X₁, T, Y) = P(X₂∣X₁, T, Y, R = 1) so there is no concern about CL model being incompatible with PS and OR models. For continuous X₁, we used method such as semiparametric location-scale model with mean parameter modeled as a regression splines of X₁. In practice, using a CL model that approximates data well in AIPW(MD)-AIPW(MD) can still improve over alternative estimators that do not have a CL component, even if the CL model is not exactly true as shown in our numerical studies. Based on (13), one might consider an alternative that specifies and estimates the model of P(X₂∣X₁) to deduce corresponding CL model. A problem with this approach is that P(X₂∣X₁) might not equal to P(X₂∣X₁, R = 1). So to estimate P(X₂∣X₁) based on the samples with R = 1, one would have to account for missing data using methods such as IPW and requires correct specification of both X₂∣X₁ model and MD model.

Although here we focused on addressing missingness in covariates, similar methodology can be developed to allow for missingness of other variables, such as treatment status or disease outcome under the missingness at random assumption. Those methodology will differ with respect to the construction of missing data model and calibration model as sketched in Supplementary Appendix E.

Finally, note that the proposed AIPW estimator can target not only scenarios with one covariate missing, but also scenarios with several covariates always missing at the same time. The latter can happen when missingness is due to sampling design or in studies where some participants might miss a clinical visit for measuring a few lab assays, but whoever comes to a visit will have all assays measured. Our paper focuses on a direct-optimization based approach for optimal treatment-regimen development that emphasizes robustness, which makes it naturally appealing to adopt the augmented inverse probability weighting approach when feasible to maximize robustness. Using weighting approach to handle more complicated general missingness such as nonmonotone missingness among several covariates requires non-trivial extension. In the presence of general missingness, one need to handle multiple missingness patterns, depending on the subsets of covariates that are missing in an individual, and assumptions need to be made about every missing data pattern conditional on observed data. Relevant work on solving inverse probability weighted estimating equation for non-monotone missing at random data has been proposed earlier³⁰. How to extend the idea to the optimal treatment-regimen problem will be a future research topic. Meanwhile, an alternative strategy for handling general missingness is to use the multiple imputation based technique as in Shortreed et al¹⁴, which on the other hand will require proper modeling of the relationship among various covariates and outcome.