Evaluating Marker-Guided Treatment Selection Strategies

Summary

A potential venue to improve healthcare efficiency is to effectively tailor individualized treatment strategies by incorporating patient level predictor information such as environmental exposure, biological, and genetic marker measurements. Many useful statistical methods for deriving individualized treatment rules (ITR) have become available in recent years. Prior to adopting any ITR in clinical practice, it is crucial to evaluate its value in improving patient outcomes. Existing methods for quantifying such values mainly consider either a single marker or semi-parametric methods that are subject to bias under model misspecification. In this paper, we consider a general setting with multiple markers and propose a two-step robust method to derive ITRs and evaluate their values. We also propose procedures for comparing different ITRs, which can be used to quantify the incremental value of new markers in improving treatment selection. While working models are used in step I to approximate optimal ITRs, we add a layer of calibration to guard against model misspecification and further assess the value of the ITR non-parametrically, which ensures the validity of the inference. To account for the sampling variability of the estimated rules and their corresponding values, we propose a resampling procedure to provide valid confidence intervals for the value functions as well as for the incremental value of new markers for treatment selection. Our proposals are examined through extensive simulation studies and illustrated with the data from a clinical trial that studies the effects of two drug combinations on HIV-1 infected patients.

1. Introduction

The standard analyses of randomized clinical trial evaluate the treatment effect based on the overall treatment difference on the entire study population. However, such overall the treatment effect assessment may not be adequate when a new treatment benefits patients differentially depending on each patient’s characteristics. A treatment deemed effective on average does not guarantee that it will be beneficial to all future patients. Conversely, a negative finding on the average the treatment effect does not imply that the new treatment is entirely futile since the effectiveness of a treatment on a small subgroup of patients may be hidden by its inactivity on a large (heterogeneous) patient population (Rothwell, 1995; Rothwell et al., 2005; Kent and Hayward, 2007). When the treatment effect varies across subpopulations, it would be desirable to develop individualized treatment rules (ITR) according to individual patients’ baseline characteristics. Assigning treatments to achieve optimal patient outcomes may substantially improve healthcare efficiency (Baker et al., 2012).

Statistical methods for developing optimal ITRs have received much attention in recent years. Traditional methods based on ad hoc subgroup analyses or searching for markertreatment interactions, while useful, may not be efficient or valid due to the curse of dimensionality and multiple comparisons. More systematic approaches to deriving ITR have been recently proposed. With a single baseline marker, semi- and non-parametric procedures have been proposed to identify a subgroup of patients who would benefit from the new treatment (e.g., Song and Pepe, 2004; Bonetti and Gelber, 2000, 2004). With multiple baseline markers, a wide range of procedures have been proposed to derive ITRs that combines information across all markers (e.g., Qian and Murphy, 2011; Imai and Strauss, 2011; Foster et al., 2011; Cai et al., 2011; Zhao et al., 2012; Zhang et al., 2012a; Zhao et al., 2013).

As strategies for deriving ITRs become increasingly available, it is important to examine the net benefit of assigning treatment according to an ITR prior to recommending its wide spread use. Most current research focuses on developing ITRs with relative little attention given to making robust inference about such estimated ITRs and their value in improving population outcomes. Although a few methods have been proposed to quantify such values, these methods consider either a single marker or semi-parametric methods that are subject to bias under model misspecification (Song and Pepe, 2004; Song and Zhou, 2009; Janes et al., 2011; Huang et al., 2012, e.g). Zhang et al. (2012a) propose a robust approach to overcome model misspecification by restricting the ITR in a parametric class and estimate the ITR parameters by maximizing an empirical value function associated with the ITR. The direct maximization of the non-smooth empirical value function could suffer substantial variability in the estimated ITR parameters. As we show Section 3.2 and Web Appendix B, even for a univariate X with ITR given by I(X ≥ c), direct maximization gives an estimate of c with a cubic convergence rate. When there are multiple markers, direct maximization of an empirical value function with respect to all unknown parameters involved in the ITR, such as those proposed in Zhang et al. (2012b), could be computationally prohibitive and unstable. Here, we consider a general setting with multiple markers and adopt a two-step method to derive a class of ITRs and make inference about the value of such ITRs. We also propose procedures for comparing different ITRs, which can be used to quantify the incremental value (IncV) of new markers in improving treatment selection. Such IncV assessment is particularly important if a marker used in the ITR is expensive and/or invasive.

The remainder of this paper is organized as follows. We describe in Section 2, the general framework for quantifying the value of ITRs and deriving ITRs that attain maximal values. We also provide some simple results demonstrating that a two-step procedure could potentially lead to an ITR that is optimal (i) among all ITRs based on a set of predictors X when the fitted models in the first step are nearly correct; and (ii) within a smaller class of ITRs when the models are misspecified. In Section 3, we provide the estimation and procedures for making inference about the proposed two-step ITR as well as its value function. In Section 4, we evaluate the finite sample performance of our proposed methods through a series of simulation studies. We apply the proposed method, in Section 5, to a data set from a clinical trial (ACTG 320) conducted by the AIDS Clinical Trial Group as a further illustration. In Section 6, we provide some concluding remarks and further discussion.

2. Quantifying the Value of ITR and Optimizing ITR

2.1 Notations and Settings

Let Y be the response variable and Y^(j) denote the potential outcome (Rubin, 1974; Rubin et al., 1978; Holland, 1986; Robins, 1986) of a patient if assigned to treatment G = j, where j = 1 refers to the experimental treatment and j = 0 to the standard treatment. The potential outcome (also referred to as counterfactual outcome) Y^(j) is defined as the value of the outcome had the treatment been set to G = j by external intervention. Both Y and Y^(j) are related via the consistency assumption Y = GY⁽¹⁾ + (1 – G)Y⁽⁰⁾. We assume the standard stable unit treatment value assumption i.e. each subject’s potential response to a treatment does not depend on the treatment assignment mechanism, the treatments received by other patients or their potential responses to treatments (Rubin, 1980, 1986).Without loss of generality, we assume that a larger value of Y is more beneficial.

Let $\mathcal{I}\left(\mathbf{X}\right)$ denote a binary ITR as a function of a baseline covariate vector X with $\mathcal{I}\left(\mathbf{X}\right)=j$ indicating assignment to treatment j. Our goal is to identify an optimal $\mathcal{I}(\cdot )$ that maximizes patients’ outcomes. When the treatment selection is optimized for all patients, the resulting population average outcome is also optimized. Thus, an optimal ITR is also expected to maximize a population average value function. A sensible choice of the value function is the cost-adjusted population average outcome associated with $\mathcal{I}$:

$${\mathbb{V}}_{\mathcal{I}}=E\left\{\mathcal{I}\right(\mathbf{X}\left)\right(Y^{\left(1\right)}-\xi )+(1-\mathcal{I}\left(\mathbf{X}\right)\left)Y^{\left(0\right)}\right\},$$

where ξ is a pre-specified incremental financial and/or medical cost associated with taking the new treatment as compared to the standard treatment. It is not difficult to see that the optimal $\mathcal{I}(\cdot )$ maximizing ${\mathbb{V}}_{\mathcal{I}}$ is the Bayes rule

$${\mathcal{I}}_{\mathrm{Bayes}}\left(\mathbf{X}\right)=\underset{\mathcal{I}}{\mathrm{argmax}}{\mathbb{V}}_{\mathcal{I}}=I\left\{D\right(\mathbf{X})\ge \xi \},$$ (2.1)

where I(.) is the indicator function and D(X) = E(Y⁽¹⁾∣X – E(Y⁽⁰⁾∣X). With a given dataset, the optimal ITR ${\mathcal{I}}_{\mathrm{Bayes}}\left(\mathbf{X}\right)$ can be approximated by estimating the conditional treatment effect function D(X) or by direct optimization of ${\mathbb{V}}_{\mathcal{I}}$ within a class of $\mathcal{I}$.

Throughout, we use notation D(β,X) to denote a model based approximation to the true conditional mean difference D(X) with a model parameter value β; let $\widehat{\beta }$ denote its estimate from the data and $\stackrel{‒}{\beta }$ denote its limiting value. In addition, we let$\stackrel{‒}{\Delta }\left(s\right)=E\{Y^{\left(1\right)}-Y^{\left(0\right)}\mid D(\stackrel{‒}{\beta },\mathbf{X})=s\}$ denote a calibrated estimate of treatment difference given $D(\stackrel{‒}{\beta },\mathbf{X})=s$. We next describe the pros and cons of various approaches.

2.2 Various Approaches to Approximating ${\mathcal{I}}_{\mathrm{Bayes}}$

When X = X is univariate, we can approximate ${\mathcal{I}}_{\mathrm{Bayes}}(\cdot )$ by estimating D(X) via kernel smoothing. Note that if D(X) is an increasing function in X, the optimal rule ${\mathcal{I}}_{\mathrm{Bayes}}$(X) must take the form I(X ≥ c°), where c° = argmax_c $\mathbb{V}\left(c\right)$ and $\mathbb{V}\left(c\right)=E\left\{I\right(X\ge c\left)\right(Y^{\left(1\right)}-\xi )+I(X<c\left)Y^{\left(0\right)}\right\}$. Evaluating the utility of a single marker based on $\mathbb{V}\left(c\right)$ has been previously considered in Song and Pepe (2004) and Song and Zhou (2009). However, when D(X) is not monotone in X, the optimal ITR may not take the form of I(X ≥ c) and $\mathbb{V}\left(c\right)<{\mathbb{V}}_{{\mathcal{I}}_{\mathrm{Bayes}}}$ for any c if there exists x₁ > x₂ such that D(x₁) = D(x₂) = ξ and P(x₁ > X > x₂) > 0.

With multivariate X, using fully non-parametric methods and incorporating non-linear functional spaces for approximating ${\mathcal{I}}_{\mathrm{Bayes}}(\cdot )$ (Foster et al., 2011; Zhao et al., 2012) could be extremely valuable, especially when D(X) takes a complex form. However, these methods are subject to curse of dimensionality and pose challenges in making inference about the resulting ITR and its associated value function. On the other hand, if D(X) is estimated by imposing parametric or semi-parametric models on E(Y^(j)∣X), the plug-in estimate of ${\mathcal{I}}_{\mathrm{Bayes}}\left(\mathbf{X}\right)$ may lead to a much lower population average outcome compared to that of the true ${\mathcal{I}}_{\mathrm{Bayes}}\left(\mathbf{X}\right)$ (Qian and Murphy, 2011). One may reduce model misspecification by including non-linear bases and selecting important variables via regularized estimation (Qian and Murphy, 2011; Imai and Strauss, 2011). However, it remains challenging to efficiently choose non-linear basis functions to achieve an optimal bias and variance trade-off.

We seek to overcome model misspecification by following a two-step principal previously proposed in Cai et al. (2011) and Zhao et al. (2013): (I) obtain a parametric or semiparametric model based estimate of D(X), denoted by $D(\widehat{\beta };\mathbf{X})$, where $\widehat{\beta }$ is the estimated model parameters; and (II) non-parametrically estimate treatment effect parameters to account for possible model misspecification in step I. Both of these existing methods, while related to using working models for treatment selection, do not address the question of how to derive an optimal ITR or how to make inference about its associated value, which is the focus of this paper. Cai et al. (2011) uses $D(\widehat{\beta };\mathbf{X})$ as a univariate score to create subgroups, ${\Omega }_s=\{\mathbf{X}:D(\widehat{\beta };\mathbf{X})=s\}$ and provides inference procedures for E{Y⁽¹⁾ – Y⁽⁰⁾ ∣ X ∈ Ω_s}. Zhao et al. (2013) proposes a non-parametric estimator of $E\{Y^{\left(1\right)}-Y^{\left(0\right)}\mid D(\widehat{\beta },\mathbf{X})\ge c\}$ for a range of c in step II and hence only relate to ITRs of the form $D(\widehat{\beta },\mathbf{X})\ge c$. Although the methods given in Zhao et al (2013) can be used to compare the potential of two scores in guiding treatment selection, their measures do not have a clear clinical interpretation and cannot be used to quantify the performance of a single score.

In this paper, we propose a two-step approach to construct a calibrated ITR,

$${\mathcal{I}}_{\mathrm{calib}}\left(\mathbf{X}\right)=I\left\{\stackrel{‒}{\Delta }\right(D(\stackrel{‒}{\beta },\mathbf{X}))\ge \xi \},$$

and evaluate its value ${\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}}$. It follows from (2.1) that ${\mathcal{I}}_{\mathrm{calib}}\left(\mathbf{X}\right)$ is the optimal ITR based on the univariate score $D(\stackrel{‒}{\beta },\mathbf{X})$. We next detail some pros and cons of using ${\mathcal{I}}_{\mathrm{calib}}$ for treatment selection under various conditions.

• When the working models in the first step are nearly correct such that D(X) is an increasing function of $D(\stackrel{‒}{\beta },\mathbf{X})$, then $\stackrel{‒}{\Delta }\left(D\right(\stackrel{‒}{\beta },\mathbf{X}\left)\right)=D\left(\mathbf{X}\right)$. Hence, the two-step procedure leads to the optimal ITR with ${\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}}={\mathbb{V}}_{{\mathcal{I}}_{\mathrm{Bayes}}}$.

• Under more severe model misspecification when $P\left[I\right\{D\left(\mathbf{X}\right)>\xi \}\ne I\{\stackrel{‒}{\Delta }\left(D\right(\stackrel{‒}{\beta },\mathbf{X}\left)\right)>\xi \left\}\right]>0$, ${\mathcal{I}}_{\mathrm{calib}}$ will be sub-optimal relative to ${\mathcal{I}}_{\mathrm{Bayes}}$ with ${\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}}<{\mathbb{V}}_{{\mathcal{I}}_{\mathrm{Bayes}}}$. However, when ${\mathcal{I}}_{\mathrm{Bayes}}$ and ${\mathcal{I}}_{\mathrm{calib}}$ are replaced with their respective estimates ${\widehat{\mathcal{I}}}_{\mathrm{Bayes}}$ and ${\widehat{\mathcal{I}}}_{\mathrm{calib}}$ obtained in finite sample, applying ${\widehat{\mathcal{I}}}_{\mathrm{Bayes}}$ to a future population may not yield a value function higher than that of ${\widehat{\mathcal{I}}}_{\mathrm{calib}}$ if the sampling variability associated with ${\widehat{\mathcal{I}}}_{\mathrm{Bayes}}$ is much larger than that of ${\widehat{\mathcal{I}}}_{\mathrm{calib}}$.

• When the model misspecification is not severe with $\stackrel{‒}{\Delta }\left(s\right)$ being increasing in s, ${\mathcal{I}}_{\mathrm{calib}}\left(\mathbf{X}\right)$ takes the form $I_{c^{\circ }}=I\left(D\right(\stackrel{‒}{\beta },\mathbf{X})>c^{\circ })$, where c° = argmax_c ${\mathbb{V}}_{I_c}$ with ${\mathbb{V}}_{I_c}=E\left\{I\right(D(\stackrel{‒}{\beta },\mathbf{X})\ge c\left)\right(Y^{\left(1\right)}-\xi )+I(D(\stackrel{‒}{\beta },\mathbf{X})<c\left)Y^{\left(0\right)}\right\}$.

• When $\stackrel{‒}{\Delta }\left(s\right)$ is not monotone and there exists d₁ > d₂ such that $\stackrel{‒}{\Delta }\left(d_1\right)=\stackrel{‒}{\Delta }\left(d_2\right)=\xi$ and $P(d_1>D(\stackrel{‒}{\beta },\mathbf{X})>d_2)>0$, we have ${\mathbb{V}}_{I_{c^{\circ }}}<{\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}}<{\mathbb{V}}_{{\mathcal{I}}_{\mathrm{Bayes}}}$. In other words, under certain model misspecifications such as missing a quadratic effect, assigning treatment according to $I\left(D\right(\stackrel{‒}{\beta },\mathbf{X})\ge c)$ as in Zhao et al. (2013) will be sub-optimal when compared to ${\mathcal{I}}_{\mathrm{calib}}$.

These findings suggest the benefit in approximating ${\mathcal{I}}_{\mathrm{Bayes}}$ with ${\mathcal{I}}_{\mathrm{calib}}$ and motivate us to provide a robust estimate of the ITR as follows:

1. posit working models to approximate D(X) using a model based score $D(\stackrel{‒}{\beta },\mathbf{X})$;

2. non-parametrically estimate $\stackrel{‒}{\Delta }\left(s\right)$ as $\widehat{\Delta }\left(s\right)$ using observed responses along with $D(\stackrel{‒}{\beta },\mathbf{X})$;

3. for a future patient with X = x, the treatment assignment is determined using ${\widehat{\mathcal{I}}}_{\mathrm{calib}}\left(\mathbf{X}\right)=I\left[\widehat{\Delta }\right\{D(\widehat{\beta },\mathbf{X})\}\ge \xi ]$, which is an estimator of ${\mathcal{I}}_{\mathrm{calib}}$ with $P\left\{{\widehat{\mathcal{I}}}_{\mathrm{calib}}\right(\mathbf{X})\ne {\mathcal{I}}_{\mathrm{calib}}(\mathbf{X}\left)\right\}\to 0$;

4. evaluate the performance of ${\widehat{\mathcal{I}}}_{\mathrm{calib}}(\cdot )$ based on the observed data by estimating ${\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}}$.

We next detail our proposed estimators for ${\mathcal{I}}_{\mathrm{calib}}$ and its associated value ${\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}}$ along with their asymptotic properties.

3. Estimation and Inference Procedures for ${\mathcal{I}}_{\mathrm{calib}}$ and ${\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}}$

We assume that data available for analysis are from a randomized clinical trial with study participants randomly assigned to either a standard treatment (G = 0) or an experimental treatment (G = 1). Our data consist of n random vectors *****$\mathcal{D}=\left\{\right(Y_i,{\mathbf{X}}_i,G_i),i=1,\dots ,n\}$, where we assume that {(Y_i,X_i) : G_i = j} are $n_j={\sum }_{i=1}^{n}I(G_i=j)$ independent and identically distributed random vectors, for j = 0, 1. Furthermore, we assume that the ratio n₁/n converges to a constant π₁ ∈ (0, 1) as n → ∞.

3.1 Estimation of ${\mathcal{I}}_{\mathrm{calib}}$ and ${\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}}$

We first approximate D(X) by imposing parametric or semi-parametric working models as E(Y_i ∣ X_i = X, G_i = j) = μ_j(_βj;X) with β being the unknown model parameter for group j and μ_j being a known link function. Without loss of generality, we suppose that β_j is estimated by ${\widehat{\beta }}_j$, the solution to ${\widehat{\mathbb{U}}}_j\left(\beta \right)=0$, where

$${\widehat{\mathbb{U}}}_j\left(\beta \right)=n_j^{-1}\sum _{i=1}^{n}I(G_i=j){\mathrm{U}}_j(\beta ,Y_i,{\mathbf{X}}_i)+{\lambda }_{\mathit{jn}}\beta ,$$ (3.1)

U_j(β;·,·) is the estimating function relating the observed data to the parameters of interest, and $0\le {\lambda }_{\mathit{jn}}=o\left(n_j^{-\frac{1}{2}}\right)$ is a tuning parameter chosen to ensure stable fitting when the dimension of β is not too small compared to the sample size. For example, one may let ${\mathbf{U}}_j(\beta ;Y_i,{\mathbf{X}}_i)={\mathbf{X}}_i\{Y_i-g({\beta }_j^{⊺}\psi \left({\mathbf{X}}_i\right)\}$ under generalized linear models, where ψ(·) is a prespecified finite dimensional vector of potentially non-linear transformation functions which would allow one to capture non-linear effects. We choose penalty ${\lambda }_{\mathit{jn}}=o\left(n_j^{-\frac{1}{2}}\right)$ so that $n^{\frac{1}{2}}(\widehat{\beta }-\stackrel{‒}{\beta })$ still converges to a zero-mean normal random vector to enable easy inference. Then, the model based estimate of D(X) can be obtained as $D(\widehat{\beta };\mathbf{X})={\mu }_1({\widehat{\beta }}_0;\mathbf{X})-{\mu }_0(\widehat{\beta };\mathbf{X})$, where $\widehat{\beta }={({\widehat{\beta }}_0^{⊺},{\widehat{\beta }}_1^{⊺})}^{⊺}$ is a vector of estimated model parameters.

Next, we propose to estimate ${\mathcal{I}}_{\mathrm{calib}}$ and ${\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}}$ non-parametrically with ${\widehat{\mathcal{I}}}_{\mathrm{calib}}\left(\mathbf{X}\right)=I\left[\widehat{\Delta }\right\{D(\widehat{\beta },\mathbf{X})\}\ge \xi ]$ and ${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}$ respectively, where

$$\begin{array}{cc}\hfill & {\widehat{\mathbb{V}}}_{\mathcal{I}}=n_1^{-1}\sum _{i=1}^n\mathcal{I}\left({\mathbf{X}}_i\right)(Y_i-\xi )I(G_i=1)+n_0^{-1}\sum _{i=1}^n\{1-\mathcal{I}({\mathbf{X}}_i\left)\right\}{Y}_{i}I(G_i=0),\hfill \\ \hfill \widehat{\Delta }\left(s\right)& =\frac{\sum _{i=1}^n{K}_h\left(D\right(\widehat{\beta };{\mathbf{X}}_i)-s)Y_{i}I(G_i=1)}{\sum _{i=1}^n{K}_h\left(D\right(\widehat{\beta };{\mathbf{X}}_i)-s)I(G_i=1)}-\frac{\sum _{i=1}^n{K}_h\left(D\right(\widehat{\beta };{\mathbf{X}}_i)-s)Y_{i}I(G_i=0)}{\sum _{i=1}^n{K}_h\left(D\right(\widehat{\beta };{\mathbf{X}}_i)-s)I(G_i=0)}\hfill \end{array}$$ (3.2)

representing a non-parametric smoothed estimator of $\stackrel{‒}{\Delta }\left(s\right)$ and K(·) is a smooth symmetric density function with h → 0 as n → 1.

We show in Web Appendix A that, under mild regularity conditions, $P\left\{{\widehat{\mathcal{I}}}_{\mathrm{calib}}\right(\mathbf{X})\ne {\mathcal{I}}_{\mathrm{calib}}(\mathbf{X}\left)\right\}\to 0$ and ${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}\to {\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}}$ in probability. Furthermore, we show that $n^{\frac{1}{2}}({\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}-{\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}}=n^{\frac{1}{2}}({\widehat{\mathbb{V}}}_{{\tilde{\mathcal{I}}}_{\mathrm{calib}}}-{\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}})+o_p(1)$, which converges in distribution to a zero mean normal random variable with variance ${\sigma }_{\mathbb{V}}^2=E\left({\Psi }_{{\nu }_i}^2\right)$, where ${\Psi }_{{\nu }_i}$ is defined in (A.1) of Web Appendix A and ${\tilde{\mathcal{I}}}_{\mathrm{calib}}\left(\mathbf{X}\right)=I\left[\stackrel{‒}{\Delta }\right\{D(\widehat{\beta },\mathbf{X})\ge \xi ]$ is the estimated treatment assignment rule knowing $\stackrel{‒}{\Delta }(\cdot )$.

This indicates that $\widehat{\Delta }\left(s\right)$ in the non-parametric calibration step does not contribute any additional variability for the estimation of ${\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}}$ at the first order. Alternative choices of $\widehat{\Delta }\left(s\right)$ such as a local likelihood estimator can also be valid provided that $\widehat{\Delta }\left(s\right)-\stackrel{‒}{\Delta }\left(s\right)=o_p\left(n^{-1∕4}\right)$.

3.2 Estimation of the Threshold Under Monotonicity

Under the assumptions that $\stackrel{‒}{\Delta }\left(s\right)$ is monotone, $\stackrel{‒}{\Delta }\left(s\right)=\xi$ has a unique solution at c°, and the product $P\left\{D\right(\stackrel{‒}{\beta },\mathbf{X})>{\stackrel{‒}{\Delta }}^{-1}(\xi \left)\right\}P\left\{D\right(\stackrel{‒}{\beta },\mathbf{X})<{\stackrel{‒}{\Delta }}^{-1}(\xi \left)\right\}>0$. The optimal ITR based on $D(\stackrel{‒}{\beta },\mathbf{X})$ takes the form ${\mathcal{I}}_{\mathrm{calib}}\left(\mathbf{X}\right)=I\left(D\right(\stackrel{‒}{\beta },\mathbf{X})>c^{\circ })$, where c° = argmax_c ${\mathbb{V}}_{I_c}$ is an interior point of the support of $D(\stackrel{‒}{\beta },\mathbf{X})$. Hence, one may also approximate ${\mathcal{I}}_{\mathrm{calib}}\left(\mathbf{X}\right)$ via direct maximization as ${\widehat{I}}_{\widehat{¢}}(\mathbf{X}$, for $\widehat{¢}={\mathrm{argmax}}_c{\widehat{\mathbb{V}}}_{{\widehat{I}}_c}$, where we define ${\widehat{I}}_c\left(\mathbf{X}\right)=I\left\{D\right(\widehat{\beta },\mathbf{X})\ge c\}$.

We show, in Web Appendix B, that $\widehat{¢}$ is a consistent estimator of c°. However, due to the non-smoothness in the empirical value function ${\widehat{\mathbb{V}}}_{{\widehat{I}}_c}$, we have $\widehat{¢}-c^{\circ }=O_p\left(n^{-1∕3}\right)$, suggesting that direct maximization results in an estimator with a much slower convergence rate and hence large variability in $\widehat{¢}$. Furthermore, following from Theorem §1.1 of Kim and Pollard (1990), $n^{1∕3}(\widehat{¢}-c_0)$ converges in distribution to argmin_t $\mathcal{Z}\left(t\right)$, where $\mathcal{Z}\left(t\right)$ is a Gaussian process. However, argmin_t $\mathcal{Z}\left(t\right)$ does not generally have an explicit form.

Since co is also the solution to $\stackrel{‒}{\Delta }\left(c\right)-\xi =0$, we propose to estimate c° as $\widehat{c}$, the solution to $\stackrel{‒}{\Delta }\left(c\right)=\xi$. As shown in Web Appendix B, $\widehat{c}-c^{\circ }=O_p\{{\left(\mathit{nh}\right)}^{-1∕2}+h^2\}$ and $\sqrt{\mathit{nh}}(\widehat{c}-c^{\circ })$ converges in distribution to a normal when h = O(n^−v) with v ∈= [1/5, 1/2). In addition, ${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{c}}}\to {\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}}$ and $n^{\frac{1}{2}}({\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{c}}}-{\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}})=n^{\frac{1}{2}}({\widehat{\mathbb{V}}}_{{\widehat{I}}_{c^{\circ }}}-{\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}})+O_p\left(1\right)=n^{\frac{1}{2}}({\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}-{\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}})+O_p\left(1\right)$. This suggests that, under the monotonicity assumption, the variability of $\widehat{c}$ is ignorable at the first order when making inference about ${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{c}}}$ and the estimator ${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{c}}}$ is asymptotically equivalent to ${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}$ obtained by first estimating $\stackrel{‒}{\Delta }\left(s\right)$ via smoothing.

Finally, using similar arguments as those given inWeb Appendix B, one may also show that $n^{\frac{1}{2}}({\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{¢}}}-{\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}})=n^{\frac{1}{2}}({\widehat{\mathbb{V}}}_{{\widehat{I}}_{c^{\circ }}}-{\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}})+O_p\left(1\right)$. However, in finite sample, the large variability in $\widehat{¢}$ leads to ${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{¢}}}$ having higher variability than that of ${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{c}}}$. We further illustrate these points in the simulation section.

3.3 Bias Correction and Interval Estimation

In practice, for a small or moderate sample size n, the value of the marker-guided ITR could be substantially over-estimated due to overfitting (Zhao et al., 2013). To correct for the bias, we use the standard cross-validation (CV) technique. Specifically, we first randomly splits the data into $\mathcal{K}$ disjoint subsets of about equal size and label them as ${\mathbb{E}}_k$, where k = 1, …$\mathcal{K}$. For each k, we use all the observations not in ${\mathbb{E}}_k$ to obtain ${\widehat{\beta }}_j^{(-k)}$ for β_j via (3.1), and compute the score $D({\widehat{\beta }}^{(-k)};{\mathbf{X}}_i)$ as well as ${\widehat{\Delta }}^{(-k)}\left(s\right)$. Then, we use observations in ${\mathbb{E}}_k$ to obtain

$${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}},k}=\frac{\sum _{i\in {\mathbb{E}}_k}I\left\{{\widehat{\Delta }}^{(-k)}\right(D({\widehat{\beta }}^{-k};{\mathbf{X}}_i))\ge \xi ,G_i=1\}(Y_i-\xi )}{\sum _{i\in {\mathbb{E}}_k}I(G_i=1)}+\frac{\sum _{i\in {\mathbb{E}}_k}I\left\{{\widehat{\Delta }}^{-k}\right(D({\widehat{\beta }}^{(-k)};{\mathbf{X}}_i))<\xi ,G_i=0\}{Y}_i}{\sum _{i\in {\mathbb{E}}_k}I(G_i=0)}$$

Finally, we obtain the CV estimator for ${\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}}$ as ${\mathbb{V}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}^{\left(\mathit{cv}\right)}={\mathcal{K}}^{-1}{\sum }_{k=1}^{\mathcal{K}}{\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}},k}$

The variance of ${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}$ involves unknown density-like functions, which makes it difficult to estimate directly, especially when the number of covariates in the model is not small. To circumvent this issue, we use a perturbation-resampling technique to obtain a good approximation to the distribution of our proposed estimators. Specifically, let {W_i, i = 1; … , n} be n independent and identically distributed random variables generated from a known distribution with mean 1 and variance 1. The perturbed version of ${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}$ can be obtained as

$${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}^{\ast }=\frac{\sum _{i=1}^{n}I\left\{{\widehat{\Delta }}^{(\ast )}\right(D({\widehat{\beta }}^{\ast };{\mathbf{X}}_i))\ge \xi ,G_i=1\}(Y_i-\xi )W_i}{\sum _{i=1}^{n}I(G_i=1)W_i}+\frac{\sum _{i=1}^{n}I\left\{{\widehat{\Delta }}^{\ast }\right(D({\widehat{\beta }}^{\ast };{\mathbf{X}}_i))<\xi ,G_i=0\}{Y}_i{W}_i}{\sum _{i=1}^{n}I(G_i=0)W_i},$$

where ${\widehat{\Delta }}^{\ast }\left(s\right)={\sum }_{i=1}^n{K}_h\left(D\right({\widehat{\beta }}^{\ast };{\mathbf{X}}_i)-s)Y_{i}I(G_i=1)W_i∕\left\{{\sum }_{i=1}^n{K}_h\right(D({\widehat{\beta }}^{\ast };{\mathbf{X}}_i)-s\left)I\right(G_i=1\left)W_i\right\}-{\sum }_{i=1}^n{K}_h\left(D\right({\widehat{\beta }}^{\ast };{\mathbf{X}}_i)-s)Y_{i}I(G_i=0)W_i∕\left\{{\sum }_{i=1}^n{K}_h\right(D({\widehat{\beta }}^{\ast };{\mathbf{X}}_i)-s\left)I\right(G_i=0\left)W_i\right\}$ and ${\widehat{\beta }}_j^{\ast }$ is the solution to $n_j^{-1}{\sum }_{i=1}^{n}I(G_i=j){\mathbf{U}}_j(\beta ;Y_i,{\mathbf{X}}_i)W_i+{\lambda }_{\mathit{jn}}\beta =0$.

Using arguments similar to Park and Wei (2003) and Cai et al. (2005), we can show that, the distribution of $n^{\frac{1}{2}}({\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}-{\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}})$ can be approximated by the conditional distribution of $n^{1∕2}({\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}^{\ast }-{\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}})$ given the data. Therefore, a 100(1 – α)% confidence interval (CI) for ${\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}}$ is constructed as ${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}\pm z_{100(1-\alpha ∕2)}{\widehat{\sigma }}_{\mathbb{V}}$ or ${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}^{\left(\mathit{cv}\right)}\pm z_{100(1-\alpha ∕2)}{\widehat{\sigma }}_{\mathbb{V}}$, where ${\widehat{\sigma }}_{\mathbb{V}}$ is obtained as the standard error (SE) of ${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}^{\ast }$ and z_q is the qth percentile of the standard normal distribution. When h = O(n^−v) with v ∈ (1/5, 1/2), the resampling method can also be used to obtain CIs for c° based on $\widehat{c}$. However, we acknowledge that it is unclear whether the proposed resampling procedures or the standard bootstrap can be used to approximate the distribution of $\widehat{¢}$ due to its non-regular distribution. Our numerical results seem to suggest that the perturbation works reasonably well in finite sample.

3.4 Incremental Value (IncV) of Markers in Improving ITR

When a set of new markers X_new is available to further improve ITR, it is important to assess their IncV, especially if these markers are costly or invasive. Using the population average outcome, we may quantify the IncV of the new markers by comparing the ITRs constructed with X_old and ${\mathbf{X}}_{\mathit{update}}={({\mathbf{X}}_{\mathit{old}}^{⊺},{\mathbf{X}}_{\mathit{new}}^{⊺})}^{⊺}$, denoted respectively by ${\mathcal{I}}_{\mathrm{calib}}^{\mathit{old}}$ and ${\mathcal{I}}_{\mathrm{calib}}^{\mathit{update}}$, with respect to their value functions, i.e. the cost-adjusted population average outcomes. Specifically, the IncV of X_new can be quantified as $\mathrm{V}={\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}^{\mathit{update}}}-{\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}^{\mathit{old}}}$. Based on the inference procedures described above, we obtain a plug-in estimate of IncV as $\widehat{\mathrm{IncV}}={\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}^{\mathit{update}}}-{\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}^{\mathit{old}}}$.

The CI of IncV can be constructed via the perturbation-resampling approach as well. For each set of perturbation random variables {W_i, i = 1, … , n}, we follow the perturbation procedures described in Section 3.3 to obtain perturbed counterpart of (${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}^{\mathit{old}}}$, ${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}^{\mathit{update}}}$) as (${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}^{\mathit{old}}}^{\ast }$, ${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}^{\mathit{update}}}^{\ast }$). Then the perturbed counterpart of $\widehat{\mathrm{IncV}}$ can be obtained as ${\widehat{\mathrm{IncV}}}^{\ast }={\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}^{\mathit{update}}}^{\ast }-{\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}^{\mathit{old}}}^{\ast }$. The empirical distribution of $n^{\frac{1}{2}}({\widehat{\mathrm{IncV}}}^{\ast }-\widehat{\mathrm{IncV}})$ conditional on the data can be used to approximate the distribution of $n^{\frac{1}{2}}(\widehat{\mathrm{IncV}}-\mathrm{IncV})$ and construct the CI accordingly.

4. Simulation Studies

To evaluate the performance of the proposed method under practical settings, we conducted extensive simulation studies to examine the finite-sample properties of the proposed point and interval estimation estimators. In addition, we compared the performance of the proposed procedures to that of Qian and Murphy (2011) and Zhang et al. (2012a) with respect to the achievable value function. Throughout the simulation studies, we let n₀ = n₁ = 250 and set ξ = 0 for simplicity. All the CIs were obtained using 500 perturbed samples with W_i ~ exp(1). All the results are summarized based on 1000 simulated datasets.

4.1 Evaluation of the Proposed Method

We first evaluated the proposed point and interval estimation procedures for the value functions. We generated covariates from

$$\left[\begin{array}{c}\hfill \log \left(X_1\right)\hfill \\ \hfill X_2\hfill \\ \hfill \log \left(X_3\right)\hfill \\ \hfill X_4\hfill \end{array}\right]\sim N\left(\left[\begin{array}{c}\hfill 1.5\hfill \\ \hfill 5.0\hfill \\ \hfill -3.0\hfill \\ \hfill 2.0\hfill \end{array}\right],\left[\begin{array}{cccc}\hfill 4.0\hfill & \hfill 0.8\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0.8\hfill & \hfill 16.0\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 8.0\hfill & \hfill 3.2\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 3.2\hfill & \hfill 8.0\hfill \end{array}\right]\right);X_5\sim \mathrm{Bernoulli}\left(0.5\right)$$

Given $\check{\mathbf{X}}={(1,\log (X_1),X_2,\log (X_3),X_4,X_5)}^{⊺}$, two types of outcomes were generated: (1) a continuous outcome from a linear model

$$Y_i={\beta }_1^{⊺}{\check{\mathbf{X}}}_{i}I(G_i=1)+{\beta }_0^{⊺}{\check{\mathbf{X}}}_{i}I(G_i=0)+{∊}_i,$$ (4.1)

where ∊_i ~ N(0, 1) with β₁ = (1, 0.5, 1, 1.5, 1, 1)^⊤ and β₀ = (6, 2.5, −3, −1, 1.5, –2)^⊤; (2) a binary outcome from a logistic regression model

$$\mathrm{logit}\left\{P\right(Y_i=1\mid {\check{\mathbf{X}}}_i,G_i\left)\right\}={\beta }_1^{⊺}{\check{\mathbf{X}}}_{i}I(G_i=1)+{\beta }_0^{⊺}{\check{\mathbf{X}}}_{i}I(G_i=0),$$ (4.2)

where β₁ = (0.1, 0., 0.1, 0.15, 0.2, 0.1)^⊤ and β₀ = (0.6, 0.25, −0.2, −0.1, 0.15, −0.2)^⊤.

To derive ITRs, we fitted various working models with either linear (for continuous Y) or logistic regression (for binary Y) with the following different sets of covariates:

$$\begin{array}{cc}\hfill \left(M_1\right):{\mathbf{X}}_{M_1}& =\check{\mathbf{X}}\left(M_2\right):{\mathbf{X}}_{M_2}={({\check{\mathbf{X}}}^{⊺},X_6,X_7,\dots ,X_{20})}^{⊺}\hfill \\ \hfill \left(M_3\right):{\mathbf{X}}_{M_3}& ={(1,X_2,X_4,X_5)}^{⊺}\left(M_4\right):{\mathbf{X}}_{M_4}={(1,X_1,X_2,X_3)}^{⊺}.\hfill \end{array}$$

For the overfitted model (M₂), we generated X₆ ~ Bernoulli(0.7) and (X₇, … ,X₂₀)^⊤ N(μ, 8 · I14×14 + 8), where μ = (3, 1, 2, −3, 3, 1, 2, 1.5, 2.5, 1, 0.5, 2, 1, 1)^⊤; I_14×14 denotes a 14 × 14 identity matrix. To obtain model based estimate of D(X), we fit simple linear regression and logistic regression models with penalty parameter λ_jn set to 0 since the models sizes are reasonably small. It can be seen that Δ(·) is an increasing function under those four working models and hence ${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}$ is asymptotically equivalent to ${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{c}}}$ and ${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{¢}}}$.

The results for estimating ${\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}}$ via ${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{c}}}$, ${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{¢}}}$, and ${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}$ together with a 2-fold CV procedure are shown in Table 1. For all estimators of ${\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}}$, the biases are negligible and the estimated SEs are close to the empirical SEs. The 95% CIs have empirical coverage levels close to the nominal level. As expected, the CV procedure generally provides estimators with lower bias compared to the apparent estimates. Since ${\mathcal{I}}_{\mathrm{calib}}$ and ${\widehat{I}}_{\widehat{c}}$ are equivalent under monotonicity, the results for ${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}$ and ${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{c}}}$ are almost identical to each other.

Table 1.

Empirical bias (Bias), empirical standard error (ESE), average of the estimated standard error (ASE) and the empirical coverage level of the 95% CIs (95%-CP) for ${\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}}$ via ${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{¢}}}$ and ${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}$ under various working models.

(a) Continuous outcome
				Apparent			2-fold CV
Model	Method	True	ASE	Bias	ESE	95%-CP	Bias	ESE	95%-CP
	${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{¢}}}$		0.629	0.109	0.609	0.952	0.073	0.602	0.955
M 1	${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{c}}}$	9.510	0.621	0.047	0.605	0.946	0.026	0.602	0.947
	${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}$		0.621	0.047	0.605	0.946	0.026	0.602	0.947
	${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{¢}}}$		0.628	0.108	0.610	0.949	0.076	0.605	0.954
M 2	${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{c}}}$	9.510	0.621	0.048	0.603	0.946	0.022	0.600	0.946
	${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}$		0.621	0.048	0.603	0.946	0.022	0.600	0.946
	${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{¢}}}$		0.632	0.124	0.629	0.948	0.089	0.623	0.944
M 3	${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{c}}}$	8.842	0.621	0.044	0.620	0.945	−0.020	0.616	0.945
	${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}$		0.621	0.044	0.620	0.945	−0.020	0.616	0.945
	${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{¢}}}$		0.639	0.212	0.633	0.962	0.146	0.630	0.956
M 4	${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{c}}}$	8.834	0.637	0.137	0.627	0.955	0.107	0.623	0.944
	${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}$		0.630	0.074	0.621	0.951	0.057	0.613	0.944

(b) Binary outcome
				Apparent			2-fold CV
Model	Method	True	ASE	Bias	ESE	95%-CP	Bias	ESE	95%-CP
	${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{c}}}$		0.042	0.028	0.037	0.927	0.016	0.037	0.935
M 1	${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{c}}}$	0.766	0.039	0.018	0.036	0.948	0.008	0.036	0.963
	${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}$		0.039	0.019	0.035	0.946	0.007	0.036	0.960
	${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{¢}}}$		0.043	0.042	0.037	0.905	0.010	0.037	0.972
M 2	${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{c}}}$	0.766	0.041	0.033	0.035	0.920	−0.005	0.036	0.974
	${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}$		0.040	0.034	0.035	0.921	−0.004	0.036	0.976
	${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{¢}}}$		0.041	0.045	0.036	0.902	0.029	0.037	0.933
M 3	${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{c}}}$	0.754	0.039	0.029	0.035	0.946	0.019	0.035	0.949
	${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}$		0.038	0.030	0.035	0.940	0.020	0.035	0.943
	${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{¢}}}$		0.048	0.039	0.042	0.900	0.031	0.041	0.915
M 4	${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{c}}}$	0.749	0.042	0.032	0.041	0.915	0.024	0.040	0.924
	${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}$		0.041	0.019	0.040	0.924	0.015	0.039	0.940

Although ${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{c}}}$ and ${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{¢}}}$ are also asymptotically equivalent, we note that ${\widehat{\mathbb{V}}}_{{\widehat{I}}_{\widehat{¢}}}$ tends to have slightly larger bias and variation in finite samples. This could be in part due to the larger variability in estimating the corresponding optimal threshold values as shown in Table 2. The bias and variance of $\widehat{¢}$ are substantially larger than those of $\widehat{c}$. The efficiency of $\widehat{¢}$ relative to $\widehat{c}$ is only about 66-75% for most of the settings we considered. This confirms the disadvantage of estimating c° by directly maximizing the non-smooth empirical objective function ${\widehat{\mathbb{V}}}_{{\widehat{I}}_c}$.

Table 2.

Empirical bias (Bias), empirical standard error (ESE), average of the estimated standard errors (ASE) and empirical coverage levels of 95% CIs (95%-CP) for estimating the optimal threshold c based on $\widehat{c}$ = argmax_c ${\widehat{\mathbb{V}}}_{{\widehat{I}}_c}$ and $\widehat{c}=\{c:\widehat{\Delta }(c)=0\}$.

		Continuous					Binary
Model	Method	True	Bias	ESE	ASE	95%-CP	True	Bias	ESE	ASE	95%-CP
M 1	$\widehat{¢}$	0.00	−0.178	1.667	1.753	0.960	0.00	−0.012	0.115	0.121	0.961
	$\widehat{c}$		0.036	1.358	1.407	0.967		−0.001	0.100	0.106	0.971
M 2	$\widehat{¢}$	0.00	−0.201	1.647	1.757	0.964	0.00	−0.021	0.118	0.128	0.958
	$\widehat{c}$	−0.0	−0.007	1.355	1.407	0.972		0.004	0.112	0.118	0.972
M 3	$\widehat{¢}$	−0.019	−0.212	1.764	1.851	0.967	−0.023	−0.022	0.109	0.114	0.954
	$\widehat{c}$		0.101	1.480	1.550	0.965		−0.008	0.094	0.100	0.964
M 4	$\widehat{¢}$	−0.121	0.190	1.693	1.853	0.979	0.017	−0.030	0.103	0.105	0.951
	$\widehat{c}$		0.071	1.491	1.547	0.946		−0.018	0.099	0.102	0.964

We also evaluated the finite sample performance of the inference procedure for the IncV of new markers. In Table 3, we presented results on the estimated IncV of X₁ and X₃ in improving the value function by comparing the ITRs derived from the full model (M₁) to those from model (M₃) with no information on X₁ and X₃. Our proposed procedures based on ${\widehat{\mathcal{I}}}_{\mathrm{calib}}$ and ${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}}$ give minimally biased estimates of the IncV and the resampling procedures provide good estimates for the SEs and CIs. Since the model sizes are small, both the apparent estimates and the CV estimates lead to reasonable interval estimates with proper coverage levels. For comparison, we provided the results based on ${\widehat{I}}_{\widehat{¢}}$ as well which also leads to consistent estimates of the IncV since Δ(·) is monotone for both models. Similar to the results shown earlier, the direct maximization of ${\widehat{\mathbb{V}}}_{{\widehat{I}}_c}$ resulted in larger variability in ${\widehat{I}}_{\widehat{¢}}$, which subsequently lead to higher variability in estimating IncV in finite sample.

Table 3.

Empirical bias (Bias), empirical standard errors (ESE), average of the estimated standard errors (ASE) and empirical coverage levels of the 95% CIs for the IncV comparing the optimal ITR under the full model versus the reduced model, where the optimal ITRs are estimated based on ${\widehat{I}}_{\widehat{¢}}$, and ${\widehat{\mathcal{I}}}_{\mathrm{calib}}$.

				Apparent			2-fold CV
Response	Method	True	ASE	Bias	ESE	95%-CP	Bias	ESE	95%-CP
Continuous	${\widehat{I}}_{\widehat{¢}}$	0.663	0.215	0.025	0.217	0.946	−0.019	0.218	0.948
	${\widehat{\mathcal{I}}}_{\mathrm{calib}}$		0.187	0.013	0.183	0.948	−0.007	0.184	0.952
Binary	${\widehat{I}}_{\widehat{¢}}$	0.011	0.026	0.005	0.024	0.973	0.002	0.024	0.966
	${\widehat{\mathcal{I}}}_{\mathrm{calib}}$		0.021	0.004	0.020	0.966	0.001	0.019	0.957

4.2 Comparisons to Existing Methods

Additional simulation studies were conducted to compare our calibration methods with those proposed in Qian and Murphy (2011) (QM) and Zhang et al. (2012a) (Zhang). The QM method employs model based ITRs but guard against model mis-specification by including non-linear basis functions and then obtains stable parameter estimates by imposing L₁ penalization. In our simulations, we included all linear effects and two-way interactions for their method. To apply the Zhang method, we let the propensity score be 0.5 and search for the optimal ITR using the linear regression followed by a CART procedure. The complexity parameter was first set at 0.001 to build a large tree which is then pruned via a 10-fold CV.

To compare the performance of these methods, we generated X_20×1 ~ N(0, 2.4I_20×20+1.6) and a continuous Y that depends non-linearly to the first 5 covariates through

$$Y_i={\beta }_1^{⊺}{\tilde{\mathbf{X}}}_{i}I(G_i=1)+{\beta }_0^{⊺}{\tilde{\mathbf{X}}}_{i}I(G_i=0)+{∊}_i\mathrm{with}{∊}_i\sim N(0,1)$$ (4.3)

where $\tilde{\mathbf{X}}=(1,e^{X_1∕3},X_2,e^{X_3∕4},X_4,I(X_5>0)$, $X_2^2$, sin(X₄))^⊤, β₀ = (12, 5, −6, −2, 3, −4, 7, −2)^⊤ and β₁ = (2, 1, 2, 3, 2, 2, 3, 3)^⊤. To construct the ITRs, we investigated four sets of working linear models with covariates being

$$\begin{array}{cc}\hfill \left(M_1^{\prime }\right):{\mathbf{X}}_{M_1^{\prime }}& =\tilde{\mathbf{X}}\left(M_2^{\prime }\right):{\mathbf{X}}_{M_2^{\prime }}={({\tilde{\mathbf{X}}}^{⊺},X_6,X_7,\dots ,X_{20})}^{⊺}\hfill \\ \hfill \left(M_3^{\prime }\right):{\mathbf{X}}_{M_3^{\prime }}& ={(1,X_1,X_2,X_3,X_4,X_5)}^{⊺}\left(M_4^{\prime }\right):{\mathbf{X}}_{M_4^{\prime }}={(1,X_1,X_2,X_3)}^{⊺}.\hfill \end{array}$$

For ($M_3^{\prime }$) and ($M_4^{\prime }$), Δ(·) is non-monotone. Since our proposal also allows non-linear transformations, we derive ${\widehat{\mathcal{I}}}_{\mathrm{calib}}$ based on $D(\widehat{\beta },\mathbf{X})$ using (i) linear models (${\widehat{\mathcal{I}}}_{\mathrm{calib}}^L$) and (ii) linear + two-way interactions with ridge penalty (${\widehat{\mathcal{I}}}_{\mathrm{calib}}^{L+I}$). For comparison, we also consider the model based rule by assigning patients according to $D(\widehat{\beta },\mathbf{X})\ge 0$ for both linear effects (Model^L) and linear + interaction effects (ModelL+I). The ridge penalty parameters for ModelL+I and ${\widehat{\mathcal{I}}}_{\mathrm{calib}}^{L+I}$ were chosen via the CV. For all scenarios, we estimated ITRs with training sets of sizes n₀ = n₁ = 250 and evaluated their corresponding value functions on independent validation datasets with n₀ = n₁ = 10⁵. We use large validation sets to ensure that the variability observed in the empirical value function reects mainly the variability due to estimating the ITRs. Table 4 summarizes the mean and empirical SE (ESE) of the empirical value functions for (1) Model^L; (2) ${\widehat{\mathcal{I}}}_{\mathrm{calib}}^L$; (3) Model^L+I ; (4) ${\widehat{\mathcal{I}}}_{\mathrm{calib}}^{L+I}$; (5) QM; and (6) Zhang.

Table 4.

Empirical mean (Mean) and the corresponding empirical standard error (ESE) for ${\mathbb{V}}_{{\mathcal{I}}_{\mathrm{calib}}}$ via different methods under various working models.

		ModelL	${\widehat{\mathcal{I}}}_{\mathrm{calib}}^L$	ModelL+I	${\widehat{\mathcal{I}}}_{\mathrm{calib}}^{L+I}$	QM	Zhang
$M_1^{\prime }$	Mean	42.39	42.39	42.39	42.39	42.38	42.06
	ESE	0.13	0.13	0.13	0.13	0.13	0.15
$M_2^{\prime }$	Mean	42.39	42.39	40.90	42.01	42.35	42.04
	ESE	0.13	0.13	0.51	0.16	0.14	0.16
$M_3^{\prime }$	Mean	39.65	41.95	40.47	41.99	40.61	41.75
	ESE	0.95	0.17	0.39	0.15	0.43	0.72
$M_4^{\prime }$	Mean	39.61	41.89	40.42	41.92	40.58	41.72
	ESE	1.01	0.17	0.52	0.15	0.57	0.70

Under the true model ($M_1^{\prime }$) with 5 covariates, all methods perform very similarly. Our calibrated method performs almost identically to the model based method, suggesting that little price is paid for the additional calibration. The QM method also did not pay much price for including the non-informative two-way interactions in this case. With the over-fitted model ($M_2^{\prime }$), except for Model^L+I, all other methods achieve similar level of value function with slight difference in the variability. The QM method performs quite well considering that about 200 covariates are included in the fitting with only 5 are informative. This is not too surprising since L₁ penalization is expected to work well when the signals are strong and sparse, as in the present case. Both the QM and our method with ${\widehat{\mathcal{I}}}_{\mathrm{calib}}^L$ achieved value functions almost identical to those obtained under ($M_1^{\prime }$). Our method with ${\widehat{\mathcal{I}}}_{\mathrm{calib}}^{L+I}$ and the Zhang method result in slightly higher variability compared to the QM and ${\widehat{\mathcal{I}}}_{\mathrm{calib}}^L$. Model^L+I has a slightly lower value function with larger variability, suggesting instability in ridge penalized modeling fitting. However, the calibration appears to have the ability to reduce the instability with a much more stable ${\widehat{\mathcal{I}}}_{\mathrm{calib}}^{L+I}$ compared to Model^L+I. In general, the price paid for overfitting under correct model specification seems to be relatively low. This is in part because all methods are building on top of correctly specified models and hence the estimated regression parameters are maximizing the value function asymptotically. As a result the variability due to estimating all parameters in the ITR contributes at a second order, similar to those argued in Zhao et al. (2012).

With the mis-specified working models ($M_3^{\prime }$) and ($M_4^{\prime }$), our proposed calibration method performs better than all other competing methods with respect to the achievable value function and/or the variability. For example, for ($M_3^{\prime }$), the average value function was 41.95 and 41.99 for ${\widehat{\mathcal{I}}}_{\mathrm{calib}}^L$ and ${\widehat{\mathcal{I}}}_{\mathrm{calib}}^{L+I}$ respectively, 40.61 for the QM method, and 41.75 for the Zhang method. Since the true underlying effects are quite non-linear, the model based method with linear effects perform poorly in this setting with value function of only 39.65. It seems that there is a slight increase in the achievable value function from our calibration method by including the non-linear basis in the working models. Under mis-specified models, the calibration method tends to produce more stable ITRs than those from the QM and Zhang methods. For example, the ESE was 0.17 and 0.15 for ${\widehat{\mathcal{I}}}_{\mathrm{calib}}^L$ and ${\widehat{\mathcal{I}}}_{\mathrm{calib}}^{L+I}$ respectively, 0.43 for the QM method, and 0.72 for the Zhang method.

5. Example: Application to an HIV/AIDS Clinical Trial

In this section, we use a randomized clinical trial from the AIDS Clinical Trials Group Protocal 320 (ACTG 320) to illustrate our proposed methods. A total of 1156 zidovudine-experienced patients with advanced human immunodeficiency virus (HIV) type-I infection were enrolled in ACTG 320 and assigned to either a 2-drug combination of zidovudine and lamivudine (treatment 0) or a 3-drug combination of zidovudine, lamivudine and indinavir (treatment 1). The objective of this study was to assess the added value of a protease inhibitor (indivar) to the dual nucleoside reverse-transcriptase inhibitors (xidovudine and lamivudine) (Hammer et al., 1997). The overall success of the 3-drug combination on various study endpoints was so significant that the study was terminated early by the Data Safety Monitoring Board. However, since the benefit of 3-drug combination therapy over the 2-drug alternative potentially differs across patients, it would be interesting to identify subgroups of patients who can be managed almost as well using the less potent 2-drug therapy.

For our analyses, the potential baseline predictors for constructing ITRs consist of age (years), CD4₀, logCD4₀, and log₁₀RNA₀, where CD4_k and RNA_k denote, respectively, the CD4 cell count (cells/mm³) and the RNA measure (copies/ml) at week k. We restricted our study to the 856 subjects who had complete information on these variables and on the outcome of interest, 427 of which were in the 3-drug combination group. Because it is relatively expensive to measure RNA, particularly in resource-limited settings, it would be of interest to examine whether X_new = log₁₀RNA₀ is useful for improving the ITRs. Thus, we compared the optimal ITRs based on the various working models with the following two sets of predictors: (M_old) : X_old=(1, Age, CD4₀, logCD4₀)^⊤

$$\left(M_{\mathit{update}}\right):{\mathbf{X}}_{\mathit{update}}={(1,\mathrm{Age},\mathrm{CD}{4}_0,\mathrm{logCD}{4}_0,{\log }_{10}{\mathrm{RNA}}_0)}^{⊺}$$

To quantify the effectiveness of the therapy, we considered the immune response Y defined as the change in logCD4 from baseline to week 24. Table 5 summarizes the estimated regression coefficients for fitted linear models with (M_update) and (M_old). To construct optimal ITRs under these two models, we let ξ = 0.277, which is about the within subject variability of logCD4 counts (Hughes et al., 1994), indicating that 3-drug therapy is only preferred if the gain in immune response exceeds ξ. All estimators of the value functions are based on 500 repeated two-fold CVs. The SEs are based on 500 perturbations.

Table 5.

Estimated regression coefficients for the immune response (change in CD4 counts from baseline to week 24) based on the ACTG 320 Data.

Model	Treatment		Intercept	Age	CD40	log(CD40)	log10(RNA0)
	2-drug	Estimate	1.065	0.002	0.004	−0.338	0.011
		Std. Error	0.350	0.004	0.001	0.057	0.049
		p-value	0.002	0.568	< 0.001	< 0.001	0.829
Mupdate
	3-drug	Estimate	1.317	0.009	−0.001	−0.328	0.128
		Std. Error	0.348	0.004	0.001	0.062	0.051
		p-value	< 0.001	0.023	0.199	< 0.001	0.013
	2-drug	Estimate	1.124	0.002	0.004	−0.338
		Std. Error	0.218	0.004	0.001	0.057
		p-value	< 0.001	0.584	< 0.001	< 0.001
Mold
	3-drug	Estimate	1.984	0.008	−0.002	−0.318
		Std. Error	0.225	0.004	0.001	0.062
		p-value	< 0.001	0.035	0.066	< 0.001

Under (M_update), the maximum cost-adjusted value function is ${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}^{\mathit{update}}}$ = 0.610 with a 95% CI (0.526, 0.692). Its corresponding optimal threshold is equal to 0.236 with a 95% CI=(−0.040, 0.512). Under (M_old), i.e. without the RNA information, we have ${\widehat{\mathbb{V}}}_{{\widehat{\mathcal{I}}}_{\mathrm{calib}}^{\mathit{old}}}$ =0.606 with a 95% CI (0.522, 0.690) and $\widehat{c}$ = 0.231 with a 95% CI=(−0.004, 0.466). Finally, the estimated difference $\widehat{\mathrm{IncV}}$ is equal to 0.004 with the 95% CI (−0.035, 0.043). This implies that despite the highly significant difference between the effect of RNA on the outcome between the two treatment groups (as shown in Table 5), including RNA information in the ITR does not improve the value function. Therefore, RNA is not necessary to better assign future patients to a specific treatment. By including two-way interactions of all variables under (M_update) and (M_old), our calibrated method yields ITRs with estimated values 0.614 for (M_update) and 0.609 for (M_old), similar to their linear effect counterparts. We also applied the QM and Zhang methods to ACTG 320 data set and observed slightly lower value functions compared to those from our methods. The value function associated with the models (M_update) and (M_old) are, respectively, 0.594 and 0.585 based on the Zhang method, and 0.578 and 0.569 based on the QM method.

It would also be of interest to compare to the simple rules that assign all patients to the 3-drug combination group or to the 2-drug group. The value functions are estimated as 0.562 if all treated with 3-drug and 0.177 if all treated with 2-drug. Comparing to assigning all patients to 3-drug, ${\widehat{\mathcal{I}}}_{\mathrm{calib}}^{\mathit{update}}$ leads to an increased value of 0.48 with a 95% CI [0.003, 0.093], suggesting an improvement by adopting the ITR.

6. Remarks

In this paper, we have proposed a robust procedure that uses multiple baseline covariates to develop and evaluate ITRs. While the procedure builds upon an existing two-step framework, this paper provides additional insights into how to develop an optimal ITR based on working models and how to evaluate such ITRs. Fitting the data with semi-parametric models in step I, combined with the non-parametric estimation in step II, provides robust estimates of ITRs and valid estimates of their associated cost-adjusted population average outcomes. Our proposed ITRs are optimal across all rules based on the given set of covariates when the fitted working models are correct or nearly correct as discussed in section 2. The ITR is optimal among all rules based on the estimated scores when the fitted models are mis-specified. To account for the variability in estimating various parameters, we proposed perturbation-resampling procedures that can be used to assess the variability in the estimators.

While in traditional statistical methods, model misspecification may hinder predictions and lead to inaccurate assignment rules, the methods developed in this paper rely on a layer of calibration in step II to guard against model misspecification. We provide justifications for when the proposed procedures lead to ITRs that are globally optimal under correct or near correct model specification and optimal within a class of ITR rules under model misspecification. Under model misspecification, the proposed ITR could be suboptimal when compared to the global optimal ITR. Hence, it would be crucial to provide working models that can approximate the true model reasonably well. Incorporating non-linear effects through basis function specification could be helpful and one may use the proposed inference procedure for comparing value functions as a tool for selecting important bases functions.

Through simulation studies and theoretical studies, we also demonstrate that direct optimization of the empirical value function ${\widehat{\mathbb{V}}}_{{\widehat{I}}_c}$ could lead to rather unstable ITRs, with high variability and slow convergence rate in estimating the optimal threshold values, even when the underlying conditional treatment difference function Δ(c) is monotone in c.

When there are new biomarkers introduced to assist in treatment selection, it is important to evaluate their value in improving population average outcomes. It is important to note that a variable highly differentially associated with Y⁽¹⁾ and Y⁽⁰⁾ may not necessarily be important for improving ITRs. This is somewhat similar to the phenomenon observed in the risk prediction literature: a variable highly significant in regression modeling may not result in large improvement in prediction. However, the IncV with respect to improving ITRs is more subtle than the typical prediction setting. Taking an extreme case scenario with a single marker X and cost ξ = 0, suppose D(X) is strictly increasing and D(X) >> 0 for all X. Then obviously X will be selected as important for predicting treatment response by any variable selection procedure. On the other hand, X would be deemed as not important for treatment selection since all subjects should be assigned to treatment 1 and having the information on X does not change the treatment assignment or the corresponding value function. Thus, if one is interested in measuring the importance of markers in guiding treatment selection, it would be valuable to directly assess the IncV with respect to the value function as proposed in this paper. When the dimension of new markers is not small, it would be crucial to employ the cross-validation to correct for the overfitting bias as suggested by Zhao et al. (2013). Procedures for efficiently selecting the informative markers warrant further research.