Model-free approach to quantifying the proportion of treatment effect explained by a surrogate marker

Summary

In randomized clinical trials, the primary outcome, Y, often requires long-term follow-up and/or is costly to measure. For such settings, it is desirable to use a surrogate marker, S, to infer the treatment effect on Y, Δ. Identifying such an S and quantifying the proportion of treatment effect on Y explained by the effect on S are thus of great importance. Most existing methods for quantifying the proportion of treatment effect are model based and may yield biased estimates under model misspecification. Recently proposed nonparametric methods require strong assumptions to ensure that the proportion of treatment effect is in the range [0, 1]. Additionally, optimal use of S to approximate Δ is especially important when S relates to Y nonlinearly. In this paper we identify an optimal transformation of S, g_opt(·), such that the proportion of treatment effect explained can be inferred based on g_opt(S). In addition, we provide two novel model-free definitions of proportion of treatment effect explained and simple conditions for ensuring that it lies within [0, 1]. We provide nonparametric estimation procedures and establish asymptotic properties of the proposed estimators. Simulation studies demonstrate that the proposed methods perform well in finite samples. We illustrate the proposed procedures using a randomized study of HIV patients.

1. Introduction

When a new treatment or prevention strategy becomes available, randomized clinical trials are often conducted to compare its efficacy to a placebo or standard care. Such trials, however, are complex and costly to perform (DiMasi et al., 2010). To ensure the efficient and timely arrival of new and affordable interventions, it is thus crucial to explore effective approaches to randomized clinical trial design (Food and Drug Administration, 2004). One key challenge is that the primary outcomes of many clinical trials are often expensive to measure and/or require long-term follow-up of patients. This gives rise to an increasing interest in identifying and validating surrogate markers that can be used instead to infer treatment effect on the primary outcome. Using such surrogate endpoints can be cost effective and can reduce participant burden if the true target clinical endpoint is invasive or difficult to measure.

The potential advantages of using a surrogate endpoint as a substitute for a primary endpoint have led to a considerable number of statistical methods for evaluating the validity of surrogate endpoints. Prentice (1989) proposed a seminal framework for evaluating the validity of a surrogate via hypothesis testing. A surrogate endpoint is considered as valid only if a test for treatment effect on the surrogate endpoint is also a valid test for treatment effect on the primary outcome. Shifting the focus from testing to estimation, Freedman et al. (1992) considered estimating the proportion of treatment effect explained by a surrogate, by assessing the change in the magnitude of the treatment effect estimate when a surrogate is added to a specified regression model. Lin et al. (1997) subsequently proposed a proportion of treatment effect explained measure for failure time endpoints using a time-dependent Cox proportional hazards model. The validity of the proportion of treatment effect explained estimates from these methods relies heavily on the validity of the specified regression models with and without the surrogate marker, yet these two sets of models often do not hold simultaneously (Lin et al., 1997). In addition to the proportion of treatment effect explained, other quantities and criteria for validating a surrogate biomarker have been proposed, though they are also largely model based (Robins & Greenland, 1992; Buyse & Molenberghs, 1998; Ghosh, 2008, 2009; Huang & Gilbert, 2011; Conlon et al., 2017). Wang & Taylor (2002) proposed a more flexible approach to quantifying the proportion of treatment effect explained, by examining what the treatment effect would have been if the surrogate had identical distributions among the treatment groups. Their approach still requires modelling choices. Based on the proportion of treatment effect explained definition of Wang & Taylor (2002), Parast et al. (2016) proposed a fully nonparametric model-free estimation procedure. However, both Wang & Taylor (2002) and Parast et al. (2016) require additional assumptions, for example that the relationship between the surrogate and the primary outcome is monotone, which is needed to ensure that the proportion of treatment effect explained quantity is between 0 and 1. In addition, Parast et al. (2016) requires that the support of the surrogate biomarker distribution is the same in the treatment and control groups. While often reasonable, these assumptions may not hold for some practical settings, and as such the proportion of treatment effect explained estimates from these methods may be biased and beyond the range of [0, 1].

In this paper we propose a novel approach to quantifying the proportion of treatment effect explained by a potential surrogate marker, S, by first identifying an optimal transformation of S, S → g_opt(S), such that g_opt(S) achieves the lowest mean squared prediction error for the outcome Y. With g_opt(·) obtained, we define the proportion of treatment effect explained by quantifying how well g_opt(S) can be used to infer the true treatment effect on Y. One could argue that this framework, i.e., directly trying to find a function of the surrogate that approximates the primary outcome, is conceptually more in line with the ultimate goal of surrogate biomarker identification, namely, to eventually replace the primary outcome. We propose two proportion of treatment effect explained definitions based on g_opt(S), pte₁ and pte₂, with pte₁ being the ratio between the treatment effect on g_opt(S) and the treatment effect on Y, and pte₂ quantifying how well the subject-level treatment effect on Y can be approximated by the effect on g_opt(S). We show that pte₁ corresponds to the quantity considered in Parast et al. (2016) under a specific choice of the reference distribution required by their method, while pte₂ has the advantage of always being between 0 and 1.We also provide nonparametric inference procedures for these proportion of treatment effect explained measures and derive their theoretical properties. Simulation results suggest that the proposed estimators perform well compared to existing methods.

2. Optimal transformation and proportion of treatment effect explained definition

2.1. Setting and notation

Let Y be the primary outcome and S the surrogate marker. The outcome Y may be discrete or continuous, and the biomarker S may be discrete or continuous. Throughout we treat S as continuous, but when S is discrete, either ordinal or categorical, all derivations and theoretical results remain valid with density functions replaced by probability mass functions. Let A be the treatment indicator, with A = 1 denoting the treated group and A = 0 denoting the control group; we assume the treatment is randomly assigned to the patients at baseline. We use the standard causal inference framework to define {Y^(a),S^(a)} as the potential outcome and surrogate marker under treatment A = a. In practice, (Y⁽¹⁾, S⁽¹⁾) and (Y⁽⁰⁾, S⁽⁰⁾) cannot be observed simultaneously for an individual. We assume that the data for analysis consist of n independent and identically distributed random variables {D_i = (Y_i, S_i, A_i), i = 1, …, n} and P(A_i = 1) = p₁ ∈ (0, 1), where $Y_i=A_i{Y}_i^{(1)}+\left(1-A_i\right)Y_i^{(0)}$ and $S_i=A_i{S}_i^{(1)}+\left(1-A_i\right)S_i^{(0)}$. Without loss of generality, we assume that p₁ = 0.5 and discuss the estimation adjustment for p₁ ╪ 0.5 in the discussion section. Throughout, the treatment effect on Y is defined as

$$\mathrm{\Delta }={\mu }_1-{\mu }_0,\mathrm{where}{\mu }_a=E\left(Y^{(a)}\right)=E(Y|A=a),$$

and without loss of generality we assume that Δ ⩾ 0.

2.2. Optimal function of the surrogate biomarker

Our goal is to find an optimal function of S, g_opt(·), such that g_opt(S) can be used to approximate the primary outcome and subsequently to quantify the treatment effect on Y, where the same g_opt(·) is applied to both treatment arms. Price et al. (2018) also recently proposed the idea of finding the optimal function of a potential surrogate. Their proposed approach identifies the optimal transformation of the surrogate for each treatment group separately; that is, the optimal functions are different depending on the treatment group. Our aim is different in that we aim to identify a single optimal function of S that can be used regardless of the treatment group. We translate our problem of interest into a prediction framework and aim to identify g_opt(·) that minimizes the mean squared error loss function

$${\mathcal{L}}_{\mathrm{oracle}}(g_{\mathrm{opt}})=E{\left[(Y^{(1)}-Y^{(0)})-\{g_{\mathrm{opt}}(S^{(1)})-g_{\mathrm{opt}}(S^{(0)})\}\right]}^2.$$

Unfortunately, since (Y⁽¹⁾, S⁽¹⁾) and (Y⁽⁰⁾, S⁽⁰⁾) cannot be observed simultaneously at the individual level and the correlations between (Y⁽¹⁾, S⁽¹⁾) and (Y⁽⁰⁾, S⁽⁰⁾) are not identifiable, we instead aim to minimize the above squared loss under a working independence assumption:

$$(Y^{(1)},S^{(1)})\perp (Y^{(0)},S^{(0)}),$$ (1)

which leads to

$$\mathcal{L}(g)=E[{\{Y^{(1)}-g_{\mathrm{opt}}(S^{(1)})\}}^2+{\{Y^{(0)}-g_{\mathrm{opt}}(S^{(0)})\}}^2]-2E\{Y^{(1)}-g_{\mathrm{opt}}(S^{(1)})\}E\{Y^{(0)}-g_{\mathrm{opt}}(S^{(0)})\}.$$

Under (1), E[{Y⁽¹⁾−g(S⁽¹⁾)}{Y⁽⁰⁾−g(S⁽⁰⁾)}] can be simplified to E{Y⁽¹⁾−g(S⁽¹⁾)}E{Y⁽⁰⁾−g(S⁽⁰⁾)}. Furthermore, since g_opt(·) is only identifiable up to a constant shift, we define the optimal g, denoted by g_opt(s), as the following constrained minimizer:

$$\begin{gathered} g_{\mathrm{opt}}={\mathrm{argmin}}_{g\in \mathcal{F}:E[Y-g(S)|A=0]=0}E\left[{\{Y-g(S)\}}^{2}I(A=1)+{\{Y-g(S)\}}^{2}I(A=0)\right], \\ ={\mathrm{argmin}}_{g\in \mathcal{F}:E[Y-g(S)|A=0]=0}E\left[{\{Y-g(S)\}}^2\right], \end{gathered}$$ (2)

where $\mathcal{F}$ is the class of measurable functions. Admittedly, it may be unlikely for (1) to hold in practice. However, this assumption is only used to facilitate the derivation g_opt(·) and is not required for the interpretation of our proposed proportion of treatment effect explained measure nor for the validity of the associated inference procedures. Even when this assumption is violated, the derived g_opt(·) may still be a sensible choice for transforming the surrogate marker. We provide additional comments on the implications of violations of this working assumption in §6.

In the Supplementary Material we show that under (1) the optimal function g_opt(·) can be expressed as

$$g_{\mathrm{opt}}(s)=m(s)+\frac{\lambda {\dot{F}}_0(s)}{{\dot{F}}_0(s)+{\dot{F}}_1(s)}=m(s)+\lambda {\mathcal{P}}_0(s),$$

where

$$m(s)=E(Y|S=s)=m_1(s){\mathcal{P}}_1(s)+m_0(s){\mathcal{P}}_0(s),m_a(s)=E\left(Y^{(a)}|S^{(a)}=s\right),$$

${\dot{F}}_a(s)=\mathrm{d}{F}_a(s)/\mathrm{d}s$ and F_a(s) are the respective density and cumulative distribution functions of $S^{(a)},{\mathcal{P}}_a(s)={\dot{F}}_a(s)/\left\{{\dot{F}}_0(s)+{\dot{F}}_1(s)\right\}=P(A=a|S=s)$, and

$$\lambda =\frac{{\mu }_0-\int m(s)\mathrm{d}{F}_0(s)}{\int {\mathcal{P}}_0(s)\mathrm{d}{F}_0(s)}=\frac{\int \left\{m_0(s)-m_1(s)\right\}{\mathcal{P}}_1(s)\mathrm{d}{F}_0(s)}{\int {\mathcal{P}}_0(s)\mathrm{d}{F}_0(s)}.$$

Thus, the optimal transformation is the conditional mean function of Y given S, shifted by a scaled posterior probability function of A = 0 given S. When the treatment has no effect on the conditional expectation of Y on S, i.e., the treatment effect is completely through s, then λ = 0 and g_opt(s) reduces to g_opt(s) = E(Y | S = s) = m(s).

With g_opt(·) identified such that g_opt(S) optimally approximates Y, we may infer the treatment effect on Y based on the treatment effect on g_opt(S). This highlights a major advantage of such a framework (Price et al., 2018), which enables us to not only perform testing on the treatment effect using S, but also to directly use the treatment effect on g_opt(S), defined as

$${\mathrm{\Delta }}_{g_{\mathrm{opt}}(S)}=E\left\{g_{\mathrm{opt}}\left(S^{(1)}\right)-g_{\mathrm{opt}}\left(S^{(0)}\right)\right\},$$

to approximate the target treatment effect Δ = E{Y⁽¹⁾ − Y⁽⁰⁾}, which is the best approximation based on the surrogate marker alone. Price et al. (2018) also proposed this idea to estimate the treatment effect on the primary outcome based on the treatment effect on the transformed surrogate, though their proposed transformation is treatment specific.

2.3. Model-free definitions of the proportion of treatment effect explained

To quantify the proportion of treatment effect explained by S, a natural definition is

$${\mathrm{PTE}}_1={\mathrm{\Delta }}_{g_{\mathrm{opt}}(S)}/\mathrm{\Delta }.$$

Although pte₁ is derived from a very different perspective, it directly relates to the proportion of treatment effect explained quantity defined by Wang & Taylor (2002) and Parast et al. (2016),

$${\mathrm{PTE}}_{\mathrm{L}}={\mathrm{\Delta }}_{\mathrm{L}}/\mathrm{\Delta },\mathrm{with}{\mathrm{\Delta }}_{\mathrm{L}}=\int m_1(s)\mathrm{d}\left\{F_1(s)-\mathcal{F}(s)\right\}-\int m_0(s)\mathrm{d}\left\{F_0(s)-\mathcal{F}(s)\right\},$$

where $\mathcal{F}(\cdot )$ is some reference function which was suggested as chosen to be either F₀ or F₁ in Parast et al. (2016), though $\mathcal{F}(\cdot )$ is not restricted to be a distribution function. In the Supplementary Material we show that pte₁ corresponds to pte_L when one chooses $\mathcal{F}(s)={\int }_{-\infty }^s{\mathcal{P}}_0(s){\dot{F}}_1(s)\mathrm{d}u/\{\int {\mathcal{P}}_0(u){\dot{F}}_0(u)\mathrm{d}u\}$, which is a subdistribution function. Thus, our proposed g_opt(·) and ${\mathrm{\Delta }}_{g_{\mathrm{opt}}(S)}$ essentially provide a mechanism for selecting an optimal reference function $\mathcal{F}$ in the previously proposed definition of pte_L.

Our proposed framework has an advantage in that it allows us to relax some assumptions that are required not only by Wang & Taylor (2002) and Parast et al. (2016), but by other surrogate marker work in general. For example, Parast et al. (2016) requires that:

Condition 1. P(S ⩾ s | A = 1) > P(S ⩾ s | A = 0) for all s;

Condition 2. m₁(s) > m₀(s) for all s;

Condition 3. m₁(s) is a nondecreasing function in s;

Condition 4. S⁽¹⁾ and S⁽⁰⁾ have the same support.

These conditions are imposed to ensure their proposed proportion of treatment effect explained is between 0 and 1, but can easily fail when the supports of S⁽⁰⁾ and S⁽¹⁾ are not the same. To ensure that pte₁ is between 0 and 1, we show in the Supplementary Material that we can relax Conditions 1–4 of Parast et al. (2016) and only need to assume:

Condition 5. ${\mathbb{S}}_1(u)⩾{\mathbb{S}}_0(u)$ for all u;

Condition 6. ${\mathbb{M}}_1(u)⩾{\mathbb{M}}_0(u)$ for all u in the common support of g_opt(S⁽¹⁾) and g_opt(S⁽⁰⁾), where ${\mathbb{S}}_a(u)=P\left\{g_{\mathrm{opt}}\left(S^{(a)}\right)⩾u\right\}$ and ${\mathbb{M}}_a(u)=E\left\{Y^{(a)}|g_{\mathrm{opt}}\left(S^{(a)}\right)=u\right\}$ for a = 0, 1.

Thus, our method requires neither monotonicity nor the same surrogate support. In the Supplementary Material we show that under Conditions 5 and 6 $\mathrm{\Delta }⩾{\mathrm{\Delta }}_{g_{\mathrm{opt}}(S)}⩾0$, indicating that Δ = 0 would imply ${\mathrm{\Delta }}_{g_{\mathrm{opt}}(S)}=0$. Hence, using ${\mathrm{\Delta }}_{g_{\mathrm{opt}}(S)}$ to infer Δ will not result in a surrogate paradox situation, defined as a situation in which the treatment effect on the surrogate marker is positive and the surrogate marker is positively correlated with the primary outcome, but the treatment effect on the primary outcome is negative.

An alternative approach to define the proportion of treatment effect explained based on g_opt(S) is to frame this as the percentage of variation in Y⁽¹⁾−Y⁽⁰⁾ explained by the variation in g_opt(S⁽¹⁾)−g_opt(S⁽⁰⁾). The second definition for the proportion of treatment effect explained by g_opt(·) that we propose is based on assessing how much of the variation in Y⁽¹⁾ − Y⁽⁰⁾ is explained by g_opt(S⁽¹⁾) − g_opt(S⁽⁰⁾) under our working assumption of (Y⁽¹⁾,S⁽¹⁾) ⊥ (Y⁽⁰⁾,S⁽⁰⁾). Specifically, we define the proportion of treatment effect explained as

$${\mathrm{PTE}}_2={\left\{\frac{{\mathrm{MSE}}_{\mathrm{null}}-{\mathrm{MSE}}_{g_{\mathrm{opt}}(S)}}{{\mathrm{MSE}}_{\mathrm{null}}}\right\}}^{1/2},$$

where ${\mathrm{MSE}}_{\mathrm{null}}=E[{\{(Y_i^{(1)}-Y_j^{(0)})\}}^2]=2E\{{(Y-{\mu }_0)}^2\}$ and ${\mathrm{MSE}}_{g_{\mathrm{opt}}(S)}=E([(Y_i^{(1)}-Y_j^{(0)})-{\{g_{\mathrm{opt}}(S_i^{(1)})-g_{\mathrm{opt}}(S_j^{(0)})\}]}^2)=2E[{\{Y-g_{\mathrm{opt}}(S)\}}^2]$, for i ╪ j. Here, MSE_null represents the variation of $Y_i^{(1)}-Y_j^{(0)}$ under the null of S being completely uninformative of Y. As with pte₁, we expect pte₂ to be close to 1 if g_opt(S) is a good surrogate, and close to 0 if g_opt(S) is a useless surrogate. By the definition of g_opt(·), we know that ${\mathrm{MSE}}_{g_{\mathrm{opt}}(S)}⩽{\mathrm{MSE}}_{\mathrm{null}}$. Therefore, pte₂ is guaranteed to be between 0 and 1.

Though pte₂ has some advantages over pte₁, one important benefit of pte₁ is the interpretability of the quantity and the appeal of such an interpretation to clinicians and applied researchers. Both pte₁ and pte_L can be described as the proportion of the treatment effect on the primary outcome that is captured by the treatment effect on the surrogate marker, or the transformation of the surrogate marker. In addition to being a very intuitive concept, the pte₁ formulation also directly allows us to approximate Δ based on ${\mathrm{\Delta }}_{g_{\mathrm{opt}}(S)}$ for future studies, where the primary outcome Y is not measured.

3. Nonparametric estimation of pte₁ and pte₂

In this section we propose nonparametric estimation procedures for the optimal transformation function g_opt(s), as well as for the resulting proportion of treatment effect explained parameters. Noting that g_opt(s) involves ${\dot{F}}_a(s)$, m(s) and λ, we propose to first estimate these quantities nonparametrically via standard kernel smoothing. Specifically, we let

$${\widehat{\dot{F}}}_a(s)=n_a^{-1}\sum _{i:A_i=a}{K}_h\left(S_i-s\right),\widehat{m}(s)=\frac{{\sum }_{i=1}^n{K}_h\left(S_i-s\right)Y_i}{{\sum }_{i=1}^n{K}_h\left(S_i-s\right)},\widehat{\lambda }=\frac{{\widehat{\mu }}_0-\int \widehat{m}(s)\mathrm{d}{\widehat{F}}_0(s)}{\int {\widehat{\mathcal{P}}}_0(s)\mathrm{d}{\widehat{F}}_0(s)},$$

$${\widehat{\mu }}_a=n_a^{-1}\sum _{i:A_i=a}{Y}_i,{\widehat{\mathcal{P}}}_0(s)=\frac{{\widehat{\dot{F}}}_0(s)}{{\widehat{\dot{F}}}_1(s)+{\widehat{\dot{F}}}_0(s)}\mathrm{and}{\widehat{F}}_a(s)=n_a^{-1}\sum _{i:A_i=a}I\left(S_i⩽s\right),$$

where K_h(·) = K(·/h)/h, K(·) is a symmetric kernel function, the bandwidth h = O(n^−ν) with ν ∈ (1/5, 1/2), and $n_a={\sum }_{i=1}^{n}I\left(A_i=a\right)$. When S is discrete, the above kernel estimators can be simplified by replacing K_h(S_i − s) with I (S_i = s). Subsequently, we estimate g_opt(s) as

$$\widehat{g}(s)=\widehat{m}(s)+\widehat{\lambda }{\widehat{\mathcal{P}}}_0(s).$$

Since both ${\widehat{\dot{F}}}_a(\cdot )$ and $\widehat{m}(\cdot )$ are standard kernel density and conditional mean estimators, it is straightforward to show that $\widehat{g}(s)$ is a uniformly consistent estimator of g_opt(s) under mild regularity conditions given in the Supplementary Material. In addition, we show in the Supplementary Material that ${(nh)}^{\frac{1}{2}}\left\{\widehat{g}(s)-g_{\mathrm{opt}}(s)\right\}$ converges in distribution to a normal distribution with mean 0 and variance σ²(s).

With g_opt estimated as $\widehat{g}$, we can construct plug-in estimates for ${\mathrm{\Delta }}_{g_{\mathrm{opt}}(S)}$ as ${\widehat{\mathrm{\Delta }}}_{\widehat{g}(S)}$, where

$${\widehat{\mathrm{\Delta }}}_{g(S)}={\widehat{\mu }}_{1,g(S)}-{\widehat{\mu }}_{0,g(S)}\mathrm{and}{\widehat{\mu }}_{a,g(S)}=n_a^{-1}\sum _{i:A_i=a}g\left(S_i\right).$$

Therefore, we estimate pte₁ as $\mathrm{P}\widehat{\mathrm{T}}{\mathrm{E}}_{1,\widehat{g}}={\widehat{\mathrm{\Delta }}}_{\widehat{g}(S)}/\widehat{\mathrm{\Delta }}$, where $\widehat{\mathrm{\Delta }}={\widehat{\mu }}_1-{\widehat{\mu }}_0$. Similarly, we may estimate pte₂ as $\mathrm{P}\widehat{\mathrm{T}}{\mathrm{E}}_{2,\widehat{g}}={\left(1-\mathrm{M}\widehat{\mathrm{S}}{\mathrm{E}}_{\widehat{g}(S)}/\mathrm{M}\widehat{\mathrm{S}}{\mathrm{E}}_{\mathrm{null}}\right)}^{1/2}$,

$$\mathrm{M}\widehat{\mathrm{S}}{\mathrm{E}}_{\widehat{g}(S)}=2n^{-1}\sum _{i=1}^n{\left\{Y_i-\widehat{g}\left(S_i\right)\right\}}^2\mathrm{and}\mathrm{M}\widehat{\mathrm{S}}{\mathrm{E}}_{\mathrm{null}}=2n^{-1}\sum _{i=1}^n{\left(Y_i-{\widehat{\mu }}_0\right)}^2.$$

In the Supplementary Material we show that $\mathrm{P}\widehat{\mathrm{T}}{\mathrm{E}}_{1,\widehat{g}}$ and $\mathrm{P}\widehat{\mathrm{T}}{\mathrm{E}}_{2,\widehat{g}}$ are consistent estimators of pte₁ and pte₂, respectively. Furthermore, when h = O(n^−ν) with $v\in (1/4,1/2),n^{\frac{1}{2}}\left(\mathrm{P}\widehat{\mathrm{T}}{\mathrm{E}}_{1,\widehat{g}}-{\mathrm{PTE}}_1\right)$ and $n^{\frac{1}{2}}\left(\mathrm{P}\widehat{\mathrm{T}}{\mathrm{E}}_{2,\widehat{g}}-{\mathrm{PTE}}_2\right)$, respectively, converge in distribution to $N\left(0,{\sigma }_1^2\right)$ and $N\left(0,{\sigma }_2^2\right)$, where ${\sigma }_1^2$ and ${\sigma }_2^2$ are defined in the Supplementary Material. The normal approximation holds for $\mathrm{P}\widehat{\mathrm{T}}{\mathrm{E}}_{2,\widehat{g}}$ when pte₂ ∈ (0, 1). In practice, we may estimate ${\sigma }_1^2$ and ${\sigma }_2^2$ by empirically estimating the influence functions or via resampling similar to those employed in Parast et al. (2016). For resampling, we may generate V = (V₁, …, V_n) from independent and identically distributed nonnegative random variables with mean 1 and variance 1 such as the unit exponential distribution. For each set of V, we let ${\widehat{F}}_a^{*}(s)=\left\{{\sum }_{i:A_i=a}I\left(S_i⩽s\right)V_i\right\}/\left\{{\sum }_{i:A_i=a}{V}_i\right\},{\widehat{\mu }}_a^{*}=\left\{{\sum }_{i:A_i=a}{Y}_i{V}_i\right\}/\left\{{\sum }_{i:A_i=a}{V}_i\right\}$,

$$\begin{gathered} {\widehat{\dot{F}}}_a^{*}(s)=\frac{{\sum }_{i:A_i=a}{K}_h\left(S_i-s\right)V_i}{{\sum }_{i:A_i=a}{V}_i},{\widehat{m}}^{*}(s)=\frac{{\sum }_{i=1}^n{K}_h\left(S_i-s\right)Y_i{V}_i}{{\sum }_{i=1}^n{K}_h\left(S_i-s\right)V_i}, \\ {\widehat{\lambda }}^{*}=\frac{{\widehat{\mu }}_0^{*}-\int {\widehat{m}}^{*}(s)\mathrm{d}{\widehat{F}}_0^{*}(s)}{\int {\widehat{\mathcal{P}}}_0^{*}(s)\mathrm{d}{\widehat{F}}_0^{*}(s)},{\widehat{\mathcal{P}}}_0^{*}(s)=\frac{{\widehat{\dot{F}}}_0^{*}(s)}{{\widehat{\dot{F}}}_0^{*}(s)+{\widehat{\dot{F}}}_1^{*}(s)}\mathrm{and}{\widehat{\mu }}_{a,g(S)}^{*}=\frac{{\sum }_{i:A_i=a}g\left(S_i\right)V_i}{{\sum }_{i:A_i=a}{V}_i}. \end{gathered}$$

Then we may obtain the perturbed counterparts of $\widehat{g}(s)$, $\mathrm{P}\widehat{\mathrm{T}}{\mathrm{E}}_{1,\widehat{g}}$, $\mathrm{P}\widehat{\mathrm{T}}{\mathrm{E}}_{2,\widehat{g}}$, $\widehat{\mathrm{\Delta }}$ and ${\widehat{\mathrm{\Delta }}}_{\widehat{g}(S)}$ as

$${\widehat{g}}^{*}(s)={\widehat{m}}^{*}(s)+{\widehat{\lambda }}^{*}{\widehat{\mathcal{P}}}_0^{*}(s),\mathrm{P}\widehat{\mathrm{T}}{\mathrm{E}}_{1,{\widehat{g}}^{*}}^{*}={\widehat{\mathrm{\Delta }}}_{{\widehat{g}}^{*}(S)}^{*}/{\widehat{\mathrm{\Delta }}}^{*},\mathrm{P}\widehat{\mathrm{T}}{\mathrm{E}}_{2,{\widehat{g}}^{*}}^{*}={(1-\mathrm{M}\widehat{\mathrm{S}}{\mathrm{E}}_{{\widehat{g}}^{*}(S)}^{*}/{\mathrm{M}\widehat{\mathrm{S}}\mathrm{E}}_{\mathrm{null}}^{*})}^{1/2},$$

where ${\widehat{\mathrm{\Delta }}}^{*}={\widehat{\mu }}_1^{*}-{\widehat{\mu }}_0^{*},{\widehat{\mathrm{\Delta }}}_{{\widehat{g}}^{*}(S)}^{*}={\widehat{\mu }}_{1,{\widehat{g}}^{*}(S)}^{*}-{\widehat{\mu }}_{0,{\widehat{g}}^{*}(S)}^{*}$,

$$\mathrm{M}\widehat{\mathrm{S}}{\mathrm{E}}_{{\widehat{g}}^{*}(S)}^{*}=\frac{2{\sum }_{i=1}^n{\left\{Y_i-{\widehat{g}}^{*}\left(S_i\right)\right\}}^2{V}_i}{{\sum }_{i=1}^n{V}_i}\mathrm{and}\mathrm{M}\widehat{\mathrm{S}}{\mathrm{E}}_{\mathrm{null}}^{*}=\frac{{\sum }_{i=1}^{n}2{\left(Y_i-{\widehat{\mu }}_0^{*}\right)}^2{V}_i}{{\sum }_{i=1}^n{V}_i}.$$

In practice, we may generate a large number, say B, of realizations for V, and then obtain B realizations of ${\widehat{g}}^{*}(s)$, ${\widehat{\mathrm{\Delta }}}^{*}$, ${\widehat{\mathrm{\Delta }}}_{{\widehat{g}}^{*}(S)}^{*}$, $\mathrm{P}\widehat{\mathrm{T}}{\mathrm{E}}_{1,{\widehat{g}}^{*}}^{*}$ and $\mathrm{P}\widehat{\mathrm{T}}{\mathrm{E}}_{2,{\widehat{g}}^{*}}^{*}$. The variance estimation and the confidence interval can be constructed based on the empirical variances and quantiles of these realizations. We expect that resampling-based inference is particularly appealing for $\mathrm{P}\widehat{\mathrm{T}}{\mathrm{E}}_{2,\widehat{g}}$, which involves empirical mean square error estimates that tend to have a skewed distribution in finite samples. When the surrogate marker carries little information about the outcome, ${\mathrm{MSE}}_{null}\approx {\mathrm{MSE}}_{g_{opt}(S)}$ and $\mathrm{M}\widehat{\mathrm{S}}{\mathrm{E}}_{null}$ may be greater than $\mathrm{M}\widehat{\mathrm{S}}{\mathrm{E}}_{\widehat{g}(S)}$ in finite samples. In such a case, we simply let $\mathrm{P}\widehat{\mathrm{T}}{\mathrm{E}}_{2,\widehat{g}}=0$. When pte₂ ≈ 0, the aforementioned normal approximation for the proposed estimator may be poor, although the distribution of $\left\{\mathrm{M}\widehat{\mathrm{S}}{\mathrm{E}}_{\widehat{g}(S)},\mathrm{M}\widehat{\mathrm{S}}{\mathrm{E}}_{null}\right\}$ can still be approximated well by a multivariate normal distribution. In such a case, the resampling method can still provide a valid but potentially conservative confidence interval for pte₂ based on the empirical quantile of ${(1-\{{\mathrm{M}\widehat{\mathrm{S}}\mathrm{E}}_{{\widehat{g}}^{*}(S)}^{*}/\mathrm{M}\widehat{\mathrm{S}}{\mathrm{E}}_{\mathrm{null}}^{*}\}\wedge 1)}^{1/2}$.

4. Simulation studies

We first conducted simulation studies to evaluate the finite-sample performance of our methods along with several existing methods, including (i) Parast et al. (2016), denoted as pte_L; (ii) Wang & Taylor (2002), denoted as pte_W; and (iii) Freedman et al. (1992), denoted as pte_F. Across all configurations we let n = 500, 1000, 250 and 500 in each arm respectively, and chose K(·) as a Gaussian kernel with bandwidth $h=h_{\mathrm{opt}}{n}^{-c_0}$, c₀ = 0.11, where h_opt is found using the method of Scott (1992).Variances were estimated using the proposed resampling method with B = 1000. All results were summarized based on 500 simulated datasets for each configuration.

We consider four data generation scenarios. For settings k = 1, 2, 3, 4, we generate

$$\begin{gathered} S^{(0)}\sim \mathrm{Ga}(\mathrm{shape}=a_k^{(0)},\mathrm{scale}=b_k^{(0)}),S^{(1)}\sim \mathrm{Ga}(\mathrm{shape}=a_k^{(1)},\mathrm{scale}=b_k^{(1)}), \\ {Y}^{(0)}=I[E^{(0)}/\{0.2{\mathcal{G}}_k^{(0)}\left(S^{(0)}\right)\}>t],Y^{(1)}=I[E^{(1)}/\{0.2+0.22{\mathcal{G}}_k^{(1)}\left(S^{(1)}\right)\}>t], \end{gathered}$$

where E⁽⁰⁾ and E⁽¹⁾ follow the unit exponential distribution, and we let $a_1^{(0)}=b_1^{(0)}=2$, $a_1^{(1)}=9$, $b_1^{(1)}=0.5$, ${\mathcal{G}}_1^{(0)}(s)={\mathcal{G}}_1^{(1)}(s)=s$, $a_2^{(0)}=7.5$, $b_2^{(0)}=1$, $a_2^{(1)}=b_2^{(1)}=2$, ${\mathcal{G}}_2^{(0)}(s)=s/2$, ${\mathcal{G}}_2^{(1)}(s)=9/11+s$, $a_3^{(0)}=b_3^{(0)}=2$, $a_3^{(1)}=9$, $b_3^{(1)}=0.5$, ${\mathcal{G}}_3^{(0)}(s)={\mathcal{G}}_3^{(1)}(s)=s-\log (s)$; $a_4^{(0)}=b_4^{(0)}=2$, $a_4^{(1)}=9$, $b_4^{(1)}=0.5$, ${\mathcal{G}}_4^{(0)}(s)={\mathcal{G}}_4^{(1)}(s)=s-2\log (s)$. All assumptions required by Parast et al. (2016) are satisfied under setting 1, but S⁽⁰⁾ and S(1) have rather different supports under setting 2 and the effect of S on Y is nonmonotone under settings 3 and 4. Assumption (1) holds in all four settings.

Figure 1 shows the empirical biases, empirical standard errors, the average of the estimated standard errors and the empirical coverage probabilities of the 95% pointwise confidence intervals for g_opt(·) when n = 1000. The estimation results of pte₁ and pte₂ when n = 1000 are shown in Table 1. The results for the estimation of g_opt(·), pte₁ and pte₂ when n = 500 have similar patterns, as shown in the Supplementary Material. Across all settings, the point estimates for g_opt(·), pte₁ and pte₂ present negligible biases, and the estimated standard errors are close to the empirical standard errors. The coverage probabilities of the confidence intervals are also close to their nominal level.

Fig. 1.

The empirical bias, empirical standard error (solid) versus the average of the estimated standard error (dashed), coverage probabilities (dashed) of the 95% (solid) confidence intervals for $\widehat{g}(s)$ when n = 1000.

Table 1.

Estimates of pte₁, pte₂, pte_L, pte_W and pte_F along with their empirical standard errors under settings 1, 2, 3 and 4 with n = 1000. For our proposed proportion of treatment effect explained estimates, we also present the average of the estimate standard errors (shown in subscript) along with the empirical coverage probabilities of the 95% confidence intervals

n = 1000
		Proposed				pteL		pteW		pteF
		True	Est	ESEASE	CP	Est	ESE	Est	ESE	Est	ESE
1	pte1	0.614	0.616	0.0830.078	0.938	0.470	0.136	0.198	0.067	0.193	0.064
	pte2	0.418	0.423	0.0270.026	0.942
2	pte1	0.442	0.439	0.0340.034	0.942	−0.265	0.056	−0.365	0.066	−0.218	0.040
	pte2	0.383	0.394	0.0270.028	0.934
3	pte1	0.511	0.503	0.0800.081	0.954	0.281	0.135	0.192	0.065	0.194	0.065
	pte2	0.362	0.367	0.0280.026	0.934
4	pte1	0.318	0.316	0.0880.084	0.936	−0.033	0.142	0.184	0.068	0.194	0.071
	pte2	0.322	0.331	0.0270.026	0.930

Est, estimates; ESE, empirical standard errors; ASE, average of the estimate standard errors; CP, coverage probability.

Table 1 also summarizes the results of other proportion of treatment effect explained estimators. Across all settings, the Wang & Taylor (2002) and Freedman et al. (1992) methods misspecify the underlying model. As a result, pte_W and pte_F estimates differ substantially from the nonparametric estimates from pte₁, pte₂ and pte_L. In setting 2, pte_W, pte_F and pte_L all fail with their estimates being negative. This is because the assumptions in these papers are not satisfied. For example, the supports of the treatment and control groups are different. However, the proposed proportion of treatment effect explained definitions and corresponding estimates do not have such a problem here. For setting 3, where we have introduced a mild deviation from the monotone increasing assumption, similar conclusions to setting 1 can be drawn. For setting 4, we observe that, except for our proposed proportion of treatment effect explained estimates, all existing methods yield proportion of treatment effect explained estimates close to zero. This is due to the fact that E(Y | S = s) is quite nonmonotone in this case, and our proposed estimates evaluate the proportion of treatment effect explained for g_opt(S) rather than S. These results highlight the robustness of our proposed method and the corresponding nonparametric estimation procedure.

We performed further sensitivity analyses for the proposed procedures when the working assumption (1) fails to hold. We consider two general settings: (I) Y⁽¹⁾ ⊥ Y⁽⁰⁾ | (S⁽¹⁾, S⁽⁰⁾), but S⁽¹⁾ and S⁽⁰⁾ are correlated; (II) (S⁽⁰⁾, S⁽¹⁾, Y⁽⁰⁾, Y⁽¹⁾) have varying degrees of correlation and Y⁽¹⁾ ⊥ Y⁽⁰⁾ | (S⁽¹⁾, S⁽⁰⁾) may not hold. Under setting I with the conditional independence structure it is still feasible to derive $g_{\mathrm{oracle}}={\mathrm{argmin}}_g{\mathcal{L}}_{\mathrm{oracle}}(g)$, although g_oracle has a complex form and is the solution to an integral equation, as shown in the Supplementary Material. In setting I two cases are considered, corresponding to a unimodal g_oracle and a monotone g_oracle, respectively. Specifically, in the first case, Ia, we generated

$$\left[\begin{array}{c}{S}^{(1)}\\ S^{(0)}\end{array}\right]\sim N\left(\left[\begin{array}{c}0\\ 3\end{array}\right],\left[\begin{array}{cc}1& {\wp }_s\\ {\wp }_s& 1\end{array}\right]\right)\mathrm{with}{\wp }_s\mathrm{chosen}\mathrm{at}0,0.2\mathrm{and}\mathrm{0.5.}$$

We then generated Y⁽¹⁾ and Y⁽⁰⁾ from

$$P\left(Y^{(1)}|S^{(0)},S^{(1)}\right)=\exp \{-1-S^{{(1)}^2}\}\mathrm{and}P\left(Y^{(0)}|S^{(0)},S^{(1)}\right)=\exp \{-2-S^{{(0)}^2}\},$$

resulting in a unimodal g_oracle. In setting Ib,

$$\left[\begin{array}{c}{S}^{(1)}\\ S^{(0)}\end{array}\right]\sim N\left(\left[\begin{array}{c}5\\ 10\end{array}\right],\left[\begin{array}{cc}1& {\wp }_s\\ {\wp }_s& 1\end{array}\right]\right)\mathrm{with}{\wp }_s=0,0.2,0.5,$$

$$P\left(Y^{(1)}|S^{(0)},S^{(1)}\right)=\exp \{-1-0.1S^{{(1)}^2}\}\mathrm{and}P\left(Y^{(0)}|S^{(0)},S^{(1)}\right)=\exp \{-2-0.1S^{{(0)}^2}\},$$

resulting in a monotone g_oracle. In setting II, we generated

$$\left[\begin{array}{l}{S}^{(1)}\hfill \\ S^{(0)}\hfill \\ Y^{(1)}\hfill \\ Y^{(0)}\hfill \end{array}\right]\sim N\left(\left[\begin{array}{c}1\\ 3\\ 6\\ 9\end{array}\right],\left[\begin{array}{cccc}1& {\wp }_s& 0.9& 0.1\\ {\wp }_s& 1& 0.1& 0.9\\ 0.9& 0.1& 1& {\wp }_y\\ 0.1& 0.9& {\wp }_y& 1\end{array}\right]\right)\mathrm{with}{\wp }_s,{\wp }_y\in \{0,0.2,0.5\},$$

resulting in nine different correlation combinations. Under setting II with the more general correlation structure, g_oracle is no longer tractable and hence we only examine the validity of the proposed nonparametric estimation procedures under the violation of (1).

For setting I, we compare the true g_oracle to our proposed g_opt and the proportion of treatment effect explained obtained under the two transformations. As shown in Fig. 2, g_oracle and g_opt mostly coincide with each other except for the extreme tail parts for setting Ia when g_oracle is unimodal. For setting Ib, where g_oracle is monotone, the two functions are nearly identical to each other. Thus, g_opt remains a good approximation to g_oracle even when the independence assumption is violated. In Table 2 we present the proportion of treatment effect explained estimates obtained using g_oracle and using g_opt. The two sets of estimates are close to each other, suggesting that the proportion of treatment effect explained estimates are not very sensitive to these departures from the working independence assumption.

Fig. 2.

The solid line denotes the proposed optimal g_opt(s) and the dotted line denotes the true optimal g_oracle(s) by solving the integration function considering ${\wp }_s$ from setting Ia, panels (a), (b) and (c), where g_oracle is unimodal, and Ib, panels (d), (e) and (f), where g_oracle is monotone.

Table 2.

Estimates of pte₁, pte₂ derived using g_oracle versus the proposed g_opt from settings Ia and Ib when Y⁽¹⁾ ⊥ Y⁽⁰⁾ | (S⁽¹⁾, S⁽⁰⁾) and S⁽¹⁾ and S⁽⁰⁾ are multivariate normal with correlation ${\wp }_s$

		Ia		Ib
${\mathit{\wp }}_{\mathit{s}}$		goracle(·)	gopt(·)	goracle(·)	gopt(·)
0	pte1	0.935	0.935	0.995	0.995
	pte2	0.682	0.683	0.504	0.504
0.2	pte1	0.926	0.929	0.995	0.994
	pte2	0.677	0.679	0.504	0.504
0.5	pte1	0.939	0.949	0.992	0.992
	pte2	0.670	0.670	0.503	0.503

Simulation results for setting II are summarized in Table 3, where we compare the point estimates of pte₁ and pte₂ to their corresponding limiting values based on g_opt and examine the validity of the standard error estimates. Across the nine combinations of ${\wp }_s$ and ${\wp }_y$, $\mathrm{P}\widehat{\mathrm{T}}{\mathrm{E}}_1$ and $\mathrm{P}\widehat{\mathrm{T}}{\mathrm{E}}_2$ have negligible bias and the average estimated standard errors are close to the empirical standard errors. These results confirm that the proposed inference procedure is valid regardless of whether the working independence assumption holds.

Table 3.

Estimates of pte₁ and pte₂ compared with the population values along with their empirical standard errors under nine variations of setting II with varying values of ${\wp }_1={\wp }_s$ and ${\wp }_2={\wp }_y$ for n = 500. Also shown are the average of the estimated standard errors along with the empirical coverage probabilities of the 95% confidence intervals

			Truth	Est	ESE	ASE	CP
${\wp }_1=0$	${\wp }_2=0$	pte1	0.887	0.885	0.015	0.015	0.942
		pte2	0.964	0.965	0.004	0.004	0.916
	${\wp }_2=0.2$	pte1	0.887	0.885	0.015	0.015	0.944
		pte2	0.964	0.965	0.004	0.004	0.904
	${\wp }_2=0.5$	pte1	0.882	0.879	0.016	0.015	0.946
		pte2	0.957	0.958	0.005	0.004	0.900
${\wp }_1=0.2$	${\wp }_2=0$	pte1	0.887	0.885	0.015	0.015	0.946
		pte2	0.964	0.965	0.004	0.004	0.916
	${\wp }_2=0.2$	pte1	0.887	0.884	0.016	0.015	0.936
		pte2	0.964	0.965	0.004	0.004	0.914
	${\wp }_2=0.5$	pte1	0.877	0.874	0.017	0.016	0.934
		pte2	0.951	0.953	0.005	0.005	0.902
${\wp }_1=0.5$	${\wp }_2=0$	pte1	0.882	0.879	0.017	0.015	0.924
		pte2	0.958	0.958	0.005	0.004	0.916
	${\wp }_2=0.2$	pte1	0.877	0.874	0.017	0.016	0.920
		pte2	0.952	0.952	0.005	0.005	0.914
	${\wp }_2=0.5$	pte1	0.868	0.866	0.018	0.017	0.930
		pte2	0.941	0.942	0.006	0.006	0.906

Est, estimates; ESE, empirical standard errors; ASE, average of the estimate standard errors; CP, coverage probability.

5. Application

5.1. Setting

We applied the proposed procedure to evaluate the surrogacy of CD4 counts in predicting the treatment effect on plasma HIV-1 RNA concentrations using the AIDS Clinical Trials Group, ACTG, 320 Study (Hammer et al., 1997), as the suppression of plasma RNA has been accepted and is widely used as a surrogate for progression to AIDS/death in the literature. The ACTG 320 study was a randomized, double-blinded, placebo-controlled trial that compared the three-drug regimen of indinavir, zidovudine or stavudine, and lamivudine with the two-drug regimen of zidovudine or stavudine, and lamivudine in HIV-infected patients with at least three months of prior zidovudine therapy. A total of 1156 patients were randomly assigned to one of the two regimens. Outcomes of interest for this study included time to a new AIDS-defining event, changes in CD4 counts and RNA concentrations. While HIV-1 viral quantification is essential for treatment monitoring, measuring RNA concentration is relatively expensive, especially for resource-limited countries (Calmy et al., 2007). We investigate whether CD4 counts can effectively serve as a surrogate marker for RNA outcomes. Specifically, we aim to evaluate the proportion of the treatment effect on RNA viral load explained by the treatment effect on S = CD4₂₄₋₀, defined as the change in CD4 counts from baseline to week 24. The ranges of S in the two treatment arms are [−100, 733.5] in the combination therapy and [−136.5, 277] in the triple therapy arm. We considered two RNA outcomes: the reduction in log₁₀RNA from baseline to week 24, denoted by $Y_{{\mathrm{RNA}}_{24-0}}$, and the binary outcome of attaining RNA below 500 at week 24, denoted by $Y_{{\mathrm{RNA}}_{24}⩽500}$. The analysis focused on 830 patients, 418 in triple therapy and 412 in combination therapy, who had complete information on CD4 and RNA measurements.

5.2. Optimal function of S and proportion of treatment effect explained estimates

We first applied the proposed methods to examine g_opt(·) of the surrogate CD4₂₄₋₀ for predicting the treatment response as quantified by the RNA viral load. The estimated g_opt(·) for $Y_{{\mathrm{RNA}}_{24-0}}$ and $Y_{{\mathrm{RNA}}_{24}⩽500}$ along with their pointwise confidence intervals are shown in Figs. 3(a) and (b). For both outcomes, the transformation function appears to be slightly nonlinear with a slightly larger magnitude of slope for smaller values of CD4₂₄₋₀. The treatment effect on the RNA outcomes and on the surrogate outcomes are all highly significant. For $Y_{{\mathrm{RNA}}_{24-0}}$, the treatment effect is estimated as $\widehat{\mathrm{\Delta }}=1.595$, SE = 0.083, while the treatment effect on the predicted outcome based on g_opt(S) is estimated as ${\widehat{\mathrm{\Delta }}}_{\widehat{g}(S)}=0.906$, SE = 0.076. This leads to pte₁ estimated as 56.8% with 95% confidence interval [49.9%, 63.6%], and pte₂ estimated as 65.6% with 95% confidence interval [61.3%, 69.9%]. The estimated pte₁ is higher than the estimated pte_L of Parast et al. (2016), which was 41.5% with 95% confidence interval [32.3%, 50.7%]. This could be due in part to the slightly non-overlapping distribution of S within the two treatment groups, as shown in the Supplementary Material, which is a required assumption for pte_L to be valid. We observe similar patterns for the binary outcome $Y_{{\mathrm{RNA}}_{24}⩽500}$. The estimated pte₁ and pte₂ were 50.5% with 95% confidence interval [43.2%, 57.8%] and 56.0% with 95% confidence interval [50.9%, 61.1%], respectively. The pte₁ estimate is again substantially higher than the pte_L estimate of 31.3% with 95% confidence interval [21.7%, 40.9%].

Fig. 3.

Point estimates for the transformation function g_opt(·) (solid lines) along with their pointwise 95% confidence intervals (dashed lines) for (a) $Y_{{\mathrm{RNA}}_{24-0}}$, the decrease in log₁₀RNA from baseline to week 24, and (b) $Y_{{\mathrm{RNA}}_{24}⩽500}$, the binary outcome of attaining RNA below 500 at week 24.

5.3. Transportability investigation

The overall goal of this work, and surrogate marker research in general, is to identify an S, or function of S as in this paper, that can be used to replace the primary outcome to test for a treatment effect. To actually achieve this goal, certain assumptions would be necessary to ensure that S or the function of S is appropriate for a future study, which we will refer to as transportability. Transportability, the assumptions required for transportability, and how to assess whether those assumptions hold are interesting problems and warrant further work. Here, the interesting structure of the ACTG 320 trial recruitment allows us to empirically explore, just within this application, this concept of transportability.

Since the recruitment for this study was stratified on CD4₀, we partition the trial into two sub-studies, denoted by $\mathcal{H}$ and $\mathcal{L}$, with different CD4₀ distributions, to investigate the transportability of the CD4 surrogacy and treatment effect across different study populations. We may treat $\mathcal{H}$ as a current study and $\mathcal{L}$ as a separate future study, and investigate the transportability of the proportion of treatment effect explained between these two studies. We consider three different partitioning mechanisms: (i) a completely random partition; (ii) randomly assigning patients with CD4₀ < 200 into group $\mathcal{L}$ with probability expit(0.5 − 0.1CD4₀/s_max), and $\mathcal{H}$ otherwise, but those with CD4₀ 200 always remaining in group $\mathcal{H}$; (iii) randomly assigning patients with CD4₀ < 100 into group $\mathcal{L}$ with probability expit(0.5 − 0.2CD4₀/s_max), and $\mathcal{H}$ otherwise, but those with CD4₀ 100 always remaining in group $\mathcal{H}$, where s_max is the observed maximum value of CD4₀. We repeated the partitioning process 40 times for each mechanism and averaged all estimates over the 40 partitions. The average number of patients in group $\mathcal{H}$ was approximately 415, 454 and 586 for settings i, ii and iii, respectively. The patient populations of $\mathcal{H}$ and $\mathcal{L}$ were similar in setting i, with median CD4₀ of approximately 72.4 in $\mathcal{H}$ and 70.4 in $\mathcal{L}$. In setting ii, the median CD4₀ was 81.4 in $\mathcal{H}$ and 59.8 in $\mathcal{L}$. The difference between the two studies is more pronounced in setting iii, with the median CD4₀ being 109.4 in $\mathcal{H}$ and 30.8 in $\mathcal{L}$. For each dataset, we first obtained estimates of $g_{\mathrm{opt}}(\cdot ),\mathrm{\Delta },{\mathrm{\Delta }}_{g_{opt}(S)}$ and the proportion of treatment effect explained within $\mathcal{H}$ and within $\mathcal{L}$ separately. To examine the cross-study transportability, we also assessed the treatment effects on ${\widehat{g}}_{\mathrm{opt}}^{\mathcal{H}}(S)$ and ${\widehat{g}}_{\mathrm{opt}}^{\mathcal{L}}(S)$ in group $\mathcal{L}$, where ${\widehat{g}}_{\mathrm{opt}}^{\mathcal{H}}$ and ${\widehat{g}}_{\mathrm{opt}}^{\mathcal{L}}$ are the estimated g_opt(·) using data in $\mathcal{H}$ and in $\mathcal{L}$, respectively.

We report the average treatment effects and proportion of treatment effect explained estimates in Table 2 of the Supplementary Material. In setting I, the two groups are drawn from the same patient population. As expected, the estimates of Δ and ${\mathrm{\Delta }}_{g_{\mathrm{opt}}(S)}$ as well as the proportion of treatment effect explained estimates from study $\mathcal{H}$ are comparable to those from study $\mathcal{L}$. In addition, the predicted treatment effect based on ${\widehat{g}}^{\mathcal{H}}(S)$ and based on ${\widehat{g}}^{\mathcal{L}}(S)$ for those in $\mathcal{L}$ are also close to each other, indicating that if $\mathcal{H}$ were an earlier study, one could use data in $\mathcal{H}$ to estimate g_opt as ${\widehat{g}}^{\mathcal{H}}$ and predict treatment response based on ${\widehat{g}}^{\mathcal{H}}(S)$ in $\mathcal{L}$. In setting II, the two groups have slightly different populations, with group $\mathcal{L}$ representing patients with lower baseline CD4 counts. We see that the three pte₁ estimates are close, potentially indicating that it would still be appropriate to use the estimate of ${\mathrm{\Delta }}_{g_{\mathrm{opt}}(S)}$ derived using ${\widehat{g}}^{\mathcal{H}}(\cdot )$ to make inference about Δ in study $\mathcal{L}$. In setting III, the baseline CD4 counts in study $\mathcal{L}$ are substantially lower than in study $\mathcal{H}$. The three pte₁ estimates differ more than the previous settings, but not substantially. Thus, making inference about Δ in study $\mathcal{L}$ based on ${\widehat{g}}^{\mathcal{H}}(\cdot )$, though not ideal, may be relatively reasonable. These results demonstrate that the transportability of the proposed method may work well for studies with moderately different distributions of baseline CD4, but would require caution when the distributions are quite different.

6. Discussion

Throughout, we assumed that p₁ = 0.5 both in the population loss function and in the observed data. In practice, even if the observed trial has a randomization ratio different from 1:1, the proposed loss function is still an appropriate choice as it reflects a future population with p₁ = 0.5. In that case, the population g_opt(·) remains the same and our proposed estimators can be easily modified to include inverse probability of treatment assignment weights to yield consistent estimators of g_opt(·), pte₁ and pte₂. More generally, if there is a preconceived p₁ ╪ 0.5, one may modify both the objective function and the estimators with weights to allow for different treatment assignment probabilities.

Our proposed plug-in estimators for the proportion of treatment effect explained use the same data to estimate both g_opt and the proportion of treatment effect explained given g, and hence may suffer from overfitting bias. In our numerical studies, the bias appears negligible compared to the standard error. When the sample size is small, it may be necessary to correct for the overfitting bias, which can be achieved via cross validation by estimating g_opt and the proportion of treatment effect explained given g using separate data. In the HIV example, we performed sensitivity analyses considering the cross-study transportability issue, where g_opt may be estimated from a previous study and used to derive the treatment effect on the transformed surrogate outcome in a subsequent study. In our opinion, transportability is unavoidable in studying surrogate markers. We choose to assume the transportability of g_opt and the proportion of treatment effect explained instead of, for example, the complete joint distribution of outcome and surrogate marker to enhance the robustness of the approach. While these assumptions may still be violated in practice, the results based on the aforementioned simulations and numerical experiments are promising.

In principle, the proposed g_opt can still be used even if the treatment effect is measured by a relative risk contrast. We have E(Y⁽⁰⁾) = E{g_opt(S⁽⁰⁾)}, and

$$\begin{gathered} E\left(Y^{(1)}\right)/E\left(Y^{(0)}\right)=\left\{E\left(Y^{(1)}\right)-E\left(Y^{(0)}\right)\right\}/E\left(Y^{(0)}\right)+1 \\ ⩾\left[E\left\{g_{\mathrm{opt}}\left(S^{(1)}\right)\right\}-E\left\{g_{\mathrm{opt}}\left(S^{(0)}\right)\right\}\right]/E\left\{g_{\mathrm{opt}}\left(S^{(0)}\right)\right\}+1 \\ =E\left\{g_{\mathrm{opt}}\left(S^{(1)}\right)\right\}/E\left\{g_{\mathrm{opt}}\left(S^{(0)}\right)\right\}, \end{gathered}$$

which guarantees the validity of the surrogate marker after transformation, i.e., $\mathrm{\Delta }=0\Rightarrow {\mathrm{\Delta }}_{g_{\mathrm{opt}}(S)}=0$. In general, if the contrast g(μ₁, μ₀) used to measure the treatment effect is monotone increasing in μ₁ for any fixed μ₀, we can guarantee that g{E(Y⁽¹⁾), E(Y⁽⁰⁾)} >= g[E{g_opt(S⁽¹⁾)}, E{g_opt(S⁽⁰⁾)}]. The pte₁ can also be generalized as

$$\frac{g\left[E\left\{g_{\mathrm{opt}}\left(S^{(1)}\right)\right\},E\left\{g_{\mathrm{opt}}\left(S^{(0)}\right)\right\}\right]}{g\left\{E\left(Y^{(1)}\right),E\left(Y^{(0)}\right)\right\}}\in [0,1].$$

However, if the treatment effect is not measured by a contrast between μ₁ and μ₀, such as hazard ratio, the current proposal is not directly applicable.

The proposed proportion of treatment effect explained measures and the associated inference procedures are robust against the violation of the working independence assumption (1), which is mainly used to derive the specific form of g_opt(·). We employ this independence assumption because the correlation structure of the counterfactuals is not identifiable, and minimizing ${\mathcal{L}}_{\mathrm{oracle}}(g)$ is often not analytically or numerically tractable, even if the correlation structure is given. As demonstrated in the Supplementary Material, even under a simple multivariate normal setting, g_oracle involves solving complex integral equations and depends on the level of correlation, which is not identifiable. However, the correspondence between pte₁ and pte_L proposed in Parast et al. (2016) does not at all require (1) to hold. As confirmed in the simulation studies, even if (1) fails, the estimation procedures based on $\widehat{g}(s)$, $\mathrm{P}\widehat{\mathrm{T}}{\mathrm{E}}_1$ and $\mathrm{P}\widehat{\mathrm{T}}{\mathrm{E}}_2$ are valid for making inference about the population values of g_opt(s), pte₁ and pte₂ with g_opt(s) defined as (2). In addition, the simulation results suggest that our proposed g_opt(·) and proportion of treatment effect explained estimates are not very sensitive to the violation of this assumption. Lastly, even if this working independence assumption is severely violated, g_opt(·) can still be viewed as an optimal transformation for the surrogate marker to recover the difference in the primary outcome between two independent patients assigned to treatment and control arms, respectively.