Surrogacy Assessment Using Principal Stratification With Multivariate Normal and Gaussian Copula Models

Abstract

Background

The validation of intermediate markers as surrogate markers (S) for the true outcome of interest (T) in clinical trials offers the possibility for trials to be run more quickly and cheaply by using the surrogate endpoint in place of the true endpoint.

Purpose

Working within a principal stratification framework, we propose causal quantities to evaluate surrogacy using a Gaussian copula model for an ordinal surrogate and time-to-event final outcome. The methods are applied to data from four colorectal cancer clinical trials where S is tumor response and T is overall survival.

Methods

For the Gaussian copula model, a Bayesian estimation strategy is used and, as some parameters are not identifiable from the data, we explore the use of informative priors that are consistent with reasonable assumptions in the surrogate marker setting to aid in estimation.

Results

While there is some bias in the estimation of the surrogacy quantities of interest, the estimation procedure does reasonably well at distinguishing between poor and good surrogate markers.

Limitations

Some of the parameters of the proposed model are not identifiable from the data, and therefore assumptions must be made in order to aid in their estimation.

Conclusions

The proposed quantities can be used in combination to provide evidence about the validity of S as a surrogate marker for T.

1 Introduction

There is much interest in the use of intermediate outcome variables as surrogate endpoints (S) for the true outcome of interest (T) in randomized clinical trials, as they may allow trials to be run more quickly and inexpensively by basing the evaluation of the treatment effect on the earlier measured surrogate. Many previous methods of surrogate validation rely on models for the conditional distribution of T given the treatment (Z) and S. However, S is a post-randomization variable, and unobserved, simultaneous predictors of S and T may exist. When such confounders exist, these methods will not have a causal interpretation. Therefore, there has been much recent work in the area of surrogacy assessment under the principal surrogacy [1] approach, which looks at the distribution of the potential outcomes of T conditional on principal strata based on the potential outcomes of S, which are pre-randomization variables [2–4]. Treatment effect estimates that condition on these principal strata are therefore causal estimates. In this paper, using the principal surrogacy framework we describe surrogate validation measures for multivariate normal surrogate and outcome data, with extensions to non-normally distributed data through the use of a Gaussian copula model.

We consider the entire joint distribution of the potential outcomes of S and the potential outcomes of T, denoted by S(Z) and T(Z) for Z = 0; 1. We first consider the scenario where both S(Z) and T(Z) are continuous and their joint distribution is multivariate normal [5]. We then relax the multivariate normality assumption and consider the scenario of an ordinal categorical surrogate marker and a censored time-to-event true outcome [6]. Once parameter estimates from the joint distribution of (S(0), S(1), T(0), T(1)) are obtained, we examine various causal quantities that may aid in the assessment of S as a surrogate marker for T. As only two of the four potential outcomes are observed for each subject ((S(0), T(0)) for those in the control arm and (S(1), T(1)) for those in the treatment arm), the models are not fully identifiable from the data. We therefore use a Bayesian estimation strategy and propose some prior distributions that are consistent with reasonable assumptions in the surrogacy setting to aid in estimation. We apply the method to data from an advanced colorectal cancer clinical trial where tumor response is assessed as a surrogate for overall survival.

2 Measures of Surrogacy

In the setting of multivariate normal surrogate and outcome variables, we previously developed a model [5] for the joint distribution of the potential outcomes of S and T, given by:

$$\left(\begin{array}{c}S(0)\\ S(1)\\ T(0)\\ T(1)\end{array}\right)~N\left(\left(\begin{array}{c}{\mu }_{S_0}\\ {\mu }_{S_1}\\ {\mu }_{T_0}\\ {\mu }_{T_1}\end{array}\right),\left(\begin{array}{cccc}{\sigma }_{S_0}^2& {\rho }_s{\sigma }_{S_0}{\sigma }_{S_1}& {\rho }_{00}{\sigma }_{S_0}{\sigma }_{T_0}& {\rho }_{01}{\sigma }_{S_0}{\sigma }_{T_1}\\ & {\sigma }_{S_1}^2& {\rho }_{10}{\sigma }_{S_1}{\sigma }_{T_0}& {\rho }_{11}{\sigma }_{S_1}{\sigma }_{T_1}\\ & & {\sigma }_{T_0}^2& {\rho }_t{\sigma }_{T_1}{\sigma }_{T_0}\\ & & & {\sigma }_{T_1}^2\end{array}\right)\right).$$

We focus on the conditional distribution of [T(0); T(1) | S(0); S(1)] and derive quantities from this distribution to determine surrogacy. Specifically, we can consider E(T(1) T(0)|S(1) S(0) = s), which is linear in s and equal to γ₀ + γ₁s, where

$$\begin{array}{c}{\gamma }_0=({\mu }_{T_1}-{\mu }_{T_0})-\left(\frac{{\rho }_{11}{\sigma }_{S_1}{\sigma }_{T_1}-{\rho }_{10}{\sigma }_{S_1}{\sigma }_{T_0}-{\rho }_{01}{\sigma }_{S_0}{\sigma }_{T_1}+{\rho }_{00}{\sigma }_{S_0}{\sigma }_{T_0}}{{\sigma }_{S_0}^2+{\sigma }_{S_1}^2-2{\rho }_s{\sigma }_{S_0}{\sigma }_{S_1}}\right)({\mu }_{S_1}-{\mu }_{S_0})\\ {\gamma }_1=\left(\frac{{\rho }_{11}{\sigma }_{S_1}{\sigma }_{T_1}-{\rho }_{10}{\sigma }_{S_1}{\sigma }_{T_0}-{\rho }_{01}{\sigma }_{S_0}{\sigma }_{T_1}+{\rho }_{00}{\sigma }_{S_0}{\sigma }_{T_0}}{{\sigma }_{S_0}^2+{\sigma }_{S_1}^2-2{\rho }_s{\sigma }_{S_0}{\sigma }_{S_1}}\right).\end{array}$$

The value of γ₀ is then is a measure of the “associative effect” [1], which should be near zero for a valid surrogate, indicating that the average treatment effect on T is near zero when there is no treatment effect on S, and γ₁ is a measure of the “dissociative effect” which should be non-zero for a valid surrogate, indicating that there is an expected treatment effect on T when there is a treatment effect on S. We can also consider the entire curve of E(T(1) T(0)|S(1) S(0) = s) [3], which provides estimates of the expected treatment effect within principal strata of S that may be clinically interesting. For a good surrogate the curve should pass through the origin, be monotonic and be much different from zero at large values of |s|.

In the setting of an ordinal categorical S, and a time-to-event T, a Gaussian copula model is used to model the joint distribution of the two potential outcomes for each of S and T [6]. We assume that S(0) and S(1) arise from latent underlying standard normal Gaussian random variables, denoted by S̃ (0) and S̃ (1), respectively, such that:

$$S(Z)=\{\begin{array}{ll}{1}_Z,\hfill & \text{if}\stackrel{\sim }{S}(Z)\in (-\infty ,{\alpha }_{1_Z})\hfill \\ 2_Z,\hfill & \text{if}\stackrel{\sim }{S}(Z)\in ({\alpha }_{1_Z},{\alpha }_{2_Z})\hfill \\ ⋮\hfill & \hfill \\ C_Z,\hfill & \text{if}\stackrel{\sim }{S}(Z)\in ({\alpha }_{{(C-1)}_Z},\infty )\hfill \end{array}$$

where α_1Z < α_2Z < … < α_(C−1)Z are unknown cut points with α_0Z = −∞, α_CZ = ∞ and let Φ⁻¹ {P(S(Z) ≤ k_Z)} = α_KZ. We assume a Weibull distribution for T, with hazard function $\lambda (T(Z))=\left(\frac{{\gamma }_{T_Z}}{{\lambda }_{T_Z}}\right){\left(\frac{T(Z)}{{\lambda }_{T_Z}}\right)}^{{\gamma }_{T_Z}-1}$. We denote the marginal cumulative distributions of S̃(0), S̃(1), T(0) and T(1) by F_S̃(0), F_S̃(1), F_T(0) and F_T(1), respectively, and assume that their joint cumulative distribution is generated by the Gaussian copula function: $F_{\stackrel{\sim }{\mathbf{y}}}(\stackrel{\sim }{\mathbf{y}})={\mathrm{\Phi }}_4\{{\mathrm{\Phi }}_1^{-1}(F_{\stackrel{\sim }{S}(0)}(\stackrel{\sim }{S}(0))),{\mathrm{\Phi }}_1^{-1}(F_{\stackrel{\sim }{S}(1)}(\stackrel{\sim }{S}(1))),{\mathrm{\Phi }}_1^{-1}(F_{T(0)}(T(0))),{\mathrm{\Phi }}_1^{-1}(F_{T(1)}(T(1)))\mid \mathrm{\Gamma }\}$ where ỹ = (S̃(0), S̃(1), T(0) and T(1)) and Φ_P is the standard P-variate normal distribution, with correlation matrix $\mathrm{\Gamma }=\left(\begin{array}{cccc}1& {\rho }_s& {\rho }_{00}& {\rho }_{01}\\ & 1& {\rho }_{10}& {\rho }_{11}\\ & & 1& {\rho }_t\\ & & & 1\end{array}\right)$.

In this setting, we can again consider measures of the associative and dissociative effect which can be measured by E(log(T(1)=T(0))|S(1) S(0) = s). This conditional expectation provides an estimate of the dissociative effect when s = 0 and a measure of the associative effect at each value of s when s ≠ 0. We can also consider the entire curve of E(log(T(1)=T(0))|S(1) S(0) = s) versus s.

3 Estimation

In both the multivariate normal model and the Gaussian copula model there are four correlation parameters that are unidentifiable from the observed data (ρ_s, ρ_t, ρ₁₀, ρ₀₁,). Therefore, a Bayesian approach was developed [5,6] with reasonable priors in the surrogate evaluation setting used to aid in estimation. Unobserved potential outcomes are treated as missing data and imputed from the appropriate posterior distribution at each iteration of the Markov chain. Non-informative priors are placed on the parameters that define the mean and variance in the multivariate normal model, and on λ_TZ, γ_TZ and the α_KZ's in the Gaussian copula model, all of which are fully identified parameters. Marginal priors are placed on each of the correlation parameters of the covariance matrix and the use of different prior assumptions are explored. For each of these there is the additional assumption that the correlation matrix must be positive definite. The priors considered for the correlation parameters include:

1. Jointly uniform prior such that for each of the six correlations p(ρ) ~ U ni f (−1, 1)

2. Jointly uniform prior such that for each of the six correlations p(ρ) ~ U ni f (0, 1)

3. Uniform over the region where all ρ’s ≥ 0, ρ₀₁ < min (ρ₀₀, ρ₁₁, ρ_s, ρ_t) and ρ₁₀ < min (ρ₀₀, ρ₁₁, ρ_s, ρ_t)

4. Beta priors such that:

   • p(ρ₁₁) ~ U ni f (0, 1)

   • p(ρ₀₀) ~ U ni f (0, 1)

   • p(ρ₁₀) and p(ρ₀₁) ~ Beta(3α₀, 3 − 3α₀) such that P(ρ₀₁ ρ₁₀ ≤ min(ρ̂₀₀, ρ̂₁₁)) = 0.80

   • p(ρ_s) and p(ρ_t) ~ Beta(3α₁, 3 − 3α₁) such that P(ρ_s ρ_t ≥ E[ρ₁₀]) = 0.80

where ρ̂₀₀, and ρ̂₁₁, are the Pearson correlation coefficients when S and T are normally distributed, and the polyserial correlation coefficients when S is ordinal and T is a time-to-event. These are estimable from the observed data. E[ρ₁₀] is the expected value under the Beta(3α₀, 3 − 3α₀) distribution. In simulation studies [5,6] we found that prior d performed the best in terms of bias and mean squared error for both the multivariate normal model and the Gaussian copula model.

4 Applications

To illustrate the use of the Gaussian copula model, we consider the use of the ordinal variable “tumor response” as a surrogate for overall survival in advanced colorectal cancer. Tumor response and overall survival are common endpoints of interest in cancer clinical trials, and there is a large literature on the use of tumor response as a surrogate marker for overall survival [7–9]. We apply our estimation method to data from four separate meta-analyses in advanced colorectal cancer [10]. Each of the four studies is a combination of separate randomized trials. Study 1 consists of eight trials that compare uoropyrimidines given as a bolus intravenous injection (5FU bolus) to 5FU modulated by methotrexate with a total of 1,168 subjects [11], Study 2 consists of four trials that compare 5FU bolus to hepatic arterial infusion of 5FU with a total of 244 patients [12], Study 3 consists of 9 trials that compare 5FU bolus to 5FU modulated by leucovorin with a total of 1,378 patients [13] and Study 4 consists of six trials that compare 5FU bolus to 5FU given in continuous infusion with a total of 1,216 patients [14]. Within each study we pool the data from the separate trials. The results from fitting the Gaussian copula model to Study 4 have been previously presented [6]. The inclusion of these additional studies allows an assessment of whether the causal surrogacy quantity, E(log(T(1)=T(0))|S(1); S(0)) is similar across different treatment comparisons.

Patients in each trial were followed with tumor response and survival time recorded. Tumor response was defined by one of four categories: complete response (CR), partial response (PR), stable disease (SD) and progressive disease (PD). In our analysis, the true endpoint T is survival time, defined as the time from randomization to death from any cause and the surrogate end point S is tumor response, defined as a categorical variable with S = 1; 2; 3; 4 for PD, SD, PR, and CR, respectively. The binary treatment indicator for treatment Z is set to 0 for control treatment and 1 for experimental treatment. Tumor response was measured after approximately three to six months of follow-up in advance of the recorded survival time.

There are modest effects of the treatment on S and T in the 4 studies. The odds ratio for response (PR or CR) in the treatment vs. control arm was 2.05 (95% CI: 1.46, 2.88) for Study 1, 3.89 (95% CI: 2.17, 6.96) for Study 2, 2.37 (95% CI: 1.74, 3.23) for Study 3, and 1.84 (95% CI: 1.35, 2.51) for Study 4. The median survival time was longer for those in the treatment group (10.7 months, 16.3 months, 11.6 months and 12.1 months for Studies 1, 2, 3 and 4, respectively) than for those in the control group (9.1 months, 12.2 months, 11.0 months and 11.3 months for Studies 1, 2, 3 and 4, respectively) for all four studies, with an estimated hazard ratio of 0.88 (95% CI: 0.78, 0.99) in Study 1, 0.82 (95% CI: 0.64, 1.07) in Study 2, 0.98 (95% CI: 0.87, 1.10) in Study 3, and 0.89 (95% CI: 0.79, 1.00) in Study 4 for the treatment vs. control group. Figure 1 (a) and Figure 1 (b) provide a plots of E[log(T(1)=T(0))|S(1) S(0) = s] vs. S(1) S(0) = s for each of the four studies using prior a and using prior d, respectively. Figure 2 provides plots of E[log(T(1)=T(0))|S(1) S(0) = s] vs. S(1) S(0) = s with the 95% credible interval for the curve for each of the four studies using prior d.

Figure 1.

E[log(T (1)/T (0)) | S(1) − S(0) = s] vs. S(1) − S(0) from 4 studies assessing tumor response tumor response (1 =progressive disease, 2 =stable disease, 3 =partial response, 4 =complete response) as a surrogate for overall survival.

Figure 2.

E[log(T (1)/T (0)) | S(1) − S(0) = s] vs. S(1) − S(0) and 95% credible intervals from 4 studies assessing tumor response (1 =progressive disease, 2 =stable disease, 3 =partial response, 4 =complete response) as a surrogate for overall survival

Under both prior a and prior d Figure 1 shows that E[log(T(1)=T(0)) | S(1) S(0) = 0] is approximately 0 for all four studies, indicative of a good principal surrogate, with E[log(T(1)=T(0)) | S(1) S(0) = s] increasing in s. We would therefore conclude that tumor response appears to be a moderately good principal surrogate for overall survival, and the fact that the four estimated curves are quite similar suggests that this is a quantity that may generalize from one treatment comparison to the next. Figure 2 shows that there is very little uncertainty around S(1) S(0) = 0, however the credible intervals become much larger at either end of the curve when S(1) S(0) = 3 and S(1) S(0) = 3. Table 1 provides the four by four table of E[log(T(1)=T(0)) | S(0); S(1)] for every pair of values for S(0) and S(1) for each of the four studies from the estimation done using Beta priors, as well as the average fraction of the population within each of the cells across all of the iterations of the Markov chain. If S is a good principal surrogate, then E[log(T(1)=T(0)) | S(0); S(1)] in the 16 cells of this table should be similar across these four studies, even though the estimated proportion of data in each cell may differ. The estimates and uncertainty intervals don't contradict the hypothesis that E[log(T(1)=T(0)) | S(0); S(1)] is similar across studies. If E[log(T(1)=T(0)) | S(0); S(1)] is similar across studies, then it may be possible to use this information to predict treatment effects on overall survival in a future trial based only on the surrogate outcome if there is a way to predict the missing counterfacutal values of S, such as by using baseline covariate information.

Table 1.

E[log(T(1)=T(0)) | S(0); S(1)] for every pair of S(0) and S(1) and population fraction within each cell for colorectal cancer data analyzed using Beta priors

Study	S(0)	S(1)
		1	2	3	4
1	1	0.19 (0.27) 23.9%	0.72 (0.20) 18.6%	0.88 (0.37) 6.0%	0.97 (0.72) 1.1%
2		−0.31 (0.69) 23.8%	0.31 (0.54) 14.2%	0.72 (1.14) 13.2%	1.50 (2.41) 0.2%
3		0.04 (0.20) 42.8%	0.69 (0.40) 11.8%	1.12 (0.68) 5.3%	1.61 (1.30) 0.3%
4		−0.11 (0.10) 35.7%	0.69 (0.11) 12.5%	1.22 (0.19) 4.2%	1.75 (0.44) 0.4%
1	2	−0.72 (0.22) 14.3%	−0.11 (0.13) 16.7%	0.07 (0.33) 7.0%	0.22 (0.59) 1.7%
2		−0.54 (1.74) 5.0%	−0.07 (0.68) 7.9%	0.47 (0.49) 16.7%	2.00 (1.87) 0.3%
3		−0.99 (0.73) 9.5%	−0.28 (0.33) 9.3%	0.28 (0.23) 8.9%	0.96 (0.65) 1.0%
4		−0.57 (0.12) 12.3%	0.09 (0.07) 12.6%	0.61 (0.10) 8.2%	1.20 (0.24) 1.2%
1	3	−1.20 (0.45) 2.5%	−0.53 (0.29) 3.5%	−0.34 (0.41) 1.8%	−0.12 (0.70) 0.5%
2		−1.27 (2.36) 1.2%	−0.34 (1.69) 2.5%	0.27 (0.58) 13.2%	1.16 (1.50) 1.0%
3		−1.69 (1.38) 1.2%	−0.98 (0.90) 2.0%	−0.36 (0.46) 4.0%	0.40 (0.42) 1.0%
4		−0.83 (0.21) 2.0%	−0.30 (0.12) 3.6%	0.17 (0.10) 4.2%	0.69 (0.16) 1.2%
1	4	−1.47 (0.85) 0.6%	−0.91 (0.53) 0.9%	−0.81 (0.63) 0.6%	−0.62 (0.84) 0.3%
2		−1.84 (2.38) 0.02%	−0.99 (2.18) 0.04%	−0.22 (1.85) 0.4%	0.19 (1.46) 0.3%
3		−1.68 (1.76) 0.2%	−1.46 (1.45) 0.4%	−0.92 (0.82) 1.3%	−0.24 (0.55) 0.9%
4		−1.08 (0.46) 0.1%	−0.60 (0.28) 0.4%	−0.17 (0.17) 0.8%	0.36 (0.19) 0.6%

5 Discussion

We have developed methods for surrogacy assessment within the principal stratification framework when the joint distribution of the potential outcomes of S and T is multivariate normal, with an extension to non-normally distributed data through the use of a Gaussian copula model. Our simulation results [5,6] suggest that the estimation procedure is able to distinguish valid principal surrogates from invalid ones. As some parameters of the proposed model are not identifiable from the data, certain assumptions must be made in order to aid in their estimation. A Bayesian estimation strategy is used, which allows the use of context specific prior distributions on the unidentified parameters to be explored. The priors that were placed on the unidentified parameters seem reasonable in the setting that we are considering. The use of other priors or other context specific assumptions about unidentified parameters or the conditional relationships between the potential outcomes could be made to accommodate the specific setting of the trial of interest [4].