The Impact of Model Misspecification on the Individual Causal Association in Surrogate Endpoint Evaluation

ABSTRACT

Surrogate endpoints are often used in place of expensive, delayed, or rare clinical endpoints in clinical trials. However, regulatory authorities require thorough evaluation to accept these surrogate endpoints as reliable substitutes. One evaluation approach is the information‐theoretic causal inference framework, which quantifies surrogacy using the individual causal association (ICA). Like most causal inference methods, this approach relies on models that are only partially identifiable. For continuous outcomes, a normal model is often used. In this study, we explored the effects of model misspecification across various scenarios. We first considered true data‐generating mechanisms based on multivariate $t$ and log‐normal distributions. We then used D‐vine copulas with Gaussian, Clayton, Gumbel, and Frank families to vary the unidentifiable copulas involving counterfactual pairs while preserving the observable bivariate margins, and considered intuitive restrictions on nonidentified correlations, including positivity and conditional independence. In all settings, the identifiability issue was addressed through sensitivity analysis. Finally, we illustrate the proposed sensitivity analyses using clinical‐trial data from schizophrenia studies, evaluating the ICA under several modeling assumptions. The results show that, in most scenarios considered, the impact of model misspecification is small; however, certain departures from the assumed model can materially affect the surrogacy assessment.

1. Introduction

Humanitarian and commercial considerations have sparked a relentless search for methods to accelerate the development of new therapies while also reducing the associated costs. In response to this pressing need, researchers and medical professionals have eagerly explored the potential of surrogate endpoints, a general strategy that has gathered considerable interest over the last decades. Surrogate endpoints are alternative outcomes used to replace or supplement the primary clinical outcomes, known as true endpoints, in the evaluation of experimental treatments or interventions.

The primary advantage of surrogate endpoints is their ability to be measured earlier, more conveniently, or more frequently than the true endpoint. This capability not only speeds up clinical trials but also streamlines the evaluation process and makes it more resource‐friendly. As a result, regulatory agencies across the globe, especially those in the United States, Europe, and Japan, began to introduce policies to regulate the use of surrogate endpoints in the evaluation of new treatments [1, 2].

However, the critical question is how to establish the adequacy of a surrogate endpoint. In other words, how can we ensure that the treatment's effect on the surrogate accurately predicts its impact on a more clinically meaningful true endpoint? To tackle this challenge, researchers have proposed various definitions of validity and formal criteria. These approaches can be categorized into two frameworks: methodologies based on single trial data (single trial setting or STS) and methodologies using data from multiple clinical trials. Prentice [3] introduced the first statistical framework for evaluating surrogate endpoints, defining surrogacy, and establishing corresponding operational criteria in the context of STS. Freedman et al. [4] highlighted the limitations of Prentice's criteria and proposed the proportion of treatment effect (PTE) as an alternative measure. Buyse and Molenberghs [5] further extended the methodology by introducing the relative effect (RE) and adjusted association (AA) for assessing surrogates in the STS. The meta‐analytic framework introduced later by Buyse et al. [6], based on expected causal treatment effects across multiple clinical trials, is considered one of the most general and effective methods for evaluating surrogate endpoints [7]. However, its practical implementation is hindered by stringent data requirements, which are often unavailable in the early stages of drug development when surrogate endpoints are most needed [1, 2, 8]. Consequently, developing new methods for the STS remains important in the field and, in this scenario, Alonso [9] introduced a general definition of surrogacy based on individual causal treatment effects and the notion of information gain (equivalently, reduction in uncertainty). To operationalize this definition, a general surrogacy metric, the so‐called individual causal association (ICA), was proposed. The ICA quantifies the association between the individual causal treatment effects on the surrogate and the true endpoint via their mutual information. Additionally, Deliorman et al. [10] extended the ICA using Rényi divergence, with the original definition as a special case of this broader formulation. This area of research is known as the information‐theoretic causal inference (ITCI) framework.

Assessing the ICA requires a joint causal model for the potential outcomes of both variables and a suitable rescaling of mutual information with desirable mathematical properties. For continuous outcomes, Alonso et al. [11] introduced a four‐dimensional normal model together with an associated ICA definition. Because some parameters of this model are not estimable from the observed data, the ICA cannot be identified without strong, unverifiable assumptions. To address this, the authors proposed a simulation‐based sensitivity analysis in which the ICA is evaluated over ranges of unidentifiable parameters of the potential‐outcomes distribution that are compatible with the data; the resulting distribution of ICA values is then summarized using frequency plots and descriptive statistics. We refer the interested reader to Alonso et al. [11] for details.

Because the model is only partially identifiable, misspecified parametric assumptions (e.g., normality) may bias the ICA and thus the assessment of the surrogate. To our knowledge, this issue has not been systematically studied despite its practical importance. We address it here through a combination of theoretical derivations and simulation studies.

The paper is organized as follows. Section 2 introduces a single‐trial framework for assessing surrogacy with two continuous outcomes. Section 3 revisits the normal causal model. Section 4 studies misspecification in two settings: Section 4.1 analyzes the case where the true data‐generating mechanism is multivariate $t$, drawing on theoretical results, and Section 4.2 considers a multivariate log‐normal mechanism via a Monte‐Carlo simulations. Section 5 introduces D‐vine copulas and employs them to assess misspecification when the data‐generating mechanism is indistinguishable from a normal model on the basis of the observed data. In Section 6, a real‐life case study is analyzed under different modeling assumptions. Finally, Section 7 provides concluding remarks.

2. General Theoretical Background

In the following, we outline the general framework introduced by Alonso et al. [11] and Alonso [9] to evaluate the validity of a putative surrogate endpoint $S$ for a true endpoint $T$. This evaluation is carried out using data from a single randomized clinical trial with a well‐defined population, where only two treatments ($Z=0/1$) are being evaluated in a parallel study design.

The Neyman–Rubin potential outcome model assumes that each patient has a four‐dimensional vector of potential outcomes $\mathit{Y}={\left(T_0,T_1,S_0,S_1\right)}^T$ with $S_z$ and $T_z$ representing the outcome for the surrogate and the true endpoint under treatment $Z=z$. The practical implementation of the model is based on several assumptions [12]. First, the stable unit treatment value assumption (SUTVA) links the observed outcomes to the potential outcomes as follows:

$${(S,T)}^T=Z\cdot {\left(S_1,T_1\right)}^T+(1-Z)\cdot {\left(S_0,T_0\right)}^T.$$

Second, the full exchangeability assumption states that the potential outcomes are independent of the assigned treatment: ${\left(T_0,T_1,S_0,S_1\right)}^T\perp Z$. These two assumptions are typically met in randomized clinical trials and they will be used throughout the remainder of this paper.

The vector of individual causal treatment effects is defined as $\mathtt{\Delta }={(\mathrm{\Delta }T,\mathrm{\Delta }S)}^T$ where $\mathrm{\Delta }S=S_1-S_0$ and $\mathrm{\Delta }T=T_1-T_0$. Alonso et al. [9] introduced the following definition of surrogacy in the STS.

Definition 1

In the STS, we shall say that $S$ is a good surrogate for $T$ if $\mathrm{\Delta }S$ conveys a substantial amount of information on $\mathrm{\Delta }T$.

The concept of information has been rigorously defined in information theory [13]. The amount of “shared” information between $\mathrm{\Delta }S$ and $\mathrm{\Delta }T$ can be quantified using the mutual information between these individual causal treatment effects, denoted by $I(\mathrm{\Delta }T,\mathrm{\Delta }S)$. Mutual information is always nonnegative, zero if and only if $\mathrm{\Delta }S$ and $\mathrm{\Delta }T$ are independent, symmetric, and invariant under bijective transformations. Despite its attractive mathematical properties, the interpretation of mutual information can be challenging in some scenarios. For instance, it lacks an upper bound when $\mathrm{\Delta }S$ and $\mathrm{\Delta }T$ are continuous. This issue is addressed by mapping mutual information onto the unit interval, ensuring that it takes a value of zero when $\mathrm{\Delta }T$ and $\mathrm{\Delta }S$ are independent and one when there is a nontrivial transformation $\varphi$ such that $\mathrm{\Delta }T=\varphi (\mathrm{\Delta }S)$ with probability one.

The mutual information between $\mathrm{\Delta }S$ and $\mathrm{\Delta }T$ is a functional of the joint distribution $f(\mathrm{\Delta }T,\mathrm{\Delta }S)$, which is completely determined by the distribution of the vector of potential outcomes $\mathit{Y}$. When both the surrogate and true endpoints are continuous, a causal inference model for $\mathit{Y}$ is often constructed using a four‐dimensional normal distribution. Subsequently, based on this model, the surrogate's validity is assessed. This approach will be detailed in the following section.

3. The Normal Causal Model

Alonso et al. [11] assumed that $\mathit{Y}\sim \mathcal{N}(\mathit{\mu },\mathtt{\Sigma })$ with $\mathit{\mu }={\left({\mu }_{T0},{\mu }_{T1},{\mu }_{S0},{\mu }_{S1}\right)}^T$ and,

$$\mathtt{\Sigma }=\left(\begin{array}{cccc}{\sigma }_{T0T0}& {\sigma }_{T0T1}& {\sigma }_{T0S0}& {\sigma }_{T0S1}\\ {\sigma }_{T0T1}& {\sigma }_{T1T1}& {\sigma }_{T1S0}& {\sigma }_{T1S1}\\ {\sigma }_{T0S0}& {\sigma }_{T1S0}& {\sigma }_{S0S0}& {\sigma }_{S0S1}\\ {\sigma }_{T0S1}& {\sigma }_{T1S1}& {\sigma }_{S0S1}& {\sigma }_{S1S1}\end{array}\right).$$ (1)

Under this assumption, $\mathtt{\Delta }={(\mathrm{\Delta }T,\mathrm{\Delta }S)}^T=\mathbf{A}\mathit{Y}\sim \mathcal{N}\left({\mathit{\mu }}_{\mathrm{\Delta }},{\mathtt{\Sigma }}_{\mathrm{\Delta }}\right)$, where ${\mathit{\mu }}_{\mathrm{\Delta }}={(\beta ,\alpha )}^T$, $\beta =E(\mathrm{\Delta }T)$, $\alpha =E(\mathrm{\Delta }S)$ and ${\mathtt{\Sigma }}_{\mathrm{\Delta }}=\mathbf{A}\mathtt{\Sigma }{\mathbf{A}}^T$ with $\mathbf{A}$ the corresponding contrast matrix. Furthermore, these authors proposed to assess Definition 1 using the Squared Information Correlation Coefficient (SICC) [14, 15],

$$R_H^2=1-e^{-2I(\mathrm{\Delta }T,\mathrm{\Delta }S)}$$ (2)

where $I(\mathrm{\Delta }T,\mathrm{\Delta }S)$ represents the mutual information between $\mathrm{\Delta }T$ and $\mathrm{\Delta }S$, as defined by:

$$I(\mathrm{\Delta }T,\mathrm{\Delta }S)=\int f(\mathrm{\Delta }T,\mathrm{\Delta }S)\ln \frac{f(\mathrm{\Delta }T,\mathrm{\Delta }S)}{f(\mathrm{\Delta }T)f(\mathrm{\Delta }S)}d\mathrm{\Delta }\mathit{Td}\mathrm{\Delta }S$$

Basically, $I(\mathrm{\Delta }T,\mathrm{\Delta }S)$ is the Kullback–Leibler (KL) divergence between the joint density function $f(\mathrm{\Delta }T,\mathrm{\Delta }S)$ and the product of the marginal density functions $f(\mathrm{\Delta }T)f(\mathrm{\Delta }S)$ [16]. For continuous outcomes the SICC satisfies the properties given in Section 2 and, therefore, one may argue that (2) is a suitable metric to assess Definition 1, that is, one may argue that it is a good metric of surrogacy. Alonso et al. [11] called this metric the individual causal association or ICA. Under the normality assumption, $-2I(\mathrm{\Delta }T,\mathrm{\Delta }S)=\log \left(1-{\rho }_{\mathrm{\Delta }}^2\right)$ where ${\rho }_{\mathrm{\Delta }}=\text{corr}(\mathrm{\Delta }T,\mathrm{\Delta }S)$ is given by,

$${\rho }_{\mathrm{\Delta }}=\frac{\begin{array}{cc}& \sqrt{{\sigma }_{T0T0}{\sigma }_{S0S0}}{\rho }_{T0S0}+\sqrt{{\sigma }_{T1T1}{\sigma }_{S1S1}}{\rho }_{T1S1}\\ & -\sqrt{{\sigma }_{T1T1}{\sigma }_{S0S0}}{\rho }_{T1S0}-\sqrt{{\sigma }_{T0T0}{\sigma }_{S1S1}}{\rho }_{T0S1}\end{array}}{\sqrt{\begin{array}{cc}& \left({\sigma }_{T0T0}+{\sigma }_{T1T1}-2\sqrt{{\sigma }_{T0T0}{\sigma }_{T1T1}}{\rho }_{T0T1}\right)\\ & \left({\sigma }_{S0S0}+{\sigma }_{S1S1}-2\sqrt{{\sigma }_{S0S0}{\sigma }_{S1S1}}{\rho }_{S0S1}\right)\end{array}}},$$

and ${\rho }_{\mathit{XY}}$ denotes the correlation between the potential outcomes $X$ and $Y$. Therefore, under this model, one has $\mathit{ICA}=R_H^2={\rho }_{\mathrm{\Delta }}^2$.

As noted by Holland [17], only two of the four potential outcomes are observable in practice; thus, while correlations such as ${\rho }_{T_0{S}_0}$ and ${\rho }_{T_1{S}_1}$ are estimable, correlations involving the counterfactual pairs $\left(S_i,T_j\right)$, $\left(S_i,S_j\right)$, and $\left(T_i,T_j\right)$ with $i\ne j$ are not. Consequently, the $\mathit{ICA}$ cannot be assessed without untestable assumptions. Alonso et al. [11] addressed this challenge via a simulation‐based sensitivity analysis that estimates $\mathit{ICA}$ over ranges of unidentifiable parameters compatible with the observed data. This strategy, however, still assumes correct parametric specification of the distribution of $\mathit{Y}$, namely, that the vector of potential outcomes follows a four‐dimensional normal distribution. Because only the joint distributions of $\left(S_0,T_0\right)$ and $\left(S_1,T_1\right)$ are identifiable, this normality assumption for $\mathit{Y}$ cannot be fully tested. In the next sections, we investigate the consequences of incorrectly assuming normality under several nonnormal data‐generating mechanisms.

4. Model Misspecification

As mentioned previously, when the surrogate and true endpoints are continuous outcomes, the ICA is quantified under the assumption that the vector of potential outcomes $\mathit{Y}$ follows a four‐dimensional normal distribution, hereafter referred to as ${\mathit{ICA}}_N$. This assumption is critical for deriving the theoretical properties of ${\mathit{ICA}}_N$ and for its implementation in the R package Surrogate [18, 19, 20]. A critical question is the sensitivity of ${\mathit{ICA}}_N$ to deviations from the assumed normal model. In following sections, we explore the impact of model misspecification on the assessment of surrogacy. Specifically, we will consider three scenarios in which the potential outcome vector $\mathit{Y}$ is assumed to follow a four‐dimensional normal distribution but the actual data‐generating mechanism (DGM) departs from this assumption.

4.1. The t‐Distribution‐Based Data‐Generating Mechanism

The multivariate t‐distribution is a special case of a more general family, the so‐called elliptical distributions [21]. One way of defining a p‐dimensional multivariate t‐distribution is based on the fact that if $\mathit{y}\sim \mathcal{N}(\mathbf{0},\mathtt{\Sigma })$ and $u\sim {\chi }_{\nu }^2$ with $\mathit{y}\perp u$ then $\mathit{x}=\mathit{\mu }+\mathit{y}/\sqrt{u+\nu }$ follows a multivariate t‐distribution with density function,

$$f(\mathbf{x}|\mathit{\mu },\mathtt{\Sigma },\nu )=\frac{\mathrm{\Gamma }[(\nu +p)/2]}{\mathrm{\Gamma }(\nu /2){(\nu \pi )}^{p/2}{\left|\mathtt{\Sigma }\right|}^{1/2}}{\left[1+\frac{1}{\nu }{(\mathit{x}-\mathit{\mu })}^T{\mathtt{\Sigma }}^{-1}(\mathit{x}-\mathit{\mu })\right]}^{-(\nu +p)/2}.$$

The multivariate t‐distribution has parameters $\mathtt{\Sigma }$, $\mathit{\mu }$, $\nu$ and it is denoted as $\mathit{x}\sim t_p(\mathit{\mu },\mathtt{\Sigma },\nu )$. The expected value of $\mathit{x}$ equals $E(\mathit{x})=\mathit{\mu }$ for $\nu >1$ (else undefined), however, $\mathtt{\Sigma }$ is not the covariance matrix of $\mathit{x}$ since Var $(\mathit{x})=\nu /(\nu -2)\mathtt{\Sigma }$ for $(\nu >2)$. It has some interesting mathematical properties. For instance, let us consider the $p$‐dimensional vector,

$$\mathit{x}=\left(\begin{array}{c}{\mathit{x}}_1\\ {\mathit{x}}_2\end{array}\right)\sim t_p(\mathit{\mu },\mathtt{\Sigma },\nu )$$ (3)

with ${\mathit{x}}_i$ a $p_i$‐dimensional vector ($p_1+p_2=p$) and let us further consider the partitions,

$$\mathit{\mu }=\left(\begin{array}{c}{\mathit{\mu }}_1\\ {\mathit{\mu }}_2\end{array}\right)\text{and}\mathtt{\Sigma }=\left(\begin{array}{cc}{\mathtt{\Sigma }}_{11}& {\mathtt{\Sigma }}_{12}\\ {\mathtt{\Sigma }}_{21}& {\mathtt{\Sigma }}_{22}\end{array}\right).$$ (4)

Ding [22] shows that,

$${\mathit{x}}_2\mid {\mathit{x}}_1\sim t_{p_2}\left({\mathit{\mu }}_{2|1},\frac{\nu +d_1}{\nu +p_1}{\mathtt{\Sigma }}_{22|1},\nu +p_1\right)$$

with the following conditional mean ${\mathit{\mu }}_{2\mid 1}={\mathit{\mu }}_2+{\mathtt{\Sigma }}_{21}{\mathtt{\Sigma }}_{11}^{-1}\left({\mathit{x}}_1-{\mathit{\mu }}_1\right)$, conditional variance ${\mathtt{\Sigma }}_{22\mid 1}={\mathtt{\Sigma }}_{22}-{\mathtt{\Sigma }}_{12}{\mathtt{\Sigma }}_{11}^{-1}{\mathtt{\Sigma }}_{21}$ and $d_1={\left({\mathit{x}}_1-{\mathit{\mu }}_1\right)}^T{\mathtt{\Sigma }}_{11}^{-1}\left({\mathit{x}}_1-{\mathit{\mu }}_1\right)$.

The multivariate t‐distribution is also affine invariant. In other words, if $\mathit{x}\sim t_p(\mathit{\mu },\mathtt{\Sigma },\nu )$ then $\mathit{z}=\mathbf{A}\mathit{x}+\mathit{b}\sim t_p\left(\mathbf{A}\mathit{\mu }+\mathit{b},\mathbf{A}\mathtt{\Sigma }{\mathbf{A}}^T,\nu \right)$ with $\mathbf{A}\ne \mathbf{0}$ [23]. This is particularly appealing for the purposes of this work. In fact, affine invariance implies that if the vector of potential outcomes follows a four‐dimensional t‐distribution, then the bivariate vector of individual causal treatment effects will follow a bivariate t‐distribution and an analytical expression for the mutual information is available in this case. In fact, let us consider again the decomposition given in Equations ((3), (4)). Arellano‐Valle et al. [21] provided an expression for the mutual information between ${\mathit{x}}_1$ and ${\mathit{x}}_2$,

$$\begin{array}{ll}& I_t\left({\mathit{x}}_1,{\mathit{x}}_2\right)=I_N\left({\mathit{x}}_1,{\mathit{x}}_2\right)+\zeta (\nu )\text{with}\\ & \zeta (\nu )=\log \left[\frac{\mathrm{\Gamma }(\nu /2)\mathrm{\Gamma }\left[\left(\nu +p_1+p_2\right)/2\right]}{\mathrm{\Gamma }\left[\left(\nu +p_1\right)/2\right]\mathrm{\Gamma }\left[\left(\nu +p_2\right)/2\right]}\right]\\ & +\frac{\nu +p_2}{2}\psi \left(\frac{\nu +p_2}{2}\right)+\frac{\nu +p_1}{2}\psi \left(\frac{\nu +p_1}{2}\right)\\ & -\frac{\nu +p_1+p_2}{2}\psi \left(\frac{\nu +p_1+p_2}{2}\right)-\frac{\nu }{2}\psi \left(\frac{\nu }{2}\right)\end{array}$$ (5)

where $\psi (x)=d/\mathit{dx}[\mathrm{\Gamma }(x)]$ is the so‐called digamma function and,

$$I_N\left({\mathit{x}}_1,{\mathit{x}}_2\right)=-\frac{1}{2}\log \left(\frac{\left|\mathtt{\Sigma }\right|}{\left|{\mathtt{\Sigma }}_{11}\right|\left|{\mathtt{\Sigma }}_{22}\right|}\right)$$

Notice that in Equation (5) $I_N\left({\mathit{x}}_1,{\mathit{x}}_2\right)$ actually quantifies the mutual information between ${\mathit{x}}_1$ and ${\mathit{x}}_2$ under the normality assumption, which can be easily shown by considering $\mathrm{Var}(\mathit{x})=\nu /(\nu -2)\mathtt{\Sigma }$ (for $\nu >2$). Interestingly, all the information due to $\mathtt{\Sigma }$ arises only from $I_N\left({\mathit{x}}_1,{\mathit{x}}_2\right)$ while the information due to $\nu$ comes from the remaining terms. It can also be shown that as $\nu$ increases, the t mutual information converges to the normal mutual information.

Let us now assume that the true DGM for the vector of potential outcomes is given by $\mathit{Y}\sim t_4(\mathit{\mu },\mathtt{\Sigma },\nu )$ with $\mathit{\mu }$ and $\mathtt{\Sigma }$ as before. The previous properties imply that $\mathtt{\Delta }\sim t_2\left({\mathit{\mu }}_{\mathrm{\Delta }},{\mathtt{\Sigma }}_{\mathrm{\Delta }},\nu \right)$ and, hence, the true ICA value denoted by ${\mathit{ICA}}_t$ is given by:

$$\begin{array}{ll}{\mathit{ICA}}_t& =1-e^{-2I_t(\mathrm{\Delta }T,\mathrm{\Delta }S)}=1-e^{-2I_N(\mathrm{\Delta }T,\mathrm{\Delta }S)-2\zeta (\nu )}\\ & =1-\left(1-{\rho }_{\mathrm{\Delta }}^2\right)e^{-2\zeta (\nu )}\\ {\mathit{ICA}}_t& =1-\left(1-{\mathit{ICA}}_N\right)e^{-2\zeta (\nu )}\end{array}$$

where

$$\zeta (\nu )=2\log \left[\frac{\mathrm{\Gamma }(\nu /2)}{\mathrm{\Gamma }[(1+\nu )/2]}\sqrt{\frac{\nu }{2}}\right]+(1+\nu )\psi \left(\frac{1+\nu }{2}\right)-(1+\nu )\psi \left(\frac{\nu }{2}\right)-\frac{2+\nu }{\nu }.$$ (6)

To derive the final equation for $\zeta (\nu )$, we use the properties of the gamma and digamma functions: $\mathrm{\Gamma }(1+z)=z\mathrm{\Gamma }(z)$ and $\psi (1+z)=\psi (z)+1/z$.

As illustrated in Figure 1, when $\nu$ approaches infinity, $\zeta (\nu )$ converges to zero and ${\mathit{ICA}}_t={\mathit{ICA}}_N$. This is expected as the multivariate t‐distribution converges to a multivariate normal distribution as the degrees of freedom $\nu$ increase. Furthermore, Figure 2 plots the pairs $\left({\mathit{ICA}}_t,{\mathit{ICA}}_N\right)$ (dashed line) for $(\nu =\mathrm{3,4,5,7})$, with the continuous line representing the identity function $y=x$ for reference. The figure clearly shows that using the normal causal inference model, when the true DGM is characterized by a multivariate t‐distribution, will have only a mild impact on the ICA. Indeed, the effect of the misspecification is noticeable only when ${\mathit{ICA}}_t$ is small and $\nu =3$. In this scenario, the ${\mathit{ICA}}_N$ is slightly larger than the true ${\mathit{ICA}}_t$. When $\nu \ge 4$, the effect of the misspecification becomes negligible. This discussion demonstrates that the normal causal model yields practically meaningful results when the t‐causal model is the true underlying DGM.

FIGURE 1.

$\zeta (\nu )$ as a function of $\nu$.

FIGURE 2.

${\mathit{ICA}}_t$ versus ${\mathit{ICA}}_N$. The black line is the identity line ${\mathit{ICA}}_N={\mathit{ICA}}_t$.

Although encouraging, this finding should be taken with caution. The t‐distribution shares many properties with the normal distribution and converges to the normal rather quickly as the degrees of freedom increase. Therefore, the next section explores the effect of misspecification using a log‐normal distribution. Addressing this case purely from a mathematical standpoint is not feasible due to the complexity of the algebra involved. Consequently, a Monte Carlo procedure will be employed.

4.2. The Log‐Normal Data‐Generating Mechanism

In Section 4.1, we showed that assuming a normal causal inference model, when the true DGM is a multivariate t‐distribution, generally has a negligible impact on the ICA value that would be estimated. However, exploring this issue when the true DGM is based on other multivariate distributions, such as the multivariate log‐normal distribution, becomes mathematically challenging due to the lack of closed form expressions. Therefore, in order to dive deeper into this problem, we resort to a Monte Carlo procedure in this section. To that end, we now assume that the vector of potential outcomes $\mathit{Y}$ follows a four‐dimensional log‐normal distribution with the density function,

$$f\left(\mathit{y}|{\mathit{\mu }}_y,\mathtt{\Sigma }\right)=\frac{1}{\sqrt{2\pi {\left|\mathtt{\Sigma }\right|}^4}{\prod }_{i=1}^4{y}_i}{e}^{-\frac{{(\ln (\mathit{y})-\mathit{\mu })}^T{\mathtt{\Sigma }}^{-1}(\ln (\mathit{y})-\mathit{\mu })}{2}}.$$

Under the log‐normal DGM, the distribution of the individual causal treatment effects $\mathtt{\Delta }$ does not have a closed form and the true ICA, denoted hereafter as ${\mathit{ICA}}_L$, cannot be computed analytically. To approximate the distribution of $\mathtt{\Delta }$ and calculate the corresponding ${\mathit{ICA}}_L$, we used Monte Carlo simulations. Specifically, we generated 200 different pairs of $\mathit{\mu }$ and $\mathtt{\Sigma }$ for the underlying log‐normal distribution, using a normal distribution for $\mathit{\mu }$ and a Wishart distribution for $\mathtt{\Sigma }$. For each of these pairs (settings), $M_y=2000$ vectors of potential outcomes $\mathit{Y}$ and the corresponding vectors of individual causal treatment effects $\mathtt{\Delta }$ were generated. Using the previous $M_y=2000$ vectors of individual causal treatment effects $\mathtt{\Delta }$ the ${\mathit{ICA}}_L$ value (as given in Equation (2)) can be numerically approximated.

Furthermore, we also computed the ${\mathit{ICA}}_N$ under the assumption that $\mathit{Y}$ follows a four‐dimensional normal distribution, with the same mean and variance as the correct log‐normal distribution. This was done by calculating ${\rho }_{\mathrm{\Delta }}^2=\text{corr}{(\mathrm{\Delta }T,\mathrm{\Delta }S)}^2$ using the $M_y=2000$ vectors of individual causal treatment effects generated in each setting.

The main findings are summarized in Figure 3, which shows $d={\mathit{ICA}}_L-{\mathit{ICA}}_N$ across the different settings. The x‐axis displays codes that correspond to the settings, arranged in the order they were generated without any reordering. It is evident from the figure that using a misspecified model can significantly impact the results in some cases. The maximum observed value for $d$ was $0.662$, indicating that ${\mathit{ICA}}_N$ can be substantially smaller than the true ${\mathit{ICA}}_L$. Although the misspecified model generally yields smaller ICAs than the correct model ($d>0$), it can also produce larger values, with the minimum difference observed being $d=-0.042$.

FIGURE 3.

Difference between ${\mathit{ICA}}_L$ and ${\mathit{ICA}}_N$.

We further explored the settings where this difference was small and large, as defined in the Supporting Information Appendix (Section 1). We observed that in cases where the difference between ${\mathit{ICA}}_L$ and ${\mathit{ICA}}_N$ was substantial, say larger than $0.3$, the underlying log‐normal distribution used to generate the potential outcomes was notably different from a normal distribution. This includes the distribution of the identifiable margins, suggesting that one will likely be able to detect the misspecification in those cases. To further investigate these differences, we computed the Kullback–Leibler (KL) divergence between the log‐normal and normal distributions to assess whether a larger KL corresponds to a greater difference between ${\mathit{ICA}}_L$ and ${\mathit{ICA}}_N$. However, no clear pattern was observed.

The previous analyses have led to the following preliminary conclusions: When the true DGM is characterized by a four‐dimensional t‐distribution, the normal causal inference model will still deliver meaningful results. In addition, when the true DGM is characterized by a four‐dimensional log‐normal distribution, the normal causal inference model may be misleading but mostly in the scenarios where misspecification is detectableusing observable data. This emphasizes the importance of checking whether the identifiable parts of the model are compatible with the normality assumption. However, an intriguing question remains: Could a true DGM, which appears identical to the normal causal model based on observed data, lead to a different assessment of the surrogate? This issue will be examined in the following section through the use of D‐vine copulas.

5. Data Generating Mechanism Indistinguishable from the Normal Model

D‐vine copulas are a flexible class of copulas that allow for the modeling of complex dependence structures among multiple variables using bivariate copulas as building blocks. A D‐vine is a sequential tree‐like structure that allows any multivariate distribution to be factorized into a set of marginal and conditional bivariate distributions. These bivariate relationships are then described using copulas, making them easier to model and estimate individually, which is useful in the ITCI framework [24]. More details on (D‐vine) copulas and related concepts are provided in the Supporting Information Appendix (Section 2).

Let now $f_{\mathit{Y}}$ be the joint density of $\mathit{Y}={\left(T_0,S_0,S_1,T_1\right)}^T$ (note the reordering). The D‐vine density decomposition of $f_{\mathit{Y}}$ is the product of four marginal densities and six bivariate copula densities:

$$f_{\mathit{Y}}=f_{T_0}{f}_{S_0}{f}_{S_1}{f}_{T_1}\cdot c_{T_0{S}_0}{c}_{S_0{S}_1}{c}_{S_1{T}_1}\cdot c_{T_0{S}_0;S_1}{c}_{S_0{T}_1;S_1}\cdot c_{T_0{T}_1;S_0{S}_1},$$ (7)

where (i) $f_{T_0}$, $f_{S_0}$, $f_{S_1}$, and $f_{T_1}$ are univariate density functions, (ii) $c_{T_0,S_0}$, $c_{S_0,S_1}$, and $c_{S_1,T_1}$ are unconditional bivariate copula densities, and (iii) $c_{T_0,S_1;S_0}$, $c_{S_0,T_1;S_1}$, and $c_{T_0,T_1;S_0,S_1}$ are conditional bivariate copula densities. The so‐called simplifying assumption, as detailed in Definition 3 in the Supporting Information Appendix and commonly used in practice Czado [25], will be adopted in this section. This assumption posits that the three conditional copula densities in (7) do not depend on the value of the conditioning variable and it is commonly used in practice because it simplifies the D‐vine copula model. The distribution of the observable bivariate margins follows immediately from the components in (7) as follows:

$$\begin{array}{cc}& f_{S_0{T}_0}(s,t)=f_{S_0}(s)f_{T_0}(t)c_{T_0{S}_0}\left\{F_{T_0}(t),F_{S_0}(s)\right\}\\ & \text{and}{f}_{S_1{T}_1}(s,t)=f_{S_1}(s)f_{T_1}(t)c_{S_1{T}_1}\left\{F_{S_1}(s),F_{T_1}(t)\right\}\end{array}$$

where $F_{T_0},F_{S_0},F_{S_1}$, and $F_{T_1}$ are the marginal distribution functions. If the marginal distributions are normal, and both $c_{T_0{S}_0}$ and $c_{S_1{T}_1}$ are Gaussian copulas, then the densities $f_{S_0{T}_0}$ and $f_{S_1{T}_1}$ will be bivariate normal. Consequently, the distribution of $\mathit{Y}$ specified in (7) becomes indistinguishable from a four‐dimensional normal distribution based on observed data, regardless of the parametric choices for $c_{S_0{S}_1}{c}_{S_0{T}_1;S_1}{c}_{T_0{T}_1;S_0{S}_1}$.

As part of our objective to construct true DGMs that are indistinguishable from the normal causal model, we ensure that the observable bivariate margins, $f_{S_0{T}_0}$ and $f_{S_1{T}_1}$, are set as bivariate normal distributions for ${\left(S_0,T_0\right)}^T$ and ${\left(S_1,T_1\right)}^T$. The parameters of these distributions were fixed to the estimated values obtained from two distinct case studies: age‐related macular degeneration (ARMD) and a data set from five randomized clinical trials focused on schizophrenia. It is important to emphasize that, in this section, ARMD and schizophrenia are not presented as two full applied case studies. Rather, they are used as two real‐data calibrations of the identifiable parts of the causal model. This allows us to evaluate the impact of alternative assumptions on the unidentifiable copulas under realistic observable margins. The full applied case study is presented separately in Section 6, where we focus on the schizophrenia data set.

Additionally, for the four unidentifiable copulas in (7), we explore four parametric choices: Gaussian, Clayton, Gumbel, and Frank.

We evaluated all combinations of these parametric copulas, resulting in $4^4=256$ unique distributions for $\mathit{Y}$. A configuration consisting entirely of Gaussian copulas produces the four‐dimensional normal distribution. Regardless of the chosen copula types, the unidentifiable copulas are parameterized by Spearman's rho correlation parameters. Let $F_k(\mathit{\rho })$, for $k=1,2,\dots ,256$, denote one of these 256 distributions of $\mathit{Y}$, where $\mathit{\rho }$ is the four‐dimensional vector of correlations that characterizes the association of the counterfactual pairs $\left(S_i,T_j\right)$ with $i\ne j$. Following the methodology in Alonso [11], we sample 1000 $\mathit{\rho }$ vectors under three distinct sampling schemes:

1. No additional assumptions: The rho parameters are sampled from $U(-1,1)$.

2. Positive restricted associations: The rho parameters are assumed to be positive and bounded away from zero and one.

3. Conditional independence and positive restricted associations: In addition to positive restricted associations, we assume conditional independence: $T_0\perp S_1\mid S_0$ and $T_1\perp S_0\mid S_1$.

Basically, for each of the 256 distributions $F_k(\mathit{\rho })$ previously defined, we generate 1000 realizations in each of the scenarios (a)–(c). Notably, each distribution has normal observable margins, though only one of them is actually a four‐dimensional normal distribution. We analyze the differences $d(\mathit{\rho },F)={\mathit{ICA}}_F(\mathit{\rho })-{\mathit{ICA}}_N(\mathit{\rho })$ for each scenario, where ${\mathit{ICA}}_F(\mathit{\rho })$ and ${\mathit{ICA}}_N(\mathit{\rho })$ are the Individual Causal Associations calculated from the density $F_k(\mathit{\rho })$ and a four‐dimensional normal distribution, respectively, both derived from the same $\mathit{\rho }$ vector. Since both distributions exhibit the same observable margins and identical values for the unidentifiable parameters $\mathit{\rho }$, the difference $d(\mathit{\rho },F)$ effectively highlights the impact of the parametric copulas that characterize the distribution of the counterfactual pairs on the ICA.

Figure 4 summarizes the main findings. The rows correspond to the two identifiable normal margins, $f_{S_0{T}_0}$ and $f_{S_1{T}_1}$, derived from both data sets. The columns represent the sampling schemes described in (a)–(c). The results show that although the impact of misspecification is generally mild ($d(\mathit{\rho },F)<0.13$) in most cases, there are instances where distributions, indistinguishable from the normal based on observed data, can lead to significantly different assessments of surrogacy. It is important to highlight again that the differences $d(\mathit{\rho },F)$ quantify the impact of the parametric copulas used to characterize the distribution of the counterfactual pairs. Interestingly, when the association structure of these counterfactual pairs is restricted, as in setting (c), the effect of the misspecification substantially diminishes.

FIGURE 4.

D‐vine copula sensitivity analysis calibrated using identifiable margins estimated from two real data examples: ARMD and schizophrenia. The two rows correspond to the two choices of observable bivariate margins, $f_{S_0{T}_0}$ and $f_{S_1{T}_1}$, used to anchor the simulation at realistic parameter values. The columns correspond to the three sampling schemes for the unidentifiable associations: no additional assumptions, positive restricted associations (PA), and positive restricted associations plus conditional independence (CI). The plotted quantity is $d={\mathit{ICA}}_F-{\mathit{ICA}}_N$, the difference between the ICA computed under the D‐vine copula density (${\mathit{ICA}}_F$) and the ICA obtained under the corresponding multivariate normal model (${\mathit{ICA}}_N$).

6. Applied Case Study: Schizophrenia Data

Schizophrenia is a psychiatric disorder characterized by psychotic manifestations such as delusions, hallucinations, and disorganized thinking and behavior [26]. We analyze a dataset comprising five double‐blind randomized clinical trials in patients with schizophrenia, available in R via the Surrogate package [27]. In this section we focus on the principal findings; implementation details and full R code are provided in Supporting Information Appendix (Section 3).

A total of 2123 patients participated, with 1589 allocated to Risperidone and 534 to control. Symptoms were assessed using the Positive and Negative Syndrome Scale (PANSS) and the Brief Psychiatric Rating Scale (BPRS). PANSS comprises 30 items, whereas BPRS is an 18‐item PANSS subscale [28]. We treat PANSS, a reliable but comparatively complex scale, as the true endpoint and BPRS, a simpler, easier‐to‐administer subscale, as the surrogate. The objective is to determine whether the BPRS subscale can reliably serve as a substitute for the more complex PANSS scale in evaluating the efficacy of Risperidone or similar antipsychotic treatments.

In prior work, both endpoints were treated as approximately normal [29]. We began by reassessing this assumption: density and Q–Q plots (Web Appendix C) suggested near normality for both endpoints, yet the Shapiro–Wilk test rejected normality for all except $T_0$. With a large sample, this test has high power and may flag small, practically negligible departures from normality. Results from earlier sections indicate that mild deviations from normality typically have only a negligible or modest impact on surrogacy assessment; however, it is difficult to know a priori how substantial the impact may be in any concrete application. We therefore conducted the analysis under multiple modeling assumptions and compared results as a sensitivity analysis.

We first assessed the ICA under the assumption of normally distributed potential outcomes, using the closed‐form expression in Section 3. To address the identifiability issues described there, we implemented the sensitivity analysis of Alonso et al. [11], estimating the ICA over a wide range of unidentifiable correlations compatible with the observed data. Figure 5 displays the resulting frequency distribution of $\mathit{ICA}$ values under normality, denoted ${\mathit{ICA}}_N$. Although small ${\mathit{ICA}}_N$ values are compatible with the data, most values from the sensitivity analysis exceed $0.8$. Specifically, ${\mathit{ICA}}_N$ ranged from $0.39$ to $0.99$, with a mean of about $0.92$ and first and third quartiles of $0.91$ and $0.95$, respectively. Repeating the analysis with the $t$‐causal model produced essentially identical results, fully aligning with Section 4.1.

FIGURE 5.

Density plot of ICA values for the schizophrenia case study obtained from the three models as ${\mathit{ICA}}_N$, ${\mathit{ICA}}_t$, and ${\mathit{ICA}}_L$.

For the multivariate log‐normal model, we proceeded analogously: we estimated the identifiable log‐normal parameters from the data, then gridded the remaining nonidentifiable correlations while holding $\rho \left(S_0,T_0\right)$ and $\rho \left(S_1,T_1\right)$ at their estimated values. We retained only grid points yielding a positive‐definite covariance matrix and computed the ICA for each. Figure 5 shows the corresponding distribution of ${\mathit{ICA}}_L$ values. Notably, diagnostics on the observable margins suggest that a log‐normal specification is a poor description of these data, so this constitutes a misspecified scenario. Even so, the log‐normal model yielded slightly larger ICA values with a narrower range ($0.45$–$0.99$), a mean and median of $0.96$, and quartiles of $0.95$ and $0.97$. Thus, despite differing from the normal/$t$ results, this analysis provides even stronger evidence in favor of the surrogate.

Finally, we analyzed the data using a D‐vine copula specification. In this setup, four pair‐copulas are unidentifiable and must be fixed by untestable parametric choices. We considered the Gaussian, Gumbel, Frank, and Clayton families for each of these four components (Section 5), yielding $4^4=256$ candidate models. Figure 6 displays the frequency distributions for a random subset of four models under three sampling schemes for the unidentifiable associations: (i) no additional constraints, (ii) positive association, and (iii) positive association plus conditional independence. Three observations are in order. First, within each sampling scheme (i)–(iii), the frequency distributions across models look strikingly similar, indicating that the particular parametric choice for the unidentifiable pair‐copulas has little influence once the sampling is fixed. Second, imposing untestable constraints on the association structure can materially change the results; for example, restricting all associations to be positive has a pronounced effect on the distributional shape. The contrasts across sampling schemes illustrate the risks of relying on untestable association assumptions, a common practice in causal inference (e.g., PTE, RE), and underscore the value of our sensitivity analysis. Third, when all pair‐copulas are Gaussian and no restrictions are imposed on the association structure, the results closely match those obtained under the normal causal model, as expected.

FIGURE 6.

Frequency distributions of the ICA for the schizophrenia case study under various D‐vine copula models and sampling schemes.

As noted in Section 1, several methods have been proposed to assess surrogacy in the STS, including the Proportion of Treatment Effect (PTE) introduced by Freedman et al. [4] and refined by Parast et al. [30]. The PTE is defined in terms of expected causal treatment effects and, because there is no replication at this level in a single trial, its identification requires untestable assumptions. Moreover, as a ratio, the PTE can be numerically unstable when the overall expected treatment effect on the true endpoint is small. Stijven et al. [31] provide a comprehensive discussion of the PTE's definition, interpretation, and limitations. Using the robust approach of Parast et al. [30], the estimated PTE in our case study was $0.84$ ($95\%$ CI: 0.66–1), which supports the surrogate's validity. We also estimated the adjusted association (AA) as $0.963$ ($95\%$ CI: 0.9594–0.9667); the AA quantifies the association between both endpoints after adjusting for treatment. Note that, while a high AA may be necessary for surrogacy, it is not a sufficient condition because a high correlation alone is not enough to deem a surrogate valid.

Taken together, these results support BPRS as a viable surrogate for PANSS. Nonetheless, side‐by‐side comparisons of multiple metrics should be interpreted with care, because the estimands, assumptions, and sources of uncertainty differ across methods [32].

7. Discussion

In this study, we investigated the effects of model misspecification on the behavior of the ICA when the true distribution deviates from a four‐dimensional normal model. Through theoretical derivations and simulations we examined scenarios where the true DGM was characterized by a four‐dimensional t‐distribution and a log‐normal distribution. Our results show that the impact of model misspecification can vary, ranging from negligible when the underlying model follows a multivariate t‐distribution to significant under a log‐normal DGM. Notably, in the latter case, the substantial impact of misspecification was primarily observable when deviations from normality could be detected using the observed data.

Additionally, by employing D‐vine copulas, we explored scenarios where the true DGM was indistinguishable from the normal model based on observed data. In this setting, it is important to note that normality of the bivariate normal margins does not guarantee multivariate normality of the entire distribution. We found that in such cases, assuming normality might lead to misleading results, even though the observed data would support this assumption.

For any surrogacy metric, defining a threshold that guarantees good surrogacy is very challenging. The general intuition is that the further the true distribution deviates from the assumed model, such as from normality, the greater the potential impact on the results. In situations with substantial uncertainty regarding model assumptions, we recommend conducting sensitivity analyses under alternative plausible distributions to evaluate the robustness of the conclusions, as illustrated in the case study. Moreover, ICA results can be compared with those obtained from alternative surrogate evaluation measures. Although these methods may differ in methodology and computational implementation, it is desirable that they lead to conclusions in the same direction.

Our findings highlight the complexity of evaluating surrogates using causal inference models, which are only partially identifiable from the data. The unidentifiable components can substantially impact the assessment of surrogacy under certain conditions. Practically, our results underscore the importance of verifying that the identifiable components of the model are consistent with the observed data and suggest that considering different modeling assumptions for the counterfactual pairs is crucial to understanding how they influence the final conclusions.

Funding

This research was supported by the MCIN/AEI/10.13039/501100011033/ and by ERDF A way of making Europe (Grant/Award Number: PID2022‐137050NB‐I00) and Agentschap Innoveren & Ondernemen and Janssen Pharmaceutical Companies of Johnson & Johnson Innovative Medicine through a Baekeland Mandate (Grant/Award Number: HBC.2022.0145).

Ethics Statement

The authors have nothing to report.

Conflicts of Interest

The authors declare no conflicts of interest.

Supporting information

Figure S1: The density graphs of potential outcome vector when the difference is −0.00079.

Figure S2: The density graphs of potential outcome vector when the difference is 0.0058.

Figure S3: The density graphs of potential outcome vector when the difference is 0.0543.

Figure S4: The density graphs of potential outcome vector when the difference is 0.5705.

Figure S5: The density graphs of potential outcome vector when the difference is 0.3464.

Figure S6: The density graphs of potential outcome vector when the difference is 0.4398.

Table S1: Interpretation of the unidentifiable components of the D‐vine density $f_{\mathit{Y}}$ in (3).

Table S2: Estimated bivariate normal distributions in the ARMD and Schizo data sets from the Surrogate R package.

Figure S7: Histograms of endpoints.

Figure S8: Q–Q plots of endpoints.

Data Availability Statement

The data that support the findings of this study are available in R Surrogate package. These data were derived from the following resources available in the public domain: https://cran.r‐project.org/web/packages/Surrogate/index.html.