Sample size estimation using a latent variable model for mixed outcome co-primary, multiple primary and composite endpoints

Abstract

Mixed outcome endpoints that combine multiple continuous and discrete components are often employed as primary outcome measures in clinical trials. These may be in the form of co-primary endpoints, which conclude effectiveness overall if an effect occurs in all of the components, or multiple primary end-points, which require an effect in at least one of the components. Alternatively, they may be combined to form composite endpoints, which reduce the outcomes to a one-dimensional endpoint. There are many advantages to joint modeling the individual outcomes, however in order to do this in practice we require techniques for sample size estimation. In this article we show how the latent variable model can be used to estimate the joint endpoints and propose hypotheses, power calculations and sample size estimation methods for each. We illustrate the techniques using a numerical example based on a four-dimensional end-point and find that the sample size required for the co-primary endpoint is larger than that required for the individual endpoint with the smallest effect size. Conversely, the sample size required in the multiple primary case is similar to that needed for the outcome with the largest effect size. We show that the empirical power is achieved for each endpoint and that the FWER can be sufficiently controlled using a Bonferroni correction if the correlations between endpoints are less than 0.5. Otherwise, less conservative adjustments may be needed. We further illustrate empirically the efficiency gains that may be achieved in the composite endpoint setting.

1. Introduction

Sample size estimation plays an integral role in the design of a study. The objective is to determine the minimum sample size that is large enough to detect, with a specified power, a clinically meaningful treatment effect. Although it is crucial that investigators have enough patients enrolled to detect this effect, overestimating the sample size also has ethical and practical implications. Namely, in a placebo-controlled trial, more patients are subjected to a placebo arm than is necessary, therefore withholding access to potentially beneficial drugs from them and delaying access to future patients.^1–3 Furthermore it results in longer, more expensive trials, using resources that could be allocated elsewhere.

One vital aspect of sample size determination is the primary endpoint. Typically this is a single outcome, however in some instances there may be multiple outcomes of interest and so various combinations of these outcomes can be selected as the primary endpoint, depending on the hypothesis of interest. Assuming we have three outcomes of interest ν₁, ν₂ and ν₃, one option is a co-primary endpoint, which takes the form of the multivariate endpoint ν₁ ∩ ν₂ ∩ ν₃. This means that an intervention is deemed to be effective overall if it is shown to be effective in each of ν₁, ν₂, and ν₃. Alternatively multiple primary endpoints may be of interest, which take the multivariate form v₁ ∪ ν₂ ∪ ν₃, where an intervention is deemed effective if it is shown to be effective in at least one of ν₁, ν₂, or ν₃. Another possibility is a composite endpoint, involving some function that maps the multivariate outcome to a univariate outcome for inference, for example ν₁ + ν₂ + ν₃. In this case the outcomes within the composite may be assigned equal or differing degrees of relevance depending on clinical importance.⁴ Alternatively, the composite endpoint may combine outcomes by labeling patients as ‘responders’ or ‘non-responders’ based on whether they exceed predefined thresholds in each of the outcomes. For instance, we let a response indicator S = 1 if ν₁ ≤ η₁, ν₂ ≤ η₂, and ν₃ ≤ η₃, where η denotes the response cutpoints. Note that the composite case is distinct from the others in that it combines the parameters and hence test statistics for each outcome into one, rather than these remaining separate for each outcome. This will have implications for sample size estimation.

For each of these endpoints, the individual outcomes may be a mix of multiple continuous, ordinal, and binary measures. One possible way to jointly model the outcomes is using a latent variable framework, arising in the graphical modeling literature, in which discrete outcomes are assumed to be latent continuous variables subject to estimable thresholds and modeled using a multivariate normal distribution.^5,6 By employing this framework we can take account of the correlation between the outcomes, improve the handling of missing data in individual components and potentially increase efficiency. Furthermore, in the case of multiple primary outcomes, it may reduce the severity of multiple testing corrections required by accounting for correlation between endpoints.

A barrier to adopting these techniques is a lack of consensus on sample size determination. A recent and comprehensive overview of the existing literature for sample size calculation in clinical trials with co-primary and multiple primary endpoints is provided by Sozu et al.⁷ The review found many proposals for power and sample size calculations for multiple continuous outcomes. In the co-primary case, some of these were based on assuming that the endpoints were bivariate normally distributed,^8,9 and extended for the case of more than two endpoints.^10,11 Other work focused on testing procedures^12,13 and controlling the type I error rate.^14–17 Similar ideas were investigated for multiple primary endpoints.^14,17–19

Approaches to sample size estimation for composite endpoints have focused primarily on the case of multiple binary components.^20–25 In the case of binary co-primary endpoints, five methods of power and sample size calculation based on three association measures have been introduced.²⁶ Additionally, sample size calculation for trials using multiple risk ratios and odds ratios for treatment effect estimation is discussed by Hamasaki et al,²⁷ and Song²⁸ explores co-primary endpoints in non-inferiority clinical trials. Consideration has also been given to the case where two co-primary endpoints are both time-to-event measures where effects are required in both endpoints,^29–31 and at least one of the endpoints.^32,33 Furthermore, composites comprised of time-to-event measures are common, in which the composite reflects time-to-first-event variable.³⁴ Sample size estimation in this case has been considered by Sugimoto et al.³⁵

The mixed outcome setting has received substantially less consideration. One paper considers overall power functions and sample size determinations for multiple co-primary endpoints that consist of mixed continuous and binary variables.³⁶ They assume that response variables follow a multivariate normal distribution, where binary variables are observed in a dichotomized normal distribution, and use Pearson’s correlations for association. A modification was suggested by Wu and de Leon³⁷ which involved using latent-level tests and pairwise correlations, and provided increased power. Thesemethods focus on the co-primary endpoint case,where effects are required in all outcomes. The case ofmultiple primary or composite endpoints where the components are measured on different scales has not been considered, each of which will require distinct hypotheses. In practice, if a mixed outcome composite is selected as the primary endpoint in a trial then the sample size calculation may be based on an overall binary endpoint or collapsed to form multiple binary endpoints however this will result in a large loss in efficiency.³⁸

In this article we build on the existing work for co-primary continuous and binary endpoints to include any combination of continuous, ordinal, and binary outcomes for co-primary, multiple primary, and composite endpoints. We propose a framework based on the same latent variable model and show how it may be tailored to each of the three endpoints to facilitate sample size estimation. The article will proceed as follows: in Section 2 we introduce the latent variable model, detailing how it can be used in each context, and specify hypothesis tests for each of the three combinations of mixed outcomes; in Section 3 we propose power calculations and sample size estimation techniques in each case; in Section 4 we illustrate the methods on a four dimensional endpoint consisting of two continuous, one ordinal and one binary outcome using a numerical example based on the MUSE trial;³⁹ and in Section 5 we simulate the empirical power for each test and the FWER for the union-intersection test. We conclude with a discussion and recommendations for practice in Section 6, and introduce user-friendly software and documentation for implementation in Section 7.

2. Endpoints and Hypothesis Testing

2.1. Latent variable framework

Let n_T and n_C represent the number of patients in the treatment group and the control group respectively and let K be the number of outcomes measured for each patient. Let $Y_{Ti}={\left(Y_{Ti1},\dots ,Y_{TiK}\right)}^T,i=1,\dots ,n_T$ be vector of K responses for patient i on the treatment arm and $Y_{Ci}={\left(Y_{Ci1},\dots ,Y_{CiK}\right)}^T,i=1,\dots ,n_C$ the vector of K responses for patient i on the control arm. Without loss of generality, the first $1\le k\le k_m$ elements of Y_Ti and Y_Ci are observed as continuous variables, the next $k_m<k\le k_o$ are observed as ordinal and the remaining $k_o<k\le K$ are observed as binary. For instance, for a three dimensional endpoint with one continuous, one ordinal and one binary measure, k_m = 1, k_o = 2, and K = 3. We use the biserial model of association by Tate,⁴⁰ which is based on latent continuous measures manifesting as discrete variables. Formally, we say that Y_Ti and Y_Ci have latent variables $Y_{\text{Ti}}^{*}$ and $Y_{\text{Ci}}^{*}$ respectively, where $Y_{Ti}^{*}\sim N_K\left({\mu }_T,{\sum }_T\right)$ and $Y_{Ci}^{*}\sim N_K\left({\mu }_C,{\sum }_C\right),$ where ${\mu }_T=\left({\mu }_{1T},\dots {\mu }_{KT}\right),{\mu }_{kT}={\mu }_{kT0}+{\mu }_{kT1}{x}_{kT1}+\dots +{\mu }_{kTp}{x}_{kTp}$ and x_kT1…x_kTP denotes the p covariates included in the model for outcome k. Likewise ${\mu }_C=\left({\mu }_{1C},\dots {\mu }_{KC}\right)$ are the corresponding quantities for the control arm. Then for $k\ne k^{\prime }:1\le k<k\prime \le k_m$ let $Var\left(Y_{Tik}\right)={\sigma }_{Tk}^2,Var\left(Y_{Cik}\right)={\sigma }_{Ck}^2$ and $Corr\left(Y_{Tik},Y_{Ti{k}^{\prime }}\right)={\rho }_{Tk{k}^{\prime }},Corr\left(Y_{Cik},Y_{Ci{k}^{\prime }}\right)={\rho }_{Ck{k}^{\prime }}$ where ${\rho }_{Tk{k}^{\prime }}$ and ${\rho }_{Ck{k}^{\prime }}$ are the associationmeasures between the endpoints. For $k_m<k\le K,Var\left(Y_{Tik}^{*}\right)=Var\left(Y_{Cik}^{*}\right)=1$ and $Corr\left(Y_{Tik}^{*},Y_{Ti{k}^{\prime }}^{*}\right)={\rho }_{Tk{k}^{\prime }}^{*},Corr\left(Y_{Cik}^{*},Y_{Ci{k}^{\prime }}^{*}\right)={\rho }_{Ck{k}^{\prime }}^{*}.$ The latent variables can be related to the observed variables by:

• $1\le k\le k_m:Y_{Tik}=Y_{Tik}^{*}$ and $Y_{Cik}=Y_{Cik}^{*}$

• $k_m<k\le k_o:Y_{Tik}=\{\begin{array}{ccc}0& \text{if}& {\tau }_{k0}\le Y_{Tik}^{*}<{\tau }_{k1},\\ 1& \text{if}& {\tau }_{k1}\le Y_{Tik}^{*}<{\tau }_{k2},\\ ⋮& ⋮\\ w_k& \text{if}& {\tau }_{k{w}_k}\le Y_{Tik}^{*}<{\tau }_{k\left(w_k+1\right)}\end{array}{Y}_{Cik}=\{\begin{array}{ccc}0& \text{if}& {\tau }_{k0}\le Y_{Cik}^{*}<{\tau }_{k1},\\ 1& \text{if}& {\tau }_{k1}\le Y_{Cik}^{*}<{\tau }_{k2},\\ ⋮& ⋮\\ w_k& \text{if}& {\tau }_{k{w}_k}\le Y_{Cik}^{*}<{\tau }_{k\left(w_k+1\right)}\end{array}$

• $k_o<k\le K:Y_{Tik}=\{\begin{array}{ccc}0& \text{if}& {\tau }_{k0}\le Y_{Tik}^{*}<{\tau }_{k1},\\ 1& \text{if}& {\tau }_{k1}\le Y_{Tik}^{*}<{\tau }_{k2}\end{array}{Y}_{Cik}=\{\begin{array}{ccc}0& \text{if}& {\tau }_{k0}\le Y_{Cik}^{*}<{\tau }_{k1},\\ 1& \text{if}& {\tau }_{k1}\le Y_{Cik}^{*}<{\tau }_{k2}\end{array}$

We set τ_{k 0} = −∞, τ_k(w_k+1) = ∞ and the intercepts μ_kT0 and μ_kC0 equal to zero for k_m < k ≤ k_o in order to estimate the cut-points. Additionally, τ_k0 = − ∞, τ_k1 = 0, τ_k2 = ∞ for k_o < k ≤ K so that the intercepts can be estimated for the outcomes observed as binary. The mixed outcomes are then combined as follows.

2.2. Co-primary endpoint

In this case, a treatment must be shown to be effective as measured by each of the outcomes in order to be deemed effective overall. We generalize previous work for mixed continuous and binary outcomes to include ordinal outcomes, as shown below.^36,37 In many clinical trials the hypothesis of interest is based on superiority, namely that the proposed treatment will perform better than the control treatment. The null hypothesis is that the difference in treatment effects for the treatment arm and control arm is less than or equal to zero. This is straightforward to formalize in the case of one endpoint but less so when there are multiple co-primary endpoints, particularly when they are measured on different scales. The hypothesis of interest is as shown in (1)

$$\begin{array}{l}{H}_0:\exists ks.t.{\pi }_{Tk}-{\pi }_{Ck}\le 0\\ H_1:{\pi }_{Tk}-{\pi }_{Ck}>0\forall k,\end{array}$$ (1)

where π_Tk and π_Ck is the effect of the intervention in the treatment and control arm respectively. For k_o < k ≤ K we can specify ${\pi }_{Tik}=P\left(Y_{Tik}=0\right)=P\left(Y_{Tik}^{*}<0\right)$ and ${\pi }_{Cik}=P\left(Y_{cik}=0\right)=P\left(Y_{Cik}^{*}<0\right)$ for the treatment and control group.

We can generalize this assumption to account for the ordinal endpoints based on the fact that for $k_m<k\le k_o{\pi }_{Tik}P\left(Y_{Tik}=w_k\right)=P\left({\tau }_{k{w}_k}<Y_{Tik}^{*}<{\tau }_{k\left(w_k+1\right)}\right).$ The definition of treatment effect for ordinal outcomes may be modified to include multiple ordinal levels by selecting the appropriate τ thresholds. For instance, ${\pi }_{Tik}=P\left(Y_{Tik}=0\right)+P\left(Y_{Tik}=1\right)+P\left(Y_{Tik}=2\right)=P\left(-\infty <Y_{Tik}^{*}<{\tau }_{k3}\right).$ As the latent means are estimable by maximum likelihood, ${\mu }_{Ti1}^{*}={\Phi }^{-1}\left({\pi }_{Ti1}\right),\dots ,{\mu }_{Tik}^{*}={\Phi }^{-1}\left({\pi }_{Tik}\right)$ in the treatment group and ${\mu }_{Ci1}^{*}={\Phi }^{-1}\left({\pi }_{Ci1}\right),\dots ,{\mu }_{Cik}^{*}={\Phi }^{-1}\left({\pi }_{CiK}\right)$ in the control group.

We can proceed by specifying that the hypothesis in (1) holds if and only if the hypothesis

$$\begin{array}{l}{H}_0^{*}:\exists ks.t.{\delta }_k^{*}\le 0\\ H_1^{*}:{\delta }_k^{*}>0\forall k,\end{array}$$ (2)

holds, where ${\delta }_k^{*}={\mu }_{TK}^{*}-{\mu }_{CK}^{*},{\mu }_{TK}^{*}=1/n_T{\Sigma }_{i=1}^{nT}{\mu }_{Tik}^{*}$ and ${\mu }_{CK}^{*}=1/n_C{\Sigma }_{i=1}^{nC}{\mu }_{Cik}^{*}.$ The maximum likelihood estimates ${\widehat{\mu }}_{TK}^{*}$ and ${\widehat{\mu }}_{CK}^{*}$ can be used for a test of $H_0^{*}$ and the variance of this test statistic can be obtained using the inverse of the Fisher information matrix.

2.3. Multiple primary endpoint

Multiple primary endpoints conclude a treatment is effective if it is shown to work in at least one of the outcomes. We would expect the sample size required to be reduced compared with the co-primary endpoint case which would require power to detect treatments in all outcomes. We can allow for sample size estimation for multiple primary endpoints as follows.

The hypothesis of interest, accounting for the fact that a significant effect in only one outcome is required, is shown below.

$$\begin{array}{l}{H}_0:{\pi }_{Tk}-{\pi }_{Ck}\le 0\forall k\\ H_1:\exists ks.t.{\pi }_{Tk}-{\pi }_{Ck}>0.\end{array}$$ (3)

As before, π_Tk and π_Ck can be determined for k_m < k ≤ k_o using the relevant τ thresholds.

$$\begin{array}{l}{H}_0^{*}:{\delta }_k^{*}\le 0\forall k\\ H_1:\exists ks.t.{\delta }_k^{*}>0.\end{array}$$ (4)

The difference in latent means ${\delta }_k^{*}={\mu }_{Tk}^{*}-{\mu }_{Ck}^{*}$ and their variance are estimated using the maximum likelihood estimates and Fisher information matrix, as before.

2.4. Composite endpoint

A review conducted by Wason et al⁴¹ showed that composite responder endpoints are widely used and identified many clinical areas in which they are common, such as oncology, rheumatology, cardiovascular, and circulation. The latent variable framework may be used to model the underlying structure of these mixed outcome composite endpoints to greatly improve efficiency.³⁸ The joint distribution of the continuous, ordinal, and binary outcomes is modeled using the latent variable structure as before. However, in this case the endpoint of interest is a composite responder endpoint and so the required quantity is some function of the probability of response in the treatment group p_T and in the control group p_C.

For instance, an overall responder index S_i can be formed for patient i, where S_i = 1 if $Y_{i1}\le {\eta }_1,\dots ,Y_{ik}^{*}\le {\eta }_k$ and 0 otherwise, where the quantities (η₁, …, η_K) are predefined responder thresholds. Generalizations where response only requires a certain number of the components to meet the thresholds are possible, but involve more complex sums. Note that this definition of response is distinct from that commonly found in composites formed from survival endpoints or binary composites typical in cardiovascular studies. We can specify p_iT and p_iC, the probability of response for patient i in the treatment arm and control arm respectively, as shown in (5),

$$\begin{array}{l}{p}_{iT}=P\left(S_i=1|T_i=1\right)={\displaystyle {\int }_{-\infty }^{{\eta }_1}\dots }{\displaystyle {\int }_{-\infty }^{{\eta }_K}{f}_{Y_1},\dots ,Y_K\left(y_{i1},\dots ,y_{iK}|T_i=1,\theta \right)d{y}_K\dots dy1}\hfill \\ p_{iC}=P\left(S_i=1|T_i=0\right)={\displaystyle {\int }_{-\infty }^{{\eta }_1}\dots }{\displaystyle {\int }_{-\infty }^{{\eta }_K}{f}_{Y_1},\dots ,Y_K\left(y_{i1},\dots ,y_{iK}|T_i=0,\theta \right)d{y}_K\dots dy1,}\hfill \end{array}$$ (5)

where θ is the vector of model parameters and we assume that $p_T\sim N\left({\delta }_T,{\sigma }_{{\delta }_T}^2\right)$ and $p_C\sim N\left({\delta }_C,{\sigma }_{\delta }^2\right).$ As in the case of co-primary and multiple primary endpoints, the assumptions allow us to estimate latent means $\left({\mu }_{k_{m+1}}^{*},\dots ,{\mu }_K^{*}\right)$ for the observed discrete components using the model parameters.

In the mixed outcome composite endpoint setting, note that although we are exploiting the latent multivariate Gaussian structure for efficiency gains we are ultimately still interested in a one dimensional endpoint, such as the difference in response probabilities between the treatment and control arms of the trial. This is distinct from the co-primary and multiple primary endpoints cases, where the overall hypothesis test must be based on some union or intersection of the hypotheses for the individual outcomes. For the composite endpoint we can formulate the hypothesis as shown in (6),

$$\begin{array}{c}{H}_0:p_T-p_C\le 0\\ H_1:p_T-p_C>0,\end{array}$$ (6)

where p_T and p_C are as in (5). For sample size estimation, we require the distribution of δ = p_T − p_C under H₁, which we can assume to be $\delta \sim N\left({\delta }_T-{\delta }_C,{\mathit{\sigma }}_{\delta }^2\right).$ The hypothesis can therefore be stated as

$$\begin{array}{c}{H}_0:{\delta }^{*}\le 0\\ H_1:{\delta }^{*}>0,\end{array}$$ (7)

where ${\delta }^{*}={\delta }_T^{*}-{\delta }_C^{*},{\delta }_T^{*}={\Phi }_K\left({\eta }_1,\dots ,{\eta }_K;{\mathit{\mu }}_T^{*},{\Sigma }_T\right),{\delta }_C^{*}={\Phi }_K\left({\eta }_1,\dots ,{\eta }_K;{\mathit{\mu }}_C^{*},{\Sigma }_C\right)$and Φ_K(.;μ,Σ) is the K-dimensional multivariate normal distribution function, with mean vector μ and covariance matrix Σ. Estimates of the quantities can be obtained using the maximum likelihood estimates for the model parameters, as in the co-primary and multiple primary endpoint settings, so that ${\widehat{\delta }}_T^{*}={\Phi }_K\left({\eta }_1,\dots ,{\eta }_K;{\widehat{\mu }}_T^{*},{\widehat{\Sigma }}_T\right)$ and ${\widehat{\delta }}_C^{*}={\Phi }_K\left({\eta }_1,\dots ,{\eta }_K;{\widehat{\mu }}_C^{*},{\widehat{\Sigma }}_C\right),$ where ${\mathit{\mu }}_T^{*}$ is the K-dimensional vector of mean values in the treatment arm and ${\mathit{\mu }}_C^{*}$ is the corresponding vector for the control arm. Using a Taylor series expansion, we can obtain the quantity ${\sigma }_{\delta }^2$ using the fact that $\mathrm{var}\left({\widehat{\delta }}^{*}\right)\approx {\left(″\delta \right)}^{T}Cov\left(\widehat{\theta }\right)\left(\prime \delta \right).$ Then, $v\widehat{a}r\left({\widehat{\delta }}^{*}\right)={\left(″{\delta }_T\right)}^T\widehat{Cov}\left(\widehat{\theta }\right)\left(″{\delta }_T\right),$ where ″δ is the vector of partial derivatives of δ* with respect to each of the parameter estimates. We can obtain $\widehat{\theta }$ and covariance matrix $\widehat{Cov}\left(\widehat{\theta }\right)$ by fitting the model to pilot trial data.

3. Sample Size Estimation

3.1. Co-primary endpoints

To construct the power function, we define the required quantities as follows. Let ${\overline{Y}}_{Tk}-{\overline{Y}}_{Ck}$ and ${\widehat{\mu }}_{Tk}^{*}-{\widehat{\mu }}_{Ck}^{*}$ denote the difference in sample means for the continuous and discrete outcomes respectively. We assume ${\delta }_k={\mu }_{Tk}-{\mu }_{Ck},{\delta }_k^{*}={\mu }_{Tk}^{*}-{\mu }_{Ck}^{*},\kappa =n_C/n_T$ and let z_α denote the (1 − α)100th standard normal percentile, where α is the prespecified significance level. We define the z score as $Z_k=\frac{{\overline{Y}}_{Tk}-{\overline{Y}}_{Ck}}{{\sigma }_k\sqrt{\frac{1+\kappa }{\kappa n_T}}}$ and $Z_k^{*}=\frac{{\widehat{\mu }}_{Tk}^{*}-{\widehat{\mu }}_{Ck}^{*}}{\sqrt{\frac{1+\kappa }{\kappa n_T}}}$ for the observed continuous and latent continuous measures respectively. The test statistic can then be defined as shown below.

$$Z_k^{†}=\{\begin{array}{l}\begin{array}{cc}{Z}_k-\frac{{\delta }_k}{{\sigma }_k}\sqrt{\frac{\kappa n_T}{1+\kappa }}=\frac{{\overline{Y}}_{Tk}-{\overline{Y}}_{Ck}-{\delta }_k}{{\sigma }_k\sqrt{\frac{1+\kappa }{\kappa n_T}}},& k=1,\dots ,k_m\end{array}\\ \begin{array}{cc}{Z}_k^{*}-{\delta }_k^{*}\sqrt{\frac{\kappa n_T}{1+\kappa }}=\frac{{\widehat{\mu }}_{Tk}^{*}-{\widehat{\mu }}_{Ck}^{*}-{\delta }_k^{*}}{\sqrt{\frac{1+\kappa }{\kappa n_T}}},& k=k_{m+1},\dots ,K\end{array}\end{array},$$ (8)

$$z_k^{†}=\{\begin{array}{l}\begin{array}{cc}{z}_{\alpha }-\frac{{\delta }_k}{{\sigma }_k}\sqrt{\frac{\kappa n_T}{1+\kappa }},& k=1,\dots ,k_m\end{array}\\ \begin{array}{cc}{z}_{\alpha }-{\delta }_k^{*}\sqrt{\frac{\kappa n_T}{1+\kappa }},& k=k_{m+1},\dots ,K\end{array}\end{array}.$$ (9)

A useful property of $Z^{†}={\left(Z_1^{†},\dots Z_K^{†}\right)}^T$ is that it is asymptotically multivariate normal under regularity conditions.¹¹The power function for the joint co-primary endpoints is as shown in (10) and hence can be approximated by (11).

$$1-\beta =P\left(\underset{k=1}{\overset{k_m}{\cap }}\{Z_k>z_a\}\underset{k_{m+1}}{\overset{K}{\cap }}\{z_k^{*}>z_a\}|\delta \right)\simeq P\left(\underset{k=1}{\overset{K}{\cap }}\{z_k^{†}>z_k^{†}\}|\delta \right),$$ (10)

for $\delta ={\left({\delta }_1,\dots ,{\delta }_{k_m},\dots ,{\delta }_{k_{\mathrm{o}}},\dots ,{\delta }_K\right)}^T\ne 0.$

$$1-\beta \simeq P\left(\underset{k=1}{\overset{K}{\cap }}\{z_k^{†}>z_k^{†}\}|\delta \right)\text{=}{\Phi }_K\left(-z_1^{†},\dots ,-z_K^{†};\Gamma \right).$$ (11)

Assuming n_T = n_C = n it is possible to rearrange (11) to obtain a sample size formula in terms of n as shown below.⁷

$$n=\frac{{\left(C_K+z_{\alpha }\right)}^2}{{\delta }_K^2},$$ (12)

where the sample size depends on the number of outcomes and C_K is the solution of

$$1-\beta ={\displaystyle {\int }_{-\infty }^{{\gamma }_1{C}_K+z_{\alpha }\left({\gamma }_1-1\right)}\cdots }{\displaystyle {\int }_{-\infty }^{{\gamma }_{K-1}{C}_K+z_{\alpha }\left({\gamma }_{K-1}-1\right)}{\displaystyle {\int }_{-\infty }^{C_K}f\left(z_1,\dots ,z_K^{*};\mathbf{0},\Gamma \right)d{z}_K^{*}\dots d{z}_1.}}$$ (13)

Alternatively, we can input different values for n in (11) to achieve the required power.

3.2. Multiple primary endpoints

Using the $Z_k^{†}$ and $Z_k^{†}$ defined for co-primary endpoints and assuming n_T = n_C = n, we can define the overall power as in (14).

$$1-\beta =P\left(\underset{k=1}{\overset{k_m}{\cup }}\{Z_k>z_a\}\underset{k_{m+1}}{\overset{K}{\cup }}\{z_k^{*}>z_a\}|\delta \right)\simeq P\left(\underset{k=1}{\overset{K}{\cup }}\{z_k^{†}>z_k^{†}\}|\delta \right).$$ (14)

In order to obtain an appropriate power function we rely on the inclusion-exclusion principle as follows.

$$\begin{array}{lll}\hfill P\left({\displaystyle \underset{k=1}{\overset{K}{\cup }}\left\{Z_k^{†}>z_k^{†}\right\}|\delta }\right)& =\hfill & {\displaystyle \sum _{k=1}^{K}P\left(\left\{Z_k^{†}>z_k^{†}\right\}|\delta \right)-{\displaystyle \sum _{k<l}P\left(\left\{Z_k^{†}>z_k^{†}\right\}\cap \left\{Z_l^{†}>z_l^{†}\right\}|\delta \right)}}\hfill \\ +\hfill & {\displaystyle \sum _{k<l<m}P\left(\left\{Z_k^{†}>z_k^{†}\right\}\cap \left\{Z_l^{†}>z_l^{†}\right\}\cap \left\{Z_m^{†}>z_m^{†}\right\}|\delta \right)}\hfill \\ +\hfill & \cdots +{\left(-1\right)}^{K-1}{\displaystyle \sum _{k<\cdots <K}P\left({\displaystyle \underset{k=1}{\overset{K}{\cap }}\left\{Z_k^{†}>z_k^{†}\right\}|\delta }\right).}\hfill \end{array}$$

A closed form expression for the overall power is shown in (15)

$$P\left({\displaystyle \underset{k=1}{\overset{K}{\cup }}\left\{Z_k^{†}>z_k^{†}\right\}|\delta }\right)={\displaystyle \sum _{i=1}^K\left({\left(-1\right)}^{i-1}{\displaystyle \sum _{I\subseteq \left\{1,\dots ,K\right\}}P\left({\displaystyle \underset{k\in I}{\cap }\left\{Z_k^{†}>z_k^{†}\right\}|\delta }\right)}\right).}$$ (15)

We then input different values for n to achieve the required power. Note when using the union-intersection test for multiple primary endpoints that a correction must be applied to control the family-wise error rate (FWER). Approaches used for multiple primary continuous endpoints, such as Bonferroni and Holm corrections, may also be implemented in this setting.

3.3. Composite endpoints

As the endpoint of interest is specified in terms of the overall one dimensional composite endpoint, we can use the formula assumed when employing the standard test of proportions technique. As ${\sigma }_{\delta }=\sqrt{\frac{{\sigma }_{\delta T}^2}{n_T}+\frac{{\sigma }_{\delta C}^2}{n_C}},$ we can assume that σ_T = σ_C = σ and n_T = n_C = n, so that $\delta \sim N\left({\delta }_T-{\delta }_C,2{\sigma }^2/n\right).$ The power is deduced in the standard way, as demonstrated below.

$$\begin{array}{c}1-\beta =P\left({\overline{P}}_T-{\overline{P}}_C>z_{\alpha }\sqrt{2{\sigma }^2/n}|H_1\right)\\ =P\left(z>\frac{z_{\alpha }\sqrt{2{\sigma }^2/n}-{\delta }^{*}}{\sqrt{2{\sigma }^2/n}}|H_1\right)\\ \hfill =\Phi \left(\frac{{\delta }^{*}}{\sqrt{2{\sigma }^2/n}}-z_a\right).\hfill \end{array}$$ (16)

Note that ${\sigma }_{\sigma }^2=\frac{2{\sigma }^2}{n},$ however to obtain a formula in terms of the required sample size we will need to separate n from the variance estimate. By fitting the model to pilot trial data we can obtain an estimate for σ², as the value of n will be known in this instance and n can be obtained using (17).

$$n=\frac{2{\widehat{\sigma }}^2{\left(z_{1-\beta }+z_{\alpha }\right)}^2}{{\widehat{\sigma }}^{*2}}.$$ (17)

This is similar to the sample size equation used for the binary method, however σ is not derived in the standard way and δ* is obtained using latent means as opposed to provided directly.

4. Numerical Application

4.1. Muse trial

We illustrate the technique for sample size determination using the MUSE trial.³⁹ The trial was a phase IIb, randomized, double-blind, placebo-controlled study investigating the efficacy and safety of anifrolumab in adults with moderate to severe systemic lupus erythematosus (SLE). Patients (n=305) were randomized (1:1:1) to receive anifrolumab (300 or 1000 mg) or placebo, in addition to standard therapy every 4 weeks for 48 weeks. The primary endpoint in the study was the percentage of patients achieving an SLE Responder Index (SRI) response at week 24, with sustained reduction of oral corticosteroids (<10 mg/day and less than or equal to the dose at week 1 from week 12 through 24). SRI is comprised of a continuous Physician’s Global Assessment (PGA) measure, a continuous SLE Disease Activity Index (SLEDAI) measure and an ordinal British Isles Lupus Assessment Group (BILAG) measure.⁴² The study had a target sample size of 100 patients per group based on providing 88% power at the two-sided 0.10 alpha level, to detect at least 20% absolute improvement in SRI(4) response rate at week 24 for anifrolumab relative to placebo. The investigators assumed a 40% placebo response rate.

4.2. Model

In this case (Y₁, Y₂, Y₃, Y₄) are SLEDAI, PGA, BILAG and the corticosteroid tapering indicator respectively and ($Y_1,Y_2,Y_3^{*},Y_4^{*}$) ~ N₄ (${\mathit{\mu }}^{*},\Sigma$) where,

${\mathit{\mu }}^{*}={\left(\mathit{\mu }{}_1,{\mathit{\mu }}_2,{\mathit{\mu }}_3^{*},{\mathit{\mu }}_4^{*}\right)}^T\Sigma =\left(\begin{array}{cccc}{\sigma }_1^2& {\mathit{\rho }}_{12}{\sigma }_1{\sigma }_2& {\mathit{\rho }}_{13}{\sigma }_1& {\mathit{\rho }}_{14}{\sigma }_1\\ {\mathit{\rho }}_{12}{\sigma }_1{\sigma }_2& {\sigma }_2^2& {\mathit{\rho }}_{23}{\sigma }_2& {\mathit{\rho }}_{24}{\sigma }_2\\ {\mathit{\rho }}_{13}{\sigma }_1& {\mathit{\rho }}_{23}{\sigma }_2& 1& {\mathit{\rho }}_{34}\\ {\mathit{\rho }}_{14}{\sigma }_1& {\mathit{\rho }}_{24}{\sigma }_2& {\mathit{\rho }}_{34}& 1\end{array}\right),$ (18)

and the ordinal and binary components may be related to their latent variables as shown in (19). The thresholds (τ₃₁, τ₃₂, τ₃₃, τ₃₄) are estimated from the data.

$$\begin{array}{cc}{Y}_{i3}=\{\begin{array}{lll}0\hfill & \text{if}\hfill & \hfill -\infty <Y_{i3}^{*}<{\tau }_{31},\\ 1\hfill & \text{if}\hfill & \hfill {\tau }_{31}\le Y_{i3}^{*}<{\tau }_{32},\\ 2\hfill & \text{if}\hfill & \hfill {\tau }_{32}\le Y_{i3}^{*}<{\tau }_{33},\\ 3\hfill & \text{if}\hfill & \hfill {\tau }_{33}\le Y_{i3}^{*}<{\tau }_{34},\\ 4\hfill & \text{if}\hfill & \hfill {\tau }_{34}\le Y_{i3}^{*}<\infty ,\end{array}& Y_{i4}=\{\begin{array}{lll}0\hfill & \text{if}\hfill & -\infty <Y_{i4}^{*}<0,\hfill \\ 1\hfill & \text{if}\hfill & 0\le Y_{i4}^{*}<\infty \hfill \end{array}\end{array}$$ (19)

We can use the MUSE trial to design future studies where we assume that the endpoints of interest are co-primary, multiple primary and composite endpoints. The overall power functions for each are shown below.

$$\begin{array}{l}{\mathit{\text{Power}}}_{\mathit{\text{co}}}={\Phi }_4\left(-z_1^{†},-z_2^{†},-z_3^{†},-z_4^{†};\sum \right)\\ {\mathit{\text{Power}}}_{\mathit{\text{mult}}}={\displaystyle \sum _{i=1}^4\left({\left(-1\right)}^{i-1}{\displaystyle \sum _{I\subseteq \left\{1,2,3,4\right\}}{\Phi }_{k\in I}\left(-z_k^{†};\sum \right)}\right)}\\ {\mathit{\text{Power}}}_{\mathit{\text{comp}}}=\Phi \left(-z\right)\mathit{\text{,}}\end{array}$$

where $z_k^{†}=Z_{\alpha }-\frac{{\delta }_k}{\sqrt{2{\sigma }_k^2/n}}$ for k = {1, 2} and $z_k^{†}=Z_{\alpha }-\frac{{\delta }_k^{*}}{\sqrt{2/n}}$ for k = {3, 4}. In the composite setting $z=\frac{{\delta }^{*}}{\sqrt{2{\sigma }^2/n}}-z_{\alpha }$ where σ is estimated using the delta method. For the Power_mult calculation we apply the Bonferroni correction, such that each outcome is assessed at the $\frac{\alpha }{4}$ level.

4.3. Computation

We have conducted the computations in R version 4.0.2. We define functions to evaluate the power for each of the endpoints using a combination of the pnorm and pmvnorm functions. Sample size is obtained by inserting values for n until the desired power is achieved. Details of our source code and a web app for implementation is included in the Software section. Code to obtain the results shown in this article can be obtained at https://github.com/martinamcm/mcmenamin_2021_multsamp. Considerations and instructions for fitting the latent variable model are discussed in detail McMenamin et al. (2021).³⁸

4.4. Results

The power is largest for the multiple primary endpoint, where 80% is achieved for n=37 patients in each arm. The power for the composite endpoint is similar to that of PGA, the component with the highest effect size. As we would expect the power is considerably lower for co-primary endpoints, which would require n=325 for 80% power (Figure 1).

Figure 1.

Power function for individual SLEDAI (continuous), PGA (continuous), BILAG (ordinal), and Taper (binary) outcomes and the power functions with when they are treated as co-primary, multiple primary, and composite endpoints using data from the MUSE trial

Table 1 shows the sample sizes required in each group, for the co-primary and multiple primary endpoints to obtain an overall power of at least 80% to detect a difference of 0.88 in SLEDAI, 0.38 in PGA, 0.24 in BILAG and 0.40 in the taper outcome based on the values observed in the trial. We allow for uncertainty in the variance of the continuous measures by setting ${\sigma }_1^2=18,19,20$ and ${\sigma }_2^2=0.35,0.45,0.55,0.65.$ The sample sizes required for each individual endpoint are also shown, based on achieving a power of at least 80%. Allowing for uncertainty in the variance of the SLEDAI outcome varies the required sample size for the co-primary endpoint but not the multiple primary endpoint. The opposite is true when the assumed variance of the PGA outcome is changed, namely affecting the sample size required for the multiple primary endpoint but not the co-primary. This is intuitive given that the treatment effect observed in the SLEDAI outcome is smallest and is largest for the PGA outcome. For the co-primary and composite endpoints the power is largest when the correlation between the endpoints is high whereas for multiple primary endpoints the power is largest for zero correlation between endpoints (Figure 2).

Table 1.

Sample sizes n = n_C = n_T for the co-primary and multiple primary endpoints for overall power 1 − β ≈ 0.80, α = 0.025, k_m = 2, K = 4 using the MUSE trial data

SLEDAI		PGA		BILAG		Taper
δ 1	${\mathit{\sigma }}_1^2$	δ 2	${\mathit{\sigma }}_2^2$	(πT3,πC3)	${\mathit{\sigma }}_3^{*}$	(πT4,πC4)	${\mathit{\sigma }}_4^{*}$	nco	nmult	SS 1	SS 2	SS 3	SS 4
0.88	18	0.38	0.35	(0.97,0.95)	0.24	(0.54,0.38)	0.40	403	46	365	39	273	99
0.88	19	0.38	0.35	(0.97,0.95)	0.24	(0.54,0.38)	0.40	419	46	386	39	273	99
0.88	20	0.38	0.35	(0.97,0.95)	0.24	(0.54,0.38)	0.40	435	46	406	39	273	99
0.88	18	0.38	0.45	(0.97,0.95)	0.24	(0.54,0.38)	0.40	403	55	365	49	273	99
0.88	18	0.38	0.55	(0.97,0.95)	0.24	(0.54,0.38)	0.40	403	63	365	60	273	99
0.88	18	0.38	0.65	(0.97,0.95)	0.24	(0.54,0.38)	0.40	403	70	365	71	273	99

Note: SS₁, SS₂, SS₃, SS₄ are sample sizes required per group for the individual endpoints for a power of at least 1 − β = 0.80.

Figure 2.

Overall power 1 − β to detect the treatment effects assumed from the MUSE trial for the systemic lupus erythematosus co-primary, multiple primary, and composite endpoints for different sample sizes per group n = n_C = n_T and differing correlations between outcomes, where Low = 0.3, Medium = 0.5, and High = 0.8

We assume that a future trial in SLE is to be conducted using the composite responder endpoint, allowing for uncertainty in σ. The estimated variance for the risk difference from the trial dataset is ${\sigma }_{\delta }^2=0.048$ with correlation parameters ${\rho }_{12}=0.448,{\rho }_{13}=0.521,{\rho }_{14}=0.003,{\rho }_{23}=0.448,{\rho }_{24}=-0.031,{\rho }_{34}=\mathrm{0.066.}$ For a risk difference of 0.14, the required sample size per group is 50, compared to 135 for 88% power in the standard binary method. If the method were to be employed for increased power, rather than a decrease in required sample size, the estimated power of the latent variable method is over 99.99% for sample sizes giving 88% power at the 0.05 one-sided alpha level in the binary method. The empirical power is shown for the latent variable method in 1000 simulated datasets, which is approximately 88% for each sample size, as required. Note that the sample size for composite endpoints are highly dependent on the responder threshold chosen, which will be predefined by clinicians.

5. Empirical Performance Of Sample Sizes

The behavior of the sample sizes obtained for each of the endpoints can be shown empirically. Assuming the four dimensional SLE endpoint, we calculate the empirical power by simulating 100 000 datasets from the multivariate normal distribution and applying the corresponding tests for both the observed and latent continuous outcomes. The key concern for the co-primary endpoints is that the method gives the appropriate power whereas for multiple primary endpoints we must ensure the family-wise error rate is controlled.

The sample sizes required for each of the three endpoints and the corresponding empirical power is shown for effect sizes observed in the MUSE trial with low, medium, and high correlation assumed between endpoints (Table 2). The empirical power derived is approximately equal to the desired power of 80% for all endpoints. As is well recognized in the multiple testing literature, the type I error rate must be controlled when multiple primary endpoints are tested using the union-intersection test. The degree to which the type I error rate is inflated depends on the number of outcomes and the correlation between outcomes, where lower correlation between outcomes and larger number of outcomes result in larger inflations (Figure 3). The performance of the Bonferroni correction in this setting is shown, where it is conservative in the case of high correlation between endpoints. As the maximum correlation between outcomes in the MUSE trial endpoint used in the numerical example is 0.5, we expect the sample sizes shown for this application to be a good estimate. If very large positive correlations between the endpoints are expected the required sample size from this approach may be overestimated. The code to obtain these empirical results is provided in the ‘Software’ section.

Table 2.

Sample sizes and empirical power (%) for n = n_C = n_T for the co-primary, multiple primary, and composite endpoints for overall power 1 − β ≈ 0.80, α = 0.025, k_m = 2, K = 4 with observed and latent effect sizes ${\delta }_1,{\delta }_2,{\delta }_3^{*},{\delta }_4^{*}$ and correlation ρ equal to 0.3, 0.5, 0.8 where correlations are assumed to be equal between all endpoints

δ 1	δ 2	${\delta }_3^{*}$	${\delta }_4^{*}$	ρ	Co-primary	Multiple primary	Composite
0.12	0.12	0.12	0.12	0.0	1766 (80.1)	591 (80.0)	1031 (80.1)
				0.3	1692 (80.0)	744 (80.1)	883 (80.0)
				0.5	1617 (80.0)	867 (80.1)	687 (80.0)
				0.8	1439 (80.0)	1117 (80.0)	589 (79.9)
0.35	0.35	0.15	0.15	0.0	917 (79.9)	105 (80.1)	201 (79.9)
				0.3	894 (80.0)	122 (79.9)	156 (80.0)
				0.5	870 (80.0)	134 (80.0)	115 (80.1)
				0.8	815 (80.1)	153 (80.0)	92 (80.0)
0.12	0.35	0.55	0.10	0.0	1772 (80.2)	61 (80.3)	81 (80.0)
				0.3	1736 (80.2)	67 (80.5)	72 (80.1)
				0.5	1700 (80.0)	70 (79.9)	67 (80.2)
				0.8	1625 (80.2)	74 (80.6)	58 (80.1)

Figure 3.

Family-wise error rate (FWER) of the multiple primary endpoints shown both unadjusted and adjusted using the Bonferroni correction. FWERs are shown for K = (2, 3, 4) outcomes and correlations are constrained to be equal between all outcomes

6. Discussion

The work in this article demonstrated the various ways in which a latent variable framework may be employed for mixed continuous, ordinal, and binary outcomes. We illustrated sample size determination in the case of mixed continuous, ordinal, and binary co-primary outcomes. We extended this to allow for sample size determination in the case of mixed multiple primary endpoints and proposed a technique to estimate the sample size when using a latent variable model for the underlying structure of a mixed composite endpoint. For co-primary and multiple primary endpoints the resulting hypothesis is based on an intersection or union of the hypotheses for the individual outcomes and so is multivariate in nature. However, for composite responder endpoints the hypothesis of interest is stated in relation to the overall responder endpoint and so is univariate. Sample size estimation in this case can make use of the standard power and sample size functions but requires the distribution of the test statistic under the alternative hypothesis which we approximate using latent-level means and a Taylor series expansion.

We applied the methods to a numerical example based on a phase IIb study. For the correlation structure observed in the MUSE trial, the sample size required for the co-primary endpoint was greater than that required for the individual endpoint with the lowest effect size. Alternatively, the sample size required for the multiple primary endpoint changes based on the variance assumed for the outcome with the largest treatment effect, however is similar to that required by the individual endpoint. The sample size required for the composite endpoint was between that required for the individual outcome with the largest and second largest effect size. Given that in the composite case we are concerned with the overall binary response endpoint, we compared the sample sizes required for the endpoint using the latent variable model with the standard binary method which we showed offered a large gain in efficiency. Results of the simulated scenarios agree with previous findings that the inclusion of the ordinal component with five levels is only responsible for a very small proportion of the precision gains. Given that the inclusion of the ordinal component substantially increases complexity and computational demand, it may be sufficient to combine any ordinal components with the binary outcome if necessary. Detailed simulation results for the composite endpoint are shown in the Supplementary Material.

One practical consideration when calculating the sample size for a trial using the latent variable model is the need to specify a large number of parameters, even in the case of only a few outcomes. Estimates for the parameters could be obtained by fitting the model to pilot data however this is potentially challenging and restrictive for a number of reasons. First, it requires that a pilot or earlier phase trial must have already taken place. Furthermore, the pilot data could be fundamentally different to the future trial and observed effects may be imprecise. Therefore, placing too much emphasis on the existing data may lead to problems in the main trial. In theory, it is possible to specify the required covariance parameters without data however this would be difficult in practice. Additionally, in the case of composite endpoints, we cannot define the variance in terms of the model parameters only, as the treatment effect is defined for the one-dimensional composite and so is a function of the parameters. This means that the full covariance matrix of the estimated parameters is required for the Taylor series derivation. An alternative when there is no data available is to apply the method using the sample size required to achieve 80% power for the binary method and avail of the large increase in power. Alternatively, we can directly specify σ_δ based on expert elicitation, as is sometimes the case in practice for standard one-dimensional endpoints. Allowing for uncertainty in the quantities and choosing conservative values should provide an appropriate sample size estimate.

It is possible to extend this approach to use adaptive sample size re-estimation, or an internal pilot to allow for reductions in the required sample size in the trial as we collect more information about the treatment effect variability.

Software

The code to obtain the results in this article is available at https://github.com/martinamcm/mcmenamin_2021_multsamp. A Shiny application for implementing the method is available at https://martinamcm.shinyapps.io/multsampsize/. Documentation and example data are available at https://github.com/martinamcm/MultSampSize.