Testing a Primary and a Secondary Endpoint in a Group Sequential Design

Summary

We consider a clinical trial with a primary and a secondary endpoint where the secondary endpoint is tested only if the primary endpoint is significant. The trial uses a group sequential procedure with two stages. The familywise error rate (FWER) of falsely concluding significance on either endpoint is to be controlled at a nominal level α. The Type I error rate for the primary endpoint is controlled by choosing any α-level stopping boundary, e.g., the standard O’Brien-Fleming or the Pocock boundary. Given any particular α-level boundary for the primary endpoint, we study the problem of determining the boundary for the secondary endpoint to control the FWER. We study this FWER analytically and numerically and find that it is maximized when the correlation coefficient ρ between the two endpoints equals 1. For the four combinations consisting of O’Brien-Fleming and Pocock boundaries for the primary and secondary endpoints, the critical constants required to control the FWER are computed for different values of ρ. An ad-hoc boundary is proposed for the secondary endpoint to address a practical concern that may be at issue in some applications. Numerical studies indicate that the O’Brien-Fleming boundary for the primary endpoint and the Pocock boundary for the secondary endpoint generally gives the best primary as well as secondary power performance. The Pocock boundary may be replaced by the ad-hoc boundary for the secondary endpoint with a very little loss of secondary power if the practical concern is at issue. A clinical trial example is given to illustrate the methods.

1. Introduction

Since the pioneering works of Pocock (1977) and O’Brien and Fleming (1979), group sequential designs have been widely studied and are now commonly employed in clinical trials. Jennison and Turnbull (2000) is a comprehensive reference on this topic. Much of the work on group sequential designs deals with a single endpoint. Jennison and Turnbull (1993) discussed a group sequential design for bivariate endpoints (efficacy and safety). They determined two-sided decision boundaries for the test statistics computed for both endpoints so that if the new treatment is shown to be superior on both endpoints then it is accepted; if it shown to be inferior on one of the endpoints then it is rejected; otherwise sampling is continued to the next stage. We are interested in the problem of testing null hypotheses on hierarchically ordered endpoints, e.g., a primary and a secondary endpoint. For such endpoints it is generally required that statistical significance be shown on the primary endpoint before the secondary endpoint can be validly tested (O’Neill, 1997). Gatekeeping procedures (Dmitrienko and Tamhane, 2007; 2009) deal with such endpoints; however, these procedures have been studied only for nonsequential designs. The present paper studies the problem of testing a primary and a secondary endpoint in a two-stage group sequential setting with the former acting as a gatekeeper for the latter. This problem was originally studied by Hung, Wang and O’Neill (2007). While this paper was being revised, we received a manuscript by Glimm, Maurer and Bretz (2009) which addresses the same problem and reaches similar conclusions but does not give explicit analytical results concerning the FWER and the primary and secondary critical boundaries as we derive in this paper. These two works are completely independent and complementary to each other.

The outline of the paper is as follows. Section 2 describes the group sequential procedure (GSP) for the above problem. Section 3 discusses control of the FWER for this GSP. This section also gives a table of the critical constants using either the O’Brien-Fleming (OF) or the Pocock (PO) boundaries for the primary and secondary endpoints resulting in four different combinations. Two additional combinations that use the OF or the PO boundary for the primary and an ad-hoc (AH) boundary for the secondary endpoint are also studied. This ad-hoc boundary is introduced to address a practical concern that is explained in Section 3.2. The powers of these six different boundary combinations are compared in Section 4. Section 5 gives a clinical trial example. Section 6 outlines extensions for future research to deal with unknown correlation coefficient, multiple stages and multiple endpoints. The proofs of all theoretical results are included in supplementary materials under the Paper Information link at http://www.biometrics.tibs.org.

2. Group Sequential Procedure

Consider a GSP with two stages and a primary and a secondary endpoint. Let n₁ and n₂ be the sample sizes for the two stages. Suppose that the observations on the primary endpoint are i.i.d. $N({\mu }_1,{\sigma }_1^2)$ and those on the secondary endpoint are i.i.d. $N({\mu }_2,{\sigma }_2^2)$. Further suppose that the two endpoints are jointly distributed as bivariate normal with correlation coefficient ρ ≥ 0. Hung, Wang and O’Neill (2007) considered the problem of testing the null hypotheses H₁: μ₁ = 0 and H₂: μ₂ = 0 against one-sided alternatives with H₂ being tested iff H₁ is rejected either at the first stage or at the second stage. Thus the primary endpoint acts as a gatekeeper for the secondary endpoint. The FWER requirement is

$$\text{FWER}=P\{\text{Reject}\text{at}\text{least}\text{one}\text{true}{H}_i(i=1,2)\}\le \alpha$$ (1)

for specified α.

The standardized cumulative sample means at the two stages are used as the test statistics for both endpoints. Denote by (X₁, X₂) the cumulative test statistics at the two stages for the primary endpoint and (Y₁, Y₂) the corresponding test statistics for the secondary endpoint. Let (c₁, c₂) and (d₁, d₂) denote the corresponding stopping boundaries. We will use the following GSP.

Stage 1

Take n₁ observations and compute (X₁, Y₁). If X₁ ≤ c₁ continue to Stage 2. If X₁ > c₁, reject H₁ and test H₂. If Y₁ > d₁, reject H₂; otherwise accept H₂. In either case terminate the trial.

Stage 2

Take n₂ observations and compute (X₂, Y₂). If X₂ ≤ c₂, accept H₁ and stop testing; otherwise reject H₁ and test H₂. If Y₂ > d₂, reject H₂; otherwise accept H₂.

In general, X_j ~ N (Δ_1j, 1) and Y_j ~ N (Δ_2j, 1) for j = 1, 2, where

$${\mathrm{\Delta }}_{i1}={\delta }_i\sqrt{n_1},{\mathrm{\Delta }}_{i2}={\delta }_i\sqrt{n_1+n_2}\text{and}{\delta }_i=\frac{{\mu }_i}{{\sigma }_i};$$ (2)

under H_i, Δ_ij = 0 (i, j = 1, 2). Frequently, we will denote the noncentrality parameters by Δ_i = Δ_i1 = γΔ_i2 (i = 1, 2), where

$$\gamma =\sqrt{\frac{n_1}{n_1+n_2}}.$$

The correlations among (X₁, X₂, Y₁, Y₂) are given by

$$\begin{array}{l}\text{corr}(X_1,X_2)=\text{corr}(Y_1,Y_2)=\gamma \\ \text{corr}(X_1,Y_1)=\text{corr}(X_2,Y_2)=\rho \\ \text{corr}(X_1,Y_2)=\text{corr}(X_2,Y_1)=\rho \gamma .\end{array}$$ (3)

3. Familywise Error Rate Control

3.1 Choice of the Primary Boundary

To determine the critical boundaries, (c₁, c₂) and (d₁, d₂), in order to satisfy the FWER control requirement (1), we need to consider three configurations, namely, H₁ ∩ H₂, H₁ ∩ H̄₂ and H̄₁ ∩ H₂, where H̄_i denotes that H_i is false (i = 1, 2). If H₁ is true (the first two cases) then the FWER is controlled at level α if the type I error probability for H₁ is controlled at level α. This is obvious under H₁ ∩ H̄₂ since there is no type I error for rejecting H₂ in that case. It is also true under H₁ ∩ H₂ because the event R₂ = {Reject H₂} is a subset of the event R₁ = {Reject H₁} and the probability of R₁ does not depend on the truth or falsity of H₂. Hence, to control P_H1∩H2 (R₁ ∪ R₂) = P_H1(R₁) ≤ α, one must choose (c₁, c₂) to satisfy

$$P_{H_1}(X_1>c_1)+P_{H_1}(X_1\le c_1,X_2>c_2)\le \alpha .$$ (4)

Any pair of constants, (c₁, c₂), that satisfy this inequality will be referred to as an α-level boundary. For the OF boundary, c₂ = γc₁, while for the PO boundary, c₂ = c₁.

More generally, we could use the error spending function approach of Lan and DeMets (1983) to calculate the primary boundary (c₁, c₂). Let α(t) be a nondecreasing function defined over t ∈ [0, 1] such that α(0) = 0 and α(1) = α; here t is called the information fraction. Let t₁ = n₁/(n₁ + n₂) and t₂ = 1. Then (c₁, c₂) can be calculated from the following two equations:

$$P_{H_1}(X_1>c_1)=\alpha (t_1)\text{and}{P}_{H_1}(X_1>c_1)+P(X_1\le c_1,X_2>c_2)=\alpha (t_2)=\alpha .$$

For a K-stage GSP (K ≥ 2) with equal information in each stage, i.e., t_i = i/K (1 ≤ i ≤ K), an error spending function that approximates the OF boundary is given by (Lan and DeMets 1983)

$${\alpha }_1(t)=2\mathrm{\Phi }\left(-\frac{z_{\alpha /2}}{\sqrt{t}}\right),$$ (5)

where Φ(·) is the standard normal c.d.f. and z_α/2 is the (1−α/2)-quantile of the standard normal distribution. Similarly, an error spending function that approximates the PO boundary is given by (Lan and DeMets 1983)

$${\alpha }_2(t)=\alpha ln\{1+(e-1)t\}.$$ (6)

These spending functions are used in the example of Section 5.

In conclusion, the FWER control problem is: Given a choice of (c₁, c₂) satisfying (4), determine (d₁, d₂) so that (1) is satisfied under H₂. We will assume this configuration, which implies that Δ₁ ≥ 0 and Δ₂ = 0, throughout the next section.

3.2 Choice of the Secondary Boundary

To test multiple hypotheses in a group sequential setup, Tang and Geller (1999) showed that if there exists a GSP to test every intersection hypothesis at level α then application of the closure principle of Marcus, Peritz and Gabriel (1976) leads to a GSP that controls the FWER at level α. With H₁ and H₂ there are only three intersections: H₁ ∩ H₂, H₁ and H₂. As seen above, an α-level GSP of H₁ ∩ H₂ is the same as the GSP of H₁ because of the hierarchical testing of H₁ and H₂. Therefore, if we use any α-level secondary boundary for testing H₂ conditional on rejecting H₁, then the overall FWER will be controlled. However, it is possible to use a more liberal secondary boundary by exploiting the dependence of the FWER on the correlation coefficient ρ as calculations in Table 1 will show. The calculations were made using the following expression for FWER under H₂:

$$\begin{array}{l}\text{FWER}=P_{H_2}(X_1>c_1,Y_1>d_1)+P_{H_2}(X_1\le c_1,X_2>c_2,Y_2>d_2)\\ ={\int }_{c_1-{\mathrm{\Delta }}_{11}}^{\infty }\mathrm{\Phi }\left(\frac{-d_1+\rho u}{\sqrt{1-{\rho }^2}}\right)\phi (u)du+{\int }_{c_2-{\mathrm{\Delta }}_{12}}^{\infty }\mathrm{\Phi }\left(\frac{c_1-{\mathrm{\Delta }}_{11}-\gamma u}{\sqrt{1-{\gamma }^2}}\right)\mathrm{\Phi }\left(\frac{-d_2+\rho u}{\sqrt{1-{\rho }^2}}\right)\phi (u)du,\end{array}$$

where φ(·) denotes the p.d.f. of the standard normal distribution; also Δ₁₁ = γΔ₁₂ = Δ₁. This expression is obtained by setting Δ₂₁ = Δ₂₂ = 0 in the secondary power expression (8).

Table 1.

Values of Critical Constant d for Secondary Boundary Using Combinations of the O’Brien-Fleming (OF)¹, Pocock (PO)² and Ad-Hoc (AH)³ Boundaries for n₁ = n₂ and α = 0.05

Primary Boundary	Secondary Boundary	ρ
		0.0	0.2	0.4	0.6	0.8	1.0
OF1	OF2	1.407 (2.041)4	1.428 (1.773)	1.476 (1.478)	1.495 (1.150)	1.551 (0.767)	1.678 (0)
OF1	PO2	1.645 (∞)	1.663 (2.638)	1.686 (2.134)	1.717 (1.663)	1.760 (1.184)	1.876 (0.497)
OF1	AH2	1.645 (∞)	1.671 (2.505)	1.714 (.949)	1.786 (1.430)	1.926 (0.889)	2.515 (0.023)
PO1	PO2	1.645 (∞)	1.655 (2.679)	1.672 (2.080)	1.698 (1.528)	1.742 (0.952)	1.876 (0)
PO1	OF2	1.280 (2.169)	1.304 (1.874)	1.333 (1.553)	1.372 (1.202)	1.429 (0.809)	1.570 (0.216)
PO1	AH2	1.645 (∞)	1.656 (2.635)	1.679 (2.010)	1.720 (1.444)	1.801 (0.876)	2.092 (0.163)

¹OF Primary Boundary: $c_1=1.678\sqrt{2}$, c₂ = 1.678, OF Secondary Boundary: $d_1=d\sqrt{2}$, d₂ = d

²PO Primary Boundary: c₁ = c₂ = 1.876, PO Secondary Boundary: d₁ = d₂ = d

³AH Secondary Boundary: Tabled value is d₁ =, d₂ = z_.05 = 1.645

⁴The value in the parenthesis is the value of Δ₁ that maximizes FWER.

Hung et al. (2007) investigated the choice d₁ = d₂ = z_α (where z_α is the upper α critical point of the N(0, 1) distribution) as Strategy 1 on the basis that the primary and secondary endpoints are tested in a fixed sequence, and so H₂ can be tested at level α following rejection of H₁ at level α. We show in Proposition 1 that this choice does not control the FWER for all ρ and Δ₁.

Proposition 1

If d₁ = d₂ = z_α then for ρ = 0, FWER is increasing in Δ₁ and FWER → α as Δ₁ → ∞ (and thus FWER is controlled for all Δ₁). In fact, FWER → α as Δ₁ → ∞ for all ρ ≥ 0. However, for ρ = 1, max_Δ1 FWER > α and thus FWER is not controlled for all ρ and Δ₁.

To illustrate the results of Proposition 1, we considered the OF boundary for the primary endpoint with α = 0.05 and n₁ = n₂ = n which uses $(c_1,c_2)=(1.678\sqrt{2},1.678)$. Figure 1 shows the behavior of FWER as a function of ρ and Δ₁ when d₁ = d₂ = z_.05 = 1.645. We see that the maximum FWER with respect to ρ and Δ₁ is attained when ρ = 1 and Δ₁ is close to 1, and this maximum is approximately 0.08. On the other hand, when ρ = 0, FWER ≤ α for all Δ₁ ≥ 0. Furthermore, FWER → α as Δ₁ → ∞ for all ρ ≥ 0.

Figure 1.

FWER as a function of ρ and Δ₁ when $(c_1,c_2)=(1.678\sqrt{2},1.678)$, n₁ = n₂ and d₁ = d₂ = z_.05 = 1.645

As seen from Figure 1, the choice d₁ = d₂ = z_α can be liberal for some Δ₁ > 0 when ρ > 0.4. As an ad-hoc solution to this problem, Hung et al. (2007) suggested Strategy 3 which uses d₁ = d₂ = z_α/2; however, this choice can be shown to be too conservative. Instead of an ad-hoc solution, we can numerically search to find the max FWER with respect to Δ₁ and ρ for different choices of d₁ = d₂ = z_α^* such that max FWER = 0.05. Figure 2 shows that this is achieved when Δ₁ = 0, ρ = 1 and d₁ = d₂ = 1.876, which corresponds to α^* = 0.0303. Note that this is the PO boundary for α = 0.05. We will show this numerical result analytically in Proposition 3.

Figure 2.

FWER as a function of ρ and Δ₁ when $(c_1,c_2)=(1.678\sqrt{2},1.678)$, n₁ = n₂ and d₁ = d₂ = z_.0303 = 1.876

In Propositions 2–4 that follow, we study three different combinations of the α-level boundaries: c₁ = d₁, c₂ = d₂ in Proposition 2, c₁ > d₁, c₂ < d₂ in Proposition 3 and c₁ < d₁, c₂ > d₂ in Proposition 4. From Tang and Geller (1999) it follows that all three boundaries control the FWER. In the first two cases we show that max FWER = α, while in the third case we show that max FWER < α. Hence it is possible to use a more liberal (d₁, d₂) boundary with level α′ > α in the third case.

Proposition 2

If (c₁, c₂) = (d₁, d₂) is an α-level boundary for the primary and secondary endpoints then for ρ = 1, max_Δ1 FWER = α is attained at Δ₁ = 0 and for Δ₁ = 0, max_ρ FWER = α is attained at ρ = 1.

Note that this proposition does not show that the global maximum of FWER is attained at Δ₁ = 0 and ρ = 1 which we conjecture to be true but have been unable to prove. However, plots in Figure 3 of numerically evaluated FWER as functions of Δ₁ and ρ when (c₁, c₂) = (d₁, d₂) is the OF boundary for α = 0.05 show that the global maximum of the FWER with respect to ρ and Δ₁ is indeed attained at ρ = 1 and Δ₁ = 0, and this maximum equals 0.05. In fact, as seen from Figures 1 and 2, even when (c₁, c₂) ≠ (d₁, d₂), the global maximum of FWER with respect to ρ is attained at ρ = 1 and Δ₁ > 0.

Figure 3.

FWER as a function of ρ and Δ₁ when $(c_1,c_2)=(d_1,d_2)=(1.678\sqrt{2},1.678)$ and n₁ = n₂

Proposition 3

If (c₁, c₂) and (d₁, d₂) are α-level boundaries for the primary and secondary endpoints such that c₁ > d₁ and c₂ < d₂ (e.g., if (c₁, c₂) is the OF boundary and (d₁, d₂) is the PO boundary) then for ρ = 1, max_Δ1 FWER = α is attained when Δ₁ = Δ₁₁ = c₁ − d₁.

Proposition 4

If (c₁, c₂) and (d₁, d₂) are α-level boundaries for the primary and secondary endpoints such that c₁ < d₁ and c₂ > d₂ (e.g., if (c₁, c₂) is the PO boundary and (d₁, d₂) is the OF boundary) then for ρ = 1, max_Δ1 FWER < α is attained when Δ₁ = γΔ₁₂ = γ(c₂ − d₂). Therefore max FWER can be increased to α by decreasing (d₁, d₂) to ( $d_1^{\prime },d_2^{\prime }$) so that ( $d_1^{\prime },d_2^{\prime }$) forms an α′-level boundary with α′ > α.

Propositions 2–4 are illustrated by calculations in Table 1 of the secondary boundary (d₁, d₂) that controls the FWER at level α for the two choices of the α-level primary boundary (c₁, c₂) for α = 0.05, n₁ = n₂ = n and ρ = 0.0(0.2)1.0. For the OF boundary, $(c_1,c_2)=(1.678\sqrt{2},1.678)$ and for the PO boundary, (c₁, c₂) = (1.876, 1.876). The same two choices are used for the secondary stopping boundary in the following way:

$$\text{OF}\text{Boundary}:d_1=d\sqrt{2},d_2=d,\text{PO}\text{Boundary}:d_1=d_2=d.$$

Table 1 lists the d-values for the four combinations of the primary and secondary boundaries.

In addition, we considered another boundary to address a referee’s practical concern over having the secondary boundary with a level α′ > α, as shown in Proposition 4. Note that, although the marginal error rate for the secondary endpoint exceeds α, the joint FWER for the primary and secondary endpoints is controlled at α. However, the referee thought that this subtlety may be difficult to explain to clinicians, and in fact suggested that both d₁, d₂ should be ≥ z_α.

For the PO secondary boundary, (d₁ = d₂ = d), from Proposition 1 we have d = z_α for ρ = 0. From Figures 1–3 we see that max_Δ1 FWER is an increasing function of ρ and so d is an increasing function of ρ; therefore d ≥ z_α for all ρ ≥ 0, as can be checked from Table 1. Thus the OF1-PO2 and PO1-PO2 combinations satisfy the referee’s practical condition; as will be seen in the next section, the OF1-PO2 combination generally gives higher power for both the primary and the secondary endpoint, and is therefore preferred.

For the OF secondary boundary, ( $d_1=d\sqrt{2}$, d₂ = d), it is possible to have d₂ < z_α and still control FWER. If this occurs then we set d₂ = z_α and then find the smallest d₁ ≥ z_α such that max FWER = α is maintained. We refer to this modified boundary as an ad-hoc (AH) boundary. We computed this secondary boundary for two additional combinations with the OF and PO as the primary boundaries. Note that the we did not employ the AH boundary for the primary endpoint in our study. This gave a total of six combinations of the primary and secondary boundaries.

The following observations can be made from the numerical results in Table 1.

1. The value of Δ₁ that maximizes FWER for fixed ρ is a decreasing function of ρ.

2. If both primary and secondary boundaries are of the same type (e.g., both OF or both PO) then we get (c₁, c₂) = (d₁, d₂) and Δ₁ = 0, ρ = 1 gives the max FWER as shown in Proposition 2.

3. If the primary boundary is OF and the secondary boundary is PO so that c₁ > d₁ and c₂ < d₂ then for ρ = 1, ${\mathrm{\Delta }}_{11}=c_1-d_1=1.678\sqrt{2}-1.876=0.497$ gives max FWER as shown in Proposition 3. Note that in Figure 3 we found d₁ = d₂ = 1.876 by numerical search, but it could have been found directly by applying Proposition 3.

4. If the primary boundary is PO and the secondary boundary is OF so that c₁ < d₁ and c₂ > d₂ then for ρ = 1, the maximizing value of Δ₁ is given by $\gamma {\mathrm{\Delta }}_{12}=\gamma (c_2-d_2)=(1/\sqrt{2})(1.876-1.678)=0.140$. However, using (c₁, c₂) = (1.876, 1.876) and $(d_1,d_2)=(1.678\sqrt{2},1.678)$ gives FWER = 0.0386 < 0.05. So, as shown in Proposition 4, (d₁, d₂) can be decreased until max FWER increases to 0.05. The final boundary, $(d_1^{\prime },d_2^{\prime })=(1.570\sqrt{2},1.570)$, and the corresponding maximizing Δ₁ = 0.216 were found numerically.

4. Power

There are two powers of interest: (i) primary power (Power₁), the probability of rejecting a false H₁, is just the power of GSP for a single endpoint which has been studied previously in the literature, and (ii) secondary power (Power₂), the probability of rejecting a false H₂. Note that Power₁ ≥ Power₂ for the same Δ_ij (i, j = 1, 2). The overall power for rejecting H₁ and H₂ when both are false or only H₂ is false is just the secondary power.

Consider an arbitrary configuration with Δ_i ≥ 0 (i = 1, 2). The primary power is given by

$${\text{Power}}_1=P_{{\overline{H}}_1}(X_1>c_1)+P_{{\overline{H}}_1}(X_1\le c_1,X_2>c_2),$$

where X_j ~ N(Δ_1j, 1) (j = 1, 2) with corr(X₁, X₂) = γ. Noting that the conditional distribution of X₂ given U = X₁ − Δ₁₁ = u is N (Δ₁₂ + γu, 1 − γ²), we get

$${\text{Power}}_1=\mathrm{\Phi }(-c_1+{\mathrm{\Delta }}_{11})+{\int }_{-\infty }^{c_1-{\mathrm{\Delta }}_{11}}\mathrm{\Phi }\left(\frac{-c_2+{\mathrm{\Delta }}_{12}+\gamma u}{\sqrt{1-{\gamma }^2}}\right)\phi (u)du.$$ (7)

The secondary power is given by

$${\text{Power}}_2=P_{{\overline{H}}_2}(X_1>c_1,Y_1>d_1)+P_{{\overline{H}}_2}(X_1\le c_1,X_2>c_2,Y_2>d_2),$$

where X_j ~ N(Δ_1j, 1), Y_j ~ N(Δ_2j, 1) (j = 1, 2) have the correlation structure shown in (4). In the first term, the conditional distribution of Y₁ given U = X₁ − Δ₁₁ = u is N(Δ₂₁ + ρu, 1 − ρ²). Similarly, in the second term, the conditional distribution of (X₁, Y₂) given U = X₂ − Δ₁₂ = u is bivariate normal with mean vector (Δ₁₁ + γu, Δ₂₂ + ρu) and covariance matrix

$$\left[\begin{array}{cc}1& \rho \gamma \\ \rho \gamma & 1\end{array}\right]-\left[\begin{array}{c}\gamma \\ \rho \end{array}\right][\gamma ,\rho ]=\left[\begin{array}{cc}1-{\gamma }^2& 0\\ 0& 1-{\rho }^2\end{array}\right],$$

i.e., conditional on U = X₂ − Δ₁₂ = u, X₁ and Y₂ are independent N (Δ₁₁ + γu, 1 − γ²) and N(Δ₂₂ + ρu, 1 − ρ²), respectively. Using these results we get

$${\text{Power}}_2={\int }_{c_1-{\mathrm{\Delta }}_{11}}^{\infty }\mathrm{\Phi }\left(\frac{-d_1+{\mathrm{\Delta }}_{21}+\rho u}{\sqrt{1-{\rho }^2}}\right)\phi (u)du+{\int }_{c_2-{\mathrm{\Delta }}_{12}}^{\infty }\mathrm{\Phi }\left(\frac{c_1-{\mathrm{\Delta }}_{11}-\gamma u}{\sqrt{1-{\gamma }^2}}\right)\mathrm{\Phi }\left(\frac{-d_2+{\mathrm{\Delta }}_{22}+\rho u}{\sqrt{1-{\rho }^2}}\right)\phi (u)du.$$ (8)

We studied the primary and secondary powers for the six combinations of the boundaries for which Table 1 is given. The powers were numerically evaluated (not simulated) using (7) and (8). It can be shown that the OF boundary is uniformly (in Δ₁) more powerful in terms of primary power than the PO boundary. Jennison (2009) has proved this result more generally for any two α-level boundaries, (c₁, c₂) and ( $c_1^{\prime },c_2^{\prime }$): if the test statistics have the MLR distribution property then (c₁, c₂) is uniformly more powerful than ( $c_1^{\prime },c_2^{\prime }$) if $c_1>c_1^{\prime },c_2<c_2^{\prime }$.

The secondary power is more complicated to study since it depends on the particular combination of the boundaries used for the primary and secondary endpoints as well as the values of ρ and Δ_ij. The plots of secondary powers as functions of Δ₁ for the six combinations of the boundaries studied in Table 1 for α = 0.05, n₁ = n₂, ρ = 0.4 and Δ₂ = 1, 2, 3 are given in Figures 4–6; the patterns of the power functions are similar for other values of ρ and hence are not shown here. These figures show that generally the combination of the OF boundary for the primary endpoint and the PO boundary for the secondary endpoint (the OF1-PO2 combination) gives the highest secondary power. With the OF1-PO2 combination, we have d₁, d₂ ≥ 1.645, so the practical concern raised above is not an issue. The OF1-AH2 has secondary power very similar to that of the OF1-PO2 combination, because their critical boundaries are similar. The reason that the secondary powers of the OF1-PO2 and OF1-AH2 combinations are close is because their critical boundaries are close. For ρ = 0.4, the OF1-PO2 boundaries are ( $c_1=1.678\sqrt{2}$, c₂ = 1.678) and (d₁ = 1.686, d₂ = 1.686) while the OF1-AH2 boundaries are ( $c_1=1.678\sqrt{2}$, c₂ = 1.678) and (d₁ = 1.714, d₂ = 1.645).

Figure 4.

Secondary powers of the six boundary combinations (OF = O’Brien-Fleming, PO = Pocock, AH = Ad-Hoc, 1 = Primary, 2 = Secondary) for the primary and secondary endpoints as functions of Δ₁ for ρ = 0.4 and Δ₂ = 1 when α = 0.05 and n₁ = n₂

Figure 6.

Secondary powers of the six boundary combinations (OF = O’Brien-Fleming, PO = Pocock, AH = Ad-Hoc, 1 = Primary, 2 = Secondary) for the primary and secondary endpoints as functions of Δ₁ for ρ = 0.4 and Δ₂ = 3 when α = 0.05 and n₁ = n₂

The power curves show that, for large Δ₁, any combination using OF2 is markedly worse than those using either PO2 or AH2. This is explained by the fact that when Δ₁ is large, there is a high chance of rejecting H₁ at the first look; so OF2 presents a higher threshold for the secondary endpoint to exceed at that look. Also note that for large Δ₁, the secondary power of OF1-OF2 is lower than that of PO1-OF2, which in turn is uniformly dominated by OF1-PO2 for all Δ₁ as well as the three values of Δ₂ that were studied. Thus PO1-OF2 is not a contender. OF1-OF2 combination has a slight edge for small Δ₁. However, if OF1-OF2 is disqualified based on the referee’s practical concern then OF1-AH2 may be preferred for small Δ₁ as well.

In looking at these plots, one may be intrigued by the fact that the secondary power of each combination of boundaries decreases with Δ₁ for large Δ₁ (Δ₁ ≥ 2) for fixed Δ₂. The reason is that the probability of rejecting H₁ at the first stage increases as Δ₁ increases. Therefore, for large Δ₁, H₂ gets only one chance to be tested at Stage 1 and so the secondary power decreases as Δ₁ increases. approaching P (Y₁ > d₁) as Δ₁ → ∞. On the other hand, for small Δ₁, H₂ gets two chances to be tested; therefore the secondary power increases as Δ₁ increases.

5. Example

To illustrate the methods discussed in this paper we consider the CAPTURE study (The Capture investigators, 1997) in which a randomized trial was conducted to compare abciximab with placebo for coronary intervention in refractory unstable angina. It was planned to enroll 1400 patients to placebo and abciximab in equal proportions. The primary endpoint was a composite of death, myocardial infarction or urgent intervention for treatment of recurrent ischemia, within 30 days of randomization. From previous studies in this patient population the placebo event rate was estimated to be 15%. The investigators wished to provide good power to detect a 5% reduction in the event rate for the abciximab arm. With a sample size of 1400, a single-look design has 81% power to detect this treatment effect, using a one-sided test at the α = 0.025 level of significance. The trial was, however, designed for a group sequential test with three equally spaced looks and OF-type stopping boundaries derived from the error spending function (5). Due to the possibility of early stopping, there is a power loss of about 1% with the group sequential design relative to the corresponding fixed sample design. For illustrative purposes we will ignore the possibility of early stopping at the first look and consider this to be a two-look design. This is a reasonable approximation since the above spending function yields an extremely conservative stopping boundary at the first look with a negligible chance of early stopping even under the alternative hypothesis of a 5% drop in the event rate.

The main secondary endpoint for this trial was death or myocardial infarction within 30 days of randomization. Although in the actual trial there was no formal testing strategy to protect the FWER of the two endpoints, we will consider two such strategies here. Under Strategy 1, we prespecify in the protocol that the secondary endpoint will be tested only if the null hypothesis for the primary endpoint is rejected; moreover, it will be tested at the same look and with the same spending function as the primary endpoint. This may be considered a conservative strategy since the secondary endpoint will face the same conservative OF-type stopping boundary as the primary endpoint. Suppose, however, that abciximab is expected to reduce the risk for the secondary endpoint to a lesser degree than that for the primary endpoint. It might then be advantageous to consider a more aggressive Strategy 2 in which the Type I error for the secondary endpoint is spent at a faster rate, resulting in a less demanding early stopping boundary at the first look. For example, one might prespecify the use of PO-type stopping boundaries derived from the error spending function (6) for the secondary endpoint.

Let π_1e and π_1c denote the true event rates for the experimental (abciximab) and control (placebo) arms, respectively, with respect to the primary endpoint. Let π_2e and π_2c be the corresponding true event rates for the secondary endpoint. Define μ_i = π_ic − π_ie, i = 1, 2. Let n₁ and n₂ be the sample sizes per arm at the two stages and N = 2(n₁ + n₂) be the planned maximum total sample size. The Wald statistics for the primary and secondary endpoints at the interim look are given by

$$X_1=\frac{({\widehat{\pi }}_{1c}^{(1)}-{\widehat{\pi }}_{1e}^{(1)})}{{\widehat{\sigma }}_1^{(1)}}\sqrt{\frac{n_1}{2}}\text{and}{Y}_1=\frac{({\widehat{\pi }}_{2c}^{(1)}-{\widehat{\pi }}_{2e}^{(1)})}{{\widehat{\sigma }}_2^{(1)}}\sqrt{\frac{n_1}{2}}$$

where ( ${\widehat{\pi }}_{ic}^{(1)},{\widehat{\pi }}_{ie}^{(1)}$) denotes the maximum likelihood estimate of (π_ic, π_ie) at the interim look and

$${\widehat{\sigma }}_i^{(1)}=\sqrt{{\widehat{\pi }}_{ic}^{(1)}(1-{\widehat{\pi }}_{ic}^{(1)})+{\widehat{\pi }}_{ie}^{(1)}(1-{\widehat{\pi }}_{ie}^{(1)})}(i=1,2).$$

The Wald statistics for the primary and secondary endpoint at the final look are given by

$$X_2=\frac{({\widehat{\pi }}_{1c}^{(2)}-{\widehat{\pi }}_{1e}^{(2)})}{{\widehat{\sigma }}_1^{(2)}}\sqrt{\frac{n_1+n_2}{2}}\text{and}{Y}_2=\frac{({\widehat{\pi }}_{2c}^{(2)}-{\widehat{\pi }}_{2e}^{(2)})}{{\widehat{\sigma }}_2^{(2)}}\sqrt{\frac{n_1+n_2}{2}}$$

where ( ${\widehat{\pi }}_{ic}^{(2)},{\widehat{\pi }}_{ie}^{(2)}$) denotes the maximum likelihood estimate of (π_ic, π_ie) at the final look and

$${\widehat{\sigma }}_i^{(2)}=\sqrt{{\widehat{\pi }}_{ic}^{(2)}(1-{\widehat{\pi }}_{ic}^{(2)})+{\widehat{\pi }}_{ie}^{(2)}(1-{\widehat{\pi }}_{ie}^{(2)})}(i=1,2).$$

Applying the large sample results of Jennison and Turnbull (2000, Chapter 3), the above sequentially computed Wald statistics have independent increments, so that (X₁, X₂, Y₁, Y₂) have the same distributional properties as the corresponding random variables defined in Section 2. We may thus avail of the results presented in Section 3 for controlling the FWER. No data are available to estimate the correlation coefficient ρ, so we will assume the least favorable configuration ρ = 1.

In the CAPTURE study an interim analysis was performed after data were available on 1050 subjects out of a planned 1400 subjects. Thus the relevant information fractions are t₁ = 1050/1400 = 0.75, and t₂ = 1400/1400 = 1. We compute the stopping boundary (c₁; c₂) from the spending function (5) by solving

$$P_{H_1}(X_1>c_1)={\alpha }_1(t_1)\text{and}{P}_{H_1}(X_1>c_1)+P_{H_1}(X_1\le c_1,X_2>c_2)=\alpha .$$

For Strategy 1 we set (c₁, c₂) = (d₁, d₂). For α = 0.025, we obtain (c₁, c₂) = (2.340, 2.012). (The classical OF boundary would be (c₁, c₂) = (2.327, 2.015)).

For Strategy 2 we compute the stopping boundary (d₁, d₂) from the spending function (6) by solving

$$P_{H_2}(Y_1\ge d_1)={\alpha }_2(t_1)\text{and}{P}_{H_2}(Y_1>d_1)+P_{H_2}(Y_1\le d_1,Y_2>d_2)=\alpha .$$

For α = 0.025, we obtain (d₁, d₂) = (2.040, 2.258) with the same (c₁, c₂) as above. (The classical PO boundary would be (d₁, d₂) = (2.126, 2.126)).

The following event rates (presented by K. Anderson at the 2002 Conference on Clinical Trial Data Monitoring Committees, Barnett International) were obtained for the primary endpoint at the time of the interim analysis: 84/532 (15.8%) for the placebo arm and 55/518 (10.6%) for the abciximab arm. Although no data were available for the secondary endpoint, we extrapolated from the published results of Simoons et al. (1997) to give 44/532 (8.3%) for the placebo arm and 26/518 (5%) for the abciximab arm. The test statistics are displayed in Table 2 and the stopping boundaries are displayed in Table 3.

Table 2.

Interim Results and Test Statistics

Endpoint	Event Rates		Interim Statistic
	Placebo	Abciximab
Primary	84/532 (15.8%)	55/518 (10.6%)	X1 = 2.485
Secondary	44/532 (8.3%)	26/518 (5.0%)	Y1 = 2.123

Table 3.

Critical Boundaries for Two Strategies

Look	Strategy 1		Strategy 2
	Primary: αOF(t)	Secondary: αOF(t)	Primary: αOF(t)	Secondary: αPO(t)
Interim	c1 = 2.340	d1 = 2.340	c1 = 2.340	d1 = 2.040
Final	c2 = 2.012	d2 = 2.012	c2 = 2.012	d2 = 2.258

The trial was terminated at the interim look since the test statistic X₁ for the primary endpoint crossed its early stopping boundary c₁. In order to formally claim statistical significance in the product label for the secondary endpoint, the test statistic Y₁ would have to cross its early stopping boundary d₁. It is seen that the secondary statistic does not cross its early stopping boundary under Strategy 1, but it does so under Strategy 2. Thus, depending on what was prespecified in the protocol, the investigators might or might not be allowed to claim statistical significance for the secondary endpoint.

6. Extensions

The scope of the present work is somewhat limited. First, we have considered only one-sided tests on two endpoints with two stages. Second, the data on the primary and secondary endpoints are assumed to be bivariate normal with known standard deviations and a nonnegative correlation coefficient that is either assumed to be known or treated as a nuisance parameter with the FWER being maximized with respect to it to determine the critical boundaries for the endpoints. These limitations are in part dictated by the analytical and computational difficulties involved. Given the practical importance of this problem, we expect that the present work will serve as a springboard for further extensions some of which are outlined below.

6.1 Unknown Correlation

The plots of FWER in Figures 4–6 and the critical constants tabulated in Table 1 show that max_Δ1 FWER is an increasing function of ρ with ρ = 1 being the least favorable case. If an upper bound on ρ can be specified then it can be used to find a more powerful boundary for the secondary endpoint than the conservative boundary corresponding to ρ = 1. A practical approach to this problem would be to use the upper limit of a one-sided confidence interval on ρ estimated from the data after Stage 1 to find the Stage 2 boundary with a suitable adjustment to the required FWER to take into account the error probability of the true ρ exceeding the upper confidence limit. The ideas from Berger and Boos (1994) can be used in this context as follows.

Let ρ^* be a (1 − ε)-level upper confidence limit on ρ (e.g., calculated using Fisher’s large sample arctan hyperbolic transformation), i.e., P(ρ ≤ ρ^*) = 1 − ε. Then, using the property that max_Δ1 FWER(Δ₁, ρ) is an increasing function of ρ, the overall maximum FWER can be written as follows. Let ${\mathrm{\Delta }}_1^{\ast }(\rho )$ be the value of Δ₁ that maximizes FWER(Δ₁, ρ) for fixed ρ. Then

$$\begin{array}{l}\underset{\{{\mathrm{\Delta }}_1,\rho \}}{max}\text{FWER}({\mathrm{\Delta }}_{1,}\rho )\\ =\underset{\{\rho \le {\rho }^{\ast }\}}{max}\text{FWER}({\mathrm{\Delta }}_1^{\ast }(\rho ),\rho )\times P(\rho \le {\rho }^{\ast })+\underset{\{\rho >{\rho }^{\ast }\}}{max}\text{FWER}({\mathrm{\Delta }}_1^{\ast }(\rho ),\rho )\times P(\rho >{\rho }^{\ast })\\ =\text{FWER}({\mathrm{\Delta }}_1^{\ast }({\rho }^{\ast }),{\rho }^{\ast })\times (1-\epsilon )+\text{FWER}({\mathrm{\Delta }}_1^{\ast }(1),1)\times \epsilon .\end{array}$$

We want to determine the smallest possible (d₁, d₂) (so as to maximize the power) subject to the above expression being ≤ α. This problem can be solved iteratively on a computer. For illustration, suppose we choose the PO boundary for the secondary endpoint, i.e., d₁ = d₂ = d. For fixed d we can evaluate $\text{FWER}({\mathrm{\Delta }}_1^{\ast }({\rho }^{\ast }),{\rho }^{\ast })={\alpha }^{\prime }(d)$ (say) and $\text{FWER}({\mathrm{\Delta }}_1^{\ast }(1),1)={\alpha }^{″}(d)$ (say) where note that α′(d) < α″(d). Then by numerical search we can find the smallest d such that α′(d)(1 − ε) + α″(d)ε = α. In this case also one may want to restrict d₁, d₂ ≥ z_α for practical reasons.

6.2 Multiple Stages

In this paper we have restricted to a GSP with two stages. However, most practical applications involve more than two stages. Restricting still to two endpoints but K ≥ 2 stages, denote the cumulative standardized test statistics for the primary and secondary endpoints at the kth stage by X_k and Y_k, respectively. The FWER control problem is to determine the critical boundaries (c₁, c₂, …, c_K) and (d₁, d₂, …, d_K) such that the FWER is strongly controlled at α for the decision rule that continues sampling as long as X_k ≤ c_k for k ≤ K. If X_k > c_k then H₁ is rejected and H₂ is tested. H₂ is rejected if Y_k > d_k and accepted if Y_k ≤ d_k; sampling terminates in either case or if k = K. If (c₁, c₂, …, c_K) and (d₁, d₂, …, d_K) are α-level boundaries then from Tang and Geller (1999) it follows that the FWER will be controlled at level α. However, a more detailed analysis can be carried out to sharpen the secondary boundary as a function of ρ as was done for K = 2 stages. It would be useful to develop a software that would compute these boundaries for any choice of the primary boundary, e.g., from the class of boundaries based on error spending functions of Lan and DeMets (1983) and Wang and Tsiatis (1987).

6.3 Multiple Endpoints

Multiple endpoints are of less practical interest than multiple stages, but it may be of interest to study three ordered endpoints (primary, secondary and tertiary). Denote the test statistics for the three endpoints at the kth stage by (X_k, Y_k, Z_k) and let (c₁, c₂, …, c_K), (d₁, d₂, …, d_K) and (e₁, e₂, …, e_K) be the critical boundaries for the three endpoints. Let H₁, H₂ and H₃ be the null hypotheses for the three endpoints. One may consider the following decision rule: Continue sampling as long as X_k ≤ c_k for k ≤ K. If X_k > c_k then reject H₁ and test H₂. If Y_k > d_k then reject H₂; otherwise terminate sampling without testing H₃. If H₂ is rejected then test H₃; reject H₃ if Z_k > e_k; otherwise accept H₃. Terminate sampling in either case or if k = K. It is easy to show that if all three boundaries are α-level then the FWER will be controlled. A challenging problem is how the knowledge (or estimates) of the correlation coefficients between the three endpoints can be used to sharpen the boundaries for the secondary and tertiary endpoints. This problem is complicated by the fact that there are three correlation coefficients.

Finally we mention another extension for K = 2 in which one may continue sampling to the second stage and test H₂ if H₁ is rejected but H₂ is not rejected at the first stage, effectively offering a second chance for the secondary endpoint to be tested.

Figure 5.

Secondary powers of the six boundary combinations (OF = O’Brien-Fleming, PO = Pocock, AH = Ad-Hoc, 1 = Primary, 2 = Secondary) for the primary and secondary endpoints as functions of Δ₁ for ρ = 0.4 and Δ₂ = 2 when α = 0.05 and n₁ = n₂