Exact p-values for Simon's two-stage designs in clinical trials

Abstract

In a one-sided hypothesis testing problem in clinical trials, the monotonic condition of a tail probability function is fundamentally important to guarantee that the actual type I and II error rates occur at the boundary of their associated parameter spaces. Otherwise, one has to search for the actual rates over the complete parameter space, which could be very computationally intensive. This important property has been extensively studied in traditional one-stage study settings (e.g., non-inferiority or superiority between two binomial proportions), but there is very limited research for this property in a two-stage design setting, e.g., Simon’s two-stage design. In this note, we theoretically prove that the tail probability is an increasing function of the parameter in Simon’s two-stage design. This proof not only provides theoretical justification that p-value occurs at the boundary of the parameter space, but also helps to reduce the computational intensity for study design search.

1 Introduction

Two-stage designs are widely used in practice to protect participants when the investigated treatment is indeed ineffective. When the outcome is binary, Simon’s optimal two-stage designs(Simon, 1989) are the most popular designs that have been frequently used in research, such as cancer studies (Siefker-Radtke et al., 2013), AIDS research studies (Zheng et al., 2012) and gastroesophageal research studies (Katz, Gerson, & Vela, 2013).

In these studies, the new treatment is tested by conducting a one-arm study, and its activity is confirmed when a large number of responses is observed. The associated hypotheses are presented as

$$H_0:p\le p_0,\text{ against }{H}_a:p\ge p_1,$$

where p₀ and p₁ represent the unacceptable and acceptable response rates for the new treatment, respectively. The acceptable response rate, p₁, is the target response rate that the new treatment could reach, and p₀ can be estimated from historical data. In such studies, a treatment with a high response rate is often preferable; thus p₁ > p₀ is assumed.

In a two-stage design, suppose X_i is the number of responses from the i–th stage, i = 1 and 2. Then, the tail area is defined as the collection of samples with a larger X₁ and a larger X₁ + X₂ as compared to the observed sample. The tail probability is calculated as the sum of probabilities for all sample points in the tail area. Since X₁ and X₂ follow binomial distributions with the success rate of p, the tail probability is a function of the parameter p. By the definition of p-value, it should be computed as the worst case scenario of the probability over the null space: {p : p ≤ p₀}. Due to the complexity of the tail probability in a two-stage setting, it is difficult to show that the tail probability is an increasing function of p. In practice, some researchers assume that the maximum of the tail probability occurs at the boundary without any theoretical proof, and others simply change the one-sided hypothesis to a simple hypothesis to test the response rate at two values: p₀ and p₁. Obviously, these approaches to simplify the problem are not suitable to make proper statistical inference for a study in a two-stage setting.

In this work, we are going to prove the monotonic condition of the tail probability in a two-stage setting: the tail probability is an increasing function of the parameter p. The monotonic condition in a one-stage setting has been studied extensively for binomial distributions (Röhmel, 2005; Röhmel & Mansmann, 1999; Shan & Ma, 2016). Recently, Shan, Chen, and Ma (2016) partially prove this condition in a two-stage setting.

This research is motived by the work by Tsai, Chi, and Chen (2008) who developed a new exact confidence interval for the response rate in a two-stage design by using the Clopper-Pearson exact approach (Clopper & Pearson, 1934). This exact approach has to be used in conjunction with a method to order the sample space. They ordered the sample space by the sufficient test statistic X_t = X₁ + X₂, which is the total number of responses (Jung & Kim, 2004). They showed that the cumulative distribution function of X_t is a decreasing function of p. This property must be satisfied in order to construct the exact interval based on the Clopper-Pearson approach (Tsai et al., 2008). They have only one random variable in the problem, but we have two random variables, X₁ and X_t in the p-value calculation. The rest of this note is organized as follows. In Section 2, we prove the monotonic condition of the tail probability in a two-stage setting by using mathematical induction. Section 3 provides some discussion and remarks.

2 Main results

In Simon’s two-stage design, four design parameters need to be determined (R₁, n₁, R, n₂) given α, β, p₀ and p₁. It should be noted that the design parameters are derived from the exact probability calculation for the actual type I and II error rates by using binomial distributions. R₁ and R are the boundary values for the number of responses using n₁ and n = n₁ + n₂ subjects from the first stage and both stages combined, respectively. If X₁ < R₁, then a study will be terminated due to non-sufficient activity of the treatment. Otherwise, an additional n₂ subjects will be considered in stage two. The final conclusion will be made by comparing the total number of responses X_t and R. A new treatment is considered promising for further investigation when X_t ≥ R.

The null hypothesis will be rejected when X₁ ≥ R₁ and X_t ≥ R, and the rejection region is presented as

$$\mathrm{\Phi }=\{(X_1,X_2):X_1\ge R_1,(X_1+X_2)\ge R\}.$$

It follows that the actual type I error (TIE) is defined as the maximum of the tail probability, P(R₁, n₁, R, n₂|p) = P((X₁, X₂) ∈ Φ|p) over the null space {p : p ≤ p₀},

$$\underset{p\in H_0}{\text{max }}P(R_1,n_1,R,n_2|p).$$ (1)

The tail probability P(R₁, n₁, R, n₂|p) can be calculated exactly by using binomial distributions as

$$P(R_1,n_1,R,n_2|p)=\sum _{X_1=R_1}^{\text{min}(n_1,R-1)}b(X_1,n_1,p)[1-B(R-X_1-1,n_2,p)]+\sum _{X_1=\text{min}(n_1,R-1)+1}^{n_1}b(X_1,n_1,p),$$

where b(․) and B(․) are the probability density function and cumulative distribution function of a binomial distribution, respectively.

Due to the intense computation of the TIE in Equation (1), it is usually assumed that the actual TIE rate is obtained at the boundary of the null space H₀, p₀. This note fills the gap to prove the monotonic condition of the tail probability, P(R₁, n₁, R, n₂|p), for Simon’s two-stage design.

Theorem 2.1

The monotonic condition of the tail probability in a two-stage setting, P(R₁, n₁, R, n₂|p), is satisfied for any given critical values R₁ and R where 0 ≤ R₁ ≤ n₁ and R₁ ≤ R ≤ n₁ + n₂. In other words, P (X₁ ≥ R₁, (X₁ + X₂) ≥ R|p) is an increasing function of p.

Proof

The number of responses from the second stage, X₂, follows a binomial distribution with parameters (n₂, p). Then, X₂ can be expressed as X₂ = B₁ + B₂ + ⋯ + B_n2, where B_i follows a Bernoulli distribution with the success rate of p, i = 1, 2, ⋯, n₂, and these$B_i^{\prime }\mathrm{s}$ are independent from each other. Therefore, the target function can be rewritten as

$$P(X_1\ge R_1,(X_1+X_2)\ge R|p)=P(X_1\ge R_1,(X_1+B_1+B_2+\cdots +B_{n2})\ge R|p)$$

We use mathematical induction (Tsai et al., 2008) for this proof. The first step is to prove that P(X₁ ≥ R₁, (X₁ + B₁) ≥ R|p) is an increasing function of p when n₂ = 1. There are only two possible outcomes of B₁, 1 with the probability of p, and 0 with the probability of 1 − p. We have

$$P(X_1\ge R_1,(X_1+B_1)\ge R)=P(X_1\ge R_1)\times P(B_1=0)+P(X_1\ge R_1,X_1\ge R-1)\times P(B_1=1)=P(X_1\ge R_1)\times (1-p)+P(X_1\ge \text{max}(R_1,R-1))\times p$$

Based on the above equation and the constraint R ≥ R₁, we consider two cases according to R = R₁ and R ≥ R₁ + 1. In the first case with R = R₁, the tail probability can be written as follows,

$$P(X_1\ge R_1,(X_1+B_1)\ge R)=P(X_1\ge R)\times (1-p)+P(X_1\ge R)\times p=P(X_1\ge R).$$

Its derivative is calculated as

$$\frac{\partial P(X_1\ge R)}{\partial p}=\left(\begin{array}{c}\hfill n_1\hfill \\ \hfill R-1\hfill \end{array}\right)(n_1-R+1)p^{R-1}{(1-p)}^{n_1-R}.$$

Since R = R₁ ≤ n₁, (n₁ − R+ 1) is always positive. The binomial coefficient $\left(\begin{array}{c}\hfill n_1\hfill \\ \hfill R-1\hfill \end{array}\right)$ is positive when R ≥ 1, and zero when R = 0. It follows that the derivative is positive when R ≥ 1 and zero when R = 0. In practice, a study design with R₁ = R = 0 is not realistic because any study under these critical values is guaranteed to be successful regardless the number of responses observed. After excluding this unrealistic case, the derivative is always positive, then P(X₁ ≥ R₁, (X₁ + B₁) ≥ R) is an increasing function of p.

In the second case with R ≥ R₁ + 1, we have

$$P(X_1\ge R_1,(X_1+B_1)\ge R)=P(X_1\ge R)\times (1-p)+P(X_1\ge R-1)\times p.$$

In the Appendix C by Tsai et al. (Tsai et al., 2008), they showed that P(X₁ ≥ R) × (1 − p) + P(X₁ ≥ R − 1) × p is an increasing function of p by using the relationship between binomial distribution function and incomplete beta function. Therefore, P(X₁ ≥ R₁, (X₁ + B₁) ≥ R) is an increasing function of p.

In the next step of mathematical induction, we assume that P(X₁ ≥ R₁,(X₁ + B₁ + B₂ + ⋯ + B_k) ≥ R) is an increasing function of p, where R ≥ R₁, and we need to check the monotonic property of P(X₁ ≥ R₁,(X₁ + B₁ + B₂ + ⋯ + B_k + B_k+1) ≥ R). It is easy to show that

$$P(X_1\ge R_1,(X_1+B_1+\cdots +B_k+B_{k+1})\ge R)=P(X_1\ge R_1,(X_1+B_1+·+B_k)\ge R)\times (1-p)+P(X_1\ge R_1,(X_1+B_1+·+B_k)\ge R-1)\times p.$$

Its derivative is

$$\frac{\partial P(X_1\ge R_1,(X_1+B_1+\cdots +B_k+B_{k+1})\ge R)}{\partial p}=\frac{\partial P(X_1\ge R_1,(X_1+B_1+·+B_k)\ge R)}{\partial p}(1-p)-P(X_1\ge R_1,(X_1+B_1+·+B_k)\ge R)+\frac{\partial P(X_1\ge R_1,(X_1+B_1+·+B_k)\ge R-1)}{\partial p}+P(X_1\ge R_1,(X_1+B_1+·+B_k)\ge R-1)$$

It is obvious that P(X₁ ≥ R₁, (X₁ + B₁ + · + B_k) ≥ R − 1) − P(X₁ ≥ R₁, (X₁ + B₁ + · + B_k) ≥ R) = P(X₁ ≥ R₁,(X₁ + B₁ + · + B_k) = R − 1) ≥ 0. By the assumption that P(X₁ ≥ R₁, (X₁ + B₁ + B₂ + … + B_k) ≥ R) is an increasing function of p, we know that $\frac{\partial P(X_1\ge R_1,(X_1+B_1+·+B_k)\ge R)}{\partial p}>0$. For $\frac{\partial P(X_1\ge R_1,(X_1+B_1+·+B_k)\ge R-1)}{\partial p}$, we need to consider two cases: R − 1 ≥ R₁ and R = R₁. In the first case with R − 1 ≥ R₁, based on the assumption when n₂ = k, this partial derivative is positive. In the second case with R = R₁, we have

$$P(X_1\ge R_1,(X_1+B_1+·+B_k)\ge R-1)=P(X_1\ge R_1,X_1\ge R-1-(B_1+·+B_k))=P(X_1\ge R).$$

Similar to the proof in the first step, we have shown that P(X₁ ≥ R) is an increasing function of p. Thus, P(X₁ ≥ R₁,(Xs + B₁ + B₂ + · + B_k + B_k+1) ≥ R) is an increasing function of p

Hence by mathematical induction, P(R₁, n₁, R, n₂|p) is an increasing function of p when R₁ ≥ 1.

It has been proved that the tail probability P(R₁, n₁, R, n₂|p) is an increasing function of p when the trial goes to the second stage with the tail area as Φ = {(X₁, X₂) : X₁ ≥ R₁, (X₁ + X₂) ≥ R}. When a study is stopped in the first stage, the following remark is used to show that the monotonic condition is still satisifed.

Remark 2.1

In the case that a study has a very small number of responses in the first stage, the study could be stopped earlier. In this case, the tail area is

$$\mathrm{\Phi }=\{(X_1,X_2):X_1\ge R_1\}.$$

It follows that the tail probability is

$$P(R_1,n_1|p)=P(X_1\ge R_1).$$

It has been shown in Theorem 2.1 that P (X₁ ≥ R₁) is an increasing function of p when R₁ ≥ 1. When R₁ = 0, the tail probability is 1, which is independent of the parameter p.

Based on the results from Theorem 2.1 and Remark 2.1, the tail probability is an increasing function of p. This monotonic condition of the tail probability is very important to guarantee that the actual type I and II error rates occur at the boundary of their associated parameter spaces.

3 Discussion

In this note, we prove the monotonic condition of the tail probability in a two-stage setting. Then, p-value and power of a study occur at the boundary of the null parameter space and the alternative parameter space, respectively. This proof is very important to reduce the computational intensity to find an optimal two-stage design.

Recently, adaptive two-stage designs are being increasingly developed and used in practice to make traditionally used study designs flexible and effective. In these adaptive designs, the second stage sample size depends on the number of responses observed from the first stage. The sample sizes in the first stage and the second stage are pre-determined to guarantee the overall type I and II error rates. The study design is presented as

$$(n_1,R_1,n_2(X_1),R_t(X_1)),\text{ where }{X}_1=R_1,R_1+2,\cdots ,n_1.$$

If X₁ < R₁, then the study is stopped in the first stage. When X₁ ≥ R₁, the trial proceeds to the second stage with the sample size n₂(X₁). It is reasonable to assume that n₂(X₁) is a non-increasing function of X₁ when X₁ ≥ R₁. This could be considered as a natural constraint in a study design because if there are more responders in the first stage, then less participants are needed in the second stage.

Currently, most researchers assume that the tail probability function of adaptive designs (Shan, Wilding, Hutson, & Gerstenberger, 2016; Kang & Tian, 2013; Kang, Xiong, Crane, & Tian, 2013) is an increasing function of p during the design search. Although the monotonic condition of the tail probability for the final optimal design is often checked, it is very difficult and time consuming to check the monotonic condition for every possible design during the design search. The extension of proving the monotonic condition from Simon’s two-stage design to adaptive two-stage designs is not trivial as more design parameters are involved. Therefore, we consider this as future work to further investigate the monotonic condition of the tail probability in adaptive two-stage designs.

Tsou et al. (Tsou, Hsiao, Chow, & Liu, 2008) proposed a two-stage design with continuous endpoints. They developed two optimal designs for use in practice: an optimal design with the smallest expected sample size under the null hypothesis, and a minimax design with the smallest possible maximum sample size. The hypothesis is one-sided, and the null hypothesis is rejected when a large test statistic is observed from both the first stage and the two stages combined. The type I and II error rates were only evaluated at the boundary of their associated sample spaces. It would be interesting to investigate the monotonic condition of the tail probability for a two-stage design with continuous endpoints.