Bayesian single-to-double arm transition design using both short-term and long-term endpoints

Summary

The choice between single-arm designs versus randomized double-arm designs has been contentiously debated in the literature of phase II oncology trials. Recently, as a compromise, the single-to-double arm transition design was proposed, combining the two designs into one trial over two stages. Successful implementation of the two-stage transition design requires a suspension period at the end of the first stage to collect the response data of the already enrolled patients. When the evaluation of the primary efficacy endpoint is overly long, the between-stage suspension period may unfavorably prolong the trial duration and cause a delay in treating future eligible patients. To accelerate the trial, we propose a Bayesian Single-to-Double arm design with Short-term endpoints (BSDS), where an intermediate short-term end-point is used for making early termination decisions at the end of the single-arm stage, followed by an evaluation of the long-term endpoint at the end of the subsequent double-arm stage. Bayesian posterior probabilities are used as the primary decision-making tool at the end of the trial. Design calibration steps are proposed for this Bayesian monitoring process to control the frequentist operating characteristics and minimize the expected sample size. Extensive simulation studies have demonstrated that our design has comparable power and average sample size but a much shorter trial duration than conventional single-to-double arm design. Applications of the design are illustrated using two phase II oncology trials with binary endpoints.

1 |. INTRODUCTION

In the drug development process, after a safe dose level of the new treatment has been determined in a phase I trial, a phase II trial is then conducted to determine its effectiveness. Based on the phase II trial results, a decision is made on whether the trial should be terminated or proceed into a large-scale confirmatory phase III trial. Phase II trials are usually either single-arm or double-arm. Standard single-arm phase II trials test the hypothesis that the response rate of a drug is less than a fixed standard rate, usually derived from the historical control data. Flexibility is introduced into single-arm studies via the development of multi-stage designs, which allow early termination due to futility or superiority. For example, Simon¹ proposed a two-stage single-arm design that controls the frequentist type I and type II error rate, optimizing the design based on the optimal or minimax sample size criteria. Shan et al.² developed an extension of Simon’s two-stage design where the second-stage sample size depends on the responses from the first stage, achieving a smaller expected sample size than Simon’s design. Under the Bayesian paradigm, Tan and Machin³ proposed a Bayesian two-stage threshold design where the interim decision is based on a posterior probability cutoff. Sambucini⁴ extended the threshold design through incorporating the posterior predictive probability of trial success. Zhou et al.⁵ investigated the Bayesian monitoring procedure based on Bayes factors derived from local and nonlocal priors.

Due to the small sample size and the lack of a control group, single-arm designs may render insufficient evaluation of the effectiveness of the new treatment. This has led to the development of the randomized double-arm design in phase II clinical trials. Randomized phase II trials are comparative double-arm trials in which patients are randomly assigned to the experimental and control arms. A randomized two-arm trial usually requires a larger sample size and results in a longer trial duration, but it offers a less biased comparison and a more reliable conclusion. Sambucini⁶ suggested that the double-arm phase II trials are preferred over single-arm trials when the standard treatment is no longer applicable to the same patient population. Tang et al.⁷ compared error rates in single-arm and randomized designs and recommended randomized designs if the patient pool is large enough.

More recently, a new class of single-to-double arm transition designs has been proposed to combine a single- and double-arm designs into a single trial.^8,9. The design connects the single- and double-arm trials to avoid the “white space” between each discrete phase, and achieves preservation of patients’ information for a more objective and comprehensive comparison. The designs are relevant when both the single-arm and double-arm trials are planned for evaluation of the treatment effectiveness. As a recent example, Catenacci et al.¹⁰ reported a trial on the combination of gemcitabine plus a hedgehog pathway inhibitor (vismodegib) in patients with metastatic pancreatic cancer, which consisted of a single-arm stage and a double-arm stage. The first stage monitors the toxicity and efficacy of the new treatment, and the second stage performs a randomized comparison with a placebo arm.

As there is often a delay between a patient’s treatment completion and his/her outcome becoming observable and assessed, the trial duration of the two-stage single-to-double arm design might be prolonged due to the trial stalling at the end of the first stage. Specifically, after the first stage sample size has been reached, the patient enrollment is suspended until all the outcomes of the first-stage patients are observed and a decision on whether the trial should proceed into the second stage is made; depending on the time it takes to observe the patients’ outcomes, this would lead to a longer trial duration. Such an issue of trial suspension is particularly relevant for cases where the proportion of disease-free patients after a period of follow-up is used as the endpoint, e.g., the 12-month progression-free survival (PFS) rate. Li, Mick, and Heitjan¹¹ discussed different approaches to interim suspension of the trial and concluded that the optimal suspension strategy depends on the enrollment rate and the outcome availability rate. Moreover, to circumvent the issue of trial suspension after the first stage, novel trial designs have been developed to incorporate short-term endpoints. For example, Kunz et al.¹² proposed a single-arm two-stage trial that uses a correlated short-term endpoint to make the interim decision at the end of the first stage. DeVeaux et al.¹³ suggested a similar design but extended the interim analysis to assess both futility and efficacy for early termination decisions.

We propose a Bayesian Single-to-Double arm design with Short-term endpoints (BSDS) that exploits an intermediate short-term endpoint in the first stage to inform the early stopping decision. The proposed design integrates the single-arm and two-arm phases as well as short-term and long-term endpoints into a seamless two-stage trial. Under such a hybrid design, a short-term/intermediate endpoint is evaluated at the end of the first stage, followed by a randomized double-arm stage where the long-term endpoint in the experimental arm is compared with that of the control arm. By incorporating the short-term endpoints, our design improves the efficiency in treating future patients as it circumvents the issue of trial suspension and leads to a considerable reduction in the trial length, while maintaining a similar level of power and expected sample size.

The rest of the paper is organized as follows. First, we introduce the notation and the framework of the proposed BSDS design in Section 2.1; the modified type I error rate and power in Section 2.2; and the optimizing design parameters and searching algorithm in Section 2.3. In Section 3, we conduct simulation studies to examine the operating characteristics and compare them with other designs under various conditions. We show that our design leads to a considerable reduction in the trial length while maintaining a relatively similar expected sample size, thereby improving the efficiency of the drug development process. In Section 4, we apply the design to real-world phase II clinical trials, and in Section 5, we conclude the paper with a brief discussion.

2 |. METHOD

2.1 |. Notation and Proposed Design

We assume that the short-term and long-term endpoints of interest are binary and equal to 1 when observing a response. We also assume that the short-term and long-term endpoints are positively correlated. A flowchart of the proposed BSDS design is depicted in Figure 1. In the first stage, $m$ patients are enrolled only to the experimental arm. We use $x_S$ and $x_L$ to denote the numbers of short-term and long-term responses among the first $m$ patients, respectively. In the first stage, only the short-term endpoint is used to accelerate the decision-making because a large proportion of $x_L$ may not be available at the moment. Let $p_S$ be the short-term response rate of the experimental treatment, we assume $x_S$ follows a binomial distribution, i.e., $x_S\sim \mathrm{B}\mathrm{i}\mathrm{n}\left(m,p_S\right)$.

FIGURE 1.

Flow chart of the proposed Bayesian Single-to-Double arm design with Short-term endpoints (BSDS).

At the interim analysis, a one-sided test is performed to determine whether the experimental treatment is sufficiently better than a fixed response rate:

$${\mathit{H}}_0^S:p_S\le {\pi }_{S0}\text{versus}{\mathit{H}}_1^S:p_S\ge {\pi }_{S1},$$

where ${\pi }_{S0}$ represents a clinically uninteresting short-term response rate (e.g., derived from historical control data), and ${\pi }_{S1}\left({\pi }_{S1}>{\pi }_{S0}\right)$ is a clinically desired short-term response rate. Given the pre-determined constraints on the type I error rate and power in the first stage, we calculate the required minimum number of short-term responses, denoted as $r_S$, for the trial to proceed to the next stage. Thus, if the total number of first-stage short-term responses $x_S>r_S$, the trial will proceed into the second stage. Otherwise, the trial will be terminated, and the treatment claimed as unpromising.

Once the trial proceeds to the second stage, a total of $2n$ patients are enrolled and equally randomized to the experimental or control arm. Among the $n$ second-stage patients in each arm, we use $y_L$ to denote the number of long-term responses in the experimental arm, and $z_L$ the number of long-term responses in the control arm. Denoting the underlying long-term response rates of the experimental and standard treatment by $p_L$ and $q_L$, respectively, we have $x_L\sim \mathrm{B}\mathrm{i}\mathrm{n}\left(m,p_L\right),y_L\sim \mathrm{B}\mathrm{i}\mathrm{n}\left(n,p_L\right)$ and $z_L\sim \mathrm{B}\mathrm{i}\mathrm{n}\left(n,q_L\right)$. At the end of this randomized stage, we examine the superiority of the experimental treatment by comparing it with a standard treatment by testing the following hypotheses:

$$H_0:p_L\le q_L\text{versus}{H}_1:p_L>q_L.$$

We propose to use Bayesian posterior probabilities to conduct such a two-arm comparison based on the combined data from the two stages. Specifically, we assume a beta-binomial model and set the priors for the response rates $p_L$ and $q_L$ as $p_L\sim \mathrm{B}\mathrm{e}\mathrm{t}\mathrm{a}\left(a_y,b_y\right)$ and $q_L\sim \mathrm{B}\mathrm{e}\mathrm{t}\mathrm{a}\left(a_z,b_z\right)$, their posterior distributions are given by

$$\begin{gathered} p_L\mid x_L,y_L\sim \mathrm{B}\mathrm{e}\mathrm{t}\mathrm{a}\left(a_y+x_L+y_L,b_y+m+n-x_L-y_L\right), \\ {q}_L\mid z_L\sim \mathrm{B}\mathrm{e}\mathrm{t}\mathrm{a}\left(a_z+z_L,b_z+n-z_L\right). \end{gathered}$$

With that, we are able to calculate the posterior probability $\left(\mathrm{P}\mathrm{P}\right)$ that $p_L>q_L$ given $x_L,y_L$ and $z_L$:

$$\mathrm{P}\mathrm{P}\left(x_L,y_L,z_L\right)\equiv \mathrm{P}\mathrm{r}\left(p_L>q_L\mid x_L,y_L,z_L\right)={\int }_0^1{\int }_{q_L}^{1}f\left(p_L;a_y+x_L+y_L,b_y+m+n-x_L-y_L\right)f\left(q_L;a_z+z_L,b_z+n-z_L\right)d{p}_{L}d{q}_L,$$

where $f\left(p_L;a_y+x_L+y_L,b_y+m+n-x_L-y_L\right)$ and $f\left(q_L;a_z+z_L,b_z+n-z_L\right)$ are the posterior beta density functions of $p_L$ and $q_L$, respectively. To control the overall type I error rate, we assign a pre-specified cutoff $\theta$ for the posterior probability. Based on the asymptotic property of the posterior probability,¹⁴ we set $\theta =1-{\alpha }_2$, where ${\alpha }_2$ is the second-stage type I error rate. For example, if ${\alpha }_2=0.1$, then $\theta =0.9$. At the end of the second stage, we reject the null hypothesis and claim the experimental drug promising if $\mathrm{P}\mathrm{P}\left(x_L,y_L,z_L\right)>\theta$; Otherwise, we accept the null hypothesis and declare the experimental treatment unpromising.

As a remark, we use the Bayesian posterior probability PP to test the null hypothesis $H_0$ at the end of the trial. However, it is worth noting that the frequentist two-sample $Z$-test could also be used in this situation if the sample size is moderately large. Both PP and $Z$-statistic are constructed based on sufficient statistics and the decision cutoff is calibrated using the algorithm described in Section 2.3. Therefore, the two testing procedures would have very similar operating characteristics when the sample size is moderately large.^14,15 There may be some minor differences when the sample size is small and the normal approximation in the $Z$-test is not accurate enough. We use the Bayesian posterior probability PP as the test statistic because it provides an exact inference and a coherent Bayesian framework for trial decision-making.

2.2 |. Type I Error Rate and Power Function

In this subsection, we discuss the type I error rate and power separately for each stage of the proposed design. In the first stage, the number of responses $x_S$ and the response rate $p_S$ for the short-term outcome are used to calculate the type I error rate and power. Recall that ${\pi }_{S0}$ is a clinically uninteresting response rate, and ${\pi }_{S1}$ is a clinically desired response rate for the short-term endpoint, the type I error rate and power for the first stage are calculated as:

$${\alpha }_1=1-\sum _{x_S=0}^{r_S}b(x_S;m,{\pi }_{S0}),$$

$$1-{\beta }_1=\sum _{x_S=r_S+1}^{m}b(x_S;m,{\pi }_{S1}),$$

where $b\left(x_S;m,{\pi }_{S0}\right)=\left(\genfrac{}{}{0pt}{}{m}{x_S}\right){{\pi }_{S0}}^{x_S}{\left(1-{\pi }_{S0}\right)}^{m-x_S}$ and $b\left(x_S;m,{\pi }_{S1}\right)=\left(\genfrac{}{}{0pt}{}{m}{x_s}\right){{\pi }_{S1}}^{x_S}{\left(1-{\pi }_{S1}\right)}^{m-x_S}$ denote the probability mass functions of $x_S$ under the two assumed response rates ${\pi }_{S0}$ and ${\pi }_{S1}$, respectively.

In the second stage, let ${\pi }_{L0}$ denote a clinically uninteresting response rate and ${\pi }_{L1}$ a clinically desired response rate for the long-term endpoint. We assume $p_L=q_L={\pi }_{L0}$ under the null hypothesis $H_0$, and $p_L={\pi }_{L1}$ and $q_L={\pi }_{L0}$ under the alternative hypothesis $H_1$. For the experimental arm, we use $p_{SL}$ to denote the joint probability of both long-term and short-term successes, and $x_{SL}$ to denote the number of successes in both endpoints. Technically, the joint probability $p_{SL}$ quantifies the correlation between the long-term and short-term endpoints. We denote the probability of rejecting $H_0$ after the second stage by $\mathrm{P}\mathrm{R}\left(p_S,p_{SL},p_L,q_L\right)$, which is a function of the underlying rates $p_S,p_{SL},p_L$, and $q_L$. The expression of $\mathrm{P}\mathrm{R}\left(p_S,p_{SL},p_L,q_L\right)$ is given by

$$\begin{gathered} \text{PR}(p_S,p_{SL},p_L,q_L)=\sum _{x_S=r_S+1}^{m}b(x_S;m,p_S)- \\ \sum _{x_S=r_S+1}^m\sum _{x_{SL}=0}^{x_S}\sum _{x_L=x_{SL}}^{m-x_S+x_{SL}}\{\left(\begin{array}{c}m\\ x_S\end{array}\right)\left(\begin{array}{c}{x}_S\\ x_{SL}\end{array}\right)\left(\begin{array}{c}m-x_S\\ x_L-x_{SL}\end{array}\right){(p_S-p_{SL})}^{x_S-x_{SL}}{(p_L-p_{SL})}^{x_L-x_{SL}} \\ {p_{SL}}^{x_{SL}}{(1-p_L-p_S+p_{SL})}^{m-x_L-x_S+x_{SL}}\sum _{y_L=0}^n\sum _{z_L=0}^{n}b(y_L;n,p_L)b(z_L;n,q_L)I\{\text{PP}(x_L,y_L,z_L)\le \theta \}\}, \end{gathered}$$

where $p_L-p_{SL}$ denotes the probability of a positive long-term outcome and a negative short-term outcome, $p_S-p_{SL}$ denotes the probability of a positive short-term outcome and a negative long-term outcome, and $1-p_L-p_S+p_{SL}$ denotes the probability of a negative outcome for both endpoints. The first part of the equation gives the probability of continuing the trial after the first stage. The remaining part gives the probability that the trial proceeds into the second stage but does not reject the null hypothesis. At the end of the equation, we introduce an indicator function $I\{\cdot \}$ which filters out all combinations of $x_L,y_L,z_L$ that lead to the rejection of the null hypothesis. In the case of nested endpoints, say the short-term endpoint is stable disease in 60 days and the long-term endpoint is stable disease in 180 days, we have $x_L$ nested in $x_S,p_{SL}=p_L$ and $x_{SL}=x_L$. This is because a patient must develop a short-term outcome before they can develop a positive long-term outcome. When nested endpoints are present, the probability of rejecting $H_0$ can be further simplified to:

$$\begin{gathered} \text{PR}(p_S,p_L,q_L)=\sum _{y_L=0}^n\sum _{z_L=0}^{n}b(y_L;n,p_L)b(z_L;n,q_L) \\ \sum _{x_L=0}^m\sum _{x_S=\text{max}(r_S+1,x_L)}^m\left(\begin{array}{c}m\\ x_S\end{array}\right)\left(\begin{array}{c}{x}_S\\ x_L\end{array}\right){(p_S-p_L)}^{x_S-x_L}{p}_L^{x_L}{(1-p_S)}^{m-x_S}I\{\text{PP}(x_L,y_L,z_L)>\theta \}. \end{gathered}$$

To control the type I error rate after stage 2 at a prespecified level of ${\alpha }_2$, we propose to fix $p_L$ and $q_L$ at the clinically uninteresting long-term response rate specified under $H_0$, i.e., $p_L=q_L={\pi }_{L0}$. Additionally, as shown in the above formulas, the probability of rejecting $H_0$ also depends on the short-term rate $p_S$ and the joint rate $p_{SL}$, which are not specified under $H_0$. To ensure that the proposed design can robustly control the overall type I error rate under a range of $\left(p_S,p_{SL}\right)$, we establish the relationship between $\mathrm{P}\mathrm{R}\left(p_S,p_{SL},p_L,q_L\right)$ (or $\mathrm{P}\mathrm{R}\left(p_S,p_{\mathit{L}},q_L\right)$ for nested outcomes) and $\left(p_S,p_{SL}\right)$ in Theorem 1.

Theorem 1. Assume that $p_L,q_L\in (\mathrm{0,\; 1})$, under each combination of $\left(n,m,r_S,\theta \right)$, we have

1. if the joint probability of observing both long-term and short-term responses $p_{SL}\in (\mathrm{0,\; 1})$ is fixed, $\mathrm{P}\mathrm{R}\left(p_S,p_{SL},p_L,q_L\right)$ (or $\mathrm{P}\mathrm{R}\left(p_S,p_L,q_L\right)$ for nested outcomes) is a monotonically increasing function of the short-term response rate $p_S\in \left(\mathrm{m}\mathrm{a}\mathrm{x}\left(0,p_{SL}\right),1\right);$

2. if $p_S-p_{SL}$ is fixed, $\mathrm{P}\mathrm{R}\left(p_S,p_{SL},p_L,q_L\right)$ is a monotonically increasing function of $p_{SL}\in \left(\mathrm{m}\mathrm{a}\mathrm{x}\left(0,p_S+p_L-1\right),\mathrm{m}\mathrm{i}\mathrm{n}\left(p_S,p_L\right)\right)$.

The proof of Theorem 1 is provided in Appendix. Based on this monotonic relationship, we recommend to elicit the value of $p_{SL}$ using a relatively large and clinically reasonable number, denoted by ${\pi }_{SL}$. In practice, the value of ${\pi }_{SL}$ can be derived from external data based on historical trials. In the case of nested endpoints, due to the relationship between $p_{SL}$ and $p_L,{\pi }_{SL}$ can simply be fixed at ${\pi }_{L0}$. Under a given $p_{SL}={\pi }_{SL}$, we propose to control the maximum type I error rate by taking $p_S={\pi }_{S1}$, the clinically desired response rate for the short-term endpoint. As a result, when $p_S\le {\pi }_{S1}$ and $p_{SL}={\pi }_{SL}$, we have

$$\mathrm{P}\mathrm{R}\left(p_S,p_{SL}={\pi }_{SL},p_L={\pi }_{L0},q_L={\pi }_{L0}\right)\le \mathrm{P}\mathrm{R}\left(p_S={\pi }_{S1},p_{SL}={\pi }_{SL},p_L={\pi }_{L0},q_L={\pi }_{L0}\right)\le {\alpha }_2.$$

The power in the second stage can be calculated using the above equation by substituting ${\pi }_{L1}$ for ${\pi }_{L0}$ and making appropriate assumptions for $p_S$ and $p_{SL}$. To obtain a practical sample size, we propose to fix $\left(p_S,p_{SL}\right)$ at $\left({\pi }_{L1},{\pi }_{SL}\right)$ and optimize the design parameters by controlling $\mathrm{P}\mathrm{R}\left(p_S={\pi }_{S1},p_{SL}={\pi }_{SL},p_L={\pi }_{L1},q_L={\pi }_{L1}\right)\ge 1-{\beta }_2$, where ${\beta }_2$ is the prespecified level of type II error rate after stage 2.

2.3 |. Optimizing Design Parameters

Given a pre-specified set of the assumed response rates (${\pi }_{L0},{\pi }_{L1},{\pi }_{S0},{\pi }_{S1},{\pi }_{SL}$), and type I and type II error rate constraints $\left({\alpha }_1,{\alpha }_2,{\beta }_1,{\beta }_2\right)$, we are able to find the optimal solutions of the first stage threshold response $r_S$, as well as the sample sizes $m$ and $n$. The optimal combination of $m,n$ and $r_S$ is chosen by minimizing the average sample number (ASN), which is the average of the expected sample size under the null and alternative hypotheses. ASN is a widely used criterion in designing phase II clinical trials.^9,16,17,18 The rationale behind using the ASN is to consider both the null and alternative hypotheses when choosing the optimal design parameters. The ASN is given by

$$\mathrm{A}\mathrm{S}\mathrm{N}=\left({\mathrm{E}\mathrm{S}\mathrm{S}}_0+{\mathrm{E}\mathrm{S}\mathrm{S}}_1\right)/2,$$

where

$$\begin{gathered} {\text{ESS}}_0=m+2n(1-{\text{PET}}_0), \\ {\text{ESS}}_1=m+2n(1-{\text{PET}}_1). \end{gathered}$$

The expected sample sizes ${\mathrm{E}\mathrm{S}\mathrm{S}}_0$ and ${\mathrm{E}\mathrm{S}\mathrm{S}}_1$ depend on the probabilities of early termination (PET). In the proposed BSDS design, the PET under the null and alternative hypotheses are calculated assuming the short-term response rate $p_S={\pi }_{S0}$ and $p_S={\pi }_{S1}$, respectively:

$$\begin{gathered} {\text{PET}}_0=\text{Pr}(x_S\le r_S\mid p_S={\pi }_{S0})=\sum _{x_S=0}^{r_S}b(x_S;m,{\pi }_{S0}), \\ {\text{PET}}_1=\text{Pr}(x_S\le r_S\mid p_S={\pi }_{S1})=\sum _{x_S=0}^{r_S}b(x_S;m,{\pi }_{S1}). \end{gathered}$$

The searching strategy to optimize the design parameters $m,n$, and $r_S$ is described as follows:

1. Set the searching range for $n$ to be between 10 and $N_{\mathrm{m}\mathrm{a}\mathrm{x}}/2$, where $N_{\mathrm{m}\mathrm{a}\mathrm{x}}$ is the required total sample size for a typical two-arm randomized design with a type I error rate ${\alpha }_2$ and power $1-{\beta }_2$, and is calculated as $N_{\mathrm{m}\mathrm{a}\mathrm{x}}=2{\left(z_{{\alpha }_2}+z_{{\beta }_2}\right)}^2\left\{{\pi }_{L0}\left(1-{\pi }_{L0}\right)+\right.\left.{\pi }_{L1}\left(1-{\pi }_{L1}\right)\right\}/{\left({\pi }_{L1}-{\pi }_{L0}\right)}^2$ using the pre-specified null and alternative responses rates ${\pi }_{L0}$ and ${\pi }_{L1}$, where $z_{\alpha }$ denotes the $100(1-\alpha )\mathrm{th}$ percentile of the standard normal distribution.

2. For a given $m$, enumerate and store all values of $r_S$ such that the first-stage type I error rate is below ${\alpha }_1$ and power is greater than ${\beta }_1$.

3. For each combination of $\left(m,r_S\right)$, and the probability cutoff $\theta$ for posterior probability, enumerate and store all values of $n$ that satisfy the constraints on the type I error rate ${\alpha }_2$ and power $1-{\beta }_2$, i.e., $\mathrm{P}\mathrm{R}\left(p_S={\pi }_{S1},p_{SL}={\pi }_{SL},p_L={\pi }_{L0},q_L={\pi }_{L0}\right)\le {\alpha }_2$ and $\mathrm{P}\mathrm{R}\left(p_S={\pi }_{S1},p_{SL}={\pi }_{SL},p_L={\pi }_{L1},q_L={\pi }_{L1}\right)\ge 1-{\beta }_2$.

4. For all combinations of $\left(m,n,r_S\right)$, calculate their PET and ESS under both hypotheses and obtain the ASN. Identify the combination of $\left(m,n,r_S\right)$ that minimizes the ASN.

2.4 |. Expected Trial Length

The BSDS design aims to reduce the trial length by shortening the suspension period between the two stages. We use the expected trial length (EL) to quantify the expected duration of a trial. The EL is calculated using the optimal sample sizes ($m,n$), the enrollment rate $\lambda$, the assessment times for the short-term $\left(t_1\right)$ and long-term $\left(t_2\right)$ outcomes, and the probability of early termination $\left({\mathrm{P}\mathrm{E}\mathrm{T}}_0,{\mathrm{P}\mathrm{E}\mathrm{T}}_1\right)$. Similarly to ESS, we calculate the average of ELs at the null $\left({\mathrm{E}\mathrm{L}}_0\right)$ and alternative hypotheses $\left({\mathrm{E}\mathrm{L}}_1\right)$, $\mathrm{A}\mathrm{E}\mathrm{L}=\left({\mathrm{E}\mathrm{L}}_0+{\mathrm{E}\mathrm{L}}_1\right)/2$, where

$$\begin{gathered} {\text{EL}}_0=(m-1)/\lambda +t_1+\left\{(n-1)/\lambda +t_2\right\}(1-{\text{PET}}_0), \\ {\text{EL}}_1=(m-1)/\lambda +t_1+\left\{(n-1)/\lambda +t_2\right\}(1-{\text{PET}}_1). \end{gathered}$$

The average expected length (AEL) represents the average of the expected trial lengths under the null and alternative hypotheses. In Section 3, we calculate the BSDS design AEL at different response rates and compare it with those of other designs.

3 |. SIMULATION STUDIES

Simulation studies are conducted to evaluate the operating characteristics of the proposed BSDS design. To calibrate the design parameters $\left(m,n,r_S\right)$, we need to specify the null and alternative response rates $\left.{\pi }_{L0},{\pi }_{L1},{\pi }_{S0},{\pi }_{S1}\right)$, and the type I and type II error rate constraints in both stages $\left({\alpha }_1,{\alpha }_2,{\beta }_1,{\beta }_2\right)$. We examine the operating characteristics under different configurations of $\left({\pi }_{L0},{\pi }_{L1},{\pi }_{S0},{\pi }_{S1}\right)$. For example, in Table 1, we set ${\pi }_{L1}={\pi }_{L0}+\delta$ and ${\pi }_{S1}={\pi }_{S0}+\delta$, and vary ${\pi }_{L0}$ from 0.1 to 0.4 at $\delta =(\mathrm{0.2,\; 0.3})$, respectively. To further evaluate the robustness of the BSDS design, we fix ${\pi }_{L0},{\pi }_{L1}$, and ${\pi }_{S0}$ at their initial values and increase ${\pi }_{S1}$ from its initial value until ${\pi }_{S1}={\pi }_{S0}+0.5$ (Table 2). Regarding the type I and type II error rates, we set them at $\left({\alpha }_1,{\alpha }_2,{\beta }_1,{\beta }_2\right)=(\mathrm{0.2,\; 0.1,\; 0.1,\; 0.2})$ in Tables 1 – 4. For the posterior probability, we adopt a non-informative prior distribution $\mathrm{B}\mathrm{e}\mathrm{t}\mathrm{a}\left({\pi }_{L0},1-{\pi }_{L0}\right)$ for both $p_L$ and $q_L$, and set the probability cut-off $\theta =1-{\alpha }_2$. The type I error rate ${\alpha }_2$ is calculated by setting the short-term response rate at $p_S={\pi }_{S1}$, and the long-term response rates at $p_L=q_L={\pi }_{L0}$. The power $1-{\beta }_2$ is obtained by setting $\left(p_L,q_L\right)=\left({\pi }_{L1},{\pi }_{L0}\right)$. Lastly, we calculate the average expected trial length to illustrate the expected amount of time-saving the BSDS design can achieve. For illustrative purposes, we assume $\lambda =2$, meaning that patients are being recruited at a steady pace of two patients per month. Additionally, we assume $t_1=3$ and $t_2=12$, which is equivalent to the short-term response observable at month 3 and the long-term response observable at month 12, e.g., the 3-month PFS rate and 12-month PFS rate often used as short-term/long-term endpoints in oncology trials.

TABLE 1.

Operating characteristics at the optimal design parameters of the BSDS design under various configurations of (${\pi }_{L0},{\pi }_{L1},{\pi }_{S0},{\pi }_{S1}$) at $\left({\alpha }_1,{\alpha }_2,{\beta }_1,{\beta }_2\right)=(\mathrm{0.2,0.1,0.1,0.2})$ when fixing ${\pi }_{S1}-{\pi }_{S0}=\delta$.

${\pi }_{L1}$	${\pi }_{L0}$	${\pi }_{S1}$	${\pi }_{S0}$	$r_S$	m	n	PET0	PET1	ESS0	ESS1	ASN	${\alpha }_2$	$1-{\beta }_2$	AEL
$\delta =0.2$
0.30	0.10	0.30	0.10	3	21	33	0.848	0.086	31.03	81.35	56.19	0.072	0.801	27.92
		0.40	0.20	7	28	33	0.818	0.074	40.00	89.12	64.56	0.068	0.806	32.01
		0.50	0.30	11	30	35	0.841	0.100	41.15	92.98	67.07	0.070	0.800	32.86
		0.60	0.40	14	30	35	0.825	0.097	42.28	93.21	67.74	0.070	0.800	33.13
		0.70	0.50	17	30	35	0.819	0.084	42.66	94.09	68.37	0.070	0.808	33.41
0.40	0.20	0.40	0.20	8	31	43	0.849	0.074	43.97	110.65	77.31	0.099	0.802	35.77
		0.50	0.30	13	36	41	0.837	0.066	49.34	112.57	80.95	0.087	0.806	38.05
		0.60	0.40	15	32	48	0.835	0.092	47.82	119.17	83.50	0.087	0.800	37.55
		0.70	0.50	18	32	44	0.811	0.069	48.59	113.89	81.24	0.085	0.800	37.26
		0.80	0.60	22	33	41	0.831	0.051	46.86	110.83	78.84	0.090	0.806	36.89
0.50	0.30	0.50	0.30	11	31	51	0.808	0.075	50.63	125.37	88.00	0.092	0.802	38.66
		0.60	0.40	17	37	48	0.818	0.059	54.47	127.37	90.92	0.096	0.802	40.93
		0.70	0.50	19	34	49	0.804	0.057	53.18	126.40	89.79	0.093	0.800	40.00
		0.80	0.60	20	30	51	0.824	0.061	47.98	125.77	86.88	0.095	0.800	38.13
		0.90	0.70	20	26	49	0.837	0.040	41.93	120.09	81.01	0.090	0.800	35.71
0.60	0.40	0.60	0.40	16	35	50	0.807	0.062	54.35	128.85	91.60	0.094	0.800	40.64
		0.70	0.50	19	34	50	0.804	0.057	53.58	128.29	90.93	0.096	0.802	40.29
		0.80	0.60	22	33	51	0.831	0.051	50.24	129.81	90.03	0.095	0.803	39.68
		0.90	0.70	19	25	50	0.807	0.033	44.35	121.66	83.00	0.095	0.800	36.17
$\delta =0.3$
0.40	0.10	0.40	0.10	2	15	14	0.816	0.027	20.15	42.24	31.20		0.801	20.70
		0.50	0.20	4	15	16	0.836	0.059	20.26	45.10	32.68	0.038	0.812	20.77
		0.60	0.30	6	16	16	0.825	0.058	21.61	46.13	33.87	0.042	0.802	21.39
		0.70	0.40	8	16	17	0.858	0.074	20.84	47.47	34.15	0.041	0.809	21.18
		0.80	0.50	9	15	16	0.849	0.061	19.83	45.05	32.44	0.038	0.802	20.63
0.50	0.20	0.50	0.20	5	18	18	0.867	0.048	22.78	52.27	37.53	0.083	0.805	22.62
		0.60	0.30	6	16	21	0.825	0.058	23.36	55.55	39.46	0.072	0.808	22.79
		0.70	0.40	8	17	19	0.801	0.040	24.56	53.47	39.01	0.077	0.800	23.17
		0.80	0.50	9	15	21	0.849	0.061	21.34	54.44	37.89	0.089	0.813	21.99
0.60	0.30	0.60	0.30	6	16	21	0.825	0.058	23.36	55.55	39.46	0.080	0.800	22.79
		0.70	0.40	8	17	20	0.801	0.040	24.96	55.39	40.17	0.095	0.800	23.46
		0.80	0.50	9	15	22	0.849	0.061	21.64	56.31	38.98	0.094	0.801	22.26
		0.90	0.60	11	15	22	0.909	0.056	18.98	56.56	37.77	0.094	0.802	21.64
0.70	0.40	0.70	0.40	8	16	23	0.858	0.074	22.54	58.58	40.56	0.096	0.809	22.78
		0.80	0.50	9	15	22	0.849	0.061	21.64	56.31	38.98	0.100	0.804	22.26
		0.90	0.60	11	15	22	0.909	0.056	18.98	56.56	37.77	0.094	0.805	21.64

TABLE 2.

Operating characteristics at the optimal design parameters of the BSDS design under various configurations of (${\pi }_{L0},{\pi }_{L1},{\pi }_{S0},{\pi }_{S1}$) at $\left({\alpha }_1,{\alpha }_2,{\beta }_1,{\beta }_2\right)=(\mathrm{0.2,0.1,0.1,0.2})$ with ${\pi }_{S0}={\pi }_{L0}$ and varying ${\pi }_{S1}$.

${\pi }_{L1}$	${\pi }_{L0}$	${\pi }_{S1}$	${\pi }_{S0}$	$r_S$	m	n	PET0	PET1	ESS0	ESS1	ASN	${\alpha }_2$	$1-{\beta }_2$	AEL
$\delta =0.2$
0.30	0.10	0.30	0.10	3	21	33	0.848	0.086	31.03	81.35	56.19	0.030	0.801	27.92
		0.40	0.10	3	19	29	0.885	0.023	25.67	75.67	50.67	0.067	0.806	26.20
		0.50	0.10	2	10	36	0.930	0.055	15.05	78.06	46.56	0.065	0.804	22.47
		0.60	0.10	3	12	32	0.974	0.015	13.64	75.02	44.33	0.073	0.805	22.40
0.40	0.20	0.40	0.20	8	31	43	0.849	0.074	43.97	110.65	77.31	0.036	0.802	35.77
		0.50	0.20	5	18	45	0.867	0.048	29.96	103.67	66.82	0.090	0.804	29.95
		0.60	0.20	5	15	44	0.939	0.033	20.37	100.02	60.20	0.094	0.804	27.22
		0.70	0.20	5	13	43	0.970	0.018	15.58	97.43	56.51	0.093	0.800	25.70
0.50	0.30	0.50	0.30	11	31	51	0.808	0.075	50.63	125.37	88.00	0.040	0.802	38.66
		0.60	0.30	9	24	46	0.847	0.022	38.06	114.01	76.03	0.093	0.800	34.01
		0.70	0.30	8	18	48	0.940	0.021	23.72	111.99	67.85	0.092	0.800	29.94
		0.80	0.30	7	13	51	0.982	0.030	14.86	111.94	63.40	0.096	0.800	27.28
0.60	0.40	0.60	0.40	16	35	50	0.807	0.062	54.35	128.85	91.60	0.041	0.800	40.64
		0.70	0.40	12	24	50	0.886	0.032	35.43	120.86	78.14	0.097	0.802	34.25
		0.80	0.40	10	17	53	0.965	0.038	20.69	119.01	69.85	0.097	0.800	29.94
		0.90	0.40	6	10	53	0.945	0.013	15.80	114.64	65.22	0.097	0.801	27.30
$\delta =0.3$
0.40	0.10	0.40	0.10	2	15	14	0.816	0.027	20.15	42.24	31.20	0.038	0.801	20.70
		0.50	0.10	3	11	20	0.981	0.113	11.74	46.47	29.11	0.065	0.802	17.74
		0.60	0.10	4	11	20	0.997	0.099	11.11	47.03	29.07	0.065	0.810	17.72
0.50	0.20	0.50	0.20	5	18	18	0.867	0.048	22.78	52.27	37.53	0.083	0.805	22.62
		0.60	0.20	4	12	21	0.927	0.057	15.05	51.59	33.32	0.070	0.802	19.68
		0.70	0.20	4	10	21	0.967	0.047	11.38	50.01	30.69	0.086	0.802	18.35
0.60	0.30	0.60	0.30	6	16	21	0.825	0.058	23.36	55.55	39.46	0.080	0.800	22.79
		0.70	0.30	4	10	22	0.850	0.047	16.61	51.92	34.26	0.089	0.800	19.91
		0.80	0.30	5	10	22	0.953	0.033	12.08	52.56	32.32	0.090	0.808	18.91
0.70	0.40	0.70	0.40	8	16	23	0.858	0.074	22.54	58.58	40.56	0.096	0.809	22.78
		0.80	0.40	7	13	20	0.902	0.030	16.91	51.80	34.35	0.097	0.803	20.48
		0.90	0.40	7	10	25	0.988	0.070	10.61	56.49	33.55	0.090	0.800	18.80

TABLE 4.

Comparison of the operating characteristics of the BSDS design, the START design, and the randomized phase IIb design under various configurations of $\left({\pi }_{L0},{\pi }_{L1},{\pi }_{S0},{\pi }_{S1}\right)$ at $\left({\alpha }_1,{\alpha }_2,{\beta }_1,{\beta }_2\right)=(\mathrm{0.2,0.1,0.1,0.2})$ and $\delta =0.3$.

${\pi }_{L1}$	${\pi }_{L0}$	Design	${\pi }_{S1}$	${\pi }_{S0}$	$r_S$	m	n	PET0	PET1	ESS0	ESS1	ASN	${\alpha }_2$	$1-{\beta }_2$	AEL
0.40	0.10	BSDS	0.50	0.20	4	15	16	0.836	0.059	20.26	45.10	32.68	0.038	0.812	20.77
		START	0.40	0.10	2	13	15	0.866	0.058	17.02	41.26	29.14	0.033	0.819	28.22
		IIb	-	-	-	-	18	-	-	36.00	36.00	36.00	0.105	0.807	20.50
0.50	0.20	BSDS	0.60	0.30	5	13	24	0.938	0.098	26.94	56.31	38.63	0.072	0.812	18.79
		START	0.50	0.20	4	15	19	0.836	0.059	21.24	50.75	35.99	0.041	0.822	30.60
		IIb	-	-	-	-	22	-	-	44.00	36.00	40.00	0.102	0.808	22.50
0.60	0.30	BSDS	0.70	0.40	8	17	20	0.801	0.040	24.96	55.39	40.17	0.093	0.800	23.46
		START	0.60	0.30	6	16	21	0.825	0.058	23.36	55.55	39.46	0.045	0.810	31.79
		IIb	-	-	-	-	25	-	-	50.00	50.00	50.00	0.098	0.815	24.00
0.70	0.40	BSDS	0.80	0.50	9	15	22	0.849	0.061	21.64	56.32	38.98	0.094	0.809	22.26
		START	0.70	0.40	8	16	22	0.858	0.074	22.26	56.73	39.49	0.034	0.806	31.51
		IIb	-	-	-	-	25	-	-	50.00	50.00	50.00	0.107	0.815	24.00

Table 1 presents the optimal solutions of ($m,n,r_S$) that minimize the ASN under different configurations of ${\pi }_{L0},{\pi }_{L1},{\pi }_{S0}$, and ${\pi }_{S1}$ with fixed $\delta$ for both long-term and short-term response rates. We observe an increasing trend in the magnitudes of the design parameters $\left(m,n,r_S\right)$ as $\left({\pi }_{L0},{\pi }_{L1}\right)$ elevates. In terms of the probabilities of early termination, we note that the value of ${\mathrm{P}\mathrm{E}\mathrm{T}}_0$ is much bigger than ${\mathrm{P}\mathrm{E}\mathrm{T}}_1$. This is due to the early stopping in stage 1 when the short-term response rate falls below the clinically uninteresting response rate. Under various configurations of response rates, we are able to control ${\mathrm{P}\mathrm{E}\mathrm{T}}_0$ above 0.8 and ${\mathrm{P}\mathrm{E}\mathrm{T}}_1$ below 0.1, respectively, aligning with the type I and II error constraints in the first stage. For the expected sample size, similar to ${\mathrm{P}\mathrm{E}\mathrm{T}}_0$ and ${\mathrm{P}\mathrm{E}\mathrm{T}}_1,{\mathrm{E}\mathrm{S}\mathrm{S}}_0$ has a smaller value than ${\mathrm{E}\mathrm{S}\mathrm{S}}_1$. The resulting ASN increases when either the long-term rates $\left({\pi }_{L0},{\pi }_{L1}\right)$ or the short-term rates $\left({\pi }_{S0},{\pi }_{S1}\right)$ elevates.

In terms of the type I error rate ${\alpha }_2$, it is well controlled below the pre-specified type I error rate constraint of 0.1. It is worth noting that ${\alpha }_2$ is relatively low when ${\pi }_{L0}$ is at the lower range and tends to increase and approaches the constraint when ${\pi }_{L0}$ increases from 0.1 to 0.4. The second-stage power $1-{\beta }_2$ is also controlled just above the pre-specified constraint of 0.8 for each optimal set of $\left(m,n,r_S\right)$. We further evaluate the characteristics of the BSDS design when we set $\delta =0.3$ and similar trends are observed. With a bigger difference between null and alternative long-term response rates, the resulting solutions of ($m,n,r_S$) are smaller than the ones in $\delta =0.2$. Hence, we also observe smaller magnitudes of the expected sample sizes and AEL in such a setting.

Table 2 summarizes the optimal parameters when varying the difference between the short-term response rate ${\pi }_{S1}-{\pi }_{S0}$. As the difference increases from 0.2 to 0.5, the ${\mathrm{P}\mathrm{E}\mathrm{T}}_0$ exhibits an increasing trend and approaches 1 when ${\pi }_{S1}-{\pi }_{S0}$ is large, while the ${\mathrm{P}\mathrm{E}\mathrm{T}}_1$, ASN and AEL all exhibit a decreasing trend. Similar trends persist, but smaller numbers of $\left(m,n,r_S\right)$, ASN, and AEL are observed when setting $\delta =0.3$.

In Tables 3 and 4, we compare the operating characteristics of the proposed BSDS design against the single-to-double arm transition (START) design proposed by Shi and Yin,⁹ and a conventional randomized phase IIb design under different configurations of ${\pi }_{L0},{\pi }_{L1},{\pi }_{S0}$ and ${\pi }_{S1}$. The START design is a single-to-double arm two-stage design that combines a single-arm first stage that compares the experimental arm with a pre-specified value and a double-arm second stage where patients’ response rates are compared between the experimental arm and the control arm based on a two-sample Z-test using combined information from both stages.⁹ We assume the short-term and long-term endpoints are nested and set $\left({\pi }_{S0},{\pi }_{S1}\right)$ at $\left({\pi }_{L0}+0.1,{\pi }_{L1}+0.1\right)$ to represent the nested condition. With regards to the suspension strategy of the BSDS design, enrollment of new patients is delayed until the short-term outcomes of the patients enrolled in stage 1 have all been observed, which is expected to be faster than using the long-term outcomes. It is worth noting that the suspension strategy in the START design, on the other hand, is based on the long-term endpoint, i.e., delaying the enrollment of new patients until the long-term outcomes of the previously treated patients in stage 1 have been fully observed, which leads to a longer delay. The search algorithm of the START design is similar to that of the BSDS design, in which the optimal design parameters are selected using the ASN-based optimal criterion. Compared with the randomized phase IIb design, despite requiring a larger overall sample size ($m+2n$), the BSDS design has a lower ASN than the single randomized phase IIb design throughout the tables. Compared with the START design, the BSDS design has a slightly higher ASN at a lower long-term response rate. For example, when $\left({\pi }_{L0},{\pi }_{L1}\right)=(\mathrm{0.1,\; 0.3})$, the START design results in the optimal design parameters $\left(m,n,r_s\right)=(\mathrm{23,\; 29,\; 3})$ and $\mathrm{ASN}=56$. Meanwhile, BSDS leads to the design parameters $\left(m,n,r_s\right)=(\mathrm{28,\; 33,\; 7})$ and the $\mathrm{A}\mathrm{S}\mathrm{N}=64.56$. However, as $\left({\pi }_{L0},{\pi }_{L1}\right)$ elevates, the resulting design parameters become similar for the two designs, and the difference between their ASN also diminishes. Regarding the average expected length of trial AEL, at $\delta =0.3$, the BSDS design leads to a comparable and even shorter AEL than the phase IIb design. Furthermore, the BSDS design leads to a shorter AEL than that of the START design in all configurations.

TABLE 3.

Comparison of the operating characteristics of the BSDS design, the START design, and the randomized phase IIb design under various configurations of $\left({\pi }_{L0},{\pi }_{L1},{\pi }_{S0},{\pi }_{S1}\right)$ at $\left({\alpha }_1,{\alpha }_2,{\beta }_1,{\beta }_2\right)=(\mathrm{0.2,0.1,0.1,0.2})$ and $\delta =0.2$.

${\pi }_{L1}$	${\pi }_{L0}$	Design	${\pi }_{S1}$	${\pi }_{S0}$	$r_S$	m	n	PET0	PET1	ESS0	ESS1	ASN	${\alpha }_2$	$1-{\beta }_2$	AEL
0.30	0.10	BSDS	0.40	0.20	7	28	33	0.818	0.074	40.00	89.12	64.56	0.068	0.806	32.01
		START	0.30	0.10	3	23	29	0.807	0.054	34.18	77.90	56.00	0.042	0.800	37.81
		IIb	-	-	-	-	36	-	-	72.00	72.00	72.00	0.105	0.807	29.50
0.40	0.20	BSDS	0.50	0.30	13	36	41	0.837	0.066	49.34	112.57	80.95	0.087	0.806	38.04
		START	0.40	0.20	8	32	38	0.825	0.057	45.27	103.60	74.60	0.040	0.805	44.54
		IIb	-	-	-	-	46	-	-	92.00	92.00	92.00	0.102	0.809	34.50
0.50	0.30	BSDS	0.60	0.40	17	37	48	0.818	0.059	54.47	127.37	90.92	0.096	0.802	40.94
		START	0.50	0.30	12	34	47	0.807	0.061	52.13	122.29	87.21	0.043	0.801	48.31
		IIb	-	-	-	-	54	-	-	108.00	108.00	108.00	0.098	0.808	38.50
0.60	0.40	BSDS	0.70	0.50	19	34	50	0.804	0.057	53.58	128.29	90.93	0.096	0.802	40.28
		START	0.60	0.40	16	35	50	0.807	0.062	54.35	128.85	91.60	0.043	0.810	49.66
		IIb	-	-	-	-	55	-	-	110.00	110.00	110.00	0.107	0.810	39.00

To further evaluate the performance of the BSDS design, we compute and compare its type I error rate, power, ESS, and EL against those of the START design and the randomized phase IIb design under $\left({\pi }_{L0},{\pi }_{L1},{\pi }_{S0},{\pi }_{S1}\right)=(\mathrm{0.2,\; 0.4,\; 0.3,\; 0.5})$ with $\left({\alpha }_1,{\alpha }_2,{\beta }_1,{\beta }_2\right)=(\mathrm{0.2,\; 0.1,\; 0.1,\; 0.2})$. Using the calculated optimal design parameters $\left(m,n,r_s\right)=(\mathrm{36,\; 41,\; 13})$, we set $\left({\pi }_{S0},{\pi }_{S1}\right)=\left({\pi }_{L0}+0.1,{\pi }_{L1}+0.1\right)$ and vary ${\pi }_{L0}$ from 0 to 0.9 to obtain the resulting type I error rate ${\alpha }_2$, power $1-{\beta }_2,{\mathrm{E}\mathrm{S}\mathrm{S}}_0$ and ${\mathrm{E}\mathrm{L}}_0$ in Figure 2. In terms of power, the BSDS design exhibits a similar trend and achieves a higher power compared with the START design. Specifically, we observe a power smaller than 0.8 when the actual response rate $p_L$ is less than ${\pi }_{L1}$ due to the screening effect in the first stage. When $p_L$ is greater than ${\pi }_{L1}$, the observed power is compensated by having more samples in the second stage, hence it is higher than the power of the phase IIb design. For type I error rate, we observe a reduction in type I error rate in both BSDS and START compared to the phase IIb design when $p_L$ is in the lower range. The BSDS design has a higher type I error rate than the START design due to the higher response rate for the short-term outcome. As for the expected sample size, similar to the START design, the BSDS design achieves a smaller ${\mathrm{E}\mathrm{S}\mathrm{S}}_0$ than the phase IIb does when $p_L$ is in the lower range. However, a larger sample size is expected when $p_L$ increases, resulting from the greater probability of entering the second stage. In terms of the expected length, the BSDS design consistently achieves a shorter ${\mathrm{E}\mathrm{L}}_0$ than the START design does. Moreover, the START design has a shorter ${\mathrm{E}\mathrm{L}}_0$ than the phase IIb when $p_L$ is lower than 0.4, while the ${\mathrm{E}\mathrm{L}}_0$ of the BSDS design remains lower than that of the phase IIb until $p_L$ is approximately 0.5. The above results suggest that our design can achieve considerable time-saving yet it has a comparable type I error rate, power, and expected sample size with the phase IIb and the START design.

FIGURE 2.

Comparison of power, type I error rate, expected sample size and expected length between the BSDS design, the START design, and the randomized phase IIb design with $\left({\alpha }_1,{\alpha }_2,{\beta }_1,{\beta }_2\right)=(\mathrm{0.2,0.1,0.1,0.2})$.

4 |. TRIAL EXAMPLES

4.1 |. Radical radiation therapy for oligometastatic breast cancer

This section considers the application of the proposed BSDS design to existing phase II trials and compares the resulting operating characteristics with those of the START design. In the first example, we consider a prospective phase II multi-centered trial to determine if radical radiation therapy to all metastatic sites might improve the progression-free survival (PFS) in oligometastatic breast cancer patients.¹⁹ The trial’s primary endpoint is the two-year PFS rate, where PFS is defined as the time from the end of treatment to local or distant progression, or death. The sample sizes were calculated using Simon’s optimal design¹ assuming a two-year PFS rate of 0.3 under the null hypothesis and 0.5 under the alternative hypothesis, with the type I and II error rates constraints $\alpha =0.05$ and $\beta =0.2$. In the first stage, the trial was scheduled to enroll 15 patients. If less than five patients were reported to have observed the primary responses at the end of the first stage, the trial would stop early. Otherwise, the trial would proceed to the second stage and enroll an additional 31 patients. At the end of the second stage, the null hypothesis would be rejected if more than 16 responders were observed.

We choose the one-year PFS rate as the short-term endpoint to shorten the trial length and make early termination decisions. The PFS rate was obtained from the Kaplan-Meier estimate plot from Trovo et al.¹⁹ The cutoff value $\theta$ for the posterior probability was set at $\theta =1-\alpha =0.95$. Operating characteristics under different combinations of (${\pi }_{S0},{\pi }_{S1}$) are presented in Table 5. Compared to START, when setting $\left({\pi }_{S0},{\pi }_{S1}\right)=(\mathrm{0.45,\; 0.7})$, we observe smaller values of $m,n$ and $r_S$. The resulting ASN is 103.45, which is also smaller than the ASN of the START design. Also, the AEL of the BSDS design is shortened to 56.76 months compared to START’s 74.18 months. The type I error rates and power for both designs under both conditions are similar and well contained below the pre-specified ${\alpha }_2$ and $1-{\beta }_2$ constraints. Furthermore, when we assume a larger difference between the null and alternative short-term response rates by setting $\left({\pi }_{S0},{\pi }_{S1}\right)=(\mathrm{0.40,\; 0.7})$, the resulting ASN is further reduced to 95.63, and the AEL is shortened to 54.98 months. This is almost 20 months shorter than the AEL of the START design. In this example, the BSDS design can achieve a much shorter expected length with comparable power, type I error rate, and required sample size than the START design.

TABLE 5.

Comparison of the operating characteristics of the BSDS design and the START design with optimal design parameters in trial applications.

${\pi }_{L1}$	${\pi }_{L0}$	Design	${\pi }_{S1}$	${\pi }_{S0}$	$r_S$	m	n	PET0	PET1	ESS0	ESS1	ASN	${\alpha }_2$	$1-{\beta }_2$	AEL
Example 4.1 1										.
0.50	0.30	START	-	-	16	42	70	0.903	0.082	55.54	170.51	113.02	0.043	0.801	74.18
0.50	0.30	BSDS	0.7	0.45	16	30	66	0.952	0.040	36.35	156.71	95.63	0.047	0.802	54.98
		BSDS	0.7	0.4	16	29	72	0.901	0.065	43.29	163.61	103.45	0.047	0.800	56.76
Example 4.2 2
0.35	0.20	START	-	-	13	54	66	0.820	0.060	77.43	178.25	127.84	0.042	0.800	69.04
0.35	0.20	BSDS	0.5	0.3	13	37	74	0.807	0.049	65.55	177.68	121.62	0.094	0.800	57.12
		BSDS	0.5	0.25	13	38	71	0.929	0.036	48.08	174.82	111.45	0.093	0.801	51.95

¹For Example 4.1, $\left({\alpha }_1,{\alpha }_2,{\beta }_1,{\beta }_2\right)=(\mathrm{0.1,\; 0.05,\; 0.1,\; 0.2})$.

²For Example 4.2, $\left({\alpha }_1,{\alpha }_2,{\beta }_1,{\beta }_2\right)=(\mathrm{0.2,\; 0.1,\; 0.1,\; 0.2})$.

4.2 |. ADXS-HPV for platinum-refractory cervical carcinoma

The second example involves a vaccine trial. Huh et al.²⁰ studied the therapeutic vaccine, Axalimogene filolisbac (ADXS-HPV), in women who had progression in metastatic cervical cancer following at least one prior line of therapy. The primary endpoint of the trial is the one-year overall survival (OS) rate. Similar to the first example, the sample size and the minimum required number of responses were calculated using Simon’s optimal design based on the assumption that the one-year OS rate is 0.2 under the null hypothesis and 0.35 under the alternative hypothesis. The study was designed to proceed to stage 2 if the conditional power at the end of stage 1 was greater than 0.2. The target sample size was 27 in stage 1 and 36 in stage 2, and the thresholds at the end of the two stages are $r_S=2$ and $r=12$, respectively (with $\alpha =0.1$ and $\beta =0.2$).

We use the three-month PFS rate as the short-term endpoint for the one-year OS rate. According to Belin et al.²¹, PFS is widely used as a surrogate outcome for overall survival. Here we obtain the three-month PFS rate using the PFS curve from Figure 3B in Huh et al.²⁰ Like the previous example, we assume different ${\pi }_{S0}$ values for the three-month PFS rate but keep ${\pi }_{L0}$ fixed. Using the given PFS and OS rate, operating characteristics under different combinations of $\left({\pi }_{S0},{\pi }_{S1}\right)$ are calculated in Table 6. When $\left({\pi }_{L0},{\pi }_{L1}\right)=(\mathrm{0.2,\; 0.35})$ and $\left({\pi }_{S0},{\pi }_{S1}\right)=(\mathrm{0.3,\; 0.5})$, the BSDS design obtains the optimal design parameter at $\left(m,n,r_S\right)=(\mathrm{37,\; 74,\; 13})$, which has a smaller required first stage sample size $m$ and a larger required second stage sample size $n$ than that of the START design. The resulting ASN is 123.96 and is lower than the ASN of the START design (127.84). The resulting AEL is 57.12 months, which is about a year shorter than the AEL of the START design (69.04 months). When $\left({\pi }_{S0},{\pi }_{S1}\right)$ are set more aggressively to $(\mathrm{0.25,\; 0.5})$, the ASN for BSDS further reduces to 112.03 and the AEL reduces to 51.95 months. The smaller ASN is mainly driven by the smaller $m$ in the first stage due to the larger value of ${\pi }_{S0}$ and ${\pi }_{S1}$. The type I error rate for our design ${\alpha }_2$ is higher than that for START, but it is controlled below the pre-specified margin. From the results of both examples, we can conclude that the proposed BSDS design results in similar design characteristics, a comparable ASN and a shorter trial duration compared to the START design.

FIGURE 3.

Illustration of type I error rates under different assumptions for $p_S$ and $p_L$ under the null hypotheses.

5 |. DISCUSSION

In this paper, we propose an integrated phase II single-to-double arm transition design where the first stage hypothesis test is conducted based on the short-term endpoint. The subsequent second stage randomizes patients and compares the effectiveness of an experimental treatment with a standard treatment based on the long-term endpoint. Because the BSDS design allows for the incorporation of different endpoints at different stages, it can be seen as an extension of the existing START design, which is limited to using a single endpoint throughout the trial. The correlation between the two endpoints, which is unknown a priori, poses a major challenge for controlling the type I error. Theorem 1 addresses this issue by providing the maximum bound for the type I error rate under different scenarios and offering theoretical guidance on how to numerically search for the optimal design parameters. Our simulation studies have demonstrated that the performance of the proposed BSDS design is on par with the existing single-to-double arm transition design. Furthermore, our method achieves a comparable average sample size while significantly reducing the average expected trial length by incorporating a short-term endpoint in the interim analysis.

Our design requires that the relationship between the short-term and the long-term endpoints be well-understood by researchers so that the short-term response rate can be specified correctly. Successful implementation of the proposed design requires that the chosen short-term endpoint be more rapidly observed than the long-term endpoint, have a higher response rate, and be more immediately affected by the treatment. The selection and evaluation of short-term endpoints in clinical trials have been well-discussed by Wickstrom and Moseley.²²

The first stage of our design does not require a control arm, and the clinically uninteresting response rate in the hypothesis test is typically determined using historical data. Incorrect specification of historical information could limit the reliability of the trial. Several methods have been developed to accurately borrow historical information. For example, Ibrahim et al.²³ proposed the power prior approach, which uses historical information to construct an informative prior. Another popular approach is to use the robust meta-analytic-predictive prior, which is a mixture prior consisting of informative and non-informative components.²⁴ At the end of the trial, responses from the single-arm and randomized stages are pooled together to make the final decision. It should be cautioned that such a pooled comparison might not be unbiased if patients’ characteristics are different between the two stages. It is crucial to ensure that the patient characteristics are homogeneous across two stages. As suggested by Day and Altman,²⁵, one should double-blind researchers and patients to minimize the potential selection bias.

As avenues for future research, it is of interest to incorporate other methods for dealing with trial suspension into the single-to-double arm design. For example, Chen, Zhao, and Zhang²⁶ proposed a “double-check” strategy that allows a trial to continue enrollment after stage 1 without suspending the trial, where the early termination decision depends on both the stage 1 threshold and an additional rescue criterion. Combining such a method with single-to-double arm designs could be a topic of interest for future study. Another possible extension to the proposed design is to use both short-term and long-term endpoints in the final decision-making, as indicated by some phase II/III seamless designs.^27,28 This approach may lead to improved power when both endpoints are correlated.²⁷

APPENDIX: PROOF OF THEOREM 1

For simplicity, we first provide the detailed proof of Theorem 1 under the case of nested outcomes, then we describe how Theorem 1 holds under more general cases. Without loss of generality, we assume that $\left(n,m,r_S,\theta \right)$ is fixed, $0<p_L\le p_S<1$ and $0<q_L<1$. To show (2) holds in Theorem 1 for nested outcomes, we reparametrize $\mathrm{P}\mathrm{R}\left(p_S,p_L,q_L\right)$ as follows:

$$\begin{gathered} \text{PR}(p_S,p_L,q_L)=\sum _{y_L=0}^n\sum _{z_L=0}^{n}b(y_L;n,p_L)b(z_L;n,q_L) \\ \sum _{x_L=0}^m\sum _{x_S=max(r_S+1,x_L)}^m\left(\begin{array}{c}m\\ x_S\end{array}\right)\left(\begin{array}{c}{x}_S\\ x_L\end{array}\right){(p_S-p_L)}^{x_S-x_L}{p}_L^{x_L}{(1-p_S)}^{m-x_S}I\{\text{PP}(x_L,y_L,z_L)>\theta \} \\ =\sum _{y_L=0}^n\sum _{z_L=0}^n\sum _{x_L=0}^{m}b(y_L;n,p_L)b(z_L;n,q_L)b(x_L;m,p_L)I\{\text{PP}(x_L,y_L,z_L)>\theta \}\left\{\sum _{x_{SL}=\text{max}(0,r_S+1-x_L)}^{m-x_L}b(x_{S\overline{L}},m-x_L,p_{S\overline{L}})\right\} \end{gathered}$$

where $x_{S\bar{L}}$ denote the number of first stage patients who have a short-term responses but do not get a long-term response, and the response probability in the last binomial term $p_{S\bar{L}}=\left(p_S-p_L\right)/\left(1-p_L\right)$ lies inside $(\mathrm{0,\; 1})$.

To show $\mathrm{P}\mathrm{R}\left(p_S,p_L,q_L\right)$ is a monotonic function of $p_S$, it suffices to show $\mathrm{P}\mathrm{R}\left(p_S,p_L,q_L\right)$ is a monotonic function of $q_{S\bar{L}}$ because $p_L$ is a pre-fixed constant. It is observed that when $x_L\ge r_S+1$, the last summation term equals 1, and thus it is not a constant function of $q_{S\bar{L}}$. As a result, we only need to show the last term, ${\sum }_{x_{S\bar{L}}=\mathrm{m}\mathrm{a}\mathrm{x}\left(0,r_S+1-x_L\right)}^{m-x_L}b\left(x_{S\bar{L}},m-x_L,p_{S\bar{L}}\right)$, is a monotonically increasing function of $p_{S\bar{L}}$ when $x_L<r_S+1$. Denote $m-x_L=K$ and $r_S+1-x_L=k$, this is equivalent to show the cumulative distribution function (CDF) of the binomial random variable $x_{S\bar{L}}$, i.e., $F\left(k,K,p_{S\bar{L}}\right)={\sum }_{x_{S\bar{L}}=0}^{k}b\left(x_{S\bar{L}},K,p_{S\bar{L}}\right)$, is a monotonically decreasing function of $p_{S\bar{L}}$.

Denote $x_{S\bar{L}}^{\prime }\in \{0,\dots ,k\}$ a truncated variable of $x_{S\bar{L}}$ with probability mass function given by

$$\mathrm{P}\mathrm{r}\left(x_{S\bar{L}}^{\prime }=i\right)=b\left(i,K,p_{S\bar{L}}\right)/F\left(k,K,p_{S\bar{L}}\right),i=0,\dots ,k.$$

We take the derivative of $F\left(k,K,p_{S\bar{L}}\right)$ with respect to $p_{S\bar{L}}$ and obtain that

$$\begin{gathered} \frac{\partial F(k,K,p_{S\overline{L}})}{\partial p_{S\overline{L}}}=\sum _{i=1}^k\left(\begin{array}{c}K\\ i\end{array}\right)\left\{\frac{i}{p_{S\overline{L}}(1-p_{S\overline{L}})}{p}_{S\overline{L}}^i{(1-p_{S\overline{L}})}^{K-i}-\frac{K}{1-p_{S\overline{L}}}{p}_{S\overline{L}}^i{(1-p_{S\overline{L}})}^{K-i}\right\} \\ =\frac{F(k,K,p_{S\overline{L}})}{p_{S\overline{L}}(1-p_{S\overline{L}})}\sum _{i=1}^k\frac{b(i,K,p_{S\overline{L}})}{F(k,K,p_{S\overline{L}})}-\frac{p_{S\overline{L}}K}{p_{S\overline{L}}(1-p_{S\overline{L}})}F(k,K,p_{S\overline{L}}) \\ =\frac{F(k,K,p_{S\overline{L}})}{p_{S\overline{L}}(1-p_{S\overline{L}})}\left(\text{E}{x}_{S\overline{L}}^{\prime }-\text{E}{x}_{S\overline{L}}\right), \end{gathered}$$

where $\mathrm{E}{x}_{S\bar{L}}^{\prime }$ and $\mathrm{E}{x}_{S\bar{L}}$ are expectations of $x_{S\bar{L}}^{\prime }$ and $x_{S\bar{L}}$, respectively.

Since $x_{S\bar{L}}^{\prime }$ is the truncated version of $x_{S\bar{L}}$, we have $\mathrm{E}{x}_{S\bar{L}}^{\prime }<\mathrm{E}{x}_{S\bar{L}}$. Therefore, the derivative $\frac{\partial F\left(k,K,p_{S\bar{L}}\right)}{\partial p_{S\bar{L}}}<0$ and the CDF $F\left(k,K,p_{S\bar{L}}\right)$ is a monotonically decreasing function of $p_{S\bar{L}}$. In other words, in the case of nested endpoints, $\mathrm{P}\mathrm{R}\left(p_S,p_L,q_L\right)$ is an increasing function of $p_S$.

To show Theorem 1 holds under the general case, we similarly reparametrize $\mathrm{P}\mathrm{R}\left(p_S,p_{SL},p_L,q_L\right)$ as follows:

$$\begin{gathered} \text{PR}(p_S,p_{SL},p_L,q_L)=\sum _{y_L=0}^n\sum _{z_L=0}^{n}b(y_L;n,p_L)b(z_L;n,q_L)\sum _{x_L=0}^m\sum _{x_{SL}=0}^{x_L}\left(\begin{array}{c}m\\ x_L\end{array}\right)\left(\begin{array}{c}{x}_L\\ x_{SL}\end{array}\right){(p_L-p_{SL})}^{x_L-x_{SL}}{p}_{SL}^{x_{SL}}{(1-p_L)}^{m-x_L} \\ I\{\text{PP}(x_L,y_L,z_L)>\theta \}I\{(m-x_L+x_{SL})\ge (r_S+1)\}\sum _{x_{S\overline{L}}=\text{max}(0,r_S+1-x_L)}^{m-x_L}b(x_{S\overline{L}},m-x_L,p_{S\overline{L}}^{\prime }) \end{gathered}$$

where $p_{S\bar{L}}^{\prime }=\left(p_S-p_{SL}\right)/\left(1-p_L\right)$. The steps that remain would mimic these, and they are thus omitted.

DATA AVAILABILITY STATEMENT

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.