A Hybrid Phase I-II/III Clinical Trial Design Allowing Dose Re-Optimization in Phase III

Summary

Conventionally, evaluation of a new drug, A, is done in three phases. Phase I is based on toxicity to determine a “maximum tolerable dose” (MTD) of A, phase II is conducted to decide whether A at the MTD is promising in terms of response probability, and if so a large randomized phase III trial is conducted to compare A to a control treatment, C, usually based on survival time or progression free survival time. It is widely recognized that this paradigm has many flaws. A recent approach combines the first two phases by conducting a phase I-II trial, which chooses an optimal dose based on both ^efficacy and toxicity, and evaluation of A at the selected optimal phase I-II dose then is done in a phase III trial. This paper proposes a new design paradigm, motivated by the possibility that the optimal phase I-II dose may not maximize mean survival time with A. We propose a hybridized design, which we call phase I-II/III, that combines phase I-II and phase III by allowing the chosen optimal phase I-II dose of A to be re-optimized based on survival time data from phase I-II patients and the first portion of phase III. The phase I-II/III design uses adaptive randomization in phase I-II, and relies on a mixture model for the survival time distribution as a function of efficacy, toxicity, and dose. A simulation study is presented to evaluate the phase I-II/III design and compare it to the usual approach that does not re-optimize the dose of A in phase III.

1. Introduction

After a new treatment agent, A, is identified in pre-clinical studies, conventional clinical drug development and evaluation is carried out in three phases (Cancer.org, 2018). In phase I, the aim is to identify a dose, called the “maximum tolerable dose” (MTD), having acceptable toxicity probability. Phase I trials typically are small, with a wide variety of designs, including the 3+3 algorithm (Storer, 1989), continual reassessment method (O’Quigley et al., 1990), and escalation with overdose control (Babb et al., 1998). Efficacy of A at the MTD then is evaluated in phase II using the estimated probability π_E of a short-term event (“response”), such as 50% solid tumor shrinkage or complete remission of leukemia. Most phase II designs compare π_E(A) with A at the MTD to an assumed π_E(C) of a conventional therapy, C. Phase II trials often are small, and may include an early stopping rule if π_E(A) is poor compared to π_E(C). If A is found to be promising in phase II, this may motivate a randomized phase III trial of A versus C based on a long term outcome, such as survival time.

Many phase II designs have been published. Simon et al. (1985) proposed a randomized selection design for two or more experimental treatments. For single-arm phase II trials, two-stage designs were proposed by Simon (1989) based on response, and by Bryant and Day (1995) based on response and toxicity. Bayesian sequential designs were proposed by Thall and Simon (1994) for a binary response, and by Thall et al. (1995) for monitoring multiple outcomes. Lee and Liu (2005) used predictive probabilities for futility rules, and Yin et al., (2012) used adaptive randomization to favor empirically better treatment arms.

It now is recognized widely that the conventional phase I → phase II → phase III paradigm has many flaws, and has led to many negative phase III trials. Two studies (Bio, 2016; Arrowsmith, 2011) showed that only about 50% of phase III trials yield an improvement over standard therapy. Seruga, et al. (2015) discussed causes of failure in phase III, including insufficient evidence of anti-disease activity in early phase trials, disagreements about how phase II trials should be designed, and reliance on phase II efficacy events or other surrogates not associated with longer survival. Yuan, Nguyen, and Thall (2016, Chapter 1) discuss problems with the conventional phase I → phase II paradigm, mainly due to limited sample sizes and ignoring efficacy when determining an MTD in phase I.

Many alternatives have been proposed that create hybrid designs by combining conventional phases, most commonly phase I-II or phase II-III. Thall (2008) reviewed phase II-III designs and discussed problems with the conventional phase II → phase III paradigm. “Select-and-test” phase II-III designs, where two or more experimental agents are chosen in phase II and randomized against C in phase III while maintaining desired overall type I and type II error rates, are given by Thall et al. (1988), Schaid et al. (1990), Stallard and Todd (2003), and many others. A phase II-III design proposed by Inoue, Thall and Berry (2004) uses both an early efficacy (response) indicator, Y_E, and survival time, Y_S. Denote π_E = Pr(Y_E = 1), the probability density function (pdf) of Y_S by f_S(t), and the conditional pdf of [Y_S | Y_E] by f_S(t | Y_E = y) for y = 0, 1. Their approach relies on a mixture model of the general form

$$f_S(t)=f_S(t|Y_E=1){\pi }_E+f_S(t|Y_E=0)(1-{\pi }_E).$$ (1)

Denote the indicator of early toxicity by Y_T and π_T = Pr(Y_T = 1). Because phase I designs use Y_T but ignore Y_E when choosing a MTD, they are likely to choose a dose having reasonable π_T but ineffectively low π_E. For example, consider a dose-finding scenario with five doses, true toxicity probabilities (.05, .10, .20, .30, .35), and true efficacy probabilities (.05, .10, .20, .30, .60). If the CRM is used with target toxicity probability .30, this most likely will select dose 4 as optimal. By ignoring Y_E, however, dose 5 is chosen less frequently, despite the fact that it has only a .05 higher π_T than dose 4 but doubles π_E from .30 to .60. Phase I-II designs are motivated, in part, by the desire to overcome this sort of problem. Examples include the two-stage design of Hoering et al. (2011), studying combination therapies (Huang et al., 2006), using the odds ratio between π_E and π_T (Yin et al, 2006), and basing decisions on elicited numerical utilities of the possible elementary events determined by efficacy and toxicity (Thall and Nguyen, 2012). Thall and Cook (2004) proposed, and Thall et al. (2014) refined, the so-called “Eff-Tox” phase I-II design based on maximizing an estimate of an efficacy-toxicity trade-off function, ϕ(π_E, π_T). The function ϕ(π_E, π_T) increases in π_E, decreases in π_T, and quantifies the desirability of each probability pair (π_E, π_T).

This article presents a new Bayesian hybrid design that combines a phase I-II design followed by a modified phase III design, based on both early and late outcomes. We will call this a phase I-II/III trial design. For simplicity, we will use survival time, Y_S, as the long term outcome, although progression-free survival (PFS) time will work in precisely the same way. Our approach relies on a mixture model for the distribution of Y_S that generalizes the model (1) by including both efficacy and toxicity indicators, (Y_E, Y_T), to characterize early outcome. After phase I-II and an initial stage of phase III have been completed, the phase I-II/III design may re-optimize the dose of the experimental agent A based on mean survival time, μ_S. This approach hybridizes the phase I-II → phase III paradigm, in which dose-finding for A is done using (Y_E, Y_T) in phase I-II, rather than using only Y_T for dose-finding as in the more conventional phase I → phase II → phase III paradigm.

Our proposed phase I-II/III design has K ⩾ 3 stages. In stage 1, a phase I-II trial is conducted based on the short term binary indicators (Y_E, Y_T), including adaptive randomization (AR) among doses of A based on the dose desirability criterion ϕ. The use of AR reduces the risk of getting stuck at a suboptimal dose in phase I-II. It addresses the “exploration versus exploitation” or “stickiness” problem, which is well known in sequential analysis (Sutton and Barto, 1998; Azriel, et al., 2010). AR improves the reliability of our proposed phase I-II/III design because it obtains more data on doses that may be sub-optimal in terms of the phase I-II criterion ϕ based on (Y_E, Y_T) but optimal in terms of μ_S.

Denote A given at x by A(x). At the end of phase I-II (stage 1), an optimal dose ${\widehat{x}}_{ET}^{\text{opt}}$ of A based on ϕ is determined. In stage 2, phase III begins with patients randomized fairly to C and $A\left({\widehat{x}}_{ET}^{\text{opt}}\right)$ Phase I-II patients are followed to observe their times of death or follow up. After a pre-specified number of deaths, $n_2^{\ast }$, have been observed from patients receiving C or $A\left({\widehat{x}}_{ET}^{\text{opt}}\right)$ in stage 2, all (Y_E,Y_T) and survival data of patients treated with A in stages 1 and 2 are used to determine an optimal dose ${\widehat{x}}_S^{\text{opt}}$ such that $A\left({\widehat{x}}_S^{\text{opt}}\right)$ maximizes μ_S, for use in the rest of phase III. The re-optimized dose ${\widehat{x}}_S^{\text{opt}}$ may or may not be the same as ${\widehat{x}}_{ET}^{\text{opt}}$. Stages 3, …, K are a randomized group sequential trial with up to K − 2 tests comparing the mean survival times of $A\left({\widehat{x}}_S^{\text{opt}}\right)$ versus C. To provide a concrete illustration, for stage 1 we use the Eff-Tox phase I-II design of Thall et al. (2014), extended to include AR.

The rest of the paper is organized as follows. In Section 2, the data structure, models, and decision criteria are presented. Section 3 presents details of trial conduct. Section 4 describes possible decisions, outcomes, and potential consequences of re-optimizing dose versus the conventional approach of using ${\widehat{x}}_{ET}^{\text{opt}}$ in phase III. Section 5 presents results of simulation study to compare the phase I-II/III design to the phase I-II → phase III paradigm. Section 6 concludes with a discussion. A computer program to implement the phase I-II/III design is available on CRAN in the package Phase123.

2. Data Structure, Models, and Decision Criteria

Given raw doses d = (d₁, ⋯, d_j) of the experimental agent A, denote the standardized doses by $x_j=\left(d_j-\overline{\mathit{d}}\right)/sd\left(\mathit{d}\right)$ for j = 1, …J. Let $Y_S^o$ denote the observed time to death or administrative censoring and $\delta =I\left(Y_S=Y_S^o\right)$. Denote the parameters for the distribution of [Y_E, Y_T|x] by θ_ET, and the parameters for the distribution of [Y_S|Y_E, Y_T, x] by θ_S.

The Eff-Tox design is reviewed in Web Appendix Section A. Briefly, for each m = E,T, and x_j, it is assumed that π_m(x_j, θ_ET) = P(Y_m = 1| x_j, θ_ET) = logit⁻¹ {η_m(x_j, θ_ET)}, with η_T (x_j, θ_ET) = τ_T,₁ + τ_T,₂x_j and ${\eta }_E\left(x_j,{\mathit{\theta }}_{ET}\right)={\tau }_{E,1}+{\tau }_{E,2}{x}_j+{\tau }_{E,3}{x}_j^2$, with τ_T,₂ > 0, so that π_T(x_j, θ_ET) increases with x_j, but π_E(x_j, θ_ET) may be non-monotone. An association parameter ψ determines the joint distribution of (Y_E, Y_T) from their marginals using a copula, so θ_ET = (τ_T,₁, τ_T,₂, τ_E,₁, τ_E,₂, τ_E,₃, ψ). These parameters are assumed to be independent with priors ψ ~ N(0, 1), τ_E,₂ ~ N(0, .20), and ${\tau }_{m,r}~N\left({\stackrel{\sim }{\mu }}_{m,r},{\stackrel{\sim }{\sigma }}_{m,r}^2\right)$ for m = E,T and r = 1, 2. Numerical values of $\left({\stackrel{\sim }{\mu }}_{m,r},{\stackrel{\sim }{\sigma }}_{m,r}\right)$ for m = E, T, r = 1, 2 are determined from elicited means of π_m(x_j, θ_ET), for j = 1, ⋯, J, m = E, T, and a desired prior effective sample size. Adaptive dose-finding decisions are based on a trade-off function ϕ(π_E, π_T) for π ∈ [0, 1]².

Denote ϕ_j = ϕ[E{π_E(x_j, θ_ET), π_T(x_j, θ_ET)}|data] for each dose x_j at any point during phase I-II based on the current data. The estimated optimal dose in an Eff-Tox trial is ${\widehat{x}}_{ET}^{\text{opt}}={max}_j\left\{{\varphi }_j\right\}$. To extend this design to include AR, rather than choosing ${\widehat{x}}_{ET}^{\text{opt}}$ for each cohort during phase I-II, we adaptively randomize the next cohort to dose x_j with probability

$$\frac{\exp \{({\varphi }_j-\overline{Q})/sd(\mathit{Q})\}}{{\displaystyle \sum _{r:{\varphi }_r\in \mathit{Q}}\exp \{({\varphi }_r-\overline{Q})/sd(\mathit{Q})\}}},$$

where Q is the current set of posterior mean desirabilities, ϕ_j, of doses that are acceptably safe and efficacious. This shrinks the selection probability of less desirable doses toward 0 while allowing selection of doses that are suboptimal in terms of ϕ. After the (Y_E,Y_T) outcomes of all N_ET patients in phase I-II have been evaluated, $A\left({\widehat{x}}_{ET}^{\text{opt}}\right)$ based on the final phase I-II data is moved forward to stage 2, which is the first portion of phase III.

In the phase I-II/III design, we define two different types of truly optimal doses of A. Let ${\mathit{\theta }}_m^{\mathit{true}}$ denote an assumed true value of θ_m for m = ET or S. The truly optimal dose that maximizes $\varphi \left\{{\pi }_E\left(x,{\mathit{\theta }}_{ET}^{\mathit{true}}\right),{\pi }_T\left(x,{\mathit{\theta }}_{ET}^{\mathit{true}}\right)\right\}$ is $x_{ET}^{\text{opt}}$. The truly optimal dose that maximizes the mean survival time ${\mu }_S\left(x,{\mathit{\theta }}_S^{\mathit{true}}\right)$ is $x_S^{\text{opt}}$. Let k =1, …., K index the stages of the phase I-II/III trial. Thus, k = 1 indexes the phase I-II trial, k = 2 indexes the first portion of phase III at the end of which the dose of A may be re-optimized based on μ_S, and k = 3, …, K index the subsequent group sequential stages in phase III for comparing $A\left({\widehat{x}}_S^{\text{opt}}\right)$ to C. Thus, there are up to K − 2 group sequential comparisons in phase III. Let ${\mathcal{D}}_{I-II,k}$ and ${\mathcal{D}}_{\mathit{III},k}$ denote the data for patients at the end of stage k, from the phase I-II and phase III portions of the trial, respectively. Therefore, ${\mathcal{D}}_{I-II,1}$ consists only of the (x, Y_E, Y_T) data from phase I-II patients, while ${\mathcal{D}}_{I-II,2}$ also includes these patients’ survival time data $\left(Y_S^o,\delta \right)$, up to the time at which the decision of whether to switch the dose based on mean survival time is made. ${\mathcal{D}}_{\mathit{III},1}$ does not exist because phase III has not begun in stage 1. The re-optimized dose ${\widehat{x}}_S^{\mathit{opt}}$ is chosen based on ${\mathcal{D}}_{I-II,2}\cup {\mathcal{D}}_{\mathit{III},2}$, which includes all (x, Y_E, Y_T) and $\left(Y_S^0,\delta \right)$ data at the end of stage 2.

Since the phase I-II → phase III paradigm uses ${\widehat{x}}_{ET}^{\text{opt}}$ throughout phase III, the primary motivation for our design is the possibility that $x_{ET}^{\text{opt}}\ne x_S^{\text{opt}}$, and that re-optimizing the dose of A may produce larger μ_S by comparing C to $A\left({\widehat{x}}_S^{\text{opt}}\right)$ rather than $A\left({\widehat{x}}_{ET}^{\text{opt}}\right)$ in the group sequential trial. To evaluate the effects of re-optimizing the dose of A based on μ_S during the first part of phase III, we require models for [Y_E, Y_T|x] and [Y_S|Y_E, Y_T, x], in order to formulate a mixture model for [Y_S|x]. This will include the effects of x on the indicators (Y_E, Y_T), and the effects of (Y_E, Y_T) and x on the hazard function of Y_S. Let π(y_E, y_T|x, θ_ET) denote the probability distribution of (Y_E, Y_T) at dose x, where (y_E, y_T) ∈ {0, 1}. Let f_S|E,_T(y_S | y_E, y_T, x, θ_S) denote the conditional pdf of Y_S given the early binary outcomes and dose x of A. The mixture pdf of Y_S for patients treated with A(x) is

$$f_S(y_S|x,{\mathit{\theta }}_S,{\mathit{\theta }}_{ET}){\displaystyle \sum _{y_E=0}^1{\displaystyle \sum _{y_T=0}^1{f}_{S|E,T}(y_S|y_E,y_T,x,{\mathit{\theta }}_S)\pi }(y_E,y_T|x,{\mathit{\theta }}_{ET}).}$$ (2)

The conditional mean survival time given (y_E, y_T) of a patient treated with A(x) is

$${\mu }_{S,A(x)}(y_E,y_T,{\mathit{\theta }}_S)={\displaystyle {\int }_0^{\mathrm{\infty }}{y}_S{f}_{S|E,T}(y_S|y_E,y_T,x,{\mathit{\theta }}_S)d{y}_S.}$$ (3)

At the end of stage 2 of phase I-II/III, we choose ${\widehat{x}}_S^{\text{opt}}$ based on all observed data, where ${\widehat{x}}_S^{\text{opt}}$ maximizes the posterior mean of the parametric mean survival time

$${\mu }_{S,A(x)}({\mathit{\theta }}_S,{\mathit{\theta }}_{ET})={\displaystyle \sum _{y_E=0}^1{\displaystyle \sum _{y_T=0}^1{\mu }_{S,A(x)}}(y_E,y_T,{\mathit{\theta }}_S)\pi }(y_E,y_T|x,{\mathit{\theta }}_{ET})$$ (4)

at A(x). Conventionally (Y_E, Y_T) are used as surrogates for Y_S in choosing a dose ${\widehat{x}}_{ET}^{\text{opt}}$ in phase I-II, but (Y_E, Y_T) are ignored when modeling survival in phase III.

We assume that the distribution of [Y_S | Y_E, Y_T, x] has the Cox type hazard function

$$h(t|Y_E,Y_T,x,{\mathit{\theta }}_S)=h_0(t)\exp \{{\beta }_{1}x+{\beta }_2{x}^2-e^{{\beta }_E}{Y}_E+e^{{\beta }_T}{Y}_T\},t>0.$$ (5)

We assume that β₁, β₂, β_E, and β_T are independent with identical non-informative N(0, 100) priors. For robustness, we assume that the baseline hazard is piecewise exponential with h₀(t) = exp(λ_l) for t ∈ (t_l, t_l+1] under the partition t₀ = 0 < t₁ < …. < t_L+1 = max(Y_s^o). We allow the dimension L of the baseline hazard to vary, with prior L ~ Poi(ζ_S) and assume that the locations of the split points t vary according to the even order statistics with a uniform distribution of size 2L, as in Lee et al. (2015) and Chapple et al. (2017). This prevents obtaining intervals in h₀(t) having few events for estimating λ_l. We suggest values of ζ_S ∈ {3, 4, 5, 6, 7}, since most hazard shapes can be approximated very accurately with 1 to 5 pieces. The resulting posterior distribution is not sensitive to the choice of ζ_S in this range for sample sizes greater than 50. We assume a normal prior with mean 0 and variance 25 for λ₁, denoted λ₁ ~ N(0, 25), and borrow strength when L > 1 for adjacent intervals via the prior ${\lambda }_l~N\left({\lambda }_{1-1},{\sigma }_{\lambda }^2\right)$, with the prior of σ_λ proportional to 1/σ_λ. The variance of λ₁ ensures posterior hazard values seen in practice, while maintaining prior non-informativeness.

Denote ${\widehat{\mu }}_{S,A\left(x\right)}=E\left(Y_S|x,{\mathcal{D}}_{I-II,2}\cup {\mathcal{D}}_{\mathit{III},2}\right)$, the posterior mean survival time for A(x) given the data from phases I-II and III at the end of stage 2. We compute this quantity under the mixture model (2) by estimating the posterior mean survival time

$${\widehat{\mu }}_{S,A(x)}(y_E,y_T|{\mathcal{D}}_{I-II,2}\cup {\mathcal{D}}_{\mathit{III},2})=E\{{\mu }_{S,A(x)}(y_E,y_T,{\mathit{\theta }}_S)|{\mathcal{D}}_{I-II,2}\cup {\mathcal{D}}_{\mathit{III},2}\}$$

for each pair (y_E, y_T) ∈ {0, 1}, under the formula (3), and computing the posterior mean

$$\widehat{\pi }(y_E,y_T|x,{\mathcal{D}}_{I-II,2}\cup {\mathcal{D}}_{\mathit{III},2})=E\{(y_E,y_T|x,{\mathit{\theta }}_{ET})|{\mathcal{D}}_{I-II,2}\cup {\mathcal{D}}_{\mathit{III},2}\}$$

of each bivariate probability under the Eff-Tox model given in Web Appendix A.

Since there will be limited survival time follow up information after $n_2^{\ast }$ events, the design only evaluates the means until the maximum observed patient follow up time. The trial is continued after $n_2^{\ast }$ patient events using the dose ${\widehat{x}}_S^{\text{opt}}$ of A having the highest posterior mean ${\widehat{\mu }}_S^{\text{opt}}={max}_{x_j}\left\{{\widehat{\mu }}_{S,A\left(x_j\right)}\right\}$. After making this decision, the design does not use data from patients who were treated at doses $x_j\ne {\widehat{x}}_S^{\text{opt}}$. After obtaining values of $n_3^{\ast },\dots ,n_K^{\ast }$ from East 6 statistical software (2016), $n_2^{\ast }$ is chosen such that the design can switch doses with high accuracy, but can still yield high power for phase III trials, given the truly optimal dose $x_S^{\text{opt}}$ has been selected. Suitable values of $n_2^{\ast }$ can be determined using the function SimPhase123 in the package Phase123. This approach will result in a larger sample size of patients in the C arm being compared to $A\left({\widehat{x}}_S^{\text{opt}}\right)$ if ${\widehat{x}}_{ET}^{\text{opt}}\ne {\widehat{x}}_S^{\text{opt}}$ We use Markov chain Monte Carlo to obtain posterior distributions for θ_ET and θ_S, using 2000 iterations and 1000 discarded as burnim This gives good convergence of the parameters, shown by the posterior of L settling on one or two values as well as traceplots for the parameters λ|L, s|L, and the coefficients in the linear terms of the Eff-Tox and survival hazard models. A detailed account of computational algorithms used to simulate posterior samples is given in Web Appendix B.

3. Trial Conduct

In this section, we give specific rules for conducting a phase I-II/III clinical trial. Each of the computer functions described below is contained in the R package Phase123, available on CRAN, including documentation of inputs and examples. Additional information on the trial parameters is given in Web Appendix C and a tutorial on several of the functions is given in Web Appendix D. When designing a phase I-II/III trial, the statistician should consult with the physician to establish design parameters, such as ϕ, maximum sample sizes N_ET and N_S, and the number of comparative tests K − 2 following dose re-optimization. The group sequential boundaries for stopping the trial due to futility ${\underset{\_}{u}}_k$ or superiority ${\overline{u}}_k$ may be obtained using East 6 statistical software (2016), specifying a null value of μ_C, desired improvement Δ, type I error, power under the alternative, maximum sample size N_S, and information proportions for determining $n_k^{\ast }$ for k = 3, …, K. If no futility decision is desired at look k then ${\underset{\_}{u}}_k=0$. The information proportions used to determine $n_k^{\ast }$ should be large enough (> 30%) to avoid making unreliable decisions based on a small amount of patient data if a dose is re-optimized for A.

The phase I-II/III design parameters must be calibrated to obtain good operating characteristics (OCs) under a reasonable array of possible scenarios. A smaller value of $n_2^{\ast }$ (≤ 20% of the total information proportion) may be obtained by simulating the phase I-II/III trial under sets of different (a) Eff-Tox scenarios quantifying effects of x on (Y_E,Y_T), (b) effects of (x,Y_E,Y_T) on survival, and (c) survival distributions. The stage 2 sample size $n_2^{\ast }$ should be set by examining the design’s OCs for several different values, to find $n_2^{\ast }$ (1) large enough to give a high probability of selecting the optimal dose, but (2) small enough so, given that the design switches to a true optimal dose in stage 2, it has good generalized power figures. This can be done using the function SimPhase123. Specific rules for conducting a phase I-II/III trial are as follows:

1. Enroll the first cohort of patients in the phase I-II portion at the lowest dose. For each subsequent cohort until N^F patients have been treated, use the function AssignEffTox to obtain the next dose to give.

2. Once N^F patients have been enrolled in phase I-II, use the function RandomEffTox to adaptively randomize the next cohort of patients among acceptable doses, which allows doses that are empirically suboptimal in terms of ϕ(π_E, π_T) to be chosen.

3. After N_ET patients have been enrolled in phase I-II and their efficacy and toxicity outcomes have been evaluated, use the function AssignEffTox to obtain the dose ${\widehat{x}}_{ET}^{\text{opt}}$ to continue to phase III.

4. Start phase III, randomizing patients equally between C and $A\left({\widehat{x}}_{ET}^{\text{opt}}\right)$.

5. After $n_2^{\ast }$ deaths have been observed, use the function Reoptimize to determine the dose ${\widehat{x}}_{ET}^{\text{opt}}$ to continue with for the remainder of the trial.

6. Remove any patients treated with ${\widehat{x}}_{ET}^{\text{opt}}$ from consideration if the dose was switched and begin randomizing patients between C and $A\left({\widehat{x}}_S^{\text{opt}}\right)$.

7. For each stage k = 3, …, K, after $n_k^{\ast }$ deaths occur, do two-sided tests for superiority or futility using the logrank test in R. Denoting the Z-score corresponding to the logrank statistic by Z, for futility bound ${\underset{\_}{u}}_k$ and superiority bound ${\overline{u}}_k$, stop the trial if

   $$|Z|>{\overline{u}}_k\text{for}\text{superiority}\text{or}|Z|<{\underset{\_}{u}}_k\text{for}\text{futility}.$$

8. Stop accrual after N_S patients have been enrolled in the phase III portion, including patients treated with a dose that is no longer considered optimal.

4. Possible Trial Outcomes

Before presenting our simulation results, we discuss possible design decisions and comment on each under different true states of nature. Because the phase I-II/III design may change the phase I-II selected dose of A in phase III before comparing A to C, the sequence of decisions that it makes may be correct and optimal, correct but suboptimal, wrong, or disastrously wrong, depends on $x_S^{\text{opt}}$ and $x_{ET}^{\text{opt}}$, their estimates, and whether ${\mu }_{A\left(x_S^{\text{opt}}\right)}={\mu }_C$ or ${\mu }_{A\left(x_S^{\text{opt}}\right)}⩾{\mu }_C+\mathrm{\Delta }$. Since more than one dose of A may provide the desired improvement in μ_S of at least Δ over C, we denote the set of all such doses by $X^{\text{opt}}=\{x_j:{\mu }_{A(x_j)}⩾{\mu }_C+\mathrm{\Delta }\}$. We define the generalized power (GP) to be the probability of (1) selecting a dose x_j ∈ X^opt in stage 2 and (2) declaring A(x_j) superior to C in one of stages 3, ⋯, K. The GP is the sum over x_j ∈ X^opt of the probability of selecting x_j and declaring A(x_j) superior to C. If X^opt contains more than one dose, then the GP is larger than the probability of the best possible decision, which is to select the optimal dose $x_S^{\text{opt}}\in X^{\text{opt}}$ that maximizes μ_S with A and declare $A\left(x_S^{\text{opt}}\right)$ superior to C. We denote the probability of making this best decision by γ₁ and the GP by γ₂. Thus, γ₁ ≤ γ₂, with γ₁ = γ₂ if X^opt contains exactly one dose, which in this case must be $x_S^{\text{opt}}$.

To help sort this out, Table 1 provides explanatory comments on scenarios in stages k = 1 (phase I-II) and k = 2 (the first portion of phase III) regarding the true relationship between the optimal doses $x_S^{\text{opt}}$ and $x_{ET}^{\text{opt}}$ and their posterior estimates ${\widehat{x}}_S^{\text{opt}}$ and ${\widehat{x}}_{ET}^{\text{opt}}$. If ${\widehat{x}}_{ET}^{\text{opt}}={\widehat{x}}_S^{\text{opt}}$, then the phase I-II/III and phase I-II → phase III designs make equivalent decisions. However, if ${\widehat{x}}_{ET}^{\text{opt}}\ne {\widehat{x}}_S^{\text{opt}}$, then switching provides a potential advantage. In this case survival data from patients who were treated with ${\widehat{x}}_{ET}^{\text{opt}}$ during phase III are no longer relevant. Depending on the accrual rate, maximum sample size N_S, and number of patient events $n_2^{\ast }$ needed to re-optimize dose, this may result in 30 to 100 patients being treated at doses no longer considered a part of the trial as phase III proceeds.

Table 1.

Possible relationships between ${\widehat{x}}_{ET}^{\mathit{opt}}$, $x_{ET}^{\mathit{opt}}$, ${\widehat{x}}_S^{\mathit{opt}}$, and $x_S^{\mathit{opt}}$, including comments related to the phase I-II → phase III and phase I-II/III designs. ${\widehat{x}}_{ET}^{\mathit{opt}}\stackrel{?}{=}{x}_{ET}^{\mathit{opt}}$ refers to whether or not the optimal dose is selected at the end of phase I-II based on ϕ, and $x_{ET}^{\mathit{opt}}\stackrel{?}{=}{x}_S^{\mathit{opt}}$ refers to whether the optimal dose based on μ_S is the same as that based on ϕ

${\widehat{x}}_{ET}^{\text{opt}}\stackrel{?}{=}{x}_{ET}^{\text{opt}}$	$x_{ET}^{\text{opt}}\stackrel{?}{=}{x}_S^{\text{opt}}$	Comments
${\widehat{x}}_{ET}^{\text{opt}}=x_{ET}^{\text{opt}}$	$x_{ET}^{\text{opt}}=x_S^{\text{opt}}$	The optimal dose in terms of μS was selected in phase I-II, so it is not desirable to switch doses at stage 2. In this scenario, the phase I-II/III design cannot provide an improvement over phase I-II → III.
${\widehat{x}}_{ET}^{\text{opt}}=x_{ET}^{\text{opt}}$	$x_{ET}^{\text{opt}}\ne x_S^{\text{opt}}$	The dose selected in phase I-II is optimal in terms of ϕ but is not optimal in terms of μS. This illustrates the advantage of the phase I-II/III design over phase I-II → III design.
${\widehat{x}}_{ET}^{\text{opt}}\ne x_{ET}^{\text{opt}}$	$x_{ET}^{\text{opt}}=x_S^{\text{opt}}$	The dose selected in phase I-II is suboptimal based on ϕ, but the optimal doses in terms of ϕ and μS are identical. This scenario illustrates the advantage of the phase I-II/III design over phase I-II → III design.
${\widehat{x}}_{ET}^{\text{opt}}\ne x_{ET}^{\text{opt}}$	$x_{ET}^{\text{opt}}\ne x_S^{\text{opt}}$	The dose selected in phase I-II is suboptimal based on ϕ, but the optimal doses in terms of ϕ and μS are not identical. This scenario illustrates the advantage of the phase I-II/III design over phase I-II → III design.

After choosing ${\widehat{x}}_S^{\text{opt}}$, the phase I-II/III design makes group sequential decisions comparing $A\left(x_S^{\text{opt}}\right)$ to C, so the decisions in phase III depend on the selected ${\widehat{x}}_S^{\text{opt}}$. But it may not be the case that ${\widehat{x}}_S^{\text{opt}}=x_S^{\text{opt}}$. That is, the design may not choose the truly optimal dose in terms of mean survival time in stage 2. Table 2 lists possible decisions of a phase I-II/III design in stages k = 2, …, K and how each decision may be viewed in terms of $x_S^{\text{opt}}$. Table 2 is ordered with the best outcomes listed first and the worst listed last, with outcomes 1, 2, and 3 being good and outcomes 4 and 5 being bad. In outcome 1, the design declares the dose that increases μ_S the most to be superior to C. In outcome 2, a dose of A is selected that provides a clinically meaningful improvement ⩾ Δ in μ_S compared to C, but the best dose of A is not chosen, so the decision is correct but the dose has not been truly optimized. Outcome 3 represents a correct decision, but it does not improve μ_S since it declares C superior to or equivalent to $A\left({\widehat{x}}_S^{\text{opt}}\right)$. Outcome 4 gives a false positive result, including cases where the design wrongly chooses an inferior dose for which ${\mu }_{A(x_j)}<{\mu }_C$, which is worse than a conventional type I error. Outcome 5 represents the worst possible case, since not only does the design wrongly conclude that the chosen dose gives $A\left({\widehat{x}}_S^{\text{opt}}\right)$ superior to C, but it might have obtained a successful trial result if it had correctly selected $x_S^{\text{opt}}$ in stage 2.

Table 2.

Possible Phase I-II/III Trial Outcomes, O. $x_S^{\mathit{opt}}$ is the truly optimal dose in terms of μ_A(x) and Δ is the desired improvement over μ_C. Column 2 gives the two trial decisions, the first row for selecting ${\widehat{x}}_s^{\mathit{opt}}$ and the second row for determining superiority, inferiority, or futility, with $A\left({\widehat{x}}_S^{\mathit{opt}}\right)\succ C$ indicating that $A\left({\widehat{x}}_S^{\mathit{opt}}\right)$ is declared superior to C, and $C\ge |A\left({\widehat{x}}_S^{\mathit{opt}}\right)$ indicating that the trial is stopped due to either superiority of C or futility.

O	Decision	Truth	Comments
1	$\begin{array}{l}{\widehat{x}}_S^{\text{opt}}=x_S^{\text{opt}}\\ A\left({\widehat{x}}_S^{\text{opt}}\right)\succ C\end{array}$	${\mu }_{A\left(x_S^{\text{opt}}\right)}>{\mu }_C+\mathrm{\Delta }$	This is the generalized power event at the optimal dose $x_S^{\text{opt}}$. The design correctly selects $x_S^{\text{opt}}$ as optimal and declares $A\left(x_S^{\text{opt}}\right)$ superior to C.
2	$\begin{array}{l}{\widehat{x}}_S^{\text{opt}}\ne x_S^{\text{opt}}\\ A\left({\widehat{x}}_S^{\text{opt}}\right)\succ C\end{array}$	$\begin{array}{l}{\mu }_{A\left({\widehat{x}}_S^{\mathit{opt}}\right)}>{\mu }_C+\mathrm{\Delta }\\ {\mu }_{A\left(x_S^{\text{opt}}\right)}>{\mu }_{A\left({\widehat{x}}_S^{\text{opt}}\right)}\end{array}$	This is a generalized power event in a case where the design correctly concludes $A\left({\widehat{x}}_S^{\text{opt}}\right)$ is superior to C but ${\widehat{x}}_S^{\text{opt}}$ is suboptimal, so it could have improved survival more had if it chosen the truly optimal dose $x_S^{\text{opt}}$.
3	$\begin{array}{l}{\widehat{x}}_S^{\text{opt}}=\text{any}{x}_j\\ C\ge A\left({\widehat{x}}_S^{\text{opt}}\right)\end{array}$	${\mu }_{A\left(x_S^{\text{opt}}\right)}\le {\mu }_C$	This is a correct conclusion, but the phase I-II/III design will require an increased sample size compared to the phase I-II → III design due to correctly switching to $x_S^{\text{opt}}$.
4	$\begin{array}{l}{\widehat{x}}_S^{\text{opt}}=\text{any}{x}_j\\ A\left({\widehat{x}}_S^{\text{opt}}\right)\succ C\end{array}$	${\mu }_{A\left(x_S^{\text{opt}}\right)}\le {\mu }_C$	This is a false positive conclusion. While the design may pick the best dose of A, it incorrectly concludes that A at that dose is superior to C.
5	$\begin{array}{l}{\widehat{x}}_S^{\text{opt}}\ne x_S^{\text{opt}}\\ A\left({\widehat{x}}_S^{\text{opt}}\right)\succ C\end{array}$	$\begin{array}{l}{\mu }_{A\left({\widehat{x}}_S^{\text{opt}}\right)}\le {\mu }_C\\ {\mu }_{A\left(x_S^{\text{opt}}\right)}⩾{\mu }_C+\mathrm{\Delta }\end{array}$	This is a disastrous false negative conclusion. The design chooses a suboptimal dose based on μS and incorrectly concludes $A\left({\widehat{x}}_S^{\text{opt}}\right)$ is inferior to C; instead of correctly selecting $x_S^{\text{opt}}$ and declaring $A\left(x_S^{\text{opt}}\right)$ superior to C.

These same decisions and interpretations are made in the conventional phase I-II → phase III paradigm, with the difference that ${\widehat{x}}_S^{\text{opt}}$ is replaced with ${\widehat{x}}_{ET}^{\text{opt}}$. Compared to this conventional design, allowing the optimal dose to be switched in the phase I-II/III design makes selecting ${\widehat{x}}_S^{\text{opt}}=x_S^{\text{opt}}$ more likely, which increases the probabilities of outcomes 1 and 2 and decreases the probabilities of the disastrous outcome 5. Under outcome 3, the phase I-II/III design is likely to treat more patients because it is more likely to correctly pick the dose ${\widehat{x}}_S^{\text{opt}}$ having the largest μ_S, thus making stopping the trial early for superiority of C or futility less likely. It will be more likely to switch to the dose having the longest mean survival time for outcome 4, however, which makes a false positive event more likely.

5. Simulation Study

To perform a simulation study comparing the phase I-II/III design to the phase I-II → phase III paradigm, we first specify three different Eff-Tox scenarios, consisting of true efficacy and toxicity dose-probability vectors. We will use these to specify different relationships between (Y_E, Y_T) and Y_S. We evaluate the design with J = 5 doses using raw dose values (d₁, ⋯, d₅) = (1, 2, 3, 3.5, 5). For this study, each patient’s (Y_E,Y_T) are evaluated in one month, and we assume for simplicity that no patients die before this month long window. For a dose x_j chosen in phase I-II, we test the null hypothesis $H_0:{\mu }_C={\mu }_{A(x_j)}=24$ months versus $H_0:{\mu }_C\ne {\mu }_{A(x_j)}$ with target ${\mu }_{A(x_S^{\text{opt}})}=36$ months, a Δ = 12 month improvement.

To implement phase I-II using the Eff-Tox design, the three equivalent (π_E, π_T) pairs used to establish the desirability function ϕ were (.35, 0), (.70, .40) and (1, .75). The contour created by these three pairs is seen in Web Figure 1. The upper limit on π_T was ${\overline{\pi }}_T=.40$and the lower limit on π_E was ${\underset{\_}{\pi }}_E=.30$. The threshold on the posterior probability that π_E > .30 and π_T < .40 was set to be p_E = p_T = .10 for both acceptability rules. Patients were treated in cohorts of size 3, with up to N_ET = 60 patients enrolled in phase I-II (stage 1). We calibrated the phase I-II hyperparameters to have prior effective sample size .90 as suggested by Yuan, Nguyen and Thall (2016). We used prior mean toxicity probabilities of (.05, .10, .15, .20, .30) and mean efficacy probabilities (.20, .40, .60, .65, .70) for the five doses, to produce the hyperparameter means (−4.23, 3.1, .02, 3.45, 0, 0) and standard deviations (3.13, 3.12, 2.68, 2.69, .2, 1) for the prior of θ_ET. The EffTox program is freely available on the MDAnderson biostatistics software page.

For the phase I-II portion of the simulated trials, patients were treated in cohorts of size three and assigned doses after the previous cohort was fully evaluated, assuming an accrual rate of five patients per month with adaptive randomization begun after N^F = 15 patients. The three simulation scenarios’ assumed true π_E(x_j) and π_T(x_j) are given in Table 3, with their selection percentages, true ϕ values, and numbers of patients treated, based on 5,000 simulated trials using the EffTox program.

Table 3.

Eff-Tox Scenarios. True outcome probabilities (π_E, π_T)^TR, desirabilities ϕ{(π_E, π_T)^TR}, and operating characteristics for the usual (non-adaptively randomized) EffTox phase I-II trial design.

Scenario	Value	1	2	3	4	5
1	(πE, πT)TR	(.20, .10)	(.40,.15)	(.60, .25)	(.65, .35)	(.70, .50)
	ϕ{(πE, πT)TR}	−.37	−.13	.05	−.01	−.13
	% Selected	3	26	29	27	13
	# Treated	6.4	16.0	16.1	12.1	9.0
2	(πE, πT)TR	(.2, .05)	(.25, .08)	(.35, .10)	(.40, .15)	(.55, .20)
	ϕ{(πE, πT)TR}	−.30	−.26	−.14	−.13	.04
	% Selected	5	11	18	16	49
	# Treated	8.4	9.1	10.1	9.8	22.4
3	(πE, πT)TR	(.40, .10)	(.50, .15)	(.60, .35)	(.65, .60)	(.70, .70)
	ϕ{(πE, πT)TR}	−.06	.03	−.09	−.35	−.40
	% Selected	26	51	20	2	0
	# Treated	16.7	27.3	11.6	3.3	0.8

In the three Eff-Tox scenarios in Table 3, the respective optimal doses in terms of the tradeoff contour are doses = 3, 5, and 2. We only consider simulated phase I-II trials that advance to phase III, ignoring simulation replications where the trial stopped early. In scenario 1, doses 3 and 4 have nearly equivalent desirability, so we expect most patients in the phase I-II portion of the phase I-II/III trial to be treated at these two doses. In scenario 2, the highest dose 5 is considered optimal, most patients are treated at this dose, and it is selected in 49% of the simulations. In scenario 3, the dose 2 is optimal and doses 4 and 5 have unacceptably high toxicity probabilities, so we expect to treat fewer patients at these doses. The design treats the most patients at dose 2, which is selected with probability .51, but substantial numbers of patients are treated at doses 1 and 3. The use of AR assigns more patients to doses 1 and 3, which allows the phase I-II /III design to better assess the functional relationship between dose and mean survival time. For the control group, we assume that the effects of toxicity and efficacy on overall survival are the same as those for the experimental group, and set the probabilities of toxicity and efficacy to be (.15, .40), (.10, .30), and (.35, .20) for the three Eff-Tox scenarios, respectively. For each of these simulation scenarios, we assume two different forms for the linear terms of the log hazard of Y_S. For A(x), we assume η_s(x, Y_E,Y_T) = β₀ + β₁x + β₂x² − exp(β_E)Y_E + exp(β_T)Y_T. For the simulated data from the control group, we assume η_S(C,Y_E,Y_T) = β_C − exp(β_E)Y_E + exp(β_T)Y_T and calibrate the additional parameter β_C so that we obtain the desired null value of 24 months for mean survival time. We first consider an exponential distribution with pdf f(t|ρ) = (1/ρ) exp(−t/ρ) where ρ = exp{η_S(x, Y_E,Y_T)}, since the O’Brien Fleming group sequential bounds (O’Brien and Fleming, 1979) for the logrank test are based on this assumption. Later, we will consider several other distributions to evaluate the robustness of the methodology. Table 4 displays the six scenarios considered, which correspond to the different Eff-Tox scenarios listed in Table 3, as well as differing effects of dose, efficacy, and toxicity on survival time.

Table 4.

Simulation Parameters. True Eff-Tox parameters β^TR corresponding to the phase I-II scenarios in Table 3, and true mean survival time of each A(x₁), ⋯, A(x₅) for each phase I-II/III scenario’s null and alternative hypotheses.

Scenario	Eff-Tox Scen	Hyp	(β1, β2, $e^{{\beta }_E}$, $e^{{\beta }_T}$, β0)TR	( ${\mu }_{A\left(x_1\right)}$, ${\mu }_{A\left(x_2\right)}$, ${\mu }_{A\left(x_3\right)}$, ${\mu }_{A\left(x_4\right)}$, ${\mu }_{A\left(x_5\right)}$)TR
1	1	Null	(.1, −.5, .5, .5, 2.9)	(8.3, 17.9, 24, 22.5, 9.8)
		Alt	(.25, −2, .5, .5, 3.4)	(1, 14.5, 36.2, 28.3, 1)
2	2	Null	(.1, −.1, 1, .5, 2.6)	(14.0, 17.8, 21.9, 23, 24)
		Alt	(.5, 0, 1, .5, 2.3)	(7.1, 10.3, 16.0, 19.5, 36)
3	2	Null	(.1, −.5, .3, 1, 3.1)	(9.5, 18.5, 24, 22.5, 10.4)
		Alt	(.1, −1, .3, 1, 3.6)	(6.9, 24.7, 38, 33.1, 6.3)
4	3	Null	(−.3, .3, .3, 1, 2.3)	(24,13.6, 8.9, 6.8,7.8)
		Alt	(−.1, .3, .3, 1, 3.0)	(38, 24.6, 18.4, 15.0, 21.1)
5	3	Null	(.1, −.5, .3, .1, 3.0)	(9.3, 18.7, 24, 22.7, 10.4)
		Alt	(.1, −1, .3, .1, 3.6)	(7.8, 28.8, 44, 38.6, 7.4)
6	1	Null	(.75, −.5, .3, .25, 2.8)	(3.2, 10.1, 20.4, 24, 20.4)
		Alt	(1, −.6, .3, .25, 3.3)	(3.0, 12.9, 31.8, 40, 36.6)

These scenarios encompass several qualitatively and quantitatively different possible cases in connecting phase I-II to phase III. In scenario 1, the optimal dose in terms of μ_S is dose 3, which is selected with probability .29. In scenario 2, there is a large efficacy effect, leading to dose 5 being optimal in terms of μ_S, but this dose is only selected with probability .49 in phase I-II. Thus, we expect to see a large improvement in this scenario by using a phase I-II/III design. Similarly, in scenario 3, dose 3 is optimal in terms of μ_S, but is only selected with probability .18 in phase I-II. Scenario 4 represents a case with a large toxicity effect and small efficacy effect, making dose 1 optimal in terms of μ_S, but dose 1 is only chosen in 26% of the usual phase I-II trials. In scenario 5, dose 3 is the third best dose in terms of ϕ but is best in terms of μ_S, and it is only selected with probability .20. In this scenario, dose 4 also gives a significant improvement in μ_S compared to C, with ${\mu }_{A(x_4)}=38.6$ months. In scenario 6, there is a large efficacy effect on overall survival, making dose 4 optimal in terms of both overall survival and ϕ, but dose 5 also has significantly improved survival compared to C. These two scenarios provide a basis for evaluating improvements in both γ₁ and the GP, γ₂. To control the possibility of incorrectly switching due to chance outcomes, we do not allow the design to continue with a dose that had less than 6 patients treated. These scenarios also have varying effects of toxicity and efficacy on h_S, quantified by the coefficients β_E and β_T. This will evaluate the sensitivity of the method to these effects. Since the parameters (β₁, β₂) must be changed substantially to obtain similar μ_S values for different values of (β_E, β_T), we do not perform a sensitivity analysis to these parameters within each scenario.

We assume that 10 patients, on average, are accrued each month during phase III, and that the phase III trial will begin 1 month after the phase I-II trial concludes. This waiting time could be increased to obtain longer survival follow up and thus improve the design’s ability to re-optimize doses during stage k = 2. We enroll a maximum of N_S = 500 patients in phase III, which has up to three interim looks after $n_3^{\ast }=200$, $n_4^{\ast }=300$ and $n_5^{\ast }=500$ deaths, with superiority decisions possible at each. We calibrated the stopping boundaries with East 6 statistical software (2016) using O’Brien-Fleming bounds (O’Brien and Fleming, 1979) with power .80 and type I error probability .05. We included a rule to determine if the trial should be stopped for futility, i.e. neither C nor $A\left({\widehat{x}}_S^{\text{opt}}\right)$ is superior, after $n_4^{\ast }=300$ deaths. The boundaries for declaring superiority of $A\left({\widehat{x}}_S^{\text{opt}}\right)$ or C based on the standardized logrank statistics are $({\overline{u}}_3,{\overline{u}}_4,{\overline{u}}_5)=(2.96,2.53,1.99)$, and the futility bound at the second look is ${\underset{\_}{u}}_4=1.001$. At the start of the phase III portion of the trial, we begin randomizing patients equally to $A\left({\widehat{x}}_{ET}^{\text{opt}}\right)$ and C. After $n_2^{\ast }$ deaths in the trial have occurred, we determine the dose ${\widehat{x}}_S^{\text{opt}}$ that patients receiving A should receive for the remainder of the trial. This is the re-optimization step. Survival times for patients in phase I-II and phase III are generated after their toxicity and efficacy are scored, which does not allow the possibility that a patient may die before their short term indicators are seen.

For each scenario and design, 5,000 simulation replications were performed. The simulation results are summarized in Table 5. In each of scenarios 1–4, γ₁ = γ₂, since there is one dose for which A is superior to C. Mean improvement in patient survival time with each design is denoted by $\overline{W}$, computed by averaging the differences between the true mean survival time with the selected dose of A and μ_C, if A is declared superior to C. In the simulations, $\overline{W}$ is computed as the mean over {W^b, b = 1, ⋯, 5000}, where

$$W^b=({\mu }_{A({\widehat{x}}_S^{\text{opt}})}-{\mu }_C)I[A({\widehat{x}}_S^{\text{opt}})\text{is declared superior to}C\text{in simulated trial b}].$$

Table 5.

Simulation results. α is the probability of a type I error or concluding an inferior version of A is better than C under the null. γ₁ is the generalized power at $A\left(x_S^{\mathit{opt}}\right)$ (probability of selecting the best dose $x_S^{\mathit{opt}}$ and declaring it to be superior to C) under the alternative hypothesis. γ₂ is the generalized power (probability of selecting any truly superior dose of A and declaring it superior to C). $\overline{W}$ is the mean improvement in patient survival under the alternative hypothesis, $\overline{\mathit{Dur}}$ is the mean trial duration, and $\overline{N}$ is the mean sample size.

Scenario	Design	Alternative Hypothesis						Null Hypothesis
		$\overline{W}$	γ1	γ2	$\overline{\mathit{Dur}}$	$\overline{N}$		α	$\overline{\mathit{Dur}}$	$\overline{N}$
1	Phase I-II → phase III	4.04	.29	.29	4.05	431.1	–	.02	4.05	461.5
	Phase I-II/III	10.15	.83	.83	4.73	479.2	–	.03	4.32	492.0
2	Phase I-II → phase III	7.87	.66	.66	4.29	459.2	–	.06	4.18	489.4
	Phase I-II/III	8.97	.75	.75	4.45	470.7	–	.02	4.18	489.9
3	Phase I-II → phase III	1.83	.06	.06	3.10	355.0	–	< .01	3.28	385.7
	Phase I-II/III	11.51	.79	.79	4.56	476.9	–	.04	4.22	485.6
4	Phase I-II → phase III	3.52	.25	.25	4.28	475.8	–	.05	3.48	407.8
	Phase I-II/III	5.86	.42	.42	4.30	472.0	–	.05	3.81	442.0
5	Phase I-II → phase III	5.61	.21	.25	3.98	428.5	–	.01	3.83	440.4
	Phase I-II/III	16.71	.68	.88	4.24	464.4	–	.03	4.37	493.9
6	Phase I-II → phase III	9.46	.34	.52	4.16	447.9	–	.02	4.16	466.5
	Phase I-II/III	12.67	.59	.75	4.53	472.7	–	.04	4.39	494.0

Table 5 shows that, in general, the phase I-II/III design maintained type I error probability ≤ .05 under H₀ and had a uniformly higher γ₁ and GP, γ₂, compared to the conventional phase I-II → phase III approach without dose re-optimization. The values of γ₁, γ₂, and $\overline{W}$ are uniformly larger for the phase I-II/III design than for the conventional phase I-II → phase III paradigm. The differences are extremely large in scenario 3, with an improvement of .73 in γ₁, and a 9.68 month improvement in $\overline{W}$. The smallest advantage of the phase I-II/III design is seen in scenario 2, with an improvement of .09 for γ₁ and .90 for $\overline{W}$. In scenarios 5 and 6, where two doses of A give mean survival time larger than μ_C + Δ = 36 months, the phase I-II/III design provides respective improvements in γ₂ of .61 and .23, and improvements in γ₁ of .57 and .25. These scenarios illustrate the potential advantage of the phase I-II/III design compared to the conventional phase I-II → phase III approach.

The phase I-II/III design does have the drawback that it requires treating more patients and longer trial durations, on average, than the conventional paradigm. Part of this required increase is due to the design correctly switching to the best dose of A in terms of overall survival, which decreases the likelihood that a trial will stop early by declaring C to be superior or due to futility. This increase in required sample size and trial duration are the price paid for the much larger probability of a successful phase III trial in cases where dose switching increases mean survival time with A.

Since phase I-II trials may have sample sizes ranging from 24 to 90 in practice, we chose the Eff-Tox sample size N_ET = 60 in the simulations as a practical compromise that obtains a reasonable amount of information in stage 1. Web Tables 1 and 2, seen in Web Appendix E, summarize additional simulations with N_ET = 90. Values of γ₁, γ₂ and $\overline{W}$ for the phase I-II/III design all increased substantially with N_ET for all six scenarios. This is because more information at different doses in phase I-II makes switching to the best dose in stage 2 more likely.

To assess robustness of the phase I-II/III design to different event time distributions, we evaluated its performance for two lognormal distributions, with variances .25 and 1, a Weibull distribution with increasing or decreasing hazard, with shape parameters 4 or .5, and a gamma distribution with scale parameter 2. The true coefficients of Y_E and Y_T in the hazard function’s linear term were kept constant for each distribution, and the remaining constant parameters β₀, β_C, β₁, β₂ were adjusted to obtain similar true means as in the exponential distribution simulation study. We exponentiated the linear term for the gamma and Weibull distribution rate parameters, but did not do this for the lognormal distribution. The means under the null and alternative hypotheses for each distribution are given in Web Table 3. Table 6 summarizes the robustness study, showing that under the alternative hypothesis, for each distribution, the phase I-II/III design has uniformly higher values of γ₁, γ₂, and $\overline{W}$, with substantially higher values for scenarios 1,3 and 5. For the Weibull distribution with decreasing hazards in each scenario, the decrease in γ₁ and γ₂ for both designs is due to the assumptions of the logrank test being grossly violated by a high early failure rate. Because so many patients have early failures, patients are not followed as long before the final group sequential test. For this distribution, however, the phase I-II/III design still improved the probability of selecting the optimal dose of A compared to the conventional paradigm by .49, .21, .15, .59, .46 and .14 in the six scenarios, respectively. Similar improvements are seen under the other distributions. This shows that the logrank test is not robust to the Weibull distribution with decreasing hazard. An extension of the phase I-II/III design might incorporate a robust group sequential test in place of the logrank test, to reduce the loss in power under a Weibull with decreasing hazard. The type I error constraints are nearly met for each distribution. Some slight inflation in α above .05 may be attributed to the proportional hazards assumption being violated. For each distribution, the phase I-II/III design treats more patients, on average, under both H₀ and H₁, and has slightly longer trial duration, but makes the correct decision much more often.

Table 6.

Robustness simulation results. α = probability of a type I error or concluding that an inferior version of A is better than C under the null. γ₁ = generalized power at $x_S^{\mathit{opt}}$ under the alternative hypothesis (probability of selecting the best dose $x_S^{\mathit{opt}}$ and declaring $A\left(x_S^{\mathit{opt}}\right)$ superior to C). γ₂ = generalized power (probability of selecting any superior dose and declaring $A\left({\widehat{x}}_S^{\mathit{opt}}\right)$ superior to C). $\overline{W}$ is the mean improvement in survival time under the alternative hypothesis. $\overline{\mathit{Dur}}$ and $\overline{N}$ are the mean trial duration and sample size, respectively.

Scenario	Distribution	Phase I-II → phase III				Phase I-II/III Design
		$\overline{W}$	γ1	γ2	α	$\overline{W}$	γ1	γ2	α
1	Lognormal, σ = .5	7.72	.30	.30	.02	14.56	.80	.80	.03
	Lognormal, σ = 1	6.18	.30	.30	.02	14.88	.85	.85	.03
	Weibull Increasing	6.37	.30	.30	.02	12.00	.77	.77	.03
	Weibull Decreasing	1.96	.13	.13	.02	4.14	.32	.32	.02
	Gamma	5.02	.30	.30	.02	11.72	.89	.89	.04
2	Lognormal, σ = .5	12.18	.73	.73	.05	15.14	.91	.91	.05
	Lognormal, σ = 1	11.70	.70	.70	.06	13.78	.82	.82	.03
	Weibull Increasing	9.08	.73	.73	.06	11.20	.90	.90	.06
	Weibull Decreasing	3.98	.32	.32	.05	4.75	.38	.38	.03
	Gamma	9.11	.73	.73	.06	11.30	.90	.90	.03
3	Lognormal, σ = .5	2.17	.06	.06	< .01	13.91	.85	.85	.04
	Lognormal, σ = 1	2.02	.06	.06	< .01	12.91	.81	.81	.03
	Weibull Increasing	2.14	.06	.06	< .01	13.63	.84	.84	.03
	Weibull Decreasing	.90	.03	.03	< .01	4.94	.32	.32	.03
	Gamma	2.07	.06	.06	< .01	13.23	.86	.86	.03
4	Lognormal, σ = .5	4.50	.32	.32	.05	7.23	.52	.52	.05
	Lognormal, σ = 1	3.86	.28	.28	.05	5.61	.40	.40	.06
	Weibull Increasing	4.67	.33	.33	.04	6.61	.47	.47	.04
	Weibull Decreasing	4.14	.28	.28	.06	6.21	.42	.42	.07
	Gamma	4.62	.31	.31	.04	6.93	.47	.47	.04
5	Lognormal, σ = .5	6.63	.21	.25	.01	18.19	.76	.96	.03
	Lognormal, σ = 1	5.74	.21	.25	.01	16.78	.69	.87	.03
	Weibull Increasing	7.41	.21	.25	.01	15.88	.64	.77	.03
	Weibull Decreasing	4.00	.16	.18	.01	12.27	.48	.63	.03
	Gamma	6.73	.21	.25	.01	18.73	.80	.92	.03
6	Lognormal, σ = .5	11.92	.34	.52	.04	15.56	.58	.83	.06
	Lognormal, σ = 1	11.50	.34	.52	.02	15.66	.60	.80	.04
	Weibull Increasing	10.29	.34	.52	.04	12.76	.54	.77	.05
	Weibull Decreasing	5.46	.21	.31	.02	6.63	.32	.38	.03
	Gamma	10.90	.34	.52	.02	14.80	.59	.83	.04

6. Discussion

We have proposed a new drug development strategy, which we call a phase I-II/III design, that re-optimizes the dose of an experimental agent A chosen in phase I-II during phase III based on mean survival time. We use information from all patients treated with A, including their short term efficacy and toxicity indicators, dose assigned, and survival time information, in order to more accurately select the dose of A that provides the highest posterior mean survival time. The design is based on an assumed a mixture model for the survival time distribution that averages over the possible short term phase I-II outcomes. While we have used the Eff-Tox trade-off based phase I-II design for stage 1 of the phase I-II/III design, one could replace the Eff-Tox design with any phase I-II design based on (Y_E, Y_T) that uses some dose optimality criterion ϕ and includes AR. However, the necessary modifications of the design parameters and computer software to accommodate such a change would be non-trivial. Similarly, a complicated but straightforward extension of the methodology may address the problem of possible deaths before evaluation of (Y_E, Y_T).

The simulations shows that, under a range of alternative cases, the generalized power γ₂, and probability γ₁ of the best possible decision, both are greatly increased by the phase I-II/III design compared to the phase I-II → phase III paradigm. The phase I-II/III design also has a much lower probability of making the least desirable decision, where a suboptimal dose is chosen and a true treatment advance is missed. A drawback of the phase I-II/III design is that it requires more patients and a slightly longer trial duration, on average, compared to the phase I-II → phase III paradigm. This seems like a very reasonable price to pay for the much larger values of γ₁, γ₂, and $\overline{W}$, in cases where re-optimizing the dose of the experimental agent increases its associated mean survival time.