Response adaptive randomization procedures in seamless phase II/III clinical trials

Abstract

It is desirable to work efficiently and cost effectively to evaluate new therapies in a time-sensitive and ethical manner without compromising the integrity and validity of the development process. The seamless phase II/III clinical trial has been proposed to meet this need, and its efficient, ethical and economic advantages can be strengthened by its combination with innovative response adaptive randomization (RAR) procedures. In particular, well-designed frequentist RAR procedures can target theoretically optimal allocation proportions, and there are explicit asymptotic results. However, there has been little research into seamless phase II/III clinical trials with frequentist RAR because of the difficulty in performing valid statistical inference and controlling the type I error rate. In this paper, we propose the framework for a family of frequentist RAR designs for seamless phase II/III trials, derive the asymptotic distribution of the parameter estimators using martingale processes and offer solutions to control the type I error rate. The numerical studies demonstrate our theoretical findings and the advantages of the proposed methods.

1. Introduction

It is desirable to work efficiently and cost effectively to evaluate new therapies in a time-sensitive and ethical manner without compromising the integrity and validity of the development process. Adaptive seamless phase II/III designs (ASD) and response-adaptive randomization (RAR) have been proposed to meet this need (Bhatt and Mehta, 2016; Chow and Chang, 2012; Bretz et al., 2009; Hu and Rosenberger, 2006). The seamless phase II/III clinical trial can reduce the lead time between phase II and phase III trials, efficiently combine the data from the two phases for analysis, reduce the number of trials conducted in drug evaluation programs, prolong the monitoring time for patients from the phase II trial for safety reasons, and decrease the sample size. These advantages can be strengthened by its combination with innovative RAR designs that can achieve multiple competing objectives such as maximizing the power of detecting treatment effects and assigning more patients to the better treatment. However, there has been limited theoretical study of the combination of ASD and frequentist RAR designs. This hinders the development and application of this sequential procedure. In this paper, we develop approaches to adaptive seamless phase II/III clinical trials with RAR and study statistical inference for this procedure.

We briefly review the literature on ASD here. The drive to reduce development costs and shorten the time-to-market of new therapies has led to the development of the methodology for ASD. Typically, such trials have two stages separated by an interim analysis (Thall et al., 1988, 1989; Schaid et al., 1990). In the first stage, which resembles a traditional phase II trial, multiple experimental treatments or drug doses are simultaneously compared against a control. The empirically best candidates are then selected for the second stage, which corresponds to a phase III trial. The final analysis includes the patients from both stages and is performed such that the overall type I error rate is controlled. Stallard and Todd (2003) extended the approach to allow sequential analyses and early stopping in the second stage. In addition, Bretz et al. (2006) and Schmidli et al. (2006) followed the approach of Bauer and Kieser (1999) and used the closure principle (Marcus et al., 1976) and combination tests (Bauer and Köhne, 1994) to combine data from the two stages in the final hypothesis test and to control the familywise type I error rate. Liu et al. (2002) offered a solid theoretical foundation for general two-stage adaptive designs. Koenig et al. (2008) proposed the adaptive Dunnett test based on the conditional error rate (Müller and Schäfer, 2001). Most of these approaches can be classified into three categories: the group sequential approach, the combination test approach, and the adaptive Dunnett method. A detailed discussion of ASD can be found in Stallard and Todd (2011) and Korn et al. (2012). Real seamless phase II/III clinical trials include Barnes et al. (2010), Zeymer et al. (2001), Schmoll et al. (2010), and Chen et al. (2015).

Next, we briefly review the literature on RAR. RAR sequentially assigns the next patient based on the previous treatment assignments and responses to achieve greater efficiency and ethical advantages such as assigning more patients to better treatments. Eisele and Woodroofe (1995) and Hu and Zhang (2004) obtained its asymptotic properties. Zhu and Hu (2010, 2012) studied the sequential monitoring of RAR. Other important developments and discussions of the advantages of RAR can be found in Wei and Durham (1978), Rosenberger et al. (2001), Hu and Rosenberger (2003), Ivanova and Rosenberger (2000), Hu et al. (2009), Zhu and Hu (2009), Biswas et al. (2011), Tymofyeyev et al. (2007), Hu et al. (2015), Wang et al. (2017) and Antognini and Giovagnoli (2010). Real trials using RAR designs include Rout et al. (1993), Tamura et al. (1994), and Andersen (1996).

It is desirable to propose the frequentist RAR designs for seamless trials to combine the advantages of these two strategies since the well-designed frequentist RAR can target the theoretically optimal allocation proportions to achieve efficient and ethical objectives with explicit asymptotic results. However, there have been few theoretical investigations of statistical inference for this procedure. The control of the type I error rate that can be inflated by ASD because of the dual influence of multiplicity and selection (Bauer et al., 2010) is of key importance for seamless phase II/III clinical trials. In addition, RAR designs pose new challenges due to the complicated relationships among the treatment assignments and responses. Besides, the RAR design has to target different allocation proportions in the two stages. Therefore, it is a must and challenging to investigate the statistical inference for this procedure and study how to control the type I error rate and whether the advantages of both RAR and ASD can be kept under this scenario.

Other works provide insight into the topic of this paper. Work on adaptive randomization design and ASD under the Bayesian framework includes but is not limited to Thall and Wathen (2007), Huang et al. (2009), Atkinson and Biswas (2005), Yuan et al. (2011), Brannath et al. (2009), Inoue et al. (2002), Berry (2004, 2012), and Zang and Lee (2014). In addition, Zhu (2017) derived the asymptotic normality for RAR designs with fixed adjusted allocation proportion where the target allocation proportions in the second stage are known and fixed at the beginning of the trial. The results are inspiring for the theoretical investigation of RAR designs with two stages. However, the design with fixed known adjusted allocation proportion is not practical in the real seamless Phase II/III clinical trials where the allocation proportions and the treatments under study in the second stage are usually determined at the interim look.

In this paper, we conducted comprehensive research to address the following questions. First, is there a general mathematical framework for combining ASD and RAR? Second, are there fundamental properties of this sequential procedure that can provide a theoretical foundation for further investigations? Third, is it possible to protect the type I error rate? Fourth, what are the advantages of the combination? We first described the framework for the proposed seamless phase II/III trials with RAR and derived the asymptotic distribution of the parameter estimators by approximating these estimators by martingale processes. This general framework and theoretical results can be used to investigate a variety of research questions for this sequential procedure. Then we used normal responses to demonstrate how to make use of the idea of Cohen and Sackrowitz (1989) and Sampson and Sill (2005) to construct the test statistics and to find the critical value to control the type I error rate for the proposed framework. We employed the large sample theory to overcome the difficulty that the number of patients in each treatment is random and the variances of responses are unknown and need to be estimated, which is nontrivial under the settings of RAR. The numerical studies also showed that our proposed method can control the type I error rate very well, improve the power and assign more people to the better treatment. This paper opens a door and provides fundamental theoretical foundations for other solutions to controlling the type I error rate for this procedure.

In Section 2, we introduce the notation, describe the framework and state the main theorem. In Sections 3, we perform numerical studies to compare the proposed procedure with other randomization procedures. Conclusions are in Section 4, and technical proofs are given in the Appendix.

2. Framework, theory and implementation of the new design

2.1. Framework

We first describe the framework for the proposed family of RAR designs for seamless phase II/III trials. Suppose there are (K + 1) treatments under study in the clinical trial, the planned total sample size is n, and the planned sample size for the first stage is n₁. Then the sample size for the second stage is n₂ = n – n₁. Let T_i = (T_i0,T_i1,…,T_iK) indicate the treatment assignment of the ith subject, where treatment 0 represents the control arm, T_ik = 1,k = 0,1,…,K if the ith patient is in treatment k, and T_ik = 0 otherwise. Let $N_k(m)={\sum }_{i=1}^m{T}_{ik}\mathrm{,}k=\mathrm{0,1,}\dots \mathrm{,}K$ be the number of patients in treatment k after m patients have been assigned, and we write N(m) = (N₀(m),N₁(m),…,N_K(m)). Let X_i = (X_i0,X_i1,…, X_iK),i = 1,…,n be a random matrix of response variables, where X_ik,k = 0,1,…,K are d-dimensional random vectors, and only one element of X_i, say X_ik, can be observed if T_ik = 1. Assume that X₁,…, X_n are independent and identically distributed with unknown parameters (θ₀,θ₁…,θ_K). Without loss of generality, we also assume that θ_k = E(X_ik), where θ_k is the d-dimensional parameter vector (θ_k1,…,θ_kd) corresponding to treatment k (k = 0,1,…,K), since we can transform the response and treat the transformation as responses to make the assumption hold (Gwise, et al. 2008; Hu and Zhang, 2004). After responses from the first m patients have been observed, the following modified sample means are used to estimate the parameter θ_k,k = 0,1,…,K:

$${\widehat{\mathit{\theta }}}_k(m)=\frac{{\sum }_{i=1}^m{T}_{ik}{\mathit{X}}_{ik}}{N_k(m)+1}\mathrm{.}$$ (1)

We write θ = (θ₀,θ₁…,θ_K) and $\widehat{\mathit{\theta }}\left(m\right)=\left({\widehat{\mathit{\theta }}}_0\left(m\right),{\widehat{\mathit{\theta }}}_1\left(m\right),\dots ,{\widehat{\mathit{\theta }}}_K\left(m\right)\right)$.

At the first stage, patients are sequentially assigned to all the (K + 1) treatments with RAR, and one treatment, say treatment M, is chosen at the end of this stage based on certain criteria. At the second stage, the planned number of remaining patients will be sequentially assigned to treatment 0 (control arm) and treatment M with RAR. RAR designs achieve various objectives in terms of ethics and efficiency by targeting optimal allocation proportions that are usually obtained based on certain optimal criteria and depend on the unknown parameter θ (Tymofyeyev et al. 2007; Rosenberger et al. 2001). For example, to maximize the power when comparing two normal distributions $N({\mu }_1\mathrm{,}{\sigma }_1^2)$ and $N({\mu }_2\mathrm{,}{\sigma }_2^2)$, the target optimal allocation proportions for the two treatments are σ₁/(σ₁ + σ₂) and σ₂/(σ₁ + σ₂), respectively, which is the so-called Neyman allocation. Let v₁ = (v₁₀, v₁₁,…,v_1K) = ρ₁(θ) = (ρ₁₀(θ), ρ₁₁(θ),…,ρ_1K(θ)) be the target allocation proportions for stage 1, and v₂ = (v₂₀, v_2M) = ρ₂(θ) = (ρ₂₀(θ), ρ_2M(θ)) be the target allocation proportions for stage 2, where ${\mathit{\rho }}_1(\mathit{z})=\left({\rho }_{10}(\mathit{z})\mathrm{,}{\rho }_{11}(\mathit{z})\mathrm{,}\dots \mathrm{,}{\rho }_{1K}(\mathit{z})\right):{\Re }^{d\times (K+1)}\to {\left(\mathrm{0,1}\right)}^{\left(K+1\right)}$ and ${\mathit{\rho }}_2(\mathit{z})=\left({\rho }_{20}(\mathit{z})\mathrm{,}{\rho }_{2M}(\mathit{z})\right):{\Re }^{d\times 2}\to {\left(\mathrm{0,1}\right)}^2$ are vector-valued functions satisfying ${\mathit{\rho }}_j(\mathit{z})1\text{'}=\mathrm{1,}j=\mathrm{1,2}$. In practice, we need to sequentially estimate the unknown parameters and update the allocation probability according to previous treatment assignments and responses so that we can target the optimal allocation proportions. Therefore, the allocation probabilities for different patients are different, but the allocation probability will converge for a variety of RAR designs (Hu and Zhang, 2004). Hu and Rosenberger (2003) studied the bias of the randomization procedure from the target allocation proportion and the variability induced by the randomization.

Specifically, we use the following family of DBCD to sequentially allocate patients in seamless phase II/III clinical trials:

1. Assign (K + 1)m₀ patients to all the treatments by restricted randomization procedures such as permuted block randomization (Rosenberger and Lachin, 2015) to obtain initial parameter estimates.

2. When the mth (m > (K + 1))m₀ patient who is in the jth (j = 1,2) stage enters the trial, compute $\widehat{\mathit{\theta }}(m-1)$ and ${\widehat{\mathit{\rho }}}_j={\mathit{\rho }}_j(\widehat{\mathit{\theta }}(m-1))$ based on all the previous responses and treatment assignments.

3. At the first stage, assign the mth patient to treatment k with probability

   $$g_{1k}\left(\mathit{N}\left(m-1\right)/\left(m-1\right)\mathrm{,}{\mathit{\rho }}_1\left(\widehat{\mathit{\theta }}\left(m-1\right)\right)\right),$$

   where g_1k(s,r) = g_1k((s₀, s₁,…, s_K), (r₀, r₁,…, r_K)): (0,1)^(K+1)×(0,1)^(K+1) → (0,1) is the allocation probability function for treatment k and ${\sum }_{k=0}^K{g}_{1k}(\mathit{s}\mathrm{,}\mathit{r})=1$. We write g₁ = (g₁₀,g₁₁,…,g_1K). This allocation probability function takes into account both the estimated targeted allocation proportions and the current allocation proportions, and it has been well studied in the literature (e.g., Hu and Zhang, 2004; Zhu and Hu, 2010). In the numerical study, we use the following allocation probability function

   $$g_{1k}(\mathit{s}\mathrm{,}\mathit{r})=\frac{r_k{(r_k/s_k)}^2}{{\sum }_{j=0}^K\left\{r_j{(r_j/s_j)}^2\right\}}\mathrm{.}$$

4. At the end of the first stage, compare all the treatments and choose one (say treatment M) for continuation into the second stage based on certain criteria, along with the control arm.

5. At the second stage, assign the mth patient to treatment k, k = 0,M with probability

   $$g_{2k}\left(\left(N_0\left(m-1\right)/\left(m-1\right)\mathrm{,}{N}_M\left(m-1\right)/\left(m-1\right)\right),{\mathit{\rho }}_2\left(\widehat{\mathit{\theta }}\left(m-1\right)\right)\right),$$

   where g_2k(s,r): (0,1)²×(0,1)² → (0,1) is the allocation probability function for treatment k and g₂₀(s,r) + g_2M(s,r)=1. We write g₂ = (g₂₀, g_2M). In the numerical study, we use the following allocation probability function

   $$g_{2k}(\mathit{s}\mathrm{,}\mathit{r})=\frac{r_k{(r_k/s_k)}^2}{\{r_0{(r_0/s_0)}^2+r_M{(r_M/s_M)}^2\}}\mathrm{,}k=\mathrm{0,}M\mathrm{.}$$

Our results are valid under the following conditions for the response, the target allocation proportions, and the allocation probability function:

• (A1) For some ε > 0, $E{\Vert {\mathit{X}}_{1k}\Vert }^{2+\epsilon }<\infty$, k=0,…,K.

• (A2) g_j(s,r),j = 1,2, is continuous on {(s, v):s1′ = 1} and twice differentiable at (v, v).

• (A3) g_j(v,v) = v,j = 1,2, for all v ∈ (0,1)^K.

• (A4) ${\mathit{\rho }}_j(\mathit{z})$ is a continuous function and twice continuously differentiable in a small neighborhood of θ.

• (A5) N_k(n)→∞, a.s.,k = 0,1,…,K.

• (A6) There is a relatively large number of patients in the second stage compared to the first stage, and $\frac{\mathit{N}(n)}{n}\to {\mathit{v}}_{2}a\mathrm{.}s\mathrm{.}$ and as the sample size goes to infinity. In addition, by Hu and Zhang (2004), $\frac{\mathit{N}(n)}{n}-{\mathit{v}}_2=O\left(\sqrt{\frac{\text{loglog}n}{n}}\right)a\mathrm{.}s\mathrm{.}$

Theorem 1 Let ${\widehat{\mathit{\theta }}}_M^{\text{'}}=\frac{{\sum }_{i=n_1+1}^n{T}_{iM}{\mathit{X}}_{iM}}{n_{M^{\prime }}+1}$, where $n_M^{\text{'}}={\sum }_{i=n_1+1}^n{T}_{iM}$. Under Conditions (A1)–(A6), we have

$$\sqrt{N_k(n_1)}\left({\widehat{\mathit{\theta }}}_k(n_1)-{\mathit{\theta }}_k\right)\to N\left(\mathrm{0,}Var({\mathit{X}}_{1k})\right)indistribution\mathrm{,}k=\mathrm{0,1,}\dots \mathrm{,}K;$$

$$\sqrt{N_0(n)}\left({\widehat{\mathit{\theta }}}_0(n)-{\mathit{\theta }}_0\right)\to N\left(\mathrm{0,}Var({\mathit{X}}_{10})\right)indistribution\mathrm{,};$$

and

$$\sqrt{n_M\text{'}}\left({\widehat{\mathit{\theta }}}_{M^{\prime }}-{\mathit{\theta }}_M\right)\to N\left(\mathrm{0,}Var({\mathit{X}}_{1M})\right)indistribution\mathrm{.}$$

This theorem offers fundamental theoretical foundations for the proposed general framework and different solutions for controlling the type I error rate might be proposed based on this result. Next, we provide a specific method.

2.2. Performing hypothesis testing for responses from normal distribution.

We have described a general framework for the proposed design and obtained the theoretical support. Next, we apply this framework when the response Y_ik of the kth treatment is from the normal distribution $N({\mu }_k\mathrm{,}{\sigma }_k^2)\mathrm{,}k=\mathrm{0,1,}\dots \mathrm{,}K$, and introduce how to construct the test statistics and find critical values to control the type I error rate. In this case, ${\mathit{\theta }}_k=({\mu }_k\mathrm{,}{\sigma }_k^2+{\mu }_k^2)$ and ${\mathit{X}}_{ik}=(Y_{ik}\mathrm{,}{Y}_{ik}^2)$.

Different methods can be used to choose treatment arms for the second stage under the proposed framework. We apply the design setting in Sampson and Sill (2005), i.e., the empirically best treatment and the control arm are continued into the second stage, and stopping for futility is not allowed. Without loss of generality, we assume the ordered outcome $Q=\left\{{\widehat{\mu }}_1(n_1)>{\widehat{\mu }}_2(n_1)>\dots >{\widehat{\mu }}_K(n_1)\right\}$ has been observed after the first stage. Then treatment 1 and the control arm enter the phase III trial. This setting is useful in many practical contexts. It can apply when some regimens or formulations of a fixed dose of a compound are under study, and only one best regimen or formulation is needed. For example, when developing topical ointments, we may have several formulations of equal potency that are expected to have minimal adverse effects. We want to choose the empirically best formulation for the phase III trial. At the end of the second stage, we would like to test the hypothesis

$$H_0:{\mu }_0={\mu }_1\text{ versus }{H}_1:{\mu }_1>{\mu }_0\mathrm{.}$$ (2)

RAR presents tremendous challenges for studying the statistical inference for this procedure. In addition to the complicated relationship between the responses and the treatment assignments, major challenges are the randomness of the patient numbers in each treatment and the variance estimators. We overcame these difficulties by martingales and large sample theory, obtaining the following conclusion under certain conditions.

Theorem 2 Consider the scenario in this section where the specific event Q has been observed and let ${\widehat{\mathit{\mu }}}^{*}=({\widehat{\mu }}_2(n_1)\mathrm{,}{\widehat{\mu }}_3(n_1)\mathrm{,}\dots \mathrm{,}{\widehat{\mu }}_K(n_1))$. Let

$$\overline{Z}\text{'}=\frac{N_1(n_1){\widehat{\mu }}_1(n_1)+n_{1^{\prime }}{\widehat{\mu }}_{1^{\prime }}}{N_1(n)}\mathrm{,}$$

$$W^{\prime }=\frac{N_1(n)N_0(n)\left(\overline{Z}\text{'}-{\widehat{\mu }}_0(n)\right)}{N_1(n){\widehat{\sigma }}_0^2(n)+N_0(n){\widehat{\sigma }}_1^2(n)}\mathrm{,}$$

$$T^{\prime }=\frac{N_1(n){\widehat{\sigma }}_0^2(n)\overline{Z}\text{'}+N_0(n){\widehat{\sigma }}_1^2(n){\widehat{\mu }}_0(n)}{N_1(n){\widehat{\sigma }}_0^2(n)+N_0(n){\widehat{\sigma }}_1^2(n)}\mathrm{,}$$

$$\overline{Z}=\frac{n_1{v}_{11}{\widehat{\mu }}_1(n_1)+(n{v}_{21}-n_1{v}_{11}){\widehat{\mu }}_{1^{\prime }}}{n{v}_{21}}\mathrm{,}$$

$$W=\frac{n{v}_{21}n{v}_{20}\left(\overline{Z}-{\widehat{\mu }}_0(n)\right)}{n{v}_{21}{\sigma }_0^2+n{v}_{20}{\sigma }_1^2}\mathrm{,}$$

$$T=\frac{n{v}_{21}{\sigma }_0^2\overline{Z}+n{v}_{20}{\sigma }_1^2{\widehat{\mu }}_0(n)}{n{v}_{21}{\sigma }_0^2+n{v}_{20}{\sigma }_1^2}\mathrm{.}$$

Then, conditioning on the event Q, the conditional cumulative distribution function of W′ given T′ and ${\widehat{\mathit{\mu }}}^{*}$ is asymptotically the conditional cumulative distribution function of W given T and ${\widehat{\mathit{\mu }}}^{*}$ whose probability density function under H₀ is

$$f\left(W|T\mathrm{,}{\widehat{\mathit{\mu }}}^{*}\right)=C_N\text{exp}\{-\frac{1}{2}{W}^2/\left(\frac{n{v}_{21}n{v}_{20}}{n{v}_{21}{\sigma }_0^2+n{v}_{20}{\sigma }_1^2}\right)\}\Phi \left\{\frac{\sqrt{n_1{v}_{11}n{v}_{21}}\left(T+\frac{{\sigma }_1^2}{n{v}_{21}}W-{\widehat{\mu }}_2(n_1)\right)}{\sqrt{(n-n_1)v_{21}}{\sigma }_1}\right\}$$

where Φ(·) is the cumulative distribution function of the standard normal distribution and C is the normalization constant, which involves integrating W over the real line. Therefore, the following two steps based on the observed data can be used to test the hypothesis (2):

• Step 1: We use the observed data to obtain C by integrating the kernel of the above density function over the real line.

• Step 2: We calculate the p-value using the observed data and the derived probability density function from Step 1, and we perform the hypothesis testing and control the type I error rate by comparing the p-value and the significance level.

Theorem 2 offers specific solutions and procedures to control the type I error rate for the proposed design.

3. Numerical Studies

We have derived the large-sample properties and proposed the solution to control the type I error rate based on these properties for this procedure. Next, we study the procedure’s finite-sample properties. We first compared our design targeting Neyman allocation (DBCD), the seamless design proposed by Sampson and Sill (2005) (SS), and the seamless designs discussed in Bretz et al. (2006) that used the closure principle (Marcus et al., 1976), combination tests (Bauer and Köhne, 1994) with either the Simes test (Simes, 1986) or the Dunnett test (Dunnett, 1955) when two experimental treatments and one control arm are under study. Then we studied the performance of our design when more treatments are involved. Take the Indacaterol to Help Achieve New COPD Treatment Excellence (INHANCE) trial (Barnes et al. 2010) as motivation example where there were four experimental treatments and several control arms. To follow the proposed framework of this paper, we revised the scenario in Barnes et al. (2010) and compared the above different designs when four experimental treatments and one control arm were under study and the best performing experimental treatment and the control arm were selected to enter the second stage. Finally, we considered the binary responses and performed the exploratory study using the normal approximation to the Binomial distribution to expand the application of our design and investigate its ability to target different allocation proportions and achieve different objectives.

In Tables 1–4, we consider responses from normal distributions. Suppose 500 patients sequentially enter the trial. There are (K + 1) treatments with responses following normal distributions $N({\mu }_k\mathrm{,}{\sigma }_k^2)\mathrm{,}k=\mathrm{0,\dots ,}K\mathrm{,}K=\mathrm{2,4,}$ under study at the first stage, and treatment 0 is the control arm. Our objective is to maximize the power. At the first stage, the target allocation proportion is ρ₁ = (ρ₁₀,…,ρ_1K), where ${\rho }_{1k}={\sigma }_k/{\sum }_{j=0}^K{\sigma }_j\mathrm{,}k=\mathrm{0,\dots ,}K\mathrm{.}$ At the end of the first stage, the treatment arm with the larger mean response is selected to enter the second stage (assuming that a larger value is better). Then we use the following Neyman allocation to maximize the power if the experimental treatment arm M is retained for the second stage:

$${\mathit{\rho }}_2=({\rho }_{20}\mathrm{,}{\rho }_{2M})=\left(\frac{{\sigma }_0}{{\sigma }_0+{\sigma }_M}\mathrm{,}\frac{{\sigma }_M}{{\sigma }_0+{\sigma }_M}\right)\mathrm{.}$$

At the end of the trial, the chosen treatment M is compared with the control arm, and the following hypothesis is tested:

$$H_0:{\mu }_0={\mu }_M\text{ versus }{H}_1:{\mu }_M>{\mu }_0\mathrm{.}$$

All the results are based on 10,000 replications, and the nominal type I error rate for the above one-sided test is 0.05.

Table 1:

Type I error rate and parameter estimation for seamless trials with three treatments, normal responses, and sample size n = 500

Allocation	t	Type I error	${\widehat{\mu }}_0$	${\widehat{\mu }}_M$
DBCD	0.1	0.049	−3.1e-4(0.06)	0.008(0.06)
SS	0.1	0.049	8.9e-4(0.06)	0.009(0.06)
Dunnett	0.1	0.054	−8.9e-4(0.06)	0.010(0.06)
Simes	0.1	0.048	6.7e-4(0.06)	0.010(0.06)
DBCD	0.2	0.051	−1.9e-4(0.07)	0.014(0.06)
SS	0.2	0.052	3.6e-6(0.07)	0.013(0.06)
Dunnett	0.2	0.049	1.3e-3(0.07)	0.013(0.06)
Simes	0.2	0.048	6.0e-4(0.07)	0.014(0.06)
DBCD	0.3	0.053	−6.9e-4(0.07)	0.017(0.06)
SS	0.3	0.050	1.4e-4(0.07)	0.019(0.06)
Dunnett	0.3	0.055	−4.9e-4(0.07)	0.018(0.06)
Simes	0.3	0.046	3.2e-4(0.07)	0.017(0.06)

Table 4:

Power of different designs for seamless trials with five treatments, normal responses, and sample size n = 500

Allocation	t	(μ0,σ0,μ1,σ1,μ2,σ2μ3,σ3,μ4,σ4)	Power	${\widehat{\mu }}_0$	Betterwin
DBCD	0.15	(1,1,1.5,2.5,1.4,2.5,1.3,2.5,1.2,2.5)	0.692	1.00(0.09)	3324
SS	0.15	(1,1,1.5,2.5,1.4,2.5,1.3,2.5,1.2,2.5)	0.645	1.00(0.07)	3332
Dunnett	0.15	(1,1,1.5,2.5,1.4,2.5,1.3,2.5,1.2,2.5)	0.657	1.00(0.07)	3283
Simes	0.15	(1,1,1.5,2.5,1.4,2.5,1.3,2.5,1.2,2.5)	0.622	1.00(0.07)	3205
DBCD	0.2	(1,1,1.6,2.5,1.5,2.5,1.4,2.5,1.3,2.5)	0.841	1.00(0.09)	3604
SS	0.2	(1,1,1.6,2.5,1.5,2.5,1.4,2.5,1.3,2.5)	0.783	1.00(0.07)	3454
Dunnett	0.2	(1,1,1.6,2.5,1.5,2.5,1.4,2.5,1.3,2.5)	0.817	1.00(0.07)	3473
Simes	0.2	(1,1,1.6,2.5,1.5,2.5,1.4,2.5,1.3,2.5)	0.772	1.00(0.07)	3490

In Tables 1 and 2, we focus on the type I error rate and the parameter estimators for studies with three treatments and five treatments, respectively. We give results for different proportions (t = n₁/n) of the sample size of the first stage when the responses of all the treatments follow N(0,1). The type I error rate can be well controlled by our methods and the designs using the Dunnett test tend to return a slightly higher type I error rate. In addition, our design can estimate the parameter μ₀ very accurately, and the parameter estimates for the selected treatment arm are consistently larger than 0 as expected due to the treatment selection at the interim look. As the proportion of the sample size of the first stage increases, the bias of the parameter estimates for the selected treatment arm becomes greater. This paper focused on hypothesis testing instead of estimation.

Table 2:

Type I error rate and parameter estimation for seamless trials with five treatments, normal responses, and sample size n = 500

Allocation	t	Type I error	${\widehat{\mu }}_0$	${\widehat{\mu }}_M$
DBCD	0.2	0.050	1.0e-3(0.07)	0.020(0.07)
SS	0.2	0.051	−2.3e-4(0.07)	0.020(0.07)
Dunnett	0.2	0.056	−4.8e-4(0.07)	0.021(0.07)
Simes	0.2	0.043	3.7e-4(0.07)	0.021(0.07)
DBCD	0.3	0.047	1.5e-3(0.07)	0.027(0.06)
SS	0.3	0.051	2.7e-4(0.07)	0.027(0.07)
Dunnett	0.3	0.058	−8.0e-5(0.07)	0.028(0.07)
Simes	0.3	0.047	2.8e-4(0.07)	0.027(0.07)
DBCD	0.4	0.051	−3.9e-4(0.07)	0.034(0.07)
SS	0.4	0.052	4.4e-4(0.07)	0.035(0.07)
Dunnett	0.4	0.054	−7.4e-4(0.07)	0.033(0.06)
Simes	0.4	0.042	5.2e-4(0.07)	0.034(0.07)

In Tables 3 and 4, we focus on the power and report how many times (“Betterwin” in the Tables) out of 10,000 replications the best experimental treatment is selected to be continued at the end of the first stage. Our design resulted in a higher power than other designs including those using the Dunnett test that have relatively higher type I error rate. Our design also performed better than other designs regarding detecting the best treatment, which is of interest to the clinical trial practitioners and represents the convergence rate of our design.

Table 3:

Power of different designs for seamless trials with three treatments, normal responses, and sample size n = 500

Allocation	t	(μ0,σ0,μ1,σ1,μ2,σ2)	Power	${\widehat{\mu }}_0$	Betterwin
DBCD	0.1	(1,1,1.4,2.5,1.2,2.5)	0.615	1.00(0.09)	6007
SS	0.1	(1,1,1.4,2.5,1.2,2.5)	0.554	1.00(0.06)	5889
Dunnett	0.1	(1,1,1.4,2.5,1.2,2.5)	0.549	1.00(0.06)	5974
Simes	0.1	(1,1,1.4,2.5,1.2,2.5)	0.521	1.00(0.06)	5957
DBCD	0.2	(1,1,1.5,2.5,1.3,2.5)	0.784	1.00(0.09)	6400
SS	0.2	(1,1,1.5,2.5,1.3,2.5)	0.727	1.00(0.07)	6188
Dunnett	0.2	(1,1,1.5,2.5,1.3,2.5)	0.745	1.00(0.07)	6191
Simes	0.2	(1,1,1.5,2.5,1.3,2.5)	0.741	1.00(0.07)	6350

In Tables 5 and 6, we used the normal approximation to the Binomial distribution to expand the application of our design in clinical trials with binary responses. Suppose 500 patients sequentially enter the trial. There are three treatments with responses following Bernoulli distribution with success rate p_k = 0,1,2, under study at the first stage, and treatment 0 is the control arm. Our objective is to assign more people to the better treatment. At the first stage, the target allocation proportion is ρ₁ = (p₀, p₁, p₂). At the end of the first stage, the treatment arm with the observed larger success rate is selected to enter the second stage. Then we use the following Urn allocation to implement DBCD if treatment M,M = 1,2, is retained for the second stage:

$${\mathit{\rho }}_2=({\rho }_{20}\mathrm{,}{\rho }_{2M})=\left(\frac{1-p_M}{(1-p_0)+(1-p_M)}\mathrm{,}\frac{1-p_0}{(1-p_0)+(1-p_M)}\right)\mathrm{.}$$

At the end of the trial the chosen treatment M is compared with the control arm, and the following hypothesis is tested:

$$H_0:p_0=p_M\text{ versus }{H}_1:p_M>p_0\mathrm{.}$$

All the results are based on 10,000 replications, and the nominal type I error rate for the above one-sided test is 0.05.

Table 5:

Type I error rate and parameter estimation for seamless trials with three treatments, binary responses, and sample size n = 500

Allocation	p	t	Type I error	${\widehat{p}}_0$	${\widehat{p}}_M$
DBCD	0.5	0.1	0.051	0.499(0.03)	0.504(0.03)
SS	0.5	0.1	0.048	0.500(0.03)	0.505(0.03)
Dunnett	0.5	0.1	0.059	0.500(0.03)	0.505(0.03)
Simes	0.5	0.1	0.050	0.500(0.03)	0.505(0.03)
DBCD	0.4	0.2	0.049	0.399(0.03)	0.405(0.03)
SS	0.4	0.2	0.051	0.400(0.03)	0.407(0.03)
Dunnett	0.4	0.2	0.052	0.400(0.03)	0.407(0.03)
Simes	0.4	0.2	0.047	0.400(0.03)	0.407(0.03)
DBCD	0.6	0.3	0.050	0.599(0.03)	0.607(0.03)
SS	0.6	0.3	0.048	0.600(0.03)	0.609(0.03)
Dunnett	0.6	0.3	0.053	0.600(0.03)	0.609(0.03)
Simes	0.6	0.3	0.046	0.600(0.03)	0.609(0.03)

Table 6:

Power for seamless trials with three treatments, binary responses, and sample size n = 500

Allocation	(p0,p1,p2)	t	Power	${\widehat{p}}_0$	${\widehat{\rho }}_0$	${\widehat{\rho }}_M$	Betterwin
DBCD	(0.7,0.9,0.8)	0.15	0.972	0.695(0.04)	0.268(0.06)	0.732(0.06)	8579
SS	(0.7,0.9,0.8)	0.15	0.970	0.700(0.03)	0.500(0.02)	0.500(0.02)	8370
Dunnett	(0.7,0.9,0.8)	0.15	0.975	0.700(0.03)	0.500(0.02)	0.500(0.02)	8416
Simes	(0.7,0.9,0.8)	0.15	0.973	0.700(0.03)	0.500(0.02)	0.500(0.02)	8382
DBCD	(0.6,0.8,0.7)	0.2	0.955	0.597(0.04)	0.344(0.05)	0.656(0.05)	8525
SS	(0.6,0.8,0.7)	0.2	0.949	0.600(0.03)	0.500(0.02)	0.500(0.02)	8333
Dunnett	(0.6,0.8,0.7)	0.2	0.958	0.600(0.03)	0.500(0.02)	0.500(0.02)	8310
Simes	(0.6,0.8,0.7)	0.2	0.959	0.600(0.03)	0.500(0.02)	0.500(0.02)	8274

In Table 5, we focus on the type I error rate and the parameter estimators for different proportions (t) of the sample size of the first stage and different success rates (p₀ = p₁ = p₂ = p). We found similar conclusions as in Tables 1 and 2. The type I error rate can be well controlled by our methods; the designs using the Dunnett test returned a higher type I error rate; our design can estimate the parameter p₀ very accurately; and the parameter estimates for the selected treatment arm were consistently larger than the estimands as expected due to the treatment selection at the interim look.

In Table 6, we focus on the actual allocation proportions as well as the power and the number of replications out of 10,000 the best experimental treatment is selected to be continued at the end of the first stage. Because a treatment arm has been dropped, the allocation proportions we reported (${\widehat{\rho }}_0\mathrm{,}{\widehat{\rho }}_M$) were calculated based on final total sample sizes of the control arm and the selected treatment arm. Our design assigned considerably more patients to the better treatment without losing much power, which is not only an ethical advantage but also an attractive feature for recruiting clinical trial participants. In addition, our design also performed better than other designs regarding detecting the best treatment.

4. Conclusion

ASD uses the responses to determine which treatment will be continued to the phase III trial, and even to determine whether to stop or continue the trial. RAR uses the previous responses and treatment assignments to sequentially estimate the parameters and update the allocation probability for the next patient. It is conceptually difficult to combine these two types of adaptive designs because (1) all the responses and treatment assignments are correlated with each other in a complicated manner; (2) the number of patients assigned to each treatment is a random variable; and (3) the data used in the treatment selection are also used for inference at the end of the trial. The main contribution of this paper is to propose a family of frequentist RAR designs which can be practically applied in seamless trials with specific methods to control the type I error rate and explicit asymptotic properties.

There are different directions to continue the research for applying RAR designs in seamless trials.

First, hypothesis testing with type I error control is the primary focus of a seamless phase II/III trial in practice, with estimation being an essential but secondary target (Cohen and Sackrowitz, 1989; Troendle and Yu, 1999; Posch et al., 2005; Stallard et al., 2008; Bowden and Glimm, 2008, 2014; Shen, 2001; Stallard and Todd, 2005; Koopmeiners et al., 2012; Kimani et al., 2013; Robertson et al., 2016). This paper focused on hypothesis testing and investigated the estimation in the numerical studies. The theoretical investigation of the bias and variance of the parameter estimators following the proposed design is worth studying in the future.

Second, RAR achieves a variety of objectives by targeting different allocation proportions. This paper studies Neyman allocation and Urn allocation. Other optimal allocations can be used to implement our design, especially when the responses are binary. For example, the optimal allocation proposed by Rosenberger et al. (2001) can be used to minimize the total failure number while fixing the power. Theoretically speaking, our design can perform better regarding different criteria by targeting different optimal allocations. However, the amount of gain may vary. Based on the numerical studies of this paper, when our design was targeting the Neyman allocation for normal responses, we improved the power, but the improvement was not significant; when our design was targeting the urn allocation for binary responses, we assigned significantly more patients to the better treatment. In practice, the choice of the allocation proportion will depend on the practical need.

Third, we can allow stopping for futility after the interim look and require the response of the best performing treatment at the end of the first stage exceed a threshold value to claim efficacy and to be continued for the second stage. In this case,

$$Q^{\prime }=\left\{{\widehat{\mu }}_1(n_1)>{\widehat{\mu }}_2(n_1)>\dots >{\widehat{\mu }}_K(n_1)\mathrm{,}{\widehat{\mu }}_1(n_1)>a\right\}\mathrm{,}$$

where α is the threshold value. Formally, we need to calculate the probability of stopping for futility based on the joint distribution of the parameter estimators at the end of the first stage, and simply adjust the above method by this probability, though no type I error will be spent due to stopping for futility. It is worth noting that these criteria for choosing the empirically best treatment for the second stage may fail to choose the truly best one. However, one major point of seamless trials is to expedite drug development. Therefore, the “near best” treatment that is safe and effective may be a better choice to benefit the population. An alternative is to choose multiple treatments such as several adjacent doses at the interim look to enter the second stage, especially when several treatments have similar efficacy. For example, in the INHANCE trial (Barnes et al. 2010), the minimum clinically important difference and the difference between the active control and the placebo are used to calculate the threshold value for choosing two adjacent doses.

Fourth, we could study other adaptive randomization designs in seamless trials. In this paper, we employed the doubly adaptive biased coin design proposed by Eisele and Woodroofe (1995) and Hu and Zhang (2004). Another important family of RAR is based on urn models, which includes the randomized play-the-winner rule (Wei and Durham, 1978), sequential estimation-adjusted urn models (Zhang et al., 2006), etc. In addition, as the development of personalized medicine, it is worth studying incorporating covariates into clinical trials. So seamless trials with the covariate-adjusted response-adaptive randomization design (Zhang et al., 2007) is also an important direction.

Fifth, we derived the asymptotic results for the proposed design in seamless phase II/III trials, based on which other specific methods could be explored to control the type I error rate. We leave these for future research.

APPENDIX

Proof of Theorem 1

Following the proof of Hu and Zhang (2004), we can obtain that

$$\sqrt{N_k(n_1)}\left({\widehat{\mathit{\theta }}}_k(n_1)-{\mathit{\theta }}_k\right)\to N\left(\mathrm{0,}Var({\mathit{X}}_{1k})\right)\text{ in distribution}\mathrm{,}k=\mathrm{0,1,}\dots \mathrm{,}K\mathrm{.}$$

Next we prove the asymptotic distribution for ${\widehat{\mathit{\theta }}}_0(n)$ that makes use of data from both stages, and the asymptotic distribution for ${\widehat{\mathit{\theta }}}_{M^{\prime }}=\frac{{\sum }_{i=n_1+1}^n{T}_{iM}{\mathit{X}}_{iM}}{n_{M^{\prime }}+1}$ that makes use of data only from the second stage.

Let F_m= σ(T₁,…,T_m,X₁,…,X_m) be the σ−algebra generated by the previous m patients. Then if the mth patient is in stage j,j = 1,2, we have

$$E\left[T_{mk}|F_{m-1}\right]=g_{jk}\left(\mathit{N}\left(m-1\right)/\left(m-1\right)\mathrm{,}{\mathit{\rho }}_j\left(\widehat{\mathit{\theta }}\left(m-1\right)\right)\right),$$

and T_m and X_m are independent under F_m−1.

Let $\Delta Q_{i\mathrm{,}kl}=T_{ik}\left(X_{ikl}-{\theta }_{kl}\right)/v_{2k}\mathrm{,}i=\mathrm{1,}\dots \mathrm{,}n\mathrm{,}k=\mathrm{1,}\dots \mathrm{,}K\mathrm{,}l=\mathrm{1,}\dots \mathrm{,}d$. Write $\Delta {\mathit{Q}}_i=\left(\Delta Q_{i\mathrm{,1}}\mathrm{,}\dots \mathrm{,}\Delta Q_{i\mathrm{,}K}\right)=\left(\Delta Q_{i\mathrm{,}kl};k=\mathrm{1,}\dots \mathrm{,}K\mathrm{,}l=\mathrm{1,}\dots \mathrm{,}d\right)$ and $\mathit{Q}\left(n\right)={\sum }_{i=1}^n\Delta {\mathit{Q}}_i$. Then Q(n) is a sequence of martingales in ${\Re }^{K\times d}$ and $Q\left(n\right)=O\left(\sqrt{n\text{loglog}n}\right)a\mathrm{.}s\mathrm{.}$ Then, we have

$$\begin{gathered} {\widehat{\theta }}_{kl}\left(n\right)-{\theta }_{kl}=\frac{1}{N_k\left(n\right)+1}\sum _{i=1}^n{T}_{ik}{X}_{ikl}-{\theta }_{kl} \\ =\frac{v_{2k}}{N_k\left(n\right)+1}\left(\sum _{i=1}^{n_1}{T}_{ik}\frac{X_{ikl}-{\theta }_{kl}}{v_{2k}}+\sum _{i=n_1+1}^n{T}_{ik}\frac{X_{ikl}-{\theta }_{kl}}{v_{2k}}\right) \\ =\frac{Q_{kl}\left(n_1\right)+\left(Q_{kl}\left(n\right)-Q_{kl}\left(n_1\right)\right)}{n}+\left(\frac{v_{2k}}{N_k\left(n\right)+1}-\frac{1}{n}\right)\left(Q_{kl}\left(n_1\right)+\left(Q_{kl}\left(n\right)-Q_{kl}\left(n_1\right)\right)\right) \\ =\frac{Q_{kl}\left(n_1\right)+\left(Q_{kl}\left(n\right)-Q_{kl}\left(n_1\right)\right)}{n}a\mathrm{.}s\mathrm{.} \end{gathered}$$

Therefore, $\widehat{\mathit{\theta }}\left(n\right)-\mathit{\theta }=\frac{\mathit{Q}\left(n\right)}{n}a\mathrm{.}s\mathrm{.}$

It is worth noting that the RAR design in this paper satisfies all the conditions of Hu and Zhang (2004) except that the target allocation proportion is adjusted once after the first stage. Hu and Zhang (2004) actually proved that Q(n) satisfied the Lindeberg’s condition; we merely need to derive the variances and covariances of the Martingale differences. When 1 ≤ i ≤ n₁, we have

$$Cov\left(\Delta Q_{i\mathrm{,}kl}\mathrm{,}\Delta Q_{i\mathrm{,}km}|F_{i-1}\right)=\frac{1}{v_{2k}^2}{g}_{1k}\left(\frac{N_k(i-1)}{i-1}\mathrm{,}{\widehat{\rho }}_{1k}\right)Cov(X_{ikl}\mathrm{,}{X}_{ikm})$$

When n₁ < i ≤ n, we have

$$Cov\left(\Delta Q_{i\mathrm{,}kl}\mathrm{,}\Delta Q_{i\mathrm{,}km}|F_{i-1}\right)=\frac{1}{v_{2k}^2}{g}_{2k}\left(\frac{N_k(i-1)}{i-1}\mathrm{,}{\widehat{\rho }}_{2k}\right)Cov(X_{ikl}\mathrm{,}{X}_{ikm})$$

Also,

$$Cov\left(\Delta Q_{i\mathrm{,}kl}\mathrm{,}\Delta Q_{i\mathrm{,}sm}|F_{i-1}\right)=\mathrm{0,}k\ne s\mathrm{.}$$

Because of Condition (A6), we have

$$\frac{1}{n}\left(\sum _{i=1}^{n_1}{g}_{1k}\left(\frac{N_k(i-1)}{i-1}\mathrm{,}{\widehat{\rho }}_{1k}\right)+\sum _{i=n_1+1}^n{g}_{2k}\left(\frac{N_k(i-1)}{i-1}\mathrm{,}{\widehat{\rho }}_{2k}\right)\right)\to v_{2k}\mathrm{.}$$

Therefore, we have

$$\sqrt{n}\left(\widehat{\mathit{\theta }}(n)-\mathit{\theta }\right)\to N\left(\mathrm{0,}\Sigma \right)\text{ in distribution}\mathrm{,}$$

where

$$\Sigma =diag\left(\frac{1}{v_{20}}Var({\mathit{X}}_{20})\mathrm{,}\dots \mathrm{,}\frac{1}{v_{2K}}Var({\mathit{X}}_{1K})\right)\mathrm{.}$$

Then we can derive

$$\sqrt{N_k(n)}\left({\widehat{\mathit{\theta }}}_k(n)-{\mathit{\theta }}_k\right)\to N\left(\mathrm{0,}Var({\mathit{X}}_{1k})\right)\text{ in distribution}\mathrm{,}k=0$$

Similarly, we can obtain

$$\sqrt{n_M\text{'}}\left({\widehat{\mathit{\theta }}}_{M^{\prime }}-{\mathit{\theta }}_M\right)\to N\left(\mathrm{0,}Var({\mathit{X}}_{1M})\right)\text{ in distribution }\mathrm{.}$$

Proof of Theorem 2

From Theorem 1, we have

$$\sqrt{N_k(n_1)}\left({\widehat{\mu }}_k(n_1)-{\mu }_k\right)~N(\mathrm{0,}{\sigma }_k^2)\mathrm{,}k=\mathrm{0,1,}\dots \mathrm{,}K\mathrm{,}$$

$$\sqrt{n_{1^{\prime }}}\left({\widehat{\mu }}_{1^{\prime }}-{\mu }_1\right)~N(\mathrm{0,}{\sigma }_1^2)\mathrm{,}$$

and

$$\sqrt{N_0(n)}\left({\widehat{\mu }}_0(n_0)-{\mu }_0\right)~N(\mathrm{0,}{\sigma }_0^2)\mathrm{.}$$

Next, we first assume the standard deviations σ₀,σ₁,…,σ_K are known and the number of patients in each treatment at both stages are fixed, i.e., N(n₁) = n₁v₁ and N(n) = nv₂. Following the idea of Sampson and Sill (2005), we can derive that the conditional pdf for W given T and ${\widehat{\mathit{\mu }}}^{*}$ under H₀ is

$$f\left(W|T\mathrm{,}{\widehat{\mathit{\mu }}}^{*}\right)=C_N\text{exp}\{-\frac{1}{2}{W}^2/\left(\frac{n{v}_{21}n{v}_{20}}{n{v}_{21}{\sigma }_0^2+n{v}_{20}{\sigma }_1^2}\right)\}\Phi \left\{\frac{\sqrt{n_1{v}_{11}n{v}_{21}}\left(T+\frac{{\sigma }_1^2}{n{v}_{21}}W-{\widehat{\mu }}_2(n_1)\right)}{\sqrt{(n-n_1)v_{21}}{\sigma }_1}\right\}$$

where C_N is the normalization constant which involves integrating W over the real line. We skip the lengthy calculation to save space.

From previous proof and conditions, we have ${\widehat{\sigma }}_k(n)\mathrm{,}k=\mathrm{0,1,}\dots \mathrm{,}K$ are consistent estimator of ${\sigma }_k$. We also have N(n₁) → n₁v₁ in Probability, and N(n₁) → n₁v₂ in Probability. Let $\mathit{A}=({\widehat{\mu }}_1(n_1)\mathrm{,}{\widehat{\mu }}_{1^{\prime }}\mathrm{,}{\widehat{\mu }}_0(n))$, $\mathit{B}=(N_1(n_1)\mathrm{,}{n}_{1^{\prime }}\mathrm{,}{N}_1(n)\mathrm{,}{N}_0(n))$, and $\mathit{c}=(n_1{v}_{11}\mathrm{,}n{v}_{21}-n_1{v}_{11}\mathrm{,}n{v}_{21}\mathrm{,}n{v}_{20})$. Then it is easy to see that W′ and T′ can be written as h₁(A,B) and h₂(A,B) respectively, where h₁ and h₂ are two continuous functions. We also have W = h₁(A,c) and T = h₂(A,c). We can prove that for all continuous bounded functions l₁ and l₂,

$$E\left[l_1(h_1(\mathit{A}\mathrm{,}\mathit{B})l_2(h_2(\mathit{A}\mathrm{,}\mathit{B}))\right]-E\left[l_1(h_1(\mathit{A}\mathrm{,}\mathit{c})l_2(h_2(\mathit{A}\mathrm{,}\mathit{c}))\right]\to 0.$$

So the conditional CDF of W′ = h₁(A,B) given T′ = h₂(A,B) and ${\widehat{\mathit{\mu }}}^{*}$ is asymptotically the conditional CDF of W = h₁(A,c) given T = h₂(A,c) and ${\widehat{\mathit{\mu }}}^{*}$