A Bayesian adaptive design with biomarkers for targeted therapies

Abstract

Background

Targeted therapies are becoming increasingly important for the treatment of various diseases. Biomarkers are a critical component of a targeted therapy as they can be used to identify patients who are more likely to benefit from a treatment. Targeted therapies, however, have created major challenges in the design, conduct, and analysis of clinical trials. In traditional clinical trials, treatment effects for various biomarkers are typically evaluated in an exploratory fashion and only limited information about the predictive values of biomarkers obtained.

Purpose

New study designs are required, which effectively evaluate both the diagnostic and the therapeutic implication of biomarkers.

Methods

The Bayesian approach provides a useful framework for optimizing the clinical trial design by directly integrating information about biomarkers and clinical outcomes as they become available. We propose a Bayesian covariate-adjusted response-adaptive randomization design, which utilizes individual biomarker profiles and patient's clinical outcomes as they become available during the course of the trial, to assign the most efficacious treatment to individual patients. Predictive biomarker subgroups are determined adaptively using a partial least squares regression approach.

Results

A series of simulation studies were conducted to examine the operating characteristics of the proposed study design. The simulation studies show that the proposed design efficiently identifies patients who benefit most from a targeted therapy and that there are substantial savings in the sample size requirements when compared to alternative designs.

Limitations

The design does not control for the type I error in the traditional sense and a positive result should be confirmed by conducting an independent phase III study focusing on the selected biomarker profile groups.

Conclusions

We conclude that the proposed design may serve a useful role in the early efficacy phase of targeted therapy development.

Introduction

With the availability of new genomic and proteomic technologies, biomarkers are becoming increasingly important in both drug discovery and development. A biomarker is defined as a characteristic that is objectively measured or evaluated as an indicator of a biological or pharmacological response to a therapeutic intervention [1]. Biomarkers have the potential to allow for the selection of patients who are more likely to benefit from a targeted therapy. For example in cancer clinical trials, biomarkers can serve as a molecular indication of drug efficacy or toxicity. This allows individual patients to be treated based on the molecular determinants of their tumor cells. Biomarkers can be used to stratify patients, to make diagnosis, and to guide treatment [2]. The development of biomarkers for diagnosis and prognosis of personalized therapies allows the targeting of individualized treatments to patients most likely to benefit. New molecular profiling technologies that allow comprehensive analysis of single nucleotide polymorphisms, gene expression, and protein profiles have resulted in a vast increase in the data available for biomarkers development [3].

At the design stage of the early clinical trial phases, the treatment effect sizes for the general patient population and the biomarker sub-populations are typically unknown. In traditional early efficacy phase clinical trials, patients are treated with the new drug and outcomes in patients with a positive biomarker are compared to outcomes in patients with a negative biomarker in an exploratory fashion. These trials may provide limited prognostic information of the biomarker; however, they do not provide any predictive information.

Several design strategies have been proposed for the prospective validation of biomarkers in the phase III setting [4–7]. Sargent et al. [6] divided clinical trial designs for predictive marker validation into two classes: biomarker-by-treatment interaction design and biomarker-based-strategy design. In the biomarker-by-treatment interaction design, patients are stratified according to their biomarker status, that is positive versus negative biomarker. Patients in each stratum are randomized to receive either the experimental arm or control arm. In the biomarker-based-strategy design, patients are randomized into a biomarker based-strategy arm and a nonbiomarker-based strategy arm. In the biomarker-based-strategy arm, patients with a positive biomarker are assigned to the experimental arm while patients with a negative biomarker are assigned to the control arm. In the nonbiomarker-based strategy arm, all patients are assigned to the control arm. In a modified version of the biomarker-based-strategy design, patients in the nonbiomarker-based strategy arm undergo a second randomization, which assigns them to the experimental or control arm. This modification allows clarification of whether any finding regarding the efficacy of the biomarker directed approach to therapy is due to a true effect of the biomarker status or to an improved regimen regardless of biomarker status [6]. Moreover, this design may also allow a retrospective assessment of an alternative classification for the biomarker. As pointed out by Sargent et al. [6], the choice of the design for any particular trial depends on the nature of the conclusion that one wishes to draw and the strength of the evidence desired at the trial's conclusion. The advantage of the biomarker-based strategy design over the biomarker-by-treatment interaction design is that it allows an assessment of the prognostic value of the biomarker. However, biomarker-based-strategy design generally requires a larger sample size than the biomarker-by-treatment interaction design.

Simon et al. [8] evaluated the relative efficiency of a targeted versus an untargeted clinical trial design for a randomized clinical trial design comparing a new treatment to a control. The untargeted trial design is the standard approach where patients are randomized into the treatment or control arm regardless of biomarker status. In the targeted trial design, only those patients who are predicted to respond, based on their biomarker profile, are enrolled and randomized into the treatment or control arm. For example, the prediction of whether a patient will respond to a treatment may be based on an assay that measures expression levels, or a multivariate gene expression model derived from transcript profiling [9–11]. Simon et al. [8] compared the two designs with regard to the number of patients required to achieve a fixed statistical power for detecting treatment effects. They showed that a targeted clinical trial design often requires fewer randomized patients than the untargeted design. The degree of reduction in sample size depends on the availability of the biomarker for identifying patients who will benefit from the new treatment and the prevalence of such patients. When the new treatment benefits only a subset of patients and those patients can be accurately identified, then the targeted design requires fewer patients than the untargeted design. However, targeted designs may lose some relative efficiency when there is a partial treatment effect for biomarker negative patients, possibly because of multiple potential mechanisms of action [8]. Therefore, while it is clear that a biomarker-based strategy can increase the efficiency of the trial, much depends on the performance of the diagnostics, the prevalence of the biomarker, and the size of the treatment effect for target negative patients.

An adaptive design is a design that allows modifications of some aspects of the trial after its initiation without undermining the validity and integrity of the trial. It provides a mechanism for incorporating biomarker information during clinical trials. Adaptive methods for clinical trials have been studied extensively by many authors [12–16]. For example, in a response adaptive clinical trial, patient outcomes can be used as they become available to adjust the allocation ratio between patients. This allows one to improve expected patient outcomes during the experiment, while still being able to reach good statistical decisions in a timely fashion. As we enter the era of personalized medicine, the role of adaptive designs in drug discovery and development has become increasingly important. Zhou et al. [17] recently proposed a Bayesian adaptive design for targeted therapy using pre-defined biomarker profile groups.

In this article, we propose a Bayesian covariate-adjusted response-adaptive (BCARA) randomization design for targeted therapies, which utilizes individual patient's biomarker profiles and clinical outcomes as they become available and which identifies subgroups of patients who respond best to a targeted therapy. Predictive biomarker subgroups are determined adaptively using a partial least squares logistic regression approach. The remainder of this article is organized as follows. In the ‘Methods’ section, the study design and computational aspects for implementing the design are described. The operating characteristics based on simulation studies are examined in the ‘Simulation studies’ section. Finally, a brief summary is given in the ‘Discussion’ section.

Methods

A clinical trial with biomarkers evaluations: Incorporating biomarker Information in assessing treatment efficacy

In the following, we assume a clinical trial with J treatments where the primary outcome is a dichotomous response variable, which can be measured within a short time period. For example, in cancer clinical trials, pathologic response, defined as the absence of residual tumors, can be typically measured via biopsy within 1 month after administrating treatment. It is assumed that measurements from a set of K biomarkers are available where the predictive or prognostic status of each biomarker is unknown. Let y_t denote the outcome for the t-th subject with

$$y_t=\{\begin{array}{cc}1& \text{if subject}t\text{has a response},\\ 0& \text{else}.\end{array}$$

Furthermore, let x_t = (x_1t, …, x_Kt)^T denote a set of K biomarkers for subject t. It is assumed that each subject's biomarker assessment is performed at or before study entry and that each biomarker is measured on a quantitative scale. For example, pharmacogenomic biomarkers are often measured quantitatively using immunohistochemistry staining with a scale ranging from 0 to 3. The main objective is to develop an adaptive clinical trial design which evaluates the therapeutic intervention of targeted therapy, identifies subset of subjects who respond better to a targeted therapy, and optimize the treatment allocation by randomizing more subjects to the superior treatment arm based on each subject's individual biomarker profile.

Bayesian logistic regression model for predicting clinical outcome

We use a Bayesian logistic regression approach to predict response to a treatment of a newly accrued subject. The Bayesian approach provides a powerful framework for optimizing the clinical trial design by integrating information about biomarkers and clinical outcomes as they become available during the trial. Bayesian logistic regression is a natural choice to this problem as it can select predictive biomarkers among a large numbers of biomarkers without loss of model performance. Let

$${\mathit{y}}^{(i)}={(y_1,\dots ,y_i)}^T$$

denote the response vector of the first i subjects.

For subject t in treatment arm j, the probability of a response is modeled as

$$Pr(y_t=1|{\mathit{\theta }}_j;{\mathit{x}}_t)=\psi ({\mathit{\theta }}_j^T{\mathit{x}}_t)=exp({\mathit{\theta }}_j^T{\mathit{x}}_t)/(1+exp({\mathit{\theta }}_j^T{\mathit{x}}_t))$$

where θ_j = (θ₁, …, θ_K)^T has the prior distribution N(μ_j, Σ_j), that is, a multivariate normal distribution, with mean μ_j and covariance matrix Σ_j.

Holmes and Held [18] proposed an auxiliary variable approach to generate samples from the posterior distribution. Let

$$y_t=\{\begin{array}{cc}1& \text{if}{Z}_t>0,\\ 0& \text{if}{Z}_t\le 0,\end{array}$$

Where $z_t={\mathit{\theta }}_j^T{\mathit{x}}_t+{\in }_t,{\in }_t\sim N(0,{\lambda }_t),{\lambda }_t={(2{\omega }_t)}^2$ where ω_t has a Kolmogorov–Smirnov distribution [19]. The error term ε_t has a scale mixture of normal form with a marginal logistic distribution. Let

$${\mathit{j}}_i={(j_1,\dots ,j_i)}^T$$ (1)

denote the i-dimensional treatment assignment vector, which specifies that subject t has been assigned to treatment j_t ε(1, …, J). Therefore, the conditional distribution of y⁽ⁱ⁾ given θ_j1, …, θ_ji follows an i-dimensional Bernoulli distribution with the means of $\psi ({\mathit{\theta }}_{j_1}^T{\mathit{x}}_1),\dots ,\psi ({\mathit{\theta }}_{j_i}^T{\mathit{x}}_i)$.

For subjects 1, …, i, let X⁽ⁱ⁾ = (x₁, …, x_i)^T, λ⁽ⁱ⁾ = (λ₁, …, λ_i)^T and Z⁽ⁱ⁾ = (z₁, …, z_i)^T, where $z_t={\mathit{\theta }}_{jt}^T{\mathit{x}}_t+{\in }_t$ for t = 1, …, i. The conditional distribution of θ_j|z⁽ⁱ⁾, λ⁽ⁱ⁾, y⁽ⁱ⁾ has a normal distribution with mean $B_j^{(i)}$ and covariance matrix ${\mathrm{\Xi }}_j^{(i)}$ where

$$\begin{array}{l}{\mathbf{B}}_j^{(i)}=({\sum }_j^{-1}+{\mathit{X}}^{(i)T}{(\text{diag}({\lambda }^{(i)}))}^{-1}{\mathit{X}}^{(i)})\times ({\sum }_j^{-1}{\mu }_j+{\mathit{X}}^{(i)T}{(\text{diag}({\lambda }^{(i)}))}^{-1}{z}^{(i)}),\\ {\mathrm{\Xi }}_j^{(i)}=({\sum }_j^{-1}+{\mathit{X}}^{(i)T}{(\text{diag}({\lambda }^{(i)}))}^{-1}{\mathit{X}}^{(i)}).\end{array}$$

A straightforward Gibbs sampling strategy can be implemented where Z_t is sampled from a truncated normal distribution with

$$Z_t|{\mathit{\theta }}_{jt},y_t,{\lambda }_t\alpha \{\begin{array}{cc}N({\mathit{\theta }}_{j_t}^T{\mathit{x}}_t,{\lambda }_t)I(Z_t>0)& \text{if}{y}_t=1,\\ N({\mathit{\theta }}_{j_t}^T{\mathit{x}}_t,{\lambda }_t)I(Z_t\le 0)& \text{if}{y}_t=0,\end{array}$$

for t = 1, …, i. The conditional distribution p(λ_t|Z_t, θ_jt) does not have a closed form. However, samples can be conveniently generated from this conditional distribution using standard rejection sampling [20]. Let

$${\mathit{\theta }}_{j_i}={\mathit{\theta }}_{j_1},\dots ,{\mathit{\theta }}_{j_i},$$

for subjects 1, …, i. Holmes and Held [18] proposed an efficient block Gibbs sampling strategy using iterative updates, that is z⁽ⁱ⁾, λ⁽ⁱ⁾|θ_ji followed by θ_ji|z⁽ⁱ⁾, λ⁽ⁱ⁾. In this approach, z_t|θ_jt, y_t follows a truncated logistic distribution, that is,

$$Z_t|{\mathit{\theta }}_{j_t},y_t,{\lambda }_t=\{\begin{array}{cc}\text{logistic}({\mathit{\theta }}_{j_t}^T{\mathit{x}}_t,1)I(Z_t>0)& \text{if}{y}_t=1,\\ \text{logistic}({\mathit{\theta }}_{j_t}^T{\mathit{x}}_t,1)I(Z_t\le 0)& \text{if}{y}_t=0,\end{array}$$

where logistic $({\mathit{\theta }}_{j_t}^T{\mathit{x}}_t,1)$ denotes the density function of the logistic distribution with mean ${\mathit{\theta }}_{j_t}^T{\mathit{x}}_t$ and scale 1.

The Gibbs sampling strategy described above can be conveniently implemented to generate samples from the posterior distribution θ_ji|z⁽ⁱ⁾.

Bayesian covariate-adjusted response-adaptive randomization

Response-adaptive randomization is a randomization technique in which the allocation of patients to the study arms is based on the responses of the previous outcomes. Response adaptive designs have been extensively studied [see, e.g., 12,21–23]. A covariate-adjusted response-adaptive randomization design can be used to incorporate biomarker information, as they become available during the conduct of the clinical trial, into a decision making process to evaluate both the diagnostic and the therapeutic intervention using biomarkers. One challenge involves the problem of how to incorporate information from multiple, possibly correlated, biomarkers into the design. The Bayesian paradigm may provide an ideal solution to both the diagnostic and the therapeutic intervention using multiple biomarkers with a covariate-adjusted response-adaptive randomization design. A BCARA randomization can be implemented using the following steps. For subject i + 1, we compute

$${\mathrm{E}}_{j_i,j}(y_{i+1}|{\mathit{\theta }}_j;x_{i+1})={Pr}_{j_i,j}(y_{i+1}=1|{\mathit{\theta }}_j;{\mathit{x}}_{i+1})$$

with respect to the posterior distribution π(θ_ji|y⁽ⁱ⁾; X⁽ⁱ⁾), that is the probability of a response for patient i + 1 treated with treatment j. The index j_i in the expectation and probability above refers to the treatment assignment vector defined in (1). The posterior distribution π(θ_ji|y⁽ⁱ⁾; X⁽ⁱ⁾) can be computed using the Gibbs sampling strategy described in the section ‘Bayesian logistic regression model for predicting clinical outcome’. Furthermore, we compute

$$p_j(y_{i+1})={Pr}_{j_i,j}(\underset{j^{\text{'}}\ne j}{\cap }{\mathbf{E}}_{{\mathbf{j}}_i,j}(y_{i+1,j}|{\mathit{\theta }}_j;{\mathit{X}}_{i+1})>{\mathrm{E}}_{j_i,j\text{'}}(y_{i+1,j\text{'}}|{\mathit{\theta }}_{j\text{'}};{\mathit{X}}_{i+1})|y_j^{(i)};{\mathit{X}}^{(i)})$$ (2)

for each j = 1, …, J. The probabilities in (2) can be used to calculate the randomization allocation probabilities, that is, the probabilities of assigning subject i + 1 to treatment arms j = 1, …, J. Specifically, the randomization rate for subject i +1 and treatment j is be computed as

$${RR}_j(i+1)={(p_j(y_{i+1}))}^c/\sum _{j\text{'}=1}^J{(p_{j\text{'}}(y_{i+1}))}^c,$$ (3)

for a constant c > 0. Following the recommendations from Thall and Wathen [24] for Bayesian response adaptive designs we set c = (i + 1)/(2N_max) in (3) where N_max is the trial's maximum sample size.

Before the BCARA randomization is implemented, a run-in phase is required. During the run-in phase, the first n* subjects are randomized to the treatment arms using a standard randomization procedure. A minimum of n* = J × (K + 1) subjects is required for the run-in phase, where J denotes the number of treatment groups and K the number of biomarkers.

Adaptive determination of predictive biomarker profile groups

A major objective of the study design is to identify subgroups of subject who are likely to respond better to a targeted therapy. We use a partial least squares regression (PLSR) approach to identify predictive biomarker subgroups. PLSR has become a popular dimension reduction tool that is based on a latent variables approach [25,26]. Contrary to principal component analysis, which attempts to find linear combinations of the predictors that explain most of the variation in these predictors using a small number of components, PLSR takes into account information about both the predictors and outcome variable in the definition of scores and loadings. Specifically, PLSR finds linear combinations of the predictors that, in addition to the maximal variance constraint, also best explain the response. Originally, PLSR methods were developed for continuous response variables. Bastien et al. [27] extended PLSR to partial least squares generalized linear regression models.

We use a partial least squares logistic regression (PLSLR) approach to classify subjects into biomarker profile groups which predict the clinical outcome. The biomarker profile groups are determined iteratively, that is, after each new subject has entered the study. Let V⁽ⁱ⁾ = (V₁, …, V_i)^T where V_t = (v_t1, …, v_t,_J-1)^T denotes the vector of a dummy variables, that is v_tj = 1 if subject t(t = 1, …, i) is assigned to treatment j (j = 1, …, J–1) and zero otherwise. Furthermore, let W⁽ⁱ⁾ denote the i × (K + J-1) matrix which consists of the standardized columns of X⁽ⁱ⁾ and V⁽ⁱ⁾. The PLSLR model of y⁽ⁱ⁾ on W⁽ⁱ⁾ with m components can be written as

$$g({\eta }^{(i)})=\sum _{h=1}^m{b}_h^{(i)}=\sum _{h=1}^m{C}_h\left(\sum _{l=1}^{K+J-1}{\beta }_{hl}^{\ast }{W}_l^{(i)}\right),$$ (4)

where c_h and ${\beta }_{\mathrm{h}\mathrm{l}}^{\ast }$ are the parameters of the PLSLR model, g(·) is the canonical link function for the binomial distribution and m ≤ K + J − 1.

The PLS components $b_h^{(i)}$ of (4) can be computed iteratively, using the steps shown below. The computation of the first PLS component $b_1^{(i)}$ is performed as follows:

1. Compute the regression coefficients a_1l of the logistic regression models of y⁽ⁱ⁾ on ${\mathit{w}}_l^{(i)}$ for each l = 1, …, K + J −1, where ${\mathit{w}}_l^{(i)}$ denotes column l of W⁽ⁱ⁾.

2. Compute ${\zeta }_1={(a_{11},\dots ,a_{1,K+J-1})}^T/{\sum }_{l=1}^{K+J-1}{a}_{1l}^2.$

3. Compute the first PLS component ${\mathit{b}}_1^{(i)}={\mathit{W}}^{(i)}{\zeta }_1/{\zeta }_1^T{\zeta }_1.$

The h-th PLS component is computed using the following steps:

1. Fit the logistic regression model of y⁽ⁱ⁾ on ${\mathit{w}}_1^{(i)}$ and ${\mathit{b}}_1^{(i)},\dots ,{\mathit{b}}_{h-1}^{(i)}$ for each l = 1, …, K + J − 1 and compute the regression coefficient a_hl, which corresponds to the predictor ${\mathit{w}}_l^{(i)}$ for each logistic regression model.

2. Compute ${\zeta }_h={(a_{hl},\dots ,a_{h,K+J-1})}_{(i)}^{\mathrm{T}}/{\sum }_{l=1}^{K+J-1}{a}_{hl}^2.$

3. Compute the residual matrix ${\mathit{R}}_{h-1}^{(i)}$ of the linear regression model of W⁽ⁱ⁾ on ${\mathit{b}}_1^{(i)},\dots ,{\mathit{b}}_{h-1•}^{(i)}$

4. Compute the h-th PLS component ${\mathit{b}}_h^{(i)}={\mathit{R}}_{h-1}^{(i)}{\zeta }_h/{\zeta }_h^T{\zeta }_h.$

The number m of PLS components to be retained can be chosen by evaluating the predictive power of the current PLSLR model after each iteration using a cross-validation procedure [27]. Specifically, the algorithm is terminated if the predictive power of the PLSLR model does not increase after adding a new PLS component.

The number of biomarker subgroups can be determined after the run-in phase of the trial when the first n* subjects are randomized to the treatment arms using a standard equal randomization procedure. In the simplest scenario, subjects can be divided into two biomarker profile groups. For example, if after the run-in phase of the trial it is concluded that the PLSLR model with the first PLS component $b_1^{(i)}$ has a sufficiently high predictive power, the two biomarker subgroups can be defined as follows:

$${\delta }_t({\mathit{y}}^{(i)})=\{\begin{array}{cc}1& \begin{array}{l}\text{if}{b}_{1t}^{(i)}>0,\text{i.e}.,\text{subject}t(t=1,\dots ,i)\\ \text{has a positive biomarker profile},\end{array}\\ 0& \begin{array}{l}\text{if}{b}_{1t}^{(i)}\le 0,\text{i.e}.,\text{subject}t(t=1,\dots ,i)\\ \text{has a negative biomarker profile},\end{array}\end{array}$$

A larger number of biomarker profile groups can be defined in a similar fashion.

Stopping rule

It is assumed that g = 1, …, G different biomarker profile groups have been identified after the run-in phase. Let

$$p_{jg}({\mathit{y}}^{(i)})={Pr}_{{\mathit{j}}_i,j}(\underset{j\text{'}\ne j}{\cap }{E}_{{\mathit{j}}_i,j}(\sum _{t=1}^i{y}_t|{\mathit{\theta }}_j;{\mathit{x}}_t)>E_{{\mathit{j}}_i,j\text{'}}(\sum _{t=1}^i{y}_t|{\mathit{\theta }}_{j\text{'}};{\mathit{x}}_t)|{\delta }_1({\mathit{y}}^{(i)})=g,\dots ,{\delta }_i({\mathit{y}}^{(i)})=g),$$ (5)

where index j_i refers to the treatment assignment vector defined in (1). The probability above can be conveniently computed using Gibbs sampling techniques described in the section ‘Bayesian Logistic regression model for predicting clinical outcome’.

The following stopping rule will be employed after the i-th subject has been enrolled and evaluated for response:

1. If max_{j=1, …, J}{P_jg(y⁽ⁱ⁾)} > δ, for a pre-specified stopping criterion δ, for example, 0.99 or 0.999, the trial is stopped after subject i has been evaluated for response. In this case, treatment j* = argmax_{j = 1, …, J}{p_jg(y⁽ⁱ⁾)} for the biomarker profile group g is declared as superior over all other treatment-biomarker group combinations.

2. If a pre-specified maximum sample size N_max is reached and P_jg(y^(N_max)) ≤ δ for all biomarker groups g = 1, …, G and treatments j = 1, …, J, then the trial is declared as inconclusive.

Step-by-step implementation

This section summarizes the steps for implementing the proposed study design:

• Step 1: Run-in phase: The first n* ≥ J × (K + 1) subjects are randomized to the treatment arms using a standard randomization procedure, where J denotes the number of treatment groups and K the number of biomarkers. It is required that at least one response is observed in each of the J treatment groups before proceeding to the Bayesian response adaptive randomization.

• Step 2: Adaptive determination of the predictive biomarker profile groups: Predictive biomarker profile groups are determined adaptively after each new patient has entered the trial. The biomarker status is determined before the randomization. The PLSLR approach is used to determine the predictive biomarker profile groups as described in the section ‘Adaptive determination of predictive biomarker profile groups’.

• Step 3: Bayesian covariate-adjusted response-adaptive randomization: After the biomarker status and biomarker profile groups of a newly accrued subject have been determined, the subject is randomized into one of the treatment arms using a BCARA randomization (3).

• Step 4: Decision:

  1. Stop the trial, if for a pre-specified stopping criterion δ, max_{j=1, …, J} {P_jg(y⁽ⁱ⁾)} > δ where max_{j=1, …, J} {P_jg(y⁽ⁱ⁾)} is defined in (5). In this case, treatment j* = argmax_{j=1, …, J}{P_jg(y⁽ⁱ⁾)} for the biomarker profile group g is declared as superior over all other treatment-biomarker group combinations.
  2. Continue the trial, if max_j=1, …, J {P_jg(y⁽ⁱ⁾)} ≤ δ.
  3. Stop the trial, if a pre-specified maximum sample size N_max is reached and $p_j^g({\mathbf{y}}^{(N_{max})})\le \delta$for all biomarker groups g = 1, …, G and treatments j = 1, …, J. In this case the trial is declared as inconclusive.

Simulation studies

Simulation study I

We performed a series of simulation studies to evaluate the operating characteristics of the proposed BCARA randomization design where subject's memberships in biomarker subgroups were determined adaptively using the PLSLR approach. In the first simulation study, the basic scenario was a three-arm study with treatments A, B, and C. A set of K = 5 biomarker measurements were generated from a multivariate normal distribution with means 0, variances 1 and correlations ρ = 0.50. For the targeted therapy (treatment A), various levels of predictive power of the five biomarkers were assumed. Specifically, four different scenarios were considered. In the first scenario, it was assumed that the biomarkers were not predictive with a polyserial correlation of ρ_ps = 0 between each of the biomarkers and response. In the second scenario, a weak association between each of the five biomarkers and treatment response to the targeted therapy, that is, treatment A, was assumed with a polyserial correlation of ρ_ps = 0.1. In the third scenario, a moderately strong association between each of the five biomarkers and treatment response was assumed with a polyserial correlation of ρ_ps = 0.5. In the last scenario, a strong association was assumed with a polyserial correlation of ρ_ps = 0.8. Furthermore, it was assumed that the biomarkers were uncorrelated with treatment response for treatment B and C. The overall response rates for the targeted therapy, that is treatment A, were assumed to be 0.5, 0.6, and 0.7. The overall response rates for treatments B and C were assumed to be 0.4. The stopping criterion parameter for terminating the trial was fixed at 0.99 and 0.999, respectively. A maximum sample size of N_max = 200 was used for each hypothetical trial. The run-in phase, during which subjects were randomized to one of the three treatment arms using equal randomization, consisted of the first 24 subjects. Afterward, the BCARA randomization design described in the section ‘Bayesian covariate-adjusted response-adaptive randomization’ was used to randomize subjects into the three treatment arms where the prior distribution for θ_j was assumed to be normally distributed with mean zero and Σ_j = 100 I₅ for j = 1, 2, 3. Biomarker groups were determined using the PLSLR approach as described in the section ‘Adaptive determination of predictive biomarker profile groups’. Subjects were adaptively classified in biomarker positive or biomarker negative groups according the sign of first PLS component, that is subjects were classified as biomarker positive if $b_{1t}^{(i)}>0$ for t = 1, …, i and i = n* + 1, …, N_max and as biomarker negative otherwise. Each scenario was simulated M = 500 times. The results are summarized in Table 1.

Table 1. Operating characteristics of BCARA randomization design where biomarker subgroups are determined adaptively using the PLSLR approach.

RRAa	ρAb	RRA+c	RRA−d	δ	Ne	PA+f	NAg	ProbA+h	ProbB+i	Problj
0.50	0.00	0.50	0.50	0.990	149	0.40	31	0.22	0.05	0.63
	0.50	0.50	0.999	195	0.36	49	0.04	0.03	0.89
	0.10	0.53	0.47	0.990	145	0.51	39	0.25	0.05	0.51
	0.53	0.47	0.999	190	0.45	77	0.05	0.02	0.89
	0.50	0.66	0.34	0.990	93	0.45	35	0.57	0.06	0.12
	0.66	0.34	0.999	162	0.45	61	0.52	0.00	0.48
	0.80	0.80	0.20	0.990	86	0.53	30	0.82	0.03	0.05
	0.80	0.20	0.999	105	0.46	46	0.71	0.03	0.17
0.60	0.00	0.60	0.60	0.990	116	0.39	24	0.27	0.05	0.34
	0.60	0.50	0.999	172	0.34	42	0.15	0.03	0.66
	0.10	0.64	0.56	0.990	110	0.45	43	0.29	0.08	0.29
	0.64	0.56	0.999	163	0.48	70	0.19	0.04	0.58
	0.50	0.76	0.44	0.990	92	0.46	35	0.82	0.02	0.06
	0.76	0.44	0.999	125	0.46	48	0.68	0.04	0.14
	0.80	0.88	0.32	0.990	83	0.48	31	0.90	0.01	0.03
	0.88	0.32	0.999	91	0.47	36	0.84	0.02	0.05
0.70	0.00	0.70	0.70	0.990	90	0.39	21	0.35	0.07	0.22
	0.70	0.70	0.999	141	0.36	35	0.19	0.06	0.39
	0.10	0.73	0.67	0.990	86	0.46	31	0.39	0.07	0.20
	0.73	0.67	0.999	140	0.44	61	0.24	0.08	0.33
	0.50	0.85	0.55	0.990	73	0.49	26	0.89	0.01	0.04
	0.85	0.55	0.999	110	0.49	37	0.70	0.03	0.16
	0.80	0.94	0.56	0.990	66	0.53	25	0.94	0.00	0.02
	0.94	0.56	0.999	84	0.48	35	0.87	0.01	0.01

^aOverall response rate of targeted therapy (treatment A).

^bPolyserial correlation between response and individual biomarkers for treatment A.

^cResponse rate of treatment A in biomarker positive group.

^dResponse rate of treatment A in biomarker negative group.

^eMean total sample size across 500 replications.

^fProportion of subjects with positive biomarker profile randomized to treatment A.

^gMean number of subjects randomized to treatment A.

^hProbability of declaring treatment A in subjects with positive biomarker profile superior.

ⁱProbability of declaring treatment B in subjects with positive biomarker profile superior.

^jProbability of inconclusive result.

The mean total sample size for the first scenario with ρ_ps = 0 and δ = 0.99 ranged from 90 to 149 and approximately 40% of the subjects with a positive biomarker profile were randomized to treatment arm A. For the second scenario with ρ_ps = 0.1 and δ = 0.99, the mean total sample size ranged from 86 to 145. As expected, more subjects from the biomarker positive group were randomized to the targeted treatment arm, that is 51% of the subjects with a positive biomarker profile were randomized to the targeted therapy arm when the overall response rate was 50%, and 46% of the subjects with a positive biomarker profile were randomized to that arm when the overall response rate was 70%. When the overall response of the targeted therapy was 50%, the probability that the targeted therapy in subjects with a positive biomarker profile was declared as superior over all other treatment-biomarker group combinations was 25%. There was a 51% probability that the maximum sample size N_max = 200 was reached without declaring one of the treatment-biomarker groups as superior when the overall response rate of the targeted therapy was 50%. However, this probability decreased to 29% and 20% when the overall response rate of the targeted therapy increased to 60% and 70%, respectively. Changing the stopping criterion parameter δ from 0.99 to 0.999 resulted in a substantial increase in the mean total sample size. For example, when the overall response rate of the targeted therapy was 50%, the mean total sample size increased from 145 to 190. Furthermore, changing the stopping criterion parameter δ from 0.99 to 0.999 resulted in a increase in the probability of an inconclusive result.

Under the third and fourth scenarios, that is, assuming a moderately strong and strong association between biomarkers and response to the targeted therapy, the mean total sample sizes were substantially less. For example, assuming a strong association between biomarkers and treatment response and using a stopping criterion parameter δ = 0.99, the mean total sample sizes ranged from 66 to 86. There were no changes in the treatment allocations. However, there was a substantial increase in the sensitivity for the fourth scenario when compared to the first and second scenarios. Specifically, assuming an overall response rate for the targeted therapy of 50% and a stopping criterion of δ = 0.99, the probability of declaring the targeted therapy as superior in subjects with a positive biomarker profile increased from 25% under the second scenario to 57% under the third scenario and to 82% under the fourth scenario.

Simulation study II

We conducted a second simulation study to evaluate the performance of the design under various scenarios regarding the predictive or prognostic status of biomarkers. For each simulated hypothetical clinical trial, a study with two treatment arms (A and B) and a set of K = 4, quantitatively measured biomarkers. The biomarker observations were simulated from a multivariate normal distribution with means of zero and a covariance matrix of $\sum =(1-\rho )I_4+\rho I_4{11}_4^T.$ Furthermore, it was assumed that there are two biomarker groups. Specifically, the true biomarker group status was classified as ‘+’ if at least two of four biomarkers have values greater or equal to 0, otherwise the biomarker group was classified as ‘−’. We considered four different scenarios. In the first scenario, it was assumed that the biomarker group status is neither predictive nor prognostic. In the second scenario, it was assumed that the biomarker group status is prognostic while in the third scenario it was assumed that it is predictive. Finally, in fourth scenario, it was assumed that the biomarker group status is both predictive and prognostic. The true response rates for the four scenarios are shown in Table 2.

Table 2. Design for simulation study II.

Treatment	Biomarker group	Scenario 1: Not predictive or prognostic	Scenario 2: Prognostic	Scenario 3: Predictive	Scenario 4: Predictive and prognostic
A	+	0.3	0.6	0.6	0.7
	−	0.3	0.3	0.3	0.1
B	+	0.3	0.6	0.3	0.5
	−	0.3	0.3	0.3	0.4

The maximum sample size for each trial was N_max = 200. Hypothetical trials were simulated M = 500 times. A standard 1:1 randomization was used for the first n* = 12 subjects during the run-in phase. The biomarker groups status was determined using the PLSLR approach as described in the section ‘Adaptive determination of predictive biomarker profile groups’ that is the biomarker status was determined based on the sign of the first PLS component. Specifically, subjects were classified in biomarker positive or biomarker negative groups according the sign of first PLS component. The stopping criterion parameter δ was fixed at 0.99 and 0.999. The results are summarized in Table 3.

Table 3. Results of simulation study II – operating characteristics BCARA randomization design under various scenarios regarding the predictive or prognostic status of biomarkers.

Scenario	δ	Na	ProbA+b	Problc
Scenario 1: Not predictive or prognostic	0.99	84	0.19	0.57
	0.999	98	0.03	0.89
Scenario 2: Prognostic	0.99	77	0.27	0.42
	0.999	96	0.08	0.85
Scenario 3: Predictive	0.99	65	0.74	0.11
	0.999	85	0.39	0.43
Scenario 4: Prognostic and predictive	0.99	72	0.49	0.44
	0.999	89	0.18	0.79

^aMean total sample size across 500 replications profile randomized to treatment A.

^bProbability of declaring treatment A in subjects with positive biomarker status superior.

^cProbability of inconclusive result.

Under the four scenarios, the total sample size ranged from 65 to 98. Under scenario one, where the true response rate for all treatment-biomarker group combinations is 0.3, the mean total sample was 84 and the probability of an inconclusive result was 57% for the stopping criterion δ = 0.99 and 89% for the stopping criterion δ = 0.999. When the biomarker group was prognostic (scenario two), the probabilties of declaring treatment A in the biomarker positive group as superior over all other treatment-biomarker group combination were 0.27 and 0.08 for δ = 0.99 and 0.999, respectively. As expected, similar values were observed for the corresponding probabilities for treatment B (results not shown). When the biomarker groups was predictive (scenario three), the probability of correctly identifying treatment A in the biomarker positive group as superior over all other treatment-biomarker group was 74% for δ = 0.99. Finally, under scenario four, where the biomarker status was both predictive and prognostic and where the overall response rate of treatment B in the unselected population was higher than in treatment A, but the response rate in tretment A in the biomarker positive group was higher than in all other treatment-biomarker group combinations, the probability of correctly identifying treatment A in the biomarker positive group as superior over all other treatment-biomarker group was 49% for δ = 0.99.

Simulation study III

In a third simulation study, the operating characteristic of the proposed BCARA randomization design where biomarker subgroups are determined using the PLSLR approach was compared to the operating characteristic of a marker-by-treatment interaction design [6] with a Bayesian response adaptive randomization [24]. In the following, we will denote the first design as BCARA-PLSLR and the second design as MBTID-BRA. In the MBTID-BRA design, instead of a standard randomization, a response adaptive randomization with a Beta(1,1) prior distribution is used to randomize patients into the treatment arms. The response rate of a targeted therapy (treatment A) was compared to the response rate of an untargeted therapy (treatment B). The response rate of the untargeted therapy was 0.5 while the overall response rates for the targeted therapy were 0.6, 0.7, and 0.8. A set of K = 3 correlated (ρ = 0.3) biomarker measurements were generated from a multivariate multinomial distribution with a scale ranging from 0 to 3. This is a standard scale used for quantifying molecular markers based on immunohistochemistry (IHC) staining with 0 (none), 1 (low), 2 (medium), or 3 (high). The frequency for each of the four categories was assumed to be 25%. Various levels of predictive power of the three biomarkers on the probability of response for the targeted therapy were assumed, with polyserial correlations ranging from 0.1 to 0.8. Furthermore, it was assumed that there were no associations between biomarker measurements and response for the untargeted therapy. The maximum sample size for each trial was N_max = 200. Hypothetical trials were simulated M = 500 times. For the BCARA-PLSLR design, equal randomization was used for the first n* = 10 subjects during the run-in phase. Biomarker subgroups were determined based on the sign of the first PLS component. The stopping criterion parameter δ was fixed at 0.99.

For the MBTID-BRA, a Beta(1,1) distribution was assumed as a prior distribution for the success probabilities for both treatment A and treatment B. The biomarker groups are defined as positive if at least one of the three biomarker has a value of 2 or higher and as negative if all values are 0 or 1, that is, ‘none’ or ‘low’ category on the standard IHC scale. The randomization ratios for this design were computed in the same fashion as in the BCARA-PLSLR design, that is, subject i + 1 was randomized to treatment A with probability ${RR}_{i+1,A}^{(g)}={(p_A^{(g)})}^c/({(p_A^{(g)})}^c+{(1-(p_A^{(g)}))}^c),$ Where c = (i + 1)/2N_max and $p_A^{(g)}$ denotes the probability that the response rate in treatment A is greater than in treatment B given the observed responses in group g for subjects 1, …, i. The same stopping rule as for the BCARA-PLSLR design was employed to terminate the trial with δ = 0.99. The results of this simulation study are summarized in Table 4.

Table 4. Comparison of operating characteristics between proposed BCARA-PLSLR design and MBTID-BRA design.

RRAa	ρ1b	ρ2b	ρ3b	Design	Nc	ProbA+d	ProbA−e	ProbB+f	ProbB−g	Problh
0.6	0.1	0.1	0.1	BCARA-PLSLRi	92	0.49	0.26	0.02	0.09	0.14
				MBTID-BRAj	82	0.48	0.43	0.04	0.05	0.00
	0.5	0.1	0.1	BCARA-PLSLRi	85	0.63	0.06	0.04	0.15	0.12
				MBTID-BRAj	76	0.40	0.53	0.04	0.03	0.00
	0.8	0.1	0.1	BCARA-PLSLRi	46	0.77	0.04	0.00	0.18	0.01
				MBTID-BRAj	73	0.45	0.47	0.04	0.04	0.00
0.7	0.1	0.1	0.1	BCARA-PLSLRi	74	0.36	0.46	0.01	0.01	0.16
				MBTID-BRAj	81	0.45	0.51	0.02	0.02	0.00
	0.5	0.1	0.1	BCARA-PLSLRi	61	0.69	0.28	0.00	0.02	0.01
				MBTID-BRAj	76	0.50	0.42	0.02	0.06	0.00
	0.8	0.1	0.1	BCARA-PLSLRi	37	0.80	0.07	0.00	0.12	0.01
				MBTID-BRAj	71	0.45	0.47	0.04	0.04	0.00
0.8	0.1	0.1	0.1	BCARA-PLSLRi	67	0.53	0.45	0.01	0.01	0.00
				MBTID-BRAj	80	0.47	0.49	0.01	0.03	0.00
	0.5	0.1	0.1	BCARA-PLSLRi	43	0.81	0.11	0.00	0.04	0.04
				MBTID-BRAj	72	0.57	0.35	0.04	0.04	0.00
	0.8	0.1	0.1	BCARA-PLSLRi	30	0.90	0.07	0.00	0.03	0.00
				MBTID-BRAj	80	0.38	0.55	0.01	0.06	0.00

^aOverall response rate of targeted therapy (treatment A).

^bPolyserial correlations between response and biomarkers 1,2, and 3 for treatment A.

^cMean total sample size across 500 replications.

^dProbability of declaring treatment A in subjects with positive biomarker profile superior.

^eProbability of declaring treatment A in subjects with negative biomarker profile superior.

^fProbability of declaring treatment B in subjects with positive biomarker profile superior.

^gProbability of declaring treatment B in subjects with negative biomarker profile superior.

^hProbability of inconclusive result.

ⁱBCARA randomization design with PLSR to determine predictive biomarker subgroups.

^jMarker-by-treatment interaction design using a Bayesian response adaptive randomization.

As expected, the operating characteristics of the BCARA-PLSLR and the MBTID-BRA design were similar when the biomarkers are nonpredictive and when the difference in response rates between the two arms is small. For example, in the scenario where the overall response rate of treatment A is 0.6 and the polyserial correlations between the three biomarkers and response are 0.1, the mean total sample size for the BCARA-PLSLR design was 92 while that for the MBTID-BRA design was 82. Under both designs, the probability that treatment A was declared as superior in the biomarker positive group was approximately 50%. There were substantial differences in the operating characteristics between the two designs when the levels of predictive power varied between the three biomarkers. For example, when the overall response rate for the targeted therapy was 0.7 and the polyserial correlations for the three biomarkers were 0.5, 0.1, and 0.1, respectively, the mean total sample size for the MBTID-BRA design was 76 while that for the BCARA-PLSLR design was 61. The probability that the targeted therapy was declared as superior in the biomarker positive group over all other treatment-biomarker group combinations was 0.69 for the BCARA-PLSLR design and 0.50 for the MBTID-BRA design.

When the overall response rate for the targeted therapy was 0.8 and there were substantial differences in the level of predictive power between the three biomarkers, with polyserial correlations of 0.8, 0.1, and 0.1, the mean total sample size for the MBTID-BRA design was 80 with a 38% probability that the targeted therapy was declared as superior in the biomarker positive group was 38% while the mean total sample size for the BCARA-PLSLR design was only 30 with a 90% probability that the targeted therapy was declared as superior in the biomarker positive group. Since in the MBTID-BRA design the biomarker groups are determined before the study is conducted, this design may not accurately capture the true predictive value of the marker in relation to the outcome of the treatments. In summary, the simulation studies show that the proposed BCARA-PLSLR design effectively identifies subjects who benefit most from a targeted therapy and that there may be substantial savings in the sample size requirements when compared to the MBTID-BRA design.

Discussion

In this article, we proposed a BCARA design that utilizes information about individual subject's biomarker profiles and clinical outcomes as they become available during the conduct of the trial. A computationally efficient algorithm using block Gibbs sampling techniques is implemented to predict patient's responses to treatment and to compute allocation probabilities adaptively. Predictive biomarker subgroups are also determined adaptively using a PLSLR approach.

Simulation studies were conducted to examine the operating characteristics. The simulation studies showed that the proposed design effectively identifies subjects who benefit most from a targeted therapy and that there may be substantial savings in the sample size requirements when compared to alternative designs.

The proposed combination of an adaptive design strategy, Bayesian approach, and biomarker classification is not meant to replace the traditional paradigm for drug development. The design does not control for the type I error in the traditional sense and a positive result should be confirmed by conducting an independent phase III study focusing on the selected biomarker profile groups. We conclude, however, that it may serve a useful role in the early efficacy phase (phase II) of targeted therapy development.