Model-based phase I designs incorporating toxicity and efficacy for single and dual agent drug combinations: Methods and challenges

Abstract

Novel therapies are challenging the standards of drug development. Agents with specific biologic targets, unknown dose-efficacy curves, and limited toxicity mandate novel designs to identify biologically optimal doses. We review two model-based designs that utilize either a proportional odds model or a continuation ratio model to identify an optimal dose of a single or two-agent combination in a Phase I setting utilizing both toxicity and efficacy data. A continual reassessment method with straightforward dose selection criterion using accumulated data from all patients treated until that time point is employed while allowing for separate toxicity and efficacy curves for each drug in a two-drug setting. The simulation studies demonstrate considerable promise, at least theoretically, in the ability of such model-based designs to identify the optimal dose. Despite such favorable operating characteristics, there are several pragmatic challenges that hinder the routine implementation of such model-based designs in practice. We review and offer practical solutions to potentially overcome some of these challenges. The acceptance and integration of these designs in practice may be quicker and easier if they are developed in concert with a clinical paradigm. Copyright © 2010 John Wiley & Sons, Ltd.

Introduction

Phase I clinical trials are first-in-man studies that primarily focus on the safety profile of a regimen. Traditionally the primary aim of a phase I clinical trial has been to establish the maximum tolerated dose (MTD) and characterize the toxicity profile of the tested agent(s). As a secondary aim, some phase I studies also include correlative studies to obtain preliminary efficacy information about the experimental agent(s); however, these data are not typically utilized in the determination of the MTD. Consequently, many phase I clinical trial designs, such as the standard cohorts of 3+3 design [1], the two-stage design [2], and the continual reassessment method (CRM) [3] do not utilize any efficacy information in the dose-finding process. This is reasonable if the general assumption of a monotonically increasing relationship between dose and efficacy holds true, as is the case with most cytotoxic agents in cancer clinical trials where the likelihood of tumor shrinkage (and thereby the clinical benefit) is hypothesized to increase with dose. The MTD that is determined from a phase I trial is then recommended for further testing in a phase II setting.

While the assumption of a monotonically increasing dose-toxicity curve is almost always appropriate from a biological standpoint, a monotonically increasing relationship between dose and efficacy has been challenged by the recent development of molecularly targeted therapies, vaccines and immunotherapy. The dose-efficacy curves for these novel therapies may follow a non-monotone pattern such as a quadratic curve or an increasing curve with a plateau. Consider for example a therapeutic approach based on immunotherapy where an agent is given to a patient in an attempt to stimulate the patient's own immune system to fight the tumor. In such cases, overstimulation of the immune system could in fact interfere with efficacy or even prove to be harmful (toxic) for the patient [4]. Under these circumstances, the focus of a phase I trial is to identify the biologically optimal dose (BOD), instead of the MTD.

The BOD is quantified by a measure of efficacy, for example, minimum effective blood concentration level of the agent, percent target inhibition of a marker, or minimum expression level of a molecular target (targeted biologic response). Of course, this requires (1) the presence of sufficient preclinical evidence demonstrating a relationship between the biomarker-based endpoint with the ultimate clinical outcome, (2) the biomarker endpoint can be assessed easily and quickly, and (3) the assays are validated. Novel designs that incorporate a measure of efficacy (in addition to toxicity) into phase I clinical trials are becoming increasingly popular. Bivariate analysis of both toxicity and efficacy to guide dose-finding is the main statistical theme of these designs. Most of these designs consider both toxicity and efficacy as binary endpoints, which leads to four possible outcomes: neither dose-limiting toxicity (DLT) nor efficacy, no DLT with efficacy, DLT without efficacy, and DLT with efficacy.

In the CRM [3] design, dose-toxicity models were first proposed to guide the dose-finding process. The dose-toxicity model represents the investigator's a priori belief in the likelihood of DLT according to delivered dose, which is thereafter updated sequentially using cumulative patient toxicity data. The model allows for the predicted probability of DLT at each dose to be estimated and subsequently facilitates the recommendation of a dose level for further testing. The classical and modified CRMs [5, 6] are model-based adaptive designs that have been demonstrated to have better operating characteristics than many algorithm-based designs: a higher proportion of patients are treated at levels close to the optimal dose level and fewer patients may be required to complete the trial. More importantly, the CRM designs have proven to be robust to model mis-specification [7] as long as the models are selected based upon clinical knowledge. The CRM can be estimated through maximum likelihood approach using classical frequentist theory, but it fits more naturally into a Bayesian framework.

Current statistical approaches extend the standard CRM approach in two directions to allow the modeling of toxicity and efficacy outcomes in a phase I setting. The first approach maintains the bivariate structure of outcomes through a joint modeling of toxicity and efficacy. The bivariate CRM (bCRM) [8] is an example of this approach applied to a bone marrow transplant study. It extends the CRM to a marginal logit dose-toxicity curve and a marginal logit dose-disease progression curve with a flexible bivariate distribution of toxicity and progression. Thall and Cook [9] considered a dose-finding algorithm based on the efficacy–toxicity trade-offs, while Yin et al. [10] proposed a dose-finding scheme using toxicity and efficacy odds ratios, both with a flavor of Bayesian analysis. Bekele and Shen [11] and Dragalin and Fedorov [12] both utilized bivariate probit models for toxicity and efficacy, with the former measuring efficacy based on the expression of a continuous biomarker.

The second approach deals with observed clinical outcomes that follow a sequential order: no DLT and no efficacy, no DLT but with efficacy, or severe DLT, which renders any efficacy irrelevant. In this case, the joint distribution of the binary toxicity and efficacy outcomes can be collapsed into an ordinal trinary variable. Examples of where such an approach would be plausible include the graft-versus-host disease (GVHD) in bone marrow transplant [9] and viral reduction in HIV studies [13]. Under suitable models, this approach has been shown to accommodate a non-monotone increasing (i.e. unimodal) dose-efficacy curve. While both O'Quigley et al. [13] and Ivanova [14] have explored clinical trial designs for ordinal toxicity and efficacy outcomes, they did not rely heavily on parametric approach for modeling the categorical data. The purpose of this paper is to review model-based trivariate CRM (TriCRM) designs, specifically the proportional odds (PO) model-based design [15] and the continuation ratio (CR) model-based designs [16, 17]. We review the theoretical model framework, the computational issues, the design specifics, and the simulation results of these TriCRM model-based designs. We conclude with a discussion of some of the practical challenges in implementing these designs.

Model-based designs

Statistical models

Let y=(y₀,y₁,y₂) denote a trinary ordinal variable representing the three possible outcomes of acceptable toxicity without efficacy, acceptable toxicity with efficacy, and severe toxicity, respectively, with corresponding probabilities denoted by: ψ(x,θ)= {ψ₀(x,θ),ψ₁(x,θ),ψ₂(x,θ)}. The probability of each outcome is a function of the dose of the agent (x) together with the parameter vector (θ). Among these, ψ₀(x,θ) is required to be a decreasing function of the dose (x) and ψ₂(x,θ) is an increasing function of the dose. The probability of acceptable toxicity with efficacy, that is treatment success, ψ₁(x,θ) can be a non-monotone function of the dose (x). Thall and Russell [15] proposed the PO regression model, given by

$$log\text{it}({\psi }_1+{\psi }_2)=\mu +\alpha +\beta \cdot x$$ (1)

$$log\text{it}({\psi }_2)=\alpha +\beta \cdot x$$ (2)

where β>0 to ensure the monotonic relationships of ψ₀(x,θ) and ψ₂(x,θ) with dose (x) described above. Alternatively, Zhang et al. [17] proposed the CR regression model given as follows:

$$log\left(\frac{{\psi }_1}{{\psi }_0}\right)=\mu +\alpha +{\beta }_1\cdot x$$ (3)

$$log\left(\frac{{\psi }_2}{{\psi }_0+{\psi }_1}\right)=\alpha +{\beta }_2\cdot x$$ (4)

where β₁>0 and β₂>0 to ensure the monotonic relationships of ψ₁(x,θ)/ψ₀(x,θ) and ψ₂(x,θ) with dose (x). Note that the left side of equation (4) is equivalent to logit(ψ₂) in equation (2) since ψ₀+ψ₁+ψ₂=1 by definition. Mandrekar et al. [16] extended the single agent CR model to accommodate two agents by including two additional slope parameters for the second agent, given as follows:

$$log\left(\frac{{\psi }_1}{{\psi }_0}\right)=\mu +\alpha +{\beta }_1\cdot x_1+{\beta }_3\cdot x_2$$ (5)

$$log\left(\frac{{\psi }_2}{{\psi }_0+{\psi }_1}\right)=\alpha +{\beta }_2\cdot x_1+{\beta }_4\cdot x_2$$ (6)

where β₁>0, β₂>0, β₃>0, and β₄>0 to ensure the monotonic relationships for each agent marginally. This is a sufficient but not necessary condition for marginal monotonicity. Both the PO and the CR models are important classes in categorical data analysis [18] for analyzing the polytomous ordinal data. However, due to the small sample size restriction in phase I clinical trials, both the PO and the CR models adopt a parsimonious simple regression model with only an intercept and a slope parameter for each agent.

Given the PO or the CR model, the likelihood for the parameter vector of θ=(μ,α,β), θ=(μ,α,β₁,β₂), or θ=(μ,α,β₁,β₂,β₃,β₄) is constructed by combining the trinomial outcomes for each cohort i=1,…,n of patients, as follows:

$$L(\theta \mid x,y)=\prod _{i=1}^n{\psi }_0{(x_i,\theta )}^{y_{0i}}\cdot {\psi }_1{(x_i,\theta )}^{y_{1i}}\cdot {\psi }_2{(x_i,\theta )}^{y_{2i}}$$ (7)

To facilitate numerical computation within the Bayesian framework, all three models chose independent uniform priors for the parameter θ with various ranges reflecting the uncertainty. Thall and Russell [15] described a didactic approach to solicit prior information. In terms of computation, different numerical analysis methods have been utilized to derive either the posterior distribution or moments of the parameters of interest.

Clinical trial design

Thall and Russell [15] formally defined the decision criteria in terms of posterior probabilities, specifically,

$$Pr\{{\psi }_1(x,\theta )<{\psi }_1^{\ast }\mid \mathit{\text{data}}\}>{\pi }_1^{\ast }$$ (8)

$$Pr\{{\psi }_2(x,\theta )>{\psi }_2^{\ast }\mid \mathit{\text{data}}\}>{\pi }_2^{\ast }$$ (9)

A dose level is deemed unacceptable if either there is very low-estimated efficacy (8) or very high-estimated toxicity rate (9). Zhang et al. [17] and Mandrekar et al. [16] adopted decision functions that utilized the estimated posterior mean of parameters θ given by,

$${\delta }_1(x,\theta )=/\{{\psi }_2(x,\theta )<{\pi }^{\ast }\}$$ (10)

$${\delta }_2(x,\theta )={\psi }_1(x,\theta )-\lambda \cdot {\psi }_2(x,\theta )$$ (11)

Based on these decision functions, a dose level must first satisfy the toxicity requirement (10), and then maximizes the toxicity-adjusted treatment success rate (11) as much as possible. Note that ${\psi }_1^{\ast },{\psi }_2^{\ast },{\pi }_1^{\ast },{\pi }_2^{\ast },$ and π* are all pre-specified a priori by the study team and 0≤λ≤1 is the weight assigned to the toxicity probability. Both the decision criteria and decision functions are evaluated with the accumulation toxicity and efficacy data for decision-making of dose escalation or de-escalation. If the current dose level has unacceptable toxicity, dose de-escalation may be introduced unless the current dose is already at the lowest level. When current dose level has acceptable toxicity but less than desired treatment success (efficacy), dose escalation is expected to occur. The conduct of a trial using these model-based TriCRM designs may vary with regard to details; however, they all follow a similar modified CRM approach. Specifically, we illustrate the scheme of the CR model-based design below:

1. Treat a cohort of three patients at a time, starting from the lowest dose level or a subset of the lowest levels for the dual agent case in Mandrekar et al. [16].

2. Doses can be escalated only by one level (or within dose neighborhood) at a time, but dose de-escalation is not necessarily restricted. For a trial with dual agents, a dose neighborhood refers to a change of at most one dose level of at most one agent [16].

3. At each interim point, the decision functions are evaluated using the accumulated toxicity and efficacy data for recommendation of dose escalation or de-escalation.

4. If the toxicity of current dose level is acceptable, then the dose of one agent is escalated until the optimal dose level is attained based on the pre-specified decision rules.

5. If the current dose level is too toxic, and the current dose level is not the lowest dose level, then the dose is de-escalated to the highest lower dose level that is estimated to have acceptable toxicity.

6. If the current dose level does not meet the toxicity criteria, and the current dose level is the lowest dose level, then the trial is terminated.

7. The trial is terminated when either (a) the maximum allowable number of patients are treated or (b) after the minimum number of patients have been treated provided that at least a pre-specified number of patients are treated at the proposed combination dose level, whichever occurs first.

A hypothetical example of a TriCRM design for dual agents is as follows. Based on pre-clinical work in multiple tumor types combining two signal transduction inhibitors, the insulin-like growth factor-1 receptor (IGF-1R) inhibitor, and the HER inhibitor [19], a potential dose-finding study with two such agents could be hypothesized in multiple tumor types. We describe here a dose-finding clinical trial design that attempts to identify an optimal dose using the TriCRM design for dual agent drug combinations [16].

Unacceptable adverse events would be defined for this combination, for example, as any grade 3 or higher treatment-related adverse event, and change in Akt phosphorylation in circulating tumor cells (CTCs) represents a potential biomarker for inhibition of the signaling pathways (‘efficacy’) in this design. The proposed design would then use the CR model with a trinary outcome: no efficacy and no unacceptable adverse events, efficacy with no unacceptable adverse events (success), and unacceptable adverse events. The dose-finding algorithm would use two decision functions: an adverse events criterion that requires the estimated unacceptable adverse events probability to be less than a pre-specified rate (say 30 per cent) for the dose range to be explored, and an efficacy criterion that maximizes the estimated success probability within the dose range that meet the adverse events criterion.

The trial would then proceed as follows: (1) assign three patients to the lowest starting dose-level combination, (2) based on the accumulated data at any point in the trial, update the parameter estimates of the CR model and evaluate the adverse events' criterion for the dose range to be explored, (3) within the dose ranges that satisfy the adverse events criterion, identify the combination dose level that has the maximum estimated success probability, (4) assign the next cohort of patients to the dose level combination that has the maximum estimated success probability, and (5) repeat the above steps until at least 9 patients are treated at the optimal dose level combination or until a maximum of 45 patients are treated, whichever occurs first. In the event that the entire dose range does not satisfy the adverse event criterion, and the current dose level is not the starting dose level, this design has the flexibility to assign the next three patients to the starting dose level instead of terminating the trial based on the premise that additional data might be needed for a more accurate estimation of the true toxicity and efficacy curves.

The trial could be designed to allow (or not allow) skipping of dose levels in the dose escalation process. An advantage of this design is that it summarizes patient outcomes in terms of both adverse events and efficacy (in this case, a biological endpoint). Moreover, the trial would be terminated early if all dose ranges to be explored are determined to have an unacceptable adverse event rate.

Simulation

Simulation studies are carried out in each of the three papers [15–17] to evaluate the performance of the model-based TriCRM designs. Figure 1 depicts the three possible dose-efficacy scenarios in a phase I setting, specifically, a monotonically increasing dose-efficacy curve, an increasing dose-efficacy curve with a plateau and a unimodal, or parabolic dose-efficacy curve.

Figure 1.

Possible dose-efficacy curves in the phase I clinical trials of molecularly targeted chemotherapy and immunotherapy.

In the Thall and Russell [15] paper, the results are embedded in the GVHD trial example. As the true dose-response (toxicity and efficacy) is unknown, several possible scenarios are considered and evaluated in these simulations. The scenarios considered in the Thall and Russell [15] paper include monotone increasing, unimodal, and monotone decreasing dose-efficacy curves with various monotone increasing dose-toxicity curves. Zhang et al. [17] considered scenarios with data generated from the PO model, the CR model, and model-free structure with the previous dose-efficacy and dose-toxicity curve combinations. The scenarios in the Mandrekar et al. [16] paper are complicated by the fact that they involve dual agents and dual outcomes; thus, each agent has a different dose-efficacy or a dose-toxicity curve of its own.

The simulation studies for all three designs have demonstrated satisfactory performance in terms of recommendation rates and experimentation rates with reasonable sample size for the most cases. In the operating characteristics table of the PO model-based GVHD trial design of Thall and Russell [15], the true optimal doses are recommended with the highest frequency (from 43 per cent to 77 per cent) with the average sample size ranging from 32 to 38 for the first five scenarios. For the scenarios where an optimal dose did not exist, the correct decisions are made (from 47 per cent to 78 per cent) with the average sample size from 19 to 30. In this design, a large proportion of patients are treated at the optimal dose or its neighborhood. Similarly, the true optimal doses are also selected a higher proportion of the times in the Zhang et al. [17] paper with some variation across the different scenarios considered. The percentage of patients treated at the optimal dose level varies from around as low as 25 per cent to as high as 75 per cent. The average sample size ranges from 16 to 25 patients depending on the location of BOD, and the no recommendation rate is high when the starting dose is too toxic. Zhang et al. [17] also showed that the TriCRM design is more efficient in identifying the optimal dose compared with a design where patients are randomly assigned to dose levels, which have been proposed as an approach for dose-finding for non-toxic agents. The simulation studies in the Mandrekar et al. [16] paper demonstrated that the optimal dose level combination can be successfully determined, but with a relatively lower frequency than when only a single agent is being studied. The authors also define a dose success region to include all those dose levels with acceptable toxicity that have a success probability within 15 per cent of the highest success probability. The percentages of patients treated at the optimal dose-level combinations (region) varied from 57 per cent to 71 per cent, with a high non-recommendation rate for toxic combinations. The average sample size ranges from 21 to 34 for the six scenarios considered in their simulations. Given the complex nature of dose-finding for dual agent trial with dual outcomes, the simulation results for the dual agent dual outcome design are promising.

Practical challenges with the implementation of model-based designs

Despite the favorable operating characteristics (primarily demonstrated through simulation studies), in our experience, there have been reluctance among the clinical community to adopt novel model-based phase I designs similar to those presented here [15–17]. Several scientific and pragmatic reasons could be considered to explain why the model-based designs are not yet ‘popular’ choices for dose-finding studies [20]. Some of the scientific reasons for not relying on model-based designs for early phase studies include (a) lack of validated biomarkers for efficacy, (b) lack of validated assays, (c) real-time assessment of the biomarker outcome not possible, (d) dichotomous efficacy outcomes inaccurate and suboptimal (model-based designs for time to event or other continuous endpoints are limited/non-existent), and (e) inability of some of the designs to accommodate categorical (as opposed to a continuum) dose-level combinations.

Zohar and Chevret [19] compared three model-based dual outcome dose finding methods and found no meaningful differences in the final estimated optimal dose, but that all these methods provided an advantage compared with a traditional 3+3 approach. Rogatko et al. [20] reported on a science citation database index search of phase I trials in cancer between 1991 and 2006. Only 20 of the 1235 trials (1.6 per cent) followed a model-based dose finding design, specifically, 17 trials used a CRM, and 3 trials used the escalation with overdose control (EWOC) design [5]. The authors include a detailed discussion of the advantages and disadvantages of the traditional up and down designs versus the model-based designs, and conclude that the adoption of the statistical model-based designs have favorable properties particularly with regard to treating a higher number of patients at the optimal or closer to the optimal dose levels compared with the traditional designs. These two reports [19, 20] highlight the pressing need to overcome the perceived ‘difficulties’ with the model-based designs.

In our experience, however, more often than not, pragmatic challenges with implementing a model-based design outweigh the scientific reasons. These pragmatic issues include (a) lack of familiarity with the design, (b) fear of the ‘black-box’ decision making framework in comparison to the straightforward decision process with the traditional non-model-based designs, (c) perceived loss of control of the data and relying on the statistical model to decide where to treat the next cohort of patients, (d) fear of lack of regulatory acceptance, and (e) finally, and most importantly, resistance to change and unwilling to be the ‘first’ to try a new approach.

Discussion

Multiple dose-finding designs for a molecularly targeted endpoint or bivariate outcomes (toxicity and efficacy) for one and two agents have been proposed in the literature [9, 14, 21–25]. The model-based TriCRM designs reviewed in this manuscript have demonstrated satisfactory performance in terms of recommendation rates and experimentation rates with reasonable sample size for the most cases [15–17]. O'Quigley et al. [13] and Ivanova [14] have explored flexible clinical trial models that may be more readily acceptable to clinician investigators. Specifically, their designs utilize ordinal toxicity and efficacy outcomes that do not rely heavily on parametric approach for modeling the categorical data. Fan and Wang [26] proposed a decision theoretic framework for dose-finding trials with multiple outcomes that allows for both an optimal dose allocation and optimal termination in a unified way. The authors explored two computationally compromised strategies as an approximation to the optimal strategy: (1) the one-step look ahead (OSLA) strategy, which assigned the best-so-far dose to the next patient, the same idea as the CRM, and (2) the two-step look ahead strategy, which was an improvement over the CRM. The simulation results demonstrated that these strategies were effective and efficient.

While we recognize that model-based or flexible designs are not perfect or to be recommended for every dose-finding study, we do feel that they provide an attractive alternative (not withstanding the above challenges) compared with the traditional algorithm-based up and down methods when one or both of the following is true: (1) number of dose levels for escalation/deescalation is large, that is 6 or higher for example and (2) agent(s) being tested are expected to have unknown or quick dose-efficacy outcomes. In the first case, the traditional designs would typically require a larger number of patients to be treated if indeed the optimal dose level is near the highest dose level. In the second case, the dose escalation/de-escalation decisions are based not only on safety but also on a measure of efficacy that is quick and reliable to assess.

The challenge of determining an optimal dose for biologic and molecularly targeted agents is considerable from both a clinical and statistical standpoint in early phase trials. It is essential to engage and educate clinical investigators as these new designs are developed. If the designs are developed in concert with a clinical paradigm, then the acceptance of these designs may be quicker and easier. As is true with any new statistical approach that is developed, a user friendly software tool with visual aids is a definite plus, and probably mandatory for these designs to be implemented in the real world. The model-based designs have been shown to have considerable promise, at least from a theoretical standpoint, improving the ability to identify optimal dose levels. It is imperative to continue efforts to translate this research knowledge into clinical practice; well designed early phase trials are the stepping stones to effective phase II and phase III trials.