Adaptive Phase I design in radiation therapy trials

Case vignette

The primary objective of an ongoing Phase I/II trial is to determine the maximum tolerated dose (MTD) for niraparib, a PARP inhibitor, in high risk and node positive prostate cancer when administered in combination with abiraterone acetate, leuprolide, and stereotactic body radiotherapy (SBRT) (NCT04194554). The trial is assessing three dose levels of niraparib (Table 1), with the MTD defined as the dose level with a dose-limiting toxicity (DLT) probability closest to the target DLT probability of 25%. The primary endpoint that will guide adaptive dose assignment is the binary indicator of DLT, defined as any persistent (does not resolve after stopping the drug) grade 4+ hematologic toxicity or any grade 3+ rectal/urinary toxicity at least possibly related to study treatment. The window of observation for DLT is from initiation of study treatment to 30 days after completion of SBRT (four 28-day cycles). The planned sample size is n=100 participants, which allows for accurate estimation of the MTD and sufficient power to estimate treatment efficacy (proportion of patients free of biochemical failure at 3 years) at the MTD. Accrual of participants is expected to occur at a rate of five patients per month.

Table 1.

Dose levels for case vignette trial.

Dose-Escalation Schedule for Niraparib (Leuprolide and abiraterone/prednisone given continuously for 6 cycles)
Dose Level	Dose of the Study Agent(s)*
Level 1	100 mg PO daily but held for 5 days (+/− 2 days) prior to RT, during SBRT, and 5 days (+/− 2 days) after last fraction of SBRT
Level 2	200 mg PO daily but held for 5 days (+/− 2 days) prior to RT, during SBRT, and 5 days (+/− 2 days) after last fraction of SBRT
Level 3	200 mg PO daily without breaks during SBRT until completion of 6 cycles

The case vignette highlights a common logistical challenge seen when designing and conducting early-phase trials of radiation therapy that examine long term systemic treatment and have a potential for late or delayed toxicity: how do we sequentially assign dose levels to participants when DLT outcomes potentially take a long time to observe relative to the expected accrual rate and simultaneously provide preliminary estimates of treatment efficacy?

The problem of late-onset toxicity

Dosing decisions in systemic therapy Phase I trials have traditionally been guided by DLTs in cycle 1 of treatment. In these trials, MTDs are defined as the highest tolerated dose from cycle 1, even though patients are administered therapy for several cycles. In radiation therapy trials, relevant toxicity events may occur in later cycles of treatment, or even after treatment has ended. Therefore, the assessment of appropriate doses based solely on DLT definitions from cycle 1 DLT outcomes is insufficient for these trials. Early toxicity does not provide a complete representation of tolerability and there is a growing need to incorporate richer toxicity information beyond DLTs observed in cycle 1. At present, late-onset DLTs are not often used in guiding initial dose allocation within the design, nor in the dose recommendation after the study. Variants of the 3+3 algorithm (Figure 1), where patients are treated in cohorts of 3 or 6, depending on the number DLTs in the first 3, prior to escalation, continue to be used in radiation therapy studies, with the significant risk of ignoring pertinent late toxicities. In the presence of late events, DLTs may be undercounted, and the resulting recommended dose is weighted in favor of higher doses that appear safer than they are^1,2. Consequently, recommended doses result in a higher than anticipated probability of toxicity. The FDA registers many novel therapies at doses different from those identified in Phase I trials². Many patients are treated at overly toxic or sub-therapeutic levels throughout the development process.

Figure 1:

Flow diagram for the conduct of the 3+3 algorithm.

One possible solution to the late-onset toxicity problem is to extend the DLT evaluation window. The logistical challenge of late-onset (or delayed) toxicity occurs because each participant must be followed throughout the DLT evaluation window without DLT to be classified as a non-DLT outcome. Moreover, in the case vignette example, DLTs can occur at any time during the approximate five-month DLT evaluation window (i.e., 4 cycles plus 30 days). If DLT outcomes can take up to five months to evaluate, then some patients will have pending DLT outcomes when new participants are accrued to the study. The case vignette presents a setting in which patients are accruing rapidly relative to the observation period for DLTs. In this situation, how should doses be allocated to newly accrued participants when participants on study have only been partially followed for DLT? If the DLT evaluation window is extended, execution of the 3+3 design may involve pausing enrollment if new patients are accrued to the study at a time when three (or six) patients have been accrued to the current dose level, but the minimum DLT evaluation window has not elapsed for at least one of these patients. This issue is characteristic of any design that requires all patients to have completed observation for DLT. Frequent pauses to trial accrual present significant logistical, statistical and ethical challenges.

Figure 2 illustrates the problem for the first three eligible participants accrued in the case vignette study. When the second participant is enrolled, the first participant has been followed without DLT for 40 days (28% of the entire DLT observation window) without DLT. When the third participant is enrolled, the first participant has been followed for 88 days without DLT (62% of the entire DLT observation window) and the second participant has been followed for 48 days without DLT (34% of the entire DLT observation window). Suppose further that an additional patient has been on treatment in the case vignette trial for four cycles without any sign of DLT. Although we have not observed the patient’s DLT outcome in the 30-day follow-up period, a four-cycle DLT-free follow-up speaks for the safety of the dose that the patient received and may be counted as 79% (112 days/142 days) of a non-DLT outcome.

Figure 2:

Illustration of the problem of late-onset toxicity. The DLT evaluation window is 4 28-day cycles of treatment plus 30-days of follow-up (142 days). Two additional patients arrive before the first patient has completed the minimum DLT evaluation window.

Among the most commonly used designs for late-onset toxicities is the rule-based Rolling 6 (R6) algorithm³, even though its operating characteristics are inferior to those of model-based strategies⁴. Innovative dose-escalation strategies are needed to efficiently and accurately identify appropriate doses that merit further research. Novel adaptive designs are based on the premise that as safety and other information accumulate, the data should guide dose assignments for future study participants. Several existing methods utilize data from patients that have been partially observed for DLT in the estimation of outcome probabilities, weighting each entered patient by a portion of the full DLT observation window for which they have been followed. The time-to-event continual reassessment method (TITE-CRM⁵) is a commonly used example of these adaptive designs and will be used to explicate its methodology, strengths and weaknesses.

Time-to-event continual reassessment method (TITE-CRM)

We now discuss the design specifications of the TITE-CRM, including how it is implemented and its operating characteristics. It has previously been shown that, relative to the standard 3+3 and rolling six³ algorithms, the TITE-CRM design identifies the MTD more accurately and treats more patients near the MTD, but does not expose patients to significant additional risk^5,22. The TITE-CRM dose assignment mechanism will operate for all patients on this trial presented earlier. This method is designed specifically for the situation where the DLT observation period is significantly greater than the mean inter-participant arrival time--in this vignette 5 months of observation versus monthly accrual--while remaining continuously open to accrual.

Design specifications in case vignette

A total of 100 evaluable patients will be enrolled with an anticipated accrual rate of five patients per month at four sites; the University of Michigan, Cornell, University of Texas Southwestern Medical Center, and University Hospitals Cleveland. When the trial began, it was open only at the University of Michigan so that the study didn’t accrue too quickly at the lowest dose level. The first dose could then be properly evaluated for safety before opening the trial to additional sites and increasing the accrual pace. Three dose levels of niraparib are described in Table 1.

Patients initiated treatment on cycle 1 day 1 with niraparib. Prostate SBRT began with cycle 4 day 5 (+/− 30 days). The MTD is defined as the dose level with probability of DLT closest to 25%, but not exceeding 30%. DLTs are defined as persistent (do not resolve after stopping the drug) grade 4+ hematologic toxicity or any grade 3+ rectal/urinary toxicity at least possibly related to study treatment and occurring at any point after initiation of study treatment through 30 days after completion of SBRT (~4 cycles). The first patient was assigned to dose level 1. Subsequently, whenever a patient presented for enrollment, the probability of DLT was re-estimated for each dose level, based on the initial expectations of toxicity (defined below) and the incidence of toxicity in patients already enrolled. Each new patient is then assigned to the dose level with an estimated DLT probability closest to the pre-specified target DLT rate because that dose is currently our best estimate of the MTD based on the updated statistical model.

The original CRM²³ for this study proposed to always treat at the target dose of niraparib. As a result, the first patient would be treated at the target dose, which may be very high. However, regulatory and ethical considerations led to starting at a low dose as well as constraining how quickly dose escalation can proceed. Three feasible constraints include (1) allowing increases of only one level between consecutive patients, (2) not visiting a level until a pre-specified number of patients have completed the previous level, or (3) not visiting a level until a pre-specified total observation time has been completed at the previous level. Every added constraint reduces the efficiency and increases the complexity of the trial. More complex constraints than the ones listed may make trial execution more challenging or perhaps infeasible. In the case vignette, the dose level that has estimated toxicity closest to the target probability of 25% was selected, subject to several safety constraints: two patients must have completed DLT observation period at dose level 1, prior to escalating to level 2 and, likewise, two patients must have completed dose level 2 prior to escalation to level 3.

Determining the dose level for each patient

During the trial design process, the first step in determining the dose escalation rule is specifying the acceptable probability of experiencing a DLT (here, 25%), which is called the target probability. The first patient is treated at a low dose and is observed for DLT. Then, as each patient presents for dose assignment, a simple dose-toxicity model is used to update the DLT probability estimate at each dose. The dose assigned to the next patient is the level with an estimated probability of DLT closest to the target probability, subject to escalation constraints that ensure the trial does not advance too fast. Patients under observation for DLT are weighted according to the proportion of the 5-month observation period they have completed.

The specification of an appropriate dose-toxicity model is central to the success of this design. The model used for the TITE-CRM in the case vignette is a one-parameter model in which the estimated probability of DLT at dose level i is given by Pr(DLT at dose level i) ≈ $p_i^{\text{exp}(a)}$, where p_i are the estimates of the DLT probability for each dose level specified by the investigators at the design stage to reflect the state of their current knowledge. Other studies may use a different model for the DLT prediction. In fact, pre-trial design specifications for CRM designs have been thoroughly studied since 1990, leading to published recommendations for executing the trial. Guidance for selecting appropriate dose-toxicity models and prior distributions for the model parameters can be found in Paoletti et al²⁴ and Lee and Cheung^25,26. The operating characteristics of the trial design, generated via simulation, evaluate the impact of the prior specifications and escalation constraints on the method’s statistical properties. Recommended specifications from these papers are intended to yield robust operating characteristics for CRM designs across a broad range of possible dose-toxicity situations.

In the case vignette trial, these probabilities at the 3 dose levels were initially estimated as 5%, 15% and 25%, respectively. The target probability was chosen to be 25%; that is, we were willing to tolerate a dose that would induce DLT in 25% of the patients. The prior distribution on the parameter a is specified by a normal distribution with a mean of 0 and a standard deviation of 0.3. The first participant is treated at the starting dose level, and observed for DLT. Each time a dose assignment is needed for a new participant, an estimate ã for the model parameter a is updated using the accumulated data at all dose levels. The estimated DLT probability estimates are then sequentially updated using the dose-toxicity model (e.g. $p_i^{\text{exp}(\tilde{a})}$). For instance, suppose that at some point in the case vignette trial, a is estimated to be 0.2 (i.e. ã = 0.2) based on all accumulated data. Our updated DLT probability estimates at each of the three dose levels would be 0.05^exp(0.2) = 0.03, 0.15^exp(0.2) = 0.10, and 0.25^exp(0.2) = 0.18. The next participant accrued to the study is assigned the dose level with an updated estimated DLT probability closest to the target DLT probability, with the constraint that untried dose levels cannot be skipped when escalating. The updating of the DLT probability estimates and the allocation of patients to the dose with updated DLT probability closest to the target continues until a pre-specified number of participants have been observed or a pre-specified stopping rule is triggered. At the end of the study, the MTD is taken to be the dose level that the next participants would have received had the trial not ended.

Estimating the operating characteristics during trial design

The “operating characteristics” of the trial are its statistical properties under assumed truths of nature²⁷. Multiple scenarios are typically considered to cover various possibilities, ranging from pessimistic (all doses are too toxic) to optimistic (all doses are safe)²⁸. In the design phase, trial data are simulated under the various assumptions, and the performance of the design is evaluated. A large number (typically, 1000) trials are simulated, including the core TITE-CRM algorithm and the dose escalation constraints and, after each simulated trial, the dose selected as the MTD, the number of DLTs at each dose and the number of participants assigned to each dose are recorded. At the end of the simulations, under each assumed state of nature, the percentage of times each study dose level is chosen as the MTD, the percentage of times the trial stopped early for safety, the number (or percentage) of participants treated at each dose level, the overall percentage of observed DLTs, and the mean (or median) trial duration are all determined. A good design should have a high probability of terminating at or near the true MTD, a low cumulative probability of stopping below the true MTD, and a low probability of escalating beyond the MTD. In the case vignette, if the prior estimates of DLT rates are correct, or if the true rates are lower than expected, the probability of correctly selecting dose level 3 as the MTD is at least 90% while the probabilities of selecting dose levels 2 and 1 are approximately 9% and <1%, respectively. If our prior estimates of toxicity are correct, and level 3 is the true MTD, an expected 65 patients will be treated at the MTD.

Part of the evaluation process at the design stage should be to assess the impact of patient accrual rates and time-to-toxicity distributions on design performance. For instance, if patient accrual is rapid and many DLTs occur late in the evaluation window, then each estimation and allocation decision will be made on limited data available at the time of accrual, especially for the first few patients. While this is also a problem in conventional dose finding designs, we want to evaluate its impact on the dose decision-making process of adaptive methods.

Illustration

In the simulated trial example in Figure 3, the first patient is enrolled at dose level 1 and the dose-assignment algorithm results in approximately 20 patients each being treated at dose levels 1 and 2 sequentially while the last 60 patients are all treated at dose level 3. In practice, enrollment was slower initially for this trial so that relatively fewer patients were treated at dose level 1. In this example trial, 2 of the first 6 patients treated at dose level 2 experienced DLT which would have prevented escalation to level 3 in a 3+3 design. Although it does not occur in this simulated trial, the dose level could de-escalate and then re-escalate to the highest dose level. Specifically, early toxicity in the trial will serve to create higher estimated toxicity rates and drive the assigned doses down. However, if the toxicity is not seen in later patients, then estimated toxicity rates will decrease and the assigned doses will start to increase. If the early toxicity was not systematic, but simply due to the vagaries of random processes (e.g., the ordering of the toxicities), this may permit testing for toxicity at a higher dose than originally expected. The dose-assignment rule acts as a built-in safety rule, so that no external rule is needed.

Figure 3:

A simulated trial illustration of the TITE-CRM.

Advantages of the adaptive design

Adaptive designs offer several benefits that make them far more appropriate for Phase I clinical trials in radiation oncology than algorithmic approaches. First, all adaptive methods have a specific goal in mind: to identify a dose whose probability of DLT is closest to a desired and known value. Although it is believed that the 3+3 method targets a DLT probability of 0.33, the observed DLT probability is more likely to be much lower and closer to 0.20²⁹. Thus, use of the 3+3 design is leading to lower doses being selected for study in Phase II trials, which has the unfortunate consequence of Phase II trials that are studying doses that may be less effective for treating and preventing recurrence of cancer. By having a design that is adapted to a specific targeted level of DLTs, designs like the TITE-CRM are better able to meet the actual goals for a Phase I trial.

Second, adaptive designs allow for both escalation and de-escalation of doses between study participants, which leads to much greater precision in the identification of the true MTD, more study participants being assigned to the true MTD, and less likelihood of early termination of the study when one or two DLTs are observed. Moreover, adaptive designs employ models that “connect” the toxicity probabilities of all dose levels to each other. Thus, at the end of the trial, the totality of the data can be used to determine an estimate and corresponding 95% confidence interval for the true DLT probability of each dose level, whereas traditional designs can only use the data collected for each dose to provide estimation and inference for that dose.

For example, imagine a study of five doses that was designed with the 3+3 algorithm. At the end of the study, suppose we have three participants each treated at doses 1, 2, and 3, six participants treated at dose 4 and two participants treated at dose 5, so that dose 4 is selected as the MTD. The estimated DLT probability for dose 4 is 1/6 = 0.17, with a corresponding 95% confidence interval of (0.00, 0.64), based upon the binomial distribution. Even if an adaptive design had produced the same numbers of participants treated at each dose, the data from all 17 participants would have been used to determine an estimate and 95% confidence interval³⁰ for the DLT probability of dose 4. This is because TITE-CRM computes confidence interval estimates for the DLT probabilities based on a model that uses all available data, not just from the final cohort. The resulting estimate will be closer to the truth (less biased), and the resulting 95% confidence interval will be much narrower.

Finally, the 3+3 design is rigid and cannot be modified to accommodate a variety of more contemporary clinical trial settings without using ad-hoc, and often statistically invalid, methods. In contrast, adaptive designs can be extended to fit the needs of any trial through the incorporation of statistically sound methods, as illustrated in the case vignette with late-onset DLTs. For example, adaptive designs can accommodate the study of both safety and efficacy simultaneously in the same group of participants^31-33, rather than require the need for an expansion cohort. Adaptive designs have also been created to examine the safety and efficacy of combinations of treatments, which is especially common with the advent of immunotherapies being used in conjunction with traditional treatment modalities. Thus, it is plausible that the dose of the traditional treatment might have to be altered because of its increased toxicity profile when used in combination with immunotherapy. If one wants to use the 3+3 design, multiple doses of only the traditional treatment or the immunotherapy could be studied and the other must be set at a fixed dose for all participants. In contrast, an adaptive design allows for multiple doses of both the traditional treatment and the immunotherapy to be studied, thereby producing a wider examination of the combined safety of both agents^34-38. Designs have also been created to examine toxicity over multiple cycles of treatment, whereby participants are treated over successive time periods (cycles), and the dose given at each cycle can vary both between participants as well as within each participant^7,39-41.

Limitations of the adaptive design

Sharon et al.⁴² describe some limitations associated with designing and implementing the TITE-CRM, many of which also apply to other adaptive designs. The process of designing a trial using the TITE-CRM can be time-consuming relative to simple algorithmic approaches, especially for a clinical trials group attempting to do so for the first time. More time and funding are needed to support the statistical team's effort at the design stage. Throughout the conduct of the study, strong collaboration between clinical, data management, and statistical teams, as well as frequent meetings with a safety review committee, may be necessary. Even though adaptive dose-finding designs intend to expedite Phase I trials with lengthy DLT evaluation windows, it may be problematic to have too many participants with pending DLT outcomes at once. This complication can result when accrual is too fast relative to the DLT observation window and can lead to difficult ethical and safety considerations in assigning future participants or a pause in accrual and longer trial duration. Nonetheless, these administrative challenges can be addressed and are far outweighed by the statistical benefits offered by adaptive designs.

Alternative designs

There are a variety of adaptive designs that can be applied to dose-finding studies, including modest extensions of TITE-CRM^6,7, the fractional CRM⁸, and the data-augmentation continual reassessment method⁹. Similar to the TITE-CRM, these designs rely on the specification of a statistical model to sequentially update estimates for the DLT probabilities at each of the dose levels and assign participants to doses based on which of these estimates is closest to a target DLT rate that defines the MTD. A more recent class of adaptive designs, known as model-assisted designs¹⁰, including the TITE Bayesian Optimal Interval (TITE-BOIN¹¹) method, were developed to combine the advantages of rule-based designs and model-based designs. These designs use a model only to derive a pre-specified set of escalation and de-escalation rules similar to that of a rule-based algorithm and estimate the MTD at the study conclusion. Unlike model-based designs, they do not adaptively fit a model based on accumulating DLT data across all dose levels after the accrual of each new cohort. Allocation decisions during the trial conduct are guided only by data observed “locally” at the current dose and the pre-determined algorithmic rules. Yuan et al¹¹ simulated 10,000 trials for each of 50,000 randomly generated dose-toxicity curves under various design conditions. Their study demonstrated that, on average, the correct dose is selected as the MTD 38.1% of the time using the R6 algorithm, 46.0% of the time using the TITE-CRM, and 44.2% of the time using the TITE-BOIN method.

Conclusion

In recent years, investigators have established that more innovative approaches are needed to address early-phase research questions, such as late-onset toxicities. Adaptive dose-finding methods have repeatedly demonstrated superior statistical properties relative to their rule-based counterparts. It has been shown that dose-finding designs with good statistical properties can significantly impact the development of new cancer therapeutics⁴³. In response, statisticians have continued to develop more flexible and efficient study designs. However, the execution of these designs in clinical practice has been limited. Novel design strategies require close collaboration between statisticians and clinical investigators to ensure that the most appropriate design is implemented correctly. Implementing an adaptive method, such as the TITE-CRM, presents design challenges such as defining dose-toxicity model parameters, specifying allocation and stopping rules, and providing support for timely data collection and reporting. Competency on these matters grows with increased application, but they can be challenging initially. Despite these challenges, designing and executing early-phase trials using adaptive designs is achievable and impactful. Design and implementation complexities have been mitigated in recent years thanks to the guidance documents such as that of Workhoven et al.⁴⁴ and available software such as https://sph.umich.edu/ccb/tite-resources.html.

Highlights.

• A common logistical challenge in designing and conducting early-phase trials of radiation therapy is how to sequentially assign dose levels to participants when DLT outcomes potentially take a long time to observe relative to the expected accrual rate and simultaneously provide preliminary estimates of treatment efficacy?

• Novel adaptive designs are based on the premise that as safety and other information accumulate, the data should guide dose assignments for future study participants.

• Several existing methods utilize data from patients that have been partially observed for dose-limiting toxicity in the estimation of outcome probabilities, weighting each entered patient by a portion of the full DLT observation window for which they have been followed.

• Adaptive designs offer several benefits that make them far more appropriate for Phase I clinical trials in radiation oncology than algorithmic approaches.

• Logistical challenges can be addressed and are far outweighed by the statistical benefits offered by adaptive designs.

Dos and don’ts.

• Do tailor the design to the science and trial objectives.

• Do not use an “off the shelf” method that does not meet the needs of the trial just because it is simple and convenient.

• Do use recommended pre-trial design specifications for the TITE-CRM and other adaptive designs.

• Do conduct simulation studies to evaluate the operating characteristics of a chosen design.

• Do use available resources and software to facilitate efficient collaboration between clinical and statistical teams.