Isotonic Phase I cancer clinical trial design utilizing patient-reported outcomes

Abstract

This article considers the concept of designing Phase I clinical trials using both clinician- and patient-reported outcomes to adaptively allocate study participants to tolerable doses and determine the maximum tolerated dose (MTD) at the study conclusion. We describe a new isotonic patient-reported outcome (PRO-ISO) Phase I design with the flexibility of allocating patients to lower, more tolerable regimens if a large number of PRO-DLT events are seen at higher doses. We conduct simulation studies of the operating characteristics of the design and compared them to the patient-reported outcomes continual reassessment method (PRO-CRM). We illustrate that the PRO-ISO makes appropriate dose assignments during the study to give investigators and reviewers an idea of how the method behaves. In simulation studies, the PRO-ISO demonstrates comparable performance to the PRO-CRM in terms of recommending target doses and allocating patients to these doses. It also performs well relative to a nonparametric optimal benchmark applied to the PRO setting. Finally, we extend our methodology to account for the problem of late-onset toxicities.

1. Introduction

This article describes an isotonic Phase I clinical trial design that uses clinician and patient-reported outcomes (PROs) to adaptively assign study participants to safe doses and determine the maximum tolerated dose (MTD) after the study. As a supplement to the National Cancer Institute Common Terminology Criteria for Adverse Events (NCI-CTCAE) used by clinicians to assess dose-limiting toxicities (DLTs), the patient-reported outcomes (PRO)-CTCAE offers an instrument for evaluating PRO-DLTs that can be incorporated into MTD estimation. The PRO-CTCAE contains 124 items, including 78 symptomatic toxicities from the CTCAE (Dueck et al., 2015). While many CTCAE categories, such as laboratory values, cannot be scored by the participants, the PRO-CTCAE (scored from 1–5) provides additional data to the NCI-CTACE in assessing overall tolerability. For instance, grade 3 nausea, according to the NCI-CTCAE, is “inadequate oral caloric or fluid intake; tube feeding; hospitalization indicated,” while grade 3 nausea, according to the PRO-CTCAE, is “moderate” severity. Patients with NCI-CTCAE grade 3 nausea will be expected to have a 4 (severe) or 5 (very severe) score on PRO-CTCAE. NCI-CTCAE objectively views toxicity from the clinician’s perspective, and PRO-CTCAE subjectively views toxicity from the patient’s perspective. If a particular dose results in a high rate of participants experiencing PRO-DLTs, an early-phase design should incorporate these events to allocate future study participants to more patient-tolerable doses (Henon et al., 2017).

To address this issue, Lee, Lu, and Cheng (2020) introduced three PRO-extensions of the likelihood-based, two-stage, continual reassessment method (CRM; O’Quigley et al., 1990) that use an NCI-DLT endpoint and a PRO-DLT endpoint in sequential dose assignments. Their patient-reported outcomes continual reassessment method (PRO-CRM) proposed a marginal modeling approach in which clinician and patient outcomes are modeled independently. The latter two approaches enforce a constraint using a joint outcome defined by clinician and patient outcomes and modeled jointly or marginally. The joint outcome with a joint modeling approach is analogous to estimating the MTD using the CRM with multiple toxicity constraints (Lee et al., 2011) and increases the number of multiple-type, multiple-toxicity phase I dose-finding designs (Bekele et al., 2004; Yuan et al., 2007; Chen et al., 2010; Ezzalfani et al., 2013; Lin, 2018; Mu et al., 2019). Simulation studies by Lee et al. (2020) show that the marginal approach has good operating characteristics and is straightforward to implement. The joint outcome based on marginal modeling indicates better performance but is more challenging to implement. Wages et al. (2022) applied a Bayesian form of the PRO-CRM using the marginal modeling framework to an ongoing Phase I study of adjuvant hypofractionated whole pelvis radiation therapy in endometrial cancer (NCT04458402).

This article proposes a new isotonic phase I design, using a marginal modeling framework, guided by both clinician- and patient-reported outcomes. In recent studies of the operating characteristics of adaptive dose-finding methods, extensions of the isotonic (ISO) method of Conaway, Dunbar, and Peddada (2004) have performed well in more complex dose-finding settings, such as drug combinations and patient heterogeneity, when compared to both model-based and model-assisted designs (Hirakawa et al., 2015; Conaway, 2017a; Conaway, 2017b; Wages, Ivanova, Marchenko, 2016; Conaway, Wages, 2017). Isotonic designs are particularly useful when studying a large number of dose levels in the drug combination setting (Wages, Reed, Keng, et al, 2021). The Conaway et al. (2004) method is a model-based method that has not yet been adapted to the patient-reported outcome problem. We hypothesize that a PRO-ISO approach will perform well in terms of the accuracy of MTD recommendation when compared to the PRO-CRM. The proposed method offers PRO dose-finding designs that (1) have good statistical properties when compared to competing methods, (2) mathematical simplicity resulting in fast computation and straightforward execution, and (3) easily understood interpretations that can be explained to clinical colleagues. The proposed design framework combines the advantages of model-based and model-assisted designs. It relies on a similar set of simple pre-trial specifications to model-assisted methods. Yet, like model-based methods, it can share DLT information across all dose levels using order-restricted inference techniques (Robertson and Dykstra, 1988), increasing efficiency. The details of our design follow in Section 2, including an extension of the method for addressing late-onset effects.

2. Methods

2.1. Modeling framework

We propose a design framework that uses order-restricted statistical inference for smoothing estimated DLT probabilities across dose levels (Leung and Wang, 2001; Ivanova and Flournoy, 2009; Wages and Conaway, 2018). Both clinician- and patient-reported DLT, termed C-DLT and P-DLT, respectively, are binary outcomes. Estimation is based on separate models for the probability of C-DLT and the probability of P-DLT, as well as the accumulated C-DLT and P-DLT data at each dose level, to sequentially allocate each new patient cohort. For general application, consider a Phase I trial evaluating $J$ discrete study dose levels $X=\left\{x_1,\dots ,x_J\right\}$. We index two binary endpoints by $i=1$ for a C-DLT and $i=2$ for a P-DLT, with $Y_1=1$ indicating the occurrence of a C-DLT, $Y_2=1$ indicating a P-DLT, and 0 otherwise. Denote the probability of observing the outcome $Y_i=1$ at dose level $x_j$ by ${\theta }_{ij},i=\mathrm{1,2};j=1,\dots ,J$. The primary objective of the study is to determine the MTD, $\gamma \in \left\{x_1,\dots ,x_J\right\}$, defined as the minimum dose with a C-DLT rate closest to the clinician-reported target C-DLT rate of ${\varphi }_1$ and the dose with a P-DLT rate closest to the patient-reported target P-DLT rate of ${\varphi }_2$ so that

$$\gamma =\text{min }\left(x_1^{*},x_2^{*}\right)$$

where $x_i^{*}\in \left\{x_1,\dots ,x_J\right\}$ and $x_i^{*}=\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}\left|{\theta }_{ij}-{\varphi }_i\right|$. The goal, both within and after the study, is to identify $\gamma$. We model the probability of observing outcome $Y_i=1$ at dose level $x_j$ via a beta-binomial model

$$y_{ij}\mid n_{ij},{\theta }_{ij}~\text{Binomial}\left(n_{ij},{\theta }_{ij}\right);{\theta }_{ij}~\text{Beta }\left({\alpha }_{ij},{\beta }_{ij}\right),$$ (1)

where $\text{Beta }\left({\alpha }_{ij},{\beta }_{ij}\right.$) is a beta distribution with parameters ${\alpha }_{ij}$ and ${\beta }_{ij}$, which are assumed to be independent for $i=\mathrm{1,2};j=1,\dots ,J$. At any point in the study, the data for outcome $i$ at dose level $x_j$ are $D_{ij}=\left\{\left(y_{ij},n_{ij}\right):i=\mathrm{1,2};j=1,\dots ,J\right\}$, where $y_{ij}$ is the number of observed occurrences of the outcome $Y_i$ at dose level $x_j$, and $n_{ij}$ is the number of patients evaluated for the occurrence of the outcome $Y_i$ at dose level $x_j$. Based on the accumulated data, the posterior distribution of ${\theta }_{ij}$ follows a beta distribution

$${\theta }_{ij}\mid D_{ij}~\text{Beta }\left({\alpha }_{ij}+y_{ij},{\beta }_{ij}+n_{ij}-y_{ij}\right).$$

Based on this distribution, the updated probability ${\hat{\theta }}_{ij}$ for outcome $i$ at dose level $j$ is given by the following approximation of the posterior median (Kerman, 2011)

$${\hat{\theta }}_{ij}=\frac{y_{ij}+{\alpha }_{ij}-1/3}{n_{ij}+{\alpha }_{ij}+{\beta }_{ij}-2/3}\text{. }$$

We also explored the use of the posterior mean but the operating characteristics of the design were slightly better when using the posterior median.

2.2. Late-onset toxicities

An assumption made in the above proposed model is that both toxicity outcomes are observed shortly after patients begin the experimental treatment. In real-world trials, however, treatments are being administered over extended periods of time, which results in relevant events that are late-onset, occurring outside of the short-term evaluation window. As an alternative to using only early outcome data, it is possible to extend the outcome evaluation window to a longer period of time in order to allow for late-onset events to be counted in allocation decisions. But a logistical challenge arises if the outcome is not captured soon enough, relative to the accrual rate. This challenge can lead to increased trial duration if completely observed toxicity data for each participant are required before dosing decisions can be made (Lin and Yuan, 2019; Yuan et al., 2018). As an alternative, it is possible to utilize available data from patients that have been partially observed in the estimation of outcome probabilities, weighting each entered patient by a portion of the full outcome observation window for which they have been followed. This idea was first proposed to accommodate late-onset toxicity in CRM (Cheung and Chappell, 2000), leading to the famous time-to-event CRM (TITE-CRM) design.

We extend the modeling framework above to the late-onset toxicity setting. We assume that the studied doses are to be administered over a pre-specified DLT outcome evaluation window $\tau$. We are assuming that a patient who experiences either type of DLT at any point before the completion of that outcome’s evaluation window is considered to reach the endpoint. They continue to be followed and observed for the occurrence of the other type of DLT. Data are still collected at the post-treatment follow-up time point (i.e., evaluation window length) for both patient-reported and physician-reported DLTs. Again, suppose that there are $J$ dose levels $d_1,\dots ,d_J$ being studied. Let ${\theta }_{ij};i=\mathrm{1,2};j=1,\dots ,J$ denote the true probability of DLT at dose level $j$ for outcome $i$. Let $z_{ik}$ denote the binary DLT (clinician- or patient-reported) outcome so that $z_{ik}=1$ if participant $k$ experiences DLT outcome $i$ within the evaluation window $(0,\tau )$ and $z_{ik}=0$ if participant $k$ does not experience DLT outcome $i$. Denote the time to DLT outcome $i$ for participants with $z_{ik}=1$ as $t_{ik}$ where $0\le t_{ik}\le \tau$. At any point in the trial, suppose that $y_{ij}=\sum _{k=1}^{n_j}{z}_{ik}$ DLT outcomes $i$ have been observed in $n_j$ participants who have been evaluated for both toxicity outcomes at dose level $j$. To model ${\theta }_{ij}$ at each dose level, we begin as above by assuming a beta-binomial model (1).

When there is a potential for late-onset outcomes and/or fast accrual, the design challenge is that there will be participants for whom $z_{ik}$ has not yet been observed at the time a dosing decision needs to be made for a newly accrued participant. Following the notation of Lin and Yuan (2020), the observed data ${\tilde{z}}_{ik},i=\mathrm{1,2};k=1,\dots ,n_j$, indicate whether participant $k$ has experienced DLT outcome $i$ at decision time $\left({\tilde{z}}_{ik}=1\right)$ or not $\left({\tilde{z}}_{ik}=0\right)$. If ${\tilde{z}}_{ik}=1$, then $z_{ik}=1$, but if ${\tilde{z}}_{ik}=0$, then $z_{ik}$ could be 0 or 1. Let ${\delta }_{ik}$ be an indicator variable for whether DLT outcome $z_{ik}$ has been determined (${\delta }_{ik}=1$) or is still pending (${\delta }_{ik}=0$) for participant $k$ at dosing decision time for the next accrual. Let $u_{ik}\le \tau$ denote the follow-up time for participant $k$ at this time. The observed interim data for outcome $i$ at dose level $j$ is $D_{ij}=\left\{\left(z_{ik},{\delta }_{ik}\right),i=\mathrm{1,2};k=1,\dots ,n_j\right\}$ and an approximate likelihood, derived by Lin and Yuan (2020), is given by

$$L(D_{ij}|{\theta }_{ij})={\theta }_{ij}^{{\tilde{y}}_{ij}}{\left(1-{\theta }_{ij}\right)}^{{\tilde{m}}_{ij}}$$ (2)

where ${\tilde{y}}_{ij}=\sum _{k=1}^{n_j} {\delta }_{ik}{z}_{ik}$ is the number of participants who have experienced DLT at the time of the next dose assignment, ${\tilde{m}}_{ij}=\sum _{k=1}^{n_j} {\delta }_{ik}\left(1-z_{ik}\right)+\sum _{k=1}^{n_j} \left(1-{\delta }_{ik}\right)w_{ik}$. In the latter expression, $m_{ik}$ is the number of participants who have completed the DLT evaluation window $\tau$ without experiencing DLT outcome $i$, and $w_{ik}$ is a weight indicating the amount of information participant $k$ is contributing to the likelihood. If participant $k$ has experienced a DLT outcome at any time prior to the current decision time or they have completed the DLT evaluation window $\tau$ without experiencing DLT, then $w_{ik}=1$. If participant $k$ has not experienced a DLT and they have not yet completed the DLT evaluation window (i.e. $u_{ik}\le \tau$), then $w_{ik}$ is a function of the time participant $k$ has been followed at the time of the next accrual. Lin and Yuan (2020) showed that the posterior distribution of ${\theta }_{ij}$ can be approximated by a $\text{Beta }\left({\alpha }_{ij}+{\tilde{y}}_{ij},{\beta }_{ij}+{\tilde{m}}_{ij}-{\tilde{y}}_{ij}\right)$ distribution. Therefore, we can estimate the probability ${\hat{\theta }}_{ij}$ for outcome $i$ at dose level $j$ is using the approximation of the posterior median (Kerman, 2011)

$${\hat{\theta }}_{ij}=\frac{{\tilde{y}}_{ij}+{\alpha }_{ij}-1/3}{{\tilde{m}}_{ij}+{\alpha }_{ij}+{\beta }_{ij}-2/3}.$$

2.3. Isotonic regression

For each outcome $i$, to impose monotonicity with respect to the dose-toxicity relationship and borrow information across dose levels, we apply the pool adjacent violators algorithm (PAVA; Robertson et al., 1988) to the posterior means ${\hat{\theta }}_{ij}$, denoting the isotonic estimates by ${\tilde{\theta }}_{ij}$. This algorithm replaces adjacent DLT probability estimates that violate the monotonicity assumption with their weighted average, where the weights are the current sample size at each dose level. We use the following PAVA implementation as described by Berry et al. (2011). Let $c=\left(c_1,\dots ,c_J\right)$ be a set of indices that indicate the pooling of adjacent values. Any doses with matching indices $c_j$ are pooled.

• Step 0:

  Let $\tilde{\theta }=\hat{\theta }$ and $c=(\mathrm{1,2},\dots ,J)$.

• Step 1:

  Let $V=\left\{j:{\tilde{\theta }}_{ij}>{\tilde{\theta }}_{i,j+1},i=\mathrm{1,2};j<J\right\}$ denote the set of adjacent violators.

• Step 2:

  If $v=\varnothing$, then stop the iteration. Otherwise, select the first violator $v=c_{V_1}$. Let $W=\left\{j:c_j=v\right.$ or $\left.c_j=v+1\right\}$. Let $m_w=\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n}\left({\tilde{\theta }}_w\right)$ denote the weighted average value over $W$, where the weights are the current sample size at each dose level.

• Step 3:

  Set $c_i\equiv v,i\in W$ and replace $\tilde{\theta }\equiv m_w,i\in W$. Repeat from Step 1.

2.4. Dose-finding algorithm

Based on interim DLT data that is available at the time a dosing decision is to be made, these DLT probability estimates can be used to make allocation decisions according to the following algorithm described by Conaway, Dunbar, and Peddada (CDP; 2004), which has been studied extensively in previous work in the early-onset DLT setting (Conaway et al, 2004; Conaway and Wages, 2017; Conaway, 2017 a,b; Wages and Fadul, 2020). Our proposed designs can be considered an extension of the CDP method to include PROs, so we designate them as the PRO-ISO method and the TITE-PRO-ISO method for early- and late-onset outcomes, respectively. Let $A$ denote the set of doses that have been tried thus far in the trial such that $A=\left\{d_j:n_j>0\right\}$. For each outcome $i$,

1. For all $d_j\in A$, compute the loss ${ℒ}_{ij}\left({\tilde{\theta }}_{ij},{\varphi }_i\right)$ based on the target DLT rate ${\varphi }_i$ such that

   $${ℒ}_{ij}({\tilde{\theta }}_{ij},{\varphi }_i)=\{\begin{array}{cc}\lambda \left({\varphi }_i-{\tilde{\theta }}_{ij}\right)& \text{ if }{\varphi }_i>{\tilde{\theta }}_{ij},\\ \left(1-\lambda \right)\left({\tilde{\theta }}_{ij}-{\varphi }_i\right)& \text{ if }{\varphi }_i\le {\tilde{\theta }}_{ij}\end{array},$$

   which, for $\lambda <0.5$, gives greater distance to estimated probabilities above ${\varphi }_i$. The value of $\lambda$ represents the severity of the penalty we place on overdosing.

2. Let $l_i={\mathrm{m}\mathrm{i}\mathrm{n}}_{d_j\in A} {ℒ}_{ij}\left({\tilde{\theta }}_{ij},{\varphi }_i\right)$, and let $H_i$ be the set of doses with losses equal to the minimum observed loss so that $H_i=\left\{d_j:{ℒ}_{ij}\left({\tilde{\theta }}_{ij},{\varphi }_i\right)=l_i\right\}$.

3. If $H_i$ contains more than one dose, then we choose from among them according to the rules:

   1. If ${\tilde{\theta }}_{ij}>{\varphi }_i\forall h\in H_i$, the suggested dose is the lowest dose in $H_i$.
   2. If ${\tilde{\theta }}_{ij}\le {\varphi }_i$ for at least one $h\in H_i$, the suggested dose is the highest dose in $H_i$ with ${\tilde{\theta }}_{ij}\le {\varphi }_i$.

4. If the suggested dose level for outcome $i$ has an estimated DLT probability that is less than ${\varphi }_i$, then the next highest dose level will be recommended if it has not yet been tried. For TITE-PRO-ISO, at least one participant must be followed for a minimum follow-up window ${\tau }^{*}<\tau$ without DLT for each outcome $i$ at the suggested dose before the next highest dose level can be recommended. Otherwise, we treat at the suggested dose level.

If at any time in the accrual process, $d_1$ is deemed too toxic, then the trial stops for safety and no dose is recommended as the MTD. Based on the posterior distribution $f\left({\theta }_{i1}\mid D_{i1}\right)$ and the pre-specified target DLT rate ${\varphi }_i$, we calculate the posterior probability that $d_1$ is too toxic and compare this probability to an upper probability cutoff. If $\mathrm{P}\mathrm{r}\left({\theta }_{i1}>{\varphi }_i\mid D_{i1}\right)>p_i$ we say that $d_1$ is not a safe dose and the trial terminates early for safety. Appropriate cut-off values $p_i$ are typically in the range from 0.80–0.95 and can be tuned via simulation studies (Yuan, Nguyen, Thall, 2016).

At each accrual decision, we use the estimated probabilities ${\tilde{\theta }}_{ij}$ for each outcome to guide allocation and identify the MTD after the study. After each cohort inclusion, we estimate $x_i^{*}$ using the estimated probabilities ${\tilde{\theta }}_{ij}$ so that ${\tilde{x}}_i^{*}=\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}\left|{\tilde{\theta }}_{ij}-{\varphi }_i\right|$. We then allocate the next cohort of participants to $\tilde{\gamma }\in \left\{x_1,\dots ,x_J\right\}$, where

$$\tilde{\gamma }=\text{min }\left({\tilde{x}}_1^{*},{\tilde{x}}_2^{*}\right),$$

with the restriction the trial is not allowed to skip dose levels when escalating. Accrual to the study will end after the maximum target sample size of $N$ participants has been accrued to the study. The MTD is defined as the dose level $\tilde{\gamma }$ that would have been administered to the next cohort had one been included.

2.5. Prior specifications

The beta prior distributions are used as smoothing parameters in the isotonic estimation. The elicitation of suitable priors rely upon practical guidelines for Bayesian adaptive clinical trial design (Thall and Simon, 1994). We ask investigators to specify the expected value of the DLT probability ${\theta }_{ij}$ at each dose level and an upper bound $v_{ij}$ such that they are 95% certain that the DLT probability will not exceed $v_{ij}$. Based on the expected value of ${\theta }_{ij}$ and a 95% upper limit $v_{ij}$ on the DLT probability, the equations

$$\text{E}\left({\theta }_{ij}\right)=\frac{{\alpha }_{ij}}{{\alpha }_{ij}+{\beta }_{ij}}\text{ and Pr}\left({\theta }_{ij}\le v_{ij}\right)=0.95$$

are solved to obtain prior specifications for ${\alpha }_{ij}+{\beta }_{ij}$. In the absence of prior information, a practical prior specification can be acquired by setting the prior mean equal to the target DLT rate ${\varphi }_i$ and setting the 95% upper limit $v_{ij}$ equal to $2\times {\varphi }_i$ at each dose level. This prior specification is recommended to avoid the problem of rigidity (Cheung, 2002) in which allocation can become confined to a sub-optimal dose level regardless of the ensuing observed data. With a smaller effective sample size (ESS) and low target DLT rate ${\theta }^{*}$, early DLTs can heavily impact the dose assignment algorithm, potentially preventing the design from ever returning to doses with only 1/1 DLTs observed, for example. Specifically, suppose for the traditional CTCAE DLT endpoint, we are targeting a DLT rate of 0.20. A Beta(0.4, 1.6) prior (ESS=2) would yield a posterior mean equal to $\frac{y_j+0.4}{n_j+1.6+0.4}$. Suppose that the first participant on dose level 1 does not experience a DLT so that $y_1=0;n_1=1$ and the posterior mean is ${\tilde{\theta }}_1=\frac{0+0.4}{1+1.6+0.4}=\frac{0.4}{3}=0.13$. According to the dose assignment algorithm described in the paper, the trial would escalate to dose level 2. If a DLT is observed on the first participant accrued to dose level 2 with this prior specification (i.e., $y_2=n_2=1$), then the posterior mean is ${\tilde{\theta }}_2=\frac{1+0.4}{1+1.6+0.4}=\frac{1.4}{3}=0.47$. At this point, the trial will return to dose level 1, and the estimate at dose level 2 will remain 0.47, meaning $\left|{\tilde{\theta }}_2-0.20\right|=0.27$ because the data collected at dose level 1 below will not affect the estimation of ${\hat{\theta }}_2$ (unless dose level 1 becomes very toxic, at which point the trial would terminate for safety). Consequently, $\left|{\tilde{\theta }}_1-0.20\right|<0.27$ and the trial will stay at dose level 1 indefinitely. A similar illustration could be provided if we encountered 2 DLTs on the first 2 participants accrued to dose level 2. Therefore, in general we want the prior to satisfy the following conditions at each dose level $j$ for each endpoint $i$ to avoid rigidity.

$$\frac{1+{\alpha }_{ij}}{1+{\alpha }_{ij}+{\beta }_{ij}}<2\times {\theta }_i^{*}\text{ and }\frac{2+{\alpha }_{ij}}{2+{\alpha }_{ij}+{\beta }_{ij}}<2\times {\theta }_i^{*}.$$

This approach to prior elicitation has been studied extensively in a variety of settings for early-onset DLTs (Conaway et al, 2004; Conaway and Wages, 2017; Conaway, 2017 a,b; Wages and Fadul, 2020), and demonstrated robust operating characteristics over a broad range of scenarios.

2.6. Nonparametric benchmark

A necessary component of the evaluation process is to have some concept for how well a design can possibly perform. A nonparametric optimal benchmark is a theoretical tool for simulations and was first described by O’Quigley, Paoletti, and Maccario (2002), as an upper bound on the accuracy of identifying the MTD based on a binary toxicity endpoint. Zohar and O’Quigley (2006) further developed the benchmark to account for two binary endpoints. Here we adapt the O’Quigley and Zohar (2006) benchmark to two binary toxicity outcomes. During the course of a Phase I trial, each patient receives a dose and is observed for the outcomes only at that dose. Therefore, we can only observe information that is partial. For instance, consider a trial investigating six available dose levels and suppose there is only one binary DLT endpoint $Y$. The following illustrations will apply to both clinican- and patient-reported DLTs. Suppose a patient is given dose level 4 and experiences a DLT. The monotonicity assumption implies that a DLT would necessarily be observed at dose levels 5 and 6. We will not have any information regarding whether the patient would have suffered a DLT for any dose below level 4, creating a vector of partial information $Y_j=\left(*,*,*,\mathrm{1,1},1\right)$ where * indicates dose levels for which DLT information is not available. Conversely, should dose 3 be deemed safe for an enrolled patient, then we can infer that he or she would experience a non-DLT outcome at dose levels 1 and 2. However, any information concerning whether the patient would have had a DLT had he or she been given any dose above level 3 is unknown, creating a vector of partial information $Y_j=\left(\mathrm{0,0},0,*,*,*\right)$.

In simulating trial data, however, we can generate each patient’s latent outcome from which we can observe complete toxicity information at all available dose levels (O’Quigley et al., 2002). The following illustration applies to both C-DLT and P-DLT outcomes, but we simply use the term DLT for the sake of brevity. For each patient, if we knew the DLT outcome at each available dose level, we would have complete information. In simulating trial data, we can generate each patient’s latent outcome from which we can observe DLT at all available dose levels. For example, suppose we have a patient that experiences a DLT from dose level 3, generating a vector of complete information $Y_j=(\mathrm{0,0},\mathrm{0,1},\mathrm{1,1})$. In the simulation of DLT outcomes in a trial, the latent toxicity tolerance for outcome $i,V_{ik}$, of patient $k$ can be considered a uniformly distributed random variable $V_{ik}~\text{Uniform }(0,1)$, which we term a patient’s latent toxicity tolerance for outcome $i$ and denote $v_{ik}$ for the $k$th entered patient Paoletti (2002). At the dose $(\left.d_j\right)$ assigned to patient $k$, if the tolerance is less than or equal to its true DLT probability (i.e. $v_k\le {\theta }_{ij}$)), then patient $j$ has a D < otherwise the patient has a non-DLT outcome. It is assumed that a patient who experiences a DLT at a particular dose level, would necessarily experience that type of DLT (i.e., clinician- or patient-reported) at any higher level. Similarly, if a patient does not have a DLT at a given dose level, he or she would also not have one any lower level. For instance, complete information for a C-DLT occurring at dose level $d_4$ and above is represented by $Y_{1j}=(\mathrm{0,0},\mathrm{0,1},\mathrm{1,1})$. Similarly, complete information for a P-DLT occurring at dose level $d_3$ and above is represented by $Y_{2j}=(\mathrm{0,0},\mathrm{1,1},\mathrm{1,1})$.

Table 1 presents the complete vectors of 18 simulated patients for true C-DLT probabilities ${\theta }_{11}=0.05,{\theta }_{12}=0.05,{\theta }_{13}=0.25,{\theta }_{14}=0.40$, and ${\theta }_{15}=0.55$, and true P-DLT probabilities ${\theta }_{21}=0.17,{\theta }_{22}=0.18,{\theta }_{23}=0.35,{\theta }_{24}=0.50$, and ${\theta }_{25}=0.65$. This set of true C-DLT and PDLT probabilities serves as Scenario 1 in our simulation studies below. Patient 3, for instance, has a latent C-DLT outcome of $v_{11}=0.31$, and a latent P-DLT outcome of $v_{21}=0.37$. Therefore, he or she will experience a non-C-DLT at dose levels 1, 2 and 3 because $v_{11}>{\theta }_{1j}$ for $j=\mathrm{1,2},3$, and suffer a DLT at levels 4 and 5 because $v_{11}<{\theta }_{1j}$ for $k=\mathrm{4,5}$. Similarly, he or she will experience a non-P-DLT at dose levels 1, 2 and 3 because $v_{21}>{\theta }_{2j}$ for $j=\mathrm{1,2},3$, and experience a P-DLT at levels 4 and 5 because $v_{21}<{\theta }_{2j}$ for $j=\mathrm{4,5}$. We can use these simulated outcomes for both endpoints to estimate the DLT probabilities by using the sample proportion of observed toxicities at each dose. That is, we can estimate ${\theta }_{1j}$ and ${\theta }_{2j};j=1,\dots ,J$ from the sample proportions

$${\hat{\theta }}_{1j}=\frac{1}{N}\sum _{k=1}^N{Y}_{1k},{\hat{\theta }}_{2j}=\frac{1}{N}\sum _{k=1}^N{Y}_{2k}.$$

Table 1:

Simulated trial of complete information

Patient	Latent	Pr (C-DLT) at					Latent	Pr (P-DLT) at
Number	C-DLT	dose level ${\mathit{x}}_{\mathit{j}}$					P-DLT	dose level ${\mathit{x}}_{\mathit{j}}$
$k$	$v_{1j}$	0.05	0.05	0.25	0.40	0.55	$v_{2j}$	0.17	0.18	0.35	0.50	0.65
1	0.04	1	1	1	1	1	0.85	0	0	0	0	0
2	0.98	0	0	0	0	0	0.46	0	0	0	1	1
3	0.31	0	0	0	1	1	0.37	0	0	0	1	1
4	0.71	0	0	0	0	0	0.41	0	0	0	1	1
5	0.43	0	0	0	0	1	0.19	0	0	1	1	1
6	0.24	0	0	1	1	1	0.56	0	0	0	0	1
7	0.47	0	0	0	0	1	0.27	0	0	1	1	1
8	0.54	0	0	0	0	1	0.13	1	1	1	1	1
9	0.48	0	0	0	0	1	0.91	0	0	0	0	0
10	0.28	0	0	0	1	1	0.71	0	0	0	0	0
11	0.76	0	0	0	0	0	0.67	0	0	0	0	0
12	0.14	0	0	1	1	1	0.36	0	0	0	1	1
13	0.97	0	0	0	0	0	0.45	0	0	0	1	1
14	0.60	0	0	0	0	0	0.70	0	0	0	0	0
15	0.06	0	0	1	1	1	0.89	0	0	0	0	0
16	0.13	0	0	1	1	1	0.48	0	0	0	1	1
17	0.97	0	0	0	0	0	0.21	0	0	1	1	1
18	0.46	0	0	0	0	1	0.49	0	0	0	1	1
	${\hat{\theta }}_{1j}$	0.06	0.06	0.28	0.39	0.67	${\hat{\theta }}_{2j}$	0.06	0.06	0.22	0.61	0.67

At the study conclusion, the benchmark of O’Quigley and Zohar (2006) aimed to identify the dose $\gamma \in \left\{x_1,\dots ,x_J\right\}$ that maximized the probability of success ${\theta }_{2\gamma }\times \left\{1-{\theta }_{1\gamma }\right\}$, where the probability of toxicity at dose $j$ is denoted ${\theta }_{1j}$ and the probability of efficacy at dose $j$ is denoted ${\theta }_{2j}$. We adapted the benchmark to select the dose $\gamma \in \left\{x_1,\dots ,x_J\right\}$, defined as the minimum dose with a C-DLT rate closest to the clinician-reported target C-DLT rate of ${\varphi }_1$ and the dose with a P-DLT rate closest to the patient-reported target P-DLT rate of ${\varphi }_2$ so that $\gamma =\mathrm{m}\mathrm{i}\mathrm{n}\left(x_1^{*},x_2^{*}\right)$, where $x_i^{*}\in x_1,\dots ,x_J$ and $x_i^{*}=\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}\left|{\theta }_{ij}-{\varphi }_i\right|$. The last row of Table 1 gives the sample proportions of the simulated trial. After 18 patients, the recommended dose is level 3, based on target DLT rates of ${\varphi }_1=0.25$ and ${\varphi }_2=0.35$.

3. Results

3.1. Operating characteristics

Simulations were run to display the performance of the design characteristics for sample sizes of $N=18$ and $N=40$. The only other existing method that adaptively accounts for both C-DLTs and P-DLTs in its design is the method of Lee et al³. This method employs a two-stage, likelihood-based CRM approach (PRO-CRM) for both toxicity endpoints (NCI and PRO), so we compare our isotonic approach (PRO-ISO) to the operating characteristics reported in their paper for the marginal modeling approach. All scenarios and PRO-CRM results are taken from Tables 2 and 3 in Lee et al (2020). Among $J=5$ study dose levels, the MTD is defined by the dose with C-DLT rate closest to the target rate of ${\varphi }_1=25\mathrm{\%}$ and a P-DLT rate closest to the target rate of ${\varphi }_2=35\mathrm{\%}$. In Scenarios 1, 3, and 4, the dose with a C-DLT rate closest to its target value of 0.25 is the same as the dose with the P-DLT rate closest to its target value of 0.35, making this dose the MTD. In Scenarios 2, 5, 6, and 7, the dose with a C-DLT rate closest to its target value of 0.25 is different from the dose with the P-DLT rate closest to its target value of 0.35, making the MTD the minimum of these two doses. In a hypothetical scenario in which there is a dose with a C-DLT rate slightly higher than 0.25 and a P-DLT rate slightly lower than 0.35, this dose would be considered the if both of these probabilities are closest to their respective target values. For instance, if the C-DLT probabilities are (0.02, 0.05, 0.10, 0.27, 0.40) and the P-DLT probabilities are (0.09, 0.17, 0.20, 0.33, 0.50), the MTD is dose level 4 based on our rule for determining the MTD. For the PRO-CRM, For the clinician constraint alone, given five dose levels, a target DLT rate of 0.25, and assuming the MTD is dose level 3, the skeleton is (0.02, 0.10, 0.25, 0.44, 0.62) for $N=18$ and (0.06, 0.14, 0.25, 0.38, 0.50) for $N=40$, respectively. For the patient constraint alone, given five dose levels, a target DLT rate of 0.35, and assuming the MTD is dose level 3, the skeleton is (0.06, 0.18, 0.35, 0.53, 0.68) for $N=18$ and (0.10, 0.21, 0.35, 0.49, 0.61) for $N=40$, respectively. For these simulations, we did not employ a safety stopping rule to make the results as comparable as possible with those in Lee et al³. For our proposed isotonic method, we used a value of $\lambda =0.5$ corresponding to a symmetric loss function, although the method has the flexibility to use a value of $\lambda <0.5$ in order to penalize overdosing.

Table 2:

MTD selection percentages for the proposed PRO-ISO, PRO-CRM, and a nonparametric optimal benchmark over 7 dose-toxicity scenarios from Lee et al. (2020). The target C-DLT rate is 0.25, the target P-DLT rate is 0.35, and the sample sizes are $N=18$ and $N=40$ participants.

	$N=18$					$N=40$
Scenario 1
Probability of C-DLT	0.05	0.05	0.25	0.40	0.50	0.05	0.05	0.25	0.40	0.50
Probability of P-DLT	0.17	0.18	0.35	0.50	0.65	0.17	0.18	0.35	0.50	0.65
PRO-CRM	6.0	30.0	57.0	7.0	0.0	1.0	18.0	76.0	5.0	0.0
PRO-ISO	3.6	28.4	58.0	8.8	1.2	1.6	21.6	72.4	4.4	0.0
TITE-PRO-ISO	3.1	27.8	55.3	12.6	0.9	0.8	22.0	71.6	5.2	0.0
Benchmark	11.7	12.9	68.8	6.5	0.0	3.9	8.9	85.5	1.7	0.0
Scenario 2
Probability of C-DLT	0.05	0.25	0.40	0.55	0.70	0.05	0.25	0.40	0.55	0.70
Probability of P-DLT	0.10	0.15	0.35	0.50	0.65	0.10	0.15	0.35	0.50	0.65
PRO-CRM	11.0	62.0	25.0	2.0	0.0	6.0	75.0	19.0	0.0	0.0
PRO-ISO	11.6	62.8	25.2	0.4	0.0	7.6	76.4	16.0	0.0	0.0
TITE-PRO-ISO	9.3	57.9	29.7	2.9	0.1	5.4	75.7	18.4	0.4	0.0
Benchmark	8.0	69.2	22.3	0.5	0.0	1.8	85.7	12.5	0.0	0.0
Scenario 3
Probability of C-DLT	0.01	0.02	0.05	0.10	0.25	0.01	0.02	0.05	0.10	0.25
Probability of P-DLT	0.04	0.09	0.17	0.20	0.35	0.04	0.09	0.17	0.20	0.35
PRO-CRM	0.0	2.0	13.0	38.0	47.0	0.0	0.0	2.0	27.0	71.0
PRO-ISO	0.0	2.0	14.4	40.0	43.6	0.0	0.4	2.8	40.0	56.8
TITE-PRO-ISO	0.0	1.0	10.0	40.1	48.8	0.0	0.1	2.2	35.8	61.8
Benchmark	0.2	0.9	8.7	27.4	62.8	0.0	0.0	1.7	18.0	80.2
Scenario 4
Probability of C-DLT	0.02	0.05	0.10	0.25	0.40	0.02	0.05	0.10	0.25	0.40
Probability of P-DLT	0.09	0.17	0.20	0.35	0.50	0.09	0.17	0.20	0.35	0.50
PRO-CRM	1.0	11.0	37.0	45.0	6.0	0.0	2.0	26.0	68.0	4.0
PRO-ISO	1.2	8.0	38.0	46.8	6.0	0.8	1.2	34.4	59.6	4.0
TITE-PRO-ISO	0.6	6.8	34.8	47.0	10.8	0.1	1.7	30.9	62.8	4.5
Benchmark	0.5	8.1	25.9	59.4	6.1	0.0	1.5	19.2	77.5	1.7
Scenario 5
Probability of C-DLT	0.05	0.10	0.16	0.25	0.40	0.05	0.10	0.16	0.25	0.40
Probability of P-DLT	0.05	0.20	0.35	0.50	0.65	0.05	0.20	0.35	0.50	0.65
PRO-CRM	2.0	31.0	55.0	12.0	0.0	0.0	15.0	73.0	12.0	0.0
PRO-ISO	1.6	27.2	50.0	20.4	0.8	0.0	25.2	61.2	13.6	0.0
TITE-PRO-ISO	0.9	24.6	51.0	21.2	2.2	0.2	17.5	66.0	16.0	0.3
Benchmark	1.0	27.3	54.6	16.7	0.5	0.0	15.5	74.2	10.2	0.0
Scenario 6
Probability of C-DLT	0.05	0.18	0.20	0.25	0.40	0.05	0.18	0.20	0.25	0.40
Probability of P-DLT	0.17	0.35	0.50	0.65	0.80	0.17	0.35	0.50	0.65	0.80
PRO-CRM	21.0	62.0	16.0	1.0	0.0	10.0	76.0	14.0	0.0	0.0
PRO-ISO	24.4	54.0	27.7	4.2	0.1	16.0	66.8	16.8	0.4	0.0
TITE-PRO-ISO	15.6	56.9	23.1	3.8	0.1	10.6	69.5	18.6	0.8	0.0
Benchmark	18.9	62.0	17.9	1.1	0.0	8.8	80.6	10.6	0.0	0.0
Scenario 7
Probability of C-DLT	0.01	0.05	0.10	0.16	0.25	0.01	0.05	0.10	0.16	0.25
Probability of P-DLT	0.04	0.05	0.20	0.35	0.50	0.04	0.05	0.20	0.35	0.50
PRO-CRM	0.0	3.0	33.0	52.0	12.0	0.0	0.0	15.0	73.0	12.0
PRO-ISO	0.0	2.8	36.0	47.6	13.6	0.0	0.0	26.0	63.6	10.4
TITE-PRO-ISO	0.0	1.8	27.5	49.0	21.8	0.0	0.3	19.1	66.4	14.2
Benchmark	0.2	1.2	28.1	54.8	15.6	0.0	0.0	16.2	73.9	10.0

Table 3:

Percentage of patient allocation for the proposed PRO-ISO, PRO-CRM, and a nonparametric optimal benchmark over 7 dose-toxicity scenarios from Lee et al. (2020). The target C-DLT rate is 0.25, the target P-DLT rate is 0.35, and the sample sizes are $N=18$ and $N=40$ participants.

	$N=18$					$N=40$
Scenario 1
Probability of C-DLT	0.05	0.05	0.25	0.40	0.50	0.05	0.05	0.25	0.40	0.50
Probability of P-DLT	0.17	0.18	0.35	0.50	0.65	0.17	0.18	0.35	0.50	0.65
PRO-CRM	20.0	32.0	38.0	9.0	1.0	11.0	26.0	53.0	9.0	1.0
PRO-ISO	14.4	31.7	39.4	12.2	2.2	7.8	30.3	52.3	8.8	1.0
TITE-PRO-ISO	25.7	30.2	31.3	10.6	2.2	12.5	27.0	47.3	11.0	2.0
Scenario 2
Probability of C-DLT	0.05	0.25	0.40	0.55	0.70	0.05	0.25	0.40	0.55	0.70
Probability of P-DLT	0.10	0.15	0.35	0.50	0.65	0.10	0.15	0.35	0.50	0.65
PRO-CRM	25.0	45.0	23.0	6.0	1.0	17.0	55.0	24.0	3.0	1.0
PRO-ISO	20.6	49.4	24.4	5.0	0.6	13.8	60.8	23.0	2.3	0.3
TITE-PRO-ISO	26.8	43.0	23.5	6.1	1.1	15.8	54.3	25.0	4.3	0.8
Scenario 3
Probability of C-DLT	0.01	0.02	0.05	0.10	0.25	0.01	0.02	0.05	0.10	0.25
Probability of P-DLT	0.04	0.09	0.17	0.20	0.35	0.04	0.09	0.17	0.20	0.35
PRO-CRM	8.0	11.0	19.0	29.0	33.0	4.0	5.0	10.0	30.0	51.0
PRO-ISO	7.2	11.1	18.9	31.7	30.6	3.0	4.8	10.0	39.3	43.0
TITE-PRO-ISO	17.9	20.1	23.5	21.8	17.3	8.0	9.3	13.3	29.0	40.3
Scenario 4
Probability of C-DLT	0.02	0.05	0.10	0.25	0.40	0.02	0.05	0.10	0.25	0.40
Probability of P-DLT	0.09	0.17	0.20	0.35	0.50	0.09	0.17	0.20	0.35	0.50
PRO-CRM	13.0	19.0	31.0	29.0	8.0	6.0	9.0	30.0	47.0	8.0
PRO-ISO	10.6	16.7	31.1	32.8	8.9	5.3	8.8	34.3	43.8	8.0
TITE-PRO-ISO	20.7	25.1	26.8	20.1	7.8	9.3	13.3	29.0	38.3	10.3
Scenario 5
Probability of C-DLT	0.05	0.10	0.16	0.25	0.40	0.05	0.10	0.16	0.25	0.40
Probability of P-DLT	0.05	0.20	0.35	0.50	0.65	0.05	0.20	0.35	0.50	0.65
PRO-CRM	16.0	34.0	36.0	12.0	2.0	8.0	25.0	52.0	14.0	1.0
PRO-ISO	12.2	30.6	36.7	17.2	3.3	5.3	29.0	48.5	16.0	1.5
TITE-PRO-ISO	21.2	33.0	29.6	12.8	3.4	9.8	26.0	43.0	17.8	3.5
Scenario 6
Probability of C-DLT	0.05	0.18	0.20	0.25	0.40	0.05	0.18	0.20	0.25	0.40
Probability of P-DLT	0.17	0.35	0.50	0.65	0.80	0.17	0.35	0.50	0.65	0.80
PRO-CRM	36.0	45.0	16.0	3.0	0.0	24.0	57.0	17.0	2.0	0.0
PRO-ISO	30.6	42.8	20.0	5.6	1.1	23.0	55.5	19.0	2.5	0.3
TITE-PRO-ISO	33.0	42.5	18.4	5.6	1.1	21.8	52.0	20.8	4.5	0.8
Scenario 7
Probability of C-DLT	0.01	0.05	0.10	0.16	0.25	0.01	0.05	0.10	0.16	0.25
Probability of P-DLT	0.04	0.05	0.20	0.35	0.50	0.04	0.05	0.20	0.35	0.50
PRO-CRM	8.0	14.0	32.0	34.0	12.0	4.0	6.0	24.0	51.0	15.0
PRO-ISO	7.2	13.3	29.4	35.6	14.4	3.0	5.5	28.5	48.3	14.8
TITE-PRO-ISO	17.9	20.7	28.5	22.3	11.2	8.0	9.8	24.8	40.0	17.5

For each scenario, 10,000 simulated trials were run, and the results are reported in Table 2. We also examine the performance of the time-to-event version of our proposed PRO-design (TITE-PRO-ISO) under late-onset scenarios. We assess the operating characteristics of TITE-PRO-ISO under a 3-month DLT evaluation window (i.e., $\tau =3$) for both toxicity endpoints, and the minimum follow-up window that at least one participant must clear without DLT before recommending a higher dose is ${\tau }^{*}=1$. We explored alternative values of ${\tau }^{*}=1.5$ and 2, but ${\tau }^{*}=1$ achieved the best balance between accuracy and safety. The value of ${\tau }^{*}$ to use in any particular setting can be calibrated via simulation studies. We generated both times to toxicity (C-DLT and P-DLT) from a uniform distribution as in Cheung and Chappell (2000). We first determine whether a patient has a P-DLT and/or C-DLT response. If so, for each observed DLT outcome, we generate a failure time uniformly on the interval (0, 3). The patient accrual rate is assumed to be 2 patients per month. We compare the TITE-PRO-ISO design with original PRO-ISO design, as the latter uses the complete data for making dose optimization decisions.

In Scenario 1, the performance of PRO-CRM and PRO-ISO are nearly identical in terms of the percentage of correctly selecting the true MTD (dose level 3) (PRO-ISO 58% vs. PRO-CRM 57%, respectively) for $N=18$. The nonparametric benchmark correctly selects dose level 3 as the MTD in 68.8% of simulated trials. In terms of participant assignment, the PRO-ISO treats a slightly higher percentage of participants on average at this dose than the PRO-CRM (39.4% vs. 38%, respectively). The relative performance of the two methods, in terms of MTD selection and participant allocation, in Scenario 1 is similar for $N=40$. In Scenario 2, the methods again are similar in terms of percentage of correct MTD selection (62% vs. 62.8%, respectively), while the PRO-ISO treats a slightly higher percentage of participants at this dose (45% vs. 49.4%) for $N=18$. The percentage of correct MTD selection for the nonparametric optimal benchmark is 69.2%, meaning that the two methods achieve approximately 90% efficiency relative to the benchmark. These findings are consistent with those observed for $N=40$ in this scenario. In Scenario 3, both for $N=18$ and $N=40$, the PRO-CRM method identifies the true MTD (dose level 5) in a higher percentage of trials than the PRO-ISO method (47% vs. 43.6%, for $N=18;$ 71% vs. 56.8%, for $N=40$), while treating a higher percentage of participants on average at this dose (33% vs. 30.6%, for $N=18;$ 51% vs. 43%, for $N=40$). The relative efficiency of the two methods compared to the benchmark drops in this scenario to approximately 70–74% for $N=18$.

In Scenario 4, the methods again are similar in terms of percentage of correct MTD selection (46.8% vs. 45%), while the PRO-ISO treats a slightly higher percentage of participants at this dose (32.8% vs. 29%) for $N=18$. The nonparametric benchmark correctly identifies dose level 4 as the MTD in 59.4% of simulated trials, meaning that the two methods are approximately 7678% efficient relative to the optimal for $N=18$. For $N=40$, the PRO-CRM has better performance than PRO-ISO in terms of correctly identifying the MTD and assigning participants to the MTD. In Scenario 5, the PRO-CRM method displays better operating characteristics than the PRO-ISO method by recommending the true MTD (level 3) in a higher percentage of trials (55% vs. 50% for $N=18$) while treating a similar percentage of participants at this dose (36% vs. 36.7% for $N=18$). Again, for $N=40$, the PRO-CRM has better performance than PRO-ISO in terms of correctly identifying the MTD and assigning participants to the MTD. Similar findings as in Scenario 5 are reported for Scenarios 6 and 7. Across all scenarios, the average percentage of correct MTD selection is 54.3% for the PRO-CRM and 51.8% for the PRO-ISO for $N=18$. The average percentage of correct selection (PCS) of the MTD for the nonparametric benchmark is 61.7%, indicating that the PRO-CRM and PRO-ISO are 88% and 84% efficient in terms of PCS relative to the optimal. These results are consistent with those reported in Cheung (2011) for a single binary endpoint, which indicated that CRM is approximately 86% efficient in terms of PCS relative to the benchmark. The average percentage of patients treated at the true MTD is 37.1% for the PRO-CRM and 38.2% for the PRO-ISO for $N=18$. In Supplemental Material, we evaluated the performance of the PRO-CRM, the PRO-ISO method, and the nonparametric benchmark over a broader range of dose-toxicity curves for $N=18$. We randomly generated 75 dose-toxicity curves from the Clertant and O’Quigley (2017) family of curves. Since we did not compare safety stopping rules for each of the methods, we eliminated 6 curves in which the lowest dose level was overly toxic (either a C-DLT or P-DLT probability exceeding its respective target rate), leaving a total of 69 curves. The results based on 1000 simulated trials under each of the 69 curves, reported in Supplemental Figures S2 and S3, indicate that the mean percent of correct selection (PCS) of the MTD is 52.7% for the PRO-CRM, 56.8% for the PRO-ISO method, and 62.6% for the nonparametric benchmark. The mean number of patients treated at the MTD for the PRO-CRM is 7.55 and the mean number of patients treated at the MTD for the PRO-ISO is 8.01. For $N=40$, the average percentage of correct MTD selection is 73.1% for the PRO-CRM and 65.38% for the PRO-ISO. The average percentage of patients treated at the true MTD is 52.3% for the PRO-CRM and 50.3% for the PRO-ISO. Overall, Table 2 indicates that the PRO-ISO method is competitive with the PRO-CRM, making it a practical alternative for designing and conducting Phase I dose-finding trials that must account for PROs.

Finally, the results of TITE-PRO-ISO design are summarized in Tables 2 and 3. It is shown that the TITE-PRO-ISO design possesses comparable operating characteristics to the PRO-ISO design under a 3-month DLT evaluation window. In most scenarios, the PCS and percentage of patients treated at the MTD for the TITE-PRO-ISO is a few percentage points less than the PRO-CRM. These results are consistent with those in Cheung (2011), which showed that the PCS of the TITE-CRM declined as the patient accrual rate increased. Simultaneously, these results demonstrated an increase in toxicity and overdose occurrences. These observed trends raise the potential issue of aggressive escalations due to insufficient follow-up of individual patients. If a substantial number of patients exhibit no DLTs within a short follow-up period, their cumulative impact on the weighted likelihood may lead to premature dose escalation. Consequently, methods based on partially observed toxicity data may expose a greater number of patients to potentially toxic doses before any actual toxicity is observed. The obvious advantage of the TITE-PRO-ISO design, however, is that it significantly shortens the duration of trial while maintaining efficient estimation of the MTD. The average trial duration for the TITE-PRO-ISO across all scenarios was $\approx 12$ months for $N=18$ and $\approx 23$ months for $N=40$. Conversely, if we had to wait until each participant completed the 3-month DLT evaluation window, the trial duration would be $\approx 54$ months for $N=18$ and $\approx 120$ months for $N=40$.

3.2. Application to a Phase I trial in endometrial cancer

The motivation for this work is an ongoing Phase I study of adjuvant hypofractionated whole pelvis radiation therapy (WPRT) in endometrial cancer (NCT04458402). This trial, which opened for accrual at the University of Cincinnati Cancer Center in March 2021, is evaluating two hypofractionated WPRT regimens in gynecologic cancer delivered in 10 or 15 fractions. The study’s primary objective is to determine the maximum tolerated dose (MTD) per fraction, defined by acceptable acute clinician-reported gastrointestinal (GI) and genitourinary (GU) toxicity and patient-reported GI toxicity, of WPRT from two biologically equivalent study dose levels: 41.25 Gy in 15 fractions for dose level 1 and 38 Gy in 10 fractions for dose level 2. When condensing equivalent radiation fractionation schemes into less fractions, there has historically been a concern that this may result in more severe side effects so these doses are considered ordered with respect to their respective DLT probabilities. This application of PRO-CRM focuses on the marginal approach because, in addition to being easy to execute, clinical investigators for the study were comfortable specifying a target PRO-DLT rate to define the MTD. Acute GI and GU toxicity are being assessed according to both the CTCAE, version 5.0, and the PRO-CTCAE. A C-DLT is defined as an acute grade 3 or higher GI or GU per CTCAE and a P-DLT is defined by GI toxicity with a score of 4 or 5 on the 5-point scale per PRO-CTCAE, occurring within three months of completing WPRT. The MTD is defined as the minimum of the dose with a C-DLT rate closest to the clinician-reported target C-DLT rate of ${\varphi }_1=20\mathrm{\%}$ and the dose with a P-DLT rate closest to the patient-reported target P-DLT rate of ${\varphi }_2=55\mathrm{\%}$. The starting dose level will be dose level 1 (treatment course 41.25 Gy in 15 fractions). The DLT evaluation window is 3 months after completing WPRT.

We illustrate the behavior of the design described under a set of true C-DLT and P-DLT probabilities. The target C-DLT rate is ${\varphi }_1=20\mathrm{\%}$ and the target P-DLT rate is ${\varphi }_2=55\mathrm{\%}$. The set of true C-DLT probabilities are {0.05, 0.15} and the set of the true P-DLT probabilities are {0.18, 0.35}, indicating that both dose levels are safe. The first three eligible participants are administered dose level 1 (15 fractions), with 0 C-DLTs and 1 P-DLT observed. Based on this data, the C-DLT probability estimate is 0.15 and the P-DLT probability estimate is 0.46. These estimates are both below the target DLT rates for each endpoint, so the dose-finding algorithm suggessts that we try the next higher dose level and escalate to dose level 2. One participant in the second cohort experiences a C-DLT and P-DLT, and another experiences a C-DLT only. The resulting C-DLT and P-DLT estimate are {0.15, 0.28} and {0.46, 0.46}, respectively, indicating that the trial should return to dose level 1 after observing these DLTs. The third cohort receives dose level 1 and no DLTs of either type are observed, but the trial returns to dose level 2 based on C-DLT and P-DLT estimate of {0.12, 0.28} and {0.32, 0.46}, respectively. In the fourth cohort, no C-DLTs and no P-DLTs are observed. Based on this data, the C-DLT probability estimates are {0.12, 0.23} and the P-DLT probability estimates are {0.32, 0.32}. These estimates indicate that dose level 2 has estimated DLT rates closest to the respective target rates for each endpoint, so the trial remains at dose level 2. The trial remains at dose level 2 for the fifth and final cohort and two additional P-DLTs are observed in the last three participants. The final model-based C-DLT and P-DLT estimate are {0.12, 0.20} and {0.32, 0.40}, respectively. Overall, in this simulated trial, $N=15$ patients were treated, yielding a final MTD recommendation of dose level 2 (10 fractions).

4. Conclusions

The ability of more comprehensive instruments to capture DLT data directly from patients has increased interest in incorporating PROs into early-phase trials to better describe treatment tolerability (Basch et al., 2015; 2016; 2021). Numerous studies report a high rate of disagreement between clinician- and patient-reported symptoms, with patients reporting symptoms more frequently and with more severity (Fromme et al., 2004; Basch et al., 2006; Qin et al., 2013; Dimaio et al., 2015; Falchook et al., 2016). The PRO-CTCAE events are more sensitive to the patient experience relative to the NCI-CTCAE events. Failing to capture and utilize these data in early development can lead to treatments that may not be tolerable for patients being carried forward into middle and late development based only on clinician assessment. It is beneficial for a method to have the flexibility of assigning patients to lower, more tolerable doses if a high rate of PRO-DLTs are seen at higher doses. We must have patient-reported upper-bound criteria in the early evaluation of new therapies. If there are concerning signals for low tolerance to treatment from a patient’s perspective, it would be valuable to know early on before the therapy advances further.

This article has described the application of a novel adaptive strategy that accounts for clinician- and patient-reported DLTs in designing Phase I cancer clinical trials. Simulation studies were performed to justify and evaluate the performance of the design characteristics. The simulation results in Tables 2 and 3 demonstrate the method’s ability to effectively recommend desirable doses, defined by acceptable toxicity according to the NCI-CTCAE and PRO-CTCAE, in a high percentage of trials with manageable sample sizes. Although the performance of the PRO-CRM and the PRO-ISO method is similar, in general, each method has its own strengths and limitations, and researchers should have the flexibility to choose the approach that aligns best with their study context and objectives. Every method has some settings under which it will perform well and no method will outperform a competitor in every possible situation. Therefore, having a range of well-performing approaches is valuable. Software in the form of an R code for both simulation of design operating characteristics and direct protocol implementation of the proposed method is available at request of the first author.

The MTD paradigm in early oncology drug development is based on the assumption that the dose–response relationship of the drug follows a monotonic pattern. Accordingly, the MTD is deemed the dose with the best efficacy. This assumption may not hold for non-cytotoxic agents. For example, immunotherapies and targeted therapies may exhibit a plateau-shaped dose–response curve, indicating that once the dose reaches a certain level, any further increases in dosage will not result in greater efficacy. In such cases, lower doses may offer comparable efficacy to the MTD, while decreasing the risk of adverse events. To this end, early drug development in oncology is moving away from an “MTD-only” paradigm in response to the FDA’s Project Optimus initiative for dose optimization. Throughout the conduct of the study, it may be reasonable to allocate participants to doses other than the one determined to be the MTD. The challenges presented by non-cytotoxic agents do not necessitate the abandonment of the framework commonly used in dose-finding studies. On the contrary, specific components of this framework can be expanded or relaxed to effectively confront these challenges. The proposed methods in this article lay the groundwork for future extensions that aim to identify multiple recommended Phase II doses (RP2Ds) to carry forward based on clinician- and patient-reported safety outcomes by relying on randomization and controlled backfill strategies (Iasonos and O’Quigley, 2021; Dehbi, O’Quigley, Iasonos, 2021). Alternatively, if suitable a short-term efficacy endpoint is available, the proposed framework can be leaned upon to identify an acceptable set of safe doses that captures patient-reported tolerability, within which doses can be optimized based on efficacy outcomes (Wages and Tait, 2015).