A Utility-based Bayesian Optimal Interval (U-BOIN) Phase I/II Design to Identify the Optimal Biological Dose for Targeted and Immune Therapies

Summary

In the era of targeted therapy and immunotherapy, the objective of dose finding is often to identify the optimal biological dose (OBD), rather than the maximum tolerated dose (MTD). We develop a utility-based Bayesian optimal interval (U-BOIN) phase I/II design to find the OBD. We jointly model toxicity and efficacy using a multinomial-Dirichlet model, and employ a utility function to measure dose risk-benefit trade-off. The U-BOIN design consists of two seamlessly connected stages. In stage I, the Bayesian optimal interval (BOIN) design is used to quickly explore the dose space and collect preliminary toxicity and efficacy data. In stage II, in light of accumulating efficacy and toxicity from both stages I and II, we continuously update the posterior estimate of the utility for each dose after each cohort, and use this information to direct the dose assignment and selection. Compared to existing phase I/II designs, one prominent advantage of the U-BOIN design is its simplicity for implementation. Once the trial is designed, it can be easily applied using predetermined decision tables, without complex model fitting and estimation. Our simulation study shows that, despite its simplicity, the U-BOIN design is robust and has high accuracy to identify the OBD. We extend the design to accommodate delayed efficacy by leveraging the short-term endpoint (e.g., immune activity or other biological activity of targeted agents), and using it to predict the delayed efficacy outcome to facilitate real-time decision making. A user-friendly software to implement the U-BOIN is freely available at www.trialdesign.org.

1 |. INTRODUCTION

Immunotherapy and targeted therapies have revolutionized cancer treatment. Unlike conventional chemotherapy, immunotherapy drugs do not target the tumor directly. Instead, they work by reactivating the immune system and, hence, reestablishing its capacity to combat tumors. A major class of immunotherapy drugs are monoclonal antibodies, known as immune checkpoint inhibitors (ICI) (e.g., nivolumab, pembrolizumab, and ipilimumab). Checkpoint proteins are receptors on immune cells that can be activated to block immune response, for example checkpoint proteins on T cells (e.g., PD-1 and CTLA-4). The ICI bind to the checkpoint receptors on T cells and release “brake” such that T cells can kill cancer cells, achieving treatment efficacy. Nivolumab and pembrolizumab are PD-1 inhibitors, and ipilimumab is CTLA-4 inhibitor.

Traditional dose-finding designs developed for chemotherapy aim to find the maximum tolerated dose (MTD). The underlying assumption is that both toxicity and efficacy monotonically increase with the dose and, thus, the MTD presents the most efficacious dose that is safe. This assumption, however, is often questionable for immunotherapy and targeted agents. Although it is reasonable to assume that toxicity increases with the dose, the same is not necessarily true for efficacy. For example, once the checkpoint binding is saturated, further incrementing the ICI dose does not increase treatment efficacy. For some monoclonal antibodies, it is observed that higher doses actually lead to lower efficacy.¹ In addition, while increasing the dose of immunotherapy agents (or targeted biological agents) may improve efficacy, it still may cause substantial toxicity, due to over-activation of the immune system. When the overall benefit is limited, the value of increasing the dose becomes questionable.

To optimize the treatment benefit of immunotherapy and targeted therapy, therefore, it is important to consider toxicity and efficacy simultaneously and their risk-benefit trade-off during dose finding. The objective of dose finding for targeted therapy and immunotherapy is to identify the optimal biological dose (OBD), defined as the dose that has the highest desirability in terms of the risk-benefit trade-off. Our research is motivated by a phase I/II trial to identify the OBD of a novel humanized antiTROP2 monoclonal antibody in patients with an advanced solid tumor. Trophoblast cell-surface antigen 2 (TROP2) is associated with increased tumor growth. It is overexpressed in the majority of human epithelial cancers, including esophageal, breast, and lung cancers.^2,3 Five doses will be studied in the trial. Toxicity will be graded according to the NCI-CTCAE version 4.03, with a 21-day assessment window using the scale of 5 grades. The dose limiting toxicity (DLT) is defined as toxicity with grade 3 or higher. Tumor response will be evaluated using RECIST version 1.1, scored as complete remission (CR), partial remission (PR), stable disease (SD) and progressive disease (PD). Response is defined as CR/PR. TROP2 expression will be measured on day 8, after dosing, to provide a biomarker for measuring the biological activity of the drug.

Here, we develop a utility-based seamless phase I/II design to find the OBD. We jointly model toxicity and efficacy using a multinomial-Dirichlet model and employ a utility function to measure dose risk-benefit trade-off. We show that the utility approach is flexible and more general, in that it contains the existing marginal toxicity-efficacy trade-off methods as special cases. The design consists of two seamlessly connected stages. In stage I, the Bayesian optimal interval (BOIN) design⁴ is used to quickly explore the dose space and collect preliminary toxicity and efficacy data. In stage II, in light of accumulating efficacy and toxicity from both stages I and II, we continuously update the posterior estimate of the utility for each dose after each cohort, and use this information to direct the dose assignment and selection. We refer to the resulting design as the utility-based BOIN (U-BOIN) design. To accommodate the delayed efficacy observed in some targeted and immunotherapy trials, we extend the U-BOIN design and use the short-term endpoint (e.g., immune activity or other biological activity of targeted agents) to predict the delayed efficacy outcome to facilitate real-time decision making.

Numerous phase I/II trial designs have been proposed. Thall and Russell⁵ developed a Bayesian phase I/II design that characterizes patient outcomes using a trinary variable that accounts for both toxicity and efficacy. Braun⁶ generalized the continual reassessment method (CRM)⁷ to accommodate toxicity and efficacy simultaneously. Thall and Cook⁸ presented the EffTox design, based on the trade-offs between toxicity and efficacy. Yin et al.⁹ proposed a phase I/II design that uses the odds ratio of the efficacy and toxicity as a measure of desirability. Yuan and Yin¹⁰ described a phase I/II design that jointly models toxicity and efficacy as time-to-event outcomes. Jin et al.¹¹ proposed a phase I/II design that accommodates late-onset toxicity and efficacy. Liu and Johnson¹² proposed a robust Bayesian phase I/II design, based on a flexible Bayesian dynamic model. Guo and Yuan¹³ proposed a personalized Bayesian phase I/II design that accounts for patient characteristics and biomarker information. Zang et al.¹⁴ proposed several practical phase I/II trial designs to find the OBD. Liu et al.¹⁵ proposed a Bayesian phase I/II trial design for immunotherapy that considers the immune response, toxicity, and efficacy. Yuan et al.¹⁶ provided a comprehensive coverage of phase I/II designs.

Compared to existing phase I/II designs, the proposed U-BOIN design has several strengths. First, the U-BOIN is simple and easy to implement. It is a model-assisted design in that it uses a model for efficient decision making, but its decision rule for dose escalation/de-escalation can be tabulated and included in the trial protocol before the trial starts.^17,18,19 Once the trial is designed, it can be easily implemented using the predetermined decision table. No complicated computation or model estimation is needed. In contrast, most existing phase I/II designs are model-based designs, and require complicated model fitting and estimation after treating each cohort. Second, the U-BOIN is robust, because it models toxicity and efficacy at each dose independently, without imposing any parametric structure on dose-toxicity and dose-efficacy curves. In contrast, most existing designs assume that the dose-toxicity and dose-efficacy curves follow certain parametric forms (e.g., logistic model), and thus are susceptible to the influence of model misspecification.

Third, most existing phase I/II designs use the trade-off in the marginal toxicity and efficacy probabilities of a dose to measure the desirability of the dose, while the U-BOIN design uses the utility, which is not only easier to elicit from physicians, but also more flexible and general. We prove that the trade-off based on marginal toxicity and efficacy probabilities is a special case of the utility approach. Although utility has been used in previous designs,^20,21,22 to the best of our knowledge, this paper is the first to formally show that the utility approach contains the trade-off based on marginal toxicity and efficacy probabilities as a special case. This provides a theoretical justification for using the utility approach. Lastly, the U-BOIN is capable of accommodating delayed efficacy, whereas most existing designs assume that the efficacy endpoint is quickly observed.

The remainder of this article proceeds as follows. Section 2 introduces the statistical model, utility function, dose-finding algorithm, and software to implement the U-BOIN design. Section 3 uses simulation to compare the U-BOIN design with an existing phase I/II design, and also assesses the robustness of the design using sensitivity analysis. Section 4 provides a brief summary.

2 |. METHODS

2.1 |. Efficacy-toxicity model

Consider a phase I/II trial with J doses under investigation. Let Y_E = 0,⋯,R − 1 denote the categorical efficacy endpoint with R levels, where a higher level represents a more desirable treatment response; and Y_T = 0,⋯,Q − 1 denote the categorical toxicity endpoint with Q levels, where a higher level represents a more severe toxicity. The bivariate outcomes (Y_E, Y_T) can be equivalently represented by a single variable Y with K = R×Q levels, where each level of Y maps to a distinct value of (Y_E, Y_T).

As an example, consider the conventional setting that both Y_E and Y_T are binary. Let Y_E = 1 denote response, 0 otherwise. Similarly, let Y_T = 1 denote DLT, 0 otherwise. Then, Y has K = 4 levels, with Y = 1, if (Y_E, Y_T) = (0, 1); Y = 2, if (Y_E, Y_T) = (0, 0); Y = 3, if (Y_E, Y_T) = (1, 1); and Y = 4, if (Y_E, Y_T) = (1, 0). The value of Y ascribed to each possible (Y_E, Y_T) is not critical, as long as we keep track of the mapping. Without loss of generality, we assume that Y = 1 denotes the least favorable clinical outcomes, and Y = K denotes the most favorable clinical outcomes.

Define ${\pi }_{jk}=\mathrm{Pr}(Y=k\mid d=j),k=1,\cdots ,K$ and $j=1,\cdots ,J$, with ${\sum }_{k=1}^K{\pi }_{jk}=1$, where d denotes the dose level. We assume that Y follows a Dirichlet-multinomial model as follows:

$$Y=k\mid d=j\sim \text{Multinomial}\left({\pi }_{j1},\cdots ,{\pi }_{jK}\right).$$ (1)

$$\left({\pi }_{j1},\cdots ,{\pi }_{jK}\right)\sim \text{Dirichlet}\left(a_1,\cdots ,a_K\right),$$ (2)

where a₁,...,a_K > 0 are hyper-parameters. We set ${\sum }_{k=1}^K{a}_k=1$, such that the prior is vague and equivalent to a prior sample size of 1.

At an interim decision time, assume that n_j patients have been treated at dose d = j, among which n_jk patients had outcome Y = k, where $n_j={\sum }_{k=1}^K{n}_{jk}$. Given the observed interim data D_j = (n_j1,⋯,n_jk), the posterior distribution of ${\pi }_j=\left({\pi }_{j1},\cdots ,{\pi }_{jK}\right)$ is

$${\pi }_j\mid D_j\sim \text{Dirichlet}\left(a_1+n_{j1},\cdots ,a_K+n_{jK}\right).$$ (3)

2.2 |. Utility

We measure the desirability of the investigational doses using utility. Let ψ_k denote the utility ascribed to outcome Y = k, k = 1,⋯,K. The utility ψ_k should be elicited from physicians to reflect the risk-benefit trade-off underlying their medical decisions, which can be done using the following procedure:

1. Fix the value of the utility for the least desirable outcome Y = 1 as ψ₁ = 0, and for the most desirable outcome Y = K as ψ_K = 100. For example, for binary Y_E and Y_T, the least desirable outcome is (Y_E = 0, Y_T = 1), i.e., (no response, DLT), and the most desirable outcome is (Y_E = 1, Y_T = 0), i.e., (response, no DLT).

2. Ask the clinician to use these two utilities as a reference to score the utility values ψ₂,⋯,ψ_K−1 for the other K − 2 possible outcomes Y = 2,⋯,K − 1 to quantify the risk-benefit trade-off under each outcome.

Table 1 shows three examples of the utility function. The first two examples consider the scenario where both toxicity and efficacy are binary outcomes. Example 1 has utility values {ψ₁ = 0, ψ₂ = 30, ψ₃ = 50, ψ₄ = 100} for the outcomes {(Y_E = 0, Y_T = 1), (Y_E = 0, Y_T = 0), (Y_E = 1, Y_T = 1), (Y_E = 1, Y_T = 0)}. Compared to example 1, example 2 rewards the response (i.e., Y_E = 1) more, in the presence of DLT (i.e., Y_T = 1), by assigning a larger value to ψ₃ (65 versus 50). This is appropriate for the trial where toxicity can be well managed and response is highly desirable (e.g., leading to long survival). Example 3 shows the case where Y_T has three levels (i.e., minor, moderate, and severe toxicity), and Y_E has also three levels (i.e., PD, SD, and CR/PR).

TABLE 1.

Examples of utility

Note. PD: partial disease, SD: stable disease, PR: partial remission, CR: complete remission.

In our experience, clinicians quickly understand what the utilities mean and provide values for ψ_k’s, since the values reflect the actual clinical practice. After completing this process, simulation should be performed to verify the operating characteristics of the design. In some cases, the simulation results may motivate slight modification of some of the numerical utility values, although such modification typically has little or no effect on the design’s operating characteristics. One possible criticism for using the utility values is that they require subjective input. However, we are inclined to view this as a strength, rather than a weakness. The process of specifying the utility requires clinicians to carefully consider the potential risks and benefits of the treatment that underlie their clinical decision making in a more formal way and incorporate that into the trial. In addition, our simulation study and previous studies^13,15,21 show that the design is generally not sensitive to the numerical values of the utility, as long as it reflects a similar trend.

Given the values of ψ_k, the true mean utility for dose j is given by

$$U_j=\sum _{k=1}^K{\psi }_k{\pi }_{jk},$$ (4)

Since the true mean utility U_j depends on π_jk, which is unknown, we need to estimate it, based on the observed data. Given the interim data D = {D_j}, the estimate of mean utility is given by

$${\widehat{U}}_j=\sum _{k=1}^K{\psi }_{k}E\left({\pi }_{jk}\mid D\right).$$ (5)

For the conventional setting with binary Y_T an Y_E, another common approach to defining the efficacy-toxicity trade-off is directly based on the marginal efficacy probability π_E,j = Pr(Y_E = 1|d = j) and marginal toxicity probability π_T,j = Pr(Y_T = 1|d = j),¹⁶ which can be expressed as

$$U_j^M={\pi }_{E,j}-\omega {\pi }_{T,j},$$ (6)

where ω is a prespecified weight. This trade-off function says that patients are willing to trade an increase of ω in the DLT rate for a unit increase in the efficacy rate. If ω = 0, we obtain the special case that the dose with the highest efficacy is the most desirable. The following theorem shows that this marginal-probability-based approach is a special case of the utility approach described above. The proof is provided in the Appendix.

Theorem 1.

Marginal-probability-based trade-off $U_j^M$, defined in (6), is a special case of the utility method defined in (4), in the sense that for a pre-specified weight ω, we can find (ψ₂, ψ₃), such that $U_j=\xi U_j^M$, where ξ is a non-zero constant.

In the Supplementary Material, we provide an example to show how to map U_j with $U_j^M$ by choosing appropriate values of (ψ₂, ψ₃).

Liu and Johnson¹² proposed another marginal-probability-based trade-off function

$$U_j^{M2}={\pi }_{E,j}-{\omega }_1{\pi }_{T,j}-{\omega }_2{\pi }_{T,j}I\left({\pi }_{T,j}>\rho \right),$$ (7)

where ω₁ and ω₂ is a prespecified weight, I(·) is an indicator function, and ρ is a prespecified toxicity threshold deemed of substantial concern. Compared to $U_j^M$ in (6), this trade-off function is more flexible and allows to impose a higher penalty (i.e., ω₁ + ω₂) when the true DLT rate π_T,j exceeds the threshold ρ. $U_j^{M2}$ becomes $U_j^M$ when ω₂ = 0. It is not clear if trade-off function $U_j^{M2}$ can be expressed equivalently in the form of U_j as (4). Nevertheless, the proposed U-BOIN design can be used with different types of utility/trade-off, including $U_j^{M2}$. We plan to incorporate this functionality in our software provided later.

2.3 |. Optimal biological dose (OBD)

To define the OBD, we first define the admissible dose to safeguard patients from toxic or futile doses. As the objective here is to rule out toxic or futile doses, it is more natural to start with the definition of an inadmissible dose. Let ${\overline{\pi }}_T$ denote the maximum tolerable DLT rate, and ${\underset{\_}{\pi }}_E$ the lowest acceptable response rate. We define that dose j is inadmissible, if it meets either one or both of the following two criteria:

$$\text{(Toxic)}\mathrm{Pr}\left({\pi }_{T,j}>{\overline{\pi }}_T\mid D\right)>C_T,$$ (8)

$$\text{(Futile)}\mathrm{Pr}\left({\pi }_{E,j}<{\underset{\_}{\pi }}_E\mid D\right)>C_E,$$ (9)

where C_E and C_T are probability cutoffs. In general, C_T = 0.95 and C_E = 0.9 work well, but should be calibrated using simulation to ensure desirable operating characteristics. This can be done easily using the software provided.

The admissible dose is then defined as the dose for which none of the criteria (8) and (9) is satisfied. When Y_T has more than 2 categories (e.g., grade 0–2, grade 3, and grade 4–5), for the purpose of defining the admissible dose, Y_T can be temporarily collapsed into DLT/no DLT (i.e., DLT = grade 3 and higher). This dichotomization simplifies the definition of the admissible dose and is often adequate to safeguard patients from overly toxic doses. While it is not necessary, more complicated safety criteria could be entertained to accommodate multiple categories. Similar dichotomization (i.e., response/no response) is also applied to Y_E, when it has more than two categories. Note that although we dichotomize Y_T and Y_E for defining the admissible dose, our model and utility are still based on their original scales. We define the OBD as the dose that is admissible and has the highest utility value, i.e.,

$$\text{OBD}={\text{argmax}}_{j\in \mathcal{A}}\left(U_j\right),$$

where $\mathcal{A}$ denotes the set of admissible doses.

2.4 |. Phase I/II OBD finding algorithm

The U-BOIN design consists of two seamless, connected stages (Figure 1). The objective of stage I is to quickly explore the dose space to identify a set of admissible doses that are reasonably efficacious and safe for stage II. In stage I, we conduct dose escalation based on only the toxicity outcome, but efficacy data are also collected and will be used for decision making in stage II. Given the exploratory nature of stage I, if Y_T has more than two categories, we dichotomize it as DLT/no-DLT to facilitate the exploration of the dose space. This is in line with the clinical practice and serves well for the purpose of stage I. Note that for the estimation and finding the OBD (i.e., the primary objective of the trial), we retain the original scale of Y_T.

FIGURE 1.

Diagram of the U-BOIN design

Stage I dose escalation/de-escalation is guided by the BOIN design,⁴ which has been widely used in variety of oncology trials, including solid tumors,^23,24 liquid tumors,^25,26 and various treatment agents.^27,28,29,30 Due to very limited data and large uncertainty, for patient safety, we set the target DLT rate ${\varphi }_T={\overline{\pi }}_T-0.05$, slightly lower than the maximum tolerable DLT rate ${\overline{\pi }}_T$, to ensure that stage I dose exploration concentrates around up to, but not exceeding, ${\overline{\pi }}_T$. Let ${\widehat{\pi }}_{j,T}$ denote the empirical (or maximum likelihood) estimate of π_j,T, given by ${\widehat{\pi }}_{j,T}=m_j/n_j$, where $m_j$ is the number of patients who experienced DLT at the dose level j; and λ_e and λ_d denote the predetermined optimal escalation boundary and de-escalation boundary. Table 2 provides the values of λ_e and λ_d for the commonly used target DLT rate ϕ_T, see Liu and Yuan⁴ for the derivation and formula to calculate λ_e and λ_d. The dose-finding algorithm in stage I proceeds as follows.

TABLE 2.

Dose escalation and de-escalation boundaries of the BOIN design

Boundaries	Target DLT rate (ϕT)
	0.15	0.20	0.25	0.30	0.35	0.40
λe (escalation)	0.118	0.157	0.197	0.236	0.276	0.316
λd (de-escalation)	0.179	0.238	0.298	0.358	0.419	0.480

A1	Patients in the first cohort are treated at dose level 1 or a prespecified starting dose.
A2	Suppose j is the current dose, use the following rules to assign a dose to the next cohort of patients.

• Escalate the dose to j + 1 if ${\widehat{\pi }}_{j,T}\le {\lambda }_e$.

• De-escalate the dose to j − 1 if ${\widehat{\pi }}_{j,T}\ge {\lambda }_d$.

• Otherwise, stay at the current dose j.

A3	Repeat step A2 until the number of patients treated on one of the doses reaches s1, and then move to stage II. We recommend s1 = 12 as the default value, while s1 = 9 to 15 generally yields good operating characteristics.

Stage II proceeds as follows:

B1	Let j* denote the highest dose level that has been tried. If ${\widehat{\pi }}_{j^{\ast },T}\le {\lambda }_e$ and j* is not the highest dose in the trial, escalate the dose to dj*+1 for treating the next cohort of patients; otherwise, proceed to step B2.
B2	Given the observed interim data D collected in both stages I and II, determine the admissible dose set $\mathcal{A}$. If no dose is admissible, terminate the trial and no dose should be selected as the OBD. Otherwise, assign the next cohort of patients to the admissible dose (i.e., ∈ $\mathcal{A}$) that has the largest posterior mean utility, which can be pre-tabulated.
B3	Repeat steps B1 and B2 until reaching the prespecified maximum sample size N or the number of patients treated at one of the doses reaches s2 (> s1), and then select the OBD as the admissible dose (i.e., ∈ $\mathcal{A}$) that has the largest posterior mean utility. For most trials, a value between 18 to 24 is a reasonable choice for s2.

In stage I, following the BOIN design, we impose an overdose control rule as follows: if $\mathrm{Pr}\left({\pi }_{T,j}>{\overline{\pi }}_T\mid m_j,n_j\right)>0.95$ and n_j ≥ 3, dose level j and higher are eliminated from the trial; the trial is terminated if the lowest dose level is eliminated, where $\mathrm{Pr}\left({\pi }_{T,j}>{\overline{\pi }}_T\mid m_j,n_j\right)>0.95$ is evaluated based on a beta-binomial model with the uniform prior. Once the trial move to stage II, this overdose control rule is seamlessly merged as the inadmissible rule (8). In the overdose control rule, we use ${\overline{\pi }}_T$, rather than ϕ_T, as the DLT rate threshold to ensure that the overdose control rule seamlessly connects with the inadmissible rule.

For stage II step B1, the reason that we perform dose escalation when ${\widehat{\pi }}_{j^{\ast },T}\le {\lambda }_e$ is to allow the trial to continue exploring the dose space, given that the highest tried dose is safe, to reduce the risk of being stuck at a local suboptimal dose, due to a large variation caused by a small sample size. Besides the pick-the-winner (PW) approach (i.e., deterministically assigning the next cohort of patients to dose j ∈ $\mathcal{A}$ that has the largest posterior mean utility), other strategies can also be used to assign patients. For example, we can adaptively randomize the next cohort of patients to dose j ∈ $\mathcal{A}$, with probability ω_j proportional to its posterior mean utility, i.e.,

$${\omega }_j=\frac{U_j}{{\sum }_{j\in \mathcal{A}}{U}_j}.$$ (10)

The adaptive randomization (AR) approach can reduce the risk of being stuck at a suboptimal dose, but as a tradeoff, it tends to treat fewer patients at the OBD. Another approach is equal randomization (ER), where the next cohort of patients are assigned to the admissible doses with equal probability, i.e.,

$${\omega }_j=\frac{1}{{\sum }_{j\in \mathcal{A}}1}.$$ (11)

We compare the performance of PW, AR, and ER in our simulation study. None of the methods dominates the others. Thus, we generally recommend the PW approach because of its simplicity.

Unlike most existing phase I/II designs, which require complicated model fitting and estimation after each cohort to make the decision of dose assignment, one prominent advantage of the U-BOIN is that its dose assignment rules can be pre-tabulated in decision tables and included in the trial protocol before the trial starts. To conduct the trial, no complicated calculation is needed. The investigator can simply use the decision tables to determine dose assignment, e.g., determine whether escalation/de-escalation is needed (steps A2 and B1) or identify admissible doses (step B2). Because of this feature, the U-BOIN can be classified as a model-assisted design.^17,18,19 Section 1 in the Supplementary Material provides an example of using the decision tables in a hypothetical phase I/II trial.

2.5 |. Delayed efficacy response

In some trials, efficacy endpoint Y_E requires a long time to be ascertained. In our motivating example, it takes three months to evaluate Y_E. The long assessment window causes a major logistics issue for decision making in stage II. For example, given that the accrual rate is three patients per month, and that patients are treated in cohorts of three, on average, six new patients will be accrued while waiting to evaluate the previous three patients’ outcomes. This begs the question: how can new patients receive timely treatment, when the previous patients’ outcomes are not yet observed?

Statistically, this means that D are not fully observed, as some Y_E’s are unavailable. As a result, the mean utility estimate (5) and the inadmissible criterion (9) cannot be evaluated for making interim decisions. This issue is known as the late-onset or delayed-outcome issue, which has been studied in literature. Cheung and Chappell³¹ proposed a weighting method to handle late-onset toxicity for phase I clinical trials. Yuan and Yin¹⁰ modeled toxicity and efficacy as time-to-event outcomes that accommodate the unobserved as censored events. Liu et al.³² and Jin et al.¹¹ treated unobserved outcomes as missing data, and proposed a Bayesian data augmentation approach to predict unobserved toxicity and/or efficacy outcomes to facilitate decision making. Cai et al.³³ took the multiple imputation approach to handle unobserved efficacy outcomes in phase II trials.

We follow the approach of Cai et al.³³ and use multiple imputation to handle unobserved Y_E. The innovation here is that we use the measure of biological activity, which is routinely recorded in targeted and immunotherapy trials, as an ancillary variable to help predict (or impute) Y_E. Examples of the measure of biological activity include immune response (e.g., CD8+ T cell count) in immunotherapy trials and gene expression related to the pathway targeted by the treatment agent. The measure of biological activity is often quickly observable after drug administration and correlated with the clinical response. Daud et al.³⁴ showed that the abundance of CD8+ T cells predict response to anti-PD1 therapy, and advocated using the immune activity to predict the likelihood of achieving a clinical response to the PD-1 pathway inhibitor. For ease of exposition, hereafter we use the immune response, denoted by Y_I, as the example to illustrate our approach, but Y_I can be any reasonable biological activity measure predictive of treatment efficacy. We assume that Y_I is quickly observable, thus its value is always available at the time of decision making.

Given the observed value of Y_I, we predict the unobserved Y_E, based on the following scaled logistic regression model:

$$\text{logit}\left(\frac{{\pi }_{E,j}}{\lambda }\right)={\beta }_0+{\beta }_1{Y}_I,$$ (12)

where π_E,j is the probability of efficacy for dose j, β₀ and β₁ are regression parameters, and 0 < λ ≤ 1 is a plateau (or scale) parameter used to reflect that the probability of clinical response often levels out after the immune activity reaches a certain level. Under this model, the probability of efficacy π_E,j increases with Y_I, and then plateaus at the value of λ when Y_I is sufficiently large. When λ = 1, it becomes a standard logistic regression model. We do not include dose level j in the model, because the treatment effect of immunotherapy is mediated by the immune response, and thus it is often reasonable to assume that, conditional on Y_I, π_E,j is independent of j. If this assumption is not plausible in some situations, one can simply add j as a covariate to the model. In addition, more biomarkers, when available, can be added to model (12) as predictors to improve the prediction accuracy. One attractive property of our imputation approach is that, when the imputation model (12) is misspecified, its impact on the design diminishes over time and eventually goes away when the trial is completed. This is because, as the trial proceeds, more and more patients’ Y_E become observed, and accordingly fewer and fewer percentage of Y_E needs prediction. As a result, our method is generally robust to model misspecification, as shown later in the sensitivity analysis.

In terms of prior specifications in model (12), following Gelman et al.,³⁵ we assume that the model parameters (λ, β₀, β₁) are independent and have their own prior distributions. We specify a uniform distribution for the scale parameter $\lambda \sim \mathit{\text{uni}}f(0,1)$. The magnitude of the coefficients (β₀ and β₁) could be very large or small, depending on different trials. This makes it difficult to specify standard prior distributions for the coefficients. To tackle this problem, we first standardize the variable Y_I. In our application, Y_I is continuous; we standardize it to have a mean of 0 and a standard deviation of 0.5. We regulate the prior distributions to make sure that a typical change in a covariate should not lead to a dramatic change in the efficacy rate.

After standardizing data, a change of 2.5 on the logit scale can move the probability of a favorable response to the therapy from 0.2 to 0.75. We assume that the effect of the immune response is unlikely to be more dramatic than that. This typically is true for immunotherapy trials, as the efficacy rate is rarely outside of that range. We assign a normal distribution with a mean of 0 and a standard deviation 1.25 for β₀: β₀ ~ N(0, 1.25²). The parameter β₁ is supposed to be positive, due to the positive relationship between immune response and tumor response, and thus we assign a gamma distribution with a shape parameter of 1 and a rate parameter of 1.2: ${\beta }_1\sim Gamma\text{(}shape=1,rate=1.2)$. The priors ensure that a change in the covariate Y_I from one standard deviation below the mean to one standard deviation above the mean will lead to an absolute change that is mostly less than 2.5 on the logit scale.

Let f(β₀, β₁, λ) denote the joint prior distribution of (β₀, β₁, λ). The posterior distribution of (β₀, β₁, λ) is given by

$$f\left({\beta }_0,{\beta }_1,\lambda \mid D\right)\propto f\left({\beta }_0,{\beta }_1,\lambda \right)\prod _{j=1}^J\prod _{i=1}^{n_j}{\left(\frac{\lambda e^{{\beta }_0+{\beta }_1{y}_{I,ji}}}{1+e^{{\beta }_0+{\beta }_1{y}_{I,ji}}}\right)}^{y_{E,ji}}{\left(\frac{1+(1-\lambda )e^{{\beta }_0+{\beta }_1{y}_{I,ji}}}{1+e^{{\beta }_0+{\beta }_1{y}_{I,ji}}}\right)}^{1-y_{E,ji}},$$ (13)

where $y_{I,ji}$ is the immune response for the ith patient treated with dose j and y_E,ji is the tumor response for this patient.

At an interim decision time, let $Y_{E,obs}$ and $Y_{E,\mathit{\text{mis}}}$ denote the observed and missing parts of Y_E. We impute the value of Y_E,mis using multiple imputations as follows.

1. Conditional on observed data $\left(Y_{E,obs},Y_I\right)$, sample L draws of (λ, β₀, β₁) from their posterior distribution (13) using the adaptive rejection Metropolis sampling.³⁶ A certain number of burn-in iterations are typically needed before collecting the posterior draws. In our simulation, we set the number of burn-in iterations as L′ = L/2.

2. Thin the posterior draws by taking a sample after every L′/H draw, resulting in a total of H sets of posterior draws of (λ, β₀, β₁), denoted as $\left({\lambda }^{(1)},{\beta }_0^{(1)},{\beta }_1^{(1)}\right),\cdots ,\left({\lambda }^{(H)},{\beta }_0^{(H)},{\beta }_1^{(H)}\right)$.

3. Impute Y_E,mis for H times, where the hth imputed value is generated by drawing a random sample from Bernoulli(q), where h = 1,…,H and $q=\lambda e^{{\beta }_0+{\beta }_1{Y}_I}/\left(1+e^{{\beta }_0+{\beta }_1{Y}_I}\right)$. We denoted the H sets of imputed values as $Y_{E,imp}^{(1)},\cdots ,Y_{E,imp}^{(H)}$.

After filling in Y_E,mis with each of the H imputed values, we obtained an H imputed complete dataset, D⁽¹⁾,⋯,D^(H), where D^(h) is obtained by filling in Y_E,mis with $Y_{E,imp}^{(h)}$, h = 1,⋯, H. Then, the estimate of mean utility (5) is given by

$${\widehat{U}}_j=\frac{1}{H}\sum _{h=1}^H\sum _{k=1}^4{\psi }_{k}E\left({\pi }_{jk}\mid D^{(h)}\right),$$ (14)

and the left side of the inadmissible criterion (9) is calculated as

$$\mathrm{Pr}\left({\pi }_{E,j}<{\underset{\_}{\pi }}_E\mid D\right)=\frac{1}{H}\sum _{h=1}^H\mathrm{Pr}\left({\pi }_{E,j}<{\underset{\_}{\pi }}_E\mid D^{(h)}\right).$$ (15)

The dose finding follows the same algorithm described in section 2.4. Little and Rubin³⁷ suggested that, for practical use, the number of multiple imputation (H) can be sufficient when H = 5 or greater. In our simulation, we perform H = 20 imputations.

To further improve the efficiency of the design, Y_I can also be used to refine the admissible rule. The rationale is that when Y_E takes a long time to be observed, the inadmissible criterion (9) is not effective for screening out ineffective doses, because a high percentage of Y_E may not be observed at the time of making interim decisions. Since Y_I is quickly observable, it can be used, supplementary to Y_E, to improve the power of identifying effective doses or safeguard patients from ineffective doses. Specifically, define ${\mu }_{I,j}=E\left(Y_I\mid d=j\right)$, and let ${\underset{\_}{\mu }}_I$ denote the lowest acceptable mean immune response. We define dose j inadmissible if it meets at least one of the toxicity and futility criteria in (8) and (9), and

$$\text{(Insufficient activity)}\mathrm{Pr}\left({\mu }_{I,j}<{\underset{\_}{\mu }}_I\mid D\right)>C_I,$$ (16)

where C_I is a pre-specified probability threshold. In other words, if a dose has little activity to activate immune system, it is deemed unpromising and inadmissible. Again, the admissible dose is defined as the dose that is not inadmissible. The posterior probability $\mathrm{Pr}\left({\mu }_{I,j}<{\underset{\_}{\mu }}_I\mid D\right)$ can be evaluated based on the Bayesian normal model:

$$\begin{gathered} Y_I\mid d=j\sim N\left({\mu }_{I,j},{\sigma }_j^2\right) \\ f\left({\mu }_{I,j},{\sigma }_j^2\right)\propto {\sigma }_j^{-2}. \end{gathered}$$ (17)

2.6 |. Software and trial implementation

To facilitate the use of the U-BOIN design, we develop an easy-to-use web app, which is freely available at http://www.trialdesign.org. Figure 2 shows the graphical user interface of the app. Extensive help files are available by clicking on the yellow question mark at the upper right corner of each tab.

FIGURE 2.

The user interface of the U-BOIN software

To design a trial, follow the steps:

• Specify the design parameters (e.g., dose information, sample size, cohort size, utility function, admissible criteria, etc.), under the Trial Setting tab. After parameters are entered, design diagram and decisions tables will be provided under the tab.

• Generate the operating characteristics of the design by supplying different scenarios (under the Simulation tab).

• Download trial protocol under Trial Protocol tab. The protocol includes (1) a brief description of the U-BOIN design, (2) decision tables based on input under Trial Setting, and (3) operating characteristics for the scenarios entered under Simulation.

To conduct a trial, use the decision tables generated under Trial Setting or use the Trial Conduct tab. The app will return the dose for treating the next cohort of patients if the data provided indicates that the trial is not completed, and it will select the OBD if the trial is completed.

3 |. NUMERICAL STUDY

3.1 |. Simulation A

Simulation A considers the case that Y_T and Y_E are quickly ascertainable. Following our motivating trial, we consider J = 5 doses, and the total sample size N = 54 patients with s₁ = 12. The minimum acceptable efficacy rate is ${\underset{\_}{\pi }}_E$ = 0.2, and the maximum acceptable DLT rate is ${\overline{\pi }}_T$ = 0.30. We set ${\pi }_T^{\ast }=0.25$ by using δ = 0.05. For the inadmissible criteria (i.e., equations (8) and (9)), probability cutoffs are set as C_T = 0.95 and C_E = 0.9, based on simulation calibration. The elicited utility is presented in Table 1 as example 1.

We consider 8 representative scenarios that differ in the shape of the dose-toxicity and dose-efficacy curves, and the location of the OBD. The scenarios are shown in Figure 3. In scenarios 1 and 2, the dose-response curve increases with the dose level, and then plateaus. In scenarios 3 and 4, the dose-response curve increases to an optimal point, and then decreases. In scenario 5, the dose-response curve levels off at the first dose level. In scenarios 6, 7, and 8, all dose-response curves monotonically increase. Scenario 6 has the last dose level being the OBD. Scenario 7 has two OBDs, located at dose level 2 and 3. Scenario 8 has no OBD since the doses are either futile or overly toxic. Under each scenario, we generate (Y_T, Y_E), based on a Gumbel model:

$$\begin{gathered} \mathrm{Pr}\left(Y_T=y_T,Y_E=y_E\mid d=j\right)={\left({\pi }_{E,j}\right)}^{y_E}{\left(1-{\pi }_{E,j}\right)}^{1-y_E}{\left({\pi }_{T,j}\right)}^{y_T}{\left(1-{\pi }_{T,j}\right)}^{1-y_T} \\ +{\pi }_{E,j}\left(1-{\pi }_{E,j}\right){\pi }_{T,j}\left(1-{\pi }_{T,j}\right){(-1)}^{y_E+y_T}\left(\frac{e^c-1}{e^c+1}\right), \end{gathered}$$ (18)

where y_E, y_T ∈ {0,1}, and the association parameter c was set as 0.2. The values of π_E,j and π_T,j (i.e., the marginal efficacy rate and DLT rate) for each dose level are provided in Table 3. We compare the proposed U-BOIN design with the EffTox design.⁸ We consider the two most important metrics for comparison: (1) percentage of correct selection (PCS), which is the probability of correctly identifying the OBD, and (2) patient allocation, which referred to the average number of patients assigned to each dose. In the EffTox design, the toxicity-efficacy trade-off is constructed based on three equally desirable trade-off target probabilities: (π_E = 0.2, π_T = 0), (π_E = 1, π_T = 0.7), and (π_E = 0.45, π_T = 0.3). Under each scenario, we performed 2,000 simulations.

FIGURE 3.

Simulation scenarios. The dash-dotted line (blue) is the dose-toxicity curve, the solid line (red) is the dose-efficacy curve, and the dashed line (black) is the dose-immune response curve. The OBD is highlighted by red asterisk in the x-axis. Simulation A considers only the efficacy and toxicity curves, while simulation B considers efficacy, toxicity, and immune response.

TABLE 3.

Results of Simulation A, including the selection percentage (selection %), the average number of patients treated at each dose (No. of patients), and the percentage of early stopping. The optimal biological dose (OBD) is bolded. In scenario 8, the OBD does not exist, and thus the percentage of early stopping is bolded.

Design		Dose Level					% of early stopping
		1	2	3	4	5
		Scenario 1
	DLT rate	0.02	0.15	0.30	0.45	0.60
	Efficacy rate	0.20	0.65	0.65	0.65	0.65
	Utility	43.0	69.0	63.0	56.0	50.0
EffTox	Selection %	2.0	50.0	45.0	2.0	0.0	0.0
	No. of patients	4.3	22.7	23.8	2.8	0.4
U-BOIN	Selection %	1.7	72.9	22.4	2.8	0.0	0.2
	No. of patients	6.2	29.9	13.8	3.5	0.5
		Scenario 2
	DLT rate	0.03	0.08	0.15	0.28	0.40
	Efficacy rate	0.10	0.22	0.60	0.60	0.60
	Utility	36.0	43.0	66.0	60.0	55.0
EffTox	Selection %	0.0	4.0	60.0	29.0	7.0	0.0
	No. of patients	3.4	4.9	26.4	14.2	5.1
U-BOIN	Selection %	1.1	3.2	65.7	24.9	4.3	0.8
	No. of patients	4.9	7.5	24.4	12.7	4.3
		Scenario 3
	DLT rate	0.05	0.15	0.30	0.45	0.60
	Efficacy rate	0.08	0.46	0.25	0.20	0.10
	Utility	34.0	56.0	37.0	29.0	18.0
EffTox	Selection %	12.0	70.0	10.0	1.0	0.0	7.0
	No. of patients	10.8	25.5	11.5	2.9	1.0
U-BOIN	Selection %	1.2	92.2	4.0	0.4	0.0	2.1
	No. of patients	6.6	34.7	8.9	2.6	0.4
		Scenario 4
	DLT rate	0.15	0.25	0.40	0.45	0.50
	Efficacy rate	0.15	0.45	0.30	0.25	0.20
	Utility	36.0	52.0	36.0	32.0	27.0
EffTox	Selection %	36.0	47.0	5.0	2.0	2.0	9.0
	No. of patients	19.4	19.0	7.5	2.6	2.3
U-BOIN	Selection %	11.9	74.1	3.7	0.4	0.0	9.9
	No. of patients	15.0	29.5	5.1	1.1	0.2
		Scenario 5
	DLT rate	0.10	0.30	0.50	0.55	0.65
	Efficacy rate	0.45	0.45	0.45	0.45	0.45
	Utility	58.0	50.0	42.0	40.0	36.0
EffTox	Selection %	69.0	27.0	2.0	0.0	0.0	2.0
	No. of patients	29.7	17.3	4.9	1.0	0.3
U-BOIN	Selection %	75.4	22.8	1.5	0.2	0.0	0.2
	No. of patients	33.1	16.5	3.7	0.5	0.1
		Scenario 6
	DLT rate	0.05	0.07	0.10	0.12	0.16
	Efficacy rate	0.35	0.45	0.50	0.55	0.75
	Utility	53.0	59.0	61.0	64.0	75.0
EffTox	Selection %	10.0	12.0	26.0	24.0	29.0	0.0
	No. of patients	8.6	7.9	14.2	11.1	12.1
U-BOIN	Selection %	5.9	11.7	13.1	13.6	55.7	0.0
	No. of patients	7.0	8.8	9.0	9.1	20.1
		Scenario 7
	DLT rate	0.03	0.16	0.27	0.45	0.55
	Efficacy rate	0.15	0.38	0.45	0.60	0.70
	Utility	40.0	51.0	51.0	53.0	55.0
EffTox	Selection %	1.0	33.0	54.0	9.0	1.0	2.0
	No. of patients	4.2	13.8	25.9	7.0	2.3
U-BOIN	Selection %	2.0	45.0	41.0	10.0	1.0	1.0
	No. of patients	5.1	20.4	20.0	6.7	1.2
		Scenario 8
	DLT rate	0.22	0.45	0.55	0.65	0.70
	Efficacy rate	0.03	0.10	0.20	0.35	0.40
	Utility	25.0	23.0	25.0	30.0	31.0
EffTox	Selection %	1.0	6.0	6.0	0.0	0.0	87.0
	No. of patients	4.6	6.2	6.3	2.3	1.2
U-BOIN	Selection %	0.8	5.5	1.7	0.0	0.0	92.0
	No. of patients	14.3	9.7	1.6	0.1	0.0

Table 3 summarizes the operating characteristics for the designs. The U-BOIN design outperforms EffTox as it has a larger PCS and allocates more patients on the OBD. In scenarios 1 and 2, the dose-response curve increases first, and then plateaus, and the OBDs (dose level 2 and 3, respectively) are one dose level lower than the MTD (i.e., dose level with DLT rate closest to the target DLT rate). The U-BOIN has a 23% higher PCS and assigns seven more patients on the OBD than EffTox does in scenario 1. Similarly, the U-BOIN has a larger PCS in scenario 2, for which the two designs have comparable patient allocation. In scenarios 3 and 4 where the dose-response curve increases to an optimal point and then decreases, U-BOIN has 22% and 27% higher PCS than EffTox, respectively. Moreover, U-BOIN assigns nine and ten more patients on the OBD, respectively in the two scenarios. In scenario 5 where the OBD is located on the first dose, and the response rate does not change with dose levels, U-BOIN has a 6% higher PCS and assigns three more patients on the OBD. In scenario 6, where all doses are safe and the OBD is the last dose level, the U-BOIN has a 27% higher PCS and assigns eight more patients on the OBD. This is because EffTox cannot distinguish well the suboptimal doses from the OBD in this scenario, subsequently assigning more patients on dose level 3 and 4 and selecting one of the dose levels as the OBD 55% of the time. U-BOIN has comparable performance to EffTox when there are two OBDs (scenario 7), in terms of PCS and patient allocation. When there is no OBD due to toxicity or futility (scenario 8), U-BOIN has a larger chance to stop the trial early.

3.2 |. Simulation B

Simulation B considers the case that Y_T is quickly ascertainable, but Y_E takes a long time to be scored with the assessment window of 3 months. We assume that patient accrual follows a Poisson process, with the rate of 3 patients per month. We simulated Y_T and Y_E based on the same Gumbel model and 8 scenarios, as described in Simulation A. The reason we chose the same simulation scenarios is that, by doing so, the results from Simulation A (i.e., Y_E is always observed) can be used as a benchmark to evaluate the performance of the design in Simulation B (i.e., Y_E is partially observed, due to delayed response). To generate a delayed response, for patients who experience efficacy in the assessment window (i.e., Y_E = 1), we simulate their time to efficacy from a truncated Weibull distribution with a support of (0, 3) months. The shape and scale parameter for the Weibull distribution are chosen such that the efficacy rate at the end of assessment time matches those in Table 3, and that 90% of the responses occur in the latter half of the assessment window (i.e., (1.5, 3) months). The immune response Y_I is generated from N(μ_I,j, 1), where μ_I,j is plotted in Figure 3. We set C_I = 0.9. Under each scenario, 2,000 simulations are performed.

Table 4 shows the simulation results. The PCS of the OBD and the number of patients allocated to the OBD are generally comparable to the results in Simulation A (i.e., the optimal benchmark with fully observed data) and U-BOIN still outperforms the EffTox design. The results indicate that U-BOIN efficiently handles the delayed efficacy response. Because the U-BOIN design does not need to suspend accrual to wait Y_E to be fully observed and allows real-time decision making, it has great potential to shorten the trial duration.

TABLE 4.

Results of Simulation B, including the selection percentage (selection %), the average number of patients treated at each dose (No. of patients), the percentage of early stopping, and the trial duration. The optimal biological dose (OBD) is bolded. In scenario 8, the OBD does not exist, and thus the percentage of early stopping is boldeded.

Design		Dose Level					% of early stopping	Duration (month)
		1	2	3	4	5
		Scenario 1
EffTox	Selection %	2.0	50.0	45.0	2.0	0.0	0.0	45.9
	No. of patients	4.3	22.7	23.8	2.8	0.4
U-BOIN	Selection %	0.9	68.9	24.1	2.9	0.3	2.8	20.8
	No. of patients	6.4	29.2	13.7	3.6	0.5
		Scenario 2
EffTox	Selection %	0.0	4.0	60.0	29.0	7.0	0.0	43.8
	No. of patients	3.4	4.9	26.4	14.2	5.1
U-BOIN	Selection %	0.0	0.2	64.1	22.4	4.9	8.3	20.0
	No. of patients	4.3	6.1	24.5	12.1	4.1
		Scenario 3
EffTox	Selection %	12.0	70.0	10.0	1.0	0.0	7.0	45.6
	No. of patients	10.8	25.5	11.5	2.9	1.0
U-BOIN	Selection %	0.8	89.6	6.0	0.8	0.0	2.8	20.7
	No. of patients	6.7	33.6	9.5	2.9	0.4	2.8
		Scenario 4
EffTox	Selection %	36.0	47.0	5.0	2.0	2.0	9.0	47.8
	No. of patients	19.4	19.0	7.5	2.6	2.3
U-BOIN	Selection %	6.8	70.4	3.4	0.4	0.0	19.1	18.8
	No. of patients	13.6	27.9	4.7	1.0	0.2	19.1
		Scenario 5
EffTox	Selection %	69.0	27.0	2.0	0.0	0.0	2.0	50.0
	No. of patients	29.7	17.3	4.9	1.0	0.3
U-BOIN	Selection %	72.9	22.7	2.2	0.2	0.0	2.0	20.7
	No. of patients	32.9	16.4	3.7	0.5	0.0	2.0
		Scenario 6
EffTox	Selection %	10.0	12.0	26.0	24.0	29.0	0.0	43.4
	No. of patients	8.6	7.9	14.2	11.1	12.1
U-BOIN	Selection %	4.2	10.3	12.2	13.6	59.2	0.4	21.0
	No. of patients	6.5	8.6	9.5	9.9	19.4	0.4
		Scenario 7
EffTox	Selection %	33.0	45.0	45.0	43.0	40.0		46.5
	No. of patients	1.0	33.0	54.0	9.0	1.0	2.0
U-BOIN	Selection %	5.0	45.6	33.1	10.5	1.7	4.1	20.7
	No. of patients	8.5	22.7	15.6	5.2	0.8	4.1
		Scenario 8
EffTox	Selection %	1.0	6.0	6.0	0.0	0.0	87.00	16.6
	No. of patients	4.7	6.1	6.3	2.4	1.3
U-BOIN	Selection %	0.0	0.4	0.4	0.0	0.0	99.10	8.8

To examine the performance of U-BOIN when there are delayed outcomes at a relatively smaller sample size, we conduct the simulation again using N = 39. Results show that U-BOIN still maintains its great operating characteristics, even when sample size is relatively small. The simulation results are provided in Table S8 of the Supplementary Material. We also provide operating characteristics for eight additional representative scenarios (Scenarios A1-A8) for both Simulation A (Table S9) and Simulation B (Table S10). The results, again, show that U-BOIN has robust performance in various scenarios in comparison to EffTox.

3.3 |. Sensitivity analysis

We conduct sensitivity analyses to assess the robustness of the U-BOIN design by using a different set of utility values Ψ′ = {ψ₁ = 0, ψ₂ = 20, ψ₃ = 55, ψ₄ = 100}, which assigns a lower score for (Y_E = 0, Y_T = 1) and a higher score for (Y_E = 1, Y_T = 1), indicating that patients are willing to tolerate a higher toxicity risk in order to attain a higher efficacy. We also evaluate the performance of U-BOIN using different patient allocation strategies (i.e., step B2 of Stage II of the dose finding algorithm): PW, AR, and ER.

The simulation results (see Figure 4) show that the U-BOIN design performs well with comparable PCS and patient allocations for the two different utilities. This characteristic is important, since two clinicians might have slightly different opinions about the specific utility value assigned to an outcome combination. Our design shows robustness for this type of deviation. Figure 5 shows the results under different patient allocation strategies. The PCS is comparable among the PW, AR, and ER strategies. The difference among the three strategies mainly lies in the number of patients treated on OBD. The PW approach, on average, assigns significantly more patients on the OBD than both the ER and AR approaches, while it has larger variability. Since the variation is more towards the higher end (i.e., more patients are treated on OBD), the PW approach is desirable in this simulation study.

FIGURE 4.

Results of sensitivity analysis for different utilities. Scenario 8 is not included, as the OBD does not exist in that scenario.

FIGURE 5.

Results of sensitivity analysis for different patient allocation strategies: pick-the-winner (PW), adaptive randomization (AR), and equal randomization (ER). Scenario 8 is not included, as the OBD does not exist in that scenario.

Note that in this Simulation A, we specify s₂ = N for fair comparison with the EffTox design, which does not stop the trial early on the basis of the number of patients treated on a dose. A sensitivity analysis using s₂ = {18, 21} is provided in Section 5 of the Supplementary Material. The result in Figure S1 shows that using the recommended values, the change in the percentage of correct selection is negligible, but the saving in sample size is substantial.

For the case with delayed Y_E, we performed three additional sensitivity analyses by (1) assuming that the efficacy assessment window is two months or four months; (2) specifying more vague prior distributions for the coefficients β₀ and β₁ with β₀ ~ N(0, 3.75²) and β₁ ~ Gamma(shape = 1, rate = 0.4), both of which have standard deviations that are three times the previous values in section 2.5; and (3) considering two additional scenarios (scenarios B1 and B2 in Table S11), where Y_I and Y_E are weakly associated with Pearson’s correlation coefficients −0.15 and 0.2, respectively. The results show that the U-BOIN design is also robust to the length of assessment window, prior distribution of the prediction model (Figure S2), and the association between Y_I and Y_E.

4 |. SUMMARY

We propose the U-BOIN, a seamless phase I/II model-assisted design, to identify the OBD for targeted and immunotherapy trials. The U-BOIN design accounts for the efficacy-toxicity trade-off using a utility function. Unlike most existing phase I/II designs, which require complicated real-time model fitting and estimation to make dose assignment decisions, the U-BOIN is simple and easy to implement. The dose assignment rules of the U-BOIN can be pre-tabulated in decision tables and included in the trial protocol before onset of the trial. To conduct the trial, no complicated calculation is needed. The investigator can simply use the decision tables to make the decision of dose escalation/de-escalation. Simulation studies show that compared to a more complicated model-based phase I/II design, the U-BOIN has higher accuracy to identify OBD and is more robust. To facilitate the use of the U-BOIN design in clinical trials, we develop a user-friendly software freely available at www.trialdesign.org.

While this article focuses on immunotherapy and targeted trials, U-BOIN also can be used for conventional cytotoxic agent trials. In such cases, both toxicity and efficacy typically increase with the dose, but may do so at different rates. It is likely that increasing the dose causes much higher toxicity, with limited efficacy gain. The idea of risk-benefit tradeoff, and thus finding the optimal dose, is still generally applicable here for use in most medical decisions in practice.

As most model-assisted designs, U-BOIN models efficacy and toxicity at each dose independently, whereas model-based phase I-II designs (e.g., EffTox design) model efficacy and toxicity across all doses, through imposing a parametric dose-efficacy and -toxicity curve model. As a result, one may worry about the potential efficiency loss for U-BOIN. Our numerical study and previous studies show that, for the purpose of dose finding, the efficiency loss caused by using only local data (in model-assisted designs, such as U-BOIN) is minimal or negligible. This can be explained as follows. First, although U-BOIN models only the local data at the current dose, its dose-finding algorithm (e.g., escalate the dose if the current dose is safe, and de-escalate the dose if the current dose is toxic) implicitly uses the dose-toxicity order information across the doses. In addition, in practice, the parametric dose-efficacy and -toxicity model assumed by model-based designs is more likely to be misspecified than correctly specified. Thus, on average, borrowing information across doses through the parametric model leads to rather limited efficiency gain. Furthermore, such limited efficiency gain does not necessarily translate into performance gain. This is because, to make correct decisions of dose assignment and selection, we only need to correctly estimate the rank of utility across the doses. A slightly more variability on the estimate of the utility has no or negligible impact on the performance of the design.

APPENDIX

A PROOF

Theorem 1.

Marginal-probability-based trade-off $U_j^M$, defined in (6), is a special case of the utility method defined in (4), in the sense that for a pre-specified weight ω, we can find (ψ₂, ψ₃), such that $U_j=\xi U_j^M$, where ξ is a non-zero constant.

Proof

If there is a constant ξ, such that $U_j^M=\xi {\tilde{U}}_j$, then, ${\tilde{U}}_j$ is a special case of $U_j^M$.

To fix scale for utility, we propose in this paper that ψ₁ = 0,ψ₄ = 100. Then $U_j^M$ is a function of only ψ₂ and ψ₃.

$$U_j^M={\psi }_2{\pi }_{j2}+{\psi }_3{\pi }_{j3}+100{\pi }_{j4}$$

It is obvious that $U_j^M>0$ by definition. Denote ${\tilde{U}}_j(\omega )={\tilde{U}}_j={\pi }_{E,j}-\omega {\pi }_{T,j}$. If $U_j^M=\xi {\tilde{U}}_j$, then $\xi \ne 0,{\tilde{U}}_j(\omega )\ne 0$, and ${\psi }_2{\pi }_{j2}+{\psi }_3{\pi }_{j3}+100{\pi }_{j4}=\xi {\tilde{U}}_j(\omega )$. Since $0<{\psi }_2{\pi }_{j2}+{\psi }_3{\pi }_{j3}<100\left({\pi }_{j2}+{\pi }_{j3}\right)$, we have $\frac{100{\pi }_{j4}}{{\tilde{U}}_j(\omega )}<\xi <\frac{100\left(1-{\pi }_{j1}\right)}{{\tilde{U}}_j(\omega )}$, which is nonzero constant.

In summary, for a pre-specified ω and ${\tilde{U}}_j(\omega )\ne 0$, we can find (ψ₂, ψ₃) and a non-zero constant ξ, such that $U_j^M=\xi {\tilde{U}}_j$, which says that ${\tilde{U}}_j$ is a special case of U_j.