Bias induced by adaptive dose-finding designs

Abstract

There is a long literature on bias in maximum likelihood estimators. Here we demonstrate that adaptive dose-finding procedures (such as Continual Reassessment Methods, Up-and-Down and Interval Designs) themselves induce bias. In particular, with Bernoulli responses and dose assignments that depend on prior responses, we provide an explicit formula for the bias of observed response rates. We illustrate the patterns of bias for designs that aim to concentrate dose allocations around a target dose, which represents a specific quantile of a cumulative response-threshold distribution. For such designs, bias tends to be positive above the target dose and negative below it. To our knowledge, this property of dose-finding designs has not previously been recognized by design developers. We discuss the implications of this bias and suggest a simple shrinkage mitigation formula that improves estimation at doses away from the target.

1. Introduction

Bias in maximum likelihood estimators is well known, but in regular exponential family models this bias goes to zero with increasing sample size (see, e.g. Firth [11]). With regard to adaptive sampling procedures, Coad and Ivanova [6] found bias and variance expressions for maximum likelihood estimators (MLEs) for five popular urn models building on prior work by Woodroofe and Coad [5,7,36]. The bias and mean squared error (MSE) of MLEs following sample size re-calculation has been explored by Liu et al. [23], Brannath [3], Graf et al. [15] and Tarima and Flournoy [31]. Bias in group sequential designs is discussed by Jennison and Turnbull [19]. Bias induced by multi-arm bandit selection procedures was studied by Xu et al. [38].

But bias in dose-finding experiments has here-to-fore been overlooked. Dose-finding designs (such as Continual Reassessment Methods [4,25], Up-and-Down [10,12,17,28] and Interval Designs [18,21]) are routine in many fields in which subjects arrive sequentially or in batches, including Phase I cancer trials, sensory studies, acute toxicity and psychometric testing. Adaptation in these designs refers to the use of prior outcome information to determine the stimulus or treatment X for the current subject or cohort.

Even though the explicit bias formula derived in Section 2 applies to a much broader family of models and goals, for simplicity of exposition, we restrict this article to the case of univariate treatment X and binary response Y. Without loss of generality, we call magnitudes of the stimuli doses and positive responses toxicities. Doses $X_1,\dots ,X_n$ are sequentially selected from a discrete ordered finite set denoted by $\mathcal{X}=\{d_1,\dots ,d_M:d_1<\cdots <d_M\}$. Assuming

$$F(x):=P\{Y=1|X=x\}=\mathrm{E}[Y|X]$$

is monotone increasing in x, it follows that $F_m:=F(d_m)$ is also monotone increasing in $m\in [1,\dots ,M]$; and it renders the region of interest interpretable as a neighborhood around a particular quantile of a cumulative distribution function of toxicity thresholds. The goal of the experiment is either (1) to estimate some pre-specified Γth quantile of $F(x)$ [i.e. $F^{-1}(\mathrm{\Gamma })$] or (2) to select the dose in $\mathcal{X}$ that is closest to $F^{-1}(\mathrm{\Gamma })$.

Most dose-finding designs attempt to concentrate dose allocations around $F^{-1}(\mathrm{\Gamma })$, which is called the target dose, rather than spread them over $\mathcal{X}$ (e.g. randomly or using the local D-optimality criterion). To achieve this concentration, the dose-assignment rule may use likelihood-based estimation, employing a parametric or nonparametric model for $F(x)$. The likelihood-based estimates are constructed from the observed toxicity rates at the doses in $\mathcal{X}$ [30]. Some dose-assignment rules that are not based explicitly on models (such as the increasingly popular family of interval designs, e.g. [18,21]) use the observed rates directly for dosing decisions. Even designs that do not rely upon estimation to allocate doses (such as the Markovian family of up-and-down designs, e.g. [10,12,17,28]) may use some type of binary regression to estimate $F^{-1}(\mathrm{\Gamma })$ at the end of the experiment, and binary regression also utilizes the observed toxicity rates.

As the sample size at a specific $X\in \mathcal{X}$ goes to ∞, the observed response rate converges to its true value [26,33]. Design developers and analysts (including the authors of this paper) have, it seems, made the implicit assumption that these rates are unbiased with finite dose-specific sample sizes as well. This is trivially true when dose assignments do not depend on outcomes, but unfortunately the assumption breaks down for adaptive designs. The very adaptive property that makes dose-finding designs useful and popular, namely their tendency to concentrate allocations in a certain region of the $F(x)$ curve, induces bias in observed dose-specific response rates.

In Section 2, we introduce the sufficient statistics for common dose-finding experiments and relate them to $\{F_m\}$. We then show the existence and form of the bias of the observed toxicity rate at $d_m$ for $F_m$, $m=1,\dots ,M$, and provide numerical illustrations of the bias for some common designs. In Section 3, we propose an adjustment to the observed toxicity rates that mitigates the bias. Section 4 includes a discussion of potential practical implications of the bias.

2. The bias of observed dose-specific toxicity rates

2.1. Preliminaries

Each treatment $X_i$ triggers a binary response $Y_i\in \{0,1\},i=1,\dots ,n$. Let ${\delta }_{im}=1$ if $X_i=d_m$ and 0 otherwise, $m=1,\dots ,M$. Dose-specific treatment and toxicity counts (ignoring their dependence on n for notational simplicity) are

$$N_m=\sum _{i=1}^n{\delta }_{im}\mathrm{and}{T}_m=\sum _{i=1}^n{Y}_i{\delta }_{im},m=1,\dots ,M.$$ (1)

Applying the factorization theorem to the likelihood $\prod _m{F}_m^{T_m}(1-F_m{)}^{(N_m-T_m)}$, with mild assumptions that typically hold as given in [30], one sees that the dose-specific summary statistics $\{T_m,N_m{\}}_1^M$ are sufficient for commonly used dose-finding experiments. But the most common statistics used in analyses are the observed dose-specific toxicity rates $\{T_m/N_m,N_m;N_m>0{\}}_1^M$, ignoring doses in $\mathcal{X}$ that are not used.

The ratio of two random variables $R_m:=T_m/N_m$, $N_m>0$ is commonly thought of as an estimate of $F_m$, but it is $\mathrm{E}[T_m]/\mathrm{E}[N_m]$ that equals $F_m$ as can be seen from the following equivalencies:

$$\begin{array}{rl}\mathrm{E}[T_m]& =\sum _{i=1}^n\mathrm{E}[Y_i{\delta }_{im}]=\sum _{i=1}^{n}P\{Y_i|{\delta }_{im}=1\}P\{{\delta }_{im}=1\}\\ & =F_m\sum _{i=1}^{n}P\{{\delta }_{im}=1\}\\ & =F_m\mathrm{E}[N_m].\end{array}$$

Therefore, $F_m$ is the ratio of expectations,

$$F_m=\mathrm{E}[T_m]/\mathrm{E}[N_m],m=1,\dots ,M.$$ (2)

We study when this ratio of expectations does or does not equal the expectation of the ratio $T_m/N_m$.

2.2. An expression for the bias of observed toxicity rates

Start with the definition of the covariance function for two random variables U and V, with $V\ne 0$:

$$\mathrm{Cov}[\frac{U}{V},V]=\mathrm{E}\left[\frac{U}{V}V\right]-\mathrm{E}\left[\frac{U}{V}\right]\mathrm{E}[V]=\mathrm{E}\left[U\right]-\mathrm{E}\left[\frac{U}{V}\right]\mathrm{E}[V].$$

Rearranging terms (as observed in [16]) yields $\mathrm{E}[U/V]=\{\mathrm{E}[U]-\mathrm{Cov}[U/V,V]\}/\mathrm{E}[V];$ substituting $T_m,N_m$ and $R_m$ for U,V and U/V, and using (2), we arrive at

$$\mathrm{E}\left[R_m\right]=\frac{\mathrm{E}\left[T_m\right]}{\mathrm{E}\left[N_m\right]}-\frac{\mathrm{Cov}[R_m,N_m]}{\mathrm{E}[N_m]}=F_m-\frac{\mathrm{Cov}[R_m,N_m]}{\mathrm{E}[N_m]}.$$ (3)

So $R_m$ is a biased estimator of $F_m$ unless its covariance with $N_m$ is zero. The bias is proportional, and opposite in sign, to $\mathrm{Cov}(R_m,N_m)$, but $|R_m-F_m|$ is always between $\pm 1$.

For adaptive dose-finding designs, $R_m$ is consistent if $N_m\to \mathrm{\infty }$ [26], but if $E[N_m]\to \mathrm{\infty }$ while $N_m$ itself remains finite with non-vanishing probability, consistency is not guaranteed. Therefore, care must be taken in using asymptotic approximations. Here we add another path to consistency via an order-of-magnitude analysis of equation (3). Because correlations are in [$-1,1$ ], the bias magnitude is bounded as

$$|\mathrm{E}[R_m]-F_m|=\frac{|\mathrm{Cov}[R_m,N_m]|}{\mathrm{E}[N_m]}\le \frac{\sqrt{\mathrm{Var}\left[R_m\right]}\sqrt{\mathrm{Var}\left[N_m\right]}}{\mathrm{E}[N_m]}\le {\displaystyle \frac{1}{2}}\mathrm{CV}[N_m],$$

where $\mathrm{CV}[N_m]$ is the coefficient of variation of $N_m$.

With Markovian up-and-down designs, $\mathrm{E}[N_m/n]$ converges to a constant while $\mathrm{Var}[N_m]/n\to 0$ (see, e.g. [10,12,17,28]), and therefore $R_m$ is consistent for all $d_m$. In contrast, when using interval designs (e.g. [18] and [20]) and likelihood-derived allocation procedures (e.g. [25]) with small n, $\{N_m\}$ are known to have high variability [29]. As $n\to \mathrm{\infty }$, these designs generally come to allocate doses exclusively to a single dose or to doses in a small interval [22,26]. Under such designs, the bias at doses outside this interval will not go away.

A straightforward rearrangement of terms in (3) reveals that

$$\begin{array}{r}\mathrm{Cor}[R_m,N_m]\{\begin{array}{ll}>0& \text{iff }\mathrm{E}[R_m]<F_m;\\ =0& \text{iff }\mathrm{E}[R_m]=F_m;\\ <0& \text{iff }\mathrm{E}[R_m]>F_m.\end{array}\end{array}$$ (4)

Figure 1 demonstrates this relationship between bias and $\mathrm{Cor}[R_m,N_m]$ for selected dose-finding designs that are described in Section 2.3. This plot shows points have roughly the same pattern of association regardless of design.

Figure 1.

Toxicity rate bias $({\overline{R}}_m-F_m)$ plotted by the correlation between $R_m$ and $N_m$ for selected dose-finding designs, n=30 left and n=120 right.

Now consider how bias might change as a function of dose. Suppose a dose-finding design causes $\{N_m{\}}_1^M$ to have a unimodal distribution around $F^{-1}(\mathrm{\Gamma })$ (as a function of m) and that $\{R_m{\}}_1^M$ is increasing with values that bound Γ. Then $\{R_m\}$ and $\{N_m\}$ are increasing together up to the point $[F^{-1}(\mathrm{\Gamma }),\mathrm{\Gamma }]$, after which $R_m$ continues to increase while $N_m$ decreases. That is, $\{R_m\}$ and $\{N_m\}$ are positively correlated for doses below $F^{-1}(\mathrm{\Gamma })$ and negatively correlated for larger doses. If a design had these characteristics, then by (4), for $d_m$ below the target dose, $R_m$ would have negative bias and above the target $R_m$ would have positive bias.

In addition, because $\mathrm{E}[N_m]$ (in the denominator of the bias) will tend to decrease as $|d_m-\mathrm{\Gamma }|$ increases, one expects the magnitude of the bias to increase as $|F_m-\mathrm{\Gamma }|$ increases. Thus we expect dose-finding designs to induce a ‘flaring out’ of the bias, pushing $\{R_m\}$ away from $F_m$, and even farther away from Γ in both directions.

2.3. Numerical illustration of observed toxicity rate bias as a function of dose

To illustrate the bias of observed toxicity rates, five common dose-finding designs are simulated, with each simulation run containing 10,000 replications and operating on the dose set $\mathcal{X}=\{1,\dots ,10\}$. To simplify our focus on bias, all simulations use the same simple logistic function defined by logit$[F(x)]=(x-5.6)/2$. The parameters of four designs studied are set with the experimental aim of estimating the 30th percentile of $F(x)$ [which is $F^{-1}(0.3)\approx 3.9$]. Studied are the Durham–Flournoy Biased Coin Design (BCD) [9,12], the Continual Reassessment Method (CRM) [4,25], the Cumulative Cohort Design (CCD) [18] and the K-in-a-row Design (KRD) [14,28,35] with K=2. For comparison, we also simulate the Classical Up-and-Down Design (CUDD) [8] whose aim is to estimate the 50th percentile [here $F^{-1}(0.5)=5.6$]. Experiments of size n=30 and n=120 are simulated for each design.

Let ${\overline{R}}_m$ denote the ensemble average of simulated $R_m$ values. Figure 1 plots the bias, ${\overline{R}}_m-F_m$, versus the correlation between $R_m$ and $N_m$ for $m=1,\dots ,M$ for all five dose-finding designs. The correlation is positive when the bias is negative and vice versa. Plots for n=30 and n=120 are quite similar, except that points of extreme bias using the CRM are tempered with the larger sample size.

Figure 2 plots the simulated bias versus dose. For all designs, the bias generally tends from negative to positive as the dose increases, crossing zero near $F^{-1}(\mathrm{\Gamma })$. The trend is more gradual and more clearly monotone under the Markovian designs (BCD, KRD, CUDD), with the long-memory CRM and CCD displaying greater bias magnitudes, and a steeper, more localized change from negative to positive near the target.

Figure 2.

Observed toxicity rate bias $({\overline{R}}_m-F_m)$ by dose for selected dose-finding designs, n=30 left and n=120 right. The target dose is 3.9 for BCD, KRD, CRM and CCD; and 5.6 for CUDD.

3. Mitigating the bias of observed toxicity rates

3.1. Potential mitigation approaches

The expression for bias (3) appears simple and straightforward at face value, suggesting there might be a way to mitigate the bias in real time, possibly even by direct substitution. Dose-specific covariances can only be observed on populations of experiments, such as we synthesized via simulation ensembles in the previous section. However, a rough on-the-fly estimate of $\mathrm{Cov}[R_m,N_m]$ is a sample covariance considering $R_{m-1},N_{m-1}$ and $R_{m+1},N_{m+1}$ to be replicates of $R_m,N_m$. The denominator of the bias, $\mathrm{E}[N_m]$, is more problematic with small overall n because the obvious substitution, $N_m$, will be very small except near the apparent target, making the overall correction rather unstable. We explored the use of this substitution and found that it tends to over-correct and to have high variance.

A far simpler approach is inspired by the work of Firth [11] on bias reduction, but makes use of our observation that the bias ‘flares out’ away from target. The fix is to slightly shrink all $R_m$'s towards target using the formula:

$${\tilde{R}}_m=\frac{{\displaystyle T_m+\mathrm{\Gamma }}}{{\displaystyle N_m+1}}=\frac{{\displaystyle N_m{R}_m+\mathrm{\Gamma }}}{{\displaystyle N_m+1}}.$$ (5)

The magnitude of shrinkage is approximately inverse to $N_m$, in agreement with the bias expression (3). When $\mathrm{\Gamma }=0.5$, this formula is identical to the commonly used correction for calculating the empirical logit in the presence of zero cell counts [2,37]. Firth's correction has been used to analyze results from a Markovian up-and-down design with $\mathrm{\Gamma }=0.5$ in anesthesiology research [1].

This mitigation formula performs rather acceptably in the simulations shown below.

3.2. Numerical examination of the shrinkage formula

We applied the shrinkage formula (5) to the simulation data that produced the figures in Section 2.3. Plots from two representative designs (KRD and CRM) in Figure 2 are repeated in Figure 3 (top left, dashed curves), augmented with plots created by applying formula 5 (solid curves). The bias is mitigated fairly well. There is some over correction at extreme doses, particularly far above the target where sample sizes are small. However, we found the overall best performance by far is obtained when shrinkage is applied without regard to dose-specific sample sizes.

Figure 3.

The impact of shrinking the bias of observed toxicity rates by dose for the KRD (red–orange) and CRM designs (blue–green). Top row: Ensemble bias (left) and root-mean-square error (right). Bottom row: Ensemble bias² (left) and Variance (right). Raw observed toxicity rates ($R_m$) are connected by dashed lines while bias-mitigated rates (${\tilde{R}}_m$) are connected by solid lines.

Figure 3 (top, right) shows that the application of formula (5) reduces root-mean-square error (RMSE), which helps $\{{\tilde{R}}_m\}$ stay generally closer than $\{R_m\}$ to the true $\{F_m\}$. RMSE is graphed to make the magnitudes of mitigation in these two figures comparable. The two graphs in the bottom row of Figure 3 clarify the bias/variance tradeoff across doses, showing the mitigating effect of (5) on both.

3.3. Impact of bias and shrinkage on parametric MLEs

We assumed the two parameter function logit$[F(x)]=(x-5.6)/2$ in order to examine how bias in the observed toxicity rates affects parametric estimates following KRD and CRM designs. Figure 4 shows the effect of using the shrinkage formula on estimates of location, slope and target dose parameters, which here are $F^{-1}(0.50)=5.6$, 0.5 and $F^{-1}(0.30)=3.9$, respectively. For reference, we include estimates generated by the D-optimal design, which assigns an equal number of samples to $F^{-1}(0.176)$ and $F^{-1}(0.824)$ [34]. The D-optimal design provides an important benchmark in that it minimizes the asymptotic confidence ellipsoid around the MLEs.

Figure 4.

The impact of bias and shrinkage upon parametric estimates from simulated KRD and CRM for n = 30 experiments. The D-optimal design experiment included for reference. True values are indicated by horizontal red lines.

CRM and KRD both underestimate location and overestimate slope, with CRM's estimates being more extreme. The shrinkage mitigates those biases, but not completely. Encouragingly, the third panel shows that the adaptive designs perform well in estimating the target dose, with shrinkage not offering any improvement. Note that KRD generally produces better estimates than CRM, for both parameters and target dose. Note also that in a very small fraction of adaptive-design runs, very extreme parametric estimates are generated that fall outside the range of values shown in Figure 4, usually because all observations are concentrated in one to two doses.

3.4. Revisiting coverage of isotonic regression interval estimates of the target dose

In our presentation of centered isotonic regression [CIR, 27], we also developed analytical confidence intervals (CIs) for the target dose. Those intervals tended to be conservative for $F^{-1}(\mathrm{\Gamma })$ with fixed non-adaptive designs. However, when applied to simulated KRD dose-finding experiments, coverage was rather deficient at n=20: only $80\mathrm{\%}$ for the nominal $90\mathrm{\%}$ CI. An empirical modification, assuming a simple random draw of doses, only increased coverage by $\sim 1\mathrm{\%}$ [27, Table 7]. Coverage was still somewhat deficient at n=40, but somewhat conservative at n=80, indicating a sample-size related issue. The deficiency in coverage was clearly asymmetric, but at the time we could not find the root cause.

Table 1 uses the same dataset that produced Table 7 in [27], showing interval coverage rates and widths calculated with and without shrinking the observed response rates toward the target rate according to (5). At n=20 coverage, while still somewhat deficient, is much closer to the nominal $90\mathrm{\%}$ level with bias mitigation. Coverage attains the nominal level before n=40 and the change in coverage with increase sample size is less pronounced. Clearly, more work is needed on interval estimation procedures following adaptive designs.

Table 1. Coverage rates (interval widths)^a for the target dose $F^{-1}(0.30)$ using 2 KRD with and without bias mitigation.

Family	n	Using raw observed rates	Using rates after shrinkage
Logistic	20	0.80 (2.14)	0.88 (2.68)
	40	0.88 (2.02)	0.92 (2.30)
	80	0.92 (1.63)	0.94 (1.75)
Weibull	20	0.80 (1.93)	0.88 (2.44)
	40	0.88 (1.85)	0.92 (2.11)
	80	0.91 (1.57)	0.93 (1.68)

^aThe nominal coverage rate is $90\mathrm{\%}$ for interval estimates after simulated KRD (K=2) experiments with n=20,40,80. Average interval widths, given in parentheses, are on the dose scale. Simulated experiments were run assuming randomly selected toxicity curves from the Logistic (top) and Weibull (bottom) family. Raw observed rates were previously published in Table 7 in [27].

4. Discussion

A pattern of bias induced by adaptive dose-finding designs has been hiding in plain sight. This article exposes and explains this discovery in the context of dose-finding experiments. It is interesting to note that a formula similar to (3) appears in the group sequential design Example 3 of [23]. Also, formula (3) can be applied when adapting treatment assignments to independent groups instead of doses (ordered points) in which case, for example, it reveals that increasing treatment assignments to those showing less toxicity causes positive bias because the correlation between toxicity rate and sample sizes is negative. For continuous responses, re-defining $R_m$ to be the observed group-specific sample mean and $T_m$ to be the observed dose-specific sum, formula (3) reveals that the sample mean will be negatively biased for the population mean when using Greedy sampling, Thompson sampling [32], Randomly Reinforced Urns [13] or other schemes that increasing the probability of sampling from high performing groups. The derivation of formula (3) provides an alternative proof to Theorem 1 in [24] and demonstrates that their conclusion that ‘adaptively collected data have negative bias’ applies to the class of adaptations they considered and not generally.

Figure 4 (right panel) demonstrates that adaptive designs can produce unbiased MLEs of the target dose despite producing biased parameter estimates.

If isotonic regression or CIR are used to produce a point estimate for the target dose, then the shrinkage helps ensure that the confidence interval of this estimate has sufficient coverage. Figure 4 demonstrates that parametric estimates of percentiles not too distant from the target (compare $F^{-1}(0.5)$ on the left with $F^{-1}(0.3)$ on the right) and estimates of the slope (center panel) are much less reliable without shrinkage. It is of concern that data generated by dose-finding designs of all kinds (including the long-memory CRM and CCD) are increasingly being used to estimate F more broadly and to make decisions regarding dropping doses away from the target (e.g. [39]). Safety exclusion rules are important, but without effective bias mitigation, these decision rules are likely to be overly aggressive. This work reinforces the warning that data from dose-finding designs that concentrate treatments at doses near a target is of limited value for making predictions except near that target.

The small-sample dose-finding problem is difficult, and some have shown more complacency than is warranted regarding the quality and reliability of existing methods. The truth is that, at present, no single design and estimation combination can provide the quality of results that some applied researchers, particularly in Phase I clinical trials, have come to expect from dose-finding experiments. Whether the expectations are unrealistic, or further breakthroughs await discovery, remains to be seen.

Markovian up-and-down designs, with their more tractable analytical properties and more balanced treatment allocations, seem to have less bias away from target, and lower RMSE, than long-memory designs like the CRM (Figures 2–4). However, parametric estimates following a CRM greatly benefit from bias-mitigating shrinkage (Figures 3 and 4).

We have been fortunate to find a simple mitigating fix for the observed toxicity rate bias that works reasonably well. Shrinkage estimators often reduce variance at the cost of some additional bias, but with dose-finding designs one can leverage knowledge of the patterns shown in Section 2.3 to reduce both estimation bias and variance via shrinkage. The shrinkage formula (5) is now implemented in the R package cir on GitHub, via the logical argument adaptiveShrink, and will be incorporated into the CRAN version as well soon.

Disclosure statement

No potential conflict of interest was reported by the authors.