Benchmark dose analysis via nonparametric regression modeling

Abstract

Estimation of benchmark doses (BMDs) in quantitative risk assessment traditionally is based upon parametric dose-response modeling. It is a well-known concern, however, that if the chosen parametric model is uncertain and/or misspecified, inaccurate and possibly unsafe low-dose inferences can result. We describe a nonparametric approach for estimating BMDs with quantal-response data based on an isotonic regression method, and also study use of corresponding, nonparametric, bootstrap-based confidence limits for the BMD. We explore the confidence limits’ small-sample properties via a simulation study, and illustrate the calculations with an example from cancer risk assessment. It is seen that this nonparametric approach can provide a useful alternative for BMD estimation when faced with the problem of parametric model uncertainty.

1. INTRODUCTION

1.1. Benchmark analysis

In quantitative risk assessment, the benchmark approach for estimating low-dose risk after exposure to a hazardous stimulus has seen substantial growth in familiarity and acceptance since its proposal in the mid-1980s⁽¹⁾. The method takes a function, R(x), relating the response to exposure at dose x ≥ 0, and manipulates components of this posited model to yield a benchmark dose (BMD) at which a specified benchmark response (BMR) is attained. (If the exposure is measured as a concentration, one refers to the exposure point as a Benchmark Concentration, or BMC.) The BMD is then further manipulated—using, e.g., uncertainty factors or modifying factors⁽²⁾—to arrive at a level of acceptable human or ecological exposure to the hazard or to otherwise establish low-exposure guidelines. One important modification is the use of lower 100(1 – α)% confidence limits on the BMD—called benchmark dose (lower) limits or simply BMDLs⁽⁹⁾—in order to incorporate statistical variability of the BMD point estimate into the calculations. Where needed for clarity, the notation adds a subscript for the BMR level at which each quantity is calculated: BMD_100BMR and BMDL_100BMR; 0 < BMR < 1. With this, many analysts use the BMDL or BMCL as a point of departure in quantitative risk assessment^(3,4), and these quantities are employed for risk characterization and management by a variety of agencies, including the U.S. Environmental Protection Agency (EPA), the U.S. Food and Drug Administration (FDA), the Organisation for Economic Co-operation and Development (OECD), and many others. The application of benchmark analysis for quantifying and managing risk with a variety of toxicological endpoints is growing in both the United States and the European Union^(5–8).

A common data-analytic application of the BMD occurs with proportions, say, Y_i/N_i, where Y_i is the number of subjects expressing the adverse event under study, N_i is the number of subjects tested, and each ith proportion is observed at a corresponding exposure dose x_i (i = 1, …, m). This is the ‘quantal response’ setting, and it is common in carcinogenicity testing, environmental toxicity analysis, and many other biomedical risk studies⁽¹⁰⁾. With quantal data the basic statistical model is the binomial: Y_i ~ indep. Bin.(N_i, R(x_i)), where R(x_i) is the function representing the probability of response at dose x_i. For risk-analytic considerations the exposed subjects’ differential risk adjusted for any spontaneous or background effects is often of interest. This leads to consideration of excess risk functions such as the extra risk R_E(x) = {R(x) − R(0)}/{1 − R(0)}⁽²⁾. The BMD is determined by setting R_E(x) = BMR over x > 0 and finding the smallest solution. When applied to data, the method is a form of inverse non-linear regression and except for the use of an excess risk function upon which to base the inversion, is similar to estimation of an ‘effective dose’ such as the well-known median effective dose, ED₅₀⁽²⁾.

1.2. Model dependence

A recognized concern with benchmark analysis is its potential sensitivity to specification of the dose-response function, R(x). A wide variety of parametric forms has been proffered for R(x) when estimating BMDs. Most operate well at (higher) doses near the range of the observed quantal outcomes; however, these different models can produce wildly different BMDs at very small levels of risk when applied to the same set of data^(11,12). Traditional strategies to avoid parametric model dependencies in low-dose risk analysis generally rely on the so-called No-Observed-Adverse-Effect Level (NOAEL) or similar variants, which does not require specification of R(·). Substantial statistical instabilities have been identified with use of the NOAEL for risk estimation, however, and contemporary analysts recommend against use of this dated technology^(13,14). (Indeed, the BMD was originally suggested as a more-stable statistical alterative to the NOAEL⁽¹⁾.) Needed is a modern quantitative methodology that can produce reliable inferences on acceptable exposure levels to the hazard under study, but that can avoid estimation biases and instabilities resulting from uncertain model specification⁽¹⁵⁾.

Recent work with the BMD has led to some intriguing advances, including model averaging techniques motivated from both statistical frequentist^(16–19) and Bayesian^(20–22) perspectives. Alternatively, in order to avoid specification of R(x) we describe below a (frequentist) nonparametric estimator for the observed dose-response pattern. Such nonparametric curve estimation has seen only limited implementation in low-dose risk assessment. Krewski et al.⁽²³⁾ made perhaps the earliest substantial effort, but they did not connect to the modern machinery of benchmark analysis. A selection of works have applied semi-parametric models for estimating BMDs, where certain portions of the model are parametrically specified. For example, Bosch et al.⁽²⁴⁾ combined a nonparametric comparison measure with a parametric dose-response specification such as the well-known probit model. Later, Fine and Bosch⁽²⁵⁾ applied quasi-likelihood-type estimating equations to extend this semi-parametric model framework. These works focused on continuous measurements, however, and not quantal data. For the quantal setting Wheeler and Bailer⁽²⁶⁾ considered model-free specifications for the dose response by exploring monotone cubic B-splines. This is similar to our approach below, except that they employed hierarchical, parametric Bayesian estimation of the dose-response parameters; also see Guha et al.⁽²⁷⁾.

Rather than appealing to the hierarchical Bayesian paradigm, Dette and colleagues^(28,29) (along with the references found therein) discussed non-hierarchical, kernel-based nonparametric estimation for effective doses, such as the ED₅₀, from a quantal-response experiment. As noted above, ED₅₀ estimation shares many similarities with BMD estimation, although Dette and his co-authors did not connect their work to problems in benchmark analysis. We consider here related methods based on the work of Bhattacharya and Lin⁽³⁰⁾. Those authors produced an adaptive, isotonic, regression-based method for estimating nonparametric effective doses in the larger area of bioassay analysis, but also considered explicitly the benchmark dose problem. Their approach extended an earlier work of Bhattacharya and Kong⁽³¹⁾, where the ED₅₀ was estimated by inverting a model-free estimator of the original risk function R(x). We adapted the Bhattacharya and Kong approach for use with excess risk functions such as R_E(x) to construct nonparametric BMDs and BMDLs⁽³²⁾. That previous work presented the theoretical features of the method and gave a corresponding BMDL based on a nonparametric bootstrap; herein we focus on application of our fully nonparametric BMD for use in toxicological risk assessment. Section 2 reviews the nonparametric estimator, while Section 3 evaluates the small-sample performance of the bootstrap-based BMDLs via a broad-scale simulation study. Section 4 illustrates the methods with an example from cancer risk assessment, and Section 5 ends with a contemplative discussion.

2. MODEL-INDEPENDENT BENCHMARK ANALYSIS

2.1. Nonparametric BMD estimation

We continue to assume a binomial structure for quantal-response data: Y_i ~ indep. Bin. (N_i, R(x_i)), i = 1, …, m. The sample proportions are written as p_i = Y_i/N_i and we assume the dose values are ordered such that 0 = x₁ < x₂ < ··· < x_m. Rather than rely on any parametric specifications on the dose-response function, however, we follow our previous construction⁽³²⁾ and only oblige R(·) to satisfy simple continuity and monotonicity assumptions. That is, we require R(x) to be a continuous function with non-negative first derivative R′(x) ≥ 0 over x ≥ 0.

We view the monotonized sequence {R(x₁), R(x₂), …, R(x_m)} as the preliminary estimation target, for which the unique maximum likelihood estimator (MLE) is the isotonic sequence⁽³³⁾

$${\stackrel{\sim }{p}}_i=\underset{1\le u\le i}{max}\underset{i\le v\le m}{min}\frac{{\sum }_{j=u}^v{Y}_j}{{\sum }_{j=u}^v{N}_j}.$$ (2.1)

This ensures p̃₁ ≤ ··· ≤ p̃_m, and may be obtained via the well-known pool-adjacent-violators (PAV) algorithm⁽³⁴⁾. To find a model-independent, isotonic estimator of the function R(x) we construct a linear interpolating spline from the PAV estimates p̃_i:

$$\stackrel{\sim }{R}(x)=\{\begin{array}{ll}{\stackrel{\sim }{p}}_i\hfill & \text{if}x=x_i;\hfill \\ {\stackrel{\sim }{p}}_i+\frac{{\stackrel{\sim }{p}}_{i+1}-{\stackrel{\sim }{p}}_i}{x_{i+1}-x_i}(x-x_i)\hfill & \text{if}{x}_i\le x<x_{i+1}\hfill \end{array}$$ (2.2)

for i = 1, …, m − 1, and R̃(x_m) = p̃_m. From this, we build an isotonic estimator for R_E(x) as the corresponding linear interpolator connecting the suitably adjusted, monotonized, pointwise estimates of the extra risks at each x_i:

$${\stackrel{\sim }{R}}_E(x)=\{\begin{array}{ll}\frac{{\stackrel{\sim }{p}}_i-{\stackrel{\sim }{p}}_1}{1-{\stackrel{\sim }{p}}_1}\hfill & \text{if}=x_i;\hfill \\ \frac{1}{1-{\stackrel{\sim }{p}}_1}\left\{{\stackrel{\sim }{p}}_i-{\stackrel{\sim }{p}}_1+\frac{{\stackrel{\sim }{p}}_{i+1}-{\stackrel{\sim }{p}}_i}{x_{i+1}-x_i}(x-x_i)\right\}\hfill & \text{if}{x}_i\le x<x_{i+1}\hfill \end{array}$$ (2.3)

for i = 1, …, m − 1, and R̃_E(x_m) = (p̃_m − p̃₁)/(1 − p̃₁). If R̃(x) satisfies the monotonicity and continuity constraints we impose above, so will the corresponding R̃_E(x).

Now, for notational convenience, denote BMD_100BMR as ξ_100BMR. Given the model-independent, isotonic estimator in (2.3), a similarly model-independent estimator for ξ_100BMR is available by inverting R̃_E(x) at the specified BMR and calculating the smallest positive solution; i.e., ξ̃_100BMR = min{x: R̃_E(x) ≥ BMR}. Since R̃_E(x) is a form of continuous, linear interpolating spline, we can write this model-independent estimator in closed form:

$$\begin{array}{l}{\stackrel{\sim }{\xi }}_{100\text{BMR}}=\{\begin{array}{ll}{x}_i+\frac{\text{BMR}-\frac{{\stackrel{\sim }{p}}_i-{\stackrel{\sim }{p}}_1}{1-{\stackrel{\sim }{p}}_1}}{\frac{{\stackrel{\sim }{p}}_{i+1}-{\stackrel{\sim }{p}}_1}{1-{\stackrel{\sim }{p}}_1}-\frac{{\stackrel{\sim }{p}}_i-{\stackrel{\sim }{p}}_1}{1-{\stackrel{\sim }{p}}_1}}(x_{i+1}-x_i)\hfill & \text{if}\frac{{\stackrel{\sim }{p}}_i-{\stackrel{\sim }{p}}_1}{1-{\stackrel{\sim }{p}}_1}<\text{BMR}<\frac{{\stackrel{\sim }{p}}_{i+1}-{\stackrel{\sim }{p}}_1}{1-{\stackrel{\sim }{p}}_1}\text{for}i=1,\dots ,m-1\hfill \\ x_i\hfill & \text{if}\text{BMR}=\frac{{\stackrel{\sim }{p}}_i-{\stackrel{\sim }{p}}_1}{1-{\stackrel{\sim }{p}}_1}\text{for}i=2,\dots ,m;\hfill \end{array}\\ =\{\begin{array}{ll}{x}_i+\frac{(1-{\stackrel{\sim }{p}}_1)\text{BMR}-({\stackrel{\sim }{p}}_i-{\stackrel{\sim }{p}}_1)}{{\stackrel{\sim }{p}}_{i+1}-{\stackrel{\sim }{p}}_i}(x_{i+1}-x_i)\hfill & \text{if}\frac{{\stackrel{\sim }{p}}_i-{\stackrel{\sim }{p}}_1}{1-{\stackrel{\sim }{p}}_1}<\text{BMR}<\frac{{\stackrel{\sim }{p}}_{i+1}-{\stackrel{\sim }{p}}_1}{1-{\stackrel{\sim }{p}}_1}\text{for}i=1,\dots ,m-1\hfill \\ x_i\hfill & \text{if}\text{BMR}=\frac{{\stackrel{\sim }{p}}_i-{\stackrel{\sim }{p}}_1}{1-{\stackrel{\sim }{p}}_1}\text{for}i=2,\dots ,m.\hfill \end{array}\end{array}$$ (2.4)

(We discuss below possible strategies to accommodate the unusual case when BMR > {p̃_m − p̃₁}/{1 − p̃₁}.) We showed⁽³²⁾ that ξ̃_100BMR possesses desirable statistical qualities: under suitable regularity conditions it converges to the target value ξ_100BMR, is asymptotically unbiased, and has a large-sample distribution converging to a Gaussian (‘normal’) form.

2.2. Model-independent bootstrap BMDLs

Unfortunately, we also discovered⁽³²⁾ that the asymptotic variance of ξ̃_100BMR depends critically on the underlying risk function R(x). Since under our basic premise we are unwilling to fully specify the form of R(x), this makes it difficult to employ the asymptotic features of ξ̃_100BMR for constructing a corresponding BMDL. For building confidence limits on an ED₅₀, Bhattacharya and Kong⁽³¹⁾ encountered an analogous predicament with their similarly constructed, isotonic, ${\widehat{\text{ED}}}_{50}$ estimator. Their solution was to appeal instead to the bootstrap⁽³⁵⁾, and their nonparametric bootstrap-based inferences achieved respectable stability. Following their lead, we also consider the bootstrap to build model-independent BMDLs.

Appeal to bootstrapping for construction of confidence limits in risk assessment has seen increasing application, and is becoming an accepted approach for calculating critical risk-analytic quantities such as the BMDL. Most authors employ it under some form of parametric specification, however^(36–39). Here, we follow our previous construction⁽³²⁾ and describe a bootstrap-based approach for building BMDLs with the model-independent estimator from (2.4). In keeping with our general estimation strategy, we do not assume any parametric form for R(x), although we do continue to impose our continuity and monotonicity assumptions. We also continue to assume a binomial parent distribution for the data.

To perform the nonparametric bootstrapping, we begin with the original proportions Y_i/N_i and resample each m-dose quantal response B > 0 times. That is, in the bth resample at each x_i we generate the pseudo-random variate $Y_{ib}^{\ast }$ from Bin.(N_i, Y_i/N_i), i = 1, …, m; b = 1, …, B. We then apply the PAV algorithm to the $Y_{ib}^{\ast }$s to yield the monotonized bootstrap sequence

$${\stackrel{\sim }{p}}_{ib}^{\ast }=\underset{1\le u\le i}{max}\underset{i\le v\le m}{min}\frac{{\sum }_{j=u}^v{Y}_{jb}^{\ast }}{{\sum }_{j=u}^v{N}_j}.$$

From these we produce an isotonic bootstrap estimate of the extra risk via (2.3). Denote this as ${\stackrel{\sim }{R}}_{E,b}^{\ast }(x)$. We then apply (2.4) to find a bootstrapped ${\stackrel{\sim }{\xi }}_b^{\ast }$. (For simplicity here we suppress the BMR subscript on ξ̃, but it is understood that all these operations are conducted for a fixed, pre-specified BMR.) Following previously validated suggestions^(37,40) for the number, B, of bootstrap resamples, we work with B = 2000.

Two special cases that require attention occur when Y_i = 0 or when Y_i = N_i at any i. If so, the resampling will always produce $Y_{ib}^{\ast }=0$ or $Y_{ib}^{\ast }=N_i$, respectively, generating no new bootstrap information⁽⁴¹⁾. Various remedies for this are possible⁽³⁸⁾; in keeping with our nonparametric strategy, we refrain from applying any parametric-model constructs. Instead, we add a small constant ε > 0 to the numerator and twice it to the denominator of the observed proportions Y_i/N_i. That is, whenever Y_i = 0 or Y_i = N_i, we replace Y_i/N_i with {Y_i + ε}/{N_i + 2ε}. This shrinks the proportion away from 0 or from 1 and towards $\frac{1}{2}$. (Shrinkage towards any fixed value is admittedly arbitrary, and we chose $\frac{1}{2}$ simply as an objective, default, shrinkage target. If in practice a different target were available based on experiment- or stimulus-specific considerations, it could easily be employed instead.) We experimented with a variety of possible values for ε, and found that only a slight amount of shrinkage was necessary to stabilize the bootstrap calculations. We settled on $\epsilon =\frac{1}{4}$, i.e., whenever Yi = 0 or Yi = Ni, replace Yi/Ni with $\{Y_i+\frac{1}{4}\}/\{N_i+\frac{1}{2}\}$.

To find a lower 100(1 − α)% confidence limit on the BMD, we collect the ${\stackrel{\sim }{\xi }}_b^{\ast }$s together to produce a bootstrap distribution for ξ̃. From this, we apply the well-known percentile method⁽⁴²⁾: order the B bootstrapped ${\stackrel{\sim }{\xi }}_b^{\ast }$ values into ${\stackrel{\sim }{\xi }}_{(1)}^{\ast }\le {\stackrel{\sim }{\xi }}_{(2)}^{\ast }\le \cdots \le {\stackrel{\sim }{\xi }}_{(B)}^{\ast }$ and select as the BMDL the lower αth percentile: ${\underset{\_}{\xi }}_{100\text{BMR}}={\stackrel{\sim }{\xi }}_{(\alpha B)}^{\ast }$. This represents a model-independent, nonparametric BMDL for use in risk assessment practice.

3. BMDL PERFORMANCE EVALUATION

Previously⁽³²⁾ we studied the performance of our model-independent BMDL, ξ_100BMR, via a series of Monte Carlo simulations. The BMDL exhibited generally stable coverage characteristics, but some undercoverage—i.e., coverage rates below the nominal 100(1 − α)% level—occurred for very small sample sizes and very shallow dose-response patterns. Our earlier focus was on introducing the isotonic estimation method and exploring its theoretical properties, however, and so our study was limited to a small, illustrative selection of dose-response functions. We recognized that more complete evaluations were needed to study the operating characteristics of our model-independent BMDL. Here, we expand upon our previous results to study how the isotonic method operates over a broader range of possible dose-response models/shapes.

3.1. Simulation design

Our focus is on the empirical coverage characteristics of the bootstrap-based confidence limit ξ_100BMR. We set the BMR to a standard default level BMR = 0.10 ⁽⁸⁾, and operate at 95% nominal coverage. We employ either four dose levels, x = 0, 0.25, 0.5, 1.0, corresponding to a standard design in cancer risk experimentation⁽⁴³⁾, or six dose levels, x = 0, 0.0625, 0.125, 0.25, 0.50, 1.0, expanding on the four-dose geometric spacing. We also include a modified six-dose design x = 0, 0.1, 0.2, 0.4, 0.6, 1.0 that gives less focus to doses near x = 0, to study how/if this affects the BMDL ξ_100BMR.

For simplicity, equal numbers of subjects, N_i = N, are taken per dose group. We fix the total sample size at (or as near as possible to) ${\sum }_{i=1}^m{N}_i=100,200,400,\text{or}4000$, producing per-dose sample sizes of N = 25, 50, 100, or 1000 for the four-dose design and N = 16, 33, 66, or 666 for the six-dose designs.

For the underlying dose-response patterns we employ models corresponding to a variety of functions available from the U.S. EPA’s BMDS software program for performing BMD calculations⁽⁴⁴⁾. Table I provides a selection of four two-parameter and four three-parameter dose-response models. These are taken from a collection of dose-response forms chosen by Wheeler and Bailer⁽³⁶⁾ in their studies of (parametric) estimators for the BMD. [Wheeler and Bailer did not include the log-logistic model (3B) in all their calculations, although they did present it as a possible data generating model. They also presented a possible data generating model based on the gamma cumulative distribution function (c.d.f.), but did not use it in their calculations. In a similar vein, we do not consider the gamma model here.] In our earlier work⁽³²⁾ we studied a subset of the models in Table I: the two-stage (3A), log-probit (3C), and Weibull (3D). Thus for these three models the simulation outcomes we give here replicate the results we reported previously. Notice that certain models impose constraints on selected parameters; these are listed in Table I to correspond with typical constraints we find in the environmental toxicology literature. In the table, for the log-probit model (3C) we define R(0) = γ₀. Note that the quantal-linear model (2A) may also be referred to as the ‘one-stage’ model (a form of ‘multi-stage’ model) or as the ‘complementary-log’ model. This may equivalently appear as R(x) = γ₀ + (1 − γ₀)(1 − e^−β₁x), where γ₀ = 1 − e^−β₀.

Table I.

Selected quantal dose-response models common in environmental toxicology.

Model	Code	R(x)	Constraints/Notes
Quantal-linear	2A	1 − exp{−β0 − β1x}	β0 ≥ 0, β1 ≥ 0
Quantal-quadratic	2B	γ0 + (1 − γ0)(1 − exp{−β1x2})	0 ≤ γ0 ≤ 1, β1 ≥ 0
Logistic	2C	(1 + exp{−β0 − β1x})−1	—
Probit	2D	Φ(β0 + β1x)	Φ(·) is the N(0,1) c.d.f.
Two-stage	3A	1 − exp{−β0 − β1x − β1x2}	βj ≥ 0, j = 0, 1, 2
Log-logistic	3B	${\gamma }_0+\frac{1-{\gamma }_0}{1+exp\{-{\beta }_0-{\beta }_{1}log[x]\}}$	0 ≤ γ0 ≤ 1, β1 ≥ 0
Log-probit	3C	γ0 + (1 − γ0)Φ(β0 + β1 log[x])	0 ≤ γ0 ≤ 1, β1 ≥ 0
Weibull	3D	γ0 + (1 − γ0)(1 − exp{−e−β0xβ1})	0 ≤ γ0 ≤ 1, β1 ≥ 1

To set the simulation parameters for each model, we fix the risk R(x) at three doses: $x=0,\frac{1}{2},1$. Based on typical patterns seen in cancer risk assessment^(41,45), background risks at x = 0 are set between 1% and 30%, and the other risk levels are increased to produce a variety of (strictly) increasing forms, ending with high-dose risks at x = 1 between 10% and 90%. Using the specifications at x = 0 and x = 1 we solve for two unknown parameters. This completes determination of the two-parameter models (2A–2D). For the three-parameter models (3A–3D), we additionally employ the response specification at $R(\frac{1}{2})$ to solve for the third unknown parameter. The actual specifications and resulting parameter configurations for the various models are given in Table II; we also include the corresponding values of ξ₁₀.

Table II.

Models and configurations (including true BMD, ξ₁₀, at BMR = 0.10) for the Monte Carlo evaluations.

	Configuration:	A	B	C	D	E	F
Constraint:	R(0) =	0.01	0.01	0.10	0.05	0.30	0.10
	$\text{R}(\frac{1}{2})=$	0.04	0.07	0.17	0.30	0.52	0.50
	R(1) =	0.10	0.20	0.30	0.50	0.75	0.90
Model	Parameters
Quantal-linear (2A)	β0	0.0101	0.0101	0.1054	0.0513	0.3567	0.1054
	β1	0.0953	0.2131	0.2513	0.6419	1.0296	2.1972
	ξ10	1.1056	0.4944	0.4193	0.1641	0.1023	0.0480
Quantal-quadratic (2B)	γ0	0.0100	0.0100	0.1000	0.0500	0.3000	0.1000
	β1	0.0953	0.2131	0.2513	0.6419	1.0296	2.1972
	ξ10	1.0514	0.7032	0.6475	0.4052	0.3199	0.2190
Logistic (2C)	β0	−4.5951	−4.5951	−2.1972	−2.9444	−0.8473	−2.1972
	β1	2.3979	3.2088	1.3499	2.9444	1.9459	4.3944
	ξ10	1.0401	0.7773	0.5535	0.3974	0.1619	0.1700
Probit (2D)	β0	−2.3263	−2.3263	−1.2816	−1.6449	−0.5244	−1.2816
	β1	1.0448	1.4847	0.7572	1.6449	1.1989	2.5631
	ξ10	1.0476	0.7372	0.5331	0.3567	0.1606	0.1575
Two-stage (3A)	β0	0.0101	0.0101	0.1054	0.0513	0.3567	0.1054
	β1	0.0278	0.0370	0.0726	0.5797	0.4796	0.1539
	β2	0.0675	0.1761	0.1788	0.0622	0.5501	2.0433
	ξ10	1.0602	0.6756	0.5911	0.1783	0.1818	0.1925
Log-logistic (3B)	γ0	0.0100	0.0100	0.1000	0.0500	0.3000	0.1000
	β0	−2.3026	−1.4376	−1.2528	−0.1054	0.5878	2.0794
	β1	1.6781	1.8802	1.7603	1.3333	1.9735	3.3219
	ξ10	1.0648	0.6676	0.5848	0.2083	0.2439	0.2760
Log-probit (3C)	γ0	0.0100	0.0100	0.1000	0.0500	0.3000	0.1000
	β0	−1.3352	−0.8708	−0.7647	−0.0660	0.3661	1.2206
	β1	0.7808	0.9794	0.9456	0.8189	1.2261	1.9626
	ξ10	1.0711	0.6575	0.5789	0.2267	0.2608	0.2794
Weibull (3D)	γ0	0.0100	0.0100	0.1000	0.0500	0.3000	0.1000
	β0	−2.3506	−1.5460	−1.3811	−0.4434	0.0292	0.7872
	β1	1.6310	1.7691	1.6341	1.0716	1.4483	1.9023
	ξ10	1.0634	0.6716	0.5874	0.1852	0.2072	0.2025

For each parameter configuration (labeled A–F), 2000 individual, pseudo-binomial, data sets are simulated and as noted in §2.2, B = 2000 bootstrap samples are generated from each data set to produce the corresponding empirical coverage value. Notice then that the approximate standard p error of the estimated coverage over all 2000 simulated data sets is $\sqrt{(0.05)(0.95)/2000}=0.005$, and this never exceeds $\sqrt{(0.5)(0.5)/2000}=0.011$. All of our calculations are performed with the R statistical programming environment⁽⁴⁶⁾.

3.2. Infinite BMDs

An unusual artifact we uncovered while conducting our Monte Carlo computations was that for some very shallow dose-response patterns, calculation of a nonparametric BMD can sometimes break down. One obvious case is when the BMR is set too high, so that the isotonic extra risk estimator never reaches the desired benchmark response over the range of the doses, i.e., R̃_E(x) < BMR, for all x ≤ x_m. If so, there is no solution to the BMD-defining relationship R̃_E(ξ) = BMR. Of course, if the extra risk were estimated using a fully parametric (non-decreasing) function one would simply extrapolate the function outside of the dose range to find the solution. To imitate this strategy with our nonparametric estimator, suppose that R̃_E(x) from (2.3) is linear and strictly increasing along its final segment between x_m−1 and x_m. Then if R̃_E(x) < BMR ∀x ≤ x_m, we simply extend this final line segment past x_m until it crosses the horizontal BMR line, and solve for ξ̃_100BMR at that intersection point. While admittedly one should apply any such extrapolations past the range of the data with great caution, this strategy nonetheless allows us to report an objective estimate for the BMD in this unusual case.

The extrapolative estimate for the BMD will still fail if the final line segment from (2.3) is flat, i.e., if p̃_m−1 = p̃_m (or when p̃_m−2 = p̃_m−1 = p̃_m, etc.). When this occurs, the data are in effect telling us that the observed dose response cannot attain the BMR, no matter how large x grows. Correspondingly, in such an instance we simply drive the estimator ξ̃_100BMR to ∞, or, equivalently, report it as undefined. In the extreme, this also occurs if the p̃_is all equal each other. In this case, (2.3) will produce R̃_E(x) = 0, so we again are forced to drive ξ̃_100BMR to ∞.

Somewhat perniciously, this issue of undefined or infinite BMDs was not uncommon with some of the very shallow dose-response configurations in Table II, especially configurations ‘A’ and ‘C’. Particularly at the smaller sample sizes, such shallow configurations could even produce non-monotone response patterns in the simulated data, despite the fact that the underlying R(x) function was strictly increasing. This forced the PAV algorithm to ‘flatten out’ the estimated extra risk over a large portion of the dose range, and when this occurred near the upper end of the range we encountered the infinite-BMD phenomenon.

Operationally, when any ‘infinite’ ξ̃_100BMR was observed in our bootstrap procedure, we set the estimate equal to machine infinity. If this occurred for more than 100(1 − α)% of the bootstrapped ξ̃^*s, we defined ξ_100BMR itself as ‘infinite’, and viewed this as failure to cover the true ξ_100BMR.

3.3. Simulation results

We summarize the empirical coverage results from our Monte Carlo study in a series of tables. Recall that we fix BMR = 0.10 and operate at nominal 95% coverage. Each initial table displays empirical coverage results recorded for all model configurations and sample sizes except the quantal-linear model (2A); we will discuss model 2A in greater detail below. Table III presents the results for the geometric four-dose design with x = 0, 0.25, 0.50, 1.0: therein, coverages lie near and generally within Monte Carlo sampling variability of the nominal 95% level. Some undercoverage is observed at the lowest per-dose sample size of N = 25 and/or with the (shallow) response configurations ‘A’ and, to a lesser extent, ‘B’; however, conservative overcoverage is at least as prevalent. Indeed, averaged across the seven models (2B–2D and 3A–3D) in Table III, the empirical coverages as a function of sample size are at or near the 95% nominal level: 94.22% for N = 25, 94.76% for N = 50, 95.84% for N = 100, and 96.95% for N = 1000.

Table III.

Empirical coverage rates of nonparametric bootstrap BMDL ξ₁₀ from Monte Carlo evaluations under geometric four-dose design for selected dose-response models given in Table I. (Model 2A is discussed separately.) Nominal coverage rate is 95%.

Model Code	Sample size, N	Configuration						Row means
		A	B	C	D	E	F
2B	25	0.9355	0.9470	0.9820	0.9640	0.9780	0.9340	0.9568
2B	50	0.8955	0.9655	0.9900	0.9740	0.9750	0.9500	0.9583
2B	100	0.9185	0.9730	0.9870	0.9820	0.9715	0.9715	0.9673
2B	1000	0.9270	0.9970	0.9930	0.9895	0.9720	0.9895	0.9780
2C	25	0.9330	0.9705	0.9680	0.9640	0.9305	0.9405	0.9511
2C	50	0.8980	0.9845	0.9765	0.9725	0.9340	0.9565	0.9537
2C	100	0.9125	0.9925	0.9760	0.9785	0.9465	0.9755	0.9636
2C	1000	0.9050	1.0000	0.9675	0.9875	0.9605	0.9950	0.9693
2D	25	0.9340	0.9640	0.9635	0.9555	0.9295	0.9440	0.9484
2D	50	0.8945	0.9780	0.9740	0.9640	0.9345	0.9570	0.9503
2D	100	0.9170	0.9865	0.9735	0.9710	0.9325	0.9765	0.9595
2D	1000	0.9125	1.0000	0.9570	0.9820	0.9605	0.9945	0.9678
3A	25	0.9370	0.9290	0.9725	0.8970	0.9260	0.9475	0.9348
3A	50	0.9060	0.9545	0.9820	0.9125	0.9350	0.9605	0.9418
3A	100	0.9185	0.9650	0.9820	0.9275	0.9450	0.9765	0.9524
3A	1000	0.9295	0.9960	0.9760	0.9335	0.9620	0.9970	0.9657
3B	25	0.9375	0.9220	0.9700	0.9045	0.9305	0.9370	0.9336
3B	50	0.9085	0.9505	0.9780	0.9330	0.9360	0.9495	0.9426
3B	100	0.9230	0.9595	0.9785	0.9470	0.9505	0.9685	0.9545
3B	1000	0.9370	0.9885	0.9695	0.9740	0.9580	0.9915	0.9698
3C	25	0.9375	0.9075	0.9695	0.9270	0.9460	0.9310	0.9364
3C	50	0.9155	0.9445	0.9815	0.9235	0.9525	0.9500	0.9446
3C	100	0.9290	0.9525	0.9805	0.9465	0.9560	0.9705	0.9558
3C	1000	0.9465	0.9725	0.9620	0.9710	0.9495	0.9900	0.9653
3D	25	0.9385	0.9230	0.9700	0.8935	0.9270	0.9520	0.9340
3D	50	0.9035	0.9515	0.9805	0.9170	0.9410	0.9595	0.9422
3D	100	0.9205	0.9600	0.9795	0.9355	0.9615	0.9775	0.9558
3D	1000	0.9360	0.9940	0.9735	0.9500	0.9760	0.9960	0.9709

Table IV presents the results for the geometric six-dose design with x = 0, 0.0625, 0.125, 0.25, 0.50, 1.0: coverage patterns therein appear slightly more stable on average than those seen in Table III, although there are also more extreme cases of undercoverage. These again occur at the lowest per-dose sample size N = 25 and/or with the (shallow) response configuration ‘A’. Overall, average empirical coverages across the seven models 2B–2D and 3A–3D are again close to nominal: 94.61% for N = 25, 95.96% for N = 50, 96.88% for N = 100, and 97.15% for N = 1000. As might be expected, decreasing the per-dose sample sizes while increasing the number of doses yields greater stability at larger sample sizes, but this effect appears to reverse somewhat when sample sizes drop.

Table IV.

Empirical coverage rates of nonparametric bootstrap BMDL ξ₁₀ from Monte Carlo evaluations under geometric six-dose design for selected dose-response models given in Table I. (Model 2A is discussed separately.) Nominal coverage rate is 95%.

Model Code	Sample size, N	Configuration						Row means
		A	B	C	D	E	F
2B	16	0.8805	0.9120	0.9795	0.9685	0.9900	0.9730	0.9506
2B	33	0.8685	0.9480	0.9940	0.9790	0.9950	0.9820	0.9611
2B	66	0.9050	0.9715	0.9950	0.9900	0.9945	0.9825	0.9731
2B	666	0.9320	0.9985	0.9965	0.9930	0.9915	0.9830	0.9824
2C	16	0.8765	0.9505	0.9690	0.9720	0.9550	0.9735	0.9494
2C	33	0.8700	0.9770	0.9885	0.9800	0.9780	0.9855	0.9632
2C	66	0.9025	0.9905	0.9835	0.9860	0.9755	0.9830	0.9702
2C	666	0.9130	1.0000	0.9705	0.9910	0.9565	0.9690	0.9667
2D	16	0.8785	0.9415	0.9680	0.9740	0.9565	0.9665	0.9475
2D	33	0.8705	0.9680	0.9890	0.9815	0.9770	0.9790	0.9608
2D	66	0.9035	0.9850	0.9825	0.9885	0.9725	0.9770	0.9682
2D	666	0.9205	1.0000	0.9725	0.9865	0.9610	0.9625	0.9672
3A	16	0.8800	0.9235	0.9760	0.9355	0.9700	0.9755	0.9434
3A	33	0.8750	0.9475	0.9915	0.9635	0.9850	0.9845	0.9578
3A	66	0.9070	0.9685	0.9915	0.9675	0.9845	0.9860	0.9675
3A	666	0.9330	0.9940	0.9855	0.9495	0.9560	0.9825	0.9668
3B	16	0.8825	0.9180	0.9715	0.9460	0.9705	0.9810	0.9449
3B	33	0.9080	0.9385	0.9900	0.9615	0.9850	0.9890	0.9620
3B	66	0.9170	0.9585	0.9885	0.9710	0.9885	0.9920	0.9693
3B	666	0.9420	0.9895	0.9830	0.9590	0.9680	0.9955	0.9728
3C	16	0.8815	0.8980	0.9740	0.9485	0.9740	0.9830	0.9432
3C	33	0.8735	0.9295	0.9895	0.9645	0.9885	0.9895	0.9558
3C	66	0.9110	0.9510	0.9885	0.9695	0.9855	0.9920	0.9663
3C	666	0.9520	0.9775	0.9760	0.9625	0.9740	0.9975	0.9733
3D	16	0.8815	0.9210	0.9725	0.9390	0.9745	0.9755	0.9440
3D	33	0.8740	0.9430	0.9900	0.9640	0.9850	0.9845	0.9568
3D	66	0.9085	0.9635	0.9895	0.9675	0.9895	0.9865	0.9675
3D	666	0.9390	0.9915	0.9835	0.9500	0.9760	0.9880	0.9713

Results for the modified six-dose design with x = 0, 0.1, 0.2, 0.4, 0.6, 1.0 are given in Table V. The pattern of coverage is roughly similar to that in Table IV, although somewhat larger undercoverage is evidenced at N = 25. Overall, average empirical coverages across the seven models 2B–2D and 3A–3D are 93.30% for N = 25, 95.19% for N = 50, 96.00% for N = 100, and 96.20% for N = 1000. At least for these designs, deemphasizing doses near to zero appeared to have limited impact.

Table V.

Empirical coverage rates of nonparametric bootstrap BMDL ξ₁₀ from Monte Carlo evaluations under modified six-dose design for selected dose-response models given in Table I. (Model 2A is discussed separately.) Nominal coverage rate is 95%.

Model Code	Sample size, N	Configuration						Row means
		A	B	C	D	E	F
2B	16	0.8840	0.9085	0.9710	0.9230	0.9750	0.9570	0.9364
2B	33	0.8815	0.9360	0.9890	0.9465	0.9855	0.9760	0.9524
2B	66	0.9100	0.9595	0.9850	0.9635	0.9900	0.9735	0.9636
2B	666	0.9335	0.9875	0.9810	0.9560	0.9855	0.9705	0.9690
2C	16	0.8795	0.9445	0.9680	0.9330	0.9325	0.9530	0.9351
2C	33	0.8760	0.9695	0.9860	0.9480	0.9680	0.9715	0.9532
2C	66	0.9065	0.9865	0.9810	0.9645	0.9590	0.9705	0.9613
2C	666	0.9185	0.9995	0.9695	0.9485	0.9480	0.9580	0.9570
2D	16	0.8840	0.9365	0.9690	0.9455	0.9325	0.9520	0.9366
2D	33	0.8810	0.9590	0.9830	0.9620	0.9665	0.9690	0.9534
2D	66	0.9100	0.9740	0.9830	0.9700	0.9590	0.9635	0.9599
2D	666	0.9220	0.9980	0.9660	0.9635	0.9465	0.9580	0.9590
3A	16	0.8885	0.8895	0.9640	0.9230	0.9355	0.9565	0.9262
3A	33	0.8845	0.9325	0.9835	0.9495	0.9730	0.9705	0.9489
3A	66	0.9135	0.9525	0.9785	0.9540	0.9665	0.9670	0.9553
3A	666	0.9385	0.9715	0.9645	0.9520	0.9560	0.9545	0.9562
3B	16	0.8895	0.8810	0.9660	0.9240	0.9640	0.9920	0.9361
3B	33	0.8835	0.9220	0.9845	0.9505	0.9810	0.9955	0.9528
3B	66	0.9145	0.9450	0.9810	0.9545	0.9785	0.9975	0.9618
3B	666	0.9425	0.9640	0.9690	0.9445	0.9770	1.0000	0.9662
3C	16	0.8840	0.8745	0.9695	0.9115	0.9740	0.9920	0.9343
3C	33	0.8935	0.9140	0.9845	0.9495	0.9880	0.9980	0.9546
3C	66	0.9165	0.9385	0.9860	0.9470	0.9865	0.9990	0.9623
3C	666	0.9370	0.9585	0.9835	0.9520	0.9845	1.0000	0.9693
3D	16	0.8885	0.8835	0.9655	0.9265	0.9435	0.9515	0.9265
3D	33	0.8845	0.9285	0.9845	0.9485	0.9690	0.9710	0.9477
3D	66	0.9150	0.9470	0.9805	0.9605	0.9635	0.9685	0.9558
3D	666	0.9400	0.9685	0.9720	0.9445	0.9600	0.9575	0.9571

On balance, our simulation results exhibit generally stable large-sample coverage characteristics for the model-independent, bootstrapped limit ξ_100BMR at the standard level of BMR = 0.10. Slight undercoverage is evidenced in selected instances, more so when sample sizes are small. We previously identified a possible explanation for this behavior⁽³²⁾, where we found that the PAV-based estimator in (2.4) exhibits slight negative bias when applied to convex response patterns. Negative bias in the estimator can translate to more conservative lower confidence bounds for the BMDL. On a relative scale the bias was not exceptional, however, and in fact is not wholly unexpected: bias can be a recurring issue with isotonic regression estimators⁽⁴⁷⁾. Indeed, the effect moderated as sample sizes increased. Nonetheless, this suggests that the method should be applied when sufficient data are available to help validate its asymptotic motivation.

We also conducted Monte Carlo coverage evaluations for the case of BMR = 0.01. While less common, this smaller BMR may be employed in practice when sufficient data are available to support inferences at extreme low doses^(8,14). Our results (not shown) were generally similar to those seen above; in particular, the coverage rates again drove towards the nominal 95% level as the sample size increased. At the lowest sample sizes, however, they appeared much more variable, and often in the direction of undercoverage. We encountered a few empirical coverage rates that dropped below 50% with some of the low-response-rate models, especially configurations ‘A’ and ‘B’. This is, in fact, consistent with practical benchmarking experience: when response rates are very small at low doses, and if the N_is do not counter by being fairly large, insufficient information will be available to perform effective inferences if the BMR is set very low. We therefore urge caution in practice when employing these methods with very small sample sizes, particularly with low-response patterns.

3.4. Simulation results for the (concave) quantal-linear model

Our Monte Carlo results for the quantal-linear model (2A) differed from the general trends seen with the other seven models in Table I, hence we discuss these separately. Among all the models we study, the quantal linear model (2A) is the only strictly concave form. That is, since model 2A has R(x) = 1−exp{−β₀ − β₁x} and β₁ ≥ 0, its first derivative is non-negative: R′(x) = β₁ exp{−β₀ − β₁x} ≥ 0 for all x > 0. As with the rest of the models in Table I, this produces an increasing dose response. (For the trivial case where β₁ = 0 the dose response will be flat and of no risk-analytic interest.) For β₁ > 0, however, the second derivative is $R^{″}(x)=-{\beta }_1^{2}exp\{-{\beta }_0-{\beta }_{1}x\}<0$ for all x > 0 and therefore the dose response increases at a decreasing rate: a concave function.

This feature impacts our linear interpolator. A concave response function will, more often than not, produce concave dose-response patterns in the p̃_is from (2.1), and linearly interpolating a concave-increasing pattern can lead to underestimation of the extra risk. This translates as overestimation of ξ̃_100BMR. The corresponding lower confidence bounds would, in turn, be driven up. If variation in the data is tight this could push them past the true, underlying vale of ξ_100BMR, collapsing the coverage rates. With very small numbers of doses, m, and large per-dose sample sizes, N_i, the rates can drop well below their nominal level. In theory the issue would quickly be remedied by increasing m, since the theoretical properties of (2.4) obtain as both min{N_i} and m grow large without limit. With small m, however, the potential undercoverage can be dramatic.

This effect was evidenced in our Monte Carlo coverage evaluations. Table VI presents Model 2A’s small-sample coverage rates for our model-independent, bootstrap BMDL ξ_100BMR at the standard level of BMR = 0.10, and with nominal coverage set to 95%. Notice the large numbers of entries below 0.95 (the zero coverage value, 0.0000, for the four-dose design under configuration ‘F’ at N = 1000 is not a typographical error). In Table II, one can determine that concavity in Model 2A increases as the configuration index moves from ‘A’ to ‘F’. Thus two clear trends emerge in Table VI: (i) coverage performance worsens as the concavity of the model increases, and (ii) adding more doses improves the performance more often than it debilitates it. In addition, comparing the two six-dose designs shows that as more sample information is placed closer to the true BMD—which of course is impossible in practice without knowledge of that true value—the coverage locates closer to its nominal level.

Table VI.

Empirical coverage rates of nonparametric bootstrap BMDL ξ₁₀ from Monte Carlo evaluations for quantal-linear dose-response model (2A) given in Table I. Nominal coverage is 95%.

Model Code	Sample size, N	Configuration						Row means
		A	B	C	D	E	F
Geometric four-dose design
2A	25	0.9330	0.9160	0.9570	0.8425	0.9110	0.7465	0.8843
2A	50	0.9120	0.9205	0.9630	0.9045	0.9115	0.6335	0.8742
2A	100	0.9305	0.9330	0.9575	0.9215	0.9115	0.4785	0.8554
2A	1000	0.9540	0.9560	0.9545	0.9145	0.8445	0.0000	0.7706
Geometric six-dose design
2A	16	0.8885	0.8860	0.9680	0.9250	0.9315	0.8665	0.9109
2A	33	0.8950	0.9155	0.9835	0.9605	0.9630	0.9140	0.9386
2A	66	0.9270	0.9430	0.9810	0.9615	0.9525	0.9205	0.9476
2A	666	0.9495	0.9400	0.9680	0.9465	0.9470	0.9280	0.9465
Modified six-dose design
2A	16	0.8945	0.8790	0.9530	0.9190	0.8035	0.8595	0.8848
2A	33	0.9150	0.9075	0.9695	0.9470	0.9010	0.9010	0.9235
2A	66	0.9330	0.9355	0.9685	0.9505	0.9000	0.9065	0.9323
2A	666	0.9455	0.9390	0.9485	0.9465	0.9395	0.8715	0.9318

The coverage results in Table VI warn that our PAV-based benchmark estimator should be applied to concave dose-response patterns with caution. For shallow response patterns (configurations ‘A’–’C’), coverage is roughly similar to that seen in Tables III–V. With greater concavity comes greater instability, however, leading to extreme degradation with the extremely concave configuration ‘F’.

This begs the question, how much concavity is too much? As reviewed above, the second derivative of R(·) measures concavity in the response. Thus as a first approximation we can quantify the concavity in any data set by calculating how the slope of the isotonically estimated response function (2.2) changes from x_i to x_i+1.

Specifically, given PAV-based estimates p̃_i from (2.1), write the slope between each adjacent pair as Δ_i = {p̃_i+1 − p̃_i}/{x_i+1 − x_i}, i = 1, …, m − 1. The change in these slopes is ${\mathrm{\Delta }}_i^{\prime }=\{{\mathrm{\Delta }}_{i+1}-{\mathrm{\Delta }}_i\}/\{x_{i+1}-x_i\}$, i = 1, …, m − 2. Then, e.g., the average change in slope, ${\overline{\mathrm{\Delta }}}^{\prime }=\frac{1}{m-2}{\sum }_{i=1}^{m-2}{\mathrm{\Delta }}_i^{\prime }$, quantifies concavity in the isotonically estimated response: smaller (more negative) values of Δ̄′ indicate greater concavity. Note, however, that concavity far away from the BMD has little effect on the coverage for the BMDL ξ_100BMR. More pertinent for our purposes is a measure of local concavity near the estimated benchmark point. For instance, suppose the calculated BMD lies between two dose values. Then, to measure local concavity we might average the two corresponding values of Δ′ associated with those bracketing doses. That is, define a measure of local concavity, Δ̃′, as

$${\stackrel{\sim }{\mathrm{\Delta }}}^{\prime }=\{\begin{array}{ll}{\mathrm{\Delta }}_1^{\prime }\hfill & \text{if}{x}_1\le {\stackrel{\sim }{\xi }}_{100\text{BMR}}<x_2\hfill \\ \frac{{\mathrm{\Delta }}_{i-1}^{\prime }+{\mathrm{\Delta }}_i^{\prime }}{2}\hfill & \text{if}{x}_i\le {\stackrel{\sim }{\xi }}_{100\text{BMR}}<x_{i+1}(i=2,\dots ,m-2)\hfill \\ {\mathrm{\Delta }}_{m-2}^{\prime }\hfill & \text{if}{x}_{m-1}\le {\stackrel{\sim }{\xi }}_{100\text{BMR}}<x_m\hfill \\ 0\hfill & \text{if}{x}_m\le {\stackrel{\sim }{\xi }}_{100\text{BMR}}\hfill \end{array}$$ (3.1)

The latter specification for Δ̃′ when ξ̃_100BMR ≥ x_m is based on our suggestion in §3.2 to extrapolate along a straight-line segment to define R̃_E beyond the upper range of the data. A possible alternative in this case is to take ${\stackrel{\sim }{\mathrm{\Delta }}}^{\prime }={\mathrm{\Delta }}_{m-2}^{\prime }$. In any case, similar to Δ̄′, smaller (more negative) values of Δ̃′ indicate greater local concavity.

We computed Δ̃′ for each design/parameter configuration combination in Table VI, to study how this measure correlates with unstable coverage performance. [Since we had available the true dose response information from Table II, we used R(x_i) in place of p̃_i and ξ_100BMR in place of ξ̃_100BMR for the calculations.] The results appear in Table VII. As expected, the concavity measure for the shallow configurations ‘A’–’C’ is fairly close to zero. The remaining, more-curvilinear configurations show smaller (more negative) local concavity, and correspond to patterns of greater coverage instability in Table VI. As a first step, therefore, we recommend that when the data indicate local concavity dropping below about Δ̃′< −0.3 for m = 4 doses or Δ̃′ < −0.6 for m = 6 doses, our PAV-based linear interpolator may not be appropriate for use in constructing BMDLs. (But, see the discussion in §5, below.)

Table VII.

Local change in slope, Δ̃′ from (3.1), under quantal-linear dose-response model (2A) given in Table I across configurations from Table II.

Design	Configuration
	A	B	C	D	E	F
Geometric four-dose	−0.0128	−0.0511	−0.0635	−0.3341	−0.5768	−2.5721
Geometric six-dose	−0.0128	−0.0615	−0.0765	−0.5112	−0.8273	−3.7932
Modified six-dose	−0.0127	−0.0499	−0.0618	−0.4365	−0.7736	−3.5018

4. EXAMPLE: FORMALDEHYDE CARCINOGENESIS IN LABORATORY ANIMALS

Formaldehyde, CH₂O, is a well-known industrial compound, exposure to which can be extensive in a variety of occupational and environmental settings. To explore the toxic and carcinogenic potential of the chemical, Schlosser et al.⁽⁴⁸⁾ reported on nasal squamous cell carcinomas observed in laboratory rats after chronic, two-year, inhalation exposure. The CH₂O exposure dose, x, is actually a concentration (in ppm) here, and so technically we will compute benchmark concentrations (BMCs) based on the quantal carcinogenicity data. Six CH₂O concentrations were studied: x = 0.0, 0.7, 2.0, 6.0, 10.0, and 15.0 ppm. Since intercurrent mortality can occur in such chronic-exposure studies, the final tumor incidences were adjusted for potential differences in animal survival. Table VIII lists the survival-adjusted proportions.

Table VIII.

Formaldehyde carcinogenicity data⁽⁴⁸⁾.

Exposure conc. (ppm), xi	0.0	0.7	2.0	6.0	10.0	15.0
Adjusted tumor incidence, Yi	0	0	0	3	21	150
Animals at risk, Ni	122	27	126	113	34	182

Of interest is calculation of the BMC, and more importantly a 95% lower confidence limit, BMCL, to help inform risk characterization on this potential carcinogen. (Our analysis here is intended primarily to illustrate the nonparametric, model-independent methodology, and not to supersede the larger risk analysis reported by Schlosser et al.⁽⁴⁸⁾ or by any previous authors regarding CH₂O carcinogenicity.) We make only the (reasonable) assumption that the true dose response is continuous and monotone non-decreasing over x ≥ 0, and operate at BMR = 0.10. From Table VIII the observed response is already non-decreasing, so the per-concentration PAV estimates p̃_i are simply the observed proportions: p̃₁ = p̃₂ = p̃₃ = 0, ${\stackrel{\sim }{p}}_4=\frac{3}{113},{\stackrel{\sim }{p}}_5=\frac{21}{34}$, and ${\stackrel{\sim }{p}}_6=\frac{150}{182}$. Constructing the isotonic extra risk estimator in (2.3) and setting it equal to BMR = 0.10 produces the model-independent estimate ξ̃₁₀ = 6.497 ppm.

For the BMCL we apply the percentile bootstrap as described in §2.2. To do so we first check that local concavity near the BMC as measured by (3.1) is acceptable: we see x₄ = 6.0 < ξ̃₁₀ = 6.50 so take ${\stackrel{\sim }{\mathrm{\Delta }}}^{\prime }=\frac{1}{2}\{{\mathrm{\Delta }}_3^{\prime }+{\mathrm{\Delta }}_4^{\prime }\}$. From Table VIII we find Δ₃ = 0.007, Δ₄ = 0.148, and Δ₅ = 0.041, with ${\mathrm{\Delta }}_3^{\prime }=+0.035$ and ${\mathrm{\Delta }}_4^{\prime }=-0.027$. Thus ${\stackrel{\sim }{\mathrm{\Delta }}}^{\prime }=\frac{1}{2}(0.035-0.027)=0.004$. This is near zero and positive, indicating slight local convexity in the neighborhood of ξ̃₁₀. Hence, no problematic issues with potential dose-response concavity are evidenced and we proceed with our nonparametric bootstrap BMCL.

For the bootstrap, we generate B = 2000 resamples from the original data, and find no occurrences of infinite ξ̃^*s. Our 95% BMCL is the lower 5th percentile from this distribution: ξ₁₀ = 6.332 ppm. By comparison, Schlosser et al.⁽⁴⁸⁾ reported a BMC₁₀ of 6.90 ppm with corresponding 95% BMCL₁₀ = 6.25 ppm under the log-probit model, a BMC₁₀ of 6.40 ppm with corresponding 95% BMCL₁₀ = 6.22 ppm under a form of the Weibull model, and a BMC₁₀ of 6.40 ppm with corresponding 95% BMCL₁₀ = 6.22 ppm under a form of multi-stage model. All sets of values rest in similar ranges, and provide comparable points of departure for conducting further risk-analytic calculations on formaldehyde carcinogenicity. Thus while our model-independent approach operates similarly to established parametric analyses for these data, it also provides an added benefit: it frees the risk assessor from uncertainties about the quality of any parametric assumptions made to support the benchmark computations.

5. DISCUSSION

Herein, we consider a model-independent, nonparametric method for estimating benchmark doses (BMDs) in quantitative risk analysis. Placing emphasis on cancer risk assessment, we describe an approach for estimating the BMD without call to any specific parametric dose-response models, relying only on the assumptions that the underlying response is continuous and monotone non-decreasing^(30,31). Lower confidence limits (BMDLs) using this estimator are derived from non-parametric bootstrap methods, building upon previous explorations into bootstrap resampling for benchmark inference^(37,49). Based on a Monte Carlo study, we find that the BMDLs exhibit relatively stable coverage for reasonably large sample sizes, but that some undercoverage can occur for very small sample sizes and very shallow dose-response patterns. Since the method is based on linearly interpolating the nonparametrically estimated risk function, the undercoverage is exacerbated if the dose-response pattern is highly concave and the number of doses is small; in such cases we cannot recommend use of this procedure. (But, see below.)

When sufficient data are available to support the nonparametric constructions, risk analysts can apply our results to build inferences on the BMD that avoid concerns over parametric model adequacy, expanding past the many, varied parametric models seen in practice. This extended operability can lead to improved risk analytic decision-making in carcinogenicity testing and other adverse-event risk assessments.

Of course, some caveats and qualifications are in order. The percentile method we used for finding the BMDL ξ_100BMR is a basic approach for constructing bootstrap inferences. While the method appears generally stable at larger sample sizes, its mixed performance with very small samples might be improved by moving to more-complex bootstrapping strategies. For instance, the bias-corrected, accelerated (BCa) bootstrap⁽³⁵⁾ is a well-known alternative to the percentile method, so we evaluated the BCa approach under the standard four-dose design using our convex-model Monte Carlo configurations from §3.1. We found that the resulting BMDLs exhibited slightly tighter empirical coverages at smaller sample sizes and with the more problematic, shallow, response patterns. Improvements were not seen across all configurations studied, however. (Details are available in a separate document⁽⁵⁰⁾.) From this, we can recommend use of BCa-based BMDLs when samples sizes are very small and/or for shallow response patterns; however, the basic percentile method appeared to operate adequately in the majority of cases we studied.

As we note in §1.2, a number of model-robust competitors to our nonparametric BMDL exist, primarily focused on (parametric) model-averaging techniques. While a complete comparison among all these methods is beyond the scope here, we can make a direct comparison with a frequentist model-averaged (FMA) BMDL proposed by Piegorsch et al.⁽¹⁹⁾. Their method employed a weighted average of BMD estimators over an ‘uncertainty class’ of different parametric models for R(x). Information-theoretic weights were used to construct the model-averaged BMD point estimate, ${\overline{\text{BMD}}}_{100\text{BMR}}$; an associated standard error, $\text{se}[{\overline{\text{BMD}}}_{100\text{BMR}}]$; and from these, a large-sample Wald-type lower confidence limit, ${\overline{\text{BMDL}}}_{100\text{BMR}}={\overline{\text{BMD}}}_{100\text{BMR}}-z_{\alpha }\text{se}[{\overline{\text{BMD}}}_{100\text{BMR}}]$, where z_α is the upper-α critical point from a standard normal distribution. Assuming the uncertainty class is constructed to capture an appropriate set of potential models for R(x), Piegorsch et al. found that their FMA BMDL exhibits substantial model-robustness for building benchmark dose lower limits.

Comparison of the FMA BMDL with our nonparametric BMDL ξ_100BMR is possible here: fortunately, Piegorsch et al. employed the exact same models and parametric configurations as in our Table II for their own small-sample simulation study of ${\overline{\text{BMDL}}}_{100\text{BMR}}$’s coverage characteristics. (They considered only the geometric four-dose design, but they did use the same BMR = 0.10 and nominal 95% confidence level as we have here.) Thus it is possible to explicitly compare the empirical coverage rates they achieved under their parametric FMA BMDL with those we find under our nonparametric BMDL. We plot in Fig. 1 the associated pairs of empirical coverages for all 192 model/configuration/sample size combinations represented in Table III and, for Model 2A with the geometric four-dose design (Table VI). In the figure, empirical coverage for the FMA BMDL is plotted on the vertical axis and that for our nonparametric BMDL is plotted on the horizontal axis. (Note the different axis scales.) If both methods operated perfectly, we would expect a tight cluster of points to lie at the crossing of the two ‘95% nominal coverage’ lines in the figure. Instead, the graphic portrays a more complex scenario. It corroborates the near-to-nominal coverage for our nonparametric BMDL seen in Table III, but it also highlights the few distressingly low coverage values under Model 2A in Table VI: see the points lying far to the left of the vertical rule marking nominal coverage. Indeed, the horizontal scale is lengthened by these few low-coverage points, making it difficult to visualize the larger comparison. To compensate, Fig. 2 repeats the plot with all the Model 2A points removed. The finer detail illustrates that by excluding Model 2A, very few cases occur where both methods produce significant undercoverage (since the lower left quadrant in the plot is almost empty), and that the FMA BMDL is often more conservative than our nonparametric BMDL (since the upper right quadrant in the plot is the most dense). Indeed, the upper half of the plot is far more populated, indicating the more-conservative coverage of the FMA BMDL. (This was, in fact, explicitly recognized by Piegorsch et al.⁽¹⁹⁾: their method included a conservative approximation for $\text{se}[{\overline{\text{BMD}}}_{100\text{BMR}}]$ to avoid cases of severe undercoverage, and this is evident in Fig. 1: to within Monte Carlo sampling error, only a handful of points lie below the nominal 95% level.) This conservatism does provide for some added ‘safety’ in use of ${\overline{\text{BMDL}}}_{100\text{BMR}}$: in contrast to the nonparametric BMDL studied here, the FMA BMDL does not fall victim to problems of benchmark overestimation with concave dose-response patterns. As Piegorsch et al. readily admitted, however, their FMA BMDL is dependent on proper development of a pertinent uncertainty class of parametric models. In cases where this is not possible, the nonparametric BMDL we suggest here can serve as a model-robust alternative, when employed cautiously with concave-increasing data (as described in §3.4).

Figure 1.

Empirical coverage rates between nonparametric BMDL ξ₁₀ from §2.2 and parametric model-averaged BMDL⁽¹⁹⁾ at BMR = 0.10 under geometric four-dose design using configurations from Table II. (Note difference in scales.) Horizontal and vertical lines indicate nominal 95% coverage level. All models from Table I are included.

Figure 2.

Empirical coverage rates between nonparametric BMDL ξ₁₀ from §2.2 and parametric model-averaged BMDL⁽¹⁹⁾ at BMR = 0.10 under geometric four-dose design using configurations from Table II, excluding Model 2A from Table I. Horizontal and vertical lines indicate nominal 95% coverage level.

It is also important to emphasize that our nonparametric method requires both the number of doses m and the per-dose sample sizes N_i to grow large⁽³²⁾. In practice, however, resource and design constraints can conspire to restrict one or both of these factors. With small m in particular, the dose spacings are often sparse and wide; this can compromise the ability of our nonparametric linear interpolator to adequately describe the dose-response pattern. Indeed, Muri et al.⁽⁵¹⁾ report that the most prevalent study design they found for estimating a BMD among 20 pesticide risk analyses employed only m = 4 doses, and did so almost twice as often as the next most-common design (which was m = 5). When studying the U.S. EPA’s Integrated Risk Information System database (http://www.epa.gov/iris), Nitcheva et al.⁽⁵²⁾ found that that the most common number of doses used among 91 rodent carcinogenicity studies was even lower, at m = 3. (Clearly, designs with as few as m = 3 doses/concentrations spread the dose placement farther apart and limit our isotonic estimator’s ability to capture information in the data. We do not recommend application of our techniques to such sparse, minimally informative designs.) Next most common was again m = 4.

Referring to our Monte Carlo results in Table III, we see that with m = 4 doses the nonparametric BMDL’s coverage characteristics are relatively stable if sufficient numbers of subjects/dose, N, are employed. Still, one questions whether we can gain greater information about the pattern of dose response, and therefore about ξ_100BMR, if we increase the number of doses. Somewhat more-stable coverage patterns were seen with m = 6 doses in Tables IV and V. So, can increasing m to, say, 10 doses improve the small-sample operating characteristics of the BMDL if resource constraints force the N_is down to perhaps only 10 subjects/dose? For example, Bhattacharya and Lin⁽⁵³⁾ were able to show that an adaptive nonparametric estimator based on inverting (2.2) proved competitive in such a setting. To examine how this extends to BMD estimation with extra risk functions, we repeated our Monte Carlo evaluations for all eight models in Table I with m = 10 doses spaced evenly between x = 0 and x = 1. We again studied constant per-dose sample sizes, N, using N = 10, 20, 40, and 400 to compare with the total sample sizes in Table III. (All other aspects of the calculations for the bootstrap BMDL ξ₁₀ remained the same.) The results appear in Table IX; note the inclusion of model 2A at the top of the table.

Table IX.

Empirical coverage rates of nonparametric bootstrap BMDL ξ₁₀ from Monte Carlo evaluations under equi-spaced ten-dose design for dose-response models given in Table I. Nominal coverage rate is 95%.

Model Code	Sample size, N	Configuration						Row means
		A	B	C	D	E	F
2A	10	0.8430	0.8810	0.9845	0.9420	0.9900	0.9215	0.9270
2A	20	0.8930	0.9170	0.9950	0.9645	0.9895	0.9555	0.9524
2A	40	0.9275	0.9295	0.9970	0.9720	0.9880	0.9675	0.9636
2A	400	0.9560	0.9505	0.9805	0.9655	0.9805	0.9615	0.9658
2B	10	0.8155	0.9540	0.9830	0.9380	0.9995	0.9970	0.9478
2B	20	0.8855	0.9445	0.9960	0.9785	1.0000	0.9980	0.9671
2B	40	0.9215	0.9765	0.9985	0.9715	1.0000	0.9990	0.9778
2B	400	0.9370	0.9980	1.0000	0.9735	0.9990	0.9950	0.9838
2C	10	0.8105	0.9495	0.9855	0.9540	0.9910	0.9870	0.9463
2C	20	0.8840	0.9615	0.9960	0.9850	0.9930	0.9870	0.9678
2C	40	0.9065	0.9860	0.9980	0.9825	0.9920	0.9865	0.9753
2C	400	0.9260	1.0000	0.9950	0.9675	0.9780	0.9795	0.9743
2D	10	0.8130	0.9500	0.9860	0.9690	0.9905	0.9795	0.9480
2D	20	0.8845	0.9580	0.9955	0.9890	0.9930	0.9825	0.9671
2D	40	0.9220	0.9840	0.9980	0.9910	0.9915	0.9835	0.9783
2D	400	0.9285	1.0000	0.9950	0.9820	0.9810	0.9815	0.9780
3A	10	0.8165	0.9530	0.9840	0.9580	0.9970	0.9935	0.9503
3A	20	0.8840	0.9500	0.9950	0.9750	0.9975	0.9955	0.9662
3A	40	0.9240	0.9750	0.9985	0.9790	0.9970	0.9950	0.9781
3A	400	0.9395	0.9965	0.9970	0.9715	0.9910	0.9930	0.9814
3B	10	0.8205	0.9555	0.9840	0.9750	0.9990	1.0000	0.9557
3B	20	0.8850	0.9440	0.9955	0.9900	0.9995	1.0000	0.9690
3B	40	0.9240	0.9670	0.9975	0.9925	0.9995	1.0000	0.9801
3B	400	0.9435	0.9930	0.9970	0.9905	1.0000	1.0000	0.9873
3C	10	0.8265	0.9590	0.9865	0.9800	1.0000	1.0000	0.9587
3C	20	0.8960	0.9360	0.9930	0.9930	1.0000	1.0000	0.9697
3C	40	0.9190	0.9560	0.9975	0.9930	0.9995	1.0000	0.9775
3C	400	0.9470	0.9915	0.9965	0.9885	0.9995	1.0000	0.9872
3D	10	0.8200	0.9545	0.9840	0.8935	0.9990	0.9935	0.9408
3D	20	0.8855	0.9465	0.9950	0.9170	0.9990	0.9960	0.9565
3D	40	0.9245	0.9715	0.9980	0.9355	0.9990	0.9940	0.9704
3D	400	0.9425	0.9950	0.9950	0.9500	0.9975	0.9915	0.9786

The patterns of coverage in Table IX appear roughly comparable to those in Tables III–VI: only with configuration ‘A’ at N = 10 does the coverage consistently weaken. Indeed, a highly encouraging indication is the improved stability in coverage for Model 2A. With m = 10 doses the 2A rates now appear commensurate with patterns seen for most other models—even at the more-concave configurations—extending the indications seen in Table VI. At least as regards estimation and inferences in benchmark risk assessment, this provides strong encouragement to include larger numbers of doses for characterizing the response when designing modern dose-response studies^(22,54,55).