Efficient experimental design for dose response modelling

Abstract

The logit binomial logistic dose response model is commonly used in applied research to model binary outcomes as a function of the dose or concentration of a substance. This model is easily tailored to assess the relative potency of two substances. Consequently, in instances where two such dose response curves are parallel so one substance can be viewed as a dilution of the other, the degree of that dilution is captured in the relative potency model parameter. It is incumbent that experimental researchers working in fields including biomedicine, environmental science, toxicology and applied sciences choose efficient experimental designs to run their studies to both fit their dose response curves and to garner important information regarding drug or substance potency. This article provides far-reaching practical design strategies for dose response model fitting and estimation of relative potency using key illustrations. These results are subsequently extended here to handle situations where the assessment of parallelism and the proper dose-scale are also of interest. Conclusions and recommended strategies are supported by both theoretical and simulation results.

1. Introduction

In situations where a binary response variable is modelled using a single explanatory variable (x), the scientific literature underscores the widespread use of the logit-link binomial (LOG2) logistic model,

$$\pi =\frac{e^{\beta (x-\gamma )}}{1+e^{\beta (x-\gamma )}}$$ (1)

This literature also highlights various design strategies implemented by subject-matter researchers, with key references given in [1,12]. In this expression, $\pi =E(Y)$ represents the ‘success’ probability of the binary/binomial response variable and $\beta$ and $\gamma$ are the respective slope and LD50 (or EC50) parameters. The LD50 parameter is the dose or concentration for which the success probability is expected to be 50%, and this equation can be easily modified for other desired quantiles. In contrast, in instances where this model fails to fit on the dose scale used in Equation (1) but fits on the log-dose scale, the LL2 format of this model may be used,

$$\pi =\frac{t}{1+t}=\frac{{\left(\frac{x}{\gamma }\right)}^{\beta }}{1+{\left(\frac{x}{\gamma }\right)}^{\beta }}$$ (2)

Again the $\beta$ and $\gamma$ parameters in the LL2 model are the slope and LD50 parameters. Thus, whereas dose or concentration enters the LOG2 model on the original scale, it enters the LL2 model on the log-scale. Furthermore, as noted in [10], selecting the logistic model to fit on the proper scale is crucial; [10] also explores robust design strategies using the generalized scaled logistic (SL3) model,

$$\pi =\frac{e^{\beta (z(x)-z(\gamma ))}}{1+e^{\beta (z(x)-z(\gamma ))}}$$ (3)

Note that in Equation (3), $x>0$, $\lambda \ne 0$ is scale hyper-parameter, and $z(x)=(x^{\lambda }-1)\text{/}\lambda$. The dose scale is indicated when $\lambda =1$ (in which case the SL3 and LOG2 models coincide), and the log-dose scale is indicated when $\lambda \to 0$ (in which case the SL3 and LL2 models coincide). Our focus here is on the LL2 model since many datasets are well represented in this format. Nonetheless, our experimental design strategies are extended in section 4.3 for situations where uncertainty exists regarding the proper scale. We specifically address the situation in which two similar (i.e. parallel-curve) substances are compared by indirect quantile assay with respective LD50’s ${\gamma }_A$ and ${\gamma }_B$ for substances A and B, and with $\rho =({\gamma }_B\text{/}{\gamma }_A)$ denoting the relative potency model parameter. Figure 1, discussed in the next section, demonstrates this situation; clearly, the relative potency can be evaluated by noting the horizontal shift in the respective LD50’s. As such, the full set of model parameters here is $(\beta ,{\gamma }_A,\rho )$ since ${\gamma }_B$ is obtained by the relation ${\gamma }_B=\rho {\gamma }_A$.

Figure 1.

Fitted LL2 logistic curves for fluoranthene [18] data with separate-curve D-optimal design points (filled circles) corresponding to cut lines at $\pi =0.2274$ and $\pi =0.7726$.

To provide relevant context for our proposed methods, we provide here four key motivating examples; these illustrations are used throughout this article to illustrate typical applications and chosen designs and to demonstrate important improvements and extensions which can be realized using the following design strategies. Each example involves the use of binomial logistic regression (i.e. indirect quantile assay) for two substances/curves and a key focus in each situation is the assessment of relative potency, $\rho ={\gamma }_B\text{/}{\gamma }_A$.

• Example A (Fluoranthene). This illustration involves the design and data given in [18] in which minnow larvae mortality was assessed using explanatory variable fluoranthene concentration ( $x$) comparing low algae (curve A) with high algae (curve B) concentrations. The chosen fluoranthene concentrations were $x=5,10,20,$ and $30$ ( $\mu$g/L) for both algal curves with respective sample sizes $n_{1j}=62,62,65,63$ for low algal concentration and $n_{2j}=71,74,60,62$ for high algal concentration. Thus, with sample sizes $n_1=252$ and $n_2=267$ for low and high algae respectively, the final sample size used in the study was $n=519$ minnow larvae. Fitting the LL2 model to these fluoranthene data yield maximum likelihood parameter estimates $\hat{\beta }=4.85,{\hat{\gamma }}_A=15.30,$ and $\hat{\rho }=1.16$. In the following sections, we evaluate this design allocation choice and provide recommendations for a more efficient use of resources.

• Example B (Peptides). The neurotensin (curve A) and somatostatin (curve B) peptide mouse mortality study data given in [8,17] provide our second illustration. In contrast with the previous example, this peptide design was highly unbalanced and involved several non-overlapping peptide doses. The eight chosen neurotensin doses (and sample sizes) were $0.01(n_{11}=30)$, $0.03(n_{12}=30)$, $0.1(n_{13}=10)$, $0.3(n_{14}=10)$, $1.0(n_{15}=10)$, $3.0(n_{16}=10)$, $10(n_{17}=10)$, and $30(n_{18}=10)$ and the six somatostatin doses (and sample sizes) were $0.3(n_{21}=10)$, $1.0(n_{22}=10)$, $3.0(n_{23}=10)$, $10(n_{24}=10)$, $30(n_{25}=10)$, and $100(n_{26}=10)$. Thus, with chosen sample sizes of $n_1=120$ and $n_2=60$ for neurotensin and somatostatin, the final sample size used in the study was $n=180$ mice. Fitting the LL2 model to these peptide data yield maximum likelihood parameter estimates $\hat{\beta }=0.72,{\hat{\gamma }}_A=5.20,$ and $\hat{\rho }=5.66$.

• Example C (Insulin). Using insulin data, occurrences of collapse/convulsion are compared in [4] in mice for standard (curve A) and test (curve B) preparations. The nine standard dose levels (and sample sizes) were $3.4(n_{11}=33)$, $5.2(n_{12}=32)$, $7.0(n_{13}=38)$, $8.5(n_{14}=37)$, $10.5(n_{15}=40)$, $13.0(n_{16}=37)$, $18.0(n_{17}=31)$, $21.0(n_{18}=37)$ and $28.0(n_{19}=30)$, and the five test dose levels (and sample sizes) were $6.5(n_{21}=40)$, $10.0(n_{22}=30)$, $14.0(n_{23}=40)$, $21.5(n_{24}=35)$, and $29.0(n_{25}=37)$. Here, the chosen final sample size was $n=497$ mice, with $n_1=315$ mice in the standard group and $n_2=182$ mice in the test group. This translates to an average of $497/14=35.5$ replicates per design support point. Further, fitting the LL2 model to these insulin data yield maximum likelihood parameter estimates $\hat{\beta }=2.30,{\hat{\gamma }}_A=11.22,$ and $\hat{\rho }=1.50$.

• Example D (Budworms). Binary mortality data are provided in [2] for tobacco budworms comparing male and female moths; the design used geometric dose levels $1,2,4,8,16$ and $32$ of cypermethrin (in $\mu$g) for each gender and with $n_{ij}=20$ moths of both genders randomized to each of the chosen six dose levels. Thus, this balanced study involved $n_1=120$ male and $n_2=120$ female moths for a total sample size of $n=240$. Fitting the LL2 model to these budworms data yield maximum likelihood parameter estimates $\hat{\beta }=1.54,{\hat{\gamma }}_A=4.69,$ and $\hat{\rho }=2.05$.

The above illustrations provide examples of actual real-world designs which range from exactly or nearly balanced ( $n_1\approx n_2$) studies with identical doses for the two substance groups to unbalanced designs involving quite disparate doses. Sample sizes in these examples ranged from $10$ to $74$ replicates per design support point. Our focus here is to provide the practitioner with clear guidelines for possible improvements in terms of design balance, respective sample sizes and recommended dose levels.

As developed and illustrated in [1,11], optimal design theory entails viewing a regression design, $\xi =\left\{\begin{array}{cccc}{x}_1& x_2& \dots & x_k\\ {\omega }_1& {\omega }_2& \dots & {\omega }_k\end{array}\right\}$, as a (probability) measure which includes both the chosen dose/concentration levels (i.e. the $x_i$, $i=1\dots k$) and the corresponding non-negative design weights/proportions (i.e. the ${\omega }_i$ $i=1\dots k$). Here, the ${\omega }_i$ all lie between 0 and 1 and sum to one. As noted in [9,12], corresponding to the specific choice of binomial logistic model in Equations (1)–(3) is a specific $p\times p$ information matrix, denoted $\mathit{M}(\xi )$, where $p$ is the number of model parameters. This notation underscores that the information matrix depends upon the chosen design ( $\xi$). In regression settings, such as those considered here, the preferred optimal design method is D-optimality which involves finding a design to maximize the determinant of $\mathit{M}(\xi )$, denoted $|\mathit{M}(\xi )|$. D-optimal designs also minimized the generalized variance, $|{\mathit{M}}^{-1}(\xi )|$; important details and reasons for preferring D-optimality (over other criteria) are given in the Appendix. In conjunction with optimization methods (as discussed in [9] and [19]), results based on the General Equivalence Theorem [6] are used here to ensure that the optimal design has been obtained. Furthermore, in instances where the model parameters are partitioned into a $p_1$-dimensional subset of so-called nuisance parameters and a $p_2$-dimensional subset of parameters of interest (so that $p_1+p_2=p$), the information matrix is partitioned,

$$\mathit{M}=\left[\begin{array}{cc}{\mathit{M}}_{11}& {\mathit{M}}_{12}\\ {\mathit{M}}_{21}& {\mathit{M}}_{22}\end{array}\right]$$ (4)

In this expression, submatrix ${\mathit{M}}_{ij}$ is of dimension $p_i\times p_j,i,j=1,2$. An example is the setting in which a design is desired to provide more information for the relative potency parameter, $\rho =({\gamma }_B\text{/}{\gamma }_A)$, but with relatively less (but non-null) information sought regarding the slope and LD50 parameters. Thus, to provide a continuum of design settings and relative information, we here use $D_{\alpha }$ design objective function (which is to be maximized in choosing the design) with $0\le \alpha \le 1$,

$${\mathrm{\Phi }}_{\alpha }(\xi )=\frac{1-\alpha }{p_1}log|{\mathit{M}}_{11}|+\frac{\alpha }{p_2}log|{\mathit{M}}_{22}-{\mathit{M}}_{21}{\mathit{M}}_{11}^{-1}{\mathit{M}}_{12}|$$ (5)

Since $|\mathit{M}|=|{\mathit{M}}_{11}|\times |{\mathit{M}}_{22}-{\mathit{M}}_{21}{\mathit{M}}_{11}^{-1}{\mathit{M}}_{12}|$, D-optimal designs are then sought if we set $\alpha =p_2/p$ whereas subset designs emphasizing only the parameter(s) of interest are obtained if we select $\alpha =1$. As noted in [15], a generalized inverse can be used here for singular ${\mathit{M}}_{11}$. It follows that for this situation, the relevant range of choices for $\alpha$ is $\alpha \in [p_2\text{/}p,1]$.

In the next section – and in the context of the examples given above as well as in the larger general setting – we obtain and discuss the theoretical optimal designs for these models. Underscoring the fact that these theoretical designs often fail to meet researcher’s needs, we extend these methods and provide useful, near-optimal robust design strategies.

This article is organized as follows. In section 2, we provide and illustrate our key result involving our so-called reflection designs; we also underscore their superiority to the ad hoc (chosen) designs and to the theoretical optimal designs. We illustrate these methods and results using several of the above examples in section 3. In section 4, we extend our key reflection design results to incorporate geometric and uniform designs as well as uncertainty regarding parallelism and/or scale. Our findings are underscored and validated by simulation studies in section 5, and our final recommendations are given in section 6. To provide ease of readability, technical/detailed results related to information matrices, variance-covariance matrices, key determinants, D-optimality and A-optimality results and illustrations are given in the Appendix (see Supplemental Material) for the design criteria developed here.

2. A novel robust design approach using reflection designs

Optimal design theory techniques have been developed to improve information regarding binomial logistic regression model parameters [see 1,13] We apply these techniques here to the above settings using the provided examples as pilot studies and from which reasonable preliminary model parameter estimates can be provided. As observed below, direct application of classical optimal design techniques, although optimal in the sense of information obtained, often fail to provide useful designs to the practitioner. In developing so-called reflection designs (below), we therefore extend these classical design methods to more fully address these practical concerns in combination with the theoretical (variance minimization) objectives.

By way of illustration, we use the parameter estimates obtained from fitting the LL2 logistic model in Equation (2) (with relative potency parameter to connect the curves) to the fluoranthene pilot data set given previously in Example A. This application provides the D-optimal design for the three model parameters ( $\beta ,{\gamma }_A,\rho$) comprising equal-weight (all ${\omega }_i=1\text{/}4$) design points (i.e. fluoranthene concentration) of $x=11.894$ and $x=19.691$ for low algae (curve A) and $x=13.847$ and $x=22.924$ for high algae (curve B). This design is shown in Figure 1 where the fitted low-concentration and high-concentration LL2 logistic curves are also displayed. Note that these design support points are the abscissas of the intersection points of the percentage cut lines $\pi =0.2274$ (i.e. $t=0.2944$) and $\pi =1-0.2274=0.7726$ (i.e. $t=1/0.2944=3.3971$) with the two fitted logistic curves. To distinguish this design from the reflection design technique we introduce and illustrate below and since the optimal design point occur on the individual curves, we call this the separate-curves D-optimal design.

Comparing the original design in [18], denoted ${\xi }_W$, with the (separate-curves) D-optimal design given above ( ${\xi }_D$), is done using the D-efficiency,

$$D_{EFFIC}={\left[\frac{|\mathit{M}({\xi }_W)|}{|\mathit{M}({\xi }_D)|}\right]}^{1/p}$$ (6)

In Equation (6), p is the number of model parameters ( $p=3$ in this illustration). Since information matrices are positive-definite and the D-optimal designs maximize the determinant-information, $D_{EFFIC}$ lies between $0$ and $1$; nearness to unity/100% is used to convey minimal information loss viz-a-viz – or proximity to – the D-optimal design. As highlighted in [1], since $D_{EFFIC}$ in Equation (6) is on a per-observation basis, to achieve the information derived from a study with sample size of $n_D$ and the D-optimal design, the sample size $n=n_D/D_{EFFIC}$ is required for a study/design with D-efficiency of $D_{EFFIC}\in (0,1)$. The D-efficiency of the design used in [18] is only $60.1\mathrm{\%}$, meaning that on a per-information basis, the authors’ chosen design and sample size of $n=519$ is equivalent to the above (separate-curves) D-optimal design with sample size of $n=312$ minnow larvae. This latter design would translate to $78$ replicates of each of the four design point concentrations. Furthermore, we note that D-optimality corresponds here to $\alpha =1\text{/}3$ in the optimality criterion in Equation (5). For other choices of $\alpha$ in the range from $1\text{/}3$ to 1, $D_{\alpha }$ designs are then obtained by finding corresponding symmetric percentage cut-lines. With the exception in the limit where $\alpha \to 1$, these designs yield two-point designs per curve analogous to the D-optimal design shown in Figure 1. Furthermore, same-concentration designs, which for this situation comprises only a total of two points used for both substance curves, are highly efficient; see [12]. For the above Example A setting, the same-concentration consists of the two fluoranthene concentrations $x=12.761$ and $x=21.366$ (for both low and high algal curves) and has a deficiency of 98.3%, and so represents only a minimal information loss when contrasted with the (separate-curves) D-optimal design.

We firmly underscore the point that the above theoretical design optimality findings notwithstanding, applied researchers typically desire and require designs for this two-curve logistic-relative-potency setting with more than two design points per curve. Coupled with the ease with which they are implemented by practitioners, this leads us to introduce and explore here designs which we call reflection designs. This is portrayed in Figure 2 for the fluoranthene example [18]. Reflections designs are easily constructed by expanding the $D_{\alpha }$-optimal design (which include D-optimality for $\alpha =p_2\text{/}p$) by using each of the (separate-curves for curve A and B) design support points for both curves, often with equal weights. For the fluoranthene illustration, the design support points are the abscissas of the points labelled $A_1-A_4$ (and $B_1-B_4$) in Figure 2.

Figure 2.

Fitted LL2 logistic curves for fluoranthene [18] data with reflection D-optimal design points (filled circles) obtained by using cut lines at $\pi =0.2274$ and $\pi =0.7726$ and including all concentrations for both curves.

To provide the details of general $D_{\alpha }$-optimal designs and characteristics of the corresponding reflection designs, we introduce the following additional notation and results. The first step is our conjecture that (separate-curves) $D_{\alpha }$-optimal designs for the LL2 logistic model given in Equation (2) constitutes two reciprocal values $t^{\ast }$ and $1\text{/}{t}^{\ast }$ with $0<t^{\ast }<1$ for both curves and with total weight $z$ placed on curve A and total weight $(1-z)$ placed on curve B with $0<z<1$. This conjecture is then established by using the above-referenced General Equivalence Theorem by ensuring that the corresponding variance function does not exceed a certain constant; additional details are given in the Appendix. In this case, our results establish that equal weighting (i.e. $z=1\text{/}2$) is optimal regardless of the chosen value of $\alpha$ within the range $\alpha \in ((p_2\text{/}p),1)$. Further, as demonstrated in the Appendix, the optimal $t^{\ast }\in (0,1)$ is the root of the equation,

$$1+t+\frac{1}{1-\alpha }(1-t)\log (t)=0$$ (7)

The cut lines in Figures 1 and 2 are then $\pi =t^{\ast }\text{/}(1+t^{\ast })$ and $\pi =1\text{/}(1+t^{\ast })$, and these cut lines are in turn used to find the corresponding reflection design points. Also, for subset designs (i.e. $\alpha =1$), we obtain $t^{\ast }=1$, (which coincides with $1\text{/}{t}^{\ast }$) and the single cut line is $\pi =1\text{/}2$; this underscores that although the subset design would be singular (only one point per curve), the reflection design is not, and could be used to estimate the model parameters. Once $t^{\ast }$ is obtained via Equation (7), since $t=(x\text{/}\gamma {)}^{\beta }$ here, algebraic results yield the coordinates of the resulting reflection design points and proportions given in Table 1.

Table 1.

Reflection design points. Note that $\varphi =({\gamma }_B\text{/}{\gamma }_A{)}^{\beta }$ and that the following percentage pairs are such that they sum to one: $({\pi }_{A1},{\pi }_{A2})$, $({\pi }_{B1},{\pi }_{B2})$, $({\pi }_{A3},{\pi }_{B4})$, $({\pi }_{A4},{\pi }_{B3})$.

Point on Figure 2	Abscissa ( $x$)	Percent ( $\pi$)
$A_1$ (obtained via lower cut line)	$x_{A1}={\gamma }_A\times (t^{\ast }{)}^{1/\beta }$	${\pi }_{A1}={\displaystyle \frac{t^{\ast }}{1+t^{\ast }}}$
$A_2$ (obtained via upper cut line)	$x_{A2}={\displaystyle \frac{{\gamma }_A}{{(t^{\ast })}^{1/\beta }}}$	${\pi }_{A2}={\displaystyle \frac{1}{1+t^{\ast }}}$
$B_1$ (obtained via lower cut line)	$x_{B1}={\gamma }_B\times (t^{\ast }{)}^{1/\beta }$	${\pi }_{B1}={\displaystyle \frac{t^{\ast }}{1+t^{\ast }}}$
$B_2$ (obtained via upper cut line)	$x_{B2}={\displaystyle \frac{{\gamma }_B}{{(t^{\ast })}^{1/\beta }}}$	${\pi }_{B2}={\displaystyle \frac{1}{1+t^{\ast }}}$
$A_3$ (reflection point)	$x_{A3}=x_{B1}$	${\pi }_{A3}={\displaystyle \frac{\varphi t^{\ast }}{1+\varphi t^{\ast }}}$
$A_4$ (reflection point)	$x_{A4}=x_{B2}$	${\pi }_{A4}={\displaystyle \frac{\varphi }{\varphi +t^{\ast }}}$
$B_3$ (reflection point)	$x_{B3}=x_{A1}$	${\pi }_{B3}={\displaystyle \frac{t^{\ast }}{\varphi +t^{\ast }}}$
$B_4$ (reflection point)	$x_{B4}=x_{A2}$	${\pi }_{B4}={\displaystyle \frac{1}{1+\varphi t^{\ast }}}$

It is certainly hoped that in order for these reflection designs to be useful, the efficiency measure given in Equation (6) (i.e. the ‘nearness’ to the optimal design) should be reasonably high – say, above 90%. Also, Equation (5) highlights that $D_{\alpha }$-optimal designs optimize a convex combination of information of the nuisance parameters (i.e. the $|{\mathit{M}}_{11}|$ term) and orthogonal information regarding the parameter(s) of interest (i.e. the $|{\mathit{M}}_{22}-{\mathit{M}}_{21}{\mathit{M}}_{11}^{-1}{\mathit{M}}_{12}|$ term). Thus, a D-subset efficiency measure comparing two designs ( ${\xi }_1$ and ${\xi }_2$) can be obtained by replacing $|\mathit{M}|$ and $p$ in Equation (6) with $|{\mathit{M}}_{22}-{\mathit{M}}_{21}{\mathit{M}}_{11}^{-1}{\mathit{M}}_{12}|$ and $p_2$ respectively. Denoting subset efficiency by $D{S}_{EFFIC}$, this measure is

$$D{S}_{EFFIC}={\left[\frac{|{\mathit{M}}_{22}({\xi }_1)-{\mathit{M}}_{21}({\xi }_1){\mathit{M}}_{11}^{-1}({\xi }_1){\mathit{M}}_{12}({\xi }_1)|}{|{\mathit{M}}_{22}({\xi }_2)-{\mathit{M}}_{21}({\xi }_2){\mathit{M}}_{11}^{-1}({\xi }_2){\mathit{M}}_{12}({\xi }_2)|}\right]}^{1/p_2}$$ (8)

In general, and certainly in the case of the LL2 logistic model setting here, as attention shifts away from D-optimality (i.e. $D_{\alpha }$-optimal designs with $\alpha =p_2\text{/}p$) to $D_S$-optimality (i.e. subset designs, $D_{\alpha }$-optimal designs with $\alpha =1$), $D_{EFFIC}$ generally decreases and $D{S}_{EFFIC}$ increases, and vice versa. As such, a balance should be met between these conflicting objectives. To illustrate, these methods are applied to several of the key examples in the next section.

3. Application of the robust reflection design strategy to example datasets

To illustrate the above novel robust design strategy, $D_{\alpha }$- reflection designs are provided here in the context of the first three examples presented and discussed in Section 1. To underscore their superiority, these designs are characterized and compared with the original (chosen) designs in terms of efficiency measures and estimated variances. In all illustrations, we use the original design and parameter estimates obtained using maximum likelihood estimation (for generalized nonlinear models) and the NLMIXED procedure in SAS and ‘gnm’ package in R/Studio; see [9]. We summarize the section with key general conclusions.

• Example A (Fluoranthene). As noted previously, the chosen design in [18] is a reflection design with fluoranthene concentrations $x=5,10,20,$ and $30$ ( $\mu$g/L) and with nearly equal weights (i.e. with approximately $519/8\approx 65$ replicates per design point). Nonetheless, this design is highly inefficient vis-à-vis the D-optimal design since its D-efficiency is only 60.1% and based also upon measures discussed below. Furthermore, with only two support points on each curve, the (separate-curve) D-optimal design (i.e. $D_{\alpha }$-optimal with $\alpha =1\text{/}3$), plotted in Figure 1 is impractical in that it provides little or no ability to test for lack of fit of the assumed model. On the other hand, the $D_{\alpha }$- reflection design (also with $\alpha =1\text{/}3$), displayed in Figure 2, provides extra support points and is highly efficient since its D-efficiency is found to be 97.1%. Here, with $\varphi =2.09$ (see Table 1), this reflection design comprises standard $(x,\pi )$ points $A_1(11.89,0.2274)$ and $A_2(19.69,0.7726)$ on curve A (low algae) and $B_1(13.85,0.2274)$ and $B_2(22.92,0.7726)$ on curve B (high algae), and (additional) reflection points $A_3(13.85,0.3810)$ and $A_4(22.92,0.8766)$ on curve A and $B_3(11.89,0.1234)$ and $B_4(19.69,0.6190)$ on curve B. This reflection design provides a reasonable range of response proportions (between $12.3\mathrm{\%}$ and $87.7\mathrm{\%}$) and is very straight-forward to implement. Note that given these D-efficiencies, to achieve the same level of information as that from the original design with $n=519$ requires a sample size of only approximately $n=320$ for the reflection design; this translates to just $40$ replicates for each of the eight design points given here and shown in Figure 2. We also note that on a per-observation basis, the estimated variances associated with the estimated model parameters ( $\hat{\beta },{\hat{\gamma }}_A,\hat{\rho }$) are $(89.57,113.27,1.31)$ for the (separate-curve) D-optimal design, $(95.75,115.27,1.34)$ for the reflection design, and $(86.89,267.86,2.87)$ for the chosen design [18]. In comparing these values, note that the D-optimal and reflection designs produce similar results in terms of parameter estimation. Also, although the chosen design [18] is slightly more efficient (i.e. slightly reduced estimated variance) for the slope parameter, it is highly inefficient in estimating the LD50 and relative potency parameters (i.e. with variance terms increasing by a factor of more than two). In sum, the reflection design’s near-optimal concentration selection and sample size reduction observed here typically represents a substantial cost savings in situations of constrained experimental budgets. Additional empirical comparisons of these designs are given in Section 5.

• Example B (Peptides). The peptide mouse mortality study [8,17] is an unbalanced study with $n_A=120$ mice randomized to a dose of neurotensin (with a dose level chosen from $0.01,0.03,0.1,0.3,1.0,3.0,10,30$ $\mu$g) and $n_B=60$ mice randomized to a dose of somatostatin (with a dose level chosen from $0.3,1.0,3.0,10,30,100$ $\mu$g). For these data and the LL2 logistic potency model, the estimated LD50’s are ${\hat{\gamma }}_A=5.20$ (neurotensin) and ${\hat{\gamma }}_B=29.47$ (somatostatin); thus, the estimated relative potency is $\hat{\rho }=5.66$. Notwithstanding the large estimated relative potency, the design techniques discussed here are helpful to obtain efficiency improvements. Here, in addition to assessing this chosen design and providing suggestions for improvement by using reflection designs, we demonstrate the effect of $\alpha$ in Equation (5) on the resulting $D_{\alpha }$-optimal designs of choice. The chosen peptide design is found to be highly inefficient since its’ respective efficiencies in Equations (6) and (8) are only $D_{EFFIC}=68.0\mathrm{\%}$ and $D{S}_{EFFIC}=43.9\mathrm{\%}$. Plotted in Figure 3 are the fitted LL2 logistic curves for the two peptides as well as the (separate-curve) $D_{\alpha }$-optimal designs with $\alpha =1\text{/}3$ (i.e. D-optimal) and with $\alpha =0.75$ (i.e. the $D_{0.75}$-optimal design). The D-optimal design corresponds to the two equal-weight design points on the respective curves using cut lines $\pi =0.2274$ and $\pi =0.7726$, and the $D_{0.75}$-optimal design corresponds to the two equal-weight design points using the cut lines $\pi =0.3269$ and $\pi =0.6731$. This shift in support points demonstrates the impact on the design of shifting $\alpha$. As noted previously, as $\alpha$ approaches 1 (i.e. the subset design for $\rho$), the cut lines approach the single cut line at $\pi =1/2$ and the single design points $x={\hat{\gamma }}_A=5.20$ and $x={\hat{\gamma }}_B=29.47$ for the individual curves. The estimated (per-observation) variances for the parameter estimates ( $\hat{\beta },{\hat{\gamma }}_A,\hat{\rho }$) are $(1.99,589.0,1395.6)$ for the D-optimal design and $(4.56,470.3,1114.4)$ for the $D_{0.75}$-optimal design. Further, the efficiencies are $D_{EFFIC}=100\mathrm{\%}$ and $D{S}_{EFFIC}=70.3\mathrm{\%}$ for the D-optimal design, and $D_{EFFIC}=88.2\mathrm{\%}$ and $D{S}_{EFFIC}=88.0\mathrm{\%}$ for the $D_{0.75}$-optimal design. These results demonstrate that as $\alpha$ shifts closer to $1$ in the $D_{\alpha }$-optimal criterion function, designs are obtained with increased subset efficiency (precision in estimating the relative potency parameter) at the expense of less precision in estimating other parameters (such as the slope parameter in this case). The practitioner is thus faced with choosing $\alpha$ for the specific study needs so that the optimal trade-off is achieved. Related additional empirical results and suggestions are provided in Section 5. As noted above and in Figure 3, both the D-optimal and the $D_{0.75}$-optimal designs outperform the chosen peptide study design in terms of precision and efficiency. Since our goal here is to obtain and demonstrate practical robust designs (i.e. additional support points per curve), we now consider the corresponding reflection designs. These reflection designs are obtained from the respective designs plotted in Figure 3 in the usual manner of using the same four concentrations for both peptides. With efficiencies of $D_{EFFIC}=92.7\mathrm{\%}$ and $D{S}_{EFFIC}=63.3\mathrm{\%}$ for the reflection D-optimal design exceeding both efficiency measures for the $D_{0.75}$-optimal design (viz, $D_{EFFIC}=87.4\mathrm{\%}$ and $D{S}_{EFFIC}=62.2\mathrm{\%}$), the reflection D-optimal design is indicated as preferred here. Furthermore, with a modest information loss (7.3%) viz-a-viz the D-optimal design and with additional design support points, this reflection D-optimal design is clearly favoured. To achieve the same total information as the $180$-point design used in [8], this reflection D-optimal design requires a sample size of only $n=132$, or an average of $16.5$ replicates per design point.

• Example C (Insulin). The mouse insulin study resulted in a reasonably high D-efficiency of $92.7\mathrm{\%}$ (per observation). However, it involved an unbalance study with $14$ support points, no repeat design support points across the standard and test preparations (which can be problematic to implement in practice) and was relatively large (i.e. $n=497$). It is desired to find instead a robust, reflection (i.e. with the same dose levels across both curves), efficient design which can be implemented possibly with a smaller size. We therefore seek a design with higher total efficiency. In this setting, the D-optimal reflection design, which comprises insulin dose levels of $x=6.59,9.87,19.10$ and $28.62$ for both preparations and possesses a (per observation) D-efficiency of $95.6\mathrm{\%}$. This simple-to-implement, four-support-point reflection design is clearly preferred and is thus recommended. The same total level of information as in the original design is obtained with $n_{ij}=60$ replicates at each design point. Should $n_i\approx 36$ be taken instead – as in the original design – the efficiency gain would be over $70\mathrm{\%}$ (i.e. $497\text{/}288=1.73$).

Figure 3.

Peptides Neurotensin (Curve A, left) and Somatostatin (Curve B, right) [8,17]. The cut lines at $\pi =0.2274$ and $\pi =0.7726$ correspond to the $D_{\alpha }$-optimal design with $\alpha =1/3$ (i.e. the D-optimal design). The cut lines at $\pi =0.3269$ and $\pi =0.6731$ correspond to the $D_{\alpha }$-optimal design with $\alpha =0.75$. Note that concentration is plotted on the log-scale due to the large separation of these curves.

For the LOG2, LL2 and SL3 logistic models in Equations (1)–(3) and the two-curve relative potency situation considered here, reflection designs typically result in four design support points for each group/curve. Indeed, for some specific choices of $\alpha$ used in Equation (5), reflection designs with only three support points can result, but these situations are indeed rare. Thus, for the fluoranthene concentration illustration in Example A and Figures 1 and 2, the reflection design for the choice $\alpha =0.933$ comprises only three support points; for all other (non-subset) situations, however, the reflection design is made up of four support points per curve.

To summarize the key results of these illustrations, it cannot be overstressed that with more than two concentration/dose support points per curve, the reflection designs introduced and illustrated here provide the practitioner designs with the ability to test for goodness of fit of the assumed model. Mindful that all assumed models are merely approximations of underlying physical processes, this ability is indeed paramount. As such, the illustrations given here demonstrate that $D_{\alpha }$-optimal (including D-optimal) reflection designs are generally highly efficient and practical across an array of settings. In the following section, we provide important extensions to this robust design strategy to address common additional experimental design and modelling concerns.

4. Extensions of the basic reflection design methodology

Here we provide key extensions to the above robust reflection optimality strategy to allow for geometric and uniform designs as well as to reflect uncertainty regarding the required parallelism or the proper dose/concentration scale.

4.1. Reflection designs with geometric and uniform design structure

In general settings beyond the relative potency logistic setting considered here, [10] introduces and explores geometric and uniform robust-optimal design strategies. Here, we apply these techniques to examine geometric and uniform reflection designs for the assessment of relative potency of two substances. In this context and for logistic models in Equations (1)–(3), we write geometric designs using the notation

$$x_1=a,x_2=ab,x_3=a{b}^2,\dots ,x_{K+1}=a{b}^K$$ (9)

We write uniform designs using the notation

$$x_1=A,x_2=A+B,x_3=A+2B,\dots ,x_{K+1}=A+KB$$ (10)

These designs are chosen whereby equal weights are assigned to each design support point, and all points are chosen so they lie within any relevant constrained design region. The value of $K\ge 1$ is often chosen in consultation with and/or consideration of the subject-matter practitioner; thus, $K$ is typically selected after viewing derived geometric and/or uniform designs and efficiencies for several meaningful, reasonable choices. In keeping with the findings from the last section, in this two-substance context, we choose these designs using the reflection (i.e. same concentration) manner for both curves. To note similarities and/or to provide meaningful contrasts with the previous examples, these approaches are exemplified and discussed using several of the key introductory illustrations.

• Example D (Budworms). The chosen tobacco budworm design is indeed a geometric reflection design. Employing the notation of Equation (9) it is such that $(a,b,K)=(1,2,5)$, and so the selected dose levels are $x=1,2,4,8,16,32$. The chosen design is balanced ( $n_i=120$ for both male and female moths) and with a total sample size of $n=240$. Using the parameter values associated with these data, the D-optimal design gives optimal cypermethrin doses $2.11$ and $10.40$ for curve A (i.e. males) and optimal doses $4.33$ and $21.30$ for curve B (i.e. females). Our reflection D-optimal design includes all four of these dose levels for both curves and yields the efficiency values of $D_{EFFIC}=94.1\mathrm{\%}$ and $D{S}_{EFFIC}=65.6\mathrm{\%}$. In contrast, the efficiencies associated with the original budworm design are $D_{EFFIC}=88.4\mathrm{\%}$ and $D{S}_{EFFIC}=56.4\mathrm{\%}$. Thus, in terms of total efficiency, our reflection design represents a modest ( $6.5\mathrm{\%}$) efficiency gain over the original budworm design. This modest efficiency gain notwithstanding, since researchers often desire geometric designs, we next obtain near-optimal reflection geometric designs. Optimizing over choices of ‘ $a$’ and ‘ $b$’ in Equation (9), we obtain the reflection geometric designs and efficiencies:

  • Four-point design: $(a,b,K)=(2.1422,2.1409,3);x=2.14,4.58,9.82,21.02;$ $D_{EFFIC}=93.7\mathrm{\%}$, $D{S}_{EFFIC}=66.1\mathrm{\%}$.
  • Five-point design: $(a,b,K)=(2.0066,1.8287,4);x=2.01,3.67,6.71,12.27,22.44;$ $D_{EFFIC}=93.6\mathrm{\%}$, $D{S}_{EFFIC}=66.0\mathrm{\%}$.

  These four- and five-point reflection geometric designs underscore that: (a) this reflection geometric design also realizes similar modest information gain over the original budworm design, (b) efficiencies can indeed be realized when the geometric design parameters in Equation (9) are intentionally/optimally chosen (as here), and (c) the imposition of the geometric structure in the reflection design in this situation results in only minimal information loss as compared to the separate-curve D-optimal design.

• Example B (Peptides). With (combined) design support points of $x=0.01,0.03,0.1,0.3,1.0,3.0,10,30$ and $100$ – but using these doses differently across the two substances – the design used in the peptide mouse mortality study is a hybrid variation on a combination of geometric designs with $(a,b,K)=(0.01,10,4)$ and $(a,b,K)=(0.03,10,3)$. As noted previously, this study’s total sample size $n=180$ (mice) was unevenly broken into $n_1=120$ for neurotensin and $n_2=60$ for somatostatin. The D-efficiency of the chosen design is only $68.0\mathrm{\%}$. This D-efficiency is increased to $79.5\mathrm{\%}$ for the same (number of and spacing of the) design points but with $n_1=n_2=90$, and this fact underscores the importance of choosing balanced designs in the assessment of relative potency in parallel-curve bioassay. Further improvements are achieved by using 4-point reflection design given in Section 3, which resulted in a D-efficiency of $92.7\mathrm{\%}$. We here pursue geometric and uniform designs for settings where end-users desire such designs. The optimal four- and five-point optimal geometric designs yield $(a,b,K)=(1.0587,5.1530,3)$ and $(a,b,K)=(0.9198,3.6696,4)$. With surprisingly high respective D-efficiencies of $92.9\mathrm{\%}$ and $92.7\mathrm{\%}$, these designs are found to be at least as efficient as the reflection D-optimal design. Note that these geometric designs are practical, easy-to-implement, and represent a large cost-savings over the original design in [8]. We also note that the optimal four-point uniform design here, which yields $(A,B,K)=(1.0322,22.4967,3)$ in Equation (10), has a D-efficiency of $89.3\mathrm{\%}$ – i.e. less than the D-efficiency $92.9\mathrm{\%}$ for the optimal four-point geometric design. This is in line with our observation that in many of our explorations and findings, geometric designs out-performed uniform designs in terms of D-efficiencies for the LL2 relative potency logistic model.

• Example C (Insulin). Notable results are also obtained in the context of the mouse insulin study. The (four-point) reflection D-optimal design reported previously yielded $D_{EFFIC}=95.6\mathrm{\%}$, and the optimal five-point geometric design results in a D-efficiency of $94.5\mathrm{\%}$, and so only minor efficiency loss. With $(a,b,K)=(6.1979,1.4884,4)$, this 5-point geometric design has design support points $x=6.20,9.23,13.73,20.44$, and $30.42$, and also is easy to implement.

These illustrations demonstrate the benefit and superiority of reflection geometric and uniform designs. For given $K$, optimization (i.e. over either ‘ $a$’ and ‘ $b$’ in Equation (9) or ‘ $A$’ and ‘ $B$’ in Equation (10)) is typically straightforward using standard nonlinear constrained optimization software such as given and discussed in [9] and [19]. Often, one may need to impose the constraint that the first and last design points need to lie in the interval $(L,U)$ with $L<U$. Since these designs are ordered $x_1<x_2<\dots <x_{K+1}$, this translates into linear and nonlinear constraints, $L<a$ and $a{b}^K<U$ for geometric designs and $L<A$ and $A+KB<U$ for uniform designs. For purposes of finding optimal reflection geometric and uniform (constrained) designs, these designs can be obtained using the suite of ‘NLP’ modules in SAS’ IML (matrix) procedure. Sample code is given in [9], and additional details are given in [19].

4.2. Reflection designs incorporating parallelism uncertainty

Our focus to this point has been to provide efficient design strategies for assessing relative potency in logistic regression modelling situations in which the practitioner is certain that the two dose–response curves are parallel. To address occasional exceptional instances, we now provide efficient design strategies for assessing relative potency where potential uncertainty exists regarding the parallelism of the dose response curves. Key references and strategies for choosing designs to solely assess parallelism are given in [12,14]. We now modify the assumed curves in Equations (1)–(3) to permit two slope parameters ( ${\beta }_A$ and ${\beta }_B$), one for each curve, so the combined model now has four parameters. We highlight that the expectation here is that the difference between these slopes, $\delta ={\beta }_B-{\beta }_A$, is zero, or that the substances or compounds are expected to be ‘similar’. In this case, the four-parameter model vector, $({\beta }_A,{\gamma }_A,\rho ,\delta )$, includes the constraints, ${\gamma }_B=\rho {\gamma }_A$ and ${\beta }_B={\beta }_A+\delta$. As demonstrated below, the geometric, uniform and general reflection D-optimal design strategies given above are easily adapted to handle this four-parameter situation. The $D_{\alpha }$ design criterion in Equation (5) is modified by writing the information matrix for the parameter vector $({\beta }_A,{\gamma }_A,\rho ,\delta )$ as

$$\begin{array}{rl}\mathit{M}& =\left[\begin{array}{ccc}{\mathit{M}}_{11}& {\mathit{M}}_{12}& {\mathit{M}}_{13}\\ {\mathit{M}}_{21}& {\mathit{M}}_{22}& {\mathit{M}}_{23}\\ {\mathit{M}}_{31}& {\mathit{M}}_{32}& {\mathit{M}}_{33}\end{array}\right]=\left[\begin{array}{cc}{\mathit{M}}_A& {\mathit{M}}_{A3}\\ {\mathit{M}}_{3A}& {\mathit{M}}_{33}\end{array}\right]\text{with}{\mathit{M}}_A=\left[\begin{array}{cc}{\mathit{M}}_{11}& {\mathit{M}}_{12}\\ {\mathit{M}}_{21}& {\mathit{M}}_{22}\end{array}\right]\end{array}$$ (11)

Here, ${\mathit{M}}_{3A}=\left[\begin{array}{cc}{\mathit{M}}_{31}& {\mathit{M}}_{32}\end{array}\right]$ and ${\mathit{M}}_{A3}^T=\left[\begin{array}{cc}{\mathit{M}}_{13}& {\mathit{M}}_{23}\end{array}\right]$, and submatrices ${\mathit{M}}_{ij}$ are of dimension $p_i\times p_j,i,j=1,2,3$. In this specific application, the partition of the parameter vector for ${\mathit{M}}_{11},{\mathit{M}}_{22}$ and ${\mathit{M}}_{33}$ is $({\beta }_A,{\gamma }_A)$, $(\rho )$ and $(\delta )$ respectively, so that $p_1=2,p_2=p_3=1$ and $p=4$. Our relevant $D_{{\alpha }_1,{\alpha }_2}$ design objective function, with constraint ${\alpha }_1+{\alpha }_2+{\alpha }_3=1$, is

$$\begin{array}{rl}{\mathrm{\Phi }}_{{\alpha }_1,{\alpha }_2}(\xi )& =\frac{{\alpha }_1}{p_1}\log |{\mathit{M}}_{11}|+\frac{{\alpha }_2}{p_2}\log |{\mathit{M}}_{22}-{\mathit{M}}_{21}{\mathit{M}}_{11}^{-1}{\mathit{M}}_{12}|\\ & +\frac{{\alpha }_3}{p_3}\log |{\mathit{M}}_{33}-{\mathit{M}}_{3A}{\mathit{M}}_A^{-1}{\mathit{M}}_{A3}|\end{array}$$ (12)

Equation (12) is the counterpart of Equation (5). Here, $|{\mathit{M}}_{11}|$ captures information in the design regarding $({\beta }_A,{\gamma }_A)$, $|{\mathit{M}}_{22}-{\mathit{M}}_{21}{\mathit{M}}_{11}^{-1}{\mathit{M}}_{12}|$ captures information regarding $(\rho )$ orthogonal to $({\beta }_A,{\gamma }_A)$, and $|{\mathit{M}}_{33}-{\mathit{M}}_{3A}{\mathit{M}}_A^{-1}{\mathit{M}}_{A3}|$ captures information regarding the hyper-parameter $(\delta )$ orthogonal to $({\beta }_A,{\gamma }_A,\rho )$.

Since $|\mathit{M}|=|{\mathit{M}}_{11}|\times |{\mathit{M}}_{22}-{\mathit{M}}_{21}{\mathit{M}}_{11}^{-1}{\mathit{M}}_{12}|\times |{\mathit{M}}_{33}-{\mathit{M}}_{3A}{\mathit{M}}_A^{-1}{\mathit{M}}_{A3}|$, weights $({\alpha }_1,{\alpha }_2,{\alpha }_3)=(p_1\text{/}p,p_2\text{/}p,p_3\text{/}p)$ – i.e. $(1\text{/}2,1\text{/}4,1\text{/}4)$ here – correspond to D-optimality for the full parameter vector, $({\beta }_A,{\gamma }_A,\rho ,\delta )$. Additionally, weights $({\alpha }_1,{\alpha }_2,{\alpha }_3)=(p_1\text{/}(p_1+p_2),p_2\text{/}(p_1+p_2),0)$ – i.e. $(2\text{/}3,1\text{/}3,0)$ here – correspond to D-optimality for the key (subset) parameter vector of interest, $({\beta }_A,{\gamma }_A,\rho )$. Thus, to enable convex combinations between these points, we explore designs of the form with $({\alpha }_1,{\alpha }_2,{\alpha }_3)=((p_1\text{/}(p_1+p_2))\times (1-a),(p_2\text{/}(p_1+p_2))\times (1-a),a)$ for $a\in [0,p_3\text{/}p]$. Then, $a=p_3\text{/}p$ focuses on $({\beta }_A,{\gamma }_A,\rho ,\delta )$ and $a=0$ focuses on $({\beta }_A,{\gamma }_A,\rho )$. An exception to this structure corresponds to subset designs focusing on the relative potency parameter $(\rho )$, and for which the relevant pattern is $({\alpha }_1,{\alpha }_2,{\alpha }_3)=(0,1-a,a)$ for $a\in (0,p_3\text{/}p]$. Finally, to assess the efficiencies of one design ( ${\xi }_1$) relative to another ( ${\xi }_2$), we have two efficiency measures. One is $D{4}_{EFFIC}$ which uses all four model parameters and the full information matrix $\mathit{M}$ from Equation (11) and $p=4$ in Equation (6). The other is $D{3}_{EFFIC}$ which uses the three key model parameters and the information matrix ${\mathit{M}}_{\mathit{A}}$ from Equation (11) and $p=3$ in Equation (6).

In Section 2, we used Equation (7) to find optimal $t^{\ast }$ values and design points for the LL2 model with 3-parameters $(\beta ,{\gamma }_A,\rho )$. Here, for the LL2 model and 4-parameter-vector, $({\beta }_A,{\gamma }_A,\rho ,\delta )$, our results given in the Appendix demonstrate that the $D_{{\alpha }_1,{\alpha }_2}$ design criterion in Equation (12) involves equal weighting (i.e. $z=1\text{/}2$) and optimal $t^{\ast }\in (0,1)$ which is the root of the equation

$$1+t+\frac{1}{{\alpha }_1+2{\alpha }_3}(1-t)\log (t)=0$$ (13)

The derived cut lines associated with Equation (13) are again $\pi =t^{\ast }\text{/}(1+t^{\ast })$ and $\pi =1/(1+t^{\ast })$, and we underscore that the calculations in Table 1 also apply here. These cut lines are then used to find the reflection points to obtain our extended reflection designs. Interestingly, the optimal subset values of $t^{\ast }$ and $\pi$ here as $a\to 0^{+}$ are as for the 3-parameter case (i.e. $\pi =1\text{/2}$). On the other hand, the full four-parameter D-optimal design yields optimal $t^{\ast }$ value of $t^{\ast }=0.2137$ and is such that $\pi =0.1760$ and $\pi =1-0.1760=0.8240$. This is in contrast with $\pi =0.2274$ and $\pi =1-0.2274=0.7726$ for the D-optimal design in the 3-parameter (i.e. parallel) case given above and plotted in Figure 2.

To again note similarities and/or to provide meaningful contrasts with the previous example, these approaches are exemplified and discussed using a key introductory illustration. Indeed, the following illustration demonstrates that the extension of $D_{\alpha }$-optimal, reflection, geometric and uniform design methodology is straightforward. The resulting design highlights the fact that reflection design here use $\pi =0.1760$ and $\pi =0.8240$ instead of $\pi =0.2274$ and $\pi =0.7726$.

• Example A (Fluoranthene). For the fluoranthene study [18], we now consider the situation in which it is felt that the curves are likely parallel but that some uncertainty exists regarding this equal-slopes assumption. We implement here the 4-parameter $D$-optimal design criterion (i.e. $D_{{\alpha }_1,{\alpha }_2}$-optimality with $({\alpha }_1,{\alpha }_2,{\alpha }_3)=(1\text{/}2,1\text{/}4,1\text{/}4)$) and obtain our corresponding reflection design and assess its efficiency. This design is plotted in Figure 4 where the four design points are labelled $A_1-A_4$ on curve A (low algal concentration) and $B_1-B_4$ on curve B (high algal concentration). The full (four parameter) D-optimal and reflection design corresponds to the horizontal cut lines $\pi =0.176$ and $\pi =0.824$. To contrast this design with the three-parameter $D$-optimal design, we also provide on this graph the dotted horizontal cut lines (at $\pi =0.2274$ and $\pi =0.7726$) and open circle reflection points corresponding to the three-parameter design plotted in Figure 2 and obtained using $({\alpha }_1,{\alpha }_2,{\alpha }_3)=(2\text{/}3,1\text{/}3,0)$. As observed here (and in general), the effect of including model uncertainty about parallelism (i.e. the parameter $\delta$) – and even though the researcher may feel that $\delta =0$ so that the curves are parallel – is to shift the cut lines in this graph further away from $\pi =1\text{/}2$. Thus, the points $A_1,A_3,B_1,B_3$ shift further to the left and the points $A_2,A_4,B_2,B_4$ shift further to the right. In other words, potential uncertainty regarding parallelism is addressed and reflected in the design by shifting the design points outwards so that they correspond to more extreme values of $\pi$ (further from $1\text{/}2$). This is reasonable when considering the assessment of whether two logistic curves are parallel since points over a greater range are more useful in assessing common slope. Further, since the D-efficiencies of the reflection design in this four-parameter case are $D{4}_{EFFIC}=96.3\mathrm{\%}$ and $D{3}_{EFFIC}=94.4\mathrm{\%}$, the benefits of this extended design (i.e. ease of implementation and ability to test for lack-of-fit with four support-point per curve) far exceeds the cost (efficiency loss values of only $3.7\mathrm{\%}$ or $5.6\mathrm{\%}$). As an additional note, the reflection geometric and uniform design methodologies discussed in Section 4.1 can also easily be extended to the four-parameter situation considered here. This is especially notable for the setting in [18] since the chosen design (with design points $x=5,10,20,30$) combines a hybrid geometric/uniform spacing format. For these data and this four-parameter LL2 model, the $D_{1/2,1/4}$-optimal four-point geometric design is $(a,b,K)=(10.6770,1.3375,3)$ i.e. $x=10.68,14.28,19.10,25.54$ and yields D-efficiencies of $D{4}_{EFFIC}=91.3\mathrm{\%}$ and $D{3}_{EFFIC}=93.3\mathrm{\%}$. Also, the $D_{1/2,1/4}$-optimal four-point uniform design is $(A,B,K)=(10.7567,4.7431,3)$ i.e. $x=10.76,15.50,20.24,24.99$ and yields D-efficiencies of $D{4}_{EFFIC}=90.4\mathrm{\%}$ and $D{3}_{EFFIC}=93.1\mathrm{\%}$. Both designs yield efficiencies in excess of $90\mathrm{\%}$ and are therefore preferred to the original design used in [18].

Figure 4.

D-optimal reflection design for the four-parameter LL2 model contrasted with the three-parameter LL2 model reflection design points. Reflection design points for four-parameter case are denoted by $A_1-A_4$ for curve A and $B_1-B_4$ for curve B (labelled filled circles). Dashed horizontal cut lines at $\pi =0.176$ and $\pi =0.824$ correspond to D-optimality cut line for this four-parameter model. Dotted horizontal cut lines (at $\pi =0.2274$ and $\pi =0.7726$) and open circle reflection points correspond to D-optimal design for the three-parameter model plotted in Figure 2 and added here to highlight change the outward shift when moving from the three-parameter to the four-parameter LL2 model.

As noted in [16], the term ‘relative potency’ applies in the sense that then one of the compounds can be thought of as a dilution of the other only in situations where compounds are deemed ‘similar’ (i.e. the logistic curves are parallel). Hence, parallelism ( $\delta =0$) is a sine qua non in one’s choice of experimental design. Nonetheless, it is important to recognize and include some degree of uncertainty in situations where such uncertainty does exist, and so the methods and results given here are then recommended. It is also important to note that even in these settings, design points are still chosen in accordance with the results given in Table 1. Further, for the D-optimal reflection design and the four-parameter LL2 model, we note that the lowest and highest values of $\pi$ are $\pi =0.0927$ (for reflection point $B_3$) and $\pi =0.9073$ (for reflection point $A_4$). This observation provides the general observation that, except in situations where the correct scale for the independent variable is uncertain (see Section 4.3 below), design points for logistic regression relative potency assessment should rarely be chosen with anticipated success probabilities less than approximately $10\mathrm{\%}$ or greater than $90\mathrm{\%}$.

4.3. Reflection designs incorporating scale uncertainty

As noted in Section 1, due to widespread use and applicability, our focus here has been on the relative potency LL2 logistic in Equation (2) and the inherently assumed log-dose-scale logistic model. In some situations, however, this model fits better on the dose scale or some other transformed scale (such as square-root-dose); in these cases the LOG2 model in Equation (1) or the SL3 model in Equation (3) may provide a better fit for a given researcher’s data. [10] underscores the importance of determining the correct scale and highlights the means by which any such uncertainty can be incorporated into the optimal design criteria function. Interestingly, [18] assesses relative potency for the fluoranthene data using the concentration scale (i.e. using LOG2 model in Equation (1)) even though our results demonstrate that this model exhibits lack of fit and the LL2 model (i.e. log-dose scale) in Equation (2) fits better. We therefore extend our geometric, uniform and general reflection design strategies given in Sections 2 and 4 to situations where it is felt that the relative potency LL2 logistic model in Equation (2) is appropriate but where some potential uncertainty exists about the proper scale. That is, we use here the SL3 model but with the belief that the hyper-parameter $\lambda \to 0$; we call this the SLL3 model. Similar results can be obtained for the LOG2 model and $\lambda =1$, but we do not pursue this extension here.

The full parameter vector here is $({\beta }_A,{\gamma }_A,\rho ,\lambda )$ with $\lambda \to 0$, and we use the approach and notation given in Section 4.2 and Equation (11). Here, ${\mathit{M}}_{11},{\mathit{M}}_{22}$ and ${\mathit{M}}_{33}$ correspond to $({\beta }_A,{\gamma }_A)$, $(\rho )$ and $(\lambda )$ respectively, so that $p_1=2,p_2=p_3=1$ and $p=4$. Similarly, we choose reflection $D_{{\alpha }_1,{\alpha }_2}$-optimal designs employing the criterion function in Equation (12) yielding the optimal $t^{\ast }\in (0,1]$ root value obtained via the Equation (13) for $({\alpha }_1,{\alpha }_2,{\alpha }_3)=((p_1/(p_1+p_2))\times (1-a),(p_2/(p_1+p_2))\times (1-a),a)$ and $a\in [0,p_3\text{/}p]$. Choosing $a=p_3\text{/}p$ corresponds to the full parameter vector $({\beta }_A,{\gamma }_A,\rho ,\lambda )$, $a=0$ corresponds to the parameter subset of interest $({\beta }_A,{\gamma }_A,\rho )$ and ignores the hyper-parameter $\lambda$, and $a$ indexes the convex combinations (i.e. line segment) connecting these two cases. In contrast with Section 4.2, our approach here is to choose low values of $a$ so that the parameter subset of interest is emphasized. We do so since our results indicate that when $a$ is chosen otherwise the resulting design can be highly inefficient in estimating the key $({\beta }_A,{\gamma }_A,\rho )$ parameters; see the example below for an illustration. It is also noteworthy that when one departs from the LL2 logistic model and the log-dose scale in the direction of the LOG2 or SL3 models, D-optimal designs often no longer have equal, rational weights. That is, for these models – as well as for the $D_{{\alpha }_1,{\alpha }_2}$-optimal designs considered here – as long as $a>0$ (i.e. reflecting that some uncertainty exist about the LL2 model and log-dose scale), a four-point optimal designs is still of the form $\xi =\left\{\begin{array}{cccc}{x}_1& x_2& x_3& x_4\\ {\omega }_1& {\omega }_2& {\omega }_3& {\omega }_4\end{array}\right\}$. However, even though each $0\le {\omega }_i\le 1$ and $\sum {\omega }_i=1$, the optimal design needn’t necessarily have weights ${\omega }_i=1\text{/}4$ and some ${\omega }_i$ may even be irrational numbers. This is often impractical, and even though algorithms exist [see 13] to convert these ‘approximate designs’ to ‘exact designs’ (i.e. with rational design weights), our empirical results demonstrate that restricting to exact designs does not result in a significant information loss. To again provide meaningful contrasts with above results, these concepts are demonstrated using the fluoranthene example.

• Example A (Fluoranthene). As noted previously, [18] fit the relative potency LOG2 logistic model to the fluoranthene data but our analysis demonstrates that the LL2 logistic model provides a better fit (and no lack of fit) for these data. Thus, the logistic model fits on the log-concentration scale but not on the concentration scale for these data. We now explore optimal design strategies here for the SL3 model with the belief that the hyper-parameter $\lambda \to 0$ (i.e. for our so-called SLL3 model). In this case, the D-optimal design for this four-parameter model (i.e. $D_{{\alpha }_1,{\alpha }_2}$-optimal with $({\alpha }_1,{\alpha }_2,{\alpha }_3)=(2\text{/}4,1\text{/}4,1\text{/}4)$) does a poor job of providing efficient estimation of the key parameters $({\beta }_A,{\gamma }_A,\rho )$; this is noted since although $D{4}_{EFFIC}=100\mathrm{\%}$, we underscore that $D{3}_{EFFIC}$ is only $83.6\mathrm{\%}$. Additionally, the increased variance estimates associated with the three key parameters are $9.8\mathrm{\%},24.9\mathrm{\%}$ and $25.2\mathrm{\%}$ respectively for $({\hat{\beta }}_A,{\hat{\gamma }}_A,\hat{\rho })$ when compared with the subset design associated only with $({\beta }_A,{\gamma }_A,\rho )$. We therefore restrict our choice of $a$ to lower values and suggest choosing $a=0.05$ as is used here. For $({\alpha }_1,{\alpha }_2,{\alpha }_3)=(0.633,0.317,0.050)$, the $D_{{\alpha }_1,{\alpha }_2}$-optimal design comprises two support points per curve but with the optimal weights are irrational (rounded to $0.2245$ and $0.2755$). As noted above, we find such approximate design weights impractical, and underscore that the restriction here to exact designs (i.e. with weights 1/4 for each of the four points) results in only minimal information loss. This exact $D_{{\alpha }_1,{\alpha }_2}$-optimal design includes the points $(x,\pi )=(10.53,0.1403)$ and $(18.25,0.7015)$ on curve A and the points $(x,\pi )=(14.94,0.2985)$ and $(25.89,0.8597)$ on curve B, and our resulting reflection design is plotted in Figure 5. Some key points regarding this reflection design are now highlighted. First, this reflection design is reasonably efficient for the key parameters $({\beta }_A,{\gamma }_A,\rho )$ since $D{3}_{EFFIC}=92.2\mathrm{\%}$, which translates to an efficiency loss of less than 8%. Similarly, the increased variance estimates associated with the three key parameters are only $7.9\mathrm{\%},9.6\mathrm{\%}$ and $10.6\mathrm{\%}$ respectively for $({\hat{\beta }}_A,{\hat{\gamma }}_A,\hat{\rho })$ when compared with the optimal subset design. Second, optimal $\pi$ values occur in pairs which sum to one across the curves (even for the reflection points), so that $0.1403+0.8597=1$ and $0.2985+0.7015=1$. Further, the formulae in Table 1 apply here to find design support points and optimal $\pi$ values again with $\varphi =({\gamma }_B\text{/}{\gamma }_A{)}^{\beta }$. Third, as clearly noted when comparing Figures 2 and 5, $D_{{\alpha }_1,{\alpha }_2}$-optimal and associated reflection designs no longer share common $\pi$ cut lines as we shift from the LL2 model to the SLL3 model – even when it is felt that $\lambda \to 0$ (i.e. as for the LL2 model function). Our results show that the same result (i.e. non-common $\pi$s) occurs for the LOG2 model function as well. Nonetheless, this should present no problem for the practitioner in using our suggested reflection designs. Also, not surprisingly, as we adjust the $a$ value ever closer to 0, the $\pi =0.1403$ and $\pi =0.2985$ cut lines in Figure 5 converge to the $\pi =0.2274$ cut line in Figure 2, and the $\pi =0.7015$ and $\pi =0.8597$ cut lines in Figure 5 converge to the $\pi =0.7726$ cut line in Figure 2. Certainly, different situations call for different choices of the value of $a$ used here to obtain the optimal reflection design. Finally, note that for the choice of $a=0.05$ here, the range of $\pi$ values for the reflection points is from $0.0724$ to $0.9276$; this compares with the range from $0.1234$ to $0.8766$ in Figure 2 (i.e. for $a=0$).

Figure 5.

Fitted LL2 logistic curves for fluoranthene [18] data with reflection design for SLL3 model. Cut lines at $\pi =0.1403$ and $\pi =0.7015$ correspond to Curve A and cut lines at $\pi =0.2985$ and $\pi =0.8597$ correspond to Curve B. As for all reflection designs, intersection points of respective curves with cut lines (i.e. points ${\mathit{A}}_1$ and ${\mathit{A}}_2$ on Curve A and ${\mathit{B}}_1$ and ${\mathit{B}}_2$ on Curve B) are used for both substances.

The examples provided in this and the previous sections underscore the ease-of-implementation and high efficiency associated with our reflection design strategies even in settings where researchers wish to build into their modelling process robustness aspects and potential uncertainty associated with the parallelism and scale parameters. Additionally, although not explicitly provided here, the reflection optimal design setting for the SLL3 model is also easily adapted to allow for geometric and uniform designs as given in Section 4.1.

5. Reflection design simulation results

The robust reflection design techniques proposed and illustrated above possess admirable assessment and efficiency measures. We now examine their performance using computer simulations. It is underscored that, although not directly assessed here, the ability to test for model lack of fit by comprising design points in excess of the number of model parameters (i.e. two per curve here), is also essential, and a concern which should be borne in mind in choosing a final design. In the following computer simulations, we performed $K=5000$ replications (per design) using R-Studio version 1.2 software and the ‘gnm’ generalized nonlinear modelling R package. To compare the above design strategies, we based these simulations on the respective fitted LL2 relative potency logistic model parameter performance using varying sample sizes in order to assess small-, moderate- and large-sample performance. In our assessments, we compare the performance of these design methodologies by calculating, recording and contrasting:

1. the percent (of 5000) of times the nonlinear model fitting failed to converge – i.e. the percent of ‘NAs’ in model fitting

2. the proportion of times the model fit’s estimated variance of the slope parameter estimate ( $\hat{\beta }$) exceeded twice (2) the estimated slope variance for the original data set

3. the proportion of times the model fit’s estimated variance of the LD50 estimate for curve A ( ${\hat{\gamma }}_A$) exceeded twice (2) the estimated LD50 (for curve A) variance for the original data set

4. the proportion of times the model fit’s estimated variance of the relative potency estimate ( $\hat{\rho }$) exceeded twice (2) the estimated relative potency variance for the original data set

5. the proportion of times the model fit’s estimated generalized variance ( $|{\mathit{M}}^{-1}|$) exceeded five (5) times the estimated generalized variance for the original data set

To again provide meaningful comparisons, we examine here several of the examples introduced in Section 1 to illustrate these simulation results and to provide meaningful empirical findings and important lessons learned.

• Example A (Fluoranthene). Here we use the design structure and LL2 binomial relative potency (RP) logistic parameter estimates obtained for the fluoranthene concentration data (viz, $\hat{\beta }=4.85,{\hat{\gamma }}_A=15.30,\hat{\rho }=1.16;{\hat{\gamma }}_B=17.82$) to explore the simulated performances for small-to-moderate and moderate-to-large sample sizes. All design strategies considered here yielded equal weights (sample sizes) for each curve (low and high algal levels) and equal weights for all design points, so the distinguishing factor are the chosen ‘optimal’ fluoranthene concentration design points (in $\mu$g/L). Each design considered here is a reflection design (i.e. same design points used for both curves), comprised of four points per curve for the first seven designs in Table 2 and two points per curve for the last design. As noted in Table 2, the designs are: (1) the original design used in [18], (2) the D-optimal (i.e. $D_{\alpha }$-optimal with $\alpha =1\text{/}3$) design, (3) the $D_{\alpha }$-optimal design with $\alpha =0.7$ (which places more emphasis on the relative potency parameter estimation), (4) the four-point geometric design, (5) the four-point uniform design, (6) the D-optimal (i.e. $D_{{\alpha }_1,{\alpha }_2}$-optimal with $({\alpha }_1,{\alpha }_2)=(1\text{/}2,1\text{/}4)$) design including the slope parameter ( $\delta$) given in Section 4.2, (7) the $D_{{\alpha }_1,{\alpha }_2}$-optimal with $({\alpha }_1,{\alpha }_2,{\alpha }_3)=(0.633,0.317,0.050)$ including the scale parameter ( $\lambda \to 0$) given in Section 4.3, and (8) the two-point same-concentration design discussed in Section 2. For this example, each design given in Table 2 was used to simulate $K=5000$ small-to-moderate datasets (total sample size $n=120$, i.e. $n_i=60$ per curve or $n_{ij}=15$ per design point per curve for four-point designs and $n_{ij}=30$ per design point per curve for two-point designs) and $K=5000$ moderate-to-large datasets (total sample size $n=240$, i.e. $n_i=120$ per curve or $n_{ij}=30$ per design point per curve for four-point designs and $n_{ij}=60$ per design point per curve for two-point designs). The 3-parameter LL2 relative potency logistic model in Equation (2) was fit to each dataset, the above-listed five design assessment measures were calculated, and the summary results are presented in Table 3. For all designs, parameter estimation convergence was achieved for practically each of the simulations; note that the $0.12\mathrm{\%}$ NA’s (for the original design and $n=120$) translates to non-convergence in only $6$ of $5000$ runs. In terms of the variance and generalized variance ( $|{\mathit{M}}^{-1}|$) estimates, Design 1 (the original design used in [18]) performed very poorly for both $n=120$ and $n=240$, with respective excessive percentages of $16.64\mathrm{\%},81.62\mathrm{\%},54.49\mathrm{\%}$ and $43.55\mathrm{\%}$ for $n=120$ and $5.74\mathrm{\%},88.22\mathrm{\%},57.04\mathrm{\%}$ and $31.58\mathrm{\%}$ for $n=240$. This underscores the highly inefficient and inferior nature of this design and is in line with the low ( $60.1\mathrm{\%}$) D-efficiency measure reported in Section 2. Other notes of interest include the fact that, with only few exceptions, these design inefficiency assessment measures decreased with increased sample sizes; this is expected and is also observed in the examples given below. It is also clear that deviating from the D-optimal design ( $\alpha =1\text{/}3$ here) by choosing for example $\alpha =0.7$ results in poor design performance here; even for the larger sample size, excessive variation was detected for $\widehat{var}(\hat{\beta })$ in $30.8\mathrm{\%}$ of the simulations and was also (surprisingly) detected for $6.54\mathrm{\%}$ of the simulations for $\widehat{var}(\hat{\rho })$. The clear ‘winners’ here are the D-optimal reflection design (Design 2) and the D-optimal design for the four-parameter potentially nonparallel model discussed in Section 4.2 (Design 6). The geometric (Design 4), uniform (Design 5) and the four-parameter SLL3 (Design 7) designs also fared reasonably well. We also highlight that in terms of the measures used here, the more problematic parameter observed here is the relative potency parameter ( $\rho$). Thus, even for the well-performing Designs 2 and 6, the design inefficiencies for this parameter are higher than for the other parameters; these inefficiencies can be reduced, however, by increasing the sample size. Finally, the two-point same-concentration (per curve) design (Design 8) performed quite well in this situation: in addition to its 98.3% D-efficiency given in Section 2, its inefficiency measures in Table 3 are equally admirable. Nonetheless, this design is not seriously considered since, with only two points per curve, this design provides little or no ability to test for model adequacy and lack of fit.

• Example B (Peptides). The mouse mortality peptide study data yielded LL2 logistic model parameter estimates $\hat{\beta }=0.72,{\hat{\gamma }}_A=5.20,\hat{\rho }=5.66;{\hat{\gamma }}_B=29.47$ and the fitted curves for neurotensin (curve A) and somatostatin (curve B) are given in Figure 3. This example provides an interesting illustration both in the unusual original design used in [8] and in the unusually high estimated relative potency. As observed below, the resulting separation of fitted logistic curves translates into key higher design inefficiency measures and the need for ever larger sample sizes. As noted in Table 4, we again use the same Designs 1–8 as in the previous example and add the separate curves (i.e. non-reflection) D-optimal design (Design 9) to provide a benchmark for comparison purposes only. Mirroring the original design’s $n=180$ sample size, we again use $K=5000$ simulations to compare the nine designs given in Table 4 with $n=180$ (i.e. $n_i=90$ per curve with roughly even splits across the design support points) and $n=360$ (i.e. $n_i=180$ per curve with exactly equal splits across the design support points). The five design assessment measures are given in Table 5 for each design upon fitting the 3-parameter LL2 relative potency logistic model in Equation (2) to each simulated dataset/design. As expected, this simulation shows that in all instances, inefficiency measures drop as the sample size is increased. In line with the low D-efficiency of only 60.1%, the poor performance of the authors’ original design is highlighted in the first row of Table 5. Surprisingly, however, we the point out that some inefficiency measures (i.e. excessive $\widehat{var}({\hat{\gamma }}_A)$ and $\widehat{var}(\hat{\rho })$) barely dropped here with increased $n$. This clearly underscores that some designs should be avoided regardless of the anticipated sample size. Understandably, convergence again improves with increased sample size, and we again see that although the D-optimal reflection design ( $D_{\alpha }$-optimal with $\alpha =1\text{/}3$ here) perform well, $D_{\alpha }$-optimal reflection designs with other choices of $\alpha$ (such as $\alpha =0.75$ here) can do well for one measure (e.g. $\widehat{var}({\hat{\gamma }}_A)$) but should in general be avoided. The superior performers here are the D-optimal reflection design (Design 2) and the geometric design (Design 4). The uniform (Design 5), D-optimal design for the 4-parameter potentially nonparallel model (Design 6) and the 4-parameter SLL3 (Design 7) designs also performed well. We also note that the more problematic parameter is again the relative potency parameter ( $\rho$). Thus, even for the well-performing designs (Designs 2 and 4), the design inefficiencies for this parameter (proportion $\widehat{var}(\hat{\rho })>C_3$) are higher than for the other parameters; increasing the sample size promises to reduce these inefficiencies. Finally, note that the two-point same-concentration (per curve) design (Design 8) again performed quite well here as did the ‘gold-standard’ two-points-per-curve D-optimal design (Design 9). With only two points per curve, however, these designs provides little or no ability to test for goodness-of-fit and are not recommended.

• Example D (Budworms). In comparing mortality rates of male and female tobacco budworm moths, the chosen geometric dose levels of cypermethrin used in [2] are $1,2,4,8,16$ and $32$ (in $\mu$g) for each gender. With $20$ moths of both genders randomized to each of the dose levels, this study involved a total sample size of $n=240$ and resulted in a D-efficiency of $88.4\mathrm{\%}$. The fitted LL2 logistic model parameter estimates, $\hat{\beta }=1.53,{\hat{\gamma }}_A=4.69,\hat{\rho }=2.05;{\hat{\gamma }}_B=9.60$, were used here in a simulation study in conjunction with the familiar eight designs given in Table 6. Some patterns observed in the previous examples are underscored here: (1) Designs 2 (D-optimal) and 6 (D-optimal design for the 4-parameter potentially nonparallel model) again do best; (2) the original design, although simple to implement, is not the best choice here; (3) $D_{\alpha }$-optimal designs with choices of $\alpha$ other than D-optimality (i.e. $\alpha =1\text{/}3$) should be avoided; and (4) although they perform well in terms of efficiency measures, two-point (same-concentration and separate-curve D-optimal) designs should be avoided as they provide little-to-no information regarding lack-of-fit. Our results here again underscore that the relative potency parameter is the more problematic parameter in terms of estimation inefficiencies but increasing the sample size is recommended to reduce these inefficiencies (Table 7).

Table 2.

Four-point (Designs 1–7) and two-point (Design 8) reflection designs used for fluoranthene [18] simulation study.

Design	Description	Design Points
1 (original)	Original design chosen	$x=5,10,20,30$ for (algae) curves A and B
2 (R/D)	D-optimal design ( $\alpha =1/3$)	$x=11.89,13.85,19.69,22.92$ for curves A and B
3 (R/ $D(0.7)$)	$D_{\alpha }$-optimal design ( $\alpha =0.7$)	$x=12.99,15.13,18.03,20.99$ for curves A and B
4 (R/geom.)	Geometric design (4 points)	$x=11.62,14.69,18.56,23.47$ for curves A and B
5 (R/uniform)	Uniform design (4 points)	$x=11.45,15.27,19.09,22.90$ for curves A and B
6 (R/slope4)	4-parameter D-optimal (slope)	$x=11.13,12.96,21.04,24.49$ for curves A and B
7 (R/scale4)	4-parameter D-optimal (scale)	$x=10.53,14.94,18.25,25.89$ for curves A and B
8 (R/SC2point)	Same-concentration D-opt.	$x=12.76,21.37$ for curves A and B

Table 3.

Simulation results for fluoranthene [18] simulation study for designs given in Table 2. Percentages given in the body of this table correspond to total sample sizes of $n=120$ (left) and $n=240$ (right).

Design	Percent NA/5000	Proportion $\widehat{var}(\hat{\beta })>C_1$	Proportion $\widehat{var}({\hat{\gamma }}_A)>C_2$	Proportion $\widehat{var}(\hat{\rho })>C_3$	Proportion $|{\mathit{M}}^{-1}|>C_4$
1 (original)	0.12/0	0.1664/0.0574	0.8162/0.8822	0.5449/0.5704	0.4355/0.3158
2 (R/D)	0/0	0.0102/0	0.0194/0.0018	0.0538/0.0178	0.0052/0
3 (R/ $D(0.7)$)	0.04/0	0.3976/0.3080	0.0628/0.0120	0.1307/0.0654	0.0770/0.0232
4 (R/geom.)	0/0	0.0416/0.0034	0.0246/0.0018	0.0714/0.0212	0.0088/0.0008
5 (R/uniform)	0/0	0.0436/0.0038	0.0226/0.0038	0.0706/0.0198	0.0088/0.0004
6 (R/slope4)	0/0	0.0078/0.0004	0.0194/0.0020	0.0462/0.0102	0.0004/0
7 (R/scale4)	0/0	0.0560/0.0094	0.0182/0.0032	0.0624/0.0188	0.0010/0
8 (R/SC2point)	0/0	0.0066/0	0.0210/0.0024	0.0610/0.0140	0.0080/0.0004

Table 4.

Eight/Six-point non-reflection design (Design 1), four-point (Designs 2–7) and two-point (Design 8) reflection designs, and two-point non-reflection design (Design 9) used for peptide [8,17] simulation study.

Design	Description	Design Points
1 (original)	Original design chosen	$x=0.01,0.03,0.1,0.3,1,3,10,30$ (curve A), $x=0.3,1,3,10,30,100$ (curve B)
2 (R/D)	D-optimal design ( $\alpha =1/3$)	$x=0.96,5.44,28.21,159.81$ for curves A and B
3 (R/ $D(0.75)$)	$D_{\alpha }$-optimal design ( $\alpha =0.75$)	$x=1.92,10.86,14.12,79.98$ for curves A and B
4 (R/geom.)	Geometric design (4 points)	$x=1.06,5.46,28.11,144.87$ for curves A and B
5 (R/uniform)	Uniform design (4 points)	$x=1.03,23.53,46.03,68.52$ for curves A and B
6 (R/slope4)	4-parameter D-optimal (slope)	$x=0.62,3.49,43.94,248.89$ for curves A and B
7 (R/scale4)	4-parameter D-optimal (scale)	$x=0.49,8.20,18.70,315.20$ for curves A and B
8 (R/SC2point)	Same-concentration D-opt.	$x=2.05,74.91$ for curves A and B
9 (D/2pt/curve)	D-optimal for separate curves	$x=0.96,28.21$ (curve A), $x=5.44,159.81$ (B)

Table 5.

Simulation results for peptide [8,17] simulation study for designs given in Table 4. Percentages given in the body of this table correspond to total sample sizes of $n=180$ (left) and $n=360$ (right).

Design	Percent NA/5000	Proportion $\widehat{var}(\hat{\beta })>C_1$	Proportion $\widehat{var}({\hat{\gamma }}_A)>C_2$	Proportion $\widehat{var}(\hat{\rho })>C_3$	Proportion $|{\mathit{M}}^{-1}|>C_4$
1 (original)	7.2/1.2	0.1134/0.0356	0.3899/0.3780	0.4362/0.3993	0.3631/0.2916
2 (R/D)	2.8/0.2	0.0138/0.0006	0.1523/0.0976	0.3029/0.2321	0.0671/0.0168
3 (R/ $D(0.75)$)	3.0/0.4	0.4049/0.2891	0.1071/0.0462	0.3436/0.2710	0.1584/0.0665
4 (R/geom.)	2.7/0.3	0.0136/0.0004	0.1441/0.0931	0.3148/0.2285	0.0773/0.0167
5 (R/uniform)	4.0/0.3	0.1212/0.0393	0.2606/0.1952	0.3149/0.2661	0.0771/0.0199
6 (R/slope4)	3.7/0.5	0.0052/0.0002	0.2233/0.1725	0.3223/0.2537	0.0636/0.0161
7 (R/scale4)	3.2/0.7	0.0494/0.0089	0.1780/0.1269	0.3259/0.2638	0.0762/0.0213
8 (R/SC2point)	3.0/0.2	0.0080/0.0004	0.1676/0.0933	0.2972/0.2232	0.0622/0.0152
9 (D/2pt/curve)	2.3/0.1	0/0	0.1624/0.0891	0.2416/0.1666	0.0266/0.0034

Table 6.

Six-point (Design 1), four-point (Designs 2–7) and two-point (Design 8) reflection designs used for tobacco budworm [2] simulation study.

Design	Description	Design Points
1 (original)	Original design chosen	$x=1,2,4,8,16,32$ for (M/F) curves A and B
2 (R/D)	D-optimal design ( $\alpha =1/3$)	$x=2.11,4.33,10.40,21.29$ for curves A and B
3 (R/ $D(0.55)$)	$D_{\alpha }$-optimal design ( $\alpha =0.55$)	$x=2.47,5.05,8.91,18.25$ for curves A and B
4 (R/geom.)	Geometric design (4 points)	$x=2.14,4.59,9.82,21.02$ for curves A and B
5 (R/uniform)	Uniform design (4 points)	$x=2.06,6.95,11.84,16.73$ for curves A and B
6 (R/slope4)	4-parameter D-optimal (slope)	$x=1.72,3.51,12.81,26.24$ for curves A and B
7 (R/scale4)	4-parameter D-optimal (scale)	$x=1.51,5.32,8.46,29.84$ for curves A and B
8 (R/SC2point)	Same-concentration D-opt.	$x=2.91,15.48$ for curves A and B

Table 7.

Simulation results for tobacco budworm [2] simulation study for designs given in Table 6. Percentages given in the body of this table correspond to total sample sizes of $n=120$ (left) and $n=240$ (right).

Design	Percent NA/5000	Proportion $\widehat{var}(\hat{\beta })>C_1$	Proportion $\widehat{var}({\hat{\gamma }}_A)>C_2$	Proportion $\widehat{var}(\hat{\rho })>C_3$	Proportion $|{\mathit{M}}^{-1}|>C_4$
1 (original)	0.06/0	0.0364/0.0032	0.1495/0.0686	0.2357/0.1708	0.0282/0.0036
2 (R/D)	0/0	0.0292/0.0022	0.0714/0.0226	0.2112/0.1252	0.0268/0.0042
3 (R/ $D(0.7)$)	0.02/0	0.0998/0.0252	0.0614/0.0122	0.2252/0.1496	0.0644/0.0138
4 (R/geom.)	0/0	0.0406/0.0052	0.0630/0.0196	0.2000/0.1360	0.0336/0.0068
5 (R/uniform)	0.04/0	0.1034/0.0322	0.0654/0.0218	0.2309/0.1508	0.0418/0.0086
6 (R/slope4)	0/0	0.0202/0.0014	0.1262/0.0602	0.2166/0.1490	0.0126/0.0012
7 (R/scale4)	0.04/0	0.0814/0.0204	0.0856/0.0320	0.2285/0.1652	0.0236/0.0028
8 (R/SC2point)	0.02/0	0.0240/0.0014	0.0864/0.0268	0.1958/0.1056	0.0276/0.0028

These simulation results corroborate and underscore the efficiency measures used in Sections 2–4 and clearly verify that our simple-to-implement reflection D-optimal design performs very well empirically and in terms of high efficiencies.

6. Concluding remarks

Synergy studies, such as those examined in [7, 9], underscore that situations exist in which researchers may seek designs with disparate concentrations for the substances or drugs. In the assessment of relative potency of similar compounds, however, [18] points out that simultaneous estimation should occur (since the curves share a common slope parameter). As such, our advice is that common concentration design should be used. The key illustrations presented and discussed here, as well as those elsewhere [see 3,4,5], highlight that in these relative-potency-assessment situations, the reflection design strategies introduced and illustrated here can and should be used to yield improved information and efficiency. Our conclusions are based on the theoretical efficiency measures and results given here in Sections 2–4 and the empirical results provided in Section 5.

Of the various reflection design techniques introduced here for the LL2 logistic regression model in Equation (2) with model parameters $(\beta ,{\gamma }_A,\rho )$, the simplest-to-implement and consistently highly efficient (using both theoretical D-efficiency measures and empirical/simulation results) design approach is the reflection D-optimal design illustrated in Figure 2. This simplicity is underscored when –based upon previous studies or subject-matter expertise – practitioners sketch the estimated LL2 dose–response curves. The intersecting $\pi =0.2274$ and $\pi =0.7726$ horizontal cut-lines are then easily overlaid, and the indicated concentrations/doses can then be read off the plot.

In Section 4, we extended this reflection design approach to geometric and uniform designs and to designs for potentially unparallel curves or uncertain scale; in general, these latter designs can only be obtained using computer software. Furthermore, as highlighted in Section 1, purely optimal designs may possess theoretical advantages, but their inability to provide the means to assess the inherent model adequacy and assumptions is a major drawback for these designs. In contrast, that the reflection design methodology introduced and illustrated here yields designs which are highly efficient, robust (i.e. providing extra support points to test for model fit) and practical makes it an invaluable and important tool for the subject-matter expert and applied statistician.

Disclosure statement

No potential conflict of interest was reported by the author(s).