A geometric characterization of optimal designs for regression models with correlated observations

Abstract

We consider the problem of optimal design of experiments for random effects models, especially population models, where a small number of correlated observations can be taken on each individual, while the observations corresponding to different individuals are assumed to be uncorrelated. We focus on c-optimal design problems and show that the classical equivalence theorem and the famous geometric characterization of Elfving (1952) from the case of uncorrelated data can be adapted to the problem of selecting optimal sets of observations for the n individual patients. The theory is demonstrated by finding optimal designs for a linear model with correlated observations and a nonlinear random effects population model, which is commonly used in pharmacokinetics.

1 Introduction

It is a common situation in pharmacokinetic trials that only a very small number of measurements can be taken on a single patient, but a larger number of n different patients are available [Sheiner et al. (1977), Schmelter (2007), Colombo et al. (2006)]. In this situation it is impossible to reliably estimate parameters of interest for each patient. However, often these individual parameters are not of primary interest, because it is assumed that the individual parameters are realizations of some global distribution. Therefore, the main aim of the experiment is the estimation of the mean and/or variance of this distribution. This results in a random effects model and is called the population approach [Retout and Mentré (2003)]. Unfortunately, the common random effect causes measurements at different timepoints on a single patient to be correlated, therefore most of the commonly used tools of classical optimal design theory are not applicable in this context. Compared to the case of uncorrelated observations the optimal design problem for dependent data is intrinsically more difficult. Most authors used asymptotic arguments to determine efficient designs [see Sacks and Ylvisaker (1968), Bickel and Herzberg (1979), Näther (1985), Müller and Pázman (2003) among others]. In general the powerful equivalence theorem [Pukelsheim (2006)] cannot be transferred to the case of dependent data [for an equivalence theorem in the case of repeated and correlated observations at the same measurement setting see Fedorov (1972)]. Note also that geometric characterizations of optimal designs are only available for the uncorrelated case [Elfving (1952), Ford et al. (1992), Haines (1995) or Studden (2005)].

The purpose of the present paper is to derive a geometric characterization of c-optimal designs, which minimize the variance of a linear combination of the estimates of the parameters (specified by a given vector c), for models with correlated observations. Note that many commonly used objectives in the statistical analysis (as designing the experiment for the estimation of the area under the curve, the maximum concentration or, in dose finding studies, the minimal effective dose) yield optimality criteria which are special cases of the c-optimality criterion [see Atkinson et al. (1993)]. In the following sections we show that if the number of available observations is the same for each patient, the total information of all observations on a single patient, accounting for correlations, can be expressed as a sum of information matrices in the usual form for uncorrelated observations. More precisely, if m observations are available for each patient, there exist vector valued functions f̃_ℓ, ℓ = 1, …, m such that the total information matrix for the set of m observations on this patient can be written in the form

$$I=\sum _{\ell =1}^m{\stackrel{\sim }{f}}_{\ell }{\stackrel{\sim }{f}}_{\ell }^T.$$ (1.1)

For this representation we introduce for the individual observations a design space of m observations for each patient in addition to the original design space. Using this representation, we can derive an equivalence theorem for c-optimal designs using the general theory in Pukelsheim (2006) and obtain a geometric characterization of c-optimal designs for the problem of allocating the n available patients to different sets of m individual observations. As a result we obtain a generalization of the famous result of Elfving to the case of dependent data.

The theoretical details are presented in Section 2. In Section 3 we demonstrate the application of these ideas in two examples, a linear model and a basic nonlinear model taken from population pharmacokinetics. Finally, some technical details are given in an Appendix. For the sake of brevity this paper concentrates on the geometric characterization of locally c-optimal designs. However, it is worthwhile to mention that in the case of uncorrelated observations Elfving-type characterizations are also available for other optimality criteria, including D-, E- and Bayesian optimality criteria [see Dette (1993a,b) and Dette (1996)]. An interesting problem of future research is the extension of the methodology developed in the present paper to obtain geometric characterizations of optimal designs for correlated observations with respect to these criteria.

2 An Elfving representation for models with correlated observations

We begin our discussion with the case of linear models where the results are slightly more transparent. The analysis of nonlinear models can easily be reduced to this situation (see Remark 2.1), while the case of random effect models is discussed in Section 3. Assume that m observations can be taken, each on a number of n individuals in the linear model

$$Y_{\mathit{\text{ij}}}={\theta }^{T}f(x_{ij})+{\varepsilon }_{\mathit{\text{ij}}};i=1,\dots ,n,j=1,\dots ,m,$$ (2.1)

where Y_ij denotes the j–th observation on the i–th individual and x_ij is the experimental condition corresponding to this observation, which is chosen from a compact interval χ ⊂ ℝ. We use x_i = (x_i1, …, x_im) to denote all experimental conditions corresponding to the individual i. The vector θ = (θ₁, …, θ_k)^T ∈ Θ ⊂ ℝ^k is the vector of parameters to be estimated, f(x) = (f₁(x), …, f_k(x))^T denotes a vector of known functions and ε_ij denotes a random error term with expectation 0 and variance σ²(x_ij) (i = 1, …, n, j = 1, …, m) depending on x_ij. Observations on the same individual are assumed to be correlated, with corr(ε_ij, ε_ij*) = c(x_ij, x_ij*), while data corresponding to different individuals are assumed to be independent, i.e. corr(ε_ij, ε_i*j*) = 0, whenever i ≠ i*. We express the total covariance matrix of errors as the block diagonal matrix V = diag(V₁, …, V_n) ∈ (ℝ^m×m)ⁿ, with matrices

$$V_i=\mathit{\text{diag}}(\sigma (x_{i1}),\dots ,\sigma (x_{\mathit{\text{im}}})){(c(x_{\mathit{\text{ir}}},x_{\mathit{\text{is}}}))}_{r,s=1,\dots ,m}\mathit{\text{diag}}(\sigma (x_{i1})),\dots ,\sigma (x_{\mathit{\text{im}}}))\in {ℝ}^{m\times m}$$

on the diagonal. We now write F_i = (f(x_i1), …, f(x_im)) ∈ ℝ^k×m as the design matrix for individual i, i = 1, …, n and define the matrix F = (F₁, …, F_n) = (f(x₁₁), …, f(x_nm)) ∈ ℝ^k×nm as the design matrix corresponding to all patients. The information matrix (proportional to the inverse of the covariance matrix) of the generalized least squares estimate for the parameter θ can be expressed as

$$M=F{V}^{-1}{F}^T=\sum _{i=1}^n{F}_i{V}_i^{-1}{F}_i^T.$$ (2.2)

The following arguments demonstrate that this expression can be rewritten in a form closer to the usual form of information matrices, which is obtained in the case of uncorrelated observations. We begin with an alternative representation for the individual information matrices $F_i{V}_i^{-1}{F}_i^T$, i = 1, …, n. For this purpose we collect all experimental conditions corresponding to one individual in a vector x_i = (x_i1, …, x_im) ∈ ℝ^m and consider χ^m as a design space. An exact design is characterized by a tuple ${(x_i,n_i)}_{i=1}^p$, where x_i ∈ χ^m and n_i ∈ ℕ such that ${\sum }_{i=1}^p{n}_i=n$. This means that n_i of the n patients are treated under the experimental condition x_i = (x_i1, …, x_im)^T (i = 1, …, p). Our first result provides the information matrix corresponding to one observation at the experimental condition x_i.

Lemma 2.1

An information matrix of the form $F_i{V}_i^{-1}{F}_i^T$ can also be expressed as

$$F_i{V}_i^{-1}{F}_i^T=\sum _{\ell =1}^m{\stackrel{\sim }{f}}_{\ell }(x_i){\stackrel{\sim }{f}}_{\ell }{(x_i)}^T,$$ (2.3)

where the functions f̃_ℓ: χ^m → ℝ^k are defined as the columns of the k × m-matrix $F^{\ast }=F_i{V}_i^{-1/2}$.

Note that the vectors f̃_ℓ(x_i) in the representation (2.3) are defined implicitly (at least for k ≥ 4) and can be easily calculated by a singular value decomposition of the matrices V_i. Using Lemma 2.1 the total information matrix for an exact design of m observations each on n subjects can therefore be written as

$$M=\sum _{i=1}^n\sum _{\ell =1}^m{\stackrel{\sim }{f}}_{\ell }(x_i){\stackrel{\sim }{f}}_{\ell }{(x_i)}^T.$$ (2.4)

Following Kiefer (1974) we define an approximate design as a probability measure ξ on χ^m with finite support. Similarly to (2.4) the information matrix of an approximate design ξ using p different sets of m single subject measurements (with weights ξ(x_i) = ξ(x_i1, …, x_im) at the points x_i) can be expressed as

$$M(\xi )=\sum _{i=1}^p\sum _{\ell =1}^m{\stackrel{\sim }{f}}_{\ell }(x_i){\stackrel{\sim }{f}}_{\ell }{(x_i)}^T\xi (x_i).$$ (2.5)

If ξ puts masses ξ_i = ξ(x_i) at points $x_i\in {\chi }^m(i=1,\dots ,p,{\sum }_{i=1}^p{\xi }_i=1)$ this means that approximately n_i ≈ nξ_i patients have to be treated under experimental conditions x_i = (x_i1, …, x_im) (i = 1, …, p). In practice the integers n_i are obtained by an appropriate rounding procedure from the quantities nξ_i [see for example Pukelsheim and Rieder (1992)]. Note that the design space here is χ^m, i.e. the space of all possible m-observation sets.

Recall that for a given vector c ∈ ℝ^k an approximate design ξ_c is called c-optimal if and only if c ∈ Range (M(ξ_c)) and ξ_c minimizes the expression c^T M⁻(ξ)c, where M⁻(ξ) denotes the generalized inverse of the matrix M(ξ) (note that this expression is approximately proportional to the variance of the generalized least squares estimate for the linear combination c^Tθ). We can now use the general theory developed in Pukelsheim (2006) and the representation (2.5) to derive a condition, which can be used to check the optimality of a given approximate design with respect to a given optimality criterion. In the special case of c-optimal designs, i.e. designs which are optimal for the estimation of a linear combination c^Tθ of the parameters (c ∈ ℝ^k), we obtain the following result.

Theorem 2.1

A design ξ_c is c-optimal in a regression model with information matrix of the form (2.5) if and only if there exists a generalized inverse G of the matrix M(ξ_c) such that the inequality

$$\sum _{\ell =1}^m\frac{{\left(c^{T}G{\stackrel{\sim }{f}}_{\ell }(x)\right)}^2}{c^T{M}^{-}({\xi }_c)c}\le 1$$ (2.6)

holds for all x ∈ χ^m. Moreover, there is equality in (2.6) at any support point of the design ξ_c.

We can use this theorem to apply Theorem 3.3 of Dette and Holland-Letz (2009) and derive a geometric characterization of c-optimal designs for models with information matrices of the form (2.5), which generalizes the classical result of Elfving (1952) to the case of dependent data. We define a generalized Elfving set by

$${ℛ}_m=\text{conv}\left\{\sum _{\ell =1}^m{\varepsilon }_{\ell }{\stackrel{\sim }{f}}_{\ell }(x)\mid x\in {\chi }^m;\sum _{\ell =1}^m{\varepsilon }_{\ell }^2=1\right\}$$ (2.7)

[note that the set ℛ_m reduces for m = 1 to the classical Elfving space considered by Elfving (1952)]. We can now formulate our main theorem.

Theorem 2.2

A design ${\xi }_c={\left\{x_r,p_r\right\}}_{r=1}^p$ is locally c-optimal in a model with information matrix of the form (2.5) if and only if there exist constants γ > 0, ε₁₁, …, ε_1p, …, ε_m1, …, ε_mp satisfying

$$\sum _{\ell =1}^m{\varepsilon }_{\ell r}^2=1;r=1,\dots ,p,$$ (2.8)

such that the point γc ∈ ℝ^k lies on the boundary of the generalized Elfving set ℛ_m defined in (2.7) and has the representation

$$\gamma c=\sum _{r=1}^p{p}_r\left\{\sum _{\ell =1}^m{\varepsilon }_{\ell r}{\stackrel{\sim }{f}}_{\ell }(x_r)\right\}\in \partial {ℛ}_m.$$ (2.9)

Remark 2.1

Theorem 2.1 and 2.2 can easily be generalized to the case of nonlinear fixed effects models of the form

$$Y_{ij}=\eta (x_{ij},\theta )+{\varepsilon }_{ij}i=1,\dots ,n;j=1,\dots ,m,$$ (2.10)

where η denotes a (not necessarily linear) function defined on χ × Θ. A rather detailed review and numerous references on optimal designs for nonlinear models can be found in Atkinson and Haines (1996). In the situation considered in this paper, standard results on nonlinear regression models show that the covariance matrix of the nonlinear generalized least squares can asymptotically be approximated by (2.2) where F_i = (f(x_i1), …, f(x_im)) ∈ ℝ^k×m and the vector f is given by

$$f(x)=\frac{\partial }{\partial \theta }\eta (x,\theta )$$ (2.11)

Here the function f depends on the unknown parameter θ. Following Chernoff (1953) we assume that a preliminary guess for θ is available. In this case the information matrix in (2.4) is well defined and all results of this section remain correct for the nonlinear model (2.10). In particular locally c-optimal designs can be characterized by the equivalence Theorem 2.1 and the geometric characterization in Theorem 2.2, using the modified design matrix F_i = (f(x_i1), …, f(x_im)) with functions f(x) defined in (2.11).

The concept of locally optimal designs has been criticized due to its sensitivity with respect to misspecification of the unknown parameter. Robust optimal designs could be obtained using a Bayesian or minimax approach [see e.g. Chaloner and Verdinelli (1995), Dette (1995), Müller and Pázman (1998)]. A geometric method of constructing Bayesian optimal designs for one-parameter models and a two-point prior distribution is given by Haines (1995) for the case of uncorrelated observations, but its generalization to models with more parameters, arbitrary prior distributions or correlated observations seems to be difficult. A generalization of Elfving's characterization to these more sophisticated criteria may be derived as a generalization of results of Dette (1996), who considered the case of uncorrelated observations. However, these investigations are extremely complicated and are deferred to future research.

3 Examples

We will demonstrate the application of the geometric characterization of Elfving type in two examples, a simple two parameter fixed effects polynomial model with intrinsically correlated observations and a nonlinear population model which is commonly used in pharmacokinetics.

3.1 Quadratic regression

As an example of a linear model we consider a two parameter fixed effects quadratic model without intercept, where m observations are taken for each of the n patients, that is

$$Y_{\mathit{\text{ij}}}={\theta }_1{x}_{\mathit{\text{ij}}}+{\theta }_2{x}_{\mathit{\text{ij}}}^2+{\varepsilon }_{\mathit{\text{ij}}}i=1,\dots ,n,j=1,\dots ,m.$$ (3.1)

We begin with the case m = 2 and assume that observations corresponding to the same patients are correlated with covariance function cov(ε_i1, ε_i2) = σ²c(x_i1, x_i2) = σ²λ^|xi1–xi2| λ ∈ [0, 1]. We obtain

$$V_i={\sigma }^2\left(\begin{array}{cc}1& {\lambda }^{|x_{i1}-x_{i2}|}\\ {\lambda }^{|x_{i1}-x_{i2}|}& 1\end{array}\right).$$

The eigenvalues of the matrix $V_i^{1/2}$ are given by $\sqrt{1-{\lambda }^{|x_{i1}-x_{i2}|}}$ and $\sqrt{1+{\lambda }^{|x_{i1}-x_{i2}|}}$ with corresponding eigenvectors (1, −1)^T and (1, 1)^T, respectively. Therefore we obtain for the matrix $V_i^{-1/2}$ the representation

$$V_i^{-1/2}=\frac{1}{2\sigma }\left(\begin{array}{cc}\frac{1}{\sqrt{1-{\lambda }^{|x_{i1}-x_{i2}|}}}+\frac{1}{\sqrt{1+{\lambda }^{|x_{i1}-x_{i2}|}}}& -\frac{1}{\sqrt{1-{\lambda }^{|x_{i1}-x_{i2}|}}}+\frac{1}{\sqrt{1+{\lambda }^{|x_{i1}-x_{i2}|}}}\\ -\frac{1}{\sqrt{1-{\lambda }^{|x_{i1}-x_{i2}|}}}+\frac{1}{\sqrt{1+{\lambda }^{|x_{i1}-x_{i2}|}}}& \frac{1}{\sqrt{1-{\lambda }^{|x_{i1}-x_{i2}|}}}+\frac{1}{\sqrt{1+{\lambda }^{|x_{i1}-x_{i2}|}}}\end{array}\right).$$

For the model (3.1) we have f(x) = (x, x²)^T, F_i = (f(x_i1), f(x_i2)) and it follows from Lemma 2.1 that the functions f̃_ℓ(x_i) (ℓ = 1, 2) are given by the columns of the matrix $F_i^{\ast }=F_i{V}_i^{-1/2}$ that is

$$\begin{array}{l}{\stackrel{\sim }{f}}_1(x_i)=\frac{1}{2\sigma }{\left(\frac{x_{i1}-x_{i2}}{\sqrt{1-{\lambda }^{|x_{i1}-x_{i2}|}}}+\frac{x_{i1}+x_{i2}}{\sqrt{1+{\lambda }^{|x_{i1}-x_{i2}|}}},\frac{x_{i1}^2-x_{i2}^2}{\sqrt{1-{\lambda }^{|x_{i1}-x_{i2}|}}}+\frac{x_{i1}^2+x_{i2}^2}{\sqrt{1+{\lambda }^{|x_{i1}-x_{i2}|}}}\right)}^T,\\ {\stackrel{\sim }{f}}_2(x_i)=\frac{1}{2\sigma }{\left(\frac{x_{i2}-x_{i1}}{\sqrt{1-{\lambda }^{|x_{i1}-x_{i2}|}}}+\frac{x_{i2}+x_{i1}}{\sqrt{1+{\lambda }^{|x_{i1}-x_{i2}|}}},\frac{x_{i2}^2-x_{i1}^2}{\sqrt{1-{\lambda }^{|x_{i1}-x_{i2}|}}}+\frac{x_{i2}^2+x_{i1}^2}{\sqrt{1+{\lambda }^{|x_{i1}-x_{i2}|}}}\right)}^T.\end{array}$$

For the choice of parameters λ = 0.6, σ² = 0.04 and the design space χ = [0, 2] the corresponding generalized Elfving set ℛ₂ defined by (2.7) is depicted in Figure 1. Every pixel in the figure is induced by a measurement set x ∈ χ^m (m = 2), where the functions f̃_ℓ and the quantities ε_ℓ in (2.7) are evaluated at a (dense) grid. Note that a more dense grid would yield a smoother figure. Both parts of the figure represent the same Elfving space, but the coloring in the left part corresponds to potential values of the first measurement x₁ of x = (x₁, x₂), while the coloring in the right part corresponds to the second measurement x₂ (see the legend of Figure 1).

Figure 1.

The Elfving set ℛ₂ defined in (2.7) for a quadratic regression model (3.1) with two observations per patient. The axes d1 and d2 represent the first and second dimension of the two dimensional elements of this set. The vector c is depicted by the red line, while the two black circles denote the points used in the Elfving representation (2.9).

Suppose we want to estimate the linear combination c^Tθ defined by the vector c = (−1, 1)^T, which is marked as the red line in Figure 1. The optimal sets of measurements are those which can be used to construct the point of the intersection of the boundary of the Elfving space with the line in the direction of the vector c. This representation may require a single point of the form

$$p(x)=\sum _{\ell =1}^m{\varepsilon }_{\ell }{\stackrel{\sim }{f}}_{\ell }(x);\sum _{\ell =1}^m{\varepsilon }_{\ell }^2=1$$

or several points p(x₁), …, p(x_p) of this type, where p ≤ k and k represent the number of parameters in the model (here k = 2). Each point x_j = (x_j1, …, x_jm) ∈ χ^m corresponds to a set of m measurements per patient (for model (3.1) we have m = 2). The weights used in the convex combination yield the weights of the optimal design, i.e. the proportions of total observations taken at the corresponding point x_j. The actual components x_j1 and x_j2 of the point x_j can be determined from the coloring of the point p(x_j) in the left and right part of Figure 1, respectively. Thus, we can easily determine the support points graphically. For example, from Figure 1 we observe that two points, say x₁ and x₂, are required to represent the boundary point γc, which are marked by two circles. From the left part of the Figure we obtain that the colour of x₁ is pink, while the colour of the second point is green, and from the legend in the right upper part of the figure we obtain the values x₁₁ = 0.0 and x₂₁ = 1.2 for the first components of x₁ and x₂, respectively. Similarly, in the right part of Figure 1 we can observe blue and red colours for the two points, which yields x₁₂ = 0.8 and x₂₂ = 2.0 for the second components of x₁ and x₂, respectively. Of course, in concrete applications the value of the components can be determined more precisely from the exact red/green/blue value of the corresponding pixel of the points in the representation (2.9) using appropriate software. Therefore the locally c-optimal design is given by

$${\xi }_i=\left(\begin{array}{cc}(0.0,0.8)& (1.2,2.00)\\ 0.48& 0.52\end{array}\right)$$ (3.2)

and advises the experimenter to use two different individual measurement sets. This means that 48% of the patients should be treated at experimental conditions x₁₁ = 0, x₁₂ = 0.8 and 52% should be treated at x₂₁ = 1.2 and x₂₂ = 2.0. Note that it seems counterintuitive to use measurements at the point x = 0, which carries no information in a model with uncorrelated observations. However, this heuristic argument is not necessarily true in the case of correlated observations.

Alternatively, we can use the figure to determine a hyperplane H supporting the Elfving space at the point γc. This plane is defined through a vector d = (d₁, d₂)^T fulfilling d^Tz = 1 for all z ∈ H, (γc)^Td = 1 and r^Td ≤ 1 for all r ∈ ℛ₂. The support points are then given as the solution of the system of equations

$$\underset{{\varepsilon }_1,{\varepsilon }_2}{max}\sum _{\ell =1}^2{\varepsilon }_{\ell }{\stackrel{\sim }{f}}_{\ell }(x_1,x_2)d=1,\sum _{i=1}^2{\varepsilon }_i^2=1$$

[see the proof of Theorem 3.3 in Dette and Holland-Letz (2009)]. This yields an alternative derivation of the design (3.2). Note that the optimality of this design can be verified by Theorem 2.1.

We now suppose that m = 3 observations are available for each individual in the quadratic regression model (3.1). The design matrix is thus given by F_i = (f(x_i1), f(x_i2), f(x_i3)) and the functions f̃_ℓ(x_i), ℓ = 1, 2, 3 can be determined similarly as in the case of m = 2 observations per individual.

The corresponding Elfving set is depicted in Figure 2. As m = 3 here, three subfigures are needed, each subfigure corresponds to one of the components x_i = (x_i1, x_i2, x_i3). We can observe that only one point is used in the Elfving representation (2.9) and we obtain by a similar reasoning as in the first part of this example that for c = (−1, 1)^T the c-optimal design is given by

Figure 2.

The Elfving set ℛ₃ defined in (2.7) for the quadratic regression model (3.1) with 3 observations per patient. The axes d1 and d2 represent the first and second dimension of the two dimensional elements of this set. The vector c is depicted by the red line, while the black circle shows the point used in the Elfving representation (2.9).

$${\xi }_c=\left(\begin{array}{c}(0.0,1.0,2.0)\\ 1\end{array}\right).$$

This means that all individuals have to be treated at experimental conditions 0, 1.0 and 2.0.

3.2 A nonlinear population model

In order to demonstrate the applicability of the methodology to population pharmacokinetic models, we consider a generic nonlinear random effects model

$$Y_{\mathit{\text{ij}}}=\eta (x_{\mathit{\text{ij}}},b_i)+{\varepsilon }_{\mathit{\text{ij}}}i=1,\dots ,n,j=1,\dots ,m,$$ (3.3)

where η : χ × ℝ^k → ℝ is a known function and the errors ε_i = (ε_i1, …, ε_im) for each patient are normally distributed with mean 0 and covariance matrix W_i ∈ ℝ^m×m, i = 1, …, n. The quantities b₁, …, b_n ∼ (θ, Ω) denote k-dimensional independent normally distributed random variables with mean θ and covariance matrix Ω representing the effect of the corresponding subject under investigation [see Beatty and Piegorsch (1997), Ette et al. (1995), Cayen and Black (1993)]. We also assume that the random variables b₁, …, b_n and the vector (ε₁₁, …, ε_nm)^T are independent.

Due to the nonlinearity of the model an explicit representation of the corresponding Fisher information matrix cannot be derived. Following Retout and Mentré (2003) we propose to use a first-order Taylor expansion to derive an approximation of this matrix. Assuming differentiability of the regression function we use the expansion

$$\eta (x,b)\approx \eta (x,\theta )+f{(x,\theta )}^T(b-\theta ),$$ (3.4)

where

$$f(x,b)+\frac{\partial \eta (x,b)}{\partial b}$$

denotes the gradient of the regression function with respect to b. This means that similarly to the case of fixed effects nonlinear models (see Remark 2.1) the nonlinear model (3.3) is approximated by the linear model (3.4). For the construction of the optimal design we assume that knowledge about the parameters θ and Ω is available from previous or similar experiments and consider the determination of locally optimal designs [see Chernoff (1953)]. As a consequence, the covariance matrix of the nonlinear least squares estimate in the model (3.3) is approximated by replacing the function f in model (2.1) with f(x) = f(x, b)∣_b=θ. The variance of the random vector Y_i = (Y_i1, …, Y_im) now includes the variance caused by the random effect and can be approximated by

$$\text{Var}(Y_i)=V_i\approx F_i^T\mathrm{\Omega }{F}_i+W_{i}i=1,\dots ,n.$$

Consider for example the simple first order elimination model with two observations for each subject (b_i = (b_i1, b_i2))

$$Y_{\mathit{\text{ij}}}=b_{i1}{e}^{-b_{i2}{x}_{ij}}+{\varepsilon }_{ij},x_{ij}\in \chi =[0,2],i=1,\dots ,n,j=1,2,$$ (3.5)

which is widely used in pharmacokinetics [see e.g. Rowland (1993)]. We assume that the errors ε_ij are homoscedastic and uncorrelated with variance σ² > 0, that is $V_i\approx F_i^T\mathrm{\Omega }{F}_i+{\sigma }^2{I}_m$ and for the parameters we consider the case

$$\theta =(5,0.8),\mathrm{\Omega }=\mathit{\text{diag}}(1,0.1)\text{and}{\sigma }^{\text{2}}=\mathrm{0.04.}$$

A straightforward calculation shows that

$$\frac{\partial \eta (x,\theta )}{\partial \theta }={(e^{-{\theta }_{2}x},-{\theta }_{1}x{e}^{-{\theta }_{2}x})}^T.$$

Therefore, we have f(x)=(e^−θ₂x, −θ₁xe^−θ₂x)^T and the functions f̃₁, f̃₂ are defined in a similar manner as it is illustrated in Example 1.

The corresponding generalized Elfving set is depicted in Figure 3. If we are interested in the optimal design for estimating the area under the curve

Figure 3.

The Elfving set ℛ₂ defined in (2.7) for the first order elimination model with 2 observations per patient. The axes d1 and d2 represent the first and second dimension of the two dimensional elements of this set. The vector c is depicted by the red line.

$$\mathit{\text{AUC}}={\int }_0^{\infty }{\theta }_1{e}^{-{\theta }_{2}x}dx=\frac{{\theta }_1}{{\theta }_2},$$

it is easy to see that this objective corresponds to a locally c-optimal design problem for the vector $c={(1/{\theta }_2,-{\theta }_1/{\theta }_2^2)}^T$, which is marked as the red line in Figure 3. From this Figure it can be seen that only one point is needed in the Elfving representation (2.9), and we obtain by a similar reasoning as in Section 3.1 that the locally c-optimal design for the estimation of the area under the curve is given by

$${\xi }_c=\left(\begin{array}{c}(0.0,2.0)\\ 1\end{array}\right).$$

This design means that all patients should be treated under experimental conditions x₁ = 0.0 and x₂ = 2.0. The optimality of this design can also be verified by Theorem 2.1.

4 Appendix: Proofs of main results

4.1 Proof of Lemma 2.1

Let $V_i=U_i{D}_i{U}_i^T$ denote the eigenvalue decomposition of the positive semidefinite matrix V_i, with U_i the matrix of eigenvectors and D_i = diag(λ_i1, …, λ_im) the diagonal matrix of eigenvalues of V_i. Then the matrix

$$V_i^{-1/2}=U_i\mathit{\text{diag}}({\lambda }_{i1}^{-1/2},\dots ,{\lambda }_{\mathit{\text{im}}}^{-1/2})U_i^T$$

is the root of $V_i^{-1}$, i.e. $V_i^{-1/2}{V}_i^{-1/2}=V_i^{-1}$, and we obtain

$$F_i{V}_i^{-1}{F}_i^T=F_i{V}_i^{-1/2}{V}_i^{-1/2}{F}_i^T.$$

Defining

$$F_i^{\ast }:=F_i{V}_i^{-1/2}\in {ℝ}^{k\times m}$$

and writing f̃_ℓ(x_i) for the ℓ-th column of $F_i^{\ast }$, we have

$$F_i{V}_i^{-1}{F}_i^T=F_i^{\ast }{(F_i^{\ast })}^T=\sum _{i=1}^m{\stackrel{\sim }{f}}_{\ell }(x_i){\stackrel{\sim }{f}}_{\ell }{(x_i)}^T,$$

which completes the proof of Lemma 2.1.

4.2 Proof of Theorem 2.1

Let Ξ denote the set of all approximative designs on χ^m and let

$$ℳ=\{M(\xi )\mid \xi \in \mathrm{\Xi }\}\subset {ℝ}^{k\times k}$$

denote the set of all information matrices of the form (2.5). The set ℳ is convex and the information matrix M(ξ_c) of a locally c-optimal design for which the linear combination c^Tθ is estimable [i.e. c ∈ Range (M(ξ))] maximizes the function (c^TM⁻c)⁻¹ in the set ℳ ∩ _c, where

$${\mathcal{A}}_c=\{M(\xi )\in ℳ\mid c\in \text{Range}(M(\xi ))\}.$$

Consequently we obtain from Theorem 7.19 in Pukelsheim (2006) that the design ξ_c is c-optimal if and only if there exists a generalized inverse, say G, of the matrix M(ξ_c) such that the inequality

$$\text{tr(}{\mathit{\text{AGcc}}}^T{G}^T\text{)}\le c^T{M}^{-}({\xi }_c\text{)}c$$

holds for all A ∈ ℳ, where there is equality for any matrix A ∈ ℳ which maximizes (c^TM⁻c) ⁻¹ in the set ℳ. Note that the family ℳ is the convex hull of the set

$$\left\{\sum _{\ell =1}^m{\stackrel{\sim }{f}}_{\ell }(x){\stackrel{\sim }{f}}_{\ell }{(x)}^T\mid x\in {\chi }^m\right\},$$

and therefore the assertion of Theorem 2.1 follows by a standard argument of optimal design theory [see e.g. Silvey (1980)].

4.3 Proof of Theorem 2.2

Recall that the information matrix at the experimental condition x = (x₁, …, x_m) is of the form

$$I(x)=\sum _{\ell =1}^m{\stackrel{\sim }{f}}_{\ell }(x){\stackrel{\sim }{f}}_{\ell }{(x)}^T.$$ (4.1)

Therefore, the result is a direct consequence of Theorem 3.3 in Dette and Holland-Letz (2009), which presents a geometric characterization of Elfving type for c-optimal designs in models with an information matrix of the form (4.1).