Optimal designs in regression with correlated errors

Abstract

This paper discusses the problem of determining optimal designs for regression models, when the observations are dependent and taken on an interval. A complete solution of this challenging optimal design problem is given for a broad class of regression models and covariance kernels.

We propose a class of estimators which are only slightly more complicated than the ordinary least-squares estimators. We then demonstrate that we can design the experiments, such that asymptotically the new estimators achieve the same precision as the best linear unbiased estimator computed for the whole trajectory of the process. As a by-product we derive explicit expressions for the BLUE in the continuous time model and analytic expressions for the optimal designs in a wide class of regression models. We also demonstrate that for a finite number of observations the precision of the proposed procedure, which includes the estimator and design, is very close to the best achievable. The results are illustrated on a few numerical examples.

1 Introduction

Optimal design theory is a classical field of mathematical statistics with numerous applications in life sciences, physics and engineering. In many cases the use of optimal or efficient designs yields to a reduction of costs by a statistical inference with a minimal number of experiments without loosing any accuracy. Most work on optimal design theory concentrates on experiments with independent observations. Under this assumption the field is very well developed and a powerful methodology for the construction of optimal designs has been established [see for example the monograph of Pukelsheim (2006)]. While important and elegant results have been derived in the case of independence, there exist numerous situations where correlation between different observations is present and these classical optimal designs are not applicable.

The theory of optimal design for correlated observations is much less developed and explicit results are only available in rare circumstances. The challenging difficulty consists here in the fact that - in contrast to the independent case - correlations yield to non-convex optimization problems and classical tools of convex optimization theory are not applicable. Some exact optimal designs for specific linear models have been studied in Dette et al. (2008); Kiselak and Stehlík (2008); Harman and Štulajter (2010). Because explicit solutions of optimal design problems for correlated observations are rarely available, several authors have proposed to determine optimal designs based on asymptotic arguments [see for example Sacks and Ylvisaker (1966, 1968), Bickel and Herzberg (1979), Näther (1985a), Zhigljavsky et al. (2010)], where the references differ in the asymptotic arguments used to embed the discrete (non-convex) optimization problem in a continuous (or approximate) one. However, in contrast to the uncorrelated case, this approach does not simplify the problem substantially and due to the lack of convexity the resulting approximate optimal design problems are still extremely difficult to solve. As a consequence, optimal designs have mainly been determined analytically for the location model (in this case the optimization problems are in fact convex) and for a few one-parameter linear models [see Boltze and Näther (1982), Näther (1985a), Ch. 4, Näther (1985b), Pázman and Müller (2001) and Müller and Pázman (2003) among others]. Only recently, Dette et al. (2013) determined (asymptotic) optimal designs for least squares estimation in models with more parameters under the additional assumption that the regression functions are eigenfunctions of an integral operator associated with the covariance kernel of the error process. However, due to this assumption, the class of models for which approximate optimal designs can be determined explicitly is rather small.

The present paper provides a complete solution of this challenging optimal design problem for a broad class of regression models and covariance kernels. Roughly speaking, we determine (asymptotic) optimal designs for a slightly modified ordinary least squares estimator (OLSE), such that the new estimate and the corresponding optimal design achieve the same accuracy as the best unbiased linear estimate (BLUE) with corresponding optimal designs.

To be more precise, consider a general regression observation scheme given by

$$y(t_j)={\theta }^{T}f(t_j)+\epsilon (t_j),j=1,\dots ,N,$$ (1.1)

where 𝔼[ε(t_j)] = 0, K(t_i, t_j) = 𝔼[ε(t_i)ε(t_j)] denotes the covariance between observations at the points t_i and t_j. (i, j = 1, . . ., N), θ = (θ₁, . . ., θ_m)^T is a vector of unknown parameters, f(t) = (f₁(t), . . ., f_m(t))^T is a vector of linearly independent functions, the explanatory variables t₁, . . ., t_N vary in a compact interval, say [a, b]. Parallel to model (1.1) we also consider its continuous time version

$$y(t)={\theta }^{T}f(t)+\epsilon (t),t\in [a,b],$$ (1.2)

where the full trajectory of the process {y(t)|t ∈ [a, b]} can be observed and {ε(t)|t ∈ [a, b]} is a centered Gaussian process with covariance kernel K, i.e. K(s, t) = 𝔼[ε(s)ε(t)]. This kernel is assumed to be continuous throughout this paper.

We pay much attention to the one-parameter case and develop a general method for solving the optimal design problem in model (1.2) explicitly for the OLSE, perhaps slightly modified. The new estimate and the corresponding optimal design achieve the minimal variance among all linear estimates (obtained by the BLUE). In particular, our approach allows to calculate this optimal variance explicitly. As a by-product we also identify the BLUE in the continuous time model (1.2). Based on these asymptotic considerations, we consider the finite sample case and suggest designs for a new estimation procedure (which is very similar to OLSE) with an efficiency very close to the best possible (obtained by the BLUE and the corresponding optimal design), for any number of observations. In doing this, we show how to implement the optimal strategies from the continuous time model in practice and demonstrate that even for very small sample sizes the loss of efficiency with respect to the best strategies based on the use of BLUE with a corresponding optimal design can be considered as negligible. We would like to point out at this point that - even in the one-dimensional case - the problem of numerically calculating optimal designs for the BLUE for a fixed sample size is an extremely challenging one due to the lack of convexity of the optimization problem.

In our approach, the importance of the one-parameter design problem is also related to the fact that the optimal design problem for multi-parameter models can be reduced componentwisely to problems in the one-parameter models. This gives us a way to generate analytically constructed universally optimal designs for a wide range of continuous time multi-parameter models of the form (1.2). Our technique is based on the observation that for a finite number of observations we can always emulate the BLUE in model (1.1) by a different linear estimator. To achieve that theoretically we assign signs to the support points of a discrete design and not only weights in the one-parameter models, but in the multi-parameter case we use matrix weights. We then determine “optimal” signs and weights and consider the weak convergence of these ‘designs” and estimators as the sample size converges to infinity. Finally, we prove the (universal) optimality of the limits in the continuous time model (1.2).

Theoretically, we construct a sequence of designs for either the pure or a modified OLSE, say θ̂_N, such that its variance or covariance matrix satisfies Var(θ̂_N) → D^* as the sample size N converges to infinity, where D^* is the variance (if m = 1) or covariance matrix (if m > 1) for the BLUE in the continuous time model (1.2). In other words, D^* is the smallest possible variance (or covariance matrix with respect to the Loewner ordering) of any unbiased linear estimator and any design. This makes the designs derived in this paper very competitive in applications against the designs proposed by Sacks and Ylvisaker (1966) and optimal designs constructed numerically for the BLUE (using the Brimkulov-Krug-Savanov algorithm, for example). We emphasize once again that due to non-convexity the numerical construction of optimal designs for the BLUE is extremely difficult. An additional advantage of our approach is that we can analytically compute the BLUE with the corresponding optimal variance (covariance matrix) D^* in the continuous time model (1.2) and therefore monitor the proximity of different approximations to the optimal variance D^* obtained by the BLUE.

The methodology developed in this paper results in a non-standard estimation and optimal design theory and consists in a delicate interplay between new linear estimators and designs in the models (1.1) and (1.2). For this reason let us briefly introduce various estimators, which we will often refer to in the following discussion. Consider the model (1.1) and suppose that N observations are taken at experimental conditions t₁, . . ., t_N. For the corresponding vector of observations Y = (y(t₁), . . ., y(t_N))^T, a general weighted least squares estimator (WLSE) of θ is defined by

$$\text{WLSE}:{\widehat{\theta }}_{\mathit{WLSE}}={({\mathbf{X}}^T\mathbf{WX})}^{-1}{\mathbf{X}}^T\mathbf{WY},$$ (1.3)

where $\mathbf{X}={(f_i(t_j))}_{j=1,\dots ,N}^{i=1,\dots ,m}$ is an N×m design matrix and W is some N × N matrix such that (X^TWX)⁻¹ exists. For any such W the estimator (1.3) is obviously unbiased. The covariance matrix of the estimator (1.3) is given by

$$\text{Var}({\widehat{\theta }}_{\mathit{WLSE}})={({\mathbf{X}}^T\mathbf{W}\mathbf{X})}^{-1}{\mathbf{X}}^T\mathbf{W}\mathbf{\sum }{\mathbf{W}}^T\mathbf{X}{({\mathbf{X}}^T{\mathbf{W}}^T\mathbf{X})}^{-1},$$ (1.4)

where Σ = (K(t_i, t_j))_i,j=1,...,N is an N ×N matrix of variances/covariances. For the standard WLSE the matrix W is symmetric non-negative definite; in this case θcirc;_WLSE minimizes the weighted sum of squares SS_W(θ) = (Y − Xθ)^TW(Y − Xθ) with respect to θ. Important particular cases of estimators of the form (1.3) are the OLSE, the best unbiased linear estimate (BLUE) and the signed least squares estimate (SLSE):

$$\text{OLSE}:{\widehat{\theta }}_{\mathit{OLSE}}={({\mathbf{X}}^T\mathbf{X})}^{-1}{\mathbf{X}}^T\mathbf{Y},$$ (1.5)

$$\text{BLUE}:{\widehat{\theta }}_{\mathit{BLUE}}={({\mathbf{X}}^T{\mathbf{\sum }}^{-1}\mathbf{X})}^{-1}{\mathbf{X}}^T{\mathbf{\sum }}^{-1}\mathbf{Y},$$ (1.6)

$$\text{SLSE}:{\widehat{\theta }}_{\mathit{SLSE}}={({\mathbf{X}}^T\mathbf{S}\mathbf{X})}^{-1}{\mathbf{X}}^T\mathbf{S}\mathbf{Y}.$$ (1.7)

Here S is an N×N diagonal matrix with entries +1 and −1 on the diagonal; note that if S ≠ I_N then SLSE is not a standard WLSE. While the use of BLUE and OLSE is standard, the SLSE is less common. It was introduced in Boltze and Näther (1982) and further studied in Chapter 5.3 of Näther (1985a). In the content of the present paper, the SLSE will turn out to be very useful for constructing optimal designs for OLSE and the BLUE in the model (1.2) with one parameter, where the full trajectory can be observed. Another estimate of θ, which is not a special case of the WLSE, will be introduced in Section 3 and used in the multi-parameter models.

The remaining structure of the paper is as follows. In Section 2 we derive optimal designs for continuous time one-parameter models and discuss how to implement the designs in practice. In Section 3 we extend the results of Section 2 to multi-parameter models. In Appendix B we discuss transformations of regression models and associated designs, which are a main tool in the proofs of our result but also of own interest. In particular, we provide an extension of the famous Doob representation for Gaussian processes [see Doob (1949) and Mehr and McFadden (1965)], which turns out to be a very important ingredient in proving the design optimality results of Sections 2 and 3. Finally, in Appendix A we collect some auxiliary statements and proofs for the main results of this paper.

2 Optimal designs for one-parameter models

In this section we concentrate on the one-parameter model

$$y(t_j)=\theta f(t_j)+\epsilon (t_j);j=1,\dots ,N,$$ (2.1)

on the interval [a, b] and its continuous time analogue, where 𝔼[ε(t)] = 0 and 𝔼[ε(t)ε(t′)] = K(t, t′). Our approach uses some non-standard ideas and estimators in linear models and therefore we begin this section with a careful explanation of the logic of the material.

• Sect. 2.1. Under the assumption that the design space is finite we show in Lemma 2.1 that by assigning weights and signs to the observation points {t₁, . . ., t_N} we can construct a WLSE which is equivalent to the BLUE. Then, we derive in Corollary 2.1 an explicit form for the optimal weights for a broad class of covariance kernels, which are called triangular covariance kernels.

• Sect. 2.2. We demonstrate in Theorem 2.1 that the optimal designs derived in Sect. 2.1 converge weakly to a signed measure, if the cardinality of the design space converges to infinity.

• Sect. 2.3. We consider model (2.1) under the assumption that the full trajectory of the process {y(t)|t ∈ [a, b]} can be observed. For the specific case of Brownian motion, that is K(t, t′) = min{t, t′}, we prove analytically the optimality of the signed measure derived in Theorem 2.1 for OLSE. Then, in Theorem 2.3 we establish optimality of the asymptotic measures from Theorem 2.1 for general covariance kernels. As a by-product we also identify the BLUE in the continuous time model (1.2) (in the one-dimensional case). For this purpose, we introduce a transformation which maps any regression model with a triangular covariance kernel into another model with different triangular kernels. These transformations allow us to reduce any optimization problem to the situation considered in Theorem 2.2, which refers to the case of Brownian motion. The construction of this map is based on an extension of the celebrated Doob’s representation which will be developed in Appendix B.

• Sect. 2.4. We provide some examples of asymptotic optimal measures for specific models.

• Sect. 2.5. We introduce a practical implementation of the asymptotic theory derived in the previous sections. For a finite sample size we construct WLSE with corresponding designs which can achieve very high efficiency compared to the BLUE with corresponding optimal design. It turns out that these estimators are slightly modified OLSE, where only observations at the end-points obtain a weight (and in some cases also a sign).

• Sect. 2.6. We illustrate the new methodology in several examples. In particular, we give a comparison with the best known procedures based on BLUE and show that the loss in precision for the procedures derived in this paper is negligible with our procedures being much simpler and more robust than the procedures based on BLUE.

2.1 Optimal designs for SLSE on a finite design space

In this section, we suppose that the design space for model (2.1) is finite, say 𝒯 = {t₁, . . ., t_N}, and demonstrate that in this case the approximate optimal designs for the SLSE (1.7) can be found explicitly. Since we consider the SLSE (1.7) rather than the OLSE (1.5), a generic approximate design on the design space 𝒯 = {t₁, . . ., t_N} is an arbitrary discrete signed measure ξ = {t₁, . . ., t_N; w₁, . . ., w_N}, where w_i = s_ip_i, s_i ∈ {−1, 1}, p_i ≥ 0 (i = 1, . . ., N) and ${\sum }_{i=1}^N{p}_i=1$. We assume that the support t₁, . . ., t_N of the design is fixed but the weights p₁, . . ., p_N and signs s₁, . . ., s_N, or equivalently the signed weights w_i, will be chosen to minimize the variance of the SLSE (1.7). In view of (1.4), this variance is given by

$$D(\xi )=\sum _{i=1}^N\sum _{j=1}^{N}K(t_i,t_j)w_i{w}_{j}f(t_i)f(t_j)/{\left(\sum _{i=1}^N{w}_i{f}^2(t_i)\right)}^2.$$ (2.2)

Note that this expression coincides with the variance of the WLSE (1.2), where the matrix W is defined by W = diag(w₁, . . ., w_N).

We assume that f(t_i) ≠ 0 for all i = 1, . . ., N. If f(t_j) = 0 for some j then the point t_j can be removed from the design space 𝒯 without changing the SLSE estimator, its variance and the corresponding value D(ξ). In the above definition of the weights w_i, we have ${\sum }_{i=1}^N\mid w_i\mid ={\sum }_{i=1}^N{p}_i=1$. Note, however, that the value of the criterion (2.2) does not change if we change all the weights from w_i to cw_i (i = 1, . . ., N) for arbitrary c ≠ 0.

Despite the fact that the functional D in (2.2) is not convex as a function of (w₁, . . ., w_N), the problem of determining the optimal design can be easily solved by a simple application of the Cauchy-Schwarz inequality. The proof of the following lemma is given in Appendix A [see also Theorem 5.3 in Näther (1985a), where this result was proved in a slightly different form].

Lemma 2.1

Assume that the matrix Σ = (K(t_i, t_j))_i,j=1,...,N is positive definite and f(t_i) ≠ 0 for all i = 1, . . ., N. Then the optimal weights $w_1^{\ast },\dots ,w_N^{\ast }$ minimizing (2.2) subject to the constraint ${\sum }_{i=1}^N\mid w_i\mid =1$ are given by

$$w_i^{\ast }=c\frac{{\mathbf{e}}_i^T{\mathbf{\sum }}^{-1}\mathbf{f}}{f(t_i)};i=1,\dots ,N,$$ (2.3)

where f = (f(t₁), . . ., f(t_N))^T, e_i = (0, 0, . . ., 0, 1, 0, . . ., 0)^T ∈ ℝ^N is the i-th unit vector, and

$$c={\left(\sum _{i=1}^N\mid {\mathbf{e}}_i^T{\mathbf{\sum }}^{-1}\mathbf{f}/f(t_i)\mid \right)}^{-1}.$$

Moreover, for the design ${\xi }^{\ast }=\{t_1,\dots ,t_N;w_1^{\ast },\dots ,w_N^{\ast }\}$ with weights (2.3) we have D(ξ^*) = D^*, where D^* = 1/(f^T Σ⁻¹f), the variance of the BLUE defined in (1.6) using all observations t₁, . . ., t_N.

Lemma 2.1 shows, in particular, that the pair {SLS estimate, corresponding optimal design ξ^*} provides an unbiased estimator with the best possible variance for the one-parameter model (2.1). This results in a WLSE (1.2) with ${\mathbf{W}}^{\ast }=\text{diag}(w_1^{\ast },\dots ,w_N^{\ast })$ which is BLUE. In other words, by a slight modification of the OLSE we are able to emulate the BLUE using the appropriate design or WLSE.

While the statement of Lemma 2.1 holds for arbitrary kernels, we are able to determine the optimal weights $w_i^{\ast }$ more explicitly for a broad class, which are called triangular kernels and are of the form

$$K(t,t^{\prime })=u(t)v(t^{\prime })\text{for}t\le t^{\prime },$$ (2.4)

where u(·) and v(·) are some functions on the interval [a, b]. Note that the majority of covariance kernels considered in literature belong to this class, see for example Näther (1985a); Zhigljavsky et al. (2010) or Harman and Štulajter (2011). The following result is a direct consequence of Lemma A.1 from Appendix A.

Corollary 2.1

Assume that the covariance kernel K(·, ·) has the form (2.4) so that the matrix Σ = (K(t_i, t_j))_i,j=1,...,N is positive definite and has the entries K(t_i, t_j) = u_iv_j for i ≤ j, where for k = 1, . . ., N we denote u_k = u(t_k), v_k = v(t_k), and also f_k = f(t_k), q_k = u_k/v_k. If f₁ ≠ 0 (i = 1, . . ., N), the weights in (2.3) can be represented explicitly as follows:

$$w_1^{\ast }=\frac{c}{f_1}({\stackrel{\sim }{\sigma }}_{11}{f}_1+{\stackrel{\sim }{\sigma }}_{12}{f}_2)=\frac{c{u}_2}{f_1{v}_1{v}_2(q_2-q_1)}\left(\frac{f_1}{u_1}-\frac{f_2}{u_2}\right),$$ (2.5)

$$w_N^{\ast }=\frac{c}{f_N}({\stackrel{\sim }{\sigma }}_{N,N}{f}_N+{\stackrel{\sim }{\sigma }}_{N-1,N}{f}_{N-1})=\frac{c}{f_N{v}_N(q_N-q_{N-1})}\left(\frac{f_N}{v_N}-\frac{f_{N-1}}{v_{N-1}}\right),$$ (2.6)

$$\begin{array}{l}{w}_i^{\ast }=\frac{c}{f_i}({\stackrel{\sim }{\sigma }}_{i,i}{f}_i+{\stackrel{\sim }{\sigma }}_{i-1,i}{f}_{i-1}+{\stackrel{\sim }{\sigma }}_{i,i+1}{f}_{i+1})\\ =\frac{c}{f_i{v}_i}\left(\frac{(q_{i+1}-q_{i-1})f_i}{v_i(q_{i+1}-q_i)(q_i-q_{i-1})}-\frac{f_{i-1}}{v_{i-1}(q_i-q_{i-1})}-\frac{f_{i+1}}{v_{i+1}(q_{i+1}-q_i)}\right),\end{array}$$ (2.7)

for i = 2, . . ., N − 1. In formulas (2.5), (2.6) and (2.7), the quantity σ̃_ij denotes the element in the position (i, j) of the matrix Σ⁻¹ = (s̃_ij)_i,j=1,...,N.

2.2 Weak convergence of designs

In this section, we consider the asymptotic properties of designs with weights (2.5) – (2.7). Recall that the design space is an interval, say [a, b], and that we assume a triangular covariance function of the form (2.4). According to the discussion of triangular covariance kernels provided in Section 4.1 of Appendix B, the functions u(·) and v(·) are continuous and strictly positive on the interval (a, b) and the function q(·) = u(·)/v(·) is positive, continuous and strictly increasing on (a, b). We also assume that the regression function f in (2.1) is continuous and strictly positive on the interval (a, b). We define the transformation

$$Q(t)=\frac{q(t)-q(a)}{q(b)-q(a)}$$ (2.8)

and note that the function Q : [a, b] → [0, 1] is increasing on the interval [a, b] with Q(a) = 0 and Q(b) = 1, that is Q(·) is a cumulative distribution function (c.d.f.). For fixed N and i = 1, . . ., N, define $z_{i,N}=(i-\frac{1}{2})/N$ and the design points

$$t_{i,N}=Q^{-1}(z_{i,N}),i=1,\dots ,N.$$ (2.9)

Theorem 2.1

Consider the optimal design problem for the model (2.1), where the error process ε(t) has the covariance kernel K(t, s) of the form (2.4). Assume that u(·), v(·), f(·) and q(·) are strictly positive, twice continuously differentiable functions on the interval [a, b]. Consider the sequence of signed measures

$${\xi }_N=\{t_{1,N},\dots ,t_{N,N};w_{1,N},\dots ,w_{N,N}\},$$

where the support points t_i,N are defined in (2.9) and the weights w_i,N are assigned to these points according to the rule (2.3) of Lemma 2.1. Then the sequence of measures {ξ_N}_N∈ℕ converges in distribution to a signed measure ξ^*, which has masses

$$P_a=\frac{c}{f(a)v^2(a)q^{\prime }(a)}\left[\frac{f(a)u^{\prime }(a)}{u(a)}-f^{\prime }(a)\right],P_b=c·\frac{h^{\prime }(b)}{f(b)v(b)q^{\prime }(b)}$$ (2.10)

at the points a and b, respectively, and the signed density

$$p(t)=-\frac{c}{f(t)v(t)}{\left[\frac{h^{\prime }(t)}{q^{\prime }(t)}\right]}^{\prime }$$ (2.11)

(that is, the Radon-Nikodym derivative of ξ^* with respect to the Lebesque measure) on the interval (a, b), where the function h(·) is defined by h(t) = f(t)/v(t).

The proof of Theorem 2.1 is technically complicated and therefore given in Appendix A. The constant c ≠ 0 in (2.10) and (2.11) is arbitrary. If a normalization |ξ^*|([a, b]) = 1 is required, then c can be found from the normalizing condition

$${\int }_a^b{\xi }^{\ast }(dt)=\mid P_a\mid +\mid P_b\mid +{\int }_a^b\mid p(t)\mid dt=1.$$

Throughout this paper we write the limiting designs of Theorem 2.1 in the form

$${\xi }^{\ast }(dt)=P_a{\delta }_a(dt)+P_b{\delta }_b(dt)+p(t)dt,$$ (2.12)

where δ_a(dt) and δ_b(dt) are the Dirac-measures concentrated at the points a and b, respectively, and the function p(·) is defined by (2.11). Note also that under the assumptions of Theorem 2.1, the function p(·) is continuous on the interval [a, b]. In the case of Brownian motion, the limiting design of Theorem 2.1 is particularly simple.

Example 2.1

If the error process ε in model (2.1) is the Brownian motion on the interval [a, b] with 0 < a < b < ∞, then K(t, s) = min(t, s) and hence u(t) = t, v(t) = 1, q(t) = t. This implies that the limiting design of Theorem 2.1 is given by (2.12) with

$$P_a=c\frac{f(a)-f^{\prime }(a)a}{af(a)},P_b=c\frac{f^{\prime }(b)}{f(b)}\text{and}p(t)=-c\frac{f^{″}(t)}{f(t)}.$$ (2.13)

2.3 Optimal designs and the BLUE

In this section we consider the continuous time model (1.2) in the case m = 1 and demonstrate that the limiting designs derived in Theorem 2.1 are in fact optimal. A linear estimator for the parameter θ in model (1.2) is defined by ${\widehat{\theta }}_{\mu }={\int }_a^{b}y(t)\mu (dt)$, where μ is a signed measure on the interval [a, b]. Special cases include the OLSE and SLSE $\stackrel{\sim }{{\theta }_{\xi }}={\int }_a^{b}y(t)f(t)\xi (dt)/{\int }_a^b{f}^2(t)\xi (dt)$, where ξ is a measure or a signed measure on the interval [a, b], respectively. Note that θ̂_μ is unbiased if and only ${\int }_a^{b}f(t)\mu (dt)=1$ and $\stackrel{\sim }{{\theta }_{\xi }}$ is unbiased by construction. The BLUE (in the continuous time model (1.2)) minimizes

$$\mathrm{\Phi }(\mu )=\text{Var}({\widehat{\theta }}_{\mu })={\int }_a^b{\int }_a^{b}K(x,y)\mu (dx)\mu (dy)$$

in the class of all signed measures μ satisfying ${\int }_a^{b}f(t)\mu (dt)=1$, and

$$D^{\ast }=inf\{\mathrm{\Phi }(\mu )\mid \mu \text{signed}\text{measure}\text{on}[a,b]\}$$ (2.14)

denotes the best possible variance of all linear unbiased estimators in the continuous time model (1.2).

Similarly, a signed measure ξ^* on the interval [a, b] is called optimal for least squares estimation in the one-parameter model (1.2), if it minimizes the functional

$$D(\xi )=\text{Var}(\stackrel{\sim }{{\theta }_{\xi }})={\int }_a^b{\int }_a^{b}K(t,s)f(t)f(s)\xi (dt)\xi (ds)/{\left({\int }_a^b{f}^2(t)\xi (dt)\right)}^2,$$ (2.15)

in the set of all signed measures ξ on the interval [a, b], such that ${\int }_a^b{f}^2(t)\xi (dt)\ne 0$. In the case of a Brownian motion, we are able to establish the optimality of the design of Example 2.1. A proof of the following result is given in Appendix A.

Theorem 2.2

Let {ε(t) | t ∈ [a, b]} be a Brownian motion, so that K(t, t′) = min{t, t′}, and f be a positive, twice continuously differentiable function on the interval [a, b] ⊂ ℝ⁺. Then the signed measure ξ^*, defined by (2.12) and (2.13) with arbitrary c ≠ 0, minimizes the functional (2.15). The minimal value in (2.15) is obtained as

$$D({\xi }^{\ast })=\underset{\xi }{min}D(\xi )={\left[\frac{f^2(a)}{a}+{\int }_a^b{(f^{\prime }(t))}^{2}dt\right]}^{-1}.$$

Moreover, the BLUE in model (1.2) is given by θ̂_μ^*, where μ^*(dt) = f(t)ξ^**(dt) and ξ^** is the signed measure defined by (2.12) and (2.13) with constant c^* = D(ξ^*). This further implies D^* = D(ξ^*) = Φ(μ^*).

Based on the design optimality established in Theorem 2.2 for the special case of Brownian motion and the technique of transformation of regression models described in Appendix B, we can establish the optimality of the asymptotic designs derived in Theorem 2.1 for more general covariance kernels; see Appendix A for the proof.

Theorem 2.3

Under the conditions of Theorem 2.1, the optimal design ξ^* minimizing the functional (2.15) is defined by the formulas (2.10) – (2.12) with arbitrary c ≠ 0. The minimal value in (2.15) is obtained as

$$D({\xi }^{\ast })={\left[\frac{{\stackrel{\sim }{f}}^2(q(a))}{q(a)}+{\int }_{q(a)}^{q(b)}{({\stackrel{\sim }{f}}^{\prime }(t))}^{2}dt\right]}^{-1},$$ (2.16)

where f̃(t) = f(q⁻¹(s))/v(q⁻¹(s)). Moreover, the BLUE in model (1.2) is given by θ̂_μ^*, where μ^*(dt) = f(t)ξ^**(dt), ξ^** is the signed measure defined in (2.10) – (2.12) with constant c^* = D(ξ^*), and D^* = Φ(μ^*) = D(ξ^*).

2.4 Examples of optimal designs

In this section, we provide the values of P_a, P_b and the function p(·) in the general expression (2.12) for the optimal designs in a number of important special cases for the one-parameter continuous time model (1.2), where the design space is 𝒯 = [a, b]. Specifically, optimal designs are given in Table 1 for the location model, in Table 2 for the linear model, in Table 3 for a quadratic model and in Table 4 for a trigonometric model. The last named model was especially chosen to demonstrate the existence of optimal designs with a density p which changes sign in the interval (a, b). In the tables several triangular covariance kernels are considered. The parameters of these covariance kernels satisfy the constraints c₂ > ±c₁, ∓ c₂ ∉ [a, b], γ > ω, λ > 0. For the sake of a transparent presentation, we use the factor c = 1 in all tables, but we emphasize once again that the optimal designs do not depend on the scaling factor.

Table 1.

Optimal designs for the location model: f(t) = 1, t [a, b].

u(t)	v(t)	Pa	Pb	p(t)
any	1	1	0	0
c1 + t	c2 ± t	$\frac{1}{a+c_1}$	$\frac{-1}{b\pm c_2}$	0
tγ	tω	−γa−γ−ω	ωb−γ−ω	γωt−1−γ−ω
eλt	e−γt	λea(γ−λ)	γeb(γ−λ)	λγet(γ−λ)

Table 2.

Optimal designs for the linear regression model through the origin: f(t) = t, t ∈ [a, b].

u(t)	v(t)	Pa	Pb	p(t)
t	1	0	1	0
c1 + t	c2 ± t	$\frac{-c_1}{(a+c_1)a}$	$\frac{\pm c_2}{(b\pm c_2)b}$	0
tγ	t	−(γ−1)a−γ−ω	(ω−1)b−γ−ω	(1−γ)(1−ω)t−1−γ−ω
eλt	1	$\frac{(a\lambda -1)e^{a\lambda }}{a}$	$\frac{e^{-b\lambda }}{b}$	$\frac{\lambda e^{-t\lambda }}{t}$
eλt	e−γt	$\frac{a\lambda -1}{a}{e}^{a(\gamma -\lambda )}$	$\frac{b\gamma +1}{b}{e}^{b(\gamma -\lambda )}$	$\frac{\lambda \gamma t-\gamma +\lambda }{t}{e}^{t(\gamma -\lambda )}$

Table 3.

Optimal designs for the quadratic regression model: f(t) = t² + ν, t ∈ [a, b].

u(t)	v(t)	Paf(a)	Pbf(b)	p(t)f(t)
t	1	(a2 − ν)/a	−2b	2
c1 + t	c2 ± t	$\frac{(a^2-\nu +2a{c}_1)}{a+c_1}$	$\frac{\mp (b^2-\nu \pm 2b{c}_2)}{b\pm c_2}$	2
tγ	tω	((2−γ)a2−γν)a−γ−ω	((ω−2)b2+ων)b−γ−ω	((2−ω)(2− ω)+νγω)t1−γ−ω
eλt	1	(2a − (a2 + ν)λ)e−aλ	−2be−bλ	2(1 − tλ)e−tλ
eλt	e−λt	(2a − (a2 + ν)λ)	−((b2 + ν)λ + 2b)	(2 − λ2(t2 + ν)

Table 4.

Optimal designs for the trigonometric regression model: $f(t)=1+\frac{1}{2}sin(2\pi t)$, t ∈ [1, 2].

u(t)	v(t)	Pa	Pb	p(t)f(t)
t	1	(1 − π)	π	2π2 sin(2πt)
c1 + t	c2 ± t	$\frac{1-\pi c_1-\pi }{c_1+1}$	$\mp \frac{1\mp \pi c_2-2\pi }{c_2\pm 2}$	2π2 sin(2πt)
t2	t	(2 − π)	(2π − 1)/8	2t−4((π2t2−1) sin(2πt)+πt cos(2πt)−1)
eλt	1	(λ − π)e−λ	πe−2λ	(2π2 sin(2πt) + πλcos(2πt))e−λt
eλt	e−λt	(λ − π)	(λ + π)	((2π2 + λ2/2) sin(2πt) + λ2)

As an example, if K(t, t′) = e^{−λ|t−t′|} for some λ > 0, we have from the last row of Table 2 that the optimal design for the continuous time model {θt+ε(t)|t ∈ [1, 2]} is ${\xi }^{\ast }(dt)=(\lambda -1){\delta }_1(dt)+(\lambda +\frac{1}{2}){\delta }_2(dt)+{\lambda }^{2}dt$, and as a consequence, $D^{\ast }={(\frac{5}{2}+\frac{1}{2\lambda }+\frac{7}{6}\lambda )}^{-1}$.

2.5 Practical implementation: designs for finite sample size

In practice, efficient designs and corresponding estimators for the model (1.1) have to be derived from the optimal solutions in the continuous time model (1.2), and in this section a procedure with a good finite sample performance is proposed. Roughly speaking, it consists of a slight modification of the ordinary least squares estimator and a discretization of a continuous signed measure with the asymptotic optimal density in (2.11).

We assume that the experimenter can take N +2 observations with N observations inside the interval [a, b]. In principle, any probability measure on the interval can be approximated by an (N +2)-point measure with weights 1/(N +2) and similarly any finite signed measure can be approximated by an (N +2)-point signed measure with equal weights (in absolute value). We hence could use a direct approximation of the optimal signed measures of the form (2.12) by a sequence of (N +2)-point signed measures with equal weights (in absolute value). For an increasing sample size this sequence will eventually converge to the optimal measure of Theorem 2.3. However, this convergence will typically be very slow, where we measure the speed of convergence by the differences between the variances D(ξ) of the corresponding estimates and the optimal value D^* defined in (2.16). The main difficulty lies in the fact that a typical optimal measure has masses at the boundary points a and b, in addition to some density on the interval (a, b). The convergence of discrete measures with equal (in absolute value) weights to such a measure will be very slow, especially in view of the fact that in our approximating measures the points cannot be repeated. Summarizing, approximation of the optimal signed measures by measures with equal weights is possible but cannot be accurate for small N.

In order to improve the rate of convergence we propose a slight modification of the ordinary least squares procedure. In particular, we propose a WLSE with weights at the points a and b (the end-points of the interval [a, b]), which correspond to the masses P_a and P_b of the asymptotic optimal design. We thus only need to approximate the continuous part of the optimal signed measure, which has a density on (a, b), by an N-point design with equal masses. To be precise, consider an optimal measure of the form (2.12). We assume that the density p(·) is not identically zero on the interval (a, b) and choose the constant c such that ${\int }_a^b\mid p(t)\mid dt=1$. Note that unless p(·) changes sign in (a, b), we can choose p(t) ≥ 0 for all t ∈ (a, b). Define φ(t) = |p(t)| for t ∈ (a, b) and denote by $F(t)={\int }_a^t\phi (s)ds$ the corresponding distribution function. The N-point design we use as an N-point approximation to the measure with density φ(t) is ξ̂_N = {t_1,N, . . ., t_N,N; 1/N, . . ., 1/N}, where t_i,N = F⁻¹(z_i,N) with z_i,N = i/(N + 1), i = 1, 2, . . ., N. If p(t) = 0 on a sub-interval of [a, b] and F⁻¹(z_i,N) is not uniquely defined then we choose the smallest element from the set F⁻¹(z_i,N) as t_i,N . Finally, the design we suggest as an (N +2)-point approximation to the optimal measure in (2.12) is

$${\xi }_{N+2}^{\ast }=P_a{\delta }_a+P_b{\delta }_b+P{\overline{\xi }}_N,$$

where P = 1 − |P_a| − |P_b|, ξ̄_N = {t_1,N, . . ., t_N,N; s_1,N/N, . . ., s_N,N/N} and s_i,N = sign(p(t_i,N)), i = 1, . . ., N.

The matrix W, which corresponds to the design ξ_N+2 and is used in the corresponding WLSE (1.3), is a diagonal matrix W_N = diag(NP_a, s_1,N P, s_2,NP, . . ., s_N,NP, NP_b) of size (N +2) × (N +2). The set of N +2 design points, where the observations should be taken, is given by {a, t_1,N, t_2,N, . . ., t_N,N, b} and the resulting estimate is defined by

$${\widehat{\theta }}_{\mathit{WLSE},N}={({\mathbf{X}}^T{\mathbf{W}}_N\mathbf{X})}^{-1}{\mathbf{X}}^T{\mathbf{W}}_N\mathbf{Y}.$$ (2.17)

It follows from (1.4), (2.15) and the discussion of the previous paragraph that

$$\underset{N\to \infty }{lim}\text{Var}({\widehat{\theta }}_{\text{WLSE},\mathrm{N}})=\underset{N\to \infty }{lim}D({\xi }_{N+2}^{\ast })=D^{\ast },$$

where D^* is defined in (2.14).

2.6 Some numerical results

Consider the regression model (2.1) with f(t) = t² + 1, t ∈ [1, 2], where the error process is given by the Brownian motion. The optimal design for this model can be obtained from Table 3, and we have P_a = 0, P_b = −0.55, P = 0.45 and p(t) = 1.38/(t² + 1). By computing the quantiles from the c.d.f. corresponding to p we can easily obtain support points of (N+2)-point designs. For example, $\text{supp}({\xi }_4^{\ast })=\{1,1.24,1.56,2\},\text{supp}({\xi }_5^{\ast })=\{1,1.18,1.39,1.65,2\}$ and $\text{supp}({\xi }_6^{\ast })=\{1,1.14,1.30,1.49,1.71,2\}$.

In Figure 1 we display the variance of various linear unbiased estimators for different sample sizes. We observe that the variance of the WLSE defined by (2.17) for the proposed (N+2)-point design ${\xi }_{N+2}^{\ast }$ is slightly larger than the variance of the BLUE for the proposed (N +2)-point design, which is very close to the variance of the BLUE with corresponding optimal (N+2)-point design. The calculation of these designs is complicated and has been performed numerically by the Nelder-Mead algorithm in MATLAB. We also note that due to the non-convexity of the optimization problem it is not clear that the algorithm finds the optimal design. However, by Theorem 2.2 and 2.3 we determined the optimal value (2.14), which is D^* ≃ 0.075004. This means that for the proposed designs WLSE has almost the same precision as BLUE.

Figure 1.

The variance of the WLSE defined in (2.17) for the proposed (N +2)-point designs ${\xi }_{N+2}^{\ast }(\mathit{crosses})$, of the BLUE for the proposed (N +2)-point designs (grey circles) and of the BLUE with corresponding optimal (N+2)-point designs (line). The error process in model (2.1) is given by the Brownian motion and the regression function is f(t) = t² + 1, t ∈ [1, 2].

In our second example we compare the proposed optimal designs with the designs from Sacks and Ylvisaker (1966), which are constructed for the BLUE. For this purpose we consider the model (2.1) with regression function f(t) = 1 + 0.5 sin(2πt), t ∈ [1, 2], and triangular covariance kernel of the form (2.4) with u(t) = t² and v(t) = t. The optimal design in the continuous time model can be obtained from Table 4 and its density is depicted in Figure 2.

Figure 2.

The density of the optimal design for continuous time model (2.1) with regression function f(t) = 1+0.5 sin(2πt), t ∈ [1, 2], and covariance kernel of the form (2.4) with u(t) = t² and v(t) = t.

By computing quantiles using this optimal design, we obtain that the 4-point design ${\xi }_4^{\ast }$ is supported at points 1, 1.27, 1.68 and 2. For ${\xi }_4^{\ast }$, the variance of the BLUE is ≃ 0.6129. Using the optimal density from Sacks and Ylvisaker (1966), we obtain the 4-point design ${\xi }_4^{SY}$ supported at 1, 1.25, 1.63 and 2. For ${\xi }_4^{SY}$, the variance of the BLUE is ≃ 0.6200. For N = 2, 3, . . ., 20, the variances of the BLUE for the proposed (N + 2)-point designs, the (N + 2)-point designs from Sacks and Ylvisaker (1966) and the optimal (N + 2)-point designs for the BLUE are depicted in Figure 3. We observe that for N = 2, 3, 4 the new designs yield a smaller variance of the BLUE, while for N = 5 the design of Sacks and Ylvisaker (1966) shows a better performance. In all other cases the results for both designs are very similar. In particular, for N ≥ 6 the variances from the optimal (N + 2)-point designs proposed in this paper and in the paper of Sacks and Ylvisaker (1966) are only slightly worse than the variances of the BLUE with corresponding best (N +2)-point designs (which is computed by direct optimization).

Figure 3.

The variance of BLUE for the proposed (N+2)-point designs (grey circles), the (N+2)-point designs from Sacks and Ylvisaker (1966) (crosses) and the BLUE with corresponding optimal (N + 2)-point designs (line) for the model f(t) = 1 + 0.5 sin(2πt), t ∈ [1, 2], and the covariance kernel with u(t) = t² and v(t) = t; N = 2, . . ., 20.

3 Multi-parameter models

In this section we discuss optimal design problems for models with more than one parameter. The structure of this section is somewhat similar to the structure of Section 2. In Section 3.1 we introduce a new class of linear estimators of the parameters in model (1.3), which we call matrix-weighted estimators (MWE) and show in Lemma 3.3 that for some special choices of the matrix weights the MWE can always emulate the BLUE. In Section 3.2 matrix-weighted designs associated with the MWE are defined. Then, for the case of triangular kernels, in Corollary 3.1 we derive the asymptotic forms for the sequence of designs that are associated with the version of the MWE which emulates the BLUE. In Section 3.3 we prove optimality of the asymptotic matrix-weighted measure derived in Corollary 3.1 in the continuous time model (1.2) (see Theorem 3.1), while some examples of asymptotically optimal measures are provided in Section 3.4. Finally, the practical implementation of the asymptotic measures is discussed in Section 3.5 and numerical examples are provided in Section 3.6.

The proofs of many statements in this section use the results of Section 2. This is possible as there is a lot of freedom in choosing the form of the MWE to emulate the BLUE and we choose a special form which could be considered as component-wise SLSE. Correspondingly, the resulting matrix-weighted designs (including the asymptotic ones) become combinations of designs for one-parameter models.

3.1 Matrix-weighted estimators and designs

Consider the regression model (1.1) and assume that N observations at points t_j (j = 1, . . ., N) have been made. Let O_j be an m × m matrix associated with the observation point t_j; j = 1, . . ., N. Recall the definition of the design matrix $\mathbf{X}={(f_i(t_j))}_{j=1,\dots ,N}^{i=1,\dots ,m}$ and the definition of Y = (y(t₁), . . ., y(t_N))^T. We introduce the m × N matrix C = (O₁f(t₁), . . ., O_N f(t_N)), whose j-th column is O_jf(t_j). Assuming that the m × m matrix

$$\mathbf{M}=\mathbf{CX}=\sum _{j=1}^N{\mathbf{O}}_{j}f(t_j)f^T(t_j)$$ (3.1)

is non-singular we define the linear estimator

$${\widehat{\theta }}_{\mathit{MWE}}={(\mathbf{CX})}^{-1}\mathbf{CY}$$ (3.2)

for the vector θ in model (1.1). We call this estimator the matrix-weighted estimator (MWE), because each column of the matrix X is multiplied by a matrix weight. It is easy to see that for any C the MWE θ̂_MWE is unbiased and its covariance matrix is given by

$$\text{Var}({\widehat{\theta }}_{\mathit{MWE}})={\mathbf{M}}^{-1}\mathbf{C}\mathbf{\sum }{\mathbf{C}}^T{({\mathbf{M}}^{-1})}^T,$$ (3.3)

where Σ = (K(t_i, t_j))_i,j=1,...,N is the N ×N matrix of covariances of the errors. Note that the matrix M defined in (3.1) generalizes the standard information matrix X^TX and that M is not necessarily a symmetric matrix. The following result shows that different matrices O₁, . . ., O_N may yield the same matrix-weighted estimator θ̂_MWE. Its proof is obvious and therefore omitted.

Lemma 3.1

Consider the regression model (1.1) and assume that the matrix M defined in (3.1) is non-singular. Then the estimator θ̂_MWE defined in (3.2) coincides with the estimator θ̂_MWE,Λ = (C_ΛX)⁻¹C_ΛY, where C_Λ = ΛC and Λ is an arbitrary non-singular m × m matrix.

The estimator θ̂_MWE,Λ introduced in Lemma 3.1 is the MWE defined by the matrix weights ΛO₁, . . ., O_N. Lemma 3.1 implies that the θ̂_MWE is exactly the same for any set of matrices {ΛO₁, . . ., ΛO_N} as long as Λ is non-singular. In the asymptotic considerations below it will be convenient to interpret the combination of the set of experimental conditions {t₁, . . ., t_N} and the set of corresponding matrices {O₁, . . ., O_N} in the MWE as an N-point matrix-weighted design.

Definition 3.1

Any combination of N points {t₁, . . ., t_N} and m × m matrices {O₁, . . ., O_N} will be called N-point matrix-weighted design and denoted by

$${\xi }_N=\{t_1,\dots ,t_N;\frac{1}{N}{\mathbf{O}}_1,\dots ,\frac{1}{N}{\mathbf{O}}_N\}.$$ (3.4)

The covariance matrix D(ξ_N) of a matrix-weighted design ξ_N is defined as the covariance matrix Var(θ̂_MWE) in (3.3) of the corresponding estimate θ̂_MWE.

The estimator θ̂_MWE is not necessarily a least-squares type estimator; that is, it may not be representable in the form (1.3) for some N × N weight matrix W and hence there may be no associated weighted sum of squares which is minimized by the MWE. However, for any given W, we can always find matrices O_j such that

$$\mathbf{C}={\mathbf{X}}^T\mathbf{W}$$ (3.5)

and therefore achieve θ̂_MWE = θ̂_WLSE. The following result gives a constructive solution to the matrix equations (3.5).

Lemma 3.2

Assume that f₁(t) ≠ 0 for all t ∈ [a, b]. Define ${\mathbf{O}}_j={\omega }_j{e}_1^T$,

$${\omega }_j=\frac{1}{f_1(t_j)}{({\mathbf{X}}^T\mathbf{W})}_j\in {ℝ}^m,$$ (3.6)

where e₁ = (1, 0, . . ., 0)^T ∈ ℝ^m is the first unit vector and (X^TW)_j denotes the j-th column of the m × N matrix X^TW. Then the corresponding matrix-weighted estimator satisfies θ̂_MWE = θ̂_WLSE.

Proof

The matrix equation (3.5) can be written as N vector equations

$${\mathbf{O}}_{j}f(t_j)={({\mathbf{X}}^T\mathbf{W})}_j;j=1,\dots ,N,$$ (3.7)

with respect to the matrices O_j. Assume that ${\mathbf{O}}_j={\omega }_j{e}_1^T$ for some ω_j ∈ ℝ^m. Then

$${\mathbf{O}}_{j}f(t_j)={\omega }_j{e}_1^{T}f(t_j)={\omega }_j{f}_1(t_j)$$

and equation (3.7) has the unique solutions (3.6).

The form ${\mathbf{O}}_j={\omega }_j{e}_1^T$ for the matrices O_j considered in Lemma 3.2 means that the matrix O_j has the vector ω_j as its first column while all other entries in this matrix are zero. We shall refer to this form as the one-column form. We can choose other forms for the matrices O_j, but then we would require different, somewhat stronger, assumptions regarding the vector f(t). For example, if f(t) ≠ (0, . . ., 0)^T for all t ∈ [a, b], then we can always choose diagonal matrices O_j to satisfy (3.5) (see Lemma 3.5 below).

The following choices for O_j ensure coincidence of θ̂_MWE with the three popular estimators defined in the Introduction.

If O_j = I_m for all j, then θ̂_MWE = θ̂_OLSE.

If O_j = s_jI_m for all j, then θ̂_MWE = θ̂_SLSE.

If W = Σ⁻¹ and ${\mathbf{O}}_j={\omega }_j{e}_1^T$ with ω_j = (X^TΣ⁻¹)_j/f₁(t_j), then θ̂_MWE = θ̂_BLUE.

We shall call any MWE θ̂_MWE optimal if it coincides with the BLUE. In view of the importance of the last case, the corresponding result is summarized in the following lemma.

Lemma 3.3

Consider the regression model (1.1) and let f₁(t) ≠ 0 for all t ∈ [a, b]. For a given set of N observation points {t₁, . . ., t_N} the MWE θ̂_MWE defines a BLUE if ${\mathbf{O}}_j={\omega }_j^{\ast }{e}_1^T$ with ${\omega }_j^{\ast }={({\mathbf{X}}^T{\mathbf{\sum }}^{-1})}_j/f_1(t_j)$.

If the covariance kernel of the error process has triangular form (2.4) then we can derive the explicit form for the optimal MWE. The result follows by a direct application of Lemma A.1.

Lemma 3.4

Assume that the covariance kernel K(·, ·) has the form (2.4) and that the matrix Σ = (K(t_i, t_j))_i,j=1,...,N is positive definite with entries K(t_i, t_j) = u_iv_j for i ≤ j, where for k = 1, . . ., N we denote u_k = u(t_k), v_k = v(t_k) and q_k = u_k/v_k. Then we have the following representation for the optimal vectors ${\omega }_j^{\ast }={({\mathbf{X}}^T{\mathbf{\sum }}^{-1})}_j/f_1(t_j)\in {ℝ}^m$ introduced in Lemma 3.3:

$${\omega }_1^{\ast }=\frac{c}{f_1(t_1)}({\stackrel{\sim }{\sigma }}_{11}f(t_1)+{\stackrel{\sim }{\sigma }}_{12}f(t_2))=\frac{c{u}_2}{f_1(t_1)v_1{v}_2(q_2-q_1)}\left(\frac{f(t_1)}{u_1}-\frac{f(t_2)}{u_2}\right),$$ (3.8)

$$\begin{array}{l}{\omega }_N^{\ast }=\frac{c}{f_1(t_N)}({\stackrel{\sim }{\sigma }}_{N,N}f(t_N)+{\stackrel{\sim }{\sigma }}_{N-1,N}f(t_{N-1}))\\ =\frac{c}{f_1(t_N)v_N(q_N-q_{N-1})}\left(\frac{f(t_N)}{v_N}-\frac{f(t_{N-1})}{v_{N-1}}\right),\end{array}$$ (3.9)

$$\begin{array}{l}{\omega }_i^{\ast }=\frac{c}{f_1(t_i)}({\stackrel{\sim }{\sigma }}_{i,i}f(t_i)+{\stackrel{\sim }{\sigma }}_{i-1,i}f(t_{i-1})+{\stackrel{\sim }{\sigma }}_{i,i+1}f(t_{i+1}))\\ =\frac{c}{f_1(t_i)v_i}\left(\frac{(q_{i+1}-q_{i-1})f(t_i)}{v_i(q_{i+1}-q_i)(q_i-q_{i-1})}-\frac{f(t_{i-1})}{v_{i-1}(q_i-q_{i-1})}-\frac{f(t_{i+1})}{v_{i+1}(q_{i+1}-q_i)}\right),\end{array}$$ (3.10)

for i = 2, . . ., N − 1. Here in formulas (3.8), (3.9) and (3.10) σ̃_ij denote the elements of the matrix Σ⁻¹ = (σ̃_ij)_i,j=1,...,N.

The following provides a result similar to Lemmas 3.2 and 3.3 in the case where the matrices O_j are diagonal. An extension of Lemma 3.4 to the matrices O_j of the diagonal form is straightforward and omitted for the sake of brevity.

Lemma 3.5

Consider the regression model (1.1) and let f_k(t) ≠ 0 for all t ≠ [a, b] and all k = 1, . . ., m. For each j = 1, . . ., N, define the diagonal matrix O_j by its diagonal elements

$${({\mathbf{O}}_j)}_{k,k}=\frac{1}{f_k(t_j)}{({\mathbf{X}}^T\mathbf{W})}_{k,j};k=1,\dots ,m,$$

where (X^TW)_k,j denotes the (k, j)-th element of the matrix X^TW. Then θ̂_MWE = θ̂_WLSE. If additionally W = Σ⁻¹ so that (O_j)_k,k = (X^TΣ⁻¹)_k,j/f_k(t_j), then θ̂_MWE = θ̂_BLUE.

3.2 Weak convergence of matrix-weighted designs

Let Q : [a, b] → [0, 1] be an increasing function on the interval [a, b] with Q(a) = 0 and Q(b) = 1 so that Q(·) is a c.d.f. For a fixed N and j = 1, . . ., N, define the points t_1,N, . . ., t_N,N by (2.9). Suppose that with each t ∈ [a, b] we can associate an m × m matrix O(t) and consider an N-point matrix-weighted design ξ_N of the form (3.4) with t_j = t_j,N and O_j = O(t_j,N). In view of (3.1) and (3.3) this design has the covariance matrix

$$\mathbf{D}({\xi }_N)={\mathbf{M}}^{-1}({\xi }_N)\mathbf{B}({\xi }_N){({\mathbf{M}}^{-1}({\xi }_N))}^T,$$

where the matrices M(ξ_N) and B(ξ_N) are defined by

$$\begin{array}{ll}\hfill \mathbf{M}({\xi }_N)& =\frac{1}{N}\sum _{j=1}^N\mathbf{O}(t_{j,n})f(t_{j,n})f^T(t_{j,n}),\hfill \\ \hfill \mathbf{B}({\xi }_N)& =\frac{1}{N^2}\sum _{i=1}^N\sum _{j=1}^{N}K(t_{i,n},t_{j,n})\mathbf{O}(t_{i,n})f(t_{i,n})f^T(t_{j,n}){\mathbf{O}}^T(t_{j,n}).\hfill \end{array}$$

In addition to the sequence of matrix-weighted designs ξ_N consider the sequence of uniform distributions on the set {t_1,N, . . ., t_N,N}. As N → ∞, this sequence converges weakly to the design (probability measure) on the interval [a, b] with distribution function Q. This implies

$$\begin{array}{c}\underset{N\to \infty }{lim}\mathbf{M}({\xi }_N)=\mathbf{M}(\xi )={\int }_a^b\mathbf{O}(t)f(t)f^T(t)\zeta (dt)\\ \underset{N\to \infty }{lim}\mathbf{B}({\xi }_N)=\mathbf{B}(\xi )={\int }_a^b{\int }_a^{b}K(t,s)\mathbf{O}(t)f(s)f^T(t){\mathbf{O}}^T(s)\zeta (dt)\zeta (ds),\end{array}$$

and

$$\underset{N\to \infty }{lim}\mathbf{D}({\xi }_N)=\mathbf{D}(\xi )={\mathbf{M}}^{-1}(\xi )\mathbf{B}(\xi ){({\mathbf{M}}^{-1}(\xi ))}^T$$ (3.11)

under the assumptions that the vector-valued function f, the matrix-valued function O, the kernel K are continuous on the interval [a, b] and the generalized information matrix M(ξ) are non-singular. Moreover, the sequence of estimators (3.2) converges (almost surely as N → ∞) to

$${\widehat{\theta }}_{\mathit{MWE},\infty }={\mathbf{M}}^{-1}(\xi ){\int }_a^b\mathbf{O}(t)f(t)y(t)\zeta (dt),$$ (3.12)

where {y(t) | t ∈ [a, b]} is the stochastic process in the continuous time model (1.2). Bearing these limiting expressions in mind we say that the sequence of matrix-weighted designs ξ_N defined by (3.4) converges to the limiting matrix-weighted design ξ(dt) = O(t)ζ(dt) as N → ∞. This relation justifies the notation M(ξ), B(ξ) and D(ξ) of the previous paragraph.

The (optimal) limiting matrix-weighted designs which will be constructed below will have a similar structure as the signed measures in (2.12). They will assign matrix weights O_a and O_b to the end-points of the interval [a, b] and a ‘matrix density’ O(t) to the points t ∈ (a, b); that is, these designs will have the form

$$\xi (dt)={\mathbf{O}}_a{\delta }_a(dt)+{\mathbf{O}}_b{\delta }_b(dt)+\mathbf{O}(t)dt.$$ (3.13)

In view of (3.12), the MWE in the continuous time model (1.2) associated with any design of the form (3.13) can be written as

$${\widehat{\theta }}_{\mathit{MWE}}(\xi )={\mathbf{M}}^{-1}(\xi )\left[{\mathbf{O}}_{a}f(a)y(a)+{\mathbf{O}}_{b}f(b)y(b)+{\int }_a^b\mathbf{O}(t)f(t)y(t)dt\right],$$ (3.14)

where $\mathbf{M}(\xi )={\mathbf{O}}_{a}f(a)f^T(a)+{\mathbf{O}}_{b}f(b)f^T(b)+{\int }_a^b\mathbf{O}(t)f(t)f^T(t)dt$. In the particular case associated with Lemma 3.4, we have the following structure of the matrices O_a and O_b and the matrix function O(t) in (3.13):

$${\mathbf{O}}_a={\omega }_a{e}_1^T,{\mathbf{O}}_b={\omega }_b{e}_1^T,\mathbf{O}(t)=\omega (t)e_1^T\text{for}t\in (a,b),$$ (3.15)

where ω_a and ω_b are some m-dimensional vectors and ω(t) ∈ ℝ^m is some vector-valued function defined on the interval (a, b). Note that ω(t) does not have to approach ω_a and ω_b as t → a and t → b, respectively.

When the sequence of matrix-weighted designs is defined by the formulas of Lemma 3.3 we can compute the limiting matrix-weighted design. The proof follows by similar arguments as given in the proof of Theorem 2.1 and is therefore omitted.

Corollary 3.1

Consider model (1.1), where the error process {ε(t)| t ∈ [a, b]} has a covariance kernel K of the form (2.4). Assume that u(·), v(·), q(·) are strictly positive, twice continuously differentiable functions on the interval [a, b] and that the vector-valued function f(·) is twice continuously differentiable with f₁(t) ≠ 0 for all t ∈ [a, b]. Consider the matrix-weighted design ξ_N of the form (3.4), where the support points t_j = t_j,N are generated by (2.9) and the matrix weights O_j = O_j,N are defined in Lemma 3.3. The sequence {ξ_N}_N∈N converges (in the sense defined above in the previous paragraph) to a matrix-weighted design ξ defined by (3.13) and (3.15) with

$${\omega }_a=\frac{c}{f_1(a)v^2(a)q^{\prime }(a)}\left[\frac{f\left(a\right)u^{\prime }(a)}{u(a)}-f^{\prime }(a)\right],{\omega }_b=\frac{c{h}^{\prime }(b)}{f_1(b)v(b)q^{\prime }(b)},\omega (t)=\frac{-c}{f_1(t)v(t)}{\left[\frac{h^{\prime }(t)}{q^{\prime }(t)}\right]}^{\prime }$$ (3.16)

where h(t) = f(t)/v(t) and the constant c ≠ 0 is arbitrary.

In Corollary 3.1, the one-column representation of the matrices O_j is used. The following statement contains a similar result for the case where the matrices O_j are diagonal.

Corollary 3.2

Let the conditions of Corollary 3.1 hold and assume additionally that f_k(t) ≠ 0 for all t ∈ [a, b] and all k = 1, . . . , m. Consider the matrix-weighted design ξ_N of the form (3.4), where the support points t_j = t_j,N are generated by (2.9) and the matrices O_j = O_j,N are defined in Lemma 3.5 with diagonal elements given by (O_j)_k,k = (X^TΣ⁻¹)_k,j/f_k(t_j). Then the sequence {ξ_N}_N×ℕ converges to the optimal matrix-weighted design ξ^* of the form (3.13), where the diagonal elements of the matrices O_a = diag(O_a,11, . . . , O_a,mm), O_b = diag(O_b,11, . . . , O_b,mm) and O(t) = diag(O₁₁(t), . . . , O_mm(t)) are given by

$${\mathbf{O}}_{a,jj}=\frac{c}{f_j(a)v^2(a)q^{\prime }(a)}\left[\frac{f_j(a)u^{\prime }(a)}{u(a)}-f_j^{\prime }(a)\right],{\mathbf{O}}_{b,jj}=\frac{c{h}_j^{\prime }(b)}{f_j(b)v(b)q^{\prime }(b)},{\mathbf{O}}_{jj}(t)=\frac{-c}{f_j(t)v(t)}{\left[\frac{h_j^{\prime }(t)}{q^{\prime }(t)}\right]}^{\prime }$$

respectively, h_j(t) = f_j(t)/v(t), j = 1, . . . , m and the constant c ≠ 0 is arbitrary.

3.3 Optimal designs and best linear estimators

In this section we consider again the continuous time model (1.2), where the full trajectory of the process {y(t)|t ∈ [a, b]} can be observed. We start recalling some known facts concerning best linear unbiased estimation. For details we refer the interested reader to the work of Grenander (1950) or Section 2.2 in Näther (1985a). Any linear estimator of θ can be written in the form of the integral

$${\widehat{\theta }}_{\mu }={\int }_a^{b}y(t)\mu (dt),$$ (3.17)

where μ(t) = (μ₁(t), . . . , μ_m(t))^T is a vector of signed measures on the interval [a, b]. For given μ, the estimator θ̂_μ is unbiased if and only if ${\int }_a^{b}f(t){\mu }^T(dt)={\mathbf{I}}_m$, where I_m denotes the m-dimensional identity matrix. Theorem 2.3 in Näther (1985a) states that the estimator θ̂_μ^* is BLUE if and only if ${\int }_a^{b}f(t){\mu }^{\ast T}(dt)={\mathbf{I}}_m$ and the identity

$${\int }_a^{b}K(u,v){\mu }^{\ast }(dv)=\mathbf{A}f(u)$$

holds for all u ∈ [a, b], where A is some m × m matrix. The matrix A is uniquely defined and coincides with the matrix

$${\mathbf{D}}^{\ast }=\text{Var}({\widehat{\theta }}_{{\mu }^{\ast }})=inf\left\{{\int }_a^b{\int }_a^{b}K(u,v)\mu (du){\mu }^T(dv)|\mu \text{vector}\text{of}\text{signed}\text{measures}\right\}.$$ (3.18)

The Gauss-Markov theorem further implies that D^* ≤ Var(θ̂), where θ̂ is any other linear unbiased estimator of θ.

Definition 3.2

A matrix-weighted design ξ^* is called optimal if D(ξ^*) = D^*, where D(ξ) is defined in (3.11) and D^* is defined in (3.18).

The designs we consider have the form (3.13) and the corresponding MWE are expressed by (3.14). The estimator (3.14) can be expressed in the form (3.17), that is θ̂_MWE(ξ) = θ̂_μ with

$$\mu (dt)={\mathbf{M}}^{-1}(\xi )[{\mathbf{O}}_{a}f(a){\delta }_a(dt)+{\mathbf{O}}_{b}f(b){\delta }_b(dt)+\mathbf{O}(t)f(t)dt].$$

The estimators defined in (3.14) are always unbiased and the following result provides the matrix-weighted optimal design and the BLUE in the continuous time model (1.2). The proof follows by similar arguments as given in the proof of Theorem 2.2 and 2.3 and is therefore omitted.

Theorem 3.1

Let K(t, s) be a covariance kernel of the form (2.4) and the vector-function f(·) be twice continuously differentiable with f₁(t) ≠ 0 for all t ∈ [a, b]. Under the assumptions of Corollary 3.1 the matrix-weighted design ξ^* defined by the formulas (3.13) and (3.16) with c = 1 is optimal in the sense of Definition 3.2. Moreover, if

$${\mu }^{\ast }(dt)={\mathbf{M}}^{-1}({\xi }^{\ast })[{\omega }_a{e}_1^{T}f(a){\delta }_a(dt)+{\omega }_b{e}_1^{T}f(b){\delta }_b(dt)+\omega (t)e_1^{T}f(t)dt],$$

then θ̂_μ^* defines the BLUE in model (1.2). Additionally, we have

$$\mathbf{D}({\xi }^{\ast })={\mathbf{D}}^{\ast }={\int }_a^b{\int }_a^{b}K(s,t){\mu }^{\ast }(ds){\mu }^{\ast }(dt)={\mathbf{M}}^{-1}({\xi }^{\ast }),$$

where the matrix M(ξ^*) is given by

$$\mathbf{M}({\xi }^{\ast })=\frac{\stackrel{\sim }{f}(q(a)){\stackrel{\sim }{f}}^T(q(a))}{q(a)}+{\int }_{q(a)}^{q(b)}{\stackrel{\sim }{f}}^{\prime }(s){{\stackrel{\sim }{f}}^{\prime }}^T(s)ds$$

and f̃(s) = f(q⁻¹(s))/v(q⁻¹(s)).

In Theorem 3.1 we have used the one-column representation for the matrices O(t). Similar arguments establish the optimality of the matrix-weighted designs ξ^* defined in Corollary 3.2 where the diagonal representation for the matrices O(t) is used. The details are omitted for the sake of brevity.

3.4 Examples of optimal matrix-weighted designs

Consider the polynomial regression model with f(t) = (1, t, t², . . . , t^m−1)^T, t ∈ [a, b] and the covariance kernel of the Brownian motion K(t, s) = min(t, s). For the construction of matrix-weighted designs we use matrices O(t) in the one-column and diagonal representations.

For the one-column representation we have from Corollary 3.1 and Theorem 3.1 that the optimal matrix weighted design has masses ${\mathbf{O}}_a={\omega }_a{e}_1^T$ and ${\mathbf{O}}_b=\omega (b)e_1^T$ at points a and b, respectively, and the density $\mathbf{O}(t)=\omega (t)e_1^T$. Here the vectors ω_a, ω_b and ω(t) are given by

$$\begin{array}{ll}\hfill {\omega }_a& ={(1/a,0,-a,\dots ,(2-m)a^{m-2})}^T,\hfill \\ \hfill {\omega }_b& ={(0,1,2b,3b^2,\dots ,(m-1)b^{m-2})}^T,\hfill \\ \hfill \omega (t)& ={(0,0,-2,-3·2t,\dots ,-(m-1)(m-2)t^{m-3})}^T,t\in (a,b),\hfill \end{array}$$

respectively. For the diagonal representation we have from Corollary 3.2 (and an analogue of Theorem 3.1) that the optimal matrix weighted design has masses O_a and O_b at points a and b, respectively, and the density O(t), where

$$\begin{array}{ll}\hfill {\mathbf{O}}_a& =\text{diag}(1/a,0,-1/a,\dots ,(2-m)/a),\hfill \\ \hfill {\mathbf{O}}_b& =\text{diag}(0,1/b,2/b,\dots ,(m-1)/b),\hfill \\ \hfill \mathbf{O}(t)& =\text{diag}(0,0,-2/t^2,\dots ,-(m-1)(m-2)/t^2),t\in (a,b).\hfill \end{array}$$

Note that in this case all non-vanishing diagonal elements of the matrix O(t) are proportional to the function 1/t². According to Lemma 3.1, we can use ΛO(t) instead of O(t), for any non-singular m × m matrix Λ. By taking the matrix

$$\mathrm{\Lambda }=\text{diag}(1,1,-1/2,\dots ,-1/[(m-1)(m-2)]),$$

we obtain

$$\begin{array}{ll}\hfill \mathrm{\Lambda }\mathbf{O}(a)& =\text{diag}(1/a,0,1/(2a),\dots ,1/[(m-1)a]),\hfill \\ \hfill \mathrm{\Lambda }\mathbf{O}(b)& =\text{diag}(0,1/b,-1/b,\dots ,-1/[(m-2)b]),\hfill \\ \hfill \mathrm{\Lambda }\mathbf{O}(t)& =\text{diag}(0,0,1/t^2,\dots ,1/t^2),t\in (a,b).\hfill \end{array}$$

As another example, consider the polynomial regression model with f(t) = (1, t, t², . . . , t^m−1)^T, t ∈ [a, b], and the triangular covariance kernel of the function (2.4) with u(t) = t^γ and v(t) = t^ω. For the diagonal representation we have from Corollary 3.2 that the optimal matrix weighted design has masses O_a and O_b at points a and b, respectively, and the density O(t), where

$$\begin{array}{ll}\hfill {\mathbf{O}}_a& =a^{-\gamma -\omega }\text{diag}(-\gamma ,1-\gamma ,2-\gamma ,\dots ,m-1-\gamma ),\hfill \\ \hfill {\mathbf{O}}_b& =b^{-\gamma -\omega }\text{diag}(\omega ,\omega -1,\omega -2b,\dots ,\omega +1-m),\hfill \\ \hfill \mathbf{O}(t)& =t^{-1-\gamma -\omega }\text{diag}({\tau }_1,{\tau }_2,\dots ,{\tau }_m),t\in (a,b),\hfill \end{array}$$

with τ_i = (i − 1 − γ) (i − 1 − ω), i = 1, . . . , m. If we further use Λ = diag(1/τ₁, 1/τ₂, . . . , 1/τ_m) then we obtain ΛO(t) = t^{−1−γ−ω} diag(1, 1, . . . , 1), t ∈ (a, b); that is, all components of the matrix ΛO(t) have exactly the same density.

3.5 Practical implementation

Here we only consider the diagonal representation of the matrices O_a, O_b and O(t); the case of one-column representation of the matrices O can be treated similarly. We assign matrix weights O_a and O_b to the boundary points a and b and use an N-point approximation to an absolutely continuous probability measure on (a, b) with some density φ(t). The density φ(t) is defined to be either the uniform density on (a, b) (if nonzero elements of different components of O(t) are not proportional to each other) or φ (t) = c|O_l,l(t)| for some l ∈ {1, . . . , m} (if nonzero elements of different components of O(t) are proportional to each other), where c is the normalization constant and l is such that the density φ (t) is not identically zero on the interval (a, b). Denote by $F(t)={\int }_a^t\phi (s)ds$ the corresponding c.d.f. For given N, we calculate an N-point approximation {t_1,N, . . . , t_N,N; 1/N, . . . , 1/N}, where t_i,N = F⁻¹(z_i,N) with z_i,N = i/(N + 1), i = 1, 2, . . . , N, to the probability measure with density φ (t).

To each point t_j,N we assign a vector of weights s_j = (s_j,N,1, . . . , s_j,N,m)^T such that s_j,N,k ∈ {−1, 0, 1} (k = 1, . . . , m). The values s_j,N,k = sign(O_k,k(t_j)) = ±1 correspond to the sign of the point t_j,N in the estimation of θ_k exactly as in the procedure for one-parameter models described in Section 2.5. Some of the values s_j,N,k could be 0. If s_j,N,k = 0 for some k then the point t_j,N is not used for the estimation of θ_k. By assigning zero weight to a point t_j,N in the k-th estimation direction, we perform a thinning of the sample of points t_1,N, . . . , t_N,N in k-th direction and thus achieve a required density in the each estimation direction. This is a deterministic version of the well-known ‘rejection method’ widely used to generate samples from various probability distributions. If the nonzero components of the matrix weight O(t) are proportional to each other then for these components s_j,N,k = 1 for all j and N.

The resulting estimator θ̂ has the form (3.2) where

$$\begin{array}{ll}\hfill \mathbf{C}& =\left(N{\mathbf{O}}_{a}f(a),{\mathbf{S}}_1\mathbf{P}f(t_1)\dots ,{\mathbf{S}}_N\mathbf{P}f(t_N),N{\mathbf{O}}_{b}f(b)\right),\hfill \\ \hfill {\mathbf{S}}_j& =\text{diag}(s_{j,N,1},\dots ,s_{J,N,m})\in {ℝ}^{m\times m}\hfill \end{array}$$

and P is the diagonal m × m matrix whose diagonal elements are given by

$${\mathbf{P}}_{k,k}=\frac{N}{{\sum }_{j=1}^N\mid s_{j,N,k}\mid }{\int }_a^b{\mathbf{O}}_{k,k}(t)dt.$$

If nonzero elements of different components of the matrix weight O(t) are proportional to each other (as was the case in the examples of Section 3.4) then the (N+2)-point approximations to the limiting design are very similar to the approximations in the one-parameter case considered in Section 2.5; their accuracy is also very high. Otherwise, when the diagonal elements of O(t) are possibly non-proportional, the accuracy of approximations will depend on the degree of non-homogeneity of components of the matrix weight O(t).

3.6 Some numerical results

For comparison of competing matrix weighted designs for multiparameter models it is convenient to consider a functional of the covariance matrix. Exemplarily we investigate in this Section the classical D-optimality criterion defined as Ψ (D(ξ)) = (det D(ξ))^1/m which has to be minimized.

As an example where all nonzero elements of the matrix O(t) are proportional to each other, let us consider the cubic regression model with f(t) = (1, t, t², t³)^T and the Brownian motion error process. The optimal value in the continuous time model (1.2) is Ψ (D^*) ≃ 2.7927. In Figure 4 we display the D-criterion of the covariance matrices of the MWE and the BLUE for the proposed (N+2)-point designs and the covariance matrix of the BLUE with corresponding optimal (N+2)-point designs. We can see that the D-efficiency of the proposed matrix-weighted design is very high, even for small N.

Figure 4.

The D-optimality criterion of the covariance matrix of the MWE for the proposed (N+2)-point designs (crosses), of the covariance matrix of the BLUE for the proposed (N+2)-point designs (line) and of the covariance matrix of the BLUE with corresponding D-optimal (N +2)-point designs (grey circles). The error process in model (1.1) is the Brownian motion and the vector of regression functions is given by f(t) = (1, t, t², t³), t ∈ [1, 2].

The second example of this section considers a situation where nonzero elements of the matrix O(t) are not proportional to each other. For this purpose we consider the model (2.1) with f(t) = (1, t, t²)^T, t ∈ [1, 2] and covariance kernel K(t, t′) = e^{−|t−t′|} with u(t) = e^t and v(t) = e^−t. Using the diagonal representation, we obtain for the optimal matrix-weighted designs

$${\mathbf{O}}_a=\text{diag}(1,0,-1),{\mathbf{O}}_b=\text{diag}(1,1.5,2),\mathbf{O}(t)=\text{diag}(1,1,1-2/t^2),t\in (1,2).$$

The optimal value in the continuous time model (1.2) is given by Ψ (D^*) ≃ 1.6779. Since some diagonal elements of O(t) are constant functions, we take the support points of the design ξ_N+2 to be equidistant: t_i,N = i/(N +1) for i = 1, . . . , N. Then we have s_j,N,k = 1 for all j = 1, . . . , N and k = 1, 2. However, some elements of (s_1,N,3, . . . , s_N,N,3) should be zero because O_3,3(t) is not proportional to O_1,1(t). For example, for N = 10 the vector of signs (s_1,N,3, . . . , s_N,N,3) is (−1, −1, 0, 0, 0, 1, 0, 0, 1, 0) and for N = 30 it is (−1, 0, −1, −1, 0, −1, 0, 0, −1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1).

In Figure 5 we depict the D-optimality criterion of the covariance matrices for various estimators. We observe that in this example for all N the D-optimality criterion of the covariance matrices of the MWE is slightly larger than the D-optimality criterion of the covariance matrices of the BLUE. However, we can also see that the proposed (N +2)-point designs are very efficient compared to the BLUE with corresponding D-optimal (N +2)-point designs even for small N.

Figure 5.

The D-optimality criterion of the covariance matrix of the MWE for the proposed (N+2)-point designs (crosses), of the covariance matrix of the BLUE for the proposed (N+2)-point designs (line) and of the BLUE with corresponding D-optimal (N+2)-point designs (grey circles). The covariance kernel in model (1.1) is K(t, t′) = e^{−|t−t′|} and the vector of regression functions is f(t) = (1, t, t²), t ∈ [1, 2].

A Proof of main results

A.1 Explicit form of the inverse of the covariance matrix of errors

Here we state an auxiliary result, which gives an explicit form for the inverse of the matrix $\mathbf{\sum }={(K(t_i,t_j))}_{i,j=1}^N$, with a triangular covariance kernel K. We did not find this result (as formulated below) in the literature. Versions of Lemma A.1, however, have been derived independently by different authors; see, for example, Lemma 7.3.2 in Zhigljavsky (1991) and formula (8) in Harman and Štulajter (2011). The proof follows from straightforward checking the condition Σ⁻¹Σ = ΣΣ⁻¹ = I.

Lemma A.1

Consider a symmetric N × N matrix Σ = (σ_i,j)_i,j=1,...,N which elements are defined by the formula σ_i,j = u_iv_j for 1 ≤ i ≤ j ≤ N. Assume that q₁ < q₂ < . . . < q_N where q_i = u_i/v_i. Then the inverse matrix Σ̃ = Σ⁻¹ is a symmetric tri-diagonal matrix and its elements σ̃_i,j with i ≤ j can be computed as follows:

$$\begin{array}{c}{\stackrel{\sim }{\sigma }}_{1,1}=\frac{u_2}{u_1{v}_1{v}_2(q_2-q_1)},{\stackrel{\sim }{\sigma }}_{N,N}=\frac{1}{v_N^2(q_N-q_{N-1})},\\ {\stackrel{\sim }{\sigma }}_{i,i}=\frac{q_{i+1}-q_{i-1}}{v_i^2(q_i-q_{i-1})(q_{i+1}-q_i)}(i=2,\dots ,N-1),\\ {\stackrel{\sim }{\sigma }}_{i,i+1}=-\frac{1}{v_i{v}_{i+1}(q_{i+1}-q_i)}(i=1,\dots ,N-1),\\ {\stackrel{\sim }{\sigma }}_{i,i+k}=0(i=1,\dots ,N-2,k\ge 2).\end{array}$$

In our applications of Lemma A.1 we assume that σ_i,j = K(t_i, t_j) with the covariance kernel K having the form (2.4).

A.2 Proof of Lemma 2.1

Denote K_ij = K(t_i, t_j), f (t_i) = f_i, a_i = f_iw_i, i, j = 1, . . . , N , a = (a₁, . . . , a_N)^T. Then for any signed measure ξ = {t₁, . . . , t_N; w₁, . . . , w_N} we have

$$D(\xi )=\frac{{\sum }_i{\sum }_j{K}_{ij}{f}_i{f}_j{w}_i{w}_j}{{({\sum }_i{f}_i^2{w}_i)}^2}=\frac{{\sum }_i{\sum }_j{K}_{ij}{a}_i{a}_j}{{({\sum }_i{f}_i{a}_i)}^2}=\frac{{\mathbf{a}}^T\mathbf{\sum }\mathbf{a}}{{({\mathbf{a}}^T\mathbf{f})}^2}.$$

Since Σ is symmetric and Σ > 0, there exists Σ⁻¹ and a symmetric matrix Σ^1/2 > 0 such that Σ = Σ^1/2Σ^1/2. Denote b = Σ^1/2a and d = Σ^−1/2f. Then we can write the design optimality criterion D(ξ) as D(ξ) = b^Tb/(b^Td)². The Cauchy-Schwartz inequality gives for any two vectors b and d the inequality (b^Td)² ≤ (b^Tb)(d^Td), that is, b^Tb/(b^Td)² ≥ 1/(d^Td). This inequality with b and d as above is equivalent to D(ξ) ≥ 1/f^T Σ⁻¹f for all ξ. Equality is attained if the vector b is proportional to the vector d; that is, if b_i = cd_i for all i and any c ≠ 0. Finally, the equality b_i = cd_i can be rewritten in the form w_i = c(Σ⁻¹f)_i/f(t_i).

A.3 Proof of Theorem 2.1

Before starting the main proof we recall the definition of the design points (2.9) and prove the following auxiliary result.

Lemma A.2

Assume that q(·) = u(·)/v(·) is a twice continuously differentiable function on the interval [a, b]. Then for all i = 1, . . . , N − 1, we have

$$t_{i+1,N}-t_{i,N}=\frac{1}{N{Q}^{\prime }(t_{i,N})}+O\left(\frac{1}{N^2}\right)\text{as}N\to \infty .$$ (A.1)

$${\mathrm{\Delta }}_n=(t_{i+1,N}-t_{i-1,N})/2=\frac{1}{N{Q}^{\prime }(t_{i,N})}\left(1+O\left(\frac{1}{N}\right)\right)\text{as}N\to \infty .$$ (A.2)

Proof of Lemma A.2

Recall the definition z_i,N = (i − 1/2)/N (i = 1, . . . , N) and set

$$m=q(a)=\underset{t\in [a,b]}{min}q(t),M=q(b)=\underset{t\in [a,b]}{max}q(t).$$

From the definition of the function Q in (2.8) we have

$$q(t_{i+1,N})-q(t_{i,N})=(M-m)(z_{i+1,N}-z_{i,N})=\frac{M-m}{N}$$ (A.3)

for all i = 1, . . . , N − 1. Observing Taylor’s formula yields for any z

$$Q^{-1}(z+\delta )=Q^{-1}(z)+\delta ·{(Q^{-1})}^{\prime }(z)+O({\delta }^2)\text{as}\delta \to 0.$$

In this formula, set z = z_i,N and δ_N = 1/N so that z + δ = z_i+1,N. We thus obtain

$$t_{i+1,N}-t_{i,N}=Q^{-1}(z_{i+1,N})=Q^{-1}(z_{i,N})+\frac{1}{N}·{(Q^{-1})}^{\prime }(z_{i,N})+O\left(\frac{1}{N^2}\right)\text{as}N\to \infty .$$

By using (2.9) and the relation (Q⁻¹)′ (z) = 1/Q′ (Q⁻¹(z)) we can rewrite this in the form (A.1). The second statement obviously follows from (A.1).

Proof of Theorem 2.1

In view of Lemma 2.1 and (2.5) – (2.7) we have

$$\begin{array}{c}{w}_{1,N}=\frac{c_N{u}_2}{f_1{v}_1{v}_2}\left(\frac{f_1}{u_1}-\frac{f_2}{u_2}\right),w_{N,N}=\frac{c_N}{f_N{v}_N}\left(\frac{f_N}{v_N}-\frac{f_{N-1}}{v_{N-1}}\right),\\ w_{i,N}=\frac{c_N}{f_i{v}_i}\left(\frac{2f_i}{v_i}-\frac{f_{i-1}}{v_{i-1}}-\frac{f_{i+1}}{v_{i+1}}\right)\text{for}i=2,\dots ,N-1,\end{array}$$

where we have used the relations (A.3). Here c_N is the normalization constant providing ${\sum }_{i=1}^N\mid w_{i,N}\mid =1$and we use the notation u_i = u(t_i,_N), v_i = (t_i,N) and f_i = f (t_i,N).

Consider first w_1,N. Denote g(t) = f(t)/u(t), then

$$w_{1,N}/c_N=\frac{u(t_{2,N})}{f(t_{1,N})v(t_{1,N})v(t_{2,N})}(g(t_{1,N})-g(t_{2,N})),$$ (A.4)

which gives

$$\begin{array}{c}\frac{u(t_{2,N})}{f(t_{1,N})v(t_{1,N})v(t_{2,N})}=\frac{u(a)}{f(a)v^2(a)}\left(1+O\left(\frac{1}{N}\right)\right),\\ g(t_{1,N})-g(a)=g^{\prime }(a)(t_{1,N}-a)+O\left({(t_{1,N}-a)}^2\right)=g^{\prime }(a)·\frac{1}{2N{Q}^{\prime }(a)}+O\left(\frac{1}{N^2}\right)\end{array}$$ (A.5)

as N → ∞. Similarly

$$g(t_{2,N})-g(a)=g^{\prime }(a)\frac{3}{2N{Q}^{\prime }(a)}+O\left(\frac{1}{N^2}\right)$$

yielding

$$g(t_{1,N})-g(t_{2,N})=-g^{\prime }(a)\frac{1}{N{Q}^{\prime }(a)}+O\left(\frac{1}{N^2}\right).$$ (A.6)

Combining (A.4), (A.5) and (A.6) we obtain

$$\begin{array}{l}\frac{w_{1,N}}{c_N}=-\frac{1}{N}·\frac{u(a)g^{\prime }(a)}{f(a)v^2(a)Q^{\prime }(a)}\left(1+O\left(\frac{1}{N}\right)\right)\\ =\frac{1}{N}·\frac{1}{v^2(a)Q^{\prime }(a)}\left[\frac{u^{\prime }(a)}{u(a)}-\frac{f^{\prime }(a)}{f(a)}\right]\left(1+O\left(\frac{1}{N}\right)\right)\end{array}$$ (A.7)

as N → ∞. Similarly to (A.7) we get the asymptotic expression for w_N,N :

$$\frac{w_{N,N}}{c_N}=\frac{1}{N}·\frac{h^{\prime }(b)}{f(b)v(b)Q^{\prime }(b)}\left(1+O\left(\frac{1}{N}\right)\right)$$ (A.8)

as N → ∞. Consider now the weights

$$w_{i,N}=c_N\frac{2h(t_{i,N})-h(t_{i-1,N})-h(t_{i+1,N})}{f(t_{i,N})v(t_{i,N})}(i=2,\dots ,N-1).$$ (A.9)

Assume N → ∞ and i = i(N) is such that i(N)/N = z + O(1/N) as N → ∞ for some z ∈ (0, 1), and set t = Q⁻¹(z).

We are going to prove that

$$\frac{w_{i,N}}{c_N}=\frac{2h(t_{i,N})-h(t_{i-1,N})-h(t_{i+1,N})}{f(t_{i,N})v(t_{i,N})}$$ (A.10)

$$\begin{array}{l}=\frac{1}{N^2{(Q^{\prime }(t))}^{2}f(t)v(t)}\left[\frac{h^{\prime }(t)Q^{″}(t)}{Q^{\prime }(t)}-h^{″}(t)\right]\left(1+O\left(\frac{1}{N}\right)\right)\\ =-\frac{1}{N^2{Q}^{\prime }(t)f(t)v(t)}{\left[\frac{h^{\prime }(t)}{Q^{\prime }(t)}\right]}^{\prime }\left(1+O\left(\frac{1}{N}\right)\right).\end{array}$$ (A.11)

First, in view of (2.9) we have $t_{i,n}=t+O\left(\frac{1}{N}\right)$ and hence

$$f(t_{i,N})v(t_{i,N})=f(t)v(t)\left(1+O\left(\frac{1}{N}\right)\right)\text{as}N\to \infty .$$

Consider the numerator in (A.10) and rewrite it as follows:

$$2h(t_{i,N})-h(t_{i-1,N})-h(t_{i+1,N})=[2h({\stackrel{\sim }{t}}_{i,N})-h(t_{i-1,N})-h(t_{i+1,N})]+2[h(t_{i,N})-h({\stackrel{\sim }{t}}_{i,N})]$$

where t̃_i,N = (t_i−1,N + t_i+1,N)/2. We obviously have t_i+1,N = t̃_i,N + Δ_N and t_i−1,N = t̃_i,N − Δ_N, where Δ_N = (t_i+1,N − t_i−1,N)/2 is defined in (A.2). This yields

$$\frac{2h({\stackrel{\sim }{t}}_{i,N})-h(t_{i-1,N})-h(t_{i+1,N})}{{\mathrm{\Delta }}_N^2}=-h^{″}(t)\left(1+O\left(\frac{1}{N}\right)\right).$$ (A.12)

Next we consider

$$\frac{h(t_{i,N})-h({\stackrel{\sim }{t}}_{i,N})}{{\mathrm{\Delta }}_N^2}=\frac{h(t_{i,N})-h({\stackrel{\sim }{t}}_{i,N})}{t_{i,N}-{\stackrel{\sim }{t}}_{i,N}}·\frac{t_{i,N}-{\stackrel{\sim }{t}}_{i,N}}{{\mathrm{\Delta }}_N^2}.$$

For the first factor we have

$$\frac{h(t_{i,N})-h({\stackrel{\sim }{t}}_{i,N})}{t_{i,N}-{\stackrel{\sim }{t}}_{i,N}}=h^{\prime }(t)\left(1+O\left(\frac{1}{N}\right)\right),$$

while the second factor gives

$$\begin{array}{l}\frac{t_{i,N}-{\stackrel{\sim }{t}}_{i,N}}{{\mathrm{\Delta }}_N^2}=2\frac{2t_{i,N}-t_{i+1,N}-t_{i-1,N}}{{(t_{i+1,N}-t_{i-1,N})}^2}\\ =2\frac{2Q^{-1}(z_{i,N})-Q^{-1}(z_{i+1,N})-Q^{-1}(z_{i-1,N})}{1/N^2}{\left(\frac{1/N}{Q^{-1}(z_{i+1,N})-Q^{-1}(z_{i-1,N})}\right)}^2\\ =-2{(Q^{-1})}^{″}(z)/{(2{(Q^{-1})}^{\prime }(z))}^2\left(1+O\left(\frac{1}{N}\right)\right)=Q^{″}(t)/(2Q^{\prime }(t))\left(1+O\left(\frac{1}{N}\right)\right),\end{array}$$

where we have used the relation (Q⁻¹)″ (z) = −Q″(z)/(Q′(z))³ in the last equation. This gives, as N → ∞,

$$2\frac{h(t_{i,N})-h({\stackrel{\sim }{t}}_{i,N})}{{\mathrm{\Delta }}_N^2}=2\frac{h(t_{i,N})-h({\stackrel{\sim }{t}}_{i,N})}{t_{i,N}-{\stackrel{\sim }{t}}_{i,N}}·\frac{t_{i,N}-{\stackrel{\sim }{t}}_{i,N}}{{\mathrm{\Delta }}_N^2}=\frac{h^{\prime }(t)Q^{″}(t)}{Q^{\prime }(t)}\left(1+O\left(\frac{1}{N}\right)\right).$$ (A.13)

Combining the expressions (A.2), (A.10), (A.12) and (A.13) yields the asymptotic expression (A.11) for w_i,N/c_N .

By noting that $c_N=NC\left(1+O\left(\frac{1}{N}\right)\right)$ as N → ∞ and that the asymptotic density of the points t_i,N (i = 1, . . . , N) is Q′(t) on the interval [a, b], we deduce the statement of the theorem as a consequence of the asymptotic formulas (A.7), (A.8) and (A.11) for w_1,N/c_N, w_N,N /c_N and w_i,N/c_N respectively.

A.4 Proof of Theorem 2.2

By Theorem 3.3 in Dette et al. (2013) a design minimizes the functional (2.15) if the identity

$${\int }_a^{b}min(s,t)f(t)d\xi (t)=\lambda f(s)$$ (A.14)

holds ξ –a.e., where λ is some constant. We consider the design ξ = ξ^* defined by (2.12) and (2.13) and verify for this design the condition (A.14). To do this we calculate by partial integration

$$\begin{array}{l}\frac{1}{c}{\int }_a^{b}min(s,t)f(t)p(t)dt={\int }_a^{s}t(-f^{″}(t))dt+{\int }_s^{b}s(-f^{″}(t))dt\\ =-\left\{{t{f}^{\prime }(t)|}_a^s-{\int }_a^s{f}^{\prime }(t)dt\right\}-s\left\{f^{\prime }(b)-f^{\prime }(s)\right\}\\ =(a{f}^{\prime }(a)-f(a))-s{f}^{\prime }(b)+f(s).\end{array}$$

Observing the definition of the masses in (2.13), the identify (A.14) follows with λ = c. This proves the first part of Theorem 2.2.

For a proof of the second statement consider a linear unbiased estimator θ̂_μ^* in model (2.1) based on the full trajectory, where μ^*(dt) = f(t)ξ^*(dt) and ξ^* is the design in (2.12), (2.13) with a constant c chosen such that θ̂_μ^* is unbiased, that is

$$c^{\ast }={\left[\frac{f^2(a)}{a}+{\int }_a^b{(f^{\prime }(t))}^{2}dt\right]}^{-1}.$$

Standard arguments of optimal design theory show that μ^* minimizes Φ (that is, θ̂_μ^* is BLUE in model (2.1) where the full trajectory can be observed) if and only if the inequality

$$\mathrm{\Phi }({\mu }^{\ast },\nu )={\int }_a^b{\int }_a^{b}K(x,y){\mu }^{\ast }(dx)\nu (dy)\ge \mathrm{\Phi }({\mu }^{\ast })$$ (A.15)

holds for all signed measures ν satisfying ${\int }_a^{b}f(t)\nu (dt)=1$. Observing this condition and the identity (A.14) we obtain

$$\mathrm{\Phi }({\mu }^{\ast },\nu )=c^{\ast }{\int }_a^{b}f(s)\nu (ds)={\left[\frac{f^2(a)}{a}+{\int }_a^b{(f^{\prime }(t))}^{2}dt\right]}^{-1}=\mathrm{\Phi }({\mu }^{\ast })$$

for all signed measures ν on [a, b] with ${\int }_a^{b}f(t)\nu (dt)=1$. By (A.15) μ* minimizes Φ. Consequently, the corresponding estimator θ̂_μ^* is BLUE with minimal variance

$$D^{\ast }=c^{\ast }={\left[\frac{f^2(a)}{a}+{\int }_a^b{(f^{\prime }(t))}^{2}dt\right]}^{-1}.$$

A.5 Proof of Theorem 2.3

Let {ε̃ (s)|s ∈ [ã, b̃] be a Brownian motion on the interval [ã, b̃] and consider the regression model (2.1) with some function f̃(s) and the error process. By Theorem 2.2 the optimal design is given by

$${\stackrel{\sim }{\xi }}^{\ast }(ds)={\stackrel{\sim }{P}}_{\stackrel{\sim }{a}}{\delta }_{\stackrel{\sim }{a}}(ds)+{\stackrel{\sim }{P}}_{\stackrel{\sim }{b}}{\delta }_{\stackrel{\sim }{b}}(ds)+\stackrel{\sim }{p}(s)ds$$

with

$${\stackrel{\sim }{P}}_{\stackrel{\sim }{a}}=c\frac{\stackrel{\sim }{f}(a)-{\stackrel{\sim }{f}}^{\prime }(\stackrel{\sim }{a})\stackrel{\sim }{a}}{\stackrel{\sim }{a}\stackrel{\sim }{f}(\stackrel{\sim }{a})},{\stackrel{\sim }{P}}_{\stackrel{\sim }{b}}=c\frac{{\stackrel{\sim }{f}}^{\prime }(\stackrel{\sim }{b})}{\stackrel{\sim }{f}(\stackrel{\sim }{b})}\text{and}\stackrel{\sim }{p}(s)=-c\frac{{\stackrel{\sim }{f}}^{″}(s)}{\stackrel{\sim }{f}(s)}.$$

We shall now use Theorem B.1 to derive the optimal design ξ*(dt) for the original regression model (2.1) with regression function f(t) and covariance kernel K(t, t′) from the design ξ̃^*(ds) for the function f̃(s) = h(q⁻¹(s)), where h(t) = f(t)/v(t).

For the Brownian motion, the covariance function is defined by (B.4) with ṽ(t) = 1 and q̃(t) = t so that by (B.6) we have β(t) = q(t), α (t) = v(t) and α̃(t) = 1/v(q⁻¹(t)). According to (B.14) the optimal design dξ̃(s) transforms to dξ^*(t) = α̃²(β(t)) dξ̃^* (β(t)) = dξ̃^* (q(t))/ν(t).

Consider first the mass at b. We have P̃_b̃ = cf̃′ (b̃)/f̃(b̃). By using the transformation of t into s = q⁻¹(t), we obtain

$$P_b=\frac{{\stackrel{\sim }{P}}_b}{v^2(b)}=c\frac{{\stackrel{\sim }{f}}^{\prime }(\stackrel{\sim }{b})}{\stackrel{\sim }{f}(\stackrel{\sim }{b})v^2(b)}=c\frac{h^{\prime }(b)}{q^{\prime }(b)v^2(b)h(b)}=c\frac{h^{\prime }(b)}{q^{\prime }(b)v(b)f(b)},$$

as required. From the representation of P̃_ãwe obtain by similar arguments

$$P_a=\frac{{\stackrel{\sim }{P}}_a}{v^2(a)}=c\frac{h(a)-a{h}^{\prime }(a)/q^{\prime }(a)}{q^{\prime }(a)v^2(a)h(a)}.$$

Let us now consider the density p̃(s), s ∈ [ã,b̃], and rewrite $d{\stackrel{\sim }{\xi }}_p^{\ast }(\beta (t))$, the absolutely continuous part of the measure ξ̃^*. The transformation of the variable s into t = q⁻¹(s) ∈ [a, b] induces the density

$$d{\stackrel{\sim }{\xi }}_p^{\ast }(\beta (t))=\stackrel{\sim }{p}(q(t))q^{\prime }(t)=-c{q}^{\prime }(t)\frac{{\stackrel{\sim }{f}}^{″}(q(t))}{\stackrel{\sim }{f}(q(t))}.$$ (A.16)

Differentiating the equality f̃(s) = h(q⁻¹(s)), we have

$${\stackrel{\sim }{f}}^{″}(s)={(h^{\prime }(q^{-1}(s))·{(q^{-1}(s))}^{\prime })}^{\prime }=h^{″}(q^{-1}(s))·{({(q^{-1}(s))}^{\prime })}^2+h^{\prime }(q^{-1}(s))·{(q^{-1}(s))}^{″}.$$

Now we obtain

$${\stackrel{\sim }{f}}^{″}(q(t))=\frac{h^{″}(t)}{{(q^{\prime }(t))}^2}-h^{\prime }(t)·\frac{q^{″}(t)}{{(q^{\prime }(t))}^3}$$

Inserting this into (A.16) and taking into account that f̃(q(t)) = h(t), we obtain the density

$$d{\stackrel{\sim }{\xi }}_p^{\ast }(\beta (t))=c\frac{1}{h(t)q^{\prime }(t)}\left(h^{\prime }(t)·\frac{q^{″}(t)}{q^{\prime }(t)}-h^{″}(t)\right)=-\frac{c}{h(t)}{\left[\frac{h^{\prime }(t)}{q^{\prime }(t)}\right]}^{\prime }.$$ (A.17)

In view of the relation dξ^*(t) = α̃²(β(t))dξ̃^*(β(t)) we need to divide the right hand side in (A.17) by v²(t) and obtain the expression for the density (2.11). This completes the proof of Theorem 2.3.

B Gaussian processes with triangular covariance kernels

B.1 Extended Doob’s representation

Assume that {ε (t)| t ∈ [a, b]} is a Gaussian process with covariance kernel K of the form (2.4); that is, K(t, t′) = u(t)v(t′) for t ≤ t′, where u(·) and v(·) are functions defined on the interval [a, b]. According to the terminology introduced in Mehr and McFadden (1965) kernels of the form (2.4) are called triangular. An alternative way of writing these covariance kernels is

$$K(t,t^{\prime })=v(t)v(t^{\prime })min\{q(t),q(t^{\prime })\}\text{for}t,t^{\prime }\in [a,b],$$ (B.1)

where q(t) = u(t)/v(t). We assume that ε(t) is non-degenerate on the open interval (a, b), which implies that the function q is strictly increasing and continuous on the interval [a, b] [see Mehr and McFadden (1965), Remark 2]. Moreover, this function is also positive on the interval (a, b) [see Remark 1 in Mehr and McFadden (1965)], which yields that the functions u and v must have the same sign and can be assumed to be positive on the interval (a, b) without loss of generality. We repeatedly use the following extension of the celebrated Doob’s representation [see Doob (1949)], which relates to two Gaussian processes (on compact intervals) by a time-space transformation.

Lemma B.1

Let {ε(t)| t ∈ [a, b]} be a non-degenerate Gaussian process with zero mean and covariance function (B.1) and let ṽ and q̃ be continuous positive functions on [ã,b̃], such that q̃ is strictly increasing and q̃ ([ã,b̃]) = q([a, b]). Define the transformations β̃ : [ã,b̃] → [a, b] and α̃ : [ã,b̃] → ℝ₊ by

$$\stackrel{\sim }{\beta }(s)=q^{-1}(\stackrel{\sim }{q}(s)),\stackrel{\sim }{\alpha }(s)=\stackrel{\sim }{v}(s)/v(\stackrel{\sim }{\beta }(s)).$$ (B.2)

Then the Gaussian process {ε̃(t)| t ∈ [ã,b̃]} defined by

$$\stackrel{\sim }{\epsilon }(s)=\stackrel{\sim }{\alpha }(s)\epsilon (\stackrel{\sim }{\beta }(s))$$ (B.3)

has zero mean and the covariance function is given by

$$\stackrel{\sim }{K}(s,s^{\prime })=\mathbb{E}[\stackrel{\sim }{\epsilon }(s)\stackrel{\sim }{\epsilon }(s^{\prime })]=\stackrel{\sim }{v}(s)\stackrel{\sim }{v}(s^{\prime })min(\stackrel{\sim }{q}(s),\stackrel{\sim }{q}(s^{\prime })).$$ (B.4)

Conversely, the Gaussian process ε(t) can be expressed via ε̃(s) by the transformation

$$\epsilon (t)=\alpha (t)\stackrel{\sim }{\epsilon }(\beta (t)),$$ (B.5)

where

$$\beta (t)={\stackrel{\sim }{q}}^{-1}(q(t)),\alpha (t)=v(t)/\stackrel{\sim }{v}(\beta (t)).$$ (B.6)

Proof

Since {ε(t)|t ∈ [a, b]} is Gaussian and has zero mean, the process defined by (B.3) is also Gaussian and has zero mean. For the covariance function of the process (B.3) we have

$$\begin{array}{l}\mathbb{E}[\stackrel{\sim }{\epsilon }(s)\stackrel{\sim }{\epsilon }(s^{\prime })]=\stackrel{\sim }{\alpha }(s)\stackrel{\sim }{\alpha }(s^{\prime })\mathbb{E}\left[\epsilon (\stackrel{\sim }{\beta }(s))\epsilon (\stackrel{\sim }{\beta }(s^{\prime }))\right]\\ =\stackrel{\sim }{\alpha }(s)\stackrel{\sim }{\alpha }(s^{\prime })v(\stackrel{\sim }{\beta }(s))v(\stackrel{\sim }{\beta }(s^{\prime }))min\left[q(\stackrel{\sim }{\beta }(s)),q(\stackrel{\sim }{\beta }(s^{\prime }))\right]\\ =\stackrel{\sim }{v}(s)\stackrel{\sim }{v}(s^{\prime })min[\stackrel{\sim }{q}(s),\stackrel{\sim }{q}(s^{\prime })]=\stackrel{\sim }{K}(s,s^{\prime }).\end{array}$$

The second part of the proof follows by the same arguments and the details are therefore omitted.

Remark B.1

1. The classical result of Doob is a particular case of (B.5) when ε̃(t) = W(t) is the Brownian motion with covariance function K̃ (t, s) = min(t, s). In this case we have ṽ (t) = 1, q̃ (t) = t, α (t) = v(t) and β(t) = q(t). Specifically, the Doob’s representation is given by ε (t) = v(t)W(q(t)) [see Doob (1949)].

2. Both functions β : [a, b] → [ã,b̃] and β̃ : [ã,b̃] → [a, b] are positive strictly increasing functions and are inverses of each other; that is,

   $$\beta (t)={\stackrel{\sim }{\beta }}^{-1}(t),\forall t\in [a,b].$$ (B.7)
3. The functions α (·) and α̃ (·) are positive and satisfy the relation

   $$\alpha (t)·\stackrel{\sim }{\alpha }(\beta (t))=\frac{v(t)}{\stackrel{\sim }{v}(\beta (t))}·\frac{\stackrel{\sim }{v}(\beta (t))}{v(\stackrel{\sim }{\beta }(\beta (t)))}=1,\forall t\in [a,b].$$ (B.8)

4. The properties (b) and (c) imply that the transformation ε̃ → ε defined by (B.5) is the inverse of the transformation ε → ε̃ defined in (B.3).

B.2 Transformation of regression models

Associated with the transformation of the triangular covariance kernels there exists a canonical transformation for the corresponding regression models. To be precise, consider the regression model (1.1) or its continuous time version (1.2), where the covariance kernel K(·, ·) has the form (B.1). Recall the definition of the transformation β : [a, b] → [ã,b̃] defined in (B.6), which maps the observation points t_j to t̃_j = β(t_j), j = 1, . . . , N and define

$$\stackrel{\sim }{f}(s)=\frac{f(\stackrel{\sim }{\beta }(s))}{\alpha (\stackrel{\sim }{\beta }(s))},\stackrel{\sim }{\epsilon }(s)=\frac{\epsilon (\stackrel{\sim }{\beta }(s))}{\alpha (\stackrel{\sim }{\beta }(s))},\stackrel{\sim }{y}({\stackrel{\sim }{t}}_j)=\frac{y(t_j)}{\alpha (t_j)},$$ (B.9)

where s ∈ [ã, b̃] so that β̃ (s) ∈ [a, b]. The regression model (1.1) can now be rewritten in the form

$$\stackrel{\sim }{y}({\stackrel{\sim }{t}}_j)={\theta }^T\stackrel{\sim }{f}(t_j)+\stackrel{\sim }{\epsilon }({\stackrel{\sim }{t}}_j),{\stackrel{\sim }{t}}_j\in [\stackrel{\sim }{a},\stackrel{\sim }{b}],j=1,\dots ,N.$$ (B.10)

The errors ε̃ (t̃_j) in (B.10) have zero mean and, by Lemma B.1 and the identity (B.8), their covariances are given by

$$\mathbb{E}[\stackrel{\sim }{\epsilon }({\stackrel{\sim }{t}}_i)\stackrel{\sim }{\epsilon }({\stackrel{\sim }{t}}_j)]=\stackrel{\sim }{K}({\stackrel{\sim }{t}}_i,{\stackrel{\sim }{t}}_j).$$ (B.11)

Hence we have transformed the regression observation scheme (1.1) with error covariances 𝔼[ε(t_i) ε (t_j)] = K(t_i, t_j) to the scheme (B.10) with covariances (B.11). Conversely, we can transform the model (B.10) with covariances (B.11) to the model (1.1) using the transformations

$$f(t)=\frac{\stackrel{\sim }{f}(\beta (t))}{\stackrel{\sim }{\alpha }(\beta (t))},\epsilon (t)=\frac{\stackrel{\sim }{\epsilon }(\beta (t))}{\stackrel{\sim }{\alpha }(\beta (t))},t\in [a,b].$$ (B.12)

Lemma B.2

The transformation f → f̃ defined in (B.9) is an inverse to the transformation f̃ → f defined in (B.12).

Proof

Inserting the expression for f̃ from (B.9) into (B.12), we have

$$f(t)=\frac{\stackrel{\sim }{f}(\beta (t))}{\stackrel{\sim }{\alpha }(\beta (t))}=\frac{f(\stackrel{\sim }{\beta }(\beta (t)))}{\alpha (\stackrel{\sim }{\beta }(\beta (t)))\stackrel{\sim }{\alpha }(\beta (t))}=\frac{f(t)}{\alpha (t)\stackrel{\sim }{\alpha }(\beta (t))}=f(t),$$

where we have used the identities β̃ (β(t)) = t, see (B.7), and α (t) α̃ (β(t)) = 1, see (B.8).

B.3 Transformation of designs

In this section we consider a transformation of the matrix-weighted designs under a given transformation of the regression models. In the one-parameter case with m = 1, these matrix-weighted designs become signed measures; that is, signed designs as considered in Section 2. In this section, it is convenient to define all integrals as Lebesgue-Stieltjes integrals with respect to the distribution functions of the measures ζ and ζ̃.

To be precise, let dξ(t) = O_ξ(t)dζ(t) be a matrix-weighted design on the interval t ∈ [a, b]. Recalling the definition of α, α̃ and β, β̃ in (B.2) and (B.6) we define a matrix-weighted design dξ̃(s) = Õ_ξ̃ (s)dζ̃(s) by

$$d\stackrel{\sim }{\zeta }(s)=d\zeta (\stackrel{\sim }{\beta }(s))\text{and}{\stackrel{\sim }{\mathbf{O}}}_{\stackrel{\sim }{\xi }}(s)={\alpha }^2(\stackrel{\sim }{\beta }(s)){\mathbf{O}}_{\xi }(\stackrel{\sim }{\beta }(s)).$$ (B.13)

Note that ζ̃ and ζ are probability measures on the intervals [ã, b̃] and [a, b], respectively. Similarly, for a given matrix-weighted design dξ̃(s) = Õ_ξ̃ (s)dζ̃(s) on the interval [ã, b̃] we define a matrix-weighted design dξ(t) = O_ξ(t)dζ(t) on the interval [a, b] by

$$d\zeta (t)=d\stackrel{\sim }{\zeta }(\beta (t))\text{and}{\mathbf{O}}_{\xi }(t)={\stackrel{\sim }{\alpha }}^2(\beta (t)){\stackrel{\sim }{\mathbf{O}}}_{\stackrel{\sim }{\xi }}(\beta (t)).$$ (B.14)

Similar to Lemma B.2 we can see that the transformation ξ̃ → ξ defined by (B.14) is the inverse to the transformation ξ → ξ̃ defined by (B.13).

For the following discussion we recall the definition of the covariance matrix D(ξ) in (3.11). For the model (B.10), the covariance matrix of the design dξ̃(s) = Õ_ξ̃ (s)dζ̃(s), defined by (B.13), is given by

$$\stackrel{\sim }{\mathbf{D}}(\stackrel{\sim }{\xi })={\stackrel{\sim }{\mathbf{M}}}^{-1}(\stackrel{\sim }{\xi })\stackrel{\sim }{\mathbf{B}}(\stackrel{\sim }{\xi }){({\stackrel{\sim }{\mathbf{M}}}^{-1}(\stackrel{\sim }{\xi }))}^T,$$ (B.15)

where

$$\stackrel{\sim }{\mathbf{B}}(\stackrel{\sim }{\xi })={\int }_{\stackrel{\sim }{a}}^{\stackrel{\sim }{b}}{\int }_{\stackrel{\sim }{a}}^{\stackrel{\sim }{b}}\stackrel{\sim }{K}(t,s){\stackrel{\sim }{\mathbf{O}}}_{\stackrel{\sim }{\xi }}(t)\stackrel{\sim }{f}(t){({\stackrel{\sim }{\mathbf{O}}}_{\stackrel{\sim }{\xi }}(s)\stackrel{\sim }{f}(s))}^{T}d\stackrel{\sim }{\zeta }(t)d\stackrel{\sim }{\zeta }(s),\stackrel{\sim }{\mathbf{M}}(\stackrel{\sim }{\xi })={\int }_{\stackrel{\sim }{a}}^{\stackrel{\sim }{b}}{\stackrel{\sim }{\mathbf{O}}}_{\stackrel{\sim }{\xi }}(t)\stackrel{\sim }{f}(t){\stackrel{\sim }{f}}^T(t)d\stackrel{\sim }{\zeta }(t)$$

and the kernel K̃ is defined by (B.4).

Theorem B.1

For any matrix-weighted design dξ(t) = O_ξ(t)dζ(t) and the corresponding matrix-weighted design ξ̃ defined by (B.13), we have D(ξ) = D̃(ξ̃). In particular, D^* = D̃^*, where D^* and D̃ ^* are the covariance matrices of the BLUE in the continuous time models (1.2) and in the model {θ^Tf̃(s) + ε̃ (s)|s ∈ [ã, b̃]}, respectively.

Proof

Using the variable transformation β̃ (s) = t and (B.9), we have

$$\begin{array}{l}\stackrel{\sim }{\mathbf{M}}(\stackrel{\sim }{\xi })=\int {\stackrel{\sim }{\mathbf{O}}}_{\stackrel{\sim }{\xi }}(s)\stackrel{\sim }{f}(s){\stackrel{\sim }{f}}^T(s)d\stackrel{\sim }{\zeta }(s)=\int \frac{{\mathbf{O}}_{\xi }(\stackrel{\sim }{\beta }(s))f(\stackrel{\sim }{\beta }(s))}{\alpha (\stackrel{\sim }{\beta }(s))}\frac{f^T(\stackrel{\sim }{\beta }(s))}{\alpha (\stackrel{\sim }{\beta }(s))}·{\alpha }^2(\stackrel{\sim }{\beta }(s))d\zeta (\stackrel{\sim }{\beta }(s))\\ =\int {\mathbf{O}}_{\xi }(t)f(t)f^T(t)d\zeta (t)=\mathbf{M}(\xi ).\end{array}$$

Next, we calculate the corresponding expression for B̃ (ξ̃), that is

$$\begin{array}{l}\stackrel{\sim }{\mathbf{B}}(\stackrel{\sim }{\xi })=\int \int \stackrel{\sim }{K}(x,y){\stackrel{\sim }{\mathbf{O}}}_{\stackrel{\sim }{\xi }}(x)\stackrel{\sim }{f}(x){({\stackrel{\sim }{\mathbf{O}}}_{\stackrel{\sim }{\xi }}(y)\stackrel{\sim }{f}(y))}^{T}d\stackrel{\sim }{\zeta }(x)d\stackrel{\sim }{\zeta }(y)\\ =\int \int \stackrel{\sim }{v}(x)\stackrel{\sim }{v}(y)min(\stackrel{\sim }{q}(x),\stackrel{\sim }{q}(y)){\stackrel{\sim }{\mathbf{O}}}_{\stackrel{\sim }{\xi }}(x)\stackrel{\sim }{f}(x){({\stackrel{\sim }{\mathbf{O}}}_{\stackrel{\sim }{\xi }}(y)\stackrel{\sim }{f}(y))}^{T}d\stackrel{\sim }{\zeta }(x)d\stackrel{\sim }{\zeta }(y)\\ =\int \int \stackrel{\sim }{v}(x)\stackrel{\sim }{v}(y)min(\stackrel{\sim }{q}(x),\stackrel{\sim }{q}(y))\frac{{\mathbf{O}}_{\xi }(\stackrel{\sim }{\beta }(x))f(\stackrel{\sim }{\beta }(x))}{\alpha (\stackrel{\sim }{\beta }(x))}\frac{{({\mathbf{O}}_{\xi }(\stackrel{\sim }{\beta }(y))f(\stackrel{\sim }{\beta }(y)))}^T}{\alpha (\stackrel{\sim }{\beta }(y))}\times {\alpha }^2(\stackrel{\sim }{\beta }(x))d\zeta (\stackrel{\sim }{\beta }(x)){\alpha }^2(\stackrel{\sim }{\beta }(y))d\zeta (\stackrel{\sim }{\beta }(y))\end{array}$$

Define s = β̃ (x) and t = β̃ (y) so that x = β̃⁻¹(s) = β(s) and similarly y = β(t). Changing the variables in the integrals above we obtain

$$\stackrel{\sim }{B}(\stackrel{\sim }{\xi })=\int \int \stackrel{\sim }{v}(\beta (s))\stackrel{\sim }{v}(\beta (t))min(\stackrel{\sim }{q}(\beta (s)),\stackrel{\sim }{q}(\beta (t))){\mathbf{O}}_{\xi }(s)f(s){({\mathbf{O}}_{\xi }(t)f(t))}^T\alpha (s)\alpha (t)d\zeta (s)d\zeta (t).$$

Using the definition of β in (B.6) yields q̃ (β(t)) = q̃ (q̃⁻¹(q(t))) = q(t) and by the definition of α in (B.6) we finally get

$$\begin{array}{l}\stackrel{\sim }{\mathbf{B}}(\stackrel{\sim }{\xi })=\int \int \stackrel{\sim }{v}(\beta (s))\stackrel{\sim }{v}(\beta (t))min(q(s),q(t)){\mathbf{O}}_{\xi }(s)f(s){({\mathbf{O}}_{\xi }(t)f(t))}^T\frac{v(s)}{\stackrel{\sim }{v}(\beta (s))}\frac{v(t)}{\stackrel{\sim }{v}(\beta (t))}d\zeta (s)d\zeta (t)\\ =\int \int min(q(s),q(t)){\mathbf{O}}_{\xi }(s)f(s){({\mathbf{O}}_{\xi }(t)f(t))}^{T}v(s)v(t)d\zeta (s)d\zeta (t)=\mathbf{B}(\xi ).\end{array}$$

The result D(ξ) = D̃ (ξ̃) follows now from the definitions (3.11) and (B.15).