Prediction Accuracy in Multivariate Repeated-Measures Bayesian Forecasting Models with Examples Drawn from Research on Sleep and Circadian Rhythms

Abstract

In study designs with repeated measures for multiple subjects, population models capturing within- and between-subjects variances enable efficient individualized prediction of outcome measures (response variables) by incorporating individuals response data through Bayesian forecasting. When measurement constraints preclude reasonable levels of prediction accuracy, additional (secondary) response variables measured alongside the primary response may help to increase prediction accuracy. We investigate this for the case of substantial between-subjects correlation between primary and secondary response variables, assuming negligible within-subjects correlation. We show how to determine the accuracy of primary response predictions as a function of secondary response observations. Given measurement costs for primary and secondary variables, we determine the number of observations that produces, with minimal cost, a fixed average prediction accuracy for a model of subject means. We illustrate this with estimation of subject-specific sleep parameters using polysomnography and wrist actigraphy. We also consider prediction accuracy in an example time-dependent, linear model and derive equations for the optimal timing of measurements to achieve, on average, the best prediction accuracy. Finally, we examine an example involving a circadian rhythm model and show numerically that secondary variables can improve individualized predictions in this time-dependent nonlinear model as well.

1. Introduction

Significant steps forward in the analysis of repeated-measures data were made with the introduction of linear and nonlinear mixed-effects models [1–3], which distinguish within-subjects variance (from multiple measurements in each subject) versus between-subjects variance (from multiple subjects being measured). Distinguishing these types of variance can also be thought of as explicitly modeling random error in the data. This can be useful in understanding how different individuals are from one another as compared to how different multiple measurements are for given individuals. In research on sleep and sleepiness, for example, breakthroughs made possible by mixed-effects models include elucidation of the dose-response effects of sustained sleep restriction on sleep architecture and neurobehavioral impairment [4, 5] and demonstration of the trait characteristics of individual differences in vulnerability to sleep loss [6]. In recent years, mixed-effects model approaches to statistical regression and analysis of variance have become widely available in statistical software packages. They are nowadays the methodology of choice for many repeated-measures investigations in sleep research and other fields of study.

A further advance was the introduction of a model individualization technique called Bayesian posterior distribution estimation or Bayesian forecasting. This technique was first used in sleep research to overcome a shortcoming of biomathematical models of fatigue and performance. Existing models did not account for individual differences in sleep regulation and vulnerability to sleep loss and therefore did not accurately predict performance for given individuals. Bayesian forecasting addressed this shortcoming by utilizing the separation of within- and between-subjects variance in model parameters as enabled by mixed-effects modeling [2]. In Bayesian forecasting, the between-subjects variance of model parameters serves as Bayesian prior information. Measurements from a new individual, not previously studied, are combined with the prior information to efficiently derive model parameters tailored to the new individual, thereby yielding a subject-specific mathematical model [2, 7, 8]. As a bonus, the Bayesian forecasting technique also yields quantitative estimates of the accuracy of individualized predictions made with the subject-specific mathematical model [9].

In a published example, Bayesian forecasting was implemented for the two-process model of sleep regulation [10, 11] to predict performance impairment of selected individuals undergoing a period of total sleep deprivation (see Figure 1). Comparisons with the individuals' actual data revealed that the model parameters converged efficiently to those that best characterized each individual, and the response predictions were significantly more accurate than could have been achieved with the original, nonindividualized two-process model [7].

Figure 1.

Lapses of attention on a psychomotor vigilance test (PVT) for a subject in a study involving 88 h of total sleep deprivation under controlled laboratory conditions. In each of the six plots, different amounts of subject data are assumed known (black dots), and the Bayesian forecasting procedure is applied to the known data to construct predictions of PVT number of lapses for a 24 h interval immediately following the most recent collected data point at time t. For the 24 h interval, 95% prediction intervals (vertical bars) are shown, along with the remainder of the data for comparison (gray dots). The beginning of the 88 h sleep deprivation period, t ₀, was at 07:30. Graphs taken from [7] with permission.

In Bayesian forecasting, individualized parameter estimates are derived from the posterior distributions of the parameters in question after combining prior distributions with the measurement(s) from the individual, and individualized model predictions for future outcomes or responses are obtained following estimation of the posterior distribution of the expected responses at given times. A variety of methods are available for obtaining parameter estimates and response predictions once posterior distributions have been estimated. We employ the Bayesian mean squared error (BMSE) [12] and make use of the Bayesian minimum mean squared error (MMSE) estimator [13, 14], which produces unbiased point estimates that minimize the BMSE [15]. The BMSE of parameter estimates and response predictions is dependent on the amount of data available for the individual at hand and the magnitude of between-subjects variance captured in the Bayesian prior distributions.

In cases where between-subjects variance is relatively large, such as performance responses to sleep loss [6, 16], measurement data for the individual at hand are more critical for making accurate individualized response predictions. Sparseness of such measurement data (e.g., due to practical or cost-based limitations) can result in unacceptably low levels of accuracy. For example, Bayesian forecasting could be used to develop a drowsy driver warning system based on a mathematical model of fatigue and performance [17, 18] calibrated to predict lateral lane deviation, using camera-based measurements of lane position to individualize the model for the driver. However, when lane markers are covered with snow and cameras are unable to determine vehicle position relative to the lane, the individualization effort becomes less effective.

To address this limitation, we consider the use of secondary variables to increase data availability, boost response prediction accuracy, and/or reduce data collection costs for individualized response prediction. For example, in the case of a drowsy driver warning system, camera-based measurements of lateral lane deviation serving as the primary response variable could be augmented with in-car secondary variables such as steering wheel variability or driver eyelid closure assessments. However, individualization of predictions based on two or more measurement variables would only be straightforward if individual differences in responses on these variables are equivalent (cf. [19]). Generally, this is not what the evidence shows. As a case in point, trait individual differences in vulnerability to performance impairment due to sleep loss vary considerably across outcome variables [6, 20], such that the most vulnerable individuals based on one variable are not necessarily also the most vulnerable individuals based on another variable. Therefore, when considering two or more response variables as the basis for individualized prediction, it is essential to account statistically for the level of congruence between the response variables.

Here we develop a multivariate statistical framework for individualized prediction of sleep or performance variables, based on Bayesian forecasting with measurements of a primary response variable, augmented with one or more measurements of secondary response variables. The response variables are assumed to follow equivalent dynamics over time, such that they can be described by the same model framework after appropriate scaling. This is a reasonable assumption in the case of, for example, models describing sleep variables measured repeatedly across multiple nights [21] and models describing performance changes over time in response to sleep loss [22]. We make use of multivariate Bayesian prior distributions of the primary and secondary variables, assumed to have been assessed in advance by means of mixed-effects modeling [2] or other suitable techniques. The between-subjects correlation(s) between the primary and secondary variables, used here to account for the level of congruence between the response variables, is assumed to have been estimated as part of the covariance matrix of the multivariate Bayesian prior distributions. The between-subjects correlation(s) are assumed to be at least moderately strong, lest the secondary variables contain essentially no information about the primary response variable that rises above the level of measurement noise.

Our focus in this paper is on prediction accuracy in the multivariate Bayesian forecasting technique. We develop the technique by first considering the details of making individualized response predictions and estimating their accuracy for a simple univariate intercept model. To demonstrate how secondary, correlated responses can be used to make more accurate individualized predictions, we expand the intercept model to include both primary and secondary responses. Assuming fixed costs of data collection for each of the responses, we show how to optimize data collection given a desired level of accuracy for predictions of the primary response variable.

We then consider a linear approximation of the homeostatic component of the two-process model [11] and derive a closed form equation of the BMSE to quantify prediction accuracy in this time-dependent model. For this example, we study the problem of optimizing the timing of measurements in order to enhance the individualized prediction accuracy most efficiently. Finally, we consider more complicated bivariate models, both linear and nonlinear, for which the BMSE cannot be determined in closed form. For these models, we describe a process of numerically assessing the BMSE for individualized prediction based on a primary response variable in Bayesian forecasting augmented with measurements of a secondary response.

2. Subject-Specific Bayesian Models

First we discuss a modeling framework for Bayesian forecasting. Consider a response variable y, dependent on a subject-specific trait parameter θ. Let i be an index for individual, ranging from 1 to N, and let j be an index for observations ordered by time, nested within individual. Suppose that the response variable can be modeled by

$$\begin{array}{c}{y}_{ij}=f\left(t_{ij},{\theta }_i\right)+{ϵ}_{ij},\end{array}$$ (1)

where f(t _ij, θ _i) represents the model function, t _ij represents a fixed measurement time, θ _i represents a random subject-specific parameter, and ϵ _ij represents additive measurement error. We assume that the distributions for θ _i and ϵ _ij are known. Equation (1) and the distributions of θ _i and ϵ _ij constitute a population model.

Limiting our focus to a particular individual, we may remove the subscript i and model the subject's responses as

$$\begin{array}{c}{y}_j=f_j+{ϵ}_j,\end{array}$$ (2)

where f _j is used to denote f(t _j, θ).

Suppose that a total of m responses (y ₁,…, y _m) have been measured for the individual at hand, and let j ^∗ denote the index of a response y _j^∗ at some future time t _j^∗. We consider a prediction (estimator) of the expected response E[y _j^∗∣θ] = f _j^∗, which we denote as ${\widehat{f}}_{j^{\ast }}$. Our interest is in constructing ${\widehat{f}}_{j^{\ast }}$ such that the expected accuracy for an arbitrary, given individual from the population is minimized.

We define the accuracy using the squared error ${(f_{j^{\ast }}-{\widehat{f}}_{j^{\ast }})}^{\mathrm{2}}$. The expected accuracy, which is referred to as the Bayesian mean squared error (BMSE), is thus given by

$$\begin{array}{c}{\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}\equiv \mathrm{B}\mathrm{M}\mathrm{S}\mathrm{E}\left({\widehat{f}}_{j^{\ast }}\right)\equiv E\left[{\left(f_{j^{\ast }}-{\widehat{f}}_{j^{\ast }}\right)}^{\mathrm{2}}\right],\end{array}$$ (3)

where the expectation is taken with respect to the marginal probability density function (pdf) of y ₁,…, y _m and θ.

We refer to the prediction that minimizes the BMSE as a minimum mean squared error (MMSE) prediction. After observing data from a particular individual, y ₁,…, y _m, and constructing the MMSE prediction ${\widehat{f}}_{j^{\ast }}$, we seek to assess the expected accuracy of this particular prediction. This can be done with confidence intervals on f _j^∗, obtained from quantiles of the posterior distribution of θ∣y ₁,…, y _m. In the following sections, we describe specific types of models and investigate the BMSE and the MMSE that minimizes it.

3. Univariate Random Intercept Model

We consider a random intercept model obtained from (2) by letting f _j = θ:

$$\begin{array}{c}{y}_j=\theta +{ϵ}_j,\\ \end{array}$$ (4)

$$\begin{array}{c}\theta ~\mathcal{N}\left(\mu ,{\delta }^{\mathrm{2}}\right),\\ \end{array}$$ (5)

$$\begin{array}{c}{ϵ}_j~\mathcal{N}\left(\mathrm{0},{\sigma }^{\mathrm{2}}\right),\\ \end{array}$$ (6)

where j = 1,…, m. For a particular individual, the expected response at time t _j^∗ is

$$\begin{array}{c}E\left[y_{j^{\ast }}\hspace{0.17em}\mid \hspace{0.17em}\theta \right]=f_{j^{\ast }}=\theta .\end{array}$$ (7)

It follows that the MMSE prediction of E[y _y^∗∣θ] is

$$\begin{array}{c}{\widehat{f}}_{j^{\ast }}=\widehat{\theta },\end{array}$$ (8)

where $\widehat{\theta }$ is the MMSE of θ. The estimator $\widehat{\theta }$ (and therefore ${\widehat{f}}_{j^{\ast }}$) is given by [23]:

$$\begin{array}{c}\widehat{\theta }=\upsilon \overline{y}+\left(\mathrm{1}-\upsilon \right)\mu ,\end{array}$$ (9)

where $\overline{y}$ is the sample mean of the measured responses and

$$\begin{array}{c}\upsilon =\frac{{\delta }^{\mathrm{2}}}{{\delta }^{\mathrm{2}}+{\sigma }^{\mathrm{2}}/m}.\end{array}$$ (10)

The variance of the posterior distribution of θ∣y ₁,…, y _m (and therefore ${\widehat{f}}_{j^{\ast }}\mid y_{\mathrm{1}},\dots ,y_m$) is [23]:

$$\begin{array}{c}\mathrm{V}\mathrm{a}\mathrm{r}\left(\theta \hspace{0.17em}\mid \hspace{0.17em}{y}_{\mathrm{1}},\dots ,y_m\right)=\frac{{\sigma }^{\mathrm{2}}}{m}\left(\frac{{\delta }^{\mathrm{2}}}{{\delta }^{\mathrm{2}}+{\sigma }^{\mathrm{2}}/m}\right).\end{array}$$ (11)

Furthermore, the BMSE of $\widehat{\theta }$ (and therefore ${\widehat{f}}_{j^{\ast }}$) is given by [23]:

$$\begin{array}{c}{\mathbf{M}}_{\widehat{\theta }}=\mathrm{V}\mathrm{a}\mathrm{r}\left(\theta \hspace{0.17em}\mid \hspace{0.17em}{y}_{\mathrm{1}},\dots ,y_m\right).\end{array}$$ (12)

The MMSE prediction ${\widehat{f}}_{j^{\ast }}$ (given by (8) and (9)) represents a trade-off between knowledge about the population and knowledge about the subject at hand. This trade-off is embodied by the weighting factor υ in (10). When no data are available for the subject at hand, υ = 0, resulting in a prediction made at ${\widehat{f}}_{j^{\ast }}=\mu$, where μ is, in this case, representative of the population mean E[θ] as well as the population mean response E[y _j^∗]:

$$\begin{array}{c}E\left[y_{j^{\ast }}\right]=E\left[\theta \right]=\mu .\end{array}$$ (13)

As we begin to collect subject-specific data, the weighting factor moves towards the value υ = 1, resulting in a prediction that converges to the subject-specific sample mean as data collection continues.

Likewise, when no data are available for the subject at hand, the expected accuracy of the prediction is ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}={\delta }^{\mathrm{2}}$. The term δ ² is, in this case, representative of the population variance as well as the variance in mean response over the population:

$$\begin{array}{c}\mathrm{V}\mathrm{a}\mathrm{r}\left[E\left[y_{j^{\ast }}\hspace{0.17em}\mid \hspace{0.17em}\theta \right]\right]\\ =E\left[{\left(E\left[y_{j^{\ast }}\hspace{0.17em}\mid \hspace{0.17em}\theta \right]-E\left[E\left[y_{j^{\ast }}\hspace{0.17em}\mid \hspace{0.17em}\theta \right]\right]\right)}^{\mathrm{2}}\right]\\ =E\left[{\left(E\left[y_{j^{\ast }}\hspace{0.17em}\mid \hspace{0.17em}\theta \right]-E\left[y_{j^{\ast }}\right]\right)}^{\mathrm{2}}\right]=E\left[{\left(\theta -\mu \right)}^{\mathrm{2}}\right]\\ ={\delta }^{\mathrm{2}},\end{array}$$ (14)

where the expectation E[y _j^∗∣θ] is evaluated with respect to the conditional distribution of y _j^∗∣θ, and all other expectations are evaluated with respect to the marginal distribution of y _j^∗. Per (12), as we begin to collect subject-specific data, the expected accuracy of the prediction improves (i.e., the BMSE decreases, where smaller is better).

To illustrate how predictions in this model depend on the amount of subject-specific data collected, we conducted a simulation of model (4) for a particular individual. For this example, the parameter θ for the individual was chosen far from the population mean (when compared to the magnitude of the population variance), in order to make the transition from the population mean to the true expected response f _j^∗ large enough so as not to be obscured by measurement noise in the example. We assumed the population parameters μ = 0 and δ = 1 and chose the subject-specific parameter value θ = 1.4. We simulated m = 10 data points for the individual, with a standard deviation of measurement error of σ = 1. The MMSE prediction ${\widehat{f}}_{j^{\ast }}$ was calculated by incorporating observations y _j iteratively. Figure 2 shows ${\widehat{f}}_{j^{\ast }}$ plotted against the number of data points used. The variance of the posterior distribution for f _j^∗ from (11) was used to construct confidence intervals on f _j^∗. As expected, for the individual considered, the prediction ${\widehat{f}}_{j^{\ast }}$ began at the population mean and moved to the true expected response with shrinking confidence interval as more simulated data were collected.

Figure 2.

MMSE predictions of the expected response ${\widehat{f}}_{j^{\ast }}$ from the univariate random intercept model specified in (4), and corresponding 95% confidence intervals, determined by incorporating simulated observations y _j iteratively. The individualized Bayesian forecasting predictions begin at the population mean expected response μ = 0 when no individual data are used and converge to the subject's expected individualized response f _j^∗ = θ = 1.4 as more subject-specific data are collected.

4. Bivariate Subject-Specific Bayesian Models

When subject-specific data are sparse, individualized predictions may not, on average, reach acceptable levels of accuracy. Accuracy may be improved by including data from a secondary subject-specific data source. However, individual differences in one response variable may not be identical to individual differences in another (e.g., [6, 21]). Therefore, data from a secondary response variable may not simply be used as a substitute for the primary response variable. Rather, to improve prediction accuracy on the primary response by incorporating data from the secondary response, the between-subjects correlation between the primary and secondary response variables must be taken into account.

Here we derive the average accuracy of individualized predictions of a primary response variable based on distinct primary and secondary subject-specific response variables, accounting for between-subjects correlation between the two responses. Let i be an index for individual, let r be an index for response type, and let j be an index for observations ordered by time, nested within individual and response type. Suppose that the response y _rij can be modeled by

$$\begin{array}{c}{y}_{rij}=f\left(t_{rij},{\theta }_{ri}\right)+{ϵ}_{rij},\end{array}$$ (15)

where f(t _rij, θ _ri) represents the model function, t _rij represents measurement time, θ _ri represents a random subject-specific parameter associated with response type r, and ϵ _rij represents additive measurement error. Limiting our focus to a particular individual, we may remove the subscript i and model the subject's responses as

$$\begin{array}{c}{y}_{rj}=f_{rj}+{ϵ}_{rj},\end{array}$$ (16)

where f _rj is used to denote f(t _rj, θ _r). Suppose that a total of m _r responses have been observed for each response type r for the individual at hand. In the next sections, we focus on constructing the MMSE prediction for the expected primary response, ${\widehat{f}}_{\mathrm{1}{j}^{\ast }}$.

5. Bivariate Random Intercept Model

We consider the bivariate random intercept model obtained from (16) by letting f _rj = θ _r:

$$\begin{array}{c}{y}_{rj}={\theta }_r+{ϵ}_{rj},\end{array}$$ (17)

where r = 1,2. The scalar model can be converted to vector form by concatenating the responses for each response type,

$$\begin{array}{c}{\mathbf{y}}_{\mathrm{1}}=\left(\begin{array}{c}{y}_{\mathrm{11}}\\ ⋮\\ y_{\mathrm{1}{m}_{\mathrm{1}}}\end{array}\right),\\ \\ {\mathbf{y}}_{\mathrm{2}}=\left(\begin{array}{c}{y}_{\mathrm{21}}\\ ⋮\\ y_{\mathrm{2}{m}_{\mathrm{2}}}\end{array}\right),\\ \end{array}$$ (18)

and then concatenating the response vectors of different types,

$$\begin{array}{c}\mathbf{y}=\left(\begin{array}{c}{\mathbf{y}}_{\mathrm{1}}\\ {\mathbf{y}}_{\mathrm{2}}\end{array}\right).\end{array}$$ (19)

A similar assembly of the measurement errors can be done so that

$$\begin{array}{c}{\mathbf{ϵ}}_{\mathrm{1}}=\left(\begin{array}{c}{ϵ}_{\mathrm{11}}\\ ⋮\\ {ϵ}_{\mathrm{1}{m}_{\mathrm{1}}}\end{array}\right),\\ \\ {\mathbf{ϵ}}_{\mathrm{2}}=\left(\begin{array}{c}{ϵ}_{\mathrm{21}}\\ ⋮\\ {ϵ}_{\mathrm{2}{m}_{\mathrm{2}}}\end{array}\right),\\ \\ \mathbf{ϵ}=\left(\begin{array}{c}{\mathbf{ϵ}}_{\mathrm{1}}\\ {\mathbf{ϵ}}_{\mathrm{2}}\end{array}\right).\\ \end{array}$$ (20)

An assembly of the parameters can be accomplished by first constructing the parameter vector,

$$\begin{array}{c}\mathbf{\theta }=\left(\begin{array}{c}{\theta }_{\mathrm{1}}\\ {\theta }_{\mathrm{2}}\end{array}\right),\end{array}$$ (21)

and the design matrix,

$$\begin{array}{c}\mathbf{H}=\left(\begin{array}{cc}\mathrm{1}& \mathrm{0}\\ ⋮& ⋮\\ \mathrm{1}& \mathrm{0}\\ \mathrm{0}& \mathrm{1}\\ ⋮& ⋮\\ \mathrm{0}& \mathrm{1}\end{array}\right),\end{array}$$ (22)

so that the single individual model of (17) can be vectorized as

$$\begin{array}{c}\mathbf{y}=\mathbf{H}\mathbf{\theta }+\mathbf{ϵ}.\end{array}$$ (23)

We consider the case where subject-specific traits and measurement errors are both normally distributed,

$$\begin{array}{c}\mathbf{\theta }~\mathcal{N}\left(\mathit{\mu },{\mathbf{C}}_{\theta }\right),\\ \end{array}$$ (24)

$$\begin{array}{c}\mathbf{ϵ}~\mathcal{N}\left(\mathbf{0},{\mathbf{C}}_{ϵ}\right),\\ \end{array}$$ (25)

where μ, C _θ, and C _ϵ are fixed population characteristics.

Correlations between primary and secondary response variables y ₁ and y ₂ can be modeled as arising from a correlation between θ ₁ and θ ₂ (between-subjects correlation) or correlations between ϵ ₁ and ϵ ₂ (within-subjects correlation). Here, we assume that response correlations arise from between-subjects correlations:

$$\begin{array}{c}{\mathbf{C}}_{\theta }=\left(\begin{array}{cc}{\delta }_{\mathrm{1}}^{\mathrm{2}}& \rho {\delta }_{\mathrm{1}}{\delta }_{\mathrm{2}}\\ \rho {\delta }_{\mathrm{1}}{\delta }_{\mathrm{2}}& {\delta }_{\mathrm{2}}^{\mathrm{2}}\end{array}\right),\end{array}$$ (26)

where ρ (−1 < ρ < 1) represents the between-subjects correlation between primary and secondary response variable means and δ _r ² represents the between-subjects variance for response variable r. Furthermore, we assume that measurement errors are uncorrelated with response variable-specific variance σ _r ², so that correlations between the response variables arise only from the between-subjects components. For subject-specific repeated-measures data with no covariates for two response variables, it may be fair to consider the error variance within subjects to be independent as long as perturbations from the intercepts do not tend to be common over both response types.

The error covariance matrix for response type r is a diagonal matrix with dimension m _r, where each nonzero element is the type-specific error variance σ _r ²,

$$\begin{array}{c}{\mathrm{\Sigma }}_r=\left(\begin{array}{ccc}{\sigma }_r^{\mathrm{2}}& \mathrm{0}& \dots \\ \mathrm{0}& {\sigma }_r^{\mathrm{2}}& \ddots \\ ⋮& \ddots & \ddots \end{array}\right).\end{array}$$ (27)

The full error covariance matrix can then be built from the type-specific blocks,

$$\begin{array}{c}{\mathbf{C}}_{ϵ}=\left(\begin{array}{cc}{\mathrm{\Sigma }}_{\mathrm{1}}& \mathbf{0}\\ \mathbf{0}& {\mathrm{\Sigma }}_{\mathrm{2}}\end{array}\right).\end{array}$$ (28)

The bivariate Bayesian model we consider here is fully characterized by (23)–(28).

As was the case for model (4), for a particular individual, the expected primary response at time t _1j^∗ is

$$\begin{array}{c}E\left[y_{\mathrm{1}{j}^{\ast }}\hspace{0.17em}\mid \hspace{0.17em}\theta \right]=f_{\mathrm{1}{j}^{\ast }}={\theta }_{\mathrm{1}}.\end{array}$$ (29)

The MMSE prediction is

$$\begin{array}{c}{\widehat{f}}_{\mathrm{1}{j}^{\ast }}={\widehat{\theta }}_{\mathrm{1}}.\end{array}$$ (30)

The MMSE estimator for model (23) is [23]:

$$\begin{array}{c}\widehat{\mathbf{\theta }}=\mathit{\mu }+{\left({\mathbf{C}}_{\theta }^{-\mathrm{1}}+{\mathbf{H}}^{\prime }{\mathbf{C}}_{ϵ}^{-\mathrm{1}}\mathbf{H}\right)}^{-\mathrm{1}}{\mathbf{H}}^{\prime }{\mathbf{C}}_{ϵ}^{-\mathrm{1}}\left(\mathbf{y}-\mathbf{H}\mathit{\mu }\right).\end{array}$$ (31)

The MMSE estimator ${\widehat{\theta }}_{\mathrm{1}}$ can be extracted as the first element of $\widehat{\mathbf{\theta }}$.

The variance of the posterior distribution of θ∣y is given by

$$\begin{array}{c}\mathrm{V}\mathrm{a}\mathrm{r}\left(\mathbf{\theta }\hspace{0.17em}\mid \hspace{0.17em}\mathbf{y}\right)={\left({\mathbf{C}}_{\theta }^{-\mathrm{1}}+{\mathbf{H}}^{\prime }{\mathbf{C}}_{ϵ}^{-\mathrm{1}}\mathbf{H}\right)}^{-\mathrm{1}}.\end{array}$$ (32)

The variance of the posterior distribution of θ ₁∣y (and therefore f _1j^∗∣y) can be obtained by extracting the first element from the diagonal of θ∣y. The BMSE of the MMSE estimator θ can be obtained from the parameter BMSE matrix ${\mathbf{M}}_{\widehat{\theta }}$, which is given by [23]:

$$\begin{array}{c}{\mathbf{M}}_{\widehat{\theta }}=\mathrm{V}\mathrm{a}\mathrm{r}\left(\mathbf{\theta }\hspace{0.17em}\mid \hspace{0.17em}\mathbf{y}\right).\end{array}$$ (33)

The BMSE of ${\widehat{\theta }}_{\mathrm{1}}$ (and therefore ${\widehat{f}}_{\mathrm{1}{j}^{\ast }}$) can be obtained by extracting the first element from the diagonal of ${\mathbf{M}}_{\widehat{\theta }}$. Substituting (22), (26) and (28) into (33), we find that the BMSE for ${\widehat{\theta }}_{\mathrm{1}}$ can be simplified as follows:

$$\begin{array}{c}{\mathbf{M}}_{{\widehat{\theta }}_{\mathrm{1}}}={\left(\frac{m_{\mathrm{1}}}{{\sigma }_{\mathrm{1}}^{\mathrm{2}}}+\frac{\mathrm{1}}{{\delta }_{\mathrm{1}}^{\mathrm{2}}}+\lambda \left(m_{\mathrm{2}}\right)\right)}^{-\mathrm{1}},\end{array}$$ (34)

where

$$\begin{array}{c}\lambda \left(m_{\mathrm{2}}\right)=\frac{{\rho }^{\mathrm{2}}{\delta }_{\mathrm{2}}^{\mathrm{2}}/{\delta }_{\mathrm{1}}^{\mathrm{2}}}{{\delta }_{\mathrm{2}}^{\mathrm{2}}\left(\mathrm{1}-{\rho }^{\mathrm{2}}\right)+{\sigma }_{\mathrm{2}}^{\mathrm{2}}/m_{\mathrm{2}}}.\end{array}$$ (35)

Figure 3 illustrates the dependence of the BMSE of ${\widehat{f}}_{j^{\ast }}$ on the number of observations from the primary and secondary responses. For this illustration, the population parameters were fixed at the values δ ₁ = 1, δ ₂ = 1, ρ = 0.85, σ ₁ = 1, and σ ₂ = 1. The figure shows the decrease of the BMSE as a function of the collection of secondary response measurements, given different numbers of primary response measurements. For a large number of primary response measurements, little change in BMSE is derived from the secondary response. However, for a small number of primary response measurements, the BMSE decreases substantially with just a few measurements of the secondary response.

Figure 3.

BMSE (i.e., accuracy) of MMSE predictions of the expected primary response ${\widehat{f}}_{j^{\ast }}$ for the bivariate random intercept model specified in (23), as a function of the amount of data collected on a secondary response variable (m ₂), shown for different amounts of data collected on the primary response variable (m ₁). The figure illustrates that the accuracy of individualized Bayesian forecasting predictions improves progressively with just a few measurements of the secondary response variable when measurements of the primary response variable are increasingly sparse.

To further illustrate the bivariate Bayesian forecasting procedure, simulated subject-specific parameter pairs θ = (θ ₁, θ ₂) from the population distribution given by (24) with δ ₁ = 1, δ ₂ = 1, and ρ = 0.9 were generated for N = 20 individuals, as shown in Figure 4. From this simulated set, individual #19 (circled in Figure 4) with subject-specific parameter vector θ = (−1.6; −2.0) was chosen to illustrate the transition of the primary response prediction from the population mean response to the true expected response f _1j^∗ through Bayesian forecasting. For this individual, we simulated errors from (25) using ${\sigma }_{\mathrm{1}}=\sqrt{\mathrm{2}}$, ${\sigma }_{\mathrm{2}}=\sqrt{\mathrm{0.5}}$, m ₁ = 10, and m ₂ = 10. Bivariate responses were then constructed using (23).

Figure 4.

Subject-specific parameter pairs, simulated from a probability distribution specified by (24), from which a simulated individual was selected for a bivariate Bayesian forecasting simulation. Each number represents the parameter pair θ = (θ ₁, θ ₂) for a different simulated individual. The circled individual (#19) is used for the illustration in Figure 5. The diagonal line shows where the points would fall given a between-subjects correlation of ρ = 1.

The MMSE prediction ${\widehat{f}}_{\mathrm{1}{j}^{\ast }}$ for individual #19 was iteratively determined after assuming only primary responses were observed and after assuming pairs of primary and secondary responses were observed. The iterative estimates for both cases are shown in Figure 5, along with the simulated data. The variance of the posterior distribution for f _1j^∗ from (32) was used to construct 95% confidence intervals on f _1j^∗. For the individual considered, the predictions based on both response variables (purple line) were more accurate than the predictions based on only the primary response (blue line) for the first few iterations of MMSE prediction. Note, however, that while this behavior of the prediction accuracy is found on average, as follows from (34) it is not necessarily true for each and every individual to which the procedure may be applied. Thus, caution is needed in relying on bivariate Bayesian forecasting for improved prediction accuracy of specific individuals; improved prediction accuracy can only be counted on in the average over individuals.

Figure 5.

MMSE predictions of the expected primary response ${\widehat{f}}_{j^{\ast }}$ from the bivariate random intercept model specified in (23), and corresponding 95% confidence intervals (shaded areas) for individual #19. The MMSE estimator ${\widehat{f}}_{j^{\ast }}$ was iteratively determined assuming only primary responses were observed, as well as assuming both primary and secondary responses were observed. For the former case, the MMSE was iteratively determined by incorporating observations y _1j; for the latter case, the MMSE was iteratively determined by incorporating pairs of observations (y _1j, y _2j). Confidence intervals were obtained from quantiles of the posterior distribution, which is defined by the posterior mean (the MMSE estimator) (31) and the posterior variance (32).

6. Data Collection Cost Minimization

When Bayesian forecasting is applied to individualize predictions, data must be collected to tailor the population model to the individual at hand. In certain sleep research applications, such as forecasting of sleep parameters across nights or predicting performance deficits across periods of sleep deprivation, and in a wide range of other biomedical contexts, this requires creating multiple opportunities for taking measurements. This may be an expensive proposition, and reducing the number of measurement bouts needed to obtain the necessary data could entail considerable cost savings. By measuring secondary responses and incorporating these through bivariate Bayesian forecasting, it may be possible to achieve a given level of prediction accuracy at lower overall cost of data acquisition. Here we explore this possibility in the case of the bivariate random intercept model.

We consider a scenario in which the cost of collecting an observation on the primary response is c ₁, the cost of collecting an observation on the secondary response is c ₂, and the total cost of data collection is the sum of primary and secondary response costs accrued,

$$\begin{array}{c}c=m_{\mathrm{1}}{c}_{\mathrm{1}}+m_{\mathrm{2}}{c}_{\mathrm{2}},\end{array}$$ (36)

where m ₁, m ₂ ≥ 0. For this scenario, we determine how many observations m ₁ and m ₂ we may expect to have to collect from each response type in order to minimize the total cost of achieving, on average, a given prediction accuracy η ²:

$$\begin{array}{c}{\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}={\eta }^{\mathrm{2}}.\end{array}$$ (37)

We can simplify the minimization problem by removing either m ₁ or m ₂ from both the total cost equation and the nonnegativity constraints on the number of data points. Using ${\widehat{f}}_{\mathrm{1}{j}^{\ast }}={\widehat{\theta }}_{\mathrm{1}}$ (see (30)), it follows that the BMSE of ${\widehat{f}}_{\mathrm{1}{j}^{\ast }}$ is equal to the BMSE of ${\widehat{\theta }}_{\mathrm{1}}$ (i.e., ${\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}={\mathbf{M}}_{{\widehat{\theta }}_{\mathrm{1}}}$). Fixing ${\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}={\eta }^{\mathrm{2}}$ (37) therefore implies that ${\mathbf{M}}_{{\widehat{\theta }}_{\mathrm{1}}}={\eta }^{\mathrm{2}}$, which can be used with (34) to obtain a relationship between the number of primary and secondary observations needed to meet the average accuracy criterion η ²:

$$\begin{array}{c}{m}_{\mathrm{1}}={\sigma }_{\mathrm{1}}^{\mathrm{2}}\left(\frac{\mathrm{1}}{{\eta }^{\mathrm{2}}}-\frac{\mathrm{1}}{{\delta }_{\mathrm{1}}^{\mathrm{2}}}-\lambda \left(m_{\mathrm{2}}\right)\right).\end{array}$$ (38)

We then substitute (38) into (36):

$$\begin{array}{c}c=c_{\mathrm{1}}{\sigma }_{\mathrm{1}}^{\mathrm{2}}\left(\frac{\mathrm{1}}{{\eta }^{\mathrm{2}}}-\frac{\mathrm{1}}{{\delta }_{\mathrm{1}}^{\mathrm{2}}}-\lambda \left(m_{\mathrm{2}}\right)\right)+c_{\mathrm{2}}{m}_{\mathrm{2}}.\end{array}$$ (39)

The constraint m ₁ ≥ 0 can be equivalently formulated as an upper bound on λ(m ₂):

$$\begin{array}{c}\lambda \left(m_{\mathrm{2}}\right)\le \frac{\mathrm{1}}{{\eta }^{\mathrm{2}}}-\frac{\mathrm{1}}{{\delta }_{\mathrm{1}}^{\mathrm{2}}}.\end{array}$$ (40)

Consideration of this constraint is only necessary if there are a certain number of measurements on the secondary response for which it is possible to obtain the desired accuracy without measurement of the primary response; that is,

$$\begin{array}{c}\underset{m_{\mathrm{2}}\to \infty }{\lim }\lambda \left(m_{\mathrm{2}}\right)\ge \frac{\mathrm{1}}{{\eta }^{\mathrm{2}}}-\frac{\mathrm{1}}{{\delta }_{\mathrm{1}}^{\mathrm{2}}}.\end{array}$$ (41)

Substituting for λ(m ₂) from (35), this latter condition can be reformulated as follows:

$$\begin{array}{c}{\delta }_{\mathrm{1}}^{\mathrm{2}}\left(\mathrm{1}-{\rho }^{\mathrm{2}}\right)\le {\eta }^{\mathrm{2}}.\end{array}$$ (42)

Thus, m ₂ has an upper bound when η ², the desired BMSE of ${\widehat{f}}_{j^{\ast }}$, is not smaller than δ ₁ ²(1 − ρ ²), the minimum BMSE of ${\widehat{f}}_{j^{\ast }}$ which can be obtained using only the secondary response. It follows that the constraint set for the minimization problem is

$$\begin{array}{c}\mathrm{0}\le m_{\mathrm{2}}\le \frac{{\sigma }_{\mathrm{2}}^{\mathrm{2}}\left(\mathrm{1}/{\eta }^{\mathrm{2}}-\mathrm{1}/{\delta }_{\mathrm{1}}^{\mathrm{2}}\right)}{{\delta }_{\mathrm{2}}^{\mathrm{2}}\left(\mathrm{1}-{\rho }^{\mathrm{2}}\right)\left(\mathrm{1}/{\delta }_{\mathrm{1}}^{\mathrm{2}}\left(\mathrm{1}-{\rho }^{\mathrm{2}}\right)-\mathrm{1}/{\eta }^{\mathrm{2}}\right)}\text{if\hspace{0.17em}\hspace{0.17em}}{\delta }_{\mathrm{1}}^{\mathrm{2}}\left(\mathrm{1}-{\rho }^{\mathrm{2}}\right)\le {\eta }^{\mathrm{2}},\\ \\ \mathrm{0}\le m_{\mathrm{2}}\text{otherwise.}\\ \end{array}$$ (43)

If m ₂ exceeds its upper bound then the number of secondary response measurements is more than what is minimally needed to meet the average accuracy criterion (37).

The minimal cost solution occurs either on the boundary of the region defined by (43) or at a local minimum in the interior of this region. Figure 6 shows how different values of the error variance σ ² can result in either boundary or interior solution types. For this demonstration, the population parameters are set at ρ = 0.85, δ ₁ = 1.0, δ ₂ = 1.0, and σ ₂ = 0.5, the costs of measurement are assumed to be c ₁ = $500 and c ₂ = $100, and the BMSE of ${\widehat{f}}_{j^{\ast }}$ is fixed at the value η ² = 0.30.

Figure 6.

Illustration of three types of absolute minima of the total cost of obtaining a fixed average prediction accuracy inside the region defined by (43). Each plot shows the total cost c = c ₁ m ₁ + c ₂ m ₂ plotted against secondary response sample size m ₂. For m ₂ given, the primary response sample size m ₁ that obtains a fixed BMSE of η ² in the expected primary response ${\widehat{f}}_{j^{\ast }}$ is computed using (38). The lower and upper boundaries of the region defined by (43) are shown with solid vertical lines. (a) shows an interior point minimum, obtained by letting σ ₁ = 1.00. This type of solution occurs when the minimum of the unconstrained cost function lies within the region defined by (43). (b) shows a lower boundary solution, obtained by letting σ ₁ = 0.15. This type of solution occurs when the minimum of the unconstrained cost function lies below the feasible region defined by (43). (c) shows an upper boundary solution, obtained by letting σ ₁ = 3.00. This type of solution occurs when the minimum of the unconstrained cost function lies above the region defined by (43).

In cases where the solution lies in the interior of (43) at a local minimum, the solution must occur at critical points of (39), which can be found by setting to zero the derivative of the total cost with respect to m ₂:

$$\begin{array}{c}\frac{\partial }{\partial m_{\mathrm{2}}}c\left(m_{\mathrm{2}}\right)=-c_{\mathrm{1}}{\sigma }_{\mathrm{1}}^{\mathrm{2}}\frac{\partial \lambda \left(m_{\mathrm{2}}\right)}{\partial m_{\mathrm{2}}}+c_{\mathrm{2}}=\mathrm{0},\end{array}$$ (44)

where

$$\begin{array}{c}\frac{\partial \lambda \left(m_{\mathrm{2}}\right)}{\partial m_{\mathrm{2}}}=\frac{{\sigma }_{\mathrm{2}}^{\mathrm{2}}{\rho }^{\mathrm{2}}{\delta }_{\mathrm{2}}^{\mathrm{2}}/{\delta }_{\mathrm{1}}^{\mathrm{2}}}{{\left(m_{\mathrm{2}}{\delta }_{\mathrm{2}}^{\mathrm{2}}\left(\mathrm{1}-{\rho }^{\mathrm{2}}\right)+{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)}^{\mathrm{2}}}.\end{array}$$ (45)

Substituting the above expression for ∂λ(m ₂)/∂m ₂ into (44) and solving for m ₂, we obtain the following two critical points:

$$\begin{array}{c}{m}_{\mathrm{2}}^{\pm }=\frac{{\sigma }_{\mathrm{2}}}{{\delta }_{\mathrm{1}}{\delta }_{\mathrm{2}}^{\mathrm{2}}\left(\mathrm{1}-{\rho }^{\mathrm{2}}\right)}\left(-{\delta }_{\mathrm{1}}{\sigma }_{\mathrm{2}}\pm \sqrt{\frac{c_{\mathrm{1}}}{c_{\mathrm{2}}}}{\delta }_{\mathrm{2}}{\sigma }_{\mathrm{1}}\left|\rho \right|\right).\end{array}$$ (46)

The smaller critical point m ₂ ⁻ can be disregarded as a possible solution since it is always less than zero. The second derivative at m ₂ ⁺,

$$\begin{array}{c}\frac{{\partial }^{\mathrm{2}}}{\partial m_{\mathrm{2}}^{\mathrm{2}}}c\left(m_{\mathrm{2}}^{+}\right)=\frac{\mathrm{2}{c}_{\mathrm{2}}\sqrt{c_{\mathrm{2}}/c_{\mathrm{1}}}\left(\mathrm{1}-{\rho }^{\mathrm{2}}\right){\delta }_{\mathrm{1}}{\delta }_{\mathrm{2}}}{\left|\rho \right|{\sigma }_{\mathrm{1}}{\sigma }_{\mathrm{2}}},\end{array}$$ (47)

is positive when |ρ | <1, which implies that the cost function exhibits a local minimum at this point. If the local minimum m ₂ ⁺ is inside the region defined by (43) (see Figure 6(a)), then the solution to the cost minimization problem is

$$\begin{array}{c}{\widehat{m}}_{\mathrm{1}}={\sigma }_{\mathrm{1}}^{\mathrm{2}}\left(\frac{\mathrm{1}}{{\eta }^{\mathrm{2}}}-\frac{\mathrm{1}}{{\delta }_{\mathrm{1}}^{\mathrm{2}}\left(\mathrm{1}-{\rho }^{\mathrm{2}}\right)}\left(\mathrm{1}-\sqrt{\frac{c_{\mathrm{2}}{\delta }_{\mathrm{1}}^{\mathrm{2}}/{\sigma }_{\mathrm{1}}^{\mathrm{2}}}{c_{\mathrm{1}}{\delta }_{\mathrm{2}}^{\mathrm{2}}/{\sigma }_{\mathrm{2}}^{\mathrm{2}}}{\rho }^{\mathrm{2}}}\right)\right),\\ \\ {\widehat{m}}_{\mathrm{2}}=m_{\mathrm{2}}^{+},\\ \end{array}$$ (48)

where ${\widehat{m}}_{\mathrm{1}}$ is determined by substituting ${\widehat{m}}_{\mathrm{2}}$ into (38), and the total cost is found from (36).

Alternatively, if m ₂ ⁺ is below the lower boundary of the region defined by (43) (see Figure 6(b)), then the minimal cost solution involves collecting no data from the secondary response. The conditions for which the secondary response is not part of the minimal cost solution are as follows:

$$\begin{array}{c}\frac{c_{\mathrm{2}}}{c_{\mathrm{1}}}>\frac{{\delta }_{\mathrm{2}}^{\mathrm{2}}/{\sigma }_{\mathrm{2}}^{\mathrm{2}}}{{\delta }_{\mathrm{1}}^{\mathrm{2}}/{\sigma }_{\mathrm{1}}^{\mathrm{2}}}{\rho }^{\mathrm{2}},\end{array}$$ (49)

where δ _r ²/σ _r ² reflects the between-to-within variance ratio for the rth response type. For this case, the solution which achieves, on average, the level of accuracy η ² is found from (38):

$$\begin{array}{c}{\widehat{m}}_{\mathrm{1}}={\sigma }_{\mathrm{1}}^{\mathrm{2}}\left(\frac{\mathrm{1}}{{\eta }^{\mathrm{2}}}-\frac{\mathrm{1}}{{\delta }_{\mathrm{1}}^{\mathrm{2}}}\right),\\ \\ {\widehat{m}}_{\mathrm{2}}=\mathrm{0}.\\ \end{array}$$ (50)

Finally, if m ₂ ⁺ is above the upper boundary of the region defined by (43) (see Figure 6(c)), then the solution for m ₂ occurs at this boundary, where all the data are collected from the secondary response and none from the primary response. For this case, the solution is as follows:

$$\begin{array}{c}{\widehat{m}}_{\mathrm{1}}=\mathrm{0},\\ \\ {\widehat{m}}_{\mathrm{2}}=\frac{{\sigma }_{\mathrm{2}}^{\mathrm{2}}\left(\mathrm{1}/{\eta }^{\mathrm{2}}-\mathrm{1}/{\delta }_{\mathrm{1}}^{\mathrm{2}}\right)}{{\delta }_{\mathrm{2}}^{\mathrm{2}}\left(\mathrm{1}-{\rho }^{\mathrm{2}}\right)\left(\mathrm{1}/{\delta }_{\mathrm{1}}^{\mathrm{2}}\left(\mathrm{1}-{\rho }^{\mathrm{2}}\right)-\mathrm{1}/{\eta }^{\mathrm{2}}\right)}.\\ \end{array}$$ (51)

Figure 7 illustrates the number of observations required from primary and secondary responses to obtain a fixed level of accuracy η ² on average for different values of the between-subjects correlation ρ between primary and secondary response parameters. For the example shown, the population model parameters and cost parameters were fixed at δ ₁ = 1, δ ₂ = 1, σ ₁ = 1, σ ₂ = 0.5, c ₁ = 5, and c ₂ = 1, and the desired level of accuracy was η ² = 0.30. The figure illustrates the three cases described by (48), (50), and (51).

Figure 7.

Example of data collection cost minimization that shows the number of observations on primary and secondary responses needed to obtain a fixed BMSE in the MMSE prediction of the expected primary response ${\widehat{f}}_{j^{\ast }}$, for different values of the between-subjects correlation between primary and secondary response parameters. The solid curve represents the number of measurements to collect from the primary response, ${\widehat{m}}_{\mathrm{1}}$, and the dashed curve represents the number of measurements to collect from the secondary response, ${\widehat{m}}_{\mathrm{2}}$. In this example, for 0.0≤|ρ | ≤ 0.18, no data is to be collected from the secondary response, and the number of data points to collect from the primary response is obtained from (50). For 0.18<|ρ | ≤ 0.85, the number of secondary observations increases and the number of primary observations decreases with increasing correlation, as specified by ${\widehat{m}}_{\mathrm{2}}$ and ${\widehat{m}}_{\mathrm{1}}$ in (48). For 0.85<|ρ | ≤ 1.0, observations are to be collected only from the secondary response, the number of which is given by (51). The equation numbers are indicated near the relevant pieces of the curves.

7. Example: Efficient Assessment of an Individual's Characteristic Wakefulness after Sleep Onset

To illustrate the cost minimization approach outlined in the previous section, we apply it in an example involving the assessment of wakefulness after sleep onset (WASO) in laboratory-based sleep studies. Here we define WASO as the duration of intermittent wakefulness during a sleep period, between the time of sleep onset and the time of final awakening. WASO can be measured by polysomnography (PSG), that is, measuring the sleep electroencephalogram (EEG) and other physiological sleep signals and scoring sleep/wake states, typically in 30 s epochs, based on those signals. PSG is the gold standard procedure for sleep/wake assessment, but it is labor-intensive and expensive to perform. WASO may also be measured in the laboratory using wrist actigraphy (i.e., wrist activity monitoring), which is considerably less expensive. Although actigraphy is not considered a gold standard for measuring WASO, the correspondence with PSG-based WASO is at least moderate in healthy populations [24].

We base our example on data from n = 33 subjects (ages 22–38; 15 females) who spent between 6 and 13 nights and days inside a controlled laboratory environment with 10 h in bed for sleep (22:00–08:00) each day. The Institutional Review Board (IRB) of Washington State University approved the research, and subjects gave written informed consent. WASO was measured using both PSG (WASO-P) and actigraphy (WASO-A). PSG recordings were performed using digital equipment (Nihon Kohden, Foothill Ranch, CA). Sleep stages and periods of wakefulness were scored in 30 s epochs using standard criteria [25], and WASO-P was calculated from the scored records. Actigraphic recordings were made with Motionlogger wrist actigraphs (Ambulatory Monitoring, Inc., Ardsley, NY). Sleep and wakefulness were assessed from the actigraphic records using the automated algorithm of [26], which calculated WASO-A.

Let WASO-P be the primary response (as it is the gold standard measure) and let WASO-A be the secondary response. Assuming that WASO-P and WASO-A are normally distributed around distinct subject-specific means, we apply the model defined by (23)–(28) to our example. We anticipate that the subject-specific means for WASO-P and WASO-A are positively correlated. We aim to determine a cost-effective data collection scheme given a specific level of desired accuracy in estimates of an individual's mean WASO-P. For illustration purposes, we assume a fixed cost of $1,250 per night for WASO-P and $150 per night for WASO-A. We wish to estimate an individual's mean WASO-P to an average accuracy (i.e., $\sqrt{\mathrm{B}\mathrm{M}\mathrm{S}\mathrm{E}}$) of η = 15 min. Using the equations derived in the previous section, we estimate the number of nights of WASO-P and WASO-A that most cost-effectively achieves the desired level of accuracy on average.

We have estimated the population model parameters using the nlme package for R [27]. This package fits mixed-effects models using the approach of [28]. The maximum likelihood estimation method tends to underestimate variance parameters [29]; therefore, we estimate these parameters using the restricted maximum likelihood method [6].

In our dataset, the overall means for WASO-P and WASO-A were estimated as follows: μ ₁ = 54 min ± 5 min and μ ₂ = 32 min ± 4 min (estimate ± standard error), indicating that WASO-A tended to underestimate the total amount of WASO as compared to PSG. The estimated variability between subjects for WASO-P and WASO-A was found to be the same: δ ₁ = δ ₂ = 21 min. There was a substantial correlation between subject means for WASO-P and WASO-A: ρ = 0.69 (95% confidence interval: [0.30, 0.90]). The within-subject variation around the subject mean was σ ₁ = 32 min for WASO-P and σ ₂ = 21 min for WASO-A.

We determine from (43) that m ₂, the number of actigraphy nights, is not bounded above; that is, we cannot achieve our accuracy with actigraphy alone. Further, we find from (46) that m ₂ ⁺ is within the feasible region defined by (43), and, therefore, the solution to the cost minimization problem is given by (48). Applying these equations, we achieve an average accuracy of η = 15 min for minimal cost by collecting 3.99 nights of actigraphy and 0.73 nights of PSG (see Figure 8(a)). An approximately optimal solution in the integer domain is found by the common practice of rounding the optimal continuous solution to the nearest integer values [30]. We verify through a grid search that the minimal cost integer value solution neighbors the analytic solution and can be obtained by rounding down to three nights of actigraphy and up to one night of PSG. This yields a total cost of $1700. In contrast, to achieve the same accuracy with PSG alone would require 2.24, or in practice three nights of measurement, for a total cost of $3750.

Figure 8.

Cost of collection and number of nights of actigraphy and polysomnography (PSG) which combined will yield a certain average accuracy of wakefulness after sleep onset (WASO) parameter estimates. For illustration purposes, PSG and actigraphy are assumed to cost $1250 and $150 per night, respectively. The cost of data collection is fixed on each diagonal dashed line (for illustrative purposes these are only shown at fixed intervals) and increases as we collect more nights of PSG and actigraphy. Subject-specific WASO estimates of an average accuracy of (a) 15 min, (b) 14 min, and (c) 13 min are obtained on the solid curve. The point that minimizes the cost of obtaining the fixed accuracy (open squares) is determined as the point on the fixed accuracy curve where the line tangent to the curve is parallel to the fixed cost lines. For comparison the solution for obtaining the same average accuracy using only polysomnography is also shown (solid squares).

Note that the results are highly dependent on the estimated between-subjects correlation and that the 95% confidence interval for this correlation was large. The minimal cost solution also depends on the level of accuracy that is required; see Figures 8(b) and 8(c) for scenarios with an average accuracy of 14 min and 13 min, respectively.

8. Linear Models with Time Dependency

For both the univariate and bivariate linear models, time dependency can be introduced by adding time as a covariate. This complicates the construction of a design matrix that enables predictions with a given average accuracy. We show that, in models with time dependency, the BMSE of a predicted response depends on the times at which responses are measured, and this dependency can be summarized by the mean and variance of the measurement times.

To illustrate, we consider a linear approximation of a time-dependent model known as the two-process model of sleep regulation [10, 31]. It has been shown that, for a range of sleep/wake scenarios, the two-process model can describe temporal changes in waking cognitive performance as the algebraic difference between two functions describing physiological processes: the homeostatic pressure for sleep and the circadian pressure for wakefulness [11]. Here we focus solely on modeling the homeostatic pressure for sleep, the dynamics of which are specified separately for sleep and for wakefulness. The dynamics can be modeled using the recursive formulation of [10]

$$\begin{array}{c}{S}_t=\left\{\begin{array}{cc}{e}^{-\mathrm{\Delta }t/{\tau }_d}{S}_{t-\mathrm{1}}\hfill & \left(\text{sleep}\right)\hfill \\ \mathrm{1}-e^{-\mathrm{\Delta }t/{\tau }_r}\left(\mathrm{1}-S_{t-\mathrm{1}}\right)\hfill & \left(\text{wake}\right),\hfill \end{array}\right.\end{array}$$ (52)

where S _t represents the homeostatic pressure after the tth time step of duration Δt, S _t−1 represents the homeostatic pressure at one time step before (i.e., at time (t − 1) · Δt), Δt is typically fixed at 0.5 h, and τ _d and τ _r are time constants for the decay and rise of the homeostatic process during sleep and wakefulness, respectively.

We divide the sleep/wake schedule into periods of sleep and periods of wake, both indexed by k. Let S ₀ ^(k) represent the initial homeostatic pressure for the kth period of wakefulness. It has been proposed that change over time in the model is better modeled as linear rather than exponential [32]. For the purpose of the present example, we adopt this idea and approximate the model over a period of continuous wakefulness using a linear interpolation between the start and end points of the wake period (see Figure 9).

Figure 9.

Homeostatic pressure for wakefulness plotted over two complete sleep/wake cycles for a repeating schedule with 16 h of wakefulness and 8 h time in bed for sleep in each cycle. The dashed line represents the homeostatic pressure for sleep as given by (52); the solid line is a linear approximation based on interpolation between the sleep/wake transition points. The parameter values τ _d = 4.2 h and τ _r = 18.2 h are taken from [10]. The initial condition (S ₀) is derived assuming steady state (54). Black bars indicate the 8 h periods in bed for sleep.

Let y _j represent (hypothetical) measurements of the build-up of homeostatic pressure for sleep during wakefulness. These data can be modeled using the following approximation:

$$\begin{array}{c}{y}_j=S_{\mathrm{0}}^{\left(k\right)}+\frac{\left(\mathrm{1}-e^{-T^{\left(k\right)}/{\tau }_r}\right)\left(\mathrm{1}-S_{\mathrm{0}}^{\left(k\right)}\right)}{T^{\left(k\right)}}{t}_j+{ϵ}_j,\end{array}$$ (53)

where t _j represents the amount of time elapsed since awakening, T ^(k) is used to denote the duration of the kth wake period, and ϵ _j represents normally distributed measurement error.

We consider the homeostatic process over a repeating schedule consisting of T ^(k) = T = 16 h of wakefulness and 8 h of sleep. In this schedule, individuals maintain a steady state for which the homeostatic pressure S ₀ ^(k) at the onset of the wake period is constant across days; that is, S ₀ ^(k) = S ₀. This allows us to derive the homeostatic pressure at the start of wakefulness as a function of τ _r and τ _d:

$$\begin{array}{c}{S}_{\mathrm{0}}=\frac{e^{-\left(\mathrm{24}-T\right)/{\tau }_d}\left(\mathrm{1}-e^{-T/{\tau }_r}\right)}{\mathrm{1}-e^{-\left(\left(\mathrm{24}-T\right)/{\tau }_d+T/{\tau }_r\right)}}.\end{array}$$ (54)

The equation for the homeostatic pressure during a particular wake period can thus be written as

$$\begin{array}{c}{y}_j=\alpha +\beta t_j+ϵ,\end{array}$$ (55)

where

$$\begin{array}{c}\alpha =S_{\mathrm{0}},\\ \end{array}$$ (56)

$$\begin{array}{c}\beta =\frac{\left(\mathrm{1}-e^{-T/{\tau }_r}\right)\left(\mathrm{1}-S_{\mathrm{0}}\right)}{T}.\\ \end{array}$$ (57)

In matrix form, the model can be written as

$$\begin{array}{c}\mathbf{y}=\mathbf{H}\mathbf{\theta }+\mathbf{ϵ},\end{array}$$ (58)

where the design matrix is given by

$$\begin{array}{c}\mathbf{H}=\left(\begin{array}{cc}\mathrm{1}& t_{\mathrm{1}}\\ ⋮& ⋮\\ \mathrm{1}& t_m\end{array}\right),\end{array}$$ (59)

and the parameter vector is given by

$$\begin{array}{c}\mathbf{\theta }=\left(\begin{array}{c}\alpha \\ \beta \end{array}\right).\end{array}$$ (60)

Analogous to (5), but for two parameters, we assume the following prior distribution on model parameters:

$$\begin{array}{c}\left(\begin{array}{c}\alpha \\ \beta \end{array}\right)~\mathcal{N}\left(\mathit{\mu },\left(\begin{array}{cc}{\delta }_{\alpha }^{\mathrm{2}}& \mathrm{0}\\ \mathrm{0}& {\delta }_{\beta }^{\mathrm{2}}\end{array}\right)\right).\end{array}$$ (61)

As in (6), we assume that the errors are independent realizations from a normal distribution with zero mean and variance σ ². Analogous to (11) and (12), the BMSE for the MMSE prediction of the expected (primary) response ${\widehat{f}}_{j^{\ast }}$ at some time t _j^∗ in the univariate case is as follows (see Appendix A):

$$\begin{array}{c}{\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}\\ =\frac{\mathrm{1}/{\delta }_{\beta }^{\mathrm{2}}+m{s}^{\mathrm{2}}/{\sigma }^{\mathrm{2}}+m{\left(\overline{t}-t_{j^{\ast }}\right)}^{\mathrm{2}}/{\sigma }^{\mathrm{2}}+t_{j^{\ast }}^{\mathrm{2}}/{\delta }_{\alpha }^{\mathrm{2}}}{\left(\mathrm{1}/{\delta }_{\alpha }^{\mathrm{2}}+m/{\sigma }^{\mathrm{2}}\right)\left(\mathrm{1}/{\delta }_{\beta }^{\mathrm{2}}+m\left(s^{\mathrm{2}}+{\overline{t}}^{\mathrm{2}}\right)/{\sigma }^{\mathrm{2}}\right)-{\left(m\overline{t}/{\sigma }^{\mathrm{2}}\right)}^{\mathrm{2}}},\end{array}$$ (62)

where $\overline{t}$ denotes the mean of these times,

$$\begin{array}{c}\overline{t}=\frac{\mathrm{1}}{m}{\displaystyle \sum }_{j=\mathrm{1}}^m{t}_j,\end{array}$$ (63)

and s ² denotes the variance of these times,

$$\begin{array}{c}{s}^{\mathrm{2}}=\frac{\mathrm{1}}{m}{\displaystyle \sum }_{j=\mathrm{1}}^m{\left(t_j-\overline{t}\right)}^{\mathrm{2}}.\end{array}$$ (64)

Our task is to determine the measurement times t ₁,…, t _m for this example that will minimize ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$. Equation (62) shows that ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ depends on the measurement times only through their mean $\overline{t}$ and variance s ². Consequently, instead of conducting an m-dimensional minimization of ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ over all the measurement times t ₁,…, t _1m, we can write ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ as a function of $\overline{t}$ and s ² and conduct a two-dimensional minimization. In doing this, we find that ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ is minimized both when s ² → ∞ (see Appendix B) and also, more practically relevant, when

$$\begin{array}{c}\overline{t}=\frac{t_{j^{\ast }}\left({\sigma }^{\mathrm{2}}/m+{\delta }_{\alpha }^{\mathrm{2}}\right)}{{\delta }_{\alpha }^{\mathrm{2}}}.\end{array}$$ (65)

The optimal mean measurement time for this example, given by (65), lies slightly above the prediction time; in the limit as δ _α ² → ∞, ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ is minimized when the data are collected at times such that ${\overline{t}}_{\mathrm{m}\mathrm{i}\mathrm{n}}=t_{j^{\ast }}$. If the prior variance on the intercept is equal to the error variance (i.e., δ _α ² = σ ²), the effect of the Bayesian prior is equivalent to increasing the value of $\overline{t}$ that produces the minimum ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ by t _j^∗/m. This adjustment is the same as what would manifest with no prior information on the intercept when adding an additional measurement time, t _m+1 = 0, to the design matrix H.

The absolute minimum of ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ in (62) is not always located in the feasible region in this example, as defined by 0 ≤ t _j ≤ T, ∀j ∈ {1,…, m}. More specifically, the absolute minimum of the unconstrained case is not located inside the feasible region if and only if (see Appendix B):

$$\begin{array}{c}{\overline{t}}_{\mathrm{m}\mathrm{i}\mathrm{n}}\ge T.\end{array}$$ (66)

Under this condition we hypothesize that, within the feasible region, ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ exhibits an absolute minimum when all the data are collected at time T. This is easy to show for the case of one measurement time (i.e., m = 1), and Appendix C contains a proof for the case of two measurement times (i.e., m = 2).

For m > 2, we conducted a simulation study to search for a counterexample (i.e., a case where the value of ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ when all data is collected at time T is not the smallest value of ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ within the feasible region). For the simulation study, we simulated 10,000,000 times with the following values:

$$\begin{array}{c}{\delta }_{\alpha }~\text{Uniform}\left(\mathrm{0.001,1}\right);\\ \\ {\delta }_{\beta }~\text{Uniform}\left(\mathrm{0.001,1}\right);\\ \\ \sigma ~\text{Uniform}\left(\mathrm{0.001,1}\right);\\ \\ m~\text{Discrete\hspace{0.17em}\hspace{0.17em}Uniform}\left(\mathrm{1,2},\dots ,\mathrm{100}\right);\\ \\ t_{j^{\ast }}~\text{Uniform}\left(T\frac{{\delta }_{\alpha }^{\mathrm{2}}}{{\sigma }^{\mathrm{2}}/m+{\delta }_{\alpha }^{\mathrm{2}}},T\right).\\ \end{array}$$ (67)

Concerning the ranges of the variance components, note that ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ is invariant to the scale of the response. This can be demonstrated by scaling the variance matrices C _θ and C _ϵ by c _y and showing that the resulting ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ is then scaled by the same factor c _y. The conclusion is that the shape of the surface of ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ depends on the variance components only through their relative magnitudes. Furthermore, we argue that if any variance component is more than three orders of magnitude greater than any other component, then it would be advantageous to simplify the model by removing the smaller component. As such, all cases for which this model is reasonable can be covered within the range from 0.001 to 1 for each variance component. Further, concerning the number of observations, we expect that if there is a counterexample, it can be found somewhere in the range 1 ≤ m ≤ 100. Finally, the range for t _j^∗ is determined specifically so that the absolute minimum lies outside the feasible region.

For each simulation, we compared ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ at the hypothesized minimum, where t _j = T, ∀j ∈ {1,…, m}, to a randomly chosen time point, where each t _j is a realization from the following distribution:

$$\begin{array}{c}{t}_j~\text{Uniform}\left(\mathrm{0},T\right).\end{array}$$ (68)

For each of the 10,000,000 simulations, ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ at the hypothesized minimum was indeed smaller than ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ at the randomly chosen time point. Thus, we found no evidence against our original hypothesis that ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ exhibits an absolute minimum if all the data are collected at time T. An analytical proof is beyond the scope of this paper.

We now extend our analysis to consider a time-dependent model with both primary and secondary responses. We formulate the model as follows:

$$\begin{array}{c}{y}_{rj}=s_{rj}+{ϵ}_{rj}={\alpha }_r+{\beta }_r{t}_{rj}+{ϵ}_{rj}.\end{array}$$ (69)

In matrix form, the model is

$$\begin{array}{c}\mathbf{y}=\mathbf{H}\mathbf{\theta }+\mathbf{ϵ},\end{array}$$ (70)

where the design matrix is given by

$$\begin{array}{c}\mathbf{H}=\left(\begin{array}{cccc}\mathrm{1}& \mathrm{0}& t_{\mathrm{11}}& \mathrm{0}\\ ⋮& ⋮& ⋮& ⋮\\ \mathrm{1}& \mathrm{0}& t_{\mathrm{1}{m}_{\mathrm{1}}}& \mathrm{0}\\ \mathrm{0}& \mathrm{1}& \mathrm{0}& t_{\mathrm{21}}\\ ⋮& ⋮& ⋮& ⋮\\ \mathrm{0}& \mathrm{1}& \mathrm{0}& t_{\mathrm{2}{m}_{\mathrm{2}}}\end{array}\right),\end{array}$$ (71)

and the parameter vector is given by

$$\begin{array}{c}\mathbf{\theta }=\left(\begin{array}{c}{\alpha }_{\mathrm{1}}\\ {\alpha }_{\mathrm{2}}\\ {\beta }_{\mathrm{1}}\\ {\beta }_{\mathrm{2}}\end{array}\right).\end{array}$$ (72)

Let us assume the following prior distribution on model parameters:

$$\begin{array}{c}\left(\begin{array}{c}{\alpha }_{\mathrm{1}}\\ {\alpha }_{\mathrm{2}}\\ {\beta }_{\mathrm{1}}\\ {\beta }_{\mathrm{2}}\end{array}\right)~\mathcal{N}\left(\mathit{\mu },\left(\begin{array}{cccc}{\delta }_{{\alpha }_{\mathrm{1}}}^{\mathrm{2}}& \rho {\delta }_{{\alpha }_{\mathrm{1}}}{\delta }_{{\alpha }_{\mathrm{2}}}& \mathrm{0}& \mathrm{0}\\ \rho {\delta }_{{\alpha }_{\mathrm{1}}}{\delta }_{{\alpha }_{\mathrm{2}}}& {\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}& \mathrm{0}& \mathrm{0}\\ \mathrm{0}& \mathrm{0}& {\delta }_{{\beta }_{\mathrm{1}}}^{\mathrm{2}}& \omega {\delta }_{{\beta }_{\mathrm{1}}}{\delta }_{{\beta }_{\mathrm{2}}}\\ \mathrm{0}& \mathrm{0}& \omega {\delta }_{{\beta }_{\mathrm{1}}}{\delta }_{{\beta }_{\mathrm{2}}}& {\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}\end{array}\right)\right).\end{array}$$ (73)

As in (28), we assume that the errors are independent realizations from a normal distribution with zero mean and σ _r  (r = 1,2). The BMSE of estimates of the primary response at time t _1j^∗ is given by

$$\begin{array}{c}{\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}={\mathbf{h}}^{\prime }{\mathbf{M}}_{\widehat{\theta }}\mathbf{h},\end{array}$$ (74)

where

$$\begin{array}{c}\mathbf{h}=\left(\begin{array}{c}\mathrm{1}\\ \mathrm{0}\\ t_{\mathrm{1}{j}^{\ast }}\\ \mathrm{0}\end{array}\right),\\ \end{array}$$ (75)

(76)

where ${\overline{t}}_{\mathrm{1}}$ denotes the mean of the primary measurement times,

$$\begin{array}{c}{\overline{t}}_{\mathrm{1}}=\frac{\mathrm{1}}{m_{\mathrm{1}}}{\displaystyle \sum }_{j=\mathrm{1}}^m{t}_{\mathrm{1}j},\end{array}$$ (77)

${\overline{t}}_{\mathrm{2}}$ denotes the mean of the secondary measurement times,

$$\begin{array}{c}{\overline{t}}_{\mathrm{2}}=\frac{\mathrm{1}}{m_{\mathrm{2}}}{\displaystyle \sum }_{j=\mathrm{1}}^m{t}_{\mathrm{2}j},\end{array}$$ (78)

s ₁ ² denotes the variance of the primary measurement times,

$$\begin{array}{c}{s}_{\mathrm{1}}^{\mathrm{2}}=\frac{\mathrm{1}}{m_{\mathrm{1}}}{\displaystyle \sum }_{j=\mathrm{1}}^{m_{\mathrm{1}}}{\left(t_{\mathrm{1}j}-{\overline{t}}_{\mathrm{1}}\right)}^{\mathrm{2}},\end{array}$$ (79)

and s ₂ ² denotes the variance of the secondary measurement times,

$$\begin{array}{c}{s}_{\mathrm{2}}^{\mathrm{2}}=\frac{\mathrm{1}}{m_{\mathrm{2}}}{\displaystyle \sum }_{j=\mathrm{1}}^{m_{\mathrm{2}}}{\left(t_{\mathrm{2}j}-{\overline{t}}_{\mathrm{2}}\right)}^{\mathrm{2}}.\end{array}$$ (80)

Our task in the bivariate case of the example is to determine the primary measurement times t ₁₁,…, t _1m1 and the secondary measurement times t ₂₁,…, t _2m2 that minimize ${\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}$. Equations (74) and (76) show that ${\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}$ depends on the measurement times only through their response-specific means and variances, ${\overline{t}}_{\mathrm{1}}$, ${\overline{t}}_{\mathrm{2}}$, s ₁ ², and s ₂ ². Consequently, it is sufficient to minimize ${\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}$ with respect to ${\overline{t}}_{\mathrm{1}}$, ${\overline{t}}_{\mathrm{2}}$, s ₁ ², and s ₂ ² and choose any set of measurement times with these means and variances. We find that ${\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}$ can be minimized by collecting data at the following times (see Appendix D):

$$\begin{array}{c}{\overline{t}}_{\mathrm{1}\mathrm{m}\mathrm{i}\mathrm{n}}\\ =\frac{t_{\mathrm{1}{j}^{\ast }}\left({\delta }_{{\alpha }_{\mathrm{1}}}^{\mathrm{2}}\left(\left(\mathrm{1}-{\rho }^{\mathrm{2}}\right){\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}/m_{\mathrm{2}}\right)+\left({\sigma }_{\mathrm{1}}^{\mathrm{2}}/m_{\mathrm{1}}\right)\left({\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}/m_{\mathrm{2}}\right)\right)}{{\delta }_{{\alpha }_{\mathrm{1}}}^{\mathrm{2}}\left(\left(\mathrm{1}-{\rho }^{\mathrm{2}}\right){\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}/m_{\mathrm{2}}\right)},\end{array}$$ (81)

$$\begin{array}{c}{\overline{t}}_{\mathrm{2}\mathrm{m}\mathrm{i}\mathrm{n}}=\mathrm{0}.\end{array}$$ (82)

The optimal mean measurement time for the primary response variable, ${\overline{t}}_{\mathrm{1}\mathrm{m}\mathrm{i}\mathrm{n}}$ in (81), lies slightly above the prediction time. In the limit as m ₂ ↓ 0, the solution becomes the univariate solution:

$$\begin{array}{c}{\overline{t}}_{\mathrm{1}\mathrm{m}\mathrm{i}\mathrm{n}}=\frac{t_{\mathrm{1}{j}^{\ast }}\left({\delta }_{{\alpha }_{\mathrm{1}}}^{\mathrm{2}}+{\sigma }_{\mathrm{1}}^{\mathrm{2}}/m_{\mathrm{1}}\right)}{{\delta }_{{\alpha }_{\mathrm{1}}}^{\mathrm{2}}}.\end{array}$$ (83)

Again, the absolute minimum of ${\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}$ is not always located in the feasible region defined by 0 ≤ t _rj ≤ T. More specifically, the absolute minimum of the unconstrained function is located outside of the feasible region if and only if (see Appendix D):

$$\begin{array}{c}{\overline{t}}_{\mathrm{1}\mathrm{m}\mathrm{i}\mathrm{n}}\ge T.\end{array}$$ (84)

Under this condition, it seems logical that the minimum would occur if we were to collect all the primary data at time T and all the secondary data at time zero. However, a simulation study similar to that described above revealed counterexamples, which occurred when ω > 0.99. When T was decreased (or the ranges of the variance components for the slope were increased), counterexamples also occurred at smaller values of ω. Therefore, the optimal measurement scheme for the bivariate case appears to depend on ω and T. When ω is small or T is large (in comparison to the magnitudes of δ _β1 and δ _β2), the optimal design seems to be collecting all the primary data at t = T and all the secondary data at t = 0. When ω is large or T is small (compared to the magnitudes of δ _β1 and δ _β2), better designs are likely to be found numerically.

In summary, the average prediction accuracy for the simple linear time-dependent model depends not only on the number of measurements collected, but also on the times when these measurements are taken. In the univariate case, the prediction accuracy depends on these times only through their mean and variance. In the example of the two-process model, the optimal mean of the measurement times is slightly after the prediction time, where the delay increases with more prior information and with fewer or less informative data. When little prior information on the intercept is available, it is usually possible to collect data so that the absolute minimum of ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ is achieved. In the case where the theoretical minimum cannot be achieved (i.e., (66) is not satisfied), minimization of ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ can be achieved by collecting all the data at time T.

In the bivariate case of our example, the prediction accuracy depends on the measurement times only through the means and variances of the primary and secondary measurement times. Furthermore, ${\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}$ is minimized by centering the primary measurement times above t _1j^∗ (see (81)) as in the univariate case and collecting all secondary measurements at time zero. As in the univariate case, when little prior information on the primary intercept is available, it is usually possible to collect data so that the absolute minimum of ${\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}$ is achieved. In the case where this minimum cannot be achieved (i.e., (84) is not satisfied), minimization of ${\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}$ can usually be obtained for parameter ranges considered in our simulation by collecting all primary data at time T, and all secondary data at time zero.

9. Nonlinear Models with Time Dependency

Lastly, we focus briefly on the nonlinear case, where the BMSE generally lacks a closed form solution. Obtaining the BMSE of the MMSE estimator for nonlinear models typically requires numerical integration of the joint probability density of y and θ. We illustrate this with an example in which we numerically estimate the prediction BMSE given a nonlinear model and a single primary response measurement and show how it can be improved by a secondary response measurement.

We consider a two-parameter sinusoidal model of circadian (i.e., 24 h) rhythmicity, defined for a given subject and a bivariate response (r = 1,2), as follows:

$$\begin{array}{c}{y}_{rj}=f_{rj}+{ϵ}_{rj}=A_r\sin \left(\frac{\mathrm{2}\pi \left(t_{rj}-\varphi \right)}{\mathrm{24}}\right)+{ϵ}_{rj},\end{array}$$ (85)

where t is in hours; A _r represents a response-specific amplitude; and ϕ represents the phase, which is assumed to be common to both response types. For our example, we assume

$$\begin{array}{c}\left(\begin{array}{c}{A}_{\mathrm{1}}\\ A_{\mathrm{2}}\end{array}\right)~\mathcal{N}\left(\left(\begin{array}{c}\mathrm{5}\\ \mathrm{5}\end{array}\right),\left(\begin{array}{cc}\mathrm{1}& \mathrm{0.95}\\ \mathrm{0.95}& \mathrm{1}\end{array}\right)\right),\\ \\ \varphi ~\mathcal{N}\left(\mathrm{0,2}\right).\\ \end{array}$$ (86)

We simulated n = 1000 individuals from this model using normally distributed errors, with primary and secondary response standard deviations of σ ₁ = 0.25 and σ ₂ = 0.1. For each individual, we simulated a single primary response at time t ₁₁ = 14 and a single secondary response at time t ₁₂ = 22. Bayesian forecasting was performed using Markov Chain Monte Carlo (MCMC) with a chain length of 100,000 to obtain the MMSE predictions f _1j^∗ for each individual at time t _1j^∗ = 24. Predictions were constructed using a primary data point alone, and also using both a primary data point and a secondary data point.

Figure 10 shows these predictions for a randomly chosen individual. The 95% confidence intervals on f _1j^∗ were constructed using the quantiles of the posterior distribution for f _1j^∗. For the individual considered, the predictions based on both response variables (purple line and shading) are at many times substantially more accurate (i.e., they have smaller posterior variance) than the predictions based on only the primary response (blue line and shading).

Figure 10.

MMSE predictions of the expected primary response ${\widehat{f}}_{\mathrm{1}{j}^{\ast }}$ and corresponding 95% confidence intervals for a randomly simulated individual from the model specified in (85). The MMSE estimator ${\widehat{f}}_{\mathrm{1}{j}^{\ast }}$ is determined assuming each of the following: no data was observed (black line and gray confidence interval), a single primary response was observed (blue line and confidence interval), and both a primary and a secondary response were observed (purple line and confidence interval). The expected primary and secondary responses using the subject's simulated parameter values are shown with blue and red dashed lines. The black vertical dashed line shows the time at which predictions are made.

The average accuracy over individuals was assessed at time t _1j^∗ = 24 by estimating the BMSE, as follows (cf. (3)):

$$\begin{array}{c}{\widehat{\mathbf{M}}}_{{\widehat{f}}_{\mathrm{1}i{j}^{\ast }}}=\frac{\mathrm{1}}{N}{\displaystyle \sum }_{i=\mathrm{1}}^N{\left(f_{\mathrm{1}i{j}^{\ast }}-{\widehat{f}}_{\mathrm{1}i{j}^{\ast }}\right)}^{\mathrm{2}}.\end{array}$$ (87)

The estimated BMSE of ${\widehat{f}}_{\mathrm{1}i{j}^{\ast }}$ when using only the primary data points was 0.53, and the estimated BMSE of ${\widehat{f}}_{\mathrm{1}i{j}^{\ast }}$ when using both the primary and secondary data points was 0.20. These results suggest that there are nonlinear modeling scenarios where secondary response variables can improve predictions of primary response variables considerably via between-subjects correlation.

10. Discussion

In this paper, we illustrated in a Bayesian, repeated-measures framework how to improve the prediction accuracy of the expected response, as measured by the BMSE, for a primary response variable in the presence of a secondary response variable that is correlated with the primary response variable between subjects. To set up the general procedure of improving prediction accuracy through Bayesian forecasting, we constructed the BMSE for a simple univariate random intercept model.

We applied the general procedure to a bivariate random intercept model and derived the BMSE of the primary response predictions for this model. We studied how the BMSE depends on the number of observations from the secondary response and found that, for a fixed number of primary response observations, the BMSE is bounded below as specified by (40). The potential value of considering a secondary response may be assessed by considering whether this lower bound represents a meaningful improvement in the expected prediction accuracy.

Assuming the availability of a reasonably highly correlated secondary variable we also addressed the problem of determining the number of primary and secondary measurements needed to obtain a given average accuracy at minimal cost. We derived equations for the solution to this problem and illustrated their use with an example from sleep research. Given previously observed means, variances, and between-subjects correlation for polysomnographic (primary) and actigraphic (secondary) measurements of wakefulness after sleep onset and assumed reasonable measurement costs, we found that, to obtain an average prediction accuracy of 15 min, the use of actigraphy in addition to polysomnography resulted in a substantial reduction in estimated data collection costs as compared to polysomnography alone.

We then considered a steady-state, linear approximation of the homeostatic component for the two-process model of sleep regulation [10, 31] in the univariate case. We found that the minimization of the BMSE with respect to the vector of times at which the data are collected can be divided into two subcases. In one subcase, the BMSE is minimized by collecting data at times with a mean value slightly above the time at which predictions are to be made, with the offset being inversely proportional to the variance of the prior on the intercept. In the second subcase, all of the data is to be collected at the maximal time point. We extended the results of this example to the bivariate case, which again can be divided into two subcases. In one subcase, the BMSE is minimized by collecting the data so that the secondary measurement times have a mean of zero, and the primary variable is collected at times with a mean value somewhat above the time at which the predictions are to be made, like in the univariate case. In the other subcase, the time points that minimize the BMSE may best be found numerically.

Finally, we considered a nonlinear circadian model and determined the improvement in individualized prediction accuracy from a single primary data point versus both a single primary data point and a single secondary data point. For this particular model (85), we found the improvement to be substantial, suggesting that there are cases for nonlinear models where a secondary variable can substantially improve prediction accuracy for the primary variable, given a reasonably high between-subjects correlation.

In conclusion, depending on the between- and within-subjects variance components and the between-subjects correlation between primary and secondary responses, using secondary response data can be effective in increasing the individualized prediction accuracy on the primary response variable in Bayesian forecasting.

This work represents an improvement over the work of Chandler and colleagues [19], who proposed incorporating secondary variables in individualized performance predictions as covariates in a generalized linear model. An advantage of their approach is that it accounts for perturbations on system dynamics from external factors that are common to the outcome variables considered. A drawback is that the approach does not accumulate information about individual differences over time and therefore does not become increasingly accurate for individualized predictions as more data are collected for the individual at hand. Furthermore, the technique requires secondary data to be measured at the same time as the primary data is to be predicted. This is not a requirement for the presently proposed method, for which individualized predictions can be made for any given time, even if secondary data are unavailable then.

That said, here we considered only Bayesian models with diagonally structured error covariance matrices. Such models do not account for correlation within subjects between response types. The work of Chandler and colleagues [19] does account for such correlation, using a fixed linear relationship between primary and secondary responses. We did not consider this possibility here, using merely a diagonal error covariance matrix with one parameter for each response type. However, the Bayesian modeling framework in this paper can be expanded to account for correlation within subjects between response types by adding additional structure to the error covariance matrix.

Finally, the models described here assume that error variance is constant over repeated measures and across different subjects. This constraint can be relaxed easily by allowing the error covariance matrix to have different elements for different individuals.

The multivariate repeated-measures Bayesian forecasting framework presented here may be useful in a variety of clinical settings. One example is modeling the disabling effects of chronic back pain, where pain-related fear may be a good choice for a secondary variable [33]. For other examples, a rich literature in this area can be found in the domain of anesthesiology [34], where clinical applications of multivariate Bayesian forecasting abound.

Appendices

A. Expression for the Univariate Linear Time-Dependent Model

Theorem A.1 A.1. —

The BMSE of MMSE predictions for prediction BMSE in the univariate, linear, time-dependent model given by (58) is given by

$$\begin{array}{c}{\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}={\mathbf{h}}^{\prime }{\left({\mathbf{C}}_{\theta }^{-\mathrm{1}}+{\mathbf{H}}^{\prime }{\mathbf{C}}_{ϵ}^{-\mathrm{1}}\mathbf{H}\right)}^{-\mathrm{1}}\mathbf{h},\end{array}$$ (A.1)

where

$$\begin{array}{c}\mathbf{h}=\left(\begin{array}{c}\mathrm{1}\\ t_{j^{\ast }}\end{array}\right),\end{array}$$ (A.2)

where t _j^∗ represents the time for which the expected response is predicted, C _θ is the prior variance matrix for the parameter vector θ, H is the design matrix, C _ϵ is the error covariance matrix, and ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ is defined in (3).

Proof —

Since

$$\begin{array}{c}{f}_{j^{\ast }}={\mathbf{h}}^{\prime }\mathbf{\theta }\end{array}$$ (A.3)

and since the MMSE estimator commutes over linear transformations [23], the MMSE estimator of the expected response is given by

$$\begin{array}{c}{\widehat{f}}_{j^{\ast }}={\mathbf{h}}^{\prime }\widehat{\mathbf{\theta }}.\end{array}$$ (A.4)

Therefore,

$$\begin{array}{c}{\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}E\left[{\left(f_{j^{\ast }}-{\widehat{f}}_{j^{\ast }}\right)}^{\mathrm{2}}\right]\\ =E\left[\left(f_{j^{\ast }}-{\widehat{f}}_{j^{\ast }}\right){\left(f_{j^{\ast }}-{\widehat{f}}_{j^{\ast }}\right)}^{\prime }\right]\\ =E\left[{\mathbf{h}}^{\prime }\left(\mathbf{\theta }-\widehat{\mathbf{\theta }}\right){\left(\mathbf{\theta }-\widehat{\mathbf{\theta }}\right)}^{\prime }\mathbf{h}\right]\\ ={\mathbf{h}}^{\prime }E\left[\left(\mathbf{\theta }-\widehat{\mathbf{\theta }}\right){\left(\mathbf{\theta }-\widehat{\mathbf{\theta }}\right)}^{\prime }\right]\mathbf{h}={\mathbf{h}}^{\prime }{\mathbf{M}}_{\widehat{\theta }}\mathbf{h},\end{array}$$ (A.5)

where the parameter BMSE matrix ${\mathbf{M}}_{\widehat{\theta }}$ is given by [23] as follows:

$$\begin{array}{c}{\mathbf{M}}_{\widehat{\theta }}\equiv E\left[\left(\mathbf{\theta }-\widehat{\mathbf{\theta }}\right){\left(\mathbf{\theta }-\widehat{\mathbf{\theta }}\right)}^{\prime }\right]={\left({\mathbf{C}}_{\theta }^{-\mathrm{1}}+{\mathbf{H}}^{\prime }{\mathbf{C}}_{ϵ}^{-\mathrm{1}}\mathbf{H}\right)}^{-\mathrm{1}}.\end{array}$$ (A.6)

Therefore, the BMSE of the estimated response is given by (A.1).

Theorem A.2 A.2. —

The BMSE of MMSE predictions for the univariate, linear, time-dependent model given by (58) is given by

$$\begin{array}{c}{\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}\\ =\frac{\mathrm{1}/{\delta }_{\beta }^{\mathrm{2}}+m{s}^{\mathrm{2}}/{\sigma }^{\mathrm{2}}+m{\left(\overline{t}-t_{j^{\ast }}\right)}^{\mathrm{2}}/{\sigma }^{\mathrm{2}}+t_{j^{\ast }}^{\mathrm{2}}/{\delta }_{\alpha }^{\mathrm{2}}}{\left(\mathrm{1}/{\delta }_{\alpha }^{\mathrm{2}}+m/{\sigma }^{\mathrm{2}}\right)\left(\mathrm{1}/{\delta }_{\beta }^{\mathrm{2}}+m{s}^{\mathrm{2}}/{\sigma }^{\mathrm{2}}\right)+m{\overline{t}}^{\mathrm{2}}/{\delta }_{\alpha }^{\mathrm{2}}{\sigma }^{\mathrm{2}}},\end{array}$$ (A.7)

where ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ is given in (A.1).

Proof —

For this model, the inverse of the between-subjects covariance matrix is given by

$$\begin{array}{c}{\mathbf{C}}_{\theta }^{-\mathrm{1}}=\left(\begin{array}{cc}\frac{\mathrm{1}}{{\delta }_{\alpha }^{\mathrm{2}}}& \mathrm{0}\\ \mathrm{0}& \frac{\mathrm{1}}{{\delta }_{\beta }^{\mathrm{2}}}\end{array}\right).\end{array}$$ (A.8)

Furthermore, the parameter BMSE matrix is given by (see Theorem A.1)

$$\begin{array}{c}{\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}={\mathbf{h}}^{\prime }{\left({\mathbf{C}}_{\theta }^{-\mathrm{1}}+{\mathbf{H}}^{\prime }{\mathbf{C}}_{ϵ}^{-\mathrm{1}}\mathbf{H}\right)}^{-\mathrm{1}}\mathbf{h},\end{array}$$ (A.9)

where h is the covariate vector at the time at which we want to make predictions:

$$\begin{array}{c}\mathbf{h}=\left(\begin{array}{c}\mathrm{1}\\ t_{j^{\ast }}\end{array}\right).\end{array}$$ (A.10)

The error variance matrix is as follows:

$$\begin{array}{c}{\mathbf{C}}_{ϵ}={\sigma }^{\mathrm{2}}\mathbf{I}.\end{array}$$ (A.11)

The design matrix is as follows:

$$\begin{array}{c}\mathbf{H}=\left(\begin{array}{cc}\mathrm{1}& t_{\mathrm{1}}\\ ⋮& ⋮\\ \mathrm{1}& t_m\end{array}\right).\end{array}$$ (A.12)

The matrix multiplication H′C _ϵ ⁻¹ H can be computed to be

$$\begin{array}{c}{\mathbf{H}}^{\prime }{\mathbf{C}}_{ϵ}^{-\mathrm{1}}\mathbf{H}=\frac{\mathrm{1}}{{\sigma }^{\mathrm{2}}}\left(\begin{array}{cc}m& {\displaystyle \sum }_{j=\mathrm{1}}^m{t}_j\\ {\displaystyle \sum }_{j=\mathrm{1}}^m{t}_j& {\displaystyle \sum }_{j=\mathrm{1}}^m{t}_j^{\mathrm{2}}\end{array}\right).\end{array}$$ (A.13)

The matrix inverse (C _θ ⁻¹ + H′C _ϵ ⁻¹ H)⁻¹ can then be computed as follows:

$$\begin{array}{c}{\left({\mathbf{C}}_{\theta }^{-\mathrm{1}}+{\mathbf{H}}^{\prime }{\mathbf{C}}_{ϵ}^{-\mathrm{1}}\mathbf{H}\right)}^{-\mathrm{1}}\\ ={\left(\begin{array}{cc}\frac{\mathrm{1}}{{\delta }_{\alpha }^{\mathrm{2}}}+\frac{m}{{\sigma }^{\mathrm{2}}}& {\displaystyle \sum }_{j=\mathrm{1}}^m\frac{t_j}{{\sigma }^{\mathrm{2}}}\\ {\displaystyle \sum }_{j=\mathrm{1}}^m\frac{t_j}{{\sigma }^{\mathrm{2}}}& \frac{\mathrm{1}}{{\delta }_{\beta }^{\mathrm{2}}}+\frac{{\sum }_{j=\mathrm{1}}^m{t}_j^{\mathrm{2}}}{{\sigma }^{\mathrm{2}}}\end{array}\right)}^{-\mathrm{1}}\\ ={\left(\left(\frac{\mathrm{1}}{{\delta }_{\alpha }^{\mathrm{2}}}+\frac{m}{{\sigma }^{\mathrm{2}}}\right)\left(\frac{\mathrm{1}}{{\delta }_{\beta }^{\mathrm{2}}}+\frac{{\sum }_{j=\mathrm{1}}^m{t}_j^{\mathrm{2}}}{{\sigma }^{\mathrm{2}}}\right)-{\left(\frac{{\sum }_{j=\mathrm{1}}^m{t}_j}{{\sigma }^{\mathrm{2}}}\right)}^{\mathrm{2}}\right)}^{-1}\\ ·\left(\begin{array}{cc}\frac{\mathrm{1}}{{\delta }_{\beta }^{\mathrm{2}}}+\frac{{\sum }_{j=\mathrm{1}}^m{t}_j^{\mathrm{2}}}{{\sigma }^{\mathrm{2}}}& -\frac{{\sum }_{j=\mathrm{1}}^m{t}_j}{{\sigma }^{\mathrm{2}}}\\ -\frac{{\sum }_{j=\mathrm{1}}^m{t}_j}{{\sigma }^{\mathrm{2}}}& \frac{\mathrm{1}}{{\delta }_{\alpha }^{\mathrm{2}}}+\frac{m}{{\sigma }^{\mathrm{2}}}\\ & \end{array}\right).\end{array}$$ (A.14)

Computing the quadratic form, we find that

$$\begin{array}{c}{\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}=\frac{\mathrm{1}/{\delta }_{\beta }^{\mathrm{2}}+\left({\sum }_{j=\mathrm{1}}^m{t}_j^{\mathrm{2}}\right)/{\sigma }^{\mathrm{2}}-\left(\mathrm{2}{t}_{j^{\ast }}{\sum }_{j=\mathrm{1}}^m{t}_j\right)/{\sigma }^{\mathrm{2}}+t_{j^{\ast }}^{\mathrm{2}}\left(\mathrm{1}/{\delta }_{\alpha }^{\mathrm{2}}+m/{\sigma }^{\mathrm{2}}\right)}{\left(\mathrm{1}/{\delta }_{\alpha }^{\mathrm{2}}+m/{\sigma }^{\mathrm{2}}\right)\left(\mathrm{1}/{\delta }_{\beta }^{\mathrm{2}}+\left({\sum }_{j=\mathrm{1}}^m{t}_j^{\mathrm{2}}\right)/{\sigma }^{\mathrm{2}}\right)-{\left(\left({\sum }_{j=\mathrm{1}}^m{t}_j\right)/{\sigma }^{\mathrm{2}}\right)}^{\mathrm{2}}}.\end{array}$$ (A.15)

We note the following decomposition of the sum of squared times:

$$\begin{array}{c}{\displaystyle \sum }_{j=\mathrm{1}}^m{t}_j^{\mathrm{2}}{\displaystyle \sum }_{j=\mathrm{1}}^m{\left(t_j-\overline{t}+\overline{t}\right)}^{\mathrm{2}}={\displaystyle \sum }_{j=\mathrm{1}}^m{\left(t_j-\overline{t}\right)}^{\mathrm{2}}+{\displaystyle \sum }_{j=\mathrm{1}}^m{\left(\overline{t}\right)}^{\mathrm{2}}\\ =m\left(s^{\mathrm{2}}+{\overline{t}}^{\mathrm{2}}\right).\end{array}$$ (A.16)

Including this decomposition, we find that

$$\begin{array}{c}{\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}\\ =\frac{\mathrm{1}/{\delta }_{\beta }^{\mathrm{2}}+m{s}^{\mathrm{2}}/{\sigma }^{\mathrm{2}}+m{\overline{t}}^{\mathrm{2}}/{\sigma }^{\mathrm{2}}-\mathrm{2}m{t}_{j^{\ast }}\overline{t}/{\sigma }^{\mathrm{2}}+t_{j^{\ast }}^{\mathrm{2}}/{\delta }_{\alpha }^{\mathrm{2}}+m{t}_{j^{\ast }}^{\mathrm{2}}/{\sigma }^{\mathrm{2}}}{\left(\mathrm{1}/{\delta }_{\alpha }^{\mathrm{2}}+m/{\sigma }^{\mathrm{2}}\right)\left(\mathrm{1}/{\delta }_{\beta }^{\mathrm{2}}+m\left(s^{\mathrm{2}}+{\overline{t}}^{\mathrm{2}}\right)/{\sigma }^{\mathrm{2}}\right)-{\left(m\overline{t}/{\sigma }^{\mathrm{2}}\right)}^{\mathrm{2}}}\\ =\frac{\mathrm{1}/{\delta }_{\beta }^{\mathrm{2}}+m{s}^{\mathrm{2}}/{\sigma }^{\mathrm{2}}+\left(m/{\sigma }^{\mathrm{2}}\right)\left({\overline{t}}^{\mathrm{2}}-\mathrm{2}{t}_{j^{\ast }}\overline{t}+t_{j^{\ast }}^{\mathrm{2}}\right)+t_{j^{\ast }}^{\mathrm{2}}/{\delta }_{\alpha }^{\mathrm{2}}}{\left(\mathrm{1}/{\delta }_{\alpha }^{\mathrm{2}}+m/{\sigma }^{\mathrm{2}}\right)\left(\mathrm{1}/{\delta }_{\beta }^{\mathrm{2}}+m\left(s^{\mathrm{2}}+{\overline{t}}^{\mathrm{2}}\right)/{\sigma }^{\mathrm{2}}\right)-{\left(m\overline{t}/{\sigma }^{\mathrm{2}}\right)}^{\mathrm{2}}}\\ =\frac{\mathrm{1}/{\delta }_{\beta }^{\mathrm{2}}+m{s}^{\mathrm{2}}/{\sigma }^{\mathrm{2}}+m{\left(\overline{t}-t_{j^{\ast }}\right)}^{\mathrm{2}}/{\sigma }^{\mathrm{2}}+t_{j^{\ast }}^{\mathrm{2}}/{\delta }_{\alpha }^{\mathrm{2}}}{\left(\mathrm{1}/{\delta }_{\alpha }^{\mathrm{2}}+m/{\sigma }^{\mathrm{2}}\right)\left(\mathrm{1}/{\delta }_{\beta }^{\mathrm{2}}+m{s}^{\mathrm{2}}/{\sigma }^{\mathrm{2}}\right)+m{\overline{t}}^{\mathrm{2}}/{\delta }_{\alpha }^{\mathrm{2}}{\sigma }^{\mathrm{2}}}.\end{array}$$ (A.17)

B. Unconstrained Minimization of the BMSE for the Univariate, Time-Dependent Linear Model

Theorem B.1 B.1. —

For the univariate, linear, time-dependent model given by (58), ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ as given by (A.17) exhibits an absolute minimum at a point $({\overline{t}}_{\mathrm{m}\mathrm{i}\mathrm{n}},s_{\mathrm{m}\mathrm{i}\mathrm{n}}^{\mathrm{2}})$ if and only if

$$\begin{array}{c}{\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}=\frac{\mathrm{1}}{\mathrm{1}/{\delta }_{\alpha }^{\mathrm{2}}+m/{\sigma }^{\mathrm{2}}}.\end{array}$$ (B.1)

Proof —

Recall that $\overline{t}$ and s ² are defined by (63) and (64). To prevent ambiguous notation, let γ = s ². Consider the derivative of ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ with respect to γ:

$$\begin{array}{c}\frac{\partial {\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}\left(\overline{t},\gamma \right)}{\partial \gamma }\\ =-\frac{m{\sigma }^{\mathrm{2}}{\left(t_{j^{\ast }}{\sigma }^{\mathrm{2}}+m\left(-\overline{t}+t_{j^{\ast }}\right){\delta }_{\alpha }^{\mathrm{2}}\right)}^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{4}}}{{\left(m{\delta }_{\alpha }^{\mathrm{2}}\left({\sigma }^{\mathrm{2}}+m\gamma {\delta }_{\beta }^{\mathrm{2}}\right)+{\sigma }^{\mathrm{2}}\left({\sigma }^{\mathrm{2}}+m\left(\gamma +{\overline{t}}^{\mathrm{2}}\right){\delta }_{\beta }^{\mathrm{2}}\right)\right)}^{\mathrm{2}}}.\end{array}$$ (B.2)

We can represent this derivative more simply as follows:

$$\begin{array}{c}\frac{\partial {\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}\left(\overline{t},\gamma \right)}{\partial \gamma }=-\frac{m{\sigma }^{\mathrm{2}}{c}_{\mathrm{1}}^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{4}}}{c_{\mathrm{2}}^{\mathrm{2}}},\end{array}$$ (B.3)

where the values of c ₁ and c ₂ are found explicitly from (B.2). It is easy to show that c ₁ ² ≥ 0 and c ₂ ² > 0, and consequently

$$\begin{array}{c}\frac{\partial {\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}\left(\overline{t},\gamma \right)}{\partial \gamma }\le \mathrm{0}.\end{array}$$ (B.4)

Therefore, for a fixed $\overline{t}$, ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ either decreases or remains constant as we increase γ. It follows that, for fixed value of $\overline{t}$, ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ is bounded below by ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ in the limit as γ → ∞:

$$\begin{array}{c}\underset{\gamma \to \infty }{\lim }{\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}\left(\overline{t},\gamma \right)=\frac{\mathrm{1}}{\mathrm{1}/{\delta }_{\alpha }^{\mathrm{2}}+m/{\sigma }^{\mathrm{2}}}.\end{array}$$ (B.5)

Since the limit of ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ does not depend on the values of $\overline{t}$ and γ, any values of $\overline{t}$ and γ for which ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ takes on the value given by (B.5) are an absolute minimum of ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$.

Theorem B.2 B.2. —

For the univariate, linear, time-dependent model given by (58), an absolute minimum of ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ as given by (A.17), inside the region defined by 0 ≤ t _j ≤ T, can be obtained by collecting data at times t _j such that

$$\begin{array}{c}\overline{t}=\frac{t_{j^{\ast }}\left({\sigma }^{\mathrm{2}}/m+{\delta }_{\alpha }^{\mathrm{2}}\right)}{{\delta }_{\alpha }^{\mathrm{2}}},\end{array}$$ (B.6)

if and only if

$$\begin{array}{c}\overline{t}\le T.\end{array}$$ (B.7)

Proof —

The critical points of the unconstrained function are found by taking the derivative of M _fj^∗ with respect to $\overline{t}$ and s ² and setting these derivatives to zero. The derivative with respect to $\overline{t}$ is as follows:

$$\begin{array}{c}\frac{\partial {\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}\left(\overline{t},s^{\mathrm{2}}\right)}{\partial \overline{t}}=-\frac{\mathrm{2}m{\sigma }^{\mathrm{2}}\left(t_{j^{\ast }}{\sigma }^{\mathrm{2}}+m\left(-\overline{t}+t_{j^{\ast }}\right){\delta }_{\alpha }^{\mathrm{2}}\right){\delta }_{\beta }^{\mathrm{2}}\left(\overline{t}{t}_{j^{\ast }}{\sigma }^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}+{\delta }_{\alpha }^{\mathrm{2}}\left({\sigma }^{\mathrm{2}}+m{s}^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}\right)\right)}{{\left(m{\delta }_{\alpha }^{\mathrm{2}}\left({\sigma }^{\mathrm{2}}+m{s}^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}\right)+{\sigma }^{\mathrm{2}}\left({\sigma }^{\mathrm{2}}+m\left(s^{\mathrm{2}}+{\overline{t}}^{\mathrm{2}}\right){\delta }_{\beta }^{\mathrm{2}}\right)\right)}^{\mathrm{2}}}.\end{array}$$ (B.8)

The derivative with respect to s ² is given by (B.2). Setting both derivatives equal to zero, we find the following critical point:

$$\begin{array}{c}{\overline{t}}_{\mathrm{m}\mathrm{i}\mathrm{n}}=\frac{t_{j^{\ast }}\left({\sigma }^{\mathrm{2}}/m+{\delta }_{\alpha }^{\mathrm{2}}\right)}{{\delta }_{\alpha }^{\mathrm{2}}}.\end{array}$$ (B.9)

Evaluating the BMSE at this critical point, we find that

$$\begin{array}{c}{\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}\left({\overline{t}}_{\mathrm{m}\mathrm{i}\mathrm{n}},s^{\mathrm{2}}\right)=\frac{\mathrm{1}}{\mathrm{1}/{\delta }_{\alpha }^{\mathrm{2}}+m/{\sigma }^{\mathrm{2}}},\end{array}$$ (B.10)

which is sufficient to show that ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ exhibits an absolute minimum at this point.

Given the constraint that 0 ≤ t _j ≤ T for each j, ${\overline{t}}_{\mathrm{m}\mathrm{i}\mathrm{n}}$ is a feasible point only when $\mathrm{0}\le {\overline{t}}_{\mathrm{m}\mathrm{i}\mathrm{n}}\le T$. It is easy to see that ${\overline{t}}_{\mathrm{m}\mathrm{i}\mathrm{n}}>\mathrm{0}$ in all cases. Therefore, the absolute minimum can be achieved by collecting all data at ${\overline{t}}_{\mathrm{m}\mathrm{i}\mathrm{n}}$ if and only if ${\overline{t}}_{\mathrm{m}\mathrm{i}\mathrm{n}}\le T$.

C. Constrained Minimization of the BMSE for the Univariate, Time-Dependent Linear Model with Two Time Points

Theorem C.1 C.1. —

For the univariate, linear, time-dependent model given by (58) with m = 2 and 0 ≤ t _j ≤ T, if

$$\begin{array}{c}\frac{t_{j^{\ast }}\left({\sigma }^{\mathrm{2}}/\mathrm{2}+{\delta }_{\alpha }^{\mathrm{2}}\right)}{{\delta }_{\alpha }^{\mathrm{2}}}>T,\end{array}$$ (C.1)

then ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ as given by (A.17) is minimized by collecting data at times such that t ₁ = t ₂ = T.

Proof —

Substituting m = 2 into (A.15) we find that

$$\begin{array}{c}{\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}=\frac{\left(t_{\mathrm{1}}^{\mathrm{2}}+t_{\mathrm{2}}^{\mathrm{2}}\right)/{\sigma }^{\mathrm{2}}-\left(\mathrm{2}\left(t_{\mathrm{1}}+t_{\mathrm{2}}\right)t_{j^{\ast }}\right)/{\sigma }^{\mathrm{2}}+t_{j^{\ast }}^{\mathrm{2}}\left(\left(\mathrm{2}/{\sigma }^{\mathrm{2}}\right)+\left(\mathrm{1}/{\delta }_{\alpha }^{\mathrm{2}}\right)\right)+\left(\mathrm{1}/{\delta }_{\beta }^{\mathrm{2}}\right)}{-{\left(t_{\mathrm{1}}+t_{\mathrm{2}}\right)}^{\mathrm{2}}/{\sigma }^{\mathrm{4}}+\left(\left(\mathrm{2}/{\sigma }^{\mathrm{2}}\right)+\left(\mathrm{1}/{\delta }_{\alpha }^{\mathrm{2}}\right)\right)\left(\left(t_{\mathrm{1}}^{\mathrm{2}}+t_{\mathrm{2}}^{\mathrm{2}}\right)/{\sigma }^{\mathrm{2}}+\left(\mathrm{1}/{\delta }_{\beta }^{\mathrm{2}}\right)\right)}.\end{array}$$ (C.2)

From Theorem B.2 we know that there are no critical points inside the feasible region, and the solution must therefore exist on the boundary of the region, which is defined by the following line segments:

$$\begin{array}{c}A\text{:\hspace{0.17em}\hspace{0.17em}}{t}_{\mathrm{1}}\in \left[\mathrm{0},T\right],t_{\mathrm{2}}=\mathrm{0};\\ \\ B\text{:\hspace{0.17em}\hspace{0.17em}}{t}_{\mathrm{1}}=\mathrm{0},t_{\mathrm{2}}\in \left[\mathrm{0},T\right];\\ \\ C\text{:\hspace{0.17em}\hspace{0.17em}}{t}_{\mathrm{1}}\in \left[\mathrm{0},T\right],t_{\mathrm{2}}=T;\\ \\ D\text{:\hspace{0.17em}\hspace{0.17em}}{t}_{\mathrm{1}}=T,t_{\mathrm{2}}\in \left[\mathrm{0},T\right].\\ \end{array}$$ (C.3)

Note that ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ is symmetric in t _j:

$$\begin{array}{c}{\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}\left(t_{\mathrm{1}}=a,\hspace{0.17em}\hspace{0.17em}{t}_{\mathrm{2}}=b\right)={\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}\left(t_{\mathrm{1}}=b,\hspace{0.17em}\hspace{0.17em}{t}_{\mathrm{2}}=a\right).\end{array}$$ (C.4)

Therefore, it is sufficient to consider only line segments A and C. We first consider finding the minimum value on line segment A. Setting t ₂ = 0 and then taking the derivative of ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ with respect to t ₁, and solving for t ₁, we find the following two critical points:

$$\begin{array}{c}{t}_{\mathrm{1}\mathrm{m}\mathrm{i}\mathrm{n}}^{A+}=\frac{t_{j^{\ast }}\left({\sigma }^{\mathrm{2}}+\mathrm{2}{\delta }_{\alpha }^{\mathrm{2}}\right)}{{\delta }_{\alpha }^{\mathrm{2}}},\\ \end{array}$$ (C.5)

$$\begin{array}{c}{t}_{\mathrm{1}\mathrm{m}\mathrm{i}\mathrm{n}}^{A-}=-\frac{{\sigma }^{\mathrm{2}}{\delta }_{\alpha }^{\mathrm{2}}}{t_{j^{\ast }}\left({\sigma }^{\mathrm{2}}+{\delta }_{\alpha }^{\mathrm{2}}\right){\delta }_{\beta }^{\mathrm{2}}}.\\ \end{array}$$ (C.6)

Applying condition (C.1) to t _1min ^A+, we find that t _1min ^A+ > 2T. Taking the second derivative of ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ with respect to t ₁, such that t ₂ = 0, and evaluating it at t _1min ^A+ result in

$$\begin{array}{c}\frac{{\partial }^{\mathrm{2}}{\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}}{\partial t_{\mathrm{1}}^{\mathrm{2}}}\\ =\frac{\mathrm{2}{\sigma }^{\mathrm{2}}{\delta }_{\alpha }^{\mathrm{8}}{\delta }_{\beta }^{\mathrm{2}}}{{\left({\sigma }^{\mathrm{2}}+\mathrm{2}{\delta }_{\alpha }^{\mathrm{2}}\right)}^{\mathrm{2}}\left({\sigma }^{\mathrm{4}}{t}_{j^{\ast }}^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}+\mathrm{3}{\sigma }^{\mathrm{2}}{t}_{j^{\ast }}^{\mathrm{2}}{\delta }_{\alpha }^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}+{\delta }_{\alpha }^{\mathrm{4}}\left({\sigma }^{\mathrm{2}}+\mathrm{2}{t}_{j^{\ast }}^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}\right)\right)},\end{array}$$ (C.7)

which is positive. Therefore (C.5) represents a minimum.

It is easy to see that t _1min ^A− is always negative. Taking the second derivative of ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ with respect to t ₁, such that t ₂ = 0, and evaluating it at t _1min ^A− (C.6) result in

$$\begin{array}{c}\frac{{\partial }^{\mathrm{2}}{\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}}{\partial t_{\mathrm{1}}^{\mathrm{2}}}\\ =-\frac{\mathrm{2}{t}_n^{\mathrm{4}}{\left({\sigma }^{\mathrm{2}}+{\delta }_{\alpha }^{\mathrm{2}}\right)}^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{6}}}{{\sigma }^{\mathrm{6}}{t}_n^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}+\mathrm{3}{\sigma }^{\mathrm{4}}{t}_n^{\mathrm{2}}{\delta }_{\alpha }^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}+{\delta }_{\alpha }^{\mathrm{4}}\left({\sigma }^{\mathrm{4}}+\mathrm{2}{\sigma }^{\mathrm{2}}{t}_n^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}\right)},\end{array}$$ (C.8)

which is negative. Therefore t _1min ^A− represents a maximum.

Since the maximum occurs at a value t _1min ^A− < 0, the minimum occurs at a value t _1min ^A+ > T, and these are the only two critical points; the BMSE must be decreasing over the line segment, and the minimum BMSE on the line segment therefore occurs at t ₁ = T, t ₂ = 0.

We next consider finding the minimum value on the line segment C. Setting t ₂ = T and then taking the derivative of ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ with respect to t ₁, and solving for t ₁, we find the following two critical points:

$$\begin{array}{c}{t}_{\mathrm{1}\mathrm{m}\mathrm{i}\mathrm{n}}^{C+}=\frac{{\sigma }^{\mathrm{2}}{t}_{j^{\ast }}-T{\delta }_{\alpha }^{\mathrm{2}}+\mathrm{2}{t}_{j^{\ast }}{\delta }_{\alpha }^{\mathrm{2}}}{{\delta }_{\alpha }^{\mathrm{2}}},\\ \\ t_{\mathrm{1}\mathrm{m}\mathrm{i}\mathrm{n}}^{C-}=\frac{{\delta }_{\alpha }^{\mathrm{2}}\left({\sigma }^{\mathrm{2}}+T^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}-T{t}_{j^{\ast }}{\delta }_{\beta }^{\mathrm{2}}\right)}{\left(-{\sigma }^{\mathrm{2}}{t}_{j^{\ast }}+T{\delta }_{\alpha }^{\mathrm{2}}-t_{j^{\ast }}{\delta }_{\alpha }^{\mathrm{2}}\right){\delta }_{\beta }^{\mathrm{2}}}.\\ \end{array}$$ (C.9)

We see that t _1min ^C+ is minimized by applying the lower bound for t _j^∗ given by (C.1), and we find that t _1min ^C+ > T. Concerning t _1min ^C−, the numerator can be minimized by applying the upper bound for t _j^∗ of T. In this case, the numerator is δ _α ² σ ², which is always positive. The maximum value of the denominator is found by substituting in the minimum value for t _j^∗ given by (C.1). We find that the maximum value is −Tσ ² δ _α ² δ _β ²/(σ ² + 2δ _α ²), which is negative. Therefore t _1min ^C− is always negative.

Taking the second derivative of ${\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}$ with respect to t ₁, such that t ₂ = T, and evaluating it at t _1min ^C+ result in

$$\begin{array}{c}\frac{{\partial }^{\mathrm{2}}{\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}}{\partial t_{\mathrm{1}}^{\mathrm{2}}}=\frac{\mathrm{2}{\sigma }^{\mathrm{2}}{\delta }_{\alpha }^{\mathrm{8}}{\delta }_{\beta }^{\mathrm{2}}}{{\left({\sigma }^{\mathrm{2}}+\mathrm{2}{\delta }_{\alpha }^{\mathrm{2}}\right)}^{\mathrm{2}}\left({\sigma }^{\mathrm{4}}{t}_{j^{\ast }}^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}+{\sigma }^{\mathrm{2}}{t}_{j^{\ast }}\left(-\mathrm{2}{t}_f+\mathrm{3}{t}_{j^{\ast }}\right){\delta }_{\alpha }^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}+{\delta }_{\alpha }^{\mathrm{4}}\left({\sigma }^{\mathrm{2}}+\mathrm{2}{t}_f^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}-\mathrm{4}{t}_f{t}_{j^{\ast }}{\delta }_{\beta }^{\mathrm{2}}+\mathrm{2}{t}_{j^{\ast }}^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}\right)\right)}.\end{array}$$ (C.10)

Evaluating (C.10) at t _j^∗ = T results in

$$\begin{array}{c}\frac{{\partial }^{\mathrm{2}}{\mathbf{M}}_{{\widehat{f}}_{j^{\ast }}}}{\partial t_{\mathrm{1}}^{\mathrm{2}}}=\frac{\mathrm{2}{\delta }_{\alpha }^{\mathrm{8}}{\delta }_{\beta }^{\mathrm{2}}}{{\left({\sigma }^{\mathrm{2}}+\mathrm{2}{\delta }_{\alpha }^{\mathrm{2}}\right)}^{\mathrm{2}}\left({\delta }_{\alpha }^{\mathrm{4}}+{\sigma }^{\mathrm{2}}{t}_f^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}+t_f^{\mathrm{2}}{\delta }_{\alpha }^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}\right)},\end{array}$$ (C.11)

which is positive. In general, (C.10) is positive when

$$\begin{array}{c}{\sigma }^{\mathrm{4}}{t}_{j^{\ast }}^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}+{\sigma }^{\mathrm{2}}{t}_{j^{\ast }}\left(-\mathrm{2}{t}_f+\mathrm{3}{t}_{j^{\ast }}\right){\delta }_{\alpha }^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}\\ +{\delta }_{\alpha }^{\mathrm{4}}\left({\sigma }^{\mathrm{2}}+\mathrm{2}{t}_f^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}-\mathrm{4}{t}_f{t}_{j^{\ast }}{\delta }_{\beta }^{\mathrm{2}}+\mathrm{2}{t}_{j^{\ast }}^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}\right)>\mathrm{0}.\end{array}$$ (C.12)

Setting the expression on the left-hand side to zero and solving for t _j^∗, we find the following roots:

$$\begin{array}{c}{t}_{j^{\ast }}=\frac{{\sigma }^{\mathrm{2}}{t}_f{\delta }_{\alpha }^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}+\mathrm{2}{t}_f{\delta }_{\alpha }^{\mathrm{4}}{\delta }_{\beta }^{\mathrm{2}}\pm \sqrt{-{\sigma }^{\mathrm{6}}{\delta }_{\alpha }^{\mathrm{4}}{\delta }_{\beta }^{\mathrm{2}}-\mathrm{3}{\sigma }^{\mathrm{4}}{\delta }_{\alpha }^{\mathrm{6}}{\delta }_{\beta }^{\mathrm{2}}-\mathrm{2}{\sigma }^{\mathrm{2}}{\delta }_{\alpha }^{\mathrm{8}}{\delta }_{\beta }^{\mathrm{2}}-{\sigma }^{\mathrm{4}}{t}_f^{\mathrm{2}}{\delta }_{\alpha }^{\mathrm{4}}{\delta }_{\beta }^{\mathrm{4}}-\mathrm{2}{\sigma }^{\mathrm{2}}{t}_f^{\mathrm{2}}{\delta }_{\alpha }^{\mathrm{6}}{\delta }_{\beta }^{\mathrm{4}}}}{{\sigma }^{\mathrm{4}}{\delta }_{\beta }^{\mathrm{2}}+\mathrm{3}{\sigma }^{\mathrm{2}}{\delta }_{\alpha }^{\mathrm{2}}{\delta }_{\beta }^{\mathrm{2}}+\mathrm{2}{\delta }_{\alpha }^{\mathrm{4}}{\delta }_{\beta }^{\mathrm{2}}}.\end{array}$$ (C.13)

These roots are imaginary, which implies that (C.10) is always positive, and therefore, t _1min ^C+ is a minimum. Given that the BMSE must always be positive, and there are only two critical points, this implies that t _1min ^C− is a maximum. Since the maximum occurs at a value of t _1min ^C− < 0 and the minimum occurs at a value of t _1min ^C+ > T, the minimum on the line segment must occur at the value t ₁ = T.

In summary, the minimum value on the line segment A occurs at t ₁ = T, t ₂ = 0. Applying symmetry, the BMSE at this point is the same as the BMSE at the points t ₁ = 0, t ₂ = T, which is the maximum point for the line segment C. Therefore, the minimum BMSE does not occur on the line segment A. Again, by symmetry, this means that the minimum BMSE does not occur on the line segment B. For the line segment C, the minimum BMSE occurs at t ₁ = T, t ₂ = T. Applying symmetry, the minimum BMSE on the line segment D occurs at t ₁ = T, t ₂ = T. Since these two points are the same, we find that the overall minimum BMSE within the feasible region occurs at the point t ₁ = t ₂ = T.

D. Unconstrained Minimization of the BMSE for the Bivariate, Time-Dependent Linear Model

Theorem D.1 D.1. —

For the bivariate, linear, time-dependent model given by (70), ${\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}$ as given by (74) exhibits an absolute minimum at a point $({\overline{t}}_{\mathrm{1}\mathrm{m}\mathrm{i}\mathrm{n}},{\overline{t}}_{\mathrm{2}\mathrm{m}\mathrm{i}\mathrm{n}},s_{\mathrm{1}\mathrm{m}\mathrm{i}\mathrm{n}}^{\mathrm{2}},s_{\mathrm{2}\mathrm{m}\mathrm{i}\mathrm{n}}^{\mathrm{2}})$ if and only if

$$\begin{array}{c}{\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}\\ =\frac{{\delta }_{{\alpha }_{\mathrm{1}}}^{\mathrm{2}}\left({\sigma }_{\mathrm{1}}^{\mathrm{2}}/m_{\mathrm{1}}\right)\left(\left(\mathrm{1}-{\rho }^{\mathrm{2}}\right){\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}/m_{\mathrm{2}}\right)}{\left({\sigma }_{\mathrm{1}}^{\mathrm{2}}/m_{\mathrm{1}}\right)\left({\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}/m_{\mathrm{2}}\right)+a_{\mathrm{1}}^{\mathrm{2}}\left(\left(\mathrm{1}-{\rho }^{\mathrm{2}}\right){\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}/m_{\mathrm{2}}\right)}.\end{array}$$ (D.1)

Proof —

Recall that ${\overline{t}}_{\mathrm{1}}$, ${\overline{t}}_{\mathrm{2}}$, s ₁ ², and s ₂ ² are defined in (77), (78), (79), and (80). To prevent ambiguous notation, let γ ₁ = s ₁ ² and γ ₂ = s ₂ ². Consider the derivative of ${\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}$ (see (74)) with respect to γ ₁:

$$\begin{array}{c}\frac{\partial {\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}\left({\overline{t}}_{\mathrm{1}},{\overline{t}}_{\mathrm{2}},{\gamma }_{\mathrm{1}},{\gamma }_{\mathrm{2}}\right)}{\partial {\gamma }_{\mathrm{1}}}=-\frac{{\delta }_{{\beta }_{\mathrm{1}}}^{\mathrm{2}}{m}_{\mathrm{1}}{\sigma }_{\mathrm{1}}^{\mathrm{2}}{c}_{\mathrm{1}}^{\mathrm{2}}}{c_{\mathrm{2}}^{\mathrm{2}}},\end{array}$$ (D.2)

where

$$\begin{array}{c}{c}_{\mathrm{1}}={\overline{t}}_{\mathrm{2}}\rho \omega {\delta }_{{\alpha }_{\mathrm{1}}}{\delta }_{{\alpha }_{\mathrm{2}}}{\delta }_{{\beta }_{\mathrm{2}}}{m}_{\mathrm{2}}{\sigma }_{\mathrm{1}}^{\mathrm{2}}{\sigma }_{\mathrm{2}}^{\mathrm{2}}\\ +t_{j^{\ast }}{\delta }_{{\beta }_{\mathrm{1}}}{\sigma }_{\mathrm{1}}^{\mathrm{2}}\left({\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}\left(s_{\mathrm{2}}^{\mathrm{2}}\left(-\mathrm{1}+{\omega }^{\mathrm{2}}\right){\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}-{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)-{\sigma }_{\mathrm{2}}^{\mathrm{2}}\left(-\left(s_{\mathrm{2}}^{\mathrm{2}}+{\overline{t}}_{\mathrm{2}}^{\mathrm{2}}\right)\left(-\mathrm{1}+{\omega }^{\mathrm{2}}\right){\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)\right)+\left({\overline{t}}_{\mathrm{1}}-t_{j^{\ast }}\right){\delta }_{{\alpha }_{\mathrm{1}}}^{\mathrm{2}}{\delta }_{{\beta }_{\mathrm{1}}}{m}_{\mathrm{1}}\left(\left(-\mathrm{1}+{\rho }^{\mathrm{2}}\right){\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}\left(s_{\mathrm{2}}^{\mathrm{2}}\left(-\mathrm{1}+{\omega }^{\mathrm{2}}\right)·{\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}-{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)+{\sigma }_{\mathrm{2}}^{\mathrm{2}}\left(-\left(s_{\mathrm{2}}^{\mathrm{2}}+{\overline{t}}_{\mathrm{2}}^{\mathrm{2}}\right)\left(-\mathrm{1}+{\omega }^{\mathrm{2}}\right){\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)\right),\\ c_{\mathrm{2}}=-\mathrm{2}{\overline{t}}_{\mathrm{1}}{\overline{t}}_{\mathrm{2}}\rho \omega {\delta }_{{\alpha }_{\mathrm{1}}}{\delta }_{{\alpha }_{\mathrm{2}}}{\delta }_{{\beta }_{\mathrm{1}}}{\delta }_{{\beta }_{\mathrm{2}}}{m}_{\mathrm{1}}{m}_{\mathrm{2}}{\sigma }_{\mathrm{1}}^{\mathrm{2}}{\sigma }_{\mathrm{2}}^{\mathrm{2}}\\ +{\sigma }_{\mathrm{1}}^{\mathrm{2}}\left({\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}\left(-\left(s_{\mathrm{1}}^{\mathrm{2}}+{\overline{t}}_{\mathrm{1}}^{\mathrm{2}}\right)·{\delta }_{{\beta }_{\mathrm{1}}}^{\mathrm{2}}{m}_{\mathrm{1}}\left(s_{\mathrm{2}}^{\mathrm{2}}\left(-\mathrm{1}+{\omega }^{\mathrm{2}}\right){\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}-{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)+{\sigma }_{\mathrm{1}}^{\mathrm{2}}\left(s_{\mathrm{2}}^{\mathrm{2}}{\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)\right)+{\sigma }_{\mathrm{2}}^{\mathrm{2}}\left(-\left(s_{\mathrm{1}}^{\mathrm{2}}+{\overline{t}}_{\mathrm{1}}^{\mathrm{2}}\right)·{\delta }_{{\beta }_{\mathrm{1}}}^{\mathrm{2}}{m}_{\mathrm{1}}\left(\left(s_{\mathrm{2}}^{\mathrm{2}}+{\overline{t}}_{\mathrm{2}}^{\mathrm{2}}\right)\left(-\mathrm{1}+{\omega }^{\mathrm{2}}\right){\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}-{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)+{\sigma }_{\mathrm{1}}^{\mathrm{2}}\left(\left(s_{\mathrm{2}}^{\mathrm{2}}+{\overline{t}}_{\mathrm{2}}^{\mathrm{2}}\right){\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)\right)\right)+a_{\mathrm{1}}^{\mathrm{2}}{m}_{\mathrm{1}}\left(\left(-\mathrm{1}+{\rho }^{\mathrm{2}}\right){\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}\left(s_{\mathrm{1}}^{\mathrm{2}}{\delta }_{{\beta }_{\mathrm{1}}}^{\mathrm{2}}{m}_{\mathrm{1}}\left(s_{\mathrm{2}}^{\mathrm{2}}\left(-\mathrm{1}+{\omega }^{\mathrm{2}}\right){\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}-{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)-{\sigma }_{\mathrm{1}}^{\mathrm{2}}\left(s_{\mathrm{2}}^{\mathrm{2}}{\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)\right)+{\sigma }_{\mathrm{2}}^{\mathrm{2}}\left({\sigma }_{\mathrm{1}}^{\mathrm{2}}\left(\left(s_{\mathrm{2}}^{\mathrm{2}}+{\overline{t}}_{\mathrm{2}}^{\mathrm{2}}\right){\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)+s_{\mathrm{1}}^{\mathrm{2}}{\delta }_{{\beta }_{\mathrm{1}}}^{\mathrm{2}}{m}_{\mathrm{1}}\left(-\left(s_{\mathrm{2}}^{\mathrm{2}}+{\overline{t}}_{\mathrm{2}}^{\mathrm{2}}\right)\left(-\mathrm{1}+{\omega }^{\mathrm{2}}\right){\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)\right)\right).\end{array}$$ (D.3)

It is easy to show that c ₁ ² ≥ 0 and c ₂ ² ≥ 0, and consequently

$$\begin{array}{c}\frac{\partial {\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}\left({\overline{t}}_{\mathrm{1}},{\overline{t}}_{\mathrm{2}},{\gamma }_{\mathrm{1}},{\gamma }_{\mathrm{2}}\right)}{\partial {\gamma }_{\mathrm{1}}}\le \mathrm{0}.\end{array}$$ (D.4)

Consider also the derivative of ${\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}$ with respect to γ ₂:

$$\begin{array}{c}\frac{{\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}\left({\overline{t}}_{\mathrm{1}},{\overline{t}}_{\mathrm{2}},{\gamma }_{\mathrm{1}},{\gamma }_{\mathrm{2}}\right)}{\partial {\gamma }_{\mathrm{2}}}=-\frac{{\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}{\sigma }_{\mathrm{1}}^{\mathrm{4}}{\sigma }_{\mathrm{2}}^{\mathrm{2}}{c}_{\mathrm{3}}^{\mathrm{2}}}{c_{\mathrm{4}}^{\mathrm{2}}},\end{array}$$ (D.5)

where

$$\begin{array}{c}{c}_{\mathrm{3}}={\overline{t}}_{\mathrm{2}}\rho {\delta }_{{\alpha }_{\mathrm{1}}}{\delta }_{{\alpha }_{\mathrm{2}}}{\delta }_{{\beta }_{\mathrm{2}}}{m}_{\mathrm{2}}\left(\left(s_{\mathrm{1}}^{\mathrm{2}}+{\overline{t}}_{\mathrm{1}}\left({\overline{t}}_{\mathrm{1}}-t_{j^{\ast }}\right)\right)\left(-\mathrm{1}+{\omega }^{\mathrm{2}}\right)·{\delta }_{{\beta }_{\mathrm{1}}}^{\mathrm{2}}{m}_{\mathrm{1}}-{\sigma }_{\mathrm{1}}^{\mathrm{2}}\right)+\left({\overline{t}}_{\mathrm{1}}-t_{j^{\ast }}\right)\omega {\delta }_{{\alpha }_{\mathrm{1}}}^{\mathrm{2}}{\delta }_{{\beta }_{\mathrm{1}}}{m}_{\mathrm{1}}\left(\left(-\mathrm{1}+{\rho }^{\mathrm{2}}\right)·{\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}-{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)+t_{j^{\ast }}\omega {\delta }_{{\beta }_{\mathrm{1}}}{\sigma }_{\mathrm{1}}^{\mathrm{2}}\left({\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right),\\ c_{\mathrm{4}}=-\mathrm{2}{\overline{t}}_{\mathrm{1}}{\overline{t}}_{\mathrm{2}}\rho \omega {\delta }_{{\alpha }_{\mathrm{1}}}{\delta }_{{\alpha }_{\mathrm{2}}}{\delta }_{{\beta }_{\mathrm{1}}}{\delta }_{{\beta }_{\mathrm{2}}}{m}_{\mathrm{1}}{m}_{\mathrm{2}}{\sigma }_{\mathrm{1}}^{\mathrm{2}}{\sigma }_{\mathrm{2}}^{\mathrm{2}}\\ +{\sigma }_{\mathrm{1}}^{\mathrm{2}}\left({\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}\left(-\left(s_{\mathrm{1}}^{\mathrm{2}}+{\overline{t}}_{\mathrm{1}}^{\mathrm{2}}\right)·{\delta }_{{\beta }_{\mathrm{1}}}^{\mathrm{2}}{m}_{\mathrm{1}}\left(s_{\mathrm{2}}^{\mathrm{2}}\left(-\mathrm{1}+{\omega }^{\mathrm{2}}\right){\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}-{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)+{\sigma }_{\mathrm{1}}^{\mathrm{2}}\left(s_{\mathrm{2}}^{\mathrm{2}}{\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)\right)+{\sigma }_{\mathrm{2}}^{\mathrm{2}}\left(-\left(s_{\mathrm{1}}^{\mathrm{2}}+{\overline{t}}_{\mathrm{1}}^{\mathrm{2}}\right)·{\delta }_{{\beta }_{\mathrm{1}}}^{\mathrm{2}}{m}_{\mathrm{1}}\left(\left(s_{\mathrm{2}}^{\mathrm{2}}+{\overline{t}}_{\mathrm{2}}^{\mathrm{2}}\right)\left(-\mathrm{1}+{\omega }^{\mathrm{2}}\right){\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}-{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)+{\sigma }_{\mathrm{1}}^{\mathrm{2}}\left(\left(s_{\mathrm{2}}^{\mathrm{2}}+{\overline{t}}_{\mathrm{2}}^{\mathrm{2}}\right){\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)\right)\right)+a_{\mathrm{1}}^{\mathrm{2}}{m}_{\mathrm{1}}\left(\left(-\mathrm{1}+{\rho }^{\mathrm{2}}\right){\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}\left(s_{\mathrm{1}}^{\mathrm{2}}{\delta }_{{\beta }_{\mathrm{1}}}^{\mathrm{2}}{m}_{\mathrm{1}}\left(s_{\mathrm{2}}^{\mathrm{2}}\left(-\mathrm{1}+{\omega }^{\mathrm{2}}\right){\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}-{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)-{\sigma }_{\mathrm{1}}^{\mathrm{2}}\left(s_{\mathrm{2}}^{\mathrm{2}}{\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)\right)+{\sigma }_{\mathrm{2}}^{\mathrm{2}}\left({\sigma }_{\mathrm{1}}^{\mathrm{2}}\left(\left(s_{\mathrm{2}}^{\mathrm{2}}+{\overline{t}}_{\mathrm{2}}^{\mathrm{2}}\right){\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)+s_{\mathrm{1}}^{\mathrm{2}}{\delta }_{{\beta }_{\mathrm{1}}}^{\mathrm{2}}{m}_{\mathrm{1}}\left(-\left(s_{\mathrm{2}}^{\mathrm{2}}+{\overline{t}}_{\mathrm{2}}^{\mathrm{2}}\right)\left(-\mathrm{1}+{\omega }^{\mathrm{2}}\right){\delta }_{{\beta }_{\mathrm{2}}}^{\mathrm{2}}{m}_{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}\right)\right)\right).\end{array}$$ (D.6)

It can be shown that c ₃ ² ≥ 0 and c ₄ ² > 0, and consequently

$$\begin{array}{c}\frac{\partial {\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}\left({\overline{t}}_{\mathrm{1}},{\overline{t}}_{\mathrm{2}},{\gamma }_{\mathrm{1}},{\gamma }_{\mathrm{2}}\right)}{\partial {\gamma }_{\mathrm{2}}}\le \mathrm{0}.\end{array}$$ (D.7)

Therefore, for fixed ${\overline{t}}_{\mathrm{1}}$ and ${\overline{t}}_{\mathrm{2}}$, ${\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}$ either decreases or remains constant as we increase γ ₁ or γ ₂. It follows that, for fixed values of ${\overline{t}}_{\mathrm{1}}$ and ${\overline{t}}_{\mathrm{2}}$, ${\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}$ is bounded below by ${\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}$ in the limit as γ ₁, γ ₂ → ∞:

$$\begin{array}{c}\underset{{\gamma }_{\mathrm{1}},{\gamma }_{\mathrm{2}}\to \infty }{\lim }{\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}\left({\overline{t}}_{\mathrm{1}},{\overline{t}}_{\mathrm{2}},{\gamma }_{\mathrm{1}},{\gamma }_{\mathrm{2}}\right)\\ =\frac{{\delta }_{{\alpha }_{\mathrm{1}}}^{\mathrm{2}}\left({\sigma }_{\mathrm{1}}^{\mathrm{2}}/m_{\mathrm{1}}\right)\left(\left(\mathrm{1}-{\rho }^{\mathrm{2}}\right){\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}/m_{\mathrm{2}}\right)}{\left({\sigma }_{\mathrm{1}}^{\mathrm{2}}/m_{\mathrm{1}}\right)\left({\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}/m_{\mathrm{2}}\right)+a_{\mathrm{1}}^{\mathrm{2}}\left(\left(\mathrm{1}-{\rho }^{\mathrm{2}}\right){\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}/m_{\mathrm{2}}\right)}.\end{array}$$ (D.8)

Since the limit of ${\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}$ does not depend on ${\overline{t}}_{\mathrm{1}}$ or ${\overline{t}}_{\mathrm{2}}$, any values of ${\overline{t}}_{\mathrm{1}},{\overline{t}}_{\mathrm{2}},{\gamma }_{\mathrm{1}}$, and γ ₂ for which ${\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}$ takes on the value given by (D.8) are an absolute minimum of ${\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}$.

Theorem D.2 D.2. —

For the bivariate, linear, time-dependent model given by (70), an absolute minimum of ${\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}$ as given by (74), inside the region defined by 0 ≤ t _rj ≤ T, can be obtained by collecting data at times t _rj such that

$$\begin{array}{c}{\overline{t}}_{\mathrm{1}\mathrm{m}\mathrm{i}\mathrm{n}}=\frac{t_{\mathrm{1}{j}^{\ast }}\left({\delta }_{{\alpha }_{\mathrm{1}}}^{\mathrm{2}}\left(\left(\mathrm{1}-{\rho }^{\mathrm{2}}\right){\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}+\left({\sigma }_{\mathrm{2}}^{\mathrm{2}}/m_{\mathrm{2}}\right)\right)+\left({\sigma }_{\mathrm{1}}^{\mathrm{2}}/m_{\mathrm{1}}\right)\left({\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}+\left({\sigma }_{\mathrm{2}}^{\mathrm{2}}/m_{\mathrm{2}}\right)\right)\right)}{{\delta }_{{\alpha }_{\mathrm{1}}}^{\mathrm{2}}\left(\left(\mathrm{1}-{\rho }^{\mathrm{2}}\right){\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}+\left({\sigma }_{\mathrm{2}}^{\mathrm{2}}/m_{\mathrm{2}}\right)\right)},\\ \\ {\overline{t}}_{\mathrm{2}\mathrm{m}\mathrm{i}\mathrm{n}}=\mathrm{0},\\ \end{array}$$ (D.9)

if and only if

$$\begin{array}{c}{\overline{t}}_{\mathrm{1}\mathrm{m}\mathrm{i}\mathrm{n}}\le T.\end{array}$$ (D.10)

Proof —

Substituting (D.9) into ${\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}$, we find that

$$\begin{array}{c}{\mathbf{M}}_{{\widehat{f}}_{\mathrm{1}{j}^{\ast }}}\left({\overline{t}}_{\mathrm{1}\mathrm{m}\mathrm{i}\mathrm{n}},{\overline{t}}_{\mathrm{2}\mathrm{m}\mathrm{i}\mathrm{n}},s_{\mathrm{1}}^{\mathrm{2}},s_{\mathrm{2}}^{\mathrm{2}}\right)\\ =\frac{{\delta }_{{\alpha }_{\mathrm{1}}}^{\mathrm{2}}\left({\sigma }_{\mathrm{1}}^{\mathrm{2}}/m_{\mathrm{1}}\right)\left(\left(\mathrm{1}-{\rho }^{\mathrm{2}}\right){\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}+\left({\sigma }_{\mathrm{2}}^{\mathrm{2}}/m_{\mathrm{2}}\right)\right)}{\left({\sigma }_{\mathrm{1}}^{\mathrm{2}}/m_{\mathrm{1}}\right)\left({\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}/m_{\mathrm{2}}\right)+a_{\mathrm{1}}^{\mathrm{2}}\left(\left(\mathrm{1}-{\rho }^{\mathrm{2}}\right){\delta }_{{\alpha }_{\mathrm{2}}}^{\mathrm{2}}+{\sigma }_{\mathrm{2}}^{\mathrm{2}}/m_{\mathrm{2}}\right)},\end{array}$$ (D.11)

which is sufficient to show that any point $({\overline{t}}_{\mathrm{1}\mathrm{m}\mathrm{i}\mathrm{n}},{\overline{t}}_{\mathrm{2}\mathrm{m}\mathrm{i}\mathrm{n}},s_{\mathrm{1}}^{\mathrm{2}},s_{\mathrm{2}}^{\mathrm{2}})$ is an absolute minimum of the unconstrained function (see Theorem D.1).

Given the constraint that 0 ≤ t _1j ≤ T for each j, ${\overline{t}}_{\mathrm{1}\mathrm{m}\mathrm{i}\mathrm{n}}$ is a feasible point only when $\mathrm{0}\le {\overline{t}}_{\mathrm{1}\mathrm{m}\mathrm{i}\mathrm{n}}\le T$. It is easy to see that ${\overline{t}}_{\mathrm{1}\mathrm{m}\mathrm{i}\mathrm{n}}>\mathrm{0}$ in all cases. Therefore, the absolute minimum can be achieved by collecting all data at ${\overline{t}}_{\mathrm{1}\mathrm{m}\mathrm{i}\mathrm{n}}$ if and only if ${\overline{t}}_{\mathrm{1}\mathrm{m}\mathrm{i}\mathrm{n}}\le T$.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.