Abstract
We propose a flexible regression model to study the association between a functional response and multiple functional covariates that are observed on the same domain. Specifically, we relate the mean of the current response to current values of the covariates by a sum of smooth unknown bivariate functions, where each of the functions depends on the current value of the covariate and the time point itself. In this framework, we develop estimation methodology that accommodates realistic scenarios where the covariates are sampled with or without error on a sparse and irregular design, and prediction that accounts for unknown model correlation structure. We also discuss the problem of testing the null hypothesis that the covariate has no association with the response. The proposed methods are evaluated numerically through simulations and two real data applications.
Keywords: Functional concurrent models, F-test, Nonlinear models, Penalized B-splines, Prediction
1. INTRODUCTION
Functional regression models where both the response and the covariate are functional have been long researched in the literature. Such models typically assume that the current response depends on the full trajectory of the covariate. The dependence is modeled through a weighted integral of the full covariate trajectory through an unknown bivariate coefficient surface as weight. Estimation procedures for this model have been discussed in [27], [43] and [40], among others. The crucial dependence assumption for this type of model may be impractical in many real data situations. To bypass this limitation, one might use the functional historical models (see, e.g., [19]), where the current response is modeled using only the past observations of the covariate. Such models quantify the relation between the response and the functional covariate/s using a linear relationship via an unknown bivariate coefficient function. Another alternative is to assume a concurrent relationship, where the current response is modeled based on only the current value of the covariate function. Functional linear concurrent models (see, e.g., [27]; [26]; [31]) assume a linear relationship; they can be thought of as a series of linear regressions for each time point, with the constraint that the regression coefficient is a univariate smooth function of time. These types of models, commonly known as varying coefficient models [11] are useful in many situations. There is a substantial amount of literature on such models for both scalar and longitudinal/functional response variables. In the latter case, several methods to estimate the model components have been developed, such as local polynomial kernel smoothing [13, 39, 5, 1], regression and smoothing spline methods [14, 4], and penalized spline and quadratic inference function based methods [25]. There have been many subsequent developments in this area such as an extension to spatial imaging [45], ridge regression [12]. We refer the readers to [6] and [22], and references therein, for detailed reviews on varying coefficient models and functional concurrent models. While the linear approach provides easy interpretation for the estimated coefficient function, it may not capture all the variability of the response in practical situations where the underlying relationship is complex.
In this article, we consider the functional concurrent model where we allow the relationship between the response and the covariate functions to be nonlinear. Specifically, we propose a model where the value of the response variable at a certain time point depends on both the time point and the values of the covariates at that time point through smooth unknown bivariate functions. Such formulation allows us to capture potential complex relationships between response and predictor functions as well as better capture out-of-sample prediction performance, as we will observe in our numerical investigation. Our model contains the standard linear concurrent model as a special case. We will show through numerical study that when the true underlying relationship is linear (that is, the linear concurrent model is correct), fitting our proposed model maintains prediction accuracy. On the other hand, when the true relationship is nonlinear, fitting the linear concurrent model results in high bias and loss of prediction accuracy.
We make three main contributions. First, we propose a general nonlinear functional concurrent model to describe the complex association between two or more functional variables measured on the same domain. We model the relationship via unknown bivariate functions, which we represent using tensor products of B-spline basis functions and estimate using a penalized least squares estimation procedure in conjunction with difference penalties ([3]; [18]; [21]). We discuss prediction of the response trajectory and develop point-wise prediction intervals that account for the correlated error structure. Accounting for the non-trivial dependence of the residuals is key for constructing valid inference in regression models with functional outcomes; see for example [10] and [23] who considered inference in a functional mixed model framework. [28] proposed inference for the fixed effects parameters in function-on-scalar regression by using estimates of the residual covariance obtained using an iterative procedure, and [9] extended these ideas to generalized multilevel function-on-scalar regression. [30], extending the work of [16], considered function-on-function regression models with flexible residual correlation structure, but did not numerically investigate the effect of different correlation structures on estimation and inference. We also assume a flexible correlation structure for the residuals and account for this non-trivial dependence in the proposed statistical inference. Specifically, we estimate the residual covariance using a two-step estimation procedure: 1) estimate the population level parameters using an independent error assumption; and 2) employ standard functional principal component analysis (FPCA) based methods (see, e.g., [43]; [46]; [8]) to the residuals. The proposed inference uses the resulting estimate of the residual covariance.
Second, our model allows us to incorporate multiple functional and scalar covariates, assuming that the effects of the covariates are additive. Specifically, we represent the effects of the covariates by a sum of smooth bivariate functions, each of which separately quantifies the dependence between the response and one of the covariates. The model involving a single functional covariate is a particular case of these models. Another way to incorporate multiple functional covariates in a functional regression is proposed by [17], where they use a single index model by first forming a linear effect of the functional covariates and then modeling the combined effect via a nonparametric function. Nevertheless, the models proposed by [17] are not our special case, and neither is our model a special case of theirs. This is because our methodology models the individual effect of each functional covariate nonlinearly and develops an additive model. We also employ a bivariate tensor product of B-spline based estimation procedure. Recently, [7] proposed a function-on-function nonlinear additive model with multiple functional and scalar covariates. They propose to summarize each functional covariate by taking an integral of the product of the covariate and an unknown two-dimensional index function, and then model the resulting single indices using a nonlinear additive model. As such, this model uses the entirety of each functional covariate to model the response function at any given time point. In contrast, we consider a concurrent relationship.
Third, we develop a testing procedure for assessing whether any of the functional covariates are related to the response variable. To the best of our knowledge, no other work considers testing for nonlinear concurrent models in functional setting and provides numerical results. We discuss two particular testing problems: 1) testing the global effect of the functional covariate against a model involving a single functional covariate, and 2) testing the null hypothesis of no association between the response and a particular covariate against a model involving two functional covariates. We consider an F-ratio type test statistic (see, e.g., [32]; [42]) and propose a resampling based algorithm to construct the null distribution of the test statistics. Our resampling procedure takes into account the correlated error structure and thus maintains the correct nominal size.
Our model is inspired by the model proposed in [21]. In particular, the nonlinear relationship that describes the conditional mean of the current response given the current value of the covariate is reminiscent of the one used in [21]. The key differences come from 1) the type of response considered, 2) ability to accommodate various sampling designs and 3) the covariance model assumed for the residuals. Specifically, we consider functional responses in this paper, whereas [21] studied scalar responses. Also, the proposed estimation and prediction procedures can accommodate various sampling designs for both responses and covariates, such as densely or sparsely sampled predictors with or without error. In contrast, the methods of [21] are presented only for densely sampled functional covariates. Additionally, we assume unknown complex dependence structure of the residuals, whereas [21] assume independent and identically distributed (iid) normal residuals, which is reasonable in their scalar-on-function regression setting. Accounting for the dependence within the error process is an important development in the proposed inference methodology.
2. ADDITIVE NONLINEAR FUNCTIONAL CONCURRENT MODEL
In this section, we introduce our model framework for a functional response and functional covariates, develop an estimation procedure for unknown model components, and discuss prediction of a new trajectory. For simplicity of presentation, our model and methodology are outlined for the setting involving two functional covariates; nevertheless, our methodology is general and can accommodate more than two scalar or functional covariates via linear or smooth effects.
2.1. Modeling Framework
Suppose for i = 1,...,n, we observe {(W1ij,t1ij) : j = 1,...,m1i}, {(W2ij,t2ij) : j = 1,...,m2i} and {(Yik, tik) : k = 1,...,mY,i}, where W1ij’s and W2ij’s denote the covariates observed at points t1ij and t2ij, respectively, and Yik’s are the response observed at tik. It is assumed that t1ij, t2ij, for all i, j and k and Wqij = Xq,i(tqij)+δqij for q = 1,2 where Xq,i(·) are realizations from random square-integrable processes, Xq(·), defined on the compact interval ; for convenience we take . It is assumed that δ1ij and δ2ij are the iid measurement errors with mean zero and variances equal to and , respectively. To illustrate our ideas, we first consider W1ij = X1,i(t1ij) and W2ij = X2,i(t2ij), which is equivalent to . Furthermore, we assume that the response and the covariates are observed on a common regularly spaced grid of points t1,...,tm. Adaptation of our methods to more realistic scenarios where and different sampling designs for Xq,i’s and Yi’s, e.g., sparsely observed data or when the covariates and response are not observed on the same grid of points, are discussed in Section 4 and Section A of the Supplementary Material.
We introduce the following additive nonlinear functional concurrent model (ANFCM)
| (1) |
where μY (t) is an unknown and smooth intercept function, F1 and F2 are smooth unknown bivariate functions defined on , and ϵi(·) is a Gaussian error process independent of the predictors X1,i(·) and X2,i(·). The error process ϵi(·) is assumed to have mean zero and unknown autocovariance function G(·,·). For identifiability of the two functions F1 and F2, we assume that E[Fq{Xq(t),t}] = 0 for any , q = 1,2, and thus μY (t) is the marginal mean function of the response. We introduce two main innovations in (1). First, the general bivariate functions Fq(·,·) allow us to model potentially complicated relationships between Y (·) and Xq(·) for q = 1,2, and extends the effect of the covariate beyond linearity. Second, we estimate and incorporate the unknown covariance structure for the residual process ϵi(·) in our inference, e.g., in prediction of a new response curve and hypothesis testing.
An important advantage of our framework is that it can easily accommodate multiple functional covariates with various types of effects; a few examples are shown in Table 1. We assume that models in Table 1 have flexible correlation structure for the residuals. The standard linear functional concurrent model with only one functional covariate is a special case of the model in (a) with F1(x,t) = β0(t) + xβ1(t) and F2(x,t) = 0, where β0(·) and β1(·) are unknown parameter functions. All the above models can easily accommodate scalar (time invariant) covariate effects of the form Xβ(t) or F(X,t) as well. While we present our methodology using only two functional covariates, the proposed framework can easily include more. This comes at the cost of computational time due to increasing number of model components and penalty parameters; although an efficient implementation (e.g., using the bam function instead of the default gam function in R package mgcv) helps to reduce computational time substantially. We have conducted a numerical study with up to five functional covariates (results shown in Supplementary Material, Section D.5), observed that our proposed methods perform reasonably well in all the cases considered, with increasing computation time.
Table 1.
Examples of general additive functional concurrent models.
| Model | Form |
|---|---|
| (a) General model | E{Y (t)|X1(t),X2(t)} = μY (t) + F1{X1(t),t} + F2{X2(t),t} |
| (b) Linear concurrent model | E{Y (t)|X1(t),X2(t)} = μY (t) + X1(t)β1(t) + X2(t)β2(t) |
| (c) Partially linear concurrent model | E{Y (t)|X1(t),X2(t)} = μY (t) + F1{X1(t),t} + X2(t)β2(t) |
2.2. Estimation
We focus on model (1) and describe estimation of the marginal mean function μY (t) and the bivariate surfaces F1(·,·) and F2(·,·). We model the mean function μY (t) by spline-based estimation methodology which represents the smooth effect by a linear combination of univariate B-spline basis functions. Let be a set of univariate B-spline basis functions defined on [0,1], where Kμ is the basis dimension. Using the basis functions, we can write , where Bμ(t) is the Kμ-dimensional vector of Bμ,d(t)’s, and Θμ is the vector of unknown parameters θμ,d’s. We also model F1(·,·) and F2(·,·) using bivariate basis expansion using tensor product of univariate B-spline basis functions ([20]; [36]; [21]; [30]). For q = 1,2 let and be the B-spline basis functions for x and t, respectively, used to model Fq(x,t). Then, F1(·,·) can be represented as , where Z1,i(t) is the Kx1Kt1-dimensional vector of BX1,k{X1,i(t)}BT1,l(t)’s, and Θ1 is the vector of unknown parameters, θ1,k,l’s. Similarly, we write , where Z2,i(t) is the Kx2Kt2-dimensional vector of BX2,k{X2,i(t)}BT2,l(t)’s, and Θ2 is the vector of unknown parameters, θ2,k,l’s. Based on the above expansions, model (1) can be written as
| (2) |
In this representation, a larger number of basis functions would result in a better but rougher fit, while a small number of basis functions results in overly smooth estimate. As is typical in the literature, we use a large number of basis functions to fully capture the complexity of the function, and penalize the coefficients to ensure smoothness of the resulting fit.
We propose to estimate Θμ, Θ1 and Θ2 by minimizing a penalized criterion , where ℙμ, ℙ1 and ℙ2 are the penalty matrices for smoothness of μY (t), F1(x,t) and F2(x,t), respectively, and contain penalty parameters that regularize the trade-off between the goodness of fit and the smoothness of fit. The notation || · ||2 is the usual L2-norm corresponding to the inner product . In practice, we observe Yi(·), X1,i(·) and X2,i(·) at fine grids of points t1,...,tm; thus, we approximate the L2-norm terms using numerical integration. The penalized sum of square fitting criterion becomes
| (3) |
Here, ℙμ is given by , where 𝔻μ represents the second order difference penalty ([3]; [20]; [21]), and λμis its penalty parameter. Also, ℙq (q = 1,2) is given by , where the notation ⊗ stands for the Kronecker product, 𝕀K is the identity matrix with dimension K, and 𝔻xq and 𝔻tq are matrices representing the row and column of second order difference penalties. The penalty parameters λxq and λtq control the roughness of the function in directions x and t, respectively.
An explicit form of the estimators and is readily available for fixed values of the penalty parameters. Define the m-dimensional vector of response Yi = [Yi(t1),...,Yi(tm)]T and the m-dimensional vector of errors Ei = [ϵi(t1),...,ϵi(tm)]T Also, we define 𝔹μ as m × Kμ-dimensional matrix with the j-th row given by and ℤq,i as m × KxqKtq-dimensional matrix with the j-th row given by . For simplicity, denote and ℙ = diag(ℙμ,ℙ1.ℙ2). Then the solution of Θ is calculated as
| (4) |
where . The penalty parameters can be chosen based on some appropriate criteria such as generalized cross validation (GCV) ([29]; [36]) or restricted maximum likelihood (REML) ([29]; [36]). Estimation under (3) can be fully implemented in R using functions of the mgcv package [37]. Then, given X1,i(t) and X2,i(t), the response curve can be estimated by . Furthermore, one can estimate the marginal mean of response by . Modification of the estimation procedure for the case where the grid of points is irregular and sparse is presented in the Supplementary Material, Section A.
2.3. Variance Estimation
The penalized criterion (3) does not account for the possibly correlated error process. For valid inference about Θ, one needs to account for the dependence of the residuals when deriving the variance of . The variance of the parameter estimate can be calculated as , where 𝔾 = cov(Ei) =[G(tj,tk)]1≤j,k≤m is the m×m covariance matrix evaluated corresponding to the observed time points. We model the non-trivial dependence of the errors process ϵ(t) assuming that the error process has the form ϵ(t) =ϵS(t) + ϵWN(t), where ϵS is a zero-mean smooth stochastic process, and ϵWN(t) is a zero-mean white noise measurement error with variance σ2. Let Σ(s,t) be the autocovariance function of ϵS. It follows that the autocovariance of the random deviation ϵ(t), G(s,t) = Σ(s,t) + σ2I(s = t) where I(·) is the indicator function, is unknown and needs to be estimated. To this end, we assume that Σ admits a spectral decomposition , where {ϕk(·),λk} are the pairs of eigenfunctions/eigenvalues. We first compute the residuals from the model fit, and employ FPCA methods (e.g., [44]; [46]) to estimate ϕk(·), λk, and σ2. Specifically, we obtain an initial smooth estimate of the covariance Σ, remove the negative eigenvalues, and obtain a final estimate of Σ, , where are the eigenfunctions/eigenvalues of the estimated covariance with , and K is the number of eigencomponents used in the estimation. Then, we estimate G by for any s,t ∈ [0,1], where is the estimated error variance. The finite truncation K is typically chosen by setting the percent of variance explained (PVE) by the first few eigencomponents to some pre-specified value, such as 90% or 95%.
2.4. Prediction
A main focus in this paper is prediction of response trajectory when a new covariate and its evaluation points are given. For example, in fire management, an important problem is the prediction of fuel moisture content, defined as proportion of free and absorbed water in the fuel. Study of fuel moisture content remains important for understanding fire dynamics and adequately predicting fire danger in an area of interest (see, e.g., [33]; [34]). However, dynamically measuring fuel moisture content on the spot over time is difficult, and a substantial amount of research has been directed to develop physical models for predicting moisture content profiles over time based on predictors that are easily available either from weather forecast (e.g., relative humidity and temperature) or predictable from seasons (e.g., solar radiation); see for example [33] for a discussion on this topic. One viable alternative is to model the past years available data using the proposed function-on-function regression model and then predict the fuel moisture trajectory for a future day based on the day’s weather forecast, so that an informative decision about fire danger can be made a priori.
However, while the research was motivated by the fire management data, we have been unable to gain access the data at the time of preparing this article. Instead, we will analyze the dietary calcium absorption data [2] and the gait data [27]. For the dietary calcium absorption data, our interest is to predict calcium absorption (as a function of age) of an individual based on their calcium intake over their age. In the context of gait data, the problem is to predict knee angle of an individual as a function of percent of the gait cycle based on the hip angle. We will analyze both the data sets in Section 6.
Suppose that we wish to predict new, unknown response Ynew(tj) when new observations X1,new(tj) and X2,new(tj) (j = 1,...,m) are given. We assume that the model Ynew(t) = μY (t) + F1{X1,new(t),t} + F2{X2,new(t),t} + ϵnew(t) ) still holds for the new data, where the error process ϵnew(t)) has the same distributional assumption as ϵi(t) in (1) and is independent of the new covariates X1,new(t) and X2,new(t). We predict the new response by
where and are estimated based on (4).
Uncertainty in the prediction depends on how small the difference is between the predicted response and the true response Ynew(t). We follow an approach similar to [29] to estimate the prediction variance. Specifically, conditional on the new covariates X1,new(t) and X2,new(t), we have . Note that ϵnew(t) is a realization of the same error process with zero-mean and covariance structure G(·,·). Let Z1,new(t) be the Kx1Kt1-dimensional vector of BX1,k{X1,new(t)}BT1,l(t)’s, and Z2,new(t) be the Kx2Kt2-dimensional vectors of BX2,k{X2,new(t)}BT2,l(t)’s as defined earlier. Also, let Ynew and be the m-dimensional vector of Ynew(tj)’s and respectively. Then the prediction variance becomes , where ℤnew = [𝔹μ|ℤ1,new|ℤ2,new] is m × (Kμ + Kx1Kt1 + Kx2Kt2)-dimensional matrix with the jth row given by . Then, the prediction variance can be estimated by plugging-in the sample estimate of G(·,·) in this formula. One can further define a 100(1 − α)% point-wise prediction interval for the new observation where Φ(·) is the standard Gaussian cumulative distribution function. In Section 4, we provide details about performing prediction in the more general case when the new covariates X1,new(·) and X2,new(·) are only observed on a sparsely sampled grid or with measurement error.
Remark.
The proposed estimation and prediction requires some preliminary steps. To be specific, we propose to transform the covariate functions by subtracting the point-wise mean and dividing by point-wise standard deviation function before applying the estimation and prediction procedures. The transformation of the covariates is important since the set of the covariate values {Xi(tj) : i,j} may not be necessarily dense over the entire domain on which the B-spline functions are defined. Therefore, there might be some situations when there are no observed data on the support of some of the B-spline basis functions. Such transformation strategies are also discussed in [21]. Details about the preliminary transformation are given in the Supplementary Material, Section B.
3. HYPOTHESIS TESTING
In many situations, testing for association between the response and the predictor variables is as important, if not more, as it is to estimate the model components. Often before performing any estimation, it is preferred to test for association first to determine whether there is association to begin with and then a more in-depth analysis is done to determine the form of the relationship. However, to the best of our knowledge, there are no currently available methods that allow testing the effect of functional covariates in non-linear concurrent models. In this section, we consider the problem of testing whether the functional predictor variable is associated with the response. To illustrate our ideas, two particular cases of the proposed framework are considered: 1) testing the no effect of the covariate against a model involving a single functional covariate (see Section 3.1); and 2) testing the significance of a particular covariate against a model involving two functional covariates (see Section 3.2).
3.1. Testing of Global effect
Our focus in this section is testing the global effect of the functional covariate. Specifically, we want to test
| (5) |
where F0(t) is univariate function and F1(x1,t) is bivariate function, both assumed unknown; see Section 5.2.2 for examples. Our testing procedure is based on first modeling the null effect F0(t) and the full model F1(x1,t) using basis function expressions in a manner that ensures that the null model is nested within the full model. Specifically, we propose to use {BT,l(t),l ≥ 1} to model F0(·) under the null model, where BT,l(t) are the B-splines evaluated at time point t. To model F1(·,·) under the full model, we use the same set of basis functions defined over the domain , but {BX,0(x1) = 1,BX,k(x1),k ≥ 1} for x1, where BX,k(x1) are the B-splines defined over - the state space of the process X1, where X1,i(·) ∼ X1(·). Under the full model, we can write . We propose to use the following F-type test statistic
| (6) |
where RSS0 and df0 are the residual sums of squares and the effective degrees-of-freedom [36] under the null model; RSS1 and df1 are defined similarly but corresponding to the full model. Here, N denotes the total number of observed data points. In this case, we have n subjects and m observations per subject, and thus the total number of observed data becomes N = nm. This can be easily generalized when each subject has different number of observations. In general, it is difficult to derive the null distribution of the proposed test statistic Tn (6) due to the smoothing techniques and the dependence in the data.
To bypass this complication, we propose to approximate the null distribution of the test statistic Tn using bootstrap of the subjects. Specifically, we follow the steps provided in Algorithm 1.
Algorithm 1.
Bootstrap algorithm for testing global effect
| 1: | Fit the full model described by the alternative hypothesis in (5) using the estimation procedure of the ANFCM. Calculate the residuals for all i and j. |
| 2: | Fit the null model described by the null hypothesis in (5) using the estimation procedure of the ANFCM and estimate . |
| 3: | Calculate the value of the test statistic in (6) based on the null and the full model fits; call this value Tn,obs. |
| 4: | Resample B sets of bootstrap residuals (b = 1,...,B) with replacement from the residuals obtained in step 1. |
| for b = 1 to B | |
| 5: | Generate response curves under the null model as . |
| 6: | Given the bootstrap data set , fit the null and the full models and evaluate the test statistic in (6), . |
| end for | |
| 7: | Compute the p-value by . |
In Algorithm 1, the test statistics obtained from each of the bootstrap samples can be viewed as realizations from the distribution of Tn under the assumption that H0 is true.
3.2. Testing of Inclusion
In the context of multiple predictors, one might be interested to know which of the predictors are related to the response variable. Suppose we want to test
| (7) |
versus the alternative H1 : E[Y (t)|X1(t) = x1,X2(t) = x2] = μY (t) + F1(x1,t) + F2(x2,t), where F1(x1,t) and F2(x2,t) are bivariate functions assumed unknown. For simplicity, denote by F0(x1,t) = μY (t) + F1(x1,t). To test the null hypothesis in (7), we represent the null and the full models using B-spline basis functions, and follow a similar logic used in the previous section. Specifically, we propose to use {BX1,0(x1) = 1,BX1,k(x1) : k ≥ 1} and {BT,l(t) : l ≥ 1} for x1 and t to model F0(x1,t) under the null hypothesis; here, BX1,k(x1) are the B-splines defined over the state space of the process X1, and BT,l(t) are the B-splines defined over . To formulate the full model, we use the same set of basis functions defined over the domain , but {BX1,0(x1) = 1,BX1,k(x1),BX2,k(x2) : k ≥ 1} for x1 and x2, where BX2,k(x2) are defined over the state space of the process X2. Therefore, under the alternative, we can write .
The null hypothesis in (7) can be tested using Algorithm 2 with the test statistic in (6).
Algorithm 2.
Bootstrap algorithm for testing significance
| 1: | Fit the full model described by the alternative hypothesis using the estimation procedure of the ANFCM. Calculate the residuals for all i and j. |
| 2: | Fit the null model described by the null hypothesis in (7) using the estimation procedure of the ANFCM and estimate . |
| 3: | Calculate the value of the test statistic in (6) based on the null and the full model fits; call this value Tn,obs. |
| 4: | Resample B sets of bootstrap residuals (b = 1,...,B) with replacement from the residuals obtained in step 1. |
| for b = 1 to B | |
| 5: | Generate response curves under the null model as . |
| 6: | Given the bootstrap data set , fit the null and the full models and evaluate the test statistic in (6), . |
| end for | |
| 7: | Compute the p-value by . |
Our proposed resampling algorithms described in Sections 3.1 and 3.2 account for the correlated error process, ϵ(·). This is done by sampling the entire residual vectors (“curve”) for each subject and thus preserving the correlation structure within the residuals. We observed in our numerical study (results not shown) that ignoring the correlation results in severely inflated type I error.
4. EXTENSIONS
This section discusses modifications of the methodology that are required by realistic situations. In particular, we consider the case when the functional covariate is observed densely with error, or sparsely with or without noise, as well as when the sparseness of the covariate is different from that of the response.
Assume first that functional covariate is observed on a fine and regular grid of points but with error; i.e., the observed predictors are Wij’s with Wij = Xi(tj)+δij, and the deviation δij has variance τ2 > 0. Several approaches have been proposed to adjust for the measurement errors; [46] proposed to first smooth each noisy trajectory using local polynomial kernel smoothing, and then estimate the mean and standard deviation of the covariate Xi(tj) by their sample estimators. The recovered trajectories, say , will estimate the latent ones Xi(·) with negligible error. The methodology described in Section 2.2 can be applied with in place of Xi(·). Numerical investigation of this approach is included in the simulation section.
Then consider the case that the functional covariate is observed on a sparse and irregular grid of points with measurement error, i.e., Wij = Xi(tij) + δij. The common assumption made for this setting is that the number of observations mi for each subject is small, but is dense in [0,1]. Reconstructing the latent trajectories Xi(·) is based on employing FPCA for sparse design ([44]) to the observed Wij’s. [44] proposed to 1) estimate the mean and covariance functions using local linear smoothers; 2) estimate eigenvalues/eigenfunctions from spectral decomposition of the estimated covariance; and 3) predict FPC scores via conditional expectation. Then, the latent trajectories are predicted by combining the newly estimated FPC scores and the estimated eigenfunctions. This method may be further applied to the response variable when the sampling design of the response is sparse as well; i.e., Yik = Yi(tik) for k = 1,...,mY,i. An alternative for the latter situation is to use the prediction of the covariates at the time points tik at which the response is observed, and then continue the estimation using the data . Preliminary investigation indicated that the latter method shows good performance in both estimation and testing evaluation; the former approach seems to yield slightly increased type I error rates.
The prediction procedure can also be extended to accommodate the more general case when the new covariate Xnew(tj) is only observed on a sparsely sampled grid. We first construct a smooth version of this new covariate using the FPCA. To this end, we compute the FPCA scores for the new covariate via the conditional expectation formula in [44] with the estimated eigenfunctions from the training data, implicitly assuming that the new covariate Xnew(·) and the originally observed covariate Xi(·) are generated from the same distribution. Then the prediction procedure can be readily applied with the smooth version of this new covariate in place of Xnew(tj).
5. NUMERICAL STUDY
In this section, we investigate the finite sample performance of our proposed methodology. Prediction accuracy is studied in Section 5.2.1, and testing performance is presented in Section 5.2.2. Finally in Section 6 we apply the proposed method to the gait study [24, 27, 15] and the dietary calcium absorption study [2, 31].
5.1. Simulation Design
We generate 1000 samples for the three simulation scenarios: (A) Yi(t) = F1{X1,i,(t), t) + ϵi(t) where F1(x1,t) = 1 + x1+t; (B) Yi(t) = F1{X1,i,(t), t) + ϵi(t)) where ; and (C) Yi(t) = F1{X1,i(t),t}+F2{X2,i(t),t}+ϵi(t) where and F2(x2,t) = 0.75exp(x2t). In all experiments, the true covariates are given by , where aq0 ∼ N(0,{2−0.5(q−1)}2), aq1 ∼ N(0,{0.85 × 2−0.5(q−1)}2) and aq2 ∼ N(0,{0.70 × 2−0.5(q−1)}2) for q = 1,2. Through-out the study, it is assumed that the covariates X1,i(t) and X2,i(t) are not observed directly. Instead we observe W1i = X1,i(t)+WN(0,0.62) and W2i = X2,i(t)+WN(0,0.62). For each of the above scenarios, the response Yi(·) is generated under all possible combinations of the following factors:
Error process Ei = [ϵi(ti1),…,ϵi(timY,i)]T : (i) ; (ii) where Σ has AR0.2(1) structure; and (iii) , where and
Sampling design: (i) dense: m = 81 equidistant time points in [0,1] for all i; and (ii) sparse: points for response and covariates
n ∈ 100,300
Our aim is to investigate the prediction accuracy of our method for both in-sample and out-of-sample prediction. To achieve this, we construct training and test data sets assuming both are independent. The test sets contain 100 subjects and are obtained using the set of 81 equally spaced points in [0,1].
5.2. Simulation Results
5.2.1. Prediction Performance
Our performance measure includes in-sample or out-of-sample root mean squared prediction error (RMSPEin and RMSPEout), integrated coverage probability (ICP) and integrated width (IW) of prediction bands. The definition of these measures is given in the Supplementary Material, Section C.
Result from Scenario A and B:
For these two scenarios, we fit both the standard linear functional concurrent model (FCM) and our proposed ANFCM for each of these choices. Our objective is two-fold: (i) When the true model is non-linear, i.e. , we expect to see that the ANFCM performs better than the FCM; and (ii) If the true model is in fact linear, i.e. F1(x1,t) = 1+x1 +t, we expect the ANFCM still maintains prediction accuracy relative to the FCM. We obtained both the ANFCM and the linear FCM fits using 7 cubic B-splines for x1 and t. For estimation of the residual covariance, the PVE is set equal to 99%.
Table 2 summarizes the predictive performance of our method for these two scenarios. We first discuss the case when the true function is nonlinear (the top two panels in Table 2). In this case, we observe that RMSPEin and RMSPEout from fitting the ANFCM are smaller than those from the linear FCM in all cases, indicating that the proposed ANFCM outperforms the linear FCM. The ICPs from the ANFCM and the linear FCM are fairly close to the nominal levels of 0.95, 0.90, and 0.85. However, on average, the linear FCM produces larger intervals, indicated by larger IW values and wider range of the estimated standard error denoted by R(SE) compared to the ANFCM. Such patterns confirm that the variability in prediction is not properly captured by the linear FCM when the true model is nonlinear. For less complicated error patterns such as (independent error structure), both models produce smaller prediction errors; nevertheless, ANFCM still produces smaller errors compared to linear FCM. The results are valid for different sample sizes as well as for dense/sparse sampling designs. The bottom two panels of Table 2 show the results when the underlying model is linear. In this case, the ANFCM continues to show very good prediction performance; the results are almost identical to the ones yielded by the linear FCM irrespective of the number of subjects, sparseness of the sampling, and the error covariance structure.
Table 2.
Summary of RMSPEin, RMSPEout, ICP, IW, and R(SE) based on 1000 simulated data sets. The models fitted by our method and the linear FCM are indicated by ANFCM and FCM, respectively.
| 1 − α = 0.95 | 1 − α = 0.90 | 1 − α = 0.85 | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| RMSPEin | RMSPEout | ICP | IW | R(SE) | ICP | IW | ICP | IW | |||||||||||
| n | Ei | ANFCM | FCM | ANFCM | FCM | ANFCM | FCM | ANFCM | FCM | ANFCM | FCM | ANFCM | FCM | ANFCM | FCM | ANFCM | FCM | ANFCM | FCM |
| Scenario B (true relationship is nonlinear), m = 81 | |||||||||||||||||||
| 100 | 0.97 | 3.55 | 0.96 | 3.75 | 0.959 | 0.943 | 3.83 | 12.67 | [3.54, 5.08] | [3.65, 21.90] | 0.916 | 0.919 | 3.21 | 10.64 | 0.871 | 0.896 | 2.81 | 9.31 | |
| 1.32 | 3.67 | 1.31 | 3.86 | 0.955 | 0.944 | 5.20 | 13.31 | [4.96, 6.26] | [5.03, 22.24] | 0.909 | 0.917 | 4.37 | 11.17 | 0.861 | 0.891 | 3.82 | 9.77 | ||
| 1.84 | 3.90 | 1.98 | 4.10 | 0.946 | 0.944 | 7.34 | 14.56 | [5.53, 9.84] | [8.37, 23.29] | 0.895 | 0.915 | 6.16 | 12.22 | 0.844 | 0.884 | 5.39 | 10.69 | ||
| 300 | 0.98 | 3.65 | 0.92 | 3.68 | 0.963 | 0.948 | 3.83 | 12.94 | [3.54, 5.11] | [3.63, 22.07] | 0.921 | 0.925 | 3.21 | 10.86 | 0.877 | 0.902 | 2.81 | 9.50 | |
| 1.33 | 3.75 | 1.28 | 3.79 | 0.958 | 0.948 | 5.20 | 13.56 | [4.96, 6.30] | [5.02, 22.39] | 0.912 | 0.923 | 4.37 | 11.38 | 0.866 | 0.898 | 3.82 | 9.96 | ||
| 1.87 | 3.99 | 1.87 | 4.02 | 0.952 | 0.949 | 7.30 | 14.79 | [5.58, 9.48] | [8.44, 23.52] | 0.903 | 0.921 | 6.12 | 12.41 | 0.854 | 0.892 | 5.36 | 10.87 | ||
| Scenario B (true relationship is nonlinear), | |||||||||||||||||||
| 100 | 1.16 | 3.55 | 1.18 | 3.76 | 0.964 | 0.928 | 4.52 | 12.60 | [3.71, 7.16] | [3.32, 23.46] | 0.929 | 0.903 | 3.79 | 10.57 | 0.891 | 0.880 | 3.32 | 9.25 | |
| 1.46 | 3.66 | 1.47 | 3.87 | 0.959 | 0.932 | 5.75 | 13.17 | [5.09, 8.04] | [4.69, 23.72] | 0.918 | 0.904 | 4.82 | 11.06 | 0.875 | 0.877 | 4.22 | 9.68 | ||
| 1.94 | 3.89 | 2.06 | 4.10 | 0.949 | 0.942 | 7.69 | 14.51 | [5.82, 11.09] | [8.13, 24.67] | 0.900 | 0.911 | 6.46 | 12.17 | 0.852 | 0.879 | 5.65 | 10.65 | ||
| 300 | 1.15 | 3.65 | 1.00 | 3.68 | 0.973 | 0.935 | 4.42 | 12.90 | [3.67, 6.90] | [3.23, 23.25] | 0.941 | 0.912 | 3.71 | 10.83 | 0.904 | 0.889 | 3.25 | 9.48 | |
| 1.46 | 3.76 | 1.33 | 3.79 | 0.966 | 0.941 | 5.66 | 13.50 | [5.06, 7.79] | [4.70, 23.50] | 0.927 | 0.915 | 4.75 | 11.33 | 0.885 | 0.889 | 4.16 | 9.91 | ||
| 1.96 | 3.99 | 1.91 | 4.03 | 0.958 | 0.948 | 7.64 | 14.80 | [5.87, 10.64] | [8.28, 24.56] | 0.913 | 0.919 | 6.41 | 12.42 | 0.866 | 0.890 | 5.61 | 10.87 | ||
| Scenario A (true relationship is linear), m = 81 | |||||||||||||||||||
| 100 | 0.90 | 0.90 | 0.90 | 0.90 | 0.951 | 0.952 | 3.53 | 3.53 | [3.52, 3.59] | [3.52, 3.59] | 0.902 | 0.902 | 2.97 | 2.97 | 0.853 | 0.853 | 2.60 | 2.60 | |
| 1.27 | 1.27 | 1.27 | 1.27 | 0.951 | 0.951 | 4.98 | 4.93 | [4.93, 5.14] | [4.93, 5.15] | 0.901 | 0.902 | 4.18 | 4.18 | 0.852 | 0.852 | 3.66 | 3.66 | ||
| 1.80 | 1.83 | 1.93 | 1.85 | 0.943 | 0.947 | 7.16 | 7.13 | [5.41, 9.10] | [5.40, 8.81] | 0.891 | 0.896 | 6.01 | 5.98 | 0.839 | 0.846 | 5.26 | 5.24 | ||
| 300 | 0.90 | 0.90 | 0.89 | 0.89 | 0.952 | 0.952 | 3.53 | 3.53 | [3.52, 3.58] | [3.52, 3.58] | 0.902 | 0.902 | 2.97 | 2.97 | 0.853 | 0.853 | 2.59 | 2.60 | |
| 1.27 | 1.27 | 1.27 | 1.27 | 0.951 | 0.951 | 4.98 | 4.98 | [4.93, 5.15] | [4.93, 5.15] | 0.902 | 0.902 | 4.18 | 4.18 | 0.852 | 0.852 | 3.66 | 3.66 | ||
| 1.83 | 1.84 | 1.86 | 1.84 | 0.948 | 0.949 | 7.13 | 7.13 | [5.44, 8.77] | [5.44, 8.72] | 0.897 | 0.899 | 5.98 | 5.98 | 0.846 | 0.849 | 5.24 | 5.24 | ||
| Scenario A (true relationship is linear), | |||||||||||||||||||
| 100 | 0.92 | 0.92 | 0.90 | 0.90 | 0.955 | 0.955 | 3.61 | 3.61 | [3.55, 3.77] | [3.55, 3.77] | 0.907 | 0.908 | 3.03 | 3.03 | 0.859 | 0.859 | 2.65 | 2.65 | |
| 1.28 | 1.28 | 1.27 | 1.27 | 0.952 | 0.952 | 5.04 | 5.04 | [4.96, 5.26] | [4.96, 5.26] | 0.904 | 0.904 | 4.23 | 4.23 | 0.854 | 0.855 | 3.70 | 3.70 | ||
| 1.81 | 1.84 | 1.89 | 1.86 | 0.943 | 0.947 | 7.14 | 7.17 | [5.38, 9.56] | [5.37, 9.54] | 0.891 | 0.896 | 6.00 | 6.01 | 0.840 | 0.846 | 5.25 | 5.26 | ||
| 300 | 0.92 | 0.92 | 0.90 | 0.90 | 0.955 | 0.955 | 3.60 | 3.60 | [3.55, 3.72] | [3.55, 3.73] | 0.908 | 0.908 | 3.02 | 3.02 | 0.859 | 0.859 | 2.64 | 2.64 | |
| 1.28 | 1.28 | 1.27 | 1.27 | 0.953 | 0.953 | 5.03 | 5.03 | [4.97, 5.19] | [4.97, 5.19] | 0.904 | 0.904 | 4.22 | 4.22 | 0.855 | 0.855 | 3.69 | 3.69 | ||
| 1.84 | 1.85 | 1.85 | 1.84 | 0.949 | 0.950 | 7.16 | 7.17 | [5.45, 9.19] | [5.45, 9.19] | 0.899 | 0.900 | 6.01 | 6.02 | 0.848 | 0.850 | 5.26 | 5.27 | ||
To aid understanding these results, Figure 1 displays prediction bands for three subject-level trajectories, when the true model is nonlinear (top panel) and linear (bottom panel), and the covariates are observed densely. In the top panel, the prediction bands for the linear FCM (dashed line) are much wider than the bands from the ANFCM (solid line). This indicates that variance estimation from the linear FCM is less accurate. In the bottom panel, the prediction bands of the ANFCM (grey solid line) and the linear FCM (dashed line) are almost identical, indicating a similar performance in terms of the variance estimation when the true model is linear.
Figure 1.
95% prediction bands constructed for three subject-level trajectories in the test data. The results are from scenario A and B (involving a single functional covariate) for the setting where sampling design is dense, n = 100, and . The top (bottom) panel corresponds to the case where the true function F1(x1,t) is nonlinear (linear) in x1. “+” are the response Ynew,i(·) in the test data, and dotted (“•”) lines are the true response without measurement errors. Solid and dashed lines are the prediction bands obtained by fitting the ANFCM and the linear FCM, respectively.
Result from Scenario C:
Now consider the setting involving two functional covariates. In this experiment, we investigate both in-sample and out-of-sample prediction performance as well as computational complexity of our algorithm with respect to the number of additional covariates. It is worth noting that the computational complexity can be affected by the number and the type of additional predictors as well as their assumed effect. Specifically, for fixed values of the smoothing parameters, the solution is readily available irrespective of the number of terms in the model. Selecting the optimal values of the smoothing parameters may be computationally demanding, and thus the computational time depends on the number of smoothing parameters. There are two available implementation tools to obtain predictions: gam and bam functions of mgcv ([37]) R package. Both functions provide almost identical model fits, but computational advantage can be gained when using the bam function with very large data sets. For completeness, we report simulation results for Scenario B (the model involving a single functional covariate) and C (the model with two functional covariates).
We obtained the ANFCM fits using 7 cubic B-splines for x1, x2 and t to model the smooth functions F1(x1,t) and F2(x2,t). For estimation of the residual covariance, we preset PVE = 99%. Table 3 summarizes RMSPEin, RMSPEout, ICP’s at the nominal levels of 0.95, 0.90, and 0.85, and average computation time (in seconds). The computation time is measured on a 2.3GHz AMD Opteron Processor and averaged over 1000 Monte Carlo replications. The top and bottom panels display the results corresponding to Scenario B and C, respectively. We observe from the results that both in-sample and out-of-sample predictive accuracy are still maintained irrespective of the number of predictive variables. We also observe that the coverage (indicated by ICP) is fairly close to the nominal levels in all cases. In terms of computational cost, both gam and bam slightly increase the average computation time with the increased sample size and model complexity. Nevertheless, the additional computation expense is rather minimal compared to the computational time corresponding to single functional covariate. The results also show that computations can be further sped up if bam is used.
Table 3.
Summary of RMSPEin, RMSPEout, ICP and average computation time (in seconds) based on 1000 simulated data sets. The models are fitted by the estimation procedure of ANFCM. The average computation times are obtained using gam and bam functions in mgcvR package.
| n | Ei | RMSPEin | RMSPEout | ICP at | ICP at | ICP at | Computation Time (seconds) | |
|---|---|---|---|---|---|---|---|---|
| 1 − α = 0.95 | 1 − α = 0.90 | 1 − α = 0.85 | gam | bam | ||||
| Scenario B (model with single functional covariate), m = 81 | ||||||||
| 100 | 0.97 | 0.96 | 0.959 | 0.916 | 0.871 | 4.05 | 2.02 | |
| 1.32 | 1.31 | 0.955 | 0.909 | 0.861 | 3.75 | 2.02 | ||
| 1.84 | 1.98 | 0.946 | 0.895 | 0.844 | 3.55 | 3.02 | ||
| 300 | 0.98 | 0.92 | 0.963 | 0.921 | 0.877 | 22.09 | 4.07 | |
| 1.33 | 1.28 | 0.958 | 0.912 | 0.866 | 15.60 | 3.90 | ||
| 1.87 | 1.87 | 0.952 | 0.903 | 0.854 | 12.03 | 3.06 | ||
| Scenario B (model with single functional covariate), | ||||||||
| 100 | 1.16 | 0.98 | 0.964 | 0.929 | 0.891 | 1.04 | 2.76 | |
| 1.46 | 1.47 | 0.959 | 0.918 | 0.875 | 0.98 | 2.88 | ||
| 1.94 | 2.06 | 0.949 | 0.900 | 0.852 | 1.02 | 2.53 | ||
| 300 | 1.15 | 1.00 | 0.973 | 0.941 | 0.904 | 3.76 | 2.64 | |
| 1.46 | 1.33 | 0.966 | 0.927 | 0.885 | 3.63 | 2.97 | ||
| 1.96 | 1.91 | 0.958 | 0.913 | 0.866 | 3.33 | 2.53 | ||
| Scenario C (model with two functional covariates), m = 81 | ||||||||
| 100 | 0.96 | 0.94 | 0.955 | 0.908 | 0.860 | 24.74 | 4.44 | |
| 1.31 | 1.28 | 0.957 | 0.912 | 0.863 | 24.37 | 4.17 | ||
| 1.80 | 1.96 | 0.937 | 0.882 | 0.831 | 24.15 | 3.37 | ||
| 300 | 0.97 | 0.92 | 0.958 | 0.912 | 0.866 | 57.74 | 4.61 | |
| 1.32 | 1.27 | 0.959 | 0.915 | 0.867 | 56.52 | 4.85 | ||
| 1.85 | 1.90 | 0.946 | 0.892 | 0.842 | 60.43 | 3.16 | ||
| Scenario C (model with two functional covariate), | ||||||||
| 100 | 1.13 | 1.07 | 0.961 | 0.921 | 0.880 | 4.20 | 3.20 | |
| 1.44 | 1.38 | 0.960 | 0.919 | 0.874 | 5.12 | 2.97 | ||
| 1.91 | 2.01 | 0.942 | 0.891 | 0.840 | 4.86 | 3.13 | ||
| 300 | 1.13 | 0.97 | 0.970 | 0.934 | 0.895 | 18.14 | 4.05 | |
| 1.44 | 1.30 | 0.967 | 0.928 | 0.885 | 14.15 | 4.33 | ||
| 1.94 | 1.91 | 0.953 | 0.903 | 0.856 | 11.15 | 3.96 | ||
To summarize, the numerical investigation shows that ANFCM may result in a significant gain in prediction accuracy over the standard linear FCM, when the true model is nonlinear; ANFCM has similar prediction performance relative to the linear FCM, when the true model is linear. Furthermore, in the presence of two functional covariates, our method still preserves high predictive accuracy, and the additional computation cost is not substantial compared to the case of single functional covariate.
In the Supplementary Material, we include additional simulation results. Section D.1 reports simulation results corresponding to another level of sparseness. Section D.2 compares the results corresponding to two competitive approaches for covariance estimation: using local linear smoothing ([44]) which is implemented in Matlab using the functions of the PACE toolbox and using a fast covariance smoothing method ([41]) which is implemented using fpca.face function of the refundR package ([15]). Section D.3 presents additional results for larger measurement error variance (τ2 = 1 and 2) in functional covariates as well as a smaller sample size n = 40 (as in the Gait data example) in addition to n = 100 and 300. We also conducted an additional simulation study to investigate the effect of different choices of the number of basis functions. The results are displayed in Section D.4 of the Supplementary Material. We found that the prediction errors and the coverage at different nominal levels did not vary too much across different number of basis functions. Section D.5 investigates the model performance with up to five functional covariates through a numerical study.
5.2.2. Testing Performance
Now we assess the performance of the proposed testing procedure for the following two scenarios: (A) E[Yd(t)|X1(t) = x1] = 1 + 2t + t2 + d(x1t/8); and (B) E[Yd(t)|X1(t) = x1,X2(t) = x2] = 2t + t2 + x1 sin(πt)/4 +d{2cos(x2t)}, where d is a constant. We generate the data corresponding to the above true models using the error covariance structure and sampling design defined in Section 5.1. In Scenario A, we are interested in testing the null hypothesis in (5). Here, F1{X(t),t} = F0(t) + d{X(t)β1(t)}, where F0(t) = 1 + 2t + t2 and β1(t) = t/8. This scenario is designed to capture the standard linear concurrent effect of the functional covariate. When d = 0, the true model is a univariate function of time point t, and X(·) has no association with Y ; when d > 0, the true model depends on both X(t) and t in a linear concurrent relationship. Thus the parameter d indexes the departure from the null hypothesis given in (5) towards stronger linear concurrent relationship. This setting is similar to the real data analysis (in Section 6), where the functional covariate is found to have significant association with the response but performance of the proposed nonlinear model is similar to that of a linear concurrent model.
In Scenario B, we are interested in testing the null hypothesis given in (7). Here, F1{X1(t),t} = X1(t)sin(πt)/4 and F2{X2(t),t)} = d{2cos(X2(t)t)}, where the parameter d ≥ 0 controls the departure from the null hypothesis given in (7). When d = 0, the response depends on X1 using a linear concurrent relationship, but does not depend on X2. However, when d > 0, Y has a linear concurrent relationship with X1 and a nonlinear relationship with X2. Thus Scenario B is designed to detect such a nonlinear relationship in the case with two covariates.
For each of the scenarios, type I error of the test is investigated by setting d = 0, and the power of the test is studied for positive values of d. We generated 2000 samples to assess the type I error rate, and 1000 samples to assess the power. The distribution of the test statistic in (6) is approximated using B = 200 bootstrap samples for each simulation.
Result from Scenario A:
We first examine the size and power performance of the global test presented in Algorithm 1. The empirical type I error rates are evaluated at nominal levels of α = 5% and 10% for sample sizes 100 and 300, and the estimated rejection probabilities are presented in top panel of Table 4. We observe that estimated type I error rates are mostly within 2 standard errors of the nominal values, and the larger sample size (n = 300) improves the size performance. The results also indicate that the performance is similar across different covariance structures and sampling designs. The power performance of our proposed test is evaluated for a fixed nominal level α = 5%. The top panel in Figure 2 displays the rejection probabilities for d = 0.1,1,2,...,7 and for different error covariance structure; in the interest of space, we only present the case of sparse design. As expected, the power of the testing procedure increases with the sample size, while the results are affected by the complexity of the error covariance: the power corresponding to non-stationary error covariance is much lower than the counterpart corresponding to AR(1) covariance error structure. Power curves for the densely sampled scenario are provided in the Supplementary Material, Section D.6. We found that the results for densely sampled data are very similar to the one obtained for sparsely sampled data.
Table 4.
Type I error probabilities (×100) based on 2000 simulations. The values in the parenthesis are the estimated standard errors (×100) of the rejection probabilities.
| α = 5% | α = 10% | |||||||
|---|---|---|---|---|---|---|---|---|
| Scenario | Sampling | n | ||||||
| A | m = 81 | 100 | 5.3(0.5) | 5.4(0.5) | 6.6(0.6) | 11.1(0.7) | 10.6(0.7) | 10.3(0.7) |
| 300 | 6.1(0.5) | 5.0(0.5) | 5.1(0.5) | 10.6(0.7) | 10.9(0.7) | 9.4(0.7) | ||
| 100 | 5.5(0.5) | 5.0(0.5) | 4.6(0.5) | 10.0(0.7) | 10.8(0.7) | 9.8(0.7) | ||
| 300 | 4.8(0.5) | 4.2(0.4) | 5.2(0.5) | 9.5(0.7) | 10.4(0.7) | 10.7(0.7) | ||
| B | m = 81 | 100 | 6.2(0.5) | 4.5 (0.5) | 5.1(0.5) | 11.5(0.7) | 9.7(0.7) | 10.0(0.7) |
| 300 | 6.4(0.5) | 4.8 (0.5) | 5.0(0.5) | 12.9(0.7) | 10.7(0.7) | 11.7(0.7) | ||
| 100 | 5.0(0.5) | 4.4(0.5) | 6.2(0.5) | 11.2(0.7) | 9.8(0.7) | 10.8(0.7) | ||
| 300 | 6.2(0.5) | 5.3(0.5) | 6.5(0.5) | 12.8(0.7) | 10.7(0.7) | 11.7(0.7) | ||
Figure 2.
Powers (×100) of the tests at significance level α = 5%. The top (bottom) panel displays the results from scenario A (scenario B) for the setting where sampling design is sparse. The error process in the left, middle and right panels is assumed to be and , respectively.
Result from Scenario B:
We now assess the size and power properties of the significance testing presented in Algorithm 2. The bottom panel of Table 4 shows the empirical type I error rates corresponding to nominal levels α = 5% and 10% for sample sizes 100 and 300. The results show that the estimated sizes are mostly within 2 standard errors of the nominal values for all significance levels, and the results are less affected by different error covariance structure and sampling designs. The bottom panel in Figure 2 displays the rejection probabilities at the α = 0.05 level for d = 0.1,1,2,...,7 and for different error covariance structure; again, we only present the case corresponding to sparse design. For the moderate sample size (n = 100), the power of the testing procedure increases with the value of d. For the larger sample size (n = 300) the empirical rejection probabilities converge to 1 at a fast rate as the value of d increases. As expected, there is some loss of power for complicated error patterns such as (non-stationary error covariance structure). However, the power performance for this case improves with the larger sample size (n = 300). We also detected that the power of the test is affected by how the data is sampled, but with only a negligible difference; the power curves corresponding to the dense design are provided in the Supplementary Material, Section D.6.
Finally, it is worthwhile noting that, when calculating the size of proposed test Tn,obs or , the difference RSS0 −RSS1 occasionally comes out negative. We detected such cases 2.6% of the time on an average, and this is true irrespective whether the sampling design is dense or sparse. In these cases, we set RSS0 −RSS1 = 0. Typically df0 −df1 is positive. However, for very few cases (less than 0.1% of the time) this difference was returned negative; we excluded such cases from our study.
6. APPLICATIONS
6.1. Gait Data
We turn our attention to data applications. We first consider the study of gait deficiency, where the objective is to understand how the joints in hip and knee interact during a gait cycle [35]. Typically, one represents the timing of events occurring during a gait cycle as a percentage of this cycle, where the initial contact of a foot is recorded as 0% and the second contact of the same foot as 100%. The data consist of longitudinal measurements of hip and knee angles taken on 39 children as they walk through a single gait cycle [24, 27]. The hip and knee angles are measured at 20 evaluation points in [0,1], which are translated from percent values of the cycle. In the Supplementary Material, Figure 2 of Section E.1 displays the observed individual trajectories of the hip and knee angles.
We consider our proposed methodology to relate the hip and knee angles, which is an example of densely observed functional covariates and response. Let Yij = Yi(tj) be the knee angle and Wij = Xi(tj) + δij be the hip angle corresponding to the ith child and the percentage of gait cycle tj, were δij are the measurement errors. The measurement error variance in the Hip angle curves are estimated (via FPCA) to be 1.714. We first employ our resampling based test to investigate whether the hip angles are associated with the knee angles. We select 7 cubic B-splines for x and t to fit the ANFCM, E[Yi(t)|Xi(t)] = F{Xi(t),t}, and B = 250 bootstrap replications are used. The bootstrap p-value is computed to be less than 0.004, thus we conclude that the hip angle measured at a specific time point has a strong effect on the knee angle at the same time point.
To assess how the hip and knee angles are related to each other, we fit our proposed ANFCM as well as the linear FCM. We assess the predictive accuracy by splitting the data into training and test sets of size 30 and 9. Assuming that the hip angles are observed with measurement errors, we smooth the covariate by FPCA and then apply the center/scaling transformation. We compare prediction errors obtained by fitting both the ANFCM and the linear FCM. Also, as a benchmark model we further fit a linear mixed effect (LME) model Yij = (β0 +b0i)+(β1 +b1i)Xij +(β2 + b2i)tij + ϵij, where (b0i,b1i,b2i)T are the subject random coefficients from N(0,R) with some 3×3 unknown covariance matrix R, ϵij are the errors from , and (b0i,b1i,b2i)T and ϵij are assumed to be independent. For the ANFCM and the linear FCM, we report the in-sample and the out-of-sample RMSPE, the ICP, the IW, and the R(SE). For the LME model, we report similar measures, but take the average over the repeated measurements instead of integrating over the time domain.
The results are summarized in Table 5 (top panel). We observe that the LME model provides a poor predictive performance compared to the others, implying that models in the framework of concurrent regression models are obviously better. ANFCM yields similar predictive performance relative to the linear FCM. Specifically, the prediction errors from the ANFCM are similar to the linear FCM. The R(SE) obtained from the ANFCM are only slightly narrower compared to FCM at all significance levels. Figure 3, top panel, shows the prediction bands obtained for few subjects in the test data set. The two competitive models, ANFCM (solid line) and the linear FCM (dashed line), show similar results. The bottom panel in Figure 3 displays a heat map plot of the predicted surface using the ANFCM. The results corroborate that the relationship between the hip angles and the knee angles is linear. This finding is also confirmed by additional simulation results using a generating model that mimics the gait data, which are included in the Supplementary Material, Section E.2.
Table 5.
Results from data examples as described in Section 6. Displayed are the summaries of RMSPEin, RMSPEout, ICP, IW, and R(SE). The models fitted by our method and the linear FCM are indicated by ANFCM and FCM, respectively.
| Gait data | |||||||||||
| 1 − α = 0.95 | 1 − α = 0.90 | 1 − α = 0.85 | |||||||||
| RMSPEin |
RMSPEout |
ICP |
IW |
R(SE) |
ICP |
IW |
R(SE) |
ICP |
IW |
R(SE) |
|
| ANFCM | 5.47 | 5.99 | 0.939 | 20.43 | [14.86, 35.75] | 0.867 | 17.15 | [12.47, 30.00] | 0.828 | 15.01 | [10.91, 26.26] |
| FCM | 5.59 | 5.69 | 0.943 | 20.60 | [14.91, 36.12] | 0.861 | 17.29 | [12.51, 30.32] | 0.822 | 15.13 | [10.95, 26.53] |
| LME | 18.93 | 19.05 | 0.972 | 76.55 | [73.75, 82.26] | 0.883 | 64.24 | [61.90, 69.04] | 0.844 | 56.22 | [54.17, 60.42] |
| Calcium absorption data | |||||||||||
| 1 − α = 0.95 | 1 − α = 0.90 | 1 − α = 0.85 | |||||||||
| RMSPEin |
RMSPEout |
ICP |
IW |
R(SE) |
ICP |
IW |
R(SE) |
ICP |
IW |
R(SE) |
|
| ANFCM | 0.079 | 0.112 | 0.950 | 0.35 | [0.32, 0.56] | 0.935 | 0.30 | [0.27, 0.47] | 0.919 | 0.26 | [0.23, 0.41] |
| FCM | 0.081 | 0.114 | 0.951 | 0.33 | [0.31, 0.41] | 0.935 | 0.28 | [0.26, 0.34] | 0.921 | 0.24 | [0.23, 0.30] |
Figure 3.
Results from gait data analysis as described in Section 6. The top panel displays 95% prediction bands obtained by fitting the ANFCM (grey solid lines) and the linear FCM (dashed lines) for three subject-level trajectories in the test data. “•” represent the knee angles in the test data. The bottom panel shows the heat map of obtained from the test data set of the gait data example.
6.2. Dietary Calcium Absorption Data
Next, we consider an application to dietary calcium absorption study [2]. In a group of 188 patients, dietary and bone measurement tests are conducted approximately every five years, and calcium intake and absorption are measured for each subject. The patients are between 35 and 45 years old at the beginning of the study, with the overall ages ranging from 35 to 64 years old. The number of repeated measurements per subject varies between 1 to 4 times. This is an example of data where both the functional response and covariate are observed on a sparse design. Our objective is to examine the pattern of calcium absorption over the ages based on calcium intake as well as body mass index (BMI) of the patients averaged over their ages. To assess their relationship, let Yij = Yi(tij) be the calcium absorption and W1ij = X1,i(t1ij) + δij be the calcium intake corresponding to ith subject and the jth time point, where δij are the noise. Let X2,i denote the average BMI for the ith subject. In the Supplementary Material, Figure 2 of Section E.1 displays the observed individual trajectories of calcium intake and absorption along the patient’s age at the visit.
In the analysis, we fit a ANFCM with a linear BMI effect, E[Yi(t)|X1,i(t),X2,i] = F{X1,i(t),t} + γ(t)X2,i, where γ(t) denotes the unknown slope function of the average BMI. As an alternative model, we also study the dependence assuming a linear FCM, E[Yi(t)|X1,i(t),X2,i] = β0(t) + β1(t)X1,i(t)+γ(t)X2,i. In the analysis, ages are transformed into the values in [0,1], and the results are considered as evaluation points of the functions. As before, we begin by testing the null hypothesis of no association between the calcium intake and the absorbtion. Specifically, we test the null hypothesis H0 : E[Y (t)|X1(t) = x1,X2 = x2] = β0(t) + x2γ(t) versus the alternative H1 : E[Y (t)|X1(t) = x1,X2 = x2] = F(x1,t) + x2γ(t) using Algorithm 2. We select 7 cubic B-splines in directions x and t respectively to fit the ANFCM, and use B = 250 bootstrap replications. The p-value of our test is obtained to be less than 0.004, indicating a significant association between the current calcium intake measured and the current absorption.
We next analyze the predictive performance of the ANFCM by using a training set of 148 random patients and a test set formed by the remaining 40 patients. Shown are also the results obtained with the linear FCM. Several adjustments are required to accommodate the sparse sampling design of the covariates, X1,i(t). The covariates in the training set are smoothed using the standard FPCA toolkit for sparse functional data [44]. The resulting estimated model (using the training data) is later used to reconstruct the trajectories in the test set. The prediction results using the ANFCM and the competitive linear FCM are presented in Table 5 (bottom panel). Both the ANFCM and the linear FCM show similar in-sample and out-of-sample performance, RMSPEin and RMSPEout; this indicates that a simple linear association between the calcium intake and absorption is more appropriate.
Furthermore, in Figure 4, the leftmost two panels present the point-wise prediction intervals/bands for two selected subjects from the test data. The differences between the ANFCM (grey solid lines) and the linear FCM (dashed lines) are rather negligible, further confirming a linearity dependence between the calcium intake and the absorption. The rightmost panel in Figure 4 displays the estimated slope function obtained by fitting the ANFCM and the linear FCM. On average, both methods indicate a positive effect of the average BMI on the absorption but with a slight difference in the estimated effects. Additional analysis has shown that the difference is due to the estimation errors.
Figure 4.
Results from calcium data analysis as described in Section 6. The leftmost two panels display 95% prediction bands obtained by fitting the ANFCM (grey solid lines) and the linear FCM (black dotted lines) for two subject-level trajectories in the test data. “•” indicate the calcium absorption measured, and thin solid lines are the predicted calcium absorption from the ANFCM estimation method. The rightmost panel displays the estimated slope function of BMI, , obtained from the ANFCM (black solid line) and the linear FCM (black dotted line).
In the Supplementary Material (Section E.3), we have also investigated the effect of choosing different number of basis functions, Kx and Kt, for the calcium absorption data. The results for three different choices (Kx,Kt) = (7,7),(9,9) and (11,11) are similar. We also investigated another ANFCM for the calcium absorption data set with a bivariate effect of BMI and age, E[Yi(t)|X1,i(t),X2,i] = F1{X1,i(t),t} + F2(X2,i,t), and obtained similar results.
7. DISCUSSION
We propose a wide class of function-on-function regression models, the additive nonlinear functional concurrent model, and discuss significance testing of no association. In particular, our proposed hypothesis testing can formally assess whether the effect of a functional covariate is significant under the assumption that the relationship between the response and the predictor is general; the linear dependence is a special case of the proposed general model, as described by our proposed modeling. In contrast, the existing literature assumes a linear dependence between the response and the covariate/s. Thus, similar significance tests are only valid when the linearity dependence assumption between the response and the covariate is true. For the two applications - the gait data and the dietary calcium absorption data - our testing procedure found significant association between the response and covariate under a more general dependence assumption. Furthermore, using the proposed methods we found evidence that the relationship between the response and covariate is indeed linear; in contrast, the linear FCM assumes a linear dependence is valid. Thus, our proposed procedure allows one to approach the problem from a more general point of view. We have implemented our proposed estimation and testing methodology using R software, and details about the implementation are provided in Section F of the Supplementary Material.
Models similar to (1) are also considered by [30]; software implementing their approach is available in the refund package in R. However, the main difference is that [30] assume that the error process is white noise, that is, iid Gaussian errors; although functional random effects are allowed. In the specific context of nonlinear concurrent modeling, no numerical investigation has been done by [30] regarding estimation or prediction. In contrast, we present methodology and numerical results for prediction, estimation of prediction uncertainty as well as algorithms for two types of hypothesis testing.
We note that the model parameters are estimated by minimizing a penalized least squares criterion (3), and the smoothing parameters are chosen using REML or GCV. While minimizing (3) is reasonable regardless whether the errors are dependent or not, it can be viewed as a penalized likelihood criterion only when errors are assumed to be independent. As a result, we propose methods for variance estimation that takes into account the error covariance. One potential issue in this procedure is that the smoothing parameters are still chosen based on (3). While our numerical results show that the proposed method still has good performance in terms of prediction error and coverage, more research is needed to address this issue. Also, the variance estimation procedure does not take into account the variability due to the estimation of the smoothing parameters. It might be possible to develop methodology to overcome this issue using recently published works such as that of [38] (see, e.g., their Section 4). Finally, in our variance estimation procedure, we have essentially adopted a frequentist approach; however, it might be possible to implement a Bayesian variance estimation procedure (see, e.g., Chapter 4.8 of [36]) in our framework. All these issues are highly interesting research topics and are reserved for future research.
Supplementary Material
ACKNOWLEDGEMENTS
Maity’s research was supported by National Institute of Health grant R00 ES017744 and an NCSU Faculty Research and Professional Development grant. Staicu’s research was supported by National Institute of Health grants R01 NS085211 and R01 MH086633 and National Science Foundation grant DMS 1454942. The authors are extremely grateful to the three anonymous referees whose suggestions and comments greatly improved the results and presentation of the article.
Footnotes
SUPPLEMENTAL MATERIAL
Online supplemental material is available in the submitted file.
Contributor Information
Janet S. Kim, Department of Statistics, North Carolina State University, Raleigh, North Carolina 27695, jskim3@ncsu.edu
Arnab Maity, Department of Statistics, North Carolina State University, Raleigh, North Carolina 27695, amaity@ncsu.edu.
Ana-Maria Staicu, Department of Statistics, North Carolina State University, Raleigh, North Carolina 27695, ana-maria_staicu@ncsu.edu.
REFERENCES
- [1].Chiang CT, Rice JA and Wu CO (2001). Smoothing spline estimation for varying coefficient models with repeatedly measured dependent variables. Journal of the American Statistical Association 96 605–619. [Google Scholar]
- [2].Davis CS (2002). Statistical methods for the analysis of repeated measurements. New York, NY: Springer. [Google Scholar]
- [3].Eilers PHC and Marx BD (1996). Flexible smoothing with B-splines and penalties. Statistical Science 11 89–121. [Google Scholar]
- [4].Eubank RL, Huang C, Muñoz Maldonado Y, Wang N, Wang S and Buchanan RJ (2004). Smoothing spline estimation in varying-coefficient models. Journal of the Royal Statistical Society, Ser. B 66 653–667. [Google Scholar]
- [5].Fan J and Zhang J (2000). Two-Step estimation of functional linear models with applications to longitudinal data. Journal of the Royal Statistical Society, Ser. B 62 303–322. [Google Scholar]
- [6].Fan J and Zhang W (2008). Statistical methods with varying coefficient models. Statistics and Its Interface 1 179–195. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [7].Fan Y, Foutz N, James GM and Jank W (2014). Functional Response additive model estimation with online virtual stock markets. Annual Review of Statistics and Its Application 8 32435–2460. [Google Scholar]
- [8].Goldsmith J, Greven S and Crainiceanu C (2013). Corrected confidence bands for functional data using principal components. Biometrics 69 41–51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [9].Goldsmith J, Zipunnikov V and Schrack J (2015). Generalized multilevel function-on-scalar regression and principal component analysis. Biometrics 71 344–353. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [10].Guo W (2002). Functional mixed effects models. Biometrics 58 121–128. [DOI] [PubMed] [Google Scholar]
- [11].Hastie T and Tibshirani R (1993). Varying coefficient models. Journal of the Royal Statistical Society, Ser. B 55 757–796. [Google Scholar]
- [12].Hilgert N, Manrique Chuquillanqui T, Crambes C and Mas A (2015). Ridge regression for the functional concurrent model. Presented at 60. World Statistics Congress ISI2015, Rio de Janeiro, BRA (2015–07-26 – 2015-07–31) http://prodinra.inra.fr/record/310110. [Google Scholar]
- [13].Hoover DR, Rice JA, Wu CO and Yang LP (1998). Nonparametric smoothing estimates of time-varying coefficient models with longitudinal data. Biometrika 85 809–822. [Google Scholar]
- [14].Huang JZ, Wu CO and Zhou L (2004). Polynomial spline estimation and inference for varying coefficient models with longitudinal data. Statistica Sinica 14 763–788. [Google Scholar]
- [15].Huang L, Scheipl F, Goldsmith J, Gellar J, Harezlak J, McLean MW, Swihart B, Xiao L, Crainiceanu C and Reiss P (2015). refund: Regression with Functional Data R package version 0.1–13.
- [16].Ivanescu AE, Staicu A-M, Scheipl F and Greven S (2015). Penalized function on function regression. Computational Statistics 30 539–568. [Google Scholar]
- [17].Jiang CR and Wang JL (2011). Functional single index models for longitudinal data. The Annals of Statistics 39 362–388. [Google Scholar]
- [18].Li Y and Ruppert D (2008). On the asymptotics of penalized splines. Biometrika 95 415–436. [Google Scholar]
- [19].Malfait N and Ramsay JO (2003). The historical functional linear model. The Canadian Journal of statistics 31 115–128. [Google Scholar]
- [20].Marx BD and Eilers PHC (2005). Multivariate penalized signal regression. Technometrics 47 13–22. [Google Scholar]
- [21].McLean MW, Hooker G, Staicu AM, Scheipl F and Ruppert D (2014). Functional generalized additive models. Journal of Computational and Graphical Statistics 23 249–269. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [22].Morris JS (2015). Functional Regression. Annual Review of Statistics and Its Application 2 321–359. [Google Scholar]
- [23].Morris JS and Carroll RJ (2006). Wavelet-based functional mixed models. Journal of the Royal Statistical Society: Series B 68 179–199. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [24].Olshen RA, Biden EN, Wyatt MP and Suther-land DH (1989). Gait analysis and the bootstrap. The Annals of Statistics 17 1419–1440. [Google Scholar]
- [25].Qu A and Li R (2006). Quadratic inference functions for varying coefficient models with longitudinal data. Biometrics 62 379–391. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [26].Ramsay JO, Hooker G and Graves S (2009). Functional data analysis in R and Matlab. New York, NY: Springer. [Google Scholar]
- [27].Ramsay JO and Silverman BW (2005). Functional data analysis, 2nd ed. New York ; Berlin: : Springer. [Google Scholar]
- [28].Reiss PT, Huang L and Mennes M (2010). Fast function-on-scalar regression with penalized basis expansions. International Journal of Biostatistics 6. [DOI] [PubMed] [Google Scholar]
- [29].Ruppert D, Wand MP and Carroll RJ (2003). Semi-parametric regression. Cambridge, New York: Cambridge University Press. [Google Scholar]
- [30].Scheipl F, Staicu AM and Greven S (2015). Functional additive mixed models. Journal of Computational and Graphical Statistics 24 477–501. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [31].Senturk D and Nguyen DV (2011). Varying coefficient models for sparse noise-contaminated longitudinal data. Statistica Sinica 21 1831–1856. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [32].Shen Q and Faraway J (2004). An F test for linear models with functional responses. Statistica Sinica 14 1239–1257. [Google Scholar]
- [33].Slijepcevic A, Anderson WR and Matthews S (2013). Testing existing models for predicting hourly variation in fine fuel moisture in eucalypt forests. Forest Ecology and Management 306 202–215. [Google Scholar]
- [34].Slijepcevic A, Anderson WR, Matthews S and Anderson DH (2015). Evaluating models to predict daily fine fuel moisture content in eucalypt forest. Forest Ecology and Management 335 261–269. [Google Scholar]
- [35].Theologis TN (2009). Children’s Orthopaedics and Fractures (Chapter 6). London: Springer. [Google Scholar]
- [36].Wood SN (2006). Generalized Additive Models: An Introduction with R. Boca Raton, Florida: Chapman and Hall/CRC. [Google Scholar]
- [37].Wood SN (2015). Mixed GAM computation vehicle with GCV/AIC/REML smoothness estimation R package version 1.8–7.
- [38].Wood SN, Pya N and Safken B (2016). Smoothing parameter and model selection for general smooth models. Journal of the American Statistical Association in press. [Google Scholar]
- [39].Wu CO, Chiang CT and Hoover DR (1998). Asymptotic confidence regions for kernel smoothing of the varying coefficients model. Journal of the American Statistical Association 93 1388–1402. [Google Scholar]
- [40].Wu Y, Fan J and Muller HG¨ (2010). Varying-coefficient functional linear regression. Bernoulli 16 730–758. [Google Scholar]
- [41].Xiao L, Zipunnikov V, Ruppert D and Crainiceanu C (2016). Fast covariance function estimation for high-dimensional functional data. Statistics and Computing 26 409–421. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [42].Xu H, Shen Q, Yang X and Shoptaw S (2011). A quasi F-test for functional linear models with functional covariates and its application to longitudinal data. Statistics in Medicine 30 2842–2853. [DOI] [PubMed] [Google Scholar]
- [43].Yao F, Muller HG¨ and Wang JL (2005). Functional linear regression analysis for longitudinal data. The Annals of Statistics 33 2873–2903. [Google Scholar]
- [44].Yao F, Muller HG¨ and Wang JL (2005). Functional data analysis for sparse longitudinal data. Journal of the American Statistical Association 100 577–590. [Google Scholar]
- [45].Zhang J, Clayton MK and Townsend PA (2011). Functional Concurrent Linear Regression Model for Spatial Images. Journal of Agricultural, Biological, and Environmental Statistics 16 105–130. [Google Scholar]
- [46].Zhang JT and Chen J (2007). Statistical inference for functional data. The Annals of Statistics 35 1052–1079. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.




