Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Apr 1.
Published in final edited form as: J Comput Graph Stat. 2014 Apr 4;24(2):477–501. doi: 10.1080/10618600.2014.901914

Functional Additive Mixed Models

Fabian Scheipl 1, Ana-Maria Staicu 2, Sonja Greven 1
PMCID: PMC4560367  NIHMSID: NIHMS589200  PMID: 26347592

Abstract

We propose an extensive framework for additive regression models for correlated functional responses, allowing for multiple partially nested or crossed functional random effects with flexible correlation structures for, e.g., spatial, temporal, or longitudinal functional data. Additionally, our framework includes linear and nonlinear effects of functional and scalar covariates that may vary smoothly over the index of the functional response. It accommodates densely or sparsely observed functional responses and predictors which may be observed with additional error and includes both spline-based and functional principal component-based terms. Estimation and inference in this framework is based on standard additive mixed models, allowing us to take advantage of established methods and robust, flexible algorithms. We provide easy-to-use open source software in the pffr() function for the R-package refund. Simulations show that the proposed method recovers relevant effects reliably, handles small sample sizes well and also scales to larger data sets. Applications with spatially and longitudinally observed functional data demonstrate the flexibility in modeling and interpretability of results of our approach.

Keywords: Functional data analysis, functional principal component analysis, P-splines, Smoothing, Varying coefficient models


In recent years, many scientific studies have collected functional data that exhibit correlation structures amenable to explicit modeling. Such structures may arise from a longitudinal study design (e.g. Goldsmith et al., 2012; Greven et al., 2010; Morris and Carroll, 2006), crossed designs (e.g. Aston et al., 2010), or spatial sampling of curves (e.g. Delicado et al., 2010; Giraldo et al., 2010; Gromenko et al., 2012; Nerini et al., 2010; Staicu et al., 2010). Simultaneously, regression for independent functional responses (e.g. Faraway, 1997) has made large advances, including both multiple scalar (e.g. Reiss et al., 2010) and multiple functional predictors in concurrent or more general relationships (e.g. Ivanescu et al., 2012). Our work is motivated by a longitudinal neuroimaging study containing repeated measurements of a functional proxy variable for neuronal health along 3 white matter tracts derived from diffusion tensor imaging (DTI). The goal of our analysis is to quantify the relationship of these function-valued proxy measures while accounting for the longitudinal correlation structure as well as the effects of patient characteristics like age and gender. This DTI study is an example of a longitudinal functional data set where models must account for the correlation structure of the data while including both scalar and functional covariates in the predictor.

To address these challenges, we propose conditional regression models for functional responses that accommodate general correlation structures via functional and scalar random effects as well as flexible linear or nonlinear effects of scalar and functional covariates. The major contributions of this paper thus consist in 1) developing a general inferential framework for additive mixed models for correlated functional responses that accommodates diverse correlation structures and flexible modeling of the mean structure extending Ivanescu et al. (2012), 2) unifying two previously separate strands of prior work, by subsuming both functional principal component- (FPC) and spline-based approaches, and 3) evaluating our implementation available in the the R-package refund (Crainiceanu et al., 2011) on real and simulated data.

Our goal is to describe and implement a framework that offers analysts of functional data similar flexibility in model specification to what is available in current implementations of (geo-)additive mixed models for scalar data. Specifically, we consider structured additive regression models of the general form

yi(t)=r=1Rfr(Xri,t)+εi(t), (1)

for functional responses yi(t),i = 1,...,n, observed over a domain I. Each term in the additive predictor is a function of a) the index t of the response and b) a subset Xr of the complete covariate set X including scalar and functional covariates and (partially) nested or crossed grouping factors. To make this more concrete, Table 1 shows the most important combinations of Xr and effect shapes available in our framework. We assume a white noise error process independent of X, such that the εi(t) are independent and identically distributed (i.i.d.) Gaussian variables with mean zero and constant variance σε2 across I. Additionally, smooth and potentially correlated error curves can be included via curve-specific random effects to model (co-)variance along t and dependence between functional observations. We assume all effects in Table 1 to be smooth but unknown functions in the covariates, and this smoothness assumptions on all components of the predictor ensures smoothness of yi(t) up to the white noise measurement error εi(t). Scalar random effects bg are mean zero Gaussian variables with general covariance structure between the different levels of g. Functional random effects bg(t) for a grouping variable g with M levels are modeled as realizations of a mean-zero Gaussian random process on {1,,M}×I with a general covariance function Kb(m,m′,t,t′) = Cov(bm(t),bm′(t′)) that is smooth in t, where m,m′ denote different levels of g. Note that our model class and software admits multiple partially or completely nested or crossed grouping factors for both scalar and functional random effects, but different random effects bg(t),bg′(t) are assumed to be mutually independent. We assume integrability for the effects of functional covariates. Our implementation for functional effects such as xi(s)β(s,t)ds also accommodates varying integration ranges with fixed, potentially observation-specific integration limits li(t),ui(t), similar to the historical functional model in Malfait and Ramsay (2003). Densely as well as sparsely observed functional responses and suitably preprocessed functional predictors with measurement error can be used in this framework. We approximate each term fr(Xr,t) by a linear combination of basis functions defined by the tensor product of marginal bases for Xr and t. Since basis dimensions have to be sufficiently large to ensure enough flexibility, maximum likelihood estimation of model (1) is likely to lead to substantial overfitting. The penalized likelihood approach described in Section 1.2 stabilizes estimates by suppressing variability not strongly supported by the data and finds a data-driven compromise between goodness of fit and simplicity of the fitted effects.

Table 1.

Forms of fr(Xr,t) depending on the covariates in Xr and linearity or smoothness in these covariates (rows), and on whether the effect is constant or varying over t (columns). For scalar categorical covariates, synthetic scalar covariates in effect or reference category coding are created. Note that effects can become interaction effects if Xr additionally contains such scalar categorical covariates. For example, we estimate group-specific effects of the functional covariates for MS patients and healthy controls in our DTI application.

Xr fr(Xr,t) constant over t fr(Xr,t) varying over t

∅ (no covariates) scalar intercept α functional intercept α(t)
functional covariate x(s) linear functional effect Yx(s)β(s)ds linear functional effect Yx(s)β(s,t)ds
smooth functional effect YF(x(s),s)ds smooth functional effect YF(x(s),s,t)ds
scalar covariate z linear effect functional linear effect (t)
smooth effect γ(z) smooth effect γ(z, t)
vector of scalar covariates z interaction effect z1z2δ functional interaction effect z1z2δ (t)
varying coefficient z1δ(z2) functional varying coefficient z1δ(z2, t)
smooth effect γ(z) smooth effect γ(z,t)
grouping variable g random intercept bg functional random intercept bg(t)
grouping variable g and scalar covariate z random slope zbg functional random slope zbg(t)

Most existing work on functional random effects has considered only special cases such as the functional random intercept model (Abramovich and Angelini, 2006; Di et al., 2009; Krafty et al., 2011), functional random intercept and slope model (Greven et al., 2010), a single level of random effects functions (Antoniadis and Sapatinas, 2007; Guo, 2002; Qin and Guo, 2006), or a two or three-level hierarchy (Baladandayuthapani et al., 2008; Bigelow and Dunson, 2007; Brumback and Rice, 1998; Li et al., 2007; Morris et al., 2003; Scarpa and Dunson, 2009; Staicu et al., 2010; Zhou et al., 2010). Aston et al. (2010) consider a general functional random effects structure under the assumption of a joint functional principal component (FPC) basis for all functional random effects in the model, which are estimated under a working independence assumption between curves. It is unclear, however, how well this approach works if the latent processes do not share the same eigenfunctions and how the correlation between functional observations affects FPC estimation. FPC estimation for correlated observations is a topic of ongoing research (c.f. Hörmann and Kokoszka, 2010, 2011; Panaretos and Tavakoli, 2013a,b). Morris and Carroll (2006); Morris et al. (2003); Zhu et al. (2011) propose a general Bayesian functional linear mixed model based on a wavelet transformation of (usually very spiky) data observed on an equidistant grid. The model proposed by Morris and Carroll (2006) includes correlation between different random effects and heterogeneous residual errors, which we do not. Our approach, on the other hand, is well suited to smooth underlying curves and allows a more general mean structure than previous functional linear mixed models; in particular we are able to estimate smooth nonlinear or linear effects of scalar and/or functional covariates within the same framework. In addition, we are able to handle data on non-equidistant or sparse grids.

To the best of our knowledge, our proposal is the first publicly available implementation that allows such a high level of flexibility for a functional regression model – prior work either limits the predictor to the effect of a single functional covariate and a functional intercept, such as the linmod function in package fda (Ramsay et al., 2011) for the R language (R Development Core Team, 2011) or to linear effects of scalar covariates, such as the fosr function for function-on-scalar regression (Reiss et al., 2010) in the R-package refund. Like the linear function-on-function regression approach in Ivanescu et al. (2012) we build on, both approaches are limited to independent functional responses. Morris and Carroll (2006) provide a closed source implementation for wavelet-based functional linear mixed models in WFMM (Herrick, 2013) that allows very general random effect and residual structures, but implement neither effects of functional covariates nor nonlinear effects of scalar covariates. The PACE package (Fang et al., 2013) for MATLAB implements FPC based regression models where the predictor is limited to the effect of a single functional or scalar covariate. Our proposal has some similarities with the regression models for independent or longitudinal scalar responses in Goldsmith et al. (2011, 2012), implemented in the pfr and lpfr functions in refund, since we also base inference on additive mixed models for scalar-on-scalar regression. However, the extension to functional responses and functional random effects with flexible correlation structure as well as the inclusion of FPC-based effects is non-trivial.

The paper is organized as follows: Section 1 develops our general approach and estimation framework for functional additive mixed models. Our method is evaluated in a simulation study and in an application to the motivating longitudinal DTI study in Section 2. Section 3 closes with a discussion and outlook.

1 Penalized regression for correlated functional data

Functional responses yi(t) are observed on a grid of Ti points ti=(ti1,,tiTi)T. To simplify notation, we assume identical grids tit=(t1,,tT)T for i = 1,...,n in the following, but note that functional responses observed on irregular and/or sparse grids are naturally accommodated in the rephrased model formulation given in (2). Then, model (1) can be expressed as

yil=r=1Rfr(Xri,tl)+εil (2)

for i = 1,...,n and l = 1,...,T. The assumption of white noise errors translates to εili.i.d.N(0,σε2). The smoothness assumption on E(yi(t)) is preserved implicitly by enforcing smoothness across I for all fr(Xr,t). To fit the model, we form y=(y1T,,ynT)T, an nT-vector that holds the concatenated function evaluation vectors yi=(yi1,,yiT)T. In the following, let Xr denote the vector or matrix containing rows of observations Xri. Let f(t) denote the vector of function evaluations of f for each entry in the vector t and let f(x,t) denote the vector of evaluations of f for each combination of rows in the vectors or matrices x,t. Let 1d=(1,,1)T denote a d-vector of ones. The row tensor product of an m × a matrix A and an m × b matrix B is defined as the m × ab matrix AB=(A1bT)(1aTB), where · denotes element-wise multiplication.

1.1 Tensor product representation of effects

Each of the R terms in model (2) can be represented as a weighted sum of basis functions defined on the product space of the covariates in Xr and t, where each marginal basis is associated with a corresponding marginal penalty. A very versatile method to construct basis functions on such a joint space is given by the row tensor product of marginal bases evaluated on Xr and t (e.g. De Boor, 1978; Wood, 2006, ch. 4.1.8). Specifically, for each of the terms,

fr(XrnT×1,t)(ΦxrnT×KxΦtrnT×Kt)θrKxKt×1=Φrθr, (3)

Φxr contains the evaluations of a suitable marginal basis for the covariate(s) in Xr and Φtr contains the evaluations of a marginal basis in t. The shape of the function is determined by the vector of coefficients θr. A corresponding penalty term can be defined by the Kronecker sum of the marginal penalty matrices Pxr and Ptr associated with each basis (Wood, 2006, ch. 4.1), i.e.

pen(θrλtr,xr)=θrT(λxrPxrKx×KxIKt+λtrIKxPtrKt×Kt)θr=θrTPr(λtr,λxr)θr. (4)

Pxr and Ptr are known and fixed positive (semi-)definite penalty matrices and λtr and λxr are positive smoothing parameters controlling the trade-off between goodness of fit and the smoothness of fr(Xr,t) in Xr and t, respectively. This flexible construction is valid for any combination of bases associated with quadratic penalties. Alternative constructions of the joint penalty such as a direct Kronecker product λr(PxrPtr) associated with a single smoothing parameter λr are possible, see Wood (2006, ch. 4.1.8) for a discussion. Typically, Kx and Kt vary with r as well, but we drop the additional index for simplicity. In the following paragraphs, we will motivate and define Φxr, Φtr,Ptr and Pxr for the different types of terms available in our implementation. Effects that are constant over t are associated with Φtr = 1nT and Ptr = 0, while users are free to choose any suitable marginal basis matrix Φtr and penalty Ptr for terms that vary over t.

Spline basis representation of effects of scalar covariates

For scalar covariates, index-varying effects are very similar to varying coefficient terms in models for scalar responses, c.f. Ivanescu et al. (2012). For the functional intercept α(t), Φxr = 1nT and Pxr = 0. For effects like zδ and zδ(t) that are linear in a scalar covariate z, the marginal basis for the covariate direction reduces to Φxr=z1T where z=(z1,,zn)T, with penalty Pxr = 0. For nonlinear effects of scalar covariates like γ(z) or γ(z,t), Φxr is a suitable marginal spline basis matrix over z and Pxr is the associated penalty.

Spline basis representation of functional effects

For linear effects of functional covariates x(s), we model β(s,t) using tensor product splines with basis functions Φks(s),ks = 1,...,Kx, over Y and a spline basis defined over I. We approximate the integral by numerical integration on the grid defined by the observation points s1,...,sH in Y. The effect in (2) then is

Yxi(s)β(s,tl)dsh=1Hwhxi(sh)ks=1Kxkt=1KtΦks(sh)Φkt(tl)θr,ks,kt.

In the notation of (3), Φxr=[xdiag(w)Φs]1T[Yxj(s)Φks(s)ds]i=1,,nks=1,,Kx1T, where w=(w1,,wH)T contains the quadrature weights for a numerical integration scheme, x=[xi(sh)]i=1,,nh=1,,H and Φs=[Φks(sh)]h=1,,Hks=1,,Kx. Pxr in the tensor product penalty (4) is the penalty associated with the Φks(s). We can extend this construction, which is equivalent to the one introduced in Ivanescu et al. (2012), to cover terms like li(t)ui(t)xi(s)β(s,t)ds (e.g. Malfait and Ramsay, 2003) with fixed, potentially observation-specific integration limits li(t),ui(t)Y. This is achieved by defining suitable weight matrices wi,l with zero entries for sh < li(tl) and sh > ui(tl). Such effects will often be required for covariates and responses that are observed on the same time domain, where responses cannot be influenced by future covariate values. In the limit, this also includes the concurrent model with terms X(t)β(t).

Our framework also extends to non-linear function-on-function effects YF(xi(s),s,t)ds, which generalize the functional generalized additive model (McLean et al., 2012) from scalar to functional responses. They offer similar flexibility to purely nonparametric approaches like Febrero-Bande and Oviedo de la Fuente (2012); Ferraty et al. (2011); Ferraty and Vieu (2006), e.g. In our framework, such terms can be represented as

YF(xi(s),s,tl)dsh=1Hwhks=1Kxkt=tKtΦks(xi(sh),sh)Φkt(tl)θkskt

with Φxr=[(wTIn)Φs]1T and Φs=[Φks(xi(sh),sh)]i=1,,n,h=1,,Hks=1,Kx, and Pxr the penalty associated with Φs. Basis functions Φks(x(s),s) can be tensor product basis functions derived from marginal bases for x(s) and s or true bivariate basis functions.

FPC basis representation of functional effects

Consider a functional covariate expanded in the Karhunen-Loève expansion xi(s)=kψk(s)ξik with ψk(s)ψk(s)ds=δkk; E(ξik)=0; Var(ξjk)=ζk. Under the assumption that Yk>Kxψk(s)ξikβ(s,t)ds0 for some Kx, i.e., that all smaller modes of variation of x(s) only have a negligible effect on y(t), we can write

Yxi(s)β(s,t)dsYk=1Kxψk(s)ξikβ(s,t)ds=k=1Kxξikβ~k(t)

with β~k(t)=Yψk(s)β(s,t)ds. Thus, a linear function-on-function effect can be represented as a sum of varying coefficient terms for the FPC loadings ξik. This representation extends FPC regression approaches (e.g. Reiss and Ogden, 2007) from scalar to functional responses. In the notation of the general framework, Φxr=[ξik]i=1,,nk=1,,Kx1T and Pxr = 0. For the t-direction, Φtr and the associated Ptr can be chosen freely. An implicit assumption here is that all βk(t),k= 1,...,Kx have similar smoothness, as they are all associated with the same smoothing parameter.

This FPC-based approach may be advantageous for functional covariates observed on irregular or sparse grids – for such data, the spline-based method requires a preprocessing step (c.f. Goldsmith et al., 2011, 2012; James, 2002) to impute the incomplete trajectories on a dense and regular grid, whereas FPCs can be estimated directly from sparse data (Yao et al., 2005). Additionally, if the shapes of the ψ^k(s) are meaningful to practitioners and Kx is small, the coefficient functions β~k(t) may be easier to interpret than a coefficient surface β(s,t). On the other hand, since inference is performed conditional on the estimated FPCs ψ^k(s) and associated loadings ξ^ik coverage issues associated with these neglected sources of estimation variability (c.f. Goldsmith et al., 2013) and bias introduced by estimation error in the FPC analysis step may occur. Additionally, Yk>Kxψk(s)ξikβ(s,t)ds0 might be a strong assumption that is hard to check in applications, as is the choice of the discrete tuning parameter Kx.

As in the spline-based case, we can extend this to FPC-based nonlinear function-on-function effects. The proposal corresponds to an extension of the functional additive model by Müller and Yao (2008) from scalar to functional responses, with the effect of the functional covariate given by fr(xi(s),t)=k=1KxFk(ξ^ik,t). In the notation of our general framework, Φxr=[Φξ1ΦξKx] where Φξk=[Φa(ξ^ik)]i=1,,na=1,,A1T for suitable spline basis functions Φa(·), such that Kx=AKx. The marginal penalty is given by Pxr=IKxPξ, where Pξ is the penalty associated with the Φa(ξ^jk). In t-direction, we are again free to choose any suitable basis Φtr and penalty Ptr. Further extensions to interaction effects of estimated FPC scores fr(xi(s),t)=k=1Kxk<kKxFk,k(ξ^ik,ξ^ik,t) are also obvious in this framework.

Spline basis representation of functional random effects

Functional random effects bg(t) are represented as smooth functions in t for each level 1,...,M of the grouping variable g. In the notation of equation (3), functional random intercepts are associated with a marginal basis Φxr=[δg(i)m]i=1,,nm=1,,M1T, where g(i) denotes the level of g for observation i. This yields an incidence matrix mapping the observations to the different levels of the grouping variable. For a functional random slope effect in a scalar covariate z, Φxr=[ziδg(i)m]i=1,,nm=1,,M1T. In the notation of equation (4), the marginal penalty Pxr for functional random effects is a M × M precision matrix that defines the dependence structure between the levels of g. The quadratic penalty (4) is mathematically equivalent to the distributional assumption θrN(0,σε2Pr(λtr,λxr)1) (Brum-back et al., 1999). Through the representation in (3), this induces a mean zero Gaussian process assumption bg(t) ~ GP(0,Kb(g(i),g(i′),t,t′)), with covariance evaluated for all nT observations Kb(g,g,t,t)=σε2ΦrPr(λtr,λxr)1ΦrT. The smoothing parameter λxr controls the relative contribution of the inter-unit variability relative to the common roughness of the functional random effects controlled by λtr.

If observations on different levels of the grouping factor are assumed independent, Pxr = IM is simply the identity matrix. More generally, Pxr can represent any fixed dependence structure between levels of g: It can be a (partially improper) precision matrix of a random field with known correlation structure, implied, for example, by the spatial or temporal arrangement of the different levels of g, such as a Gaussian Markov random field (GMRF) on geographical regions for conditionally auto-regressive (CAR) model terms. Alternatively, (Pxr)−1 can be defined using any valid correlation function based on – for example – spatial, temporal, or genetic distances between levels of g. If the grouping variable is simply the index of observations (i.e., g(i) = i), this construction yields smooth residual curves with potential for spatial or temporal autocorrelation. This innovative definition of functional random effects admits very flexible model specifications, since any combination of spline basis, smoothness penalty and between-subject correlation can be used for functional random effects. This allows, for example, for spatially correlated functional residuals with periodicity constraints for the Canadian Weather data (see Appendix C of the online supplement). Multiple (partially) nested or crossed random effects can be constructed in this way and are implemented in pffr().

FPC basis representation of functional random intercepts

For functional random intercepts without between-unit correlation, i.e., for bg(t)i.i.d.GP(0,Kb(t,t)), it can be advantageous to use the eigenfunctions of the covariance operator Kb(t,t′) as basis functions in t. Specifically, we use the Karhunen-Loève expansion of random processes to represent bg(t)k=1Ktηk(t)νgk with κk,ηk(t) the ordered eigenvalues and -functions of Kb(t,t′), νgk the associated FPC loadings, and Kt a suitable truncation lag. The marginal basis for the t-direction is then Φtr=1n[η^1(t)ηKt(t)]. Since E(νgk) = 0 and Var(νgk) = κk, a reasonable marginal penalty is Ptr=diag(κ^1,,κ^Kt)1. This encourages relative contributions of the FPCs to the random effect curves that are roughly proportional to their estimated magnitudes κ^k. As for the spline-based functional random effects, Φxr=[δg(i)m]i=1,,mm=1,,M1T is an incidence matrix for the group levels, while Pxr = IM.

In practice, ηk(t) and κk have to be estimated. An iterative procedure can be outlined as follows: (1) Use a fit without functional random effects under independence assumption to obtain working residuals E=[εil]i=1,,nl=1,,T. (2) Compute the group-level means of residual curves E=ΔTE with Δ=[δg(i)mnm]i=1,,nm=1,,M, where nm is the number of observations for the m-th level of g. (3) Perform a (truncated) spectral decomposition of K^b=[K^b(tl,tl)]l,,l=1,,T to obtain η^k(t), κ^k for k = 1, ..., Kt. A suitable estimate for K^b can be derived from smoothing the entries in the matrix M1ETE (without the diagonal) as in Yao et al. (2005). This approach for estimating K^b can only be used for random intercepts for a single grouping variable. Compared to spline-based functional random effects, FPC-based modeling holds the promise of using the optimal, most parsimonious basis to represent bg(t). Computationally, it is expected to scale much better for large M, as the number of coefficients associated with a functional random effect is MKt and Kt for FPCs will typically be much smaller than in a sufficiently flexible spline basis. On the other hand, the FPC approach requires a pilot estimate for Kb. The subsequent performance will be sensitive to the quality of the estimation of the FPCs and to the choice of Kt.

1.2 Mixed model representation

Using the tensor product representation introduced in the previous subsection for all terms, model (1) can be re-written as

y=Φθ+;N(0,σε2InT), (5)

where Φ = [Φ1|...|ΦR] contains the concatenated Φr associated with the different model terms and θ=(θ1T,,θRT)T the respective stacked coefficient vectors θr. To clear up notation, we assign a sequential index v = 1,...,V to the smoothing parameters λxr,λtr in (4), where V is the total number of smoothing parameters in the model. We pad PtrIKx and IKtPxr with rows and columns of zeros, denoting these matrices v1 and v2 such that the penalty θrT(λtrPtrIKx+λxrIKPxr)θr=λv1θTP~v1θ+λv2θTP~v2θ refers to the full coefficient vector θ. The penalized likelihood criterion to be minimized then becomes

1σε2yΦθ2+v=1Vλvσε2θTP~vθ. (6)

The total number of smoothing parameters is V ≤ 2R, as some terms are constant over t or Xr and the corresponding v are zero. Let τv=σε2λv and use similar arguments as in Ruppert et al. (2003, ch. 4.9) to obtain the solution θ^ of (6) as the best linear unbiased predictor in the linear mixed effects model (MEM)

yN(Φθ,σε2InT);θN(0,(v=1Vτv1P~v)), (7)

where S denotes the generalized inverse of S, and N(0,S) is a partially improper Gaussian distribution with positive semi-definite covariance matrix S. The impropriety results from rank deficiencies in some of the v, since roughness penalties typically define a nullspace of maximally smooth functions. Numerical difficulties posed by the positive semi-definiteness are solved by another re-parameterization that separates the various model terms into their unpenalized and penalized components, i.e. into “fixed” effects and “random” effects with a proper distribution, respectively. These are well known issues in the literature on penalized regression splines described in detail e.g. in Ruppert et al. (2003, ch. 4.9),Wood (2006, ch. 6.6.1); recent developments for tensor product splines are in Wood et al. (2013).

One of the main advantages of formulating the penalized likelihood optimization as estimation in an MEM is that the smoothing parameters λv=σε2τv can be treated as variance component parameters and thus can be estimated using restricted maximum likelihood (REML). In particular, Reiss and Ogden (2009) and Wood (2011) have shown that smoothing parameter selection with REML is more stable and results in somewhat lower MSE than generalized cross-validation (GCV) and Krivobokova and Kauermann (2007) have shown that REML estimation of penalized splines is more robust to error correlation mis-specification than AIC-based criteria. A second advantage this approach offers is that the representation of our model class (1) results in a fit criterion (7) equivalent to that of conventional additive mixed models for scalar data. This means much of the powerful and versatile inference machinery developed for scalar linear and additive mixed models (AMMs) over the last years can be applied directly to the proposed model class of functional AMMs, due to their close structural similarity. Specifically, 1) pointwise, bias-corrected confidence bands (Marra and Wood, 2012; Nychka, 1988; Ruppert et al., 2003) are available for the functional effects, 2) tests for random effects as well as tests for constant or linear effects versus more general alternatives developed for scalar responses (Crainiceanu and Ruppert, 2004; Crainiceanu et al., 2005; Greven et al., 2008; Scheipl et al., 2008; Wood, 2013), and 3) model selection approaches that have recently been proposed for scalar-response AMMs (Greven and Kneib, 2010; Marra and Wood, 2011) are directly applicable to the proposed model class. Finally, the proposed approach accommodates a large variety of effects, at no increase in the level of complexity of the algorithm itself. The tensor product representation given in Section 1.1 combined with the MEM representation (7) allows for a unified framework for smoothness parameter selection and estimation of all model components in model (1), including functional random effects and FPC-based effects.

Implementation

The full framework for functional additive models we describe here is implemented in the pffr-function in the refund package for R. The underlying inference engine is the mgcv package (Wood, 2011) for generalized additive models which also supplies most of the functionality for constructing basis and penalty matrices. pffr offers a formula-based interface similar to the established formula syntax of mgcv and returns a rich model object whose fit can be summarized, plotted and compared with other model formulations without any programming effort by the user through convenient utility functions.

2 Empirical evaluation

The following section describes an extensive simulation study and results for the motivating application to a longitudinal DTI study. A fully reproducible example analysis of the well known Canadian Weather data showcasing the flexibility of pffr can be found in Section C of the online appendix.

2.1 Simulation study

Simulation setup

We simulate data with repeated measures structure for a model yij(t)=r=1Rfr(Xrij,t)+εij(t) for the following four scenarios to investigate the sensitivity of the estimates to varying model complexity, noise levels and number of observations:

  1. Functional random intercept, functional random slope: fr(Xrij,t)=α(t)+bi0(t)+bi1(t)uij

  2. Functional random intercept, functional covariate: fr(Xrij,t)=α(t)+x1,ij(s)β1(s,t)ds+bi0(t)

  3. Functional random intercept, two functional covariates: fr(Xrij,t)=α(t)+x1,ij(s)β1(s,t)ds+x2,ij(s)β2(s,t)ds+bi0(t)

  4. Functional random intercept, functional covariate, smooth scalar covariate effect, varying coefficient term: fr(Xrij,t))=α(t)+x1,ij(s)β1(s,t)ds+γ1(z1,ij,t)+δ2(t)z2,ij+bi0(t)

Definitions of the various effect functions and descriptions of the data generating processes used for the covariates can be found in section B of the online supplement, along with unabridged simulation results and graphical displays of data and estimated effects for the replications with minimal, maximal and median error for each scenario.

For each of the four scenarios, we run 10 replications for each combination of the following settings, yielding 1920 model fits in total:

  • – number of subjects: M ∈ {10,100}

  • – mean number of observations per subject: ni {3,20}. Subject labels i ∈ {1,...,M} are drawn from a multinomial distribution with probabilities P(i=i)i to generate unbalanced designs.

  • – number of grid points for t: T ∈ {30,60}

  • – relative importance of random effects: SNRB {0.2,1,5}, where SNRB is the ratio of the standard deviation of the additive predictor without random effects divided by the standard deviation of the random effect functions; e.g. for SNRB = 5, the contribution of each functional random effect to the variability in y(t) is about 5 times smaller than that of the non-random effects.

  • – signal-to-noise ratio: SNRε {1,5}, where SNRε is the ratio of the standard deviation of the additive predictor divided by the standard deviation of the residuals σε.

Our results show that 10 replications for each combination are sufficient to derive precise estimates of effects of the setting parameters on estimation errors and computation times, c.f. Figures 1 and 2. Fits are obtained with the defaults in pffr(), i.e., cubic B-spline bases with 20 basis functions and first order difference penalty for the functional intercept, tensor products of cubic B-spline bases with five marginal basis functions for the tensor product terms, with first order difference penalties for the t- and s-directions and second order difference penalties for the covariate direction (if applicable). The smoothing parameters are REML estimates as returned by mgcv. The models for settings 1 to 4 include K = 120 to 1020 coefficients and 5 to 8 smoothing parameters.

Figure 1.

Figure 1

Baseline levels and estimated multiplicative change in rIMSE for the 4 scenarios. The scenarios are depicted with different symbols, and the segments accompanying the symbols correspond to the estimated effect ± 2 standard errors. Effects other than b0(t) only occur in a subset of scenarios. Horizontal axis on log2.

Figure 2.

Figure 2

Computation times for scenarios 1 to 4 (from left to right). Vertical axis on log10-scale. Horizontal axis for the various combinations of numbers of subjects M and average number of replications per subject ni. Results for T = 30 in dark grey and in light grey for T = 60. Timings are wall-clock time taken on an 2.2 GHz AMD Opteron 6174.

Estimation error

We use the relative integrated mean squared error defined as rIMSE(f^r(Xr,t))=n1i=1n(f^r(Xri,t)fr(Xri,t))2dtfr(Xri,t)2dt to evaluate the accuracy of the estimates. Relative errors allow comparisons across different scenarios and noise levels regardless of the ranges of the true fr(Xr,t). Note that we evaluate the estimation accuracy of the effects on the scale of the response, not on the scale of the coefficient function itself to make errors directly comparable across effects. Detailed analysis of results (see Appendix B in the supplement) shows that there are no relevant interaction effects between the setting parameters M, ni, T, SNRB and SNRε on the observed errors within scenarios, so we fit log-linear models with main effects for the setting parameters in each scenario to observed rIMSE values and proceed to analyze the estimated effects. Figure 1 shows baseline levels and the estimated multiplicative effects of the simulation parameters on the rIMSEs. The effect of increasing the number of grid points T for y(t) from 30 to 60 is not shown, as it decreased relative errors for all quantities by a factor of about 0.7 to 0.5. Baseline rIMSE values (top left panel) are given for data with SNRε = 1, M = 10, ni = 3, T = 30, SNRB = 0.2. In this very noisy setting with small sample size and dominant random effects, covariate effect estimates are not very accurate, with relative errors mostly in the vicinity of one. Since the random effects are estimated with little error, however, the error for the responses in this difficult setting is small as well. Increasing SNRε from 1 to 5 (top right panel) decreases relative errors about 16-fold, with smaller 8-fold reductions for the random effects in scenario 1. Increasing the number of groups from M = 10 to M = 100 (second row, left panel) has no substantial effect on the overall estimation accuracy of the yi(t). Estimation accuracy of the functional random effects is not improved either due to the commensurate increase in the number of parameters, while errors for the covariate effects decrease about 8-fold. An increase in the average number of observations per group from ni = 3 to ni = 20 (second row, right panel) results in a similar reduction of relative errors for the covariate effects, and also a marked four- to sixfold decrease in the errors for the response trajectories. A reduction of the relative contribution of the random effects to the linear predictor, i.e. increasing SNRB from 0.2 to 1 [5] (bottom row, left [right] panel), improves the overall estimation accuracy of y(t) only slightly if at all [factor 0.7 to 0.8]. This overall improvement is due to the large reduction of errors for the covariate effects, which compensates for the observed deterioration of random effect estimates. While the errors for the former decrease about 8-fold [16-fold], the errors for the latter increase about 1.5- to twofold [five- to 16-fold].

Comparison to other approaches

Appendix C summarizes additional results for comparisons between spline-based and FPC-based terms implemented for function-on-function effects and functional random effects in pffr as well as the wavelet-based approach for functional linear mixed models of Morris and Carroll (2006) implemented in WFMM (Herrick, 2013).

Coverage

We also evaluate coverage of approximate point-wise empirical Bayes confidence intervals (CIs) (c.f. Wood, 2006, eq. (4.35)) with constraint correction (Marra and Wood, 2012) for a nominal level of 95%. For each fitted model, we record the proportion of point-wise intervals covering the true value of each quantity evaluated on a fine grid. Note that the coverages of neighboring grid points are not independent, but for the computationally intensive models we fit this is a feasible alternative to coverage estimates based on hundreds of replicates of each setting. CI coverage was consistently very close to the nominal level for ŷ(t) (Median10% quantile90% quantile : 0.95(0.92−0.97)) and 0(t) (0.95(0.90.98)), while α^(t)(0.97(0.91)) showed some overcoverage as well as a few replicates with coverage below 0.7 for small and noisy data. Coverage for functional random slopes 1(t) was below nominal for small groups, but close to nominal for larger datasets (ni = 3: 0.9(0.810.96); ni = 20: 0.95(0.90.99)). Similarly for β^1(s,t) and β^2(s,t), overall coverage was close to the nominal level (0.95(0.850.99)), with systematic undercoverage in small and noisy data sets with dominating random effects. Both γ^1(z1,t)(0.99(0.941)) and δ^2(t)(1(0.751)) had overcoverage, the latter with many outliers with observed coverages below 0.8.

Computation times

Figure 2 shows computation times on an 2.2 GHz AMD Opteron 6174 processor for the different scenarios and sample sizes. Especially for models with multiple random effects (scenario 1) computation times increase dramatically in M. Smaller models are fit rapidly, and even for the largest data sets with nT = 1.2 · 105, computation times are not prohibitively long. Speed gains for REML inference on large data sets can be achieved by using the pffr()-option to use mgcv's bam() routine for estimating additive models on data sets that do not fit into memory, as in Section 2.2. Using GCV optimization (Wood, 2004) instead of REML-based inference in pffr() can yield up to 10-fold speedups especially for large data sets, but tends to be less stable.

Summary

Important effects that contribute relevantly to the predictor are estimated with good to excellent accuracy. Only a single replicate resulted in an rIMSE for y(t) greater than 0.1 – even in the most challenging data situations with few noisy observations and small group sizes, our approach is able to reproduce the true structure of the data well. Our results indicate that estimation accuracy of covariate effects is affected most strongly by changes in the noise levels SNRε and especially SNRB, and less strongly by changes in the available number of observations M,ni and T. The patterns of relative change in accuracy are identical for simple functional regression coefficients, index-varying smooth effects or effect surfaces for functional covariates. The estimation accuracy of the functional random effects is affected strongly by the relative importance of the random effects SNRB and the group size ni, and little by the number of groups M. FPC-based random effects seem to require a sufficiently large number of groups and low noise level to obtain usable FPC estimates. Spline-based approaches yielded superior results to FPC-based and wavelet-based approaches, but it should be noted that the data-generating process for the simulation study was spline-based itself. Overall, the observed coverage of the approximate pointwise CI was very close to the nominal level except for very small or noisy data.

2.2 Modeling spatial association of demyelination in a longitudinal MS study

Our motivating tractography study comprises 162 MS patients and 42 healthy controls who are observed at one to eight visits, spread over up to four years, with 476 visits in total. MS damages white matter tracts (WMT) in the brain due to lesions, axonal damage and demyelination. Diffusion tensor imaging (DTI) is a magnetic resonance imaging technique that is able to resolve individual WMTs in the central nervous system (Basser et al., 2000), and is thus a very useful tool in monitoring disease progression in MS patients. At each visit, fractional anisotropy (FA) was determined via DTI along the corpus callosum (CCA, connecting the left and right hemispheres of the brain), the left corticospinal tract (CST, connecting the brain and the spinal cord), and the left optic radiation tract (OPR, connecting visual cortex and thalamus). FA is derived from the estimated diffusion tensor and is equal to zero if water diffuses perfectly isotropically (Brownian motion) and to one if it diffuses with perfectly organized movement of all molecules in one direction for a given voxel. It may be decreased in MS patients and thus serves as a marker of disease progression here. Tracts are registered within and between subjects using standard biological landmarks identified by an experienced neuroradiologist. Figure 3 displays registered tract profiles as functions of tract location; profiles corresponding to four different subjects at first visit are highlighted.

Figure 3.

Figure 3

From left to right: FA profiles along CCA, OPR and CST for MS patients (red) and controls (blue). Solid line: females; dashed: males. FA-OPR and FA-CST are de-trended and smoothed.

Various aspects of this complex tractography dataset have been explored in a sequence of papers including Goldsmith et al. (2011), Goldsmith et al. (2012), Staicu et al. (2011), Ivanescu et al. (2012). This study was first introduced by Greven et al. (2010), who modeled longitudinal variability in trajectories FPC-based, but could not take into account any covariate effects. Our goal here is to better understand the spatial course of the demyelination process via its FA proxy and investigate possible differences therein between MS and healthy subjects. Ivanescu et al. (2012) considered a similar question, but used only the first measurement of each subject since their approach is unable to handle the longitudinal structure of the data. We assume a functional linear dependence between the FA along the CCA and the two functional covariates – FA along the OPR and FA along the CTS – while adjusting for the effects of other relevant covariates such as gender, age at visit, and disease status. Specifically, if yij(t) is the FA profile at location t on the CCA tract observed at visit j for subject i, we assume that the conditional mean of yij(t), μij(t), is

μij(t)=αdi(t)+δgi(t)+ν(uij,t)+x1,ij(s)β1,di(s,t)ds+x2,ij(r)β2,di(r,t)dr (8)

where x1,ij(s) and x2,ij(r) are the FA profiles at locations s and r along the OPR and CTS tracts, respectively, observed at the jth visit of the ith subject. Here di is the disease status of the subject, with di = 1 for MS patients, and di = 0 for healthy subjects; gi indicates the gender: gi = 1 for males and 0 for females; and uij is the age (in years) at the jth visit of the ith subject. Note that the effects of FA-OPR and FA-CST at the current visit are disease group-specific, with β1,0(s,t)2,0(r,t) for controls and β1,1(s,t)2,1(r,t) for MS patients. Neither age nor gender effects were found to differ between disease groups in the model-building process.

Effect estimates for a naïve model (8) along the lines of Ivanescu et al. (2012) under assumed independence with measures of uncertainty are provided in Appendix B, Figure 7 for completeness. Due to the inappropriate conditional independence assumption, this approach underestimates the variability of the estimates. We use our proposed functional additive mixed model to account for the within-subject correlation, which is the key advantage of our approach over available function-on-function regression methods. Specifically, a more appropriate model is

yij(t)=μij(t)+bi0(t)+εijt, (9)

where bi0(t) are subject-specific functional random intercepts. Model (9) can be fit using the pffr() function in the refund package. Estimating (9) took about 17 hours on an 2 GHz AMD Opteron processor.

Figure 7.

Figure 7

Estimated components of model (8) with ±2 pointwise standard errors, using Ivanescu et al. (2012). Coefficient surfaces are color-coded for sign and pointwise significance (95%): blue if sig. < 0, light blue if < 0, light red if > 0, red if sig. > 0. Top row, left to right: mean FA-CCA for healthy (blue, dotted) versus MS (red, solid) females; mean FA-CCA for female (purple, solid) and male (green, dotted) MS patients; estimated effect of age-at-visit ν^(u,t). Bottom row, left to right: Estimated coefficient surfaces β^1,0(s,t), β^1,1(s,t), β^2,0(r,t), β^2,1(r,t).

Since there are subjects with a few missing locations along the tracts and since the FA measurements are observed with noise, we preprocess the functional covariates. (Note that missing values in the functional response are not an issue for our approach.) The FA profiles are first detrended by subtracting the disease group-specific mean function to make the estimated effects easily interpretable (see Appendix A.1). They are then smoothed, which also imputes missing values. For smoothing, we use functional principal component analysis (Di et al., 2009; Yao et al., 2005) for all tract-specific FA curves under a working independence assumption between profiles on the same subject. Since the observed FA-CCA profiles exhibit a lot of small scale structure at locations 5 20 and > 85, spline based functional random intercepts would require a very large basis to provide sufficient flexibility. Instead, we use the residual curves from model (8) fitted under an independence assumption to obtain an unsmoothed FPC-basis for the random intercepts, as described on page 10.

Figure 4 shows the estimated mean of the FA profiles along the CCA tract (anterior to posterior, i.e. front of the head to the back) for female subjects with and without MS (left panel) and for male and female MS patients (second from left). The estimated mean FA profiles have similar shapes, with a sharp increase in the rostrum/genu (front), a plateau in the middle section, followed by a decrease near the isthmus and a rapid increase towards the splenium (back). As expected, MS patients tend to have lower FA-CCA, especially in the posterior section from the rostral body to the splenium. The effect of gender seems to be negligible. The estimated age effect, ν^(uij,t), indicates that FA-CCA decreases almost linearly with age over the entire tract, particularly in the anterior part, but this effect is fairly small. Not accounting for the longitudinal data structure (c.f. Figure 7), differences between MS and healthy subjects would be found to be much larger and statistically significant along the entire tract. The corresponding estimate for the age effect seems implausible. Due to the misplaced independence assumption, the variability of the estimates shown in Figure 7 is underestimated, but should be approximately correct in Figures 4 and 5. The rightmost panels in Figure 8 give covariances and correlations for εij(t) and show that the white-noise-error assumption is reasonable for model (9), but severely violated for (8). They also show that spline-based random intercepts are less successful in removing all structure from the residuals in this case, especially in the rostrum/genu.

Figure 4.

Figure 4

Estimated components of model (9) with ±2 pointwise standard errors. Coefficient surfaces are color-coded for sign and approximate pointwise significance (95%): blue if sig. < 0, light blue if < 0, light red if > 0, red if sig. > 0. Left to right: mean FA-CCA for healthy (blue, dotted) versus MS (red, solid) females; mean FA-CCA for female (purple, solid) and male (green, dotted) MS patients; estimated smooth index-varying age effect ν^(u,t).

Figure 5.

Figure 5

Left to right: Estimated coefficient surfaces β^1,0(s,t), β^1,1(s,t), β^2,0(r,t), β^2,1(r,t).

Figure 8.

Figure 8

Top row, left to right: Observed residuals ε^ij(t) for model (8); empirical covariance for ε^ij(t) for model (8); empirical covariance for ε^ij(t) for model (9) with FPC-based random intercepts; empirical covariance for ε^ij(t) for model (9) with spline-based random intercepts; legend for covariance values. Bottom row: Empirical correlations.

In healthy controls, FA values at the ends of the OPR tract (towards lateral geniculate nucleus and visual cortex, respectively) and in its middle section show a positive association with FA values along the entire CCA tract (see Figure 5). For the CTS tract, there is some indication of a positive association between FA values in the beginning of the CTS tract (medulla) and the end of the CCA (splenium) and between the end of the CTS tract (subcortical white matter) and the beginning of the CCA (rostrum/genu), the latter corresponding with spatial proximity. These patterns should be indicative of the normal ageing process, while the observed associations mostly vanish for MS patients or become much weaker. It should be noted, however, that simulation results indicate potentially low estimation accuracy of fixed effects in settings such as this one in which the random effects dominate the predictor. Figure 6 displays the predicted intercept curves i0(t) (left panel) and observed residuals ε^ij(t)=yij(t)μ^ij(t)b^i0(t) (right panel). The large variation in the predicted functional intercepts reveals large inter-subject variability. By accounting for the between-subject variability the observed integrated root mean square error of the responses with the proposed method (0.027) reduces to half of its magnitude compared to Ivanescu et al. (2012) under an independence assumption (0.05). Model (9) explains about 90% of the observed variability, while (8) explains only about 63%.

Figure 6.

Figure 6

Predicted functional intercepts i0(t) and observed residuals êij(t) for model (9).

In conclusion, using our flexible modeling framework for the FA profiles along the CCA tract shows that a large fraction of the variability in the data is captured by subject-specific random effects. Modeling the dependence on FA profiles at other well identified tracts, OPR and CTS, can provide new insights into the spatial association in normal ageing and disease processes. Interestingly, our results indicate that the associations between demyelination along the left CTS and left OPR tracts and the CCA tract are weaker for MS patients than for healthy controls. A possible interpretation of this finding could be that demyelination processes in MS patients are more strongly localized, consistent with the development of localized lesions during MS. By properly accounting for the longitudinal structure of the data the estimation uncertainty of all effects increases compared to model (8) under an independence assumption.

3 Discussion and Outlook

We propose a general framework for flexible additive regression models for correlated functional responses, allowing for multiple functional random effects with, for example, spatial, temporal, spatio-temporal or longitudinal (Section 2.2) correlation structures. Dependence structures can be modeled either implicitly by specifying smooth temporal, spatial or tempo-spatial effects or explicitly by including functional random effects with marginal between-unit correlation structures given by the precision matrices of Gaussian (Markov) random fields. Estimation and inference is performed by standard additive mixed model software, allowing us to take advantage of established robust and flexible algorithms. The approach is implemented as fully documented open-source software in the pffr()-function in the refund package (Crainiceanu et al., 2011) for R. Effects of functional covariates and functional random effects are available in both FPC- and spline-based variants and both scalar and functional covariates can have linear or more general smooth effects on the outcome trajectories, allowing analysts to choose the most suitable tools for the task at hand.

Simulation experiments show that the proposed method recovers relevant effects reliably and handles small group sizes and/or low numbers of replications well. Data sets of considerable size can be fit in acceptable time. Two applications demonstrate that our approach makes it possible to fit flexible models that do justice to complex data situations and yet still yield interpretable results that can help to understand the underlying processes.

This work opens up a number of interesting avenues for further research. A first direction concerns the covariance structure of the residuals. Since our present inference algorithms do not exploit the extreme sparsity of the design matrices for smooth observation-specific residual terms, estimating such terms dramatically increases computation time and memory requirements for large data sets. On the other hand, simply assuming i.i.d. errors εit will often be unrealistic since some degree of auto-correlation and heteroscedasticity over the index of the functional response is usually encountered in practice. We are currently investigating an iterative procedure similar to the approach in Reiss et al. (2010), where observed residuals from an initial model estimated under a working independence assumption are used to estimate a working covariance structure and the model is then re-estimated based on de-correlated data. If successful, such a marginal model specification could offer an efficient alternative to the conditional modeling approach outlined in the present paper. In a second direction, we are currently developing diagnostic measures to identify potential problems with low-rank functional covariates (c.f. Appendix A) as well as practical model-building strategies regarding the estimation of corresponding regression surfaces. A closely related avenue of inquiry are more in-depth comparisons of spline- and FPC-based approaches for modeling function-on-function terms as well as functional random effects in order to evaluate their relative strengths and weaknesses. The unifying framework implemented in pffr() will greatly facilitate such comparisons. In addition, we have begun implementing a dedicated toolbox for REML-based inference tailored to function-on-function regression. This effort is based on the computationally efficient array regression approach of Currie et al. (2006), which is expected to speed up inference for large scale problems and help to generalize the proposed methods for multidimensional functional responses and image regression.

Supplementary Material

Supplementary Materials

Acknowledgements

We thank Danny Reich and Peter Calabresi for supplying and explaining the DTI tractography data. Richard Herrick and Jeffrey S. Morris provided a Linux-version of WFMM. Sonja Greven and Fabian Scheipl were funded by Emmy Noether grant GR 3793/1-1 from the German Research Foundation. Ana-Maria Staicu's research was supported by U.S. National Science Foundation grant number DMS 1007466 and by US National Institute of Health grant number 1R01 NS085211-01.

A Identifiability

A.1 Imposing suitable identifiability constraints

Additive models for scalar responses ensure identifiability by imposing suitable constraints on the functions that make up the additive predictor β0+sfs(xs), such as a sum-to-zero constraint i=1nfs(xsi)=0 for each function fs(xs) (Wood, 2006). Otherwise, any constant could be added to one function and subtracted from the others without changing the fit criterion.

A similar issue arises in the context of our proposed model. For arbitrary functions γz(t),bg(t)

E(yi(t))=α(t)+γ(zi,t)+bg(i)(t)=(α(t)+γz(t)+bg(t))+(γ(zi,t)γz(t))+(bg(i)(t)bg(t))

obtains the same fit with two different parameterizations. To avoid this, we impose sum-to-zero constraints for each t so that n1i=1nbg(i)(t)=n1i=1nγ(zi,t)=0t.

We also center covariate trajectories xi(s) by subtracting the mean function x(s)=n1ixi(s). If both the sum-to-zero constraints for each t are imposed and functional covariates are centered, all effects that vary over the index of the response are directly interpretable as deviations from the overall mean trajectory α(t). Standard sum-to-zero constraints implemented in mgcv, which would correspond to Σi,t γ(zi,t) = 0, yield neither identifiable models nor effects that are interpretable like this. Implementationwise, we use the method described in Wood (2006, ch. 1.8.1) to absorb the sum-to-zero-for-each-t constraints into the design matrices of all effects varying over t, see section A in the online supplement for details and examples.

A.2 Limits on the identifiability of complex regression surfaces for low-rank functional covariates

For function-on-function-regression terms Yx(s)β(s,t)ds, identifiability of β(s,t) is guaranteed under conditions derived in He et al. (2003), Chiou et al. (2004) and Prchal and Sarda (2007), which are hard to verify empirically. In practice, an important quantity in this regard for the stability of spline-based estimates is the effective rank of the covariance operator of x(s), which can be defined as the number of eigenvalues that together account for at least 99.5% of the covariate's variability. If this effective rank is low, the kernel of the functional covariate's covariance operator is large. Scheipl and Greven (2012) have shown that spline-based regression surface estimates can be unstable if the kernel of the functional covariate's covariance operator overlaps the function space spanned by parameter vectors in the nullspace of the tensor product spline's roughness penalty. Based on theoretical considerations and simulation results (c.f. Scheipl and Greven, 2012), we recommend that practitioners check the effective rank of the observed covariance matrix of functional covariates and the amount of overlap between the kernel of the functional covariate's covariance operator and the nullspace of the associated roughness penalty. Utility functions to perform these checks and constructors for modified roughness penalties without nullspaces are included in refund.

B Supplementary Details for the DTI Data Analysis

C Supplementary Simulation Study Results

Comparison with FPC-based approaches

We fit models with an FPC-based function-on-function term (c.f. page 8) and models with FPC-based functional random intercepts (c.f. page 10) to each dataset generated for the second scenario. rIMSEs for the FPC-based function-on-function term were larger than those of the spline-based estimates by a mean factor of 1.5(1.2−2.1), while computation time was about the same for M = 10 (1(0.8−1.4)) and somewhat longer for M = 100 (1.3(1.1−1.6)). Results for the FPC-based functional random intercept were more different from the spline-based option. Specifically, the FPC-based functional random intercept showed fairly little improvement for SNRε = 5 compared to SNRε = 1. For the latter, the FPC performance was fairly similar (M = 10: factor of 2.4(1.3−9.4), M = 100: factor of 1.1(1.0−1.5)), while it was much less precise for the former: 3.4(1.1−17) for M = 100 and 17(2.5−106) for M = 10. As expected, however, FPC-based functional random intercepts scaled much better than spline-based ones for larger datasets in terms of computation time due to their more compact optimal basis representation – for M = 100, the iterative FPC procedure was faster than spline-based random effect models by a factor of 0.3(0.2−0.5). Also, our spline-based data generating process corresponding to five non-zero FPCs (c.f. Appendix B of the supplement) may be more difficult for FPC based approaches: previous simulation studies of FPC-based functional regression have typically used data generating processes with lower effective rank (e.g. Chen and Müller, 2012; Müller and Yao, 2008; Wu et al., 2010, with 2, 3, and 4 eigenfunctions, respectively) and simpler coefficient shapes.

Comparison with WFMM

We compare our approach to the available implementation of the wavelet-based functional linear mixed models of Morris and Carroll (2006) in WFMM (Herrick, 2013). We can only provide this comparison for scenario 1 as the other scenarios feature terms that are not available in WFMM, which can only fit random effect curves and functional linear effects zijβ(t) of scalar covariates z. Note that, differing from the results for pffr in the remainder of the article, these results are for balanced data, as the WFMM algorithm seems to fail whenever there are any subjects with 1 or 2 observations only, and 10 replicates per setting. We used the default hyper- and tuning parameters for WFMM, with 2000 iterations of burn-in followed by 10000 iterations of sampling. In general, the IMSEs for WFMM are about double to three times those of pffr. Specifically, the IMSE for y(t) is increased by a median factor of 2.5(2.1−3.1) (IMSE(b0(t): 2.3(1.6−3.8), IMSE(b1(t): 2.4(1.5−4)). Note, however, that this comparison is not entirely fair to WFMM, as it is designed for spiky data e.g. from spectrometry (i.e., it assumes sparsity in a suitable wavelet domain), not the smooth functional data that we assume and correspondingly simulated here. Although WFMM is much slower (4- to 16-fold) than pffr for small and intermediate data sizes, its computation time increases much slower than pffr for larger data sets due to its efficient data representation in the wavelet domain and its very fast C++ implementation. Detailed results for this comparison are provided in Section B.6 of the online appendix.

References

  1. Abramovich F, Angelini C. Testing in mixed-effects FANOVA models. Journal of Statistical Planning and Inference. 2006;136(12):4326–4348. [Google Scholar]
  2. Antoniadis A, Sapatinas T. Estimation and inference in functional mixed-effects models. Computational Statistics & Data Analysis. 2007;51(10):4793–4813. [Google Scholar]
  3. Aston JAD, Chiou JM, Evans JP. Linguistic pitch analysis using functional principal component mixed effect models. Journal of the Royal Statistical Society: Series C (Applied Statistics) 2010;59(2):297–317. [Google Scholar]
  4. Baladandayuthapani V, Mallick BK, Young Hong M, Lupton JR, Turner ND, Carroll RJ. Bayesian hierarchical spatially correlated functional data analysis with application to colon carcinogenesis. Biometrics. 2008;64(1):64–73. doi: 10.1111/j.1541-0420.2007.00846.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Basser P, Pajevic S, Pierpaoli C, Duda J. In vivo fiber tractography using DT-MRI data. Magnetic Resonance in Medicine. 2000;44:625–632. doi: 10.1002/1522-2594(200010)44:4<625::aid-mrm17>3.0.co;2-o. [DOI] [PubMed] [Google Scholar]
  6. Bigelow JL, Dunson DB. Bayesian adaptive regression splines for hierarchical data. Biometrics. 2007;63(3):724–732. doi: 10.1111/j.1541-0420.2007.00761.x. [DOI] [PubMed] [Google Scholar]
  7. Brumback B, Rice JA. Smoothing spline models for the analysis of nested and crossed samples of curves (with discussion). Journal of the American Statistical Association. 1998;93:961–994. [Google Scholar]
  8. Brumback B, Ruppert D, Wand MP. Comment. Journal of the American Statistical Association. 1999;94(447):794–797. [Google Scholar]
  9. Chen K, Müller H-G. Modeling repeated functional observations. Journal of the American Statistical Association. 2012;107:1599–1609. [Google Scholar]
  10. Chiou J, Müller H, Wang J. Functional response models. Statistica Sinica. 2004;14(3):675–694. [Google Scholar]
  11. Crainiceanu CM, Reiss PT, Goldsmith J, Greven S, Huang L, Scheipl F. refund: Regression with Functional Data. R package version 0.1-5. 2011 [Google Scholar]
  12. Crainiceanu CM, Ruppert D. Likelihood ratio tests in linear mixed models with one variance component. Journal of the Royal Statistical Society. Series B. 2004;66(1):165–185. [Google Scholar]
  13. Crainiceanu CM, Ruppert D, Claeskens G, Wand MP. Exact likelihood ratio tests for penalised splines. Biometrika. 2005;92(1):91–103. [Google Scholar]
  14. Currie ID, Durban M, Eilers PHC. Generalized linear array models with applications to multidimensional smoothing. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 2006;68(2):259–280. [Google Scholar]
  15. De Boor C. A practical guide to splines. Springer; New York: 1978. [Google Scholar]
  16. Delicado P, Giraldo R, Comas C, Mateu J. Statistics for spatial functional data: some recent contributions. Environmetrics. 2010;21(3-4):224–239. [Google Scholar]
  17. Di CZ, Crainiceanu CM, Caffo BS, Punjabi NM. Multilevel functional principal component analysis. Annals of Applied Statistics. 2009;3(1):458–488. doi: 10.1214/08-AOAS206SUPP. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Fang Y, Liu B, Müller H-G, Wang J-L. PACE: Principal Analysis by Conditional Expectation. Version 2.16. 2013 [Google Scholar]
  19. Faraway JJ. Regression analysis for a functional response. Technometrics. 1997;39(3):254–261. [Google Scholar]
  20. Febrero-Bande M, Oviedo de la Fuente M. Statistical computing in functional data analysis: The R package fda.usc. Journal of Statistical Software. 2012;51(4):1–28. [Google Scholar]
  21. Ferraty F, Goia A, Salinelli E, Vieu P. Recent advances on functional additive regression. In: Ferraty F, editor. Recent Advances in Functional Data Analysis and Related Topics. Springer; 2011. pp. 97–102. [Google Scholar]
  22. Ferraty F, Vieu P. Nonparametric Functional Data Analysis: Theory and Practice. Springer; 2006. [Google Scholar]
  23. Giraldo R, Delicado P, Mateu J. Continuous time-varying kriging for spatial prediction of functional data: An environmental application. Journal of Agricultural, Biological, and Environmental Statistics. 2010;15(1):66–82. [Google Scholar]
  24. Goldsmith J, Bobb J, Crainiceanu CM, Caffo B, Reich D. Penalized functional regression. Journal of Computational and Graphical Statistics. 2011;20:830–851. doi: 10.1198/jcgs.2010.10007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Goldsmith J, Crainiceanu CM, Caffo B, Reich D. Longitudinal penalized functional regression for cognitive outcomes on neuronal tract measurements. Journal of the Royal Statistical Society: Series C (to appear) 2012 doi: 10.1111/j.1467-9876.2011.01031.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Goldsmith J, Greven S, Crainiceanu CM. Corrected confidence bands for functional data using principal components. Biometrics. 2013;69(1):41–51. doi: 10.1111/j.1541-0420.2012.01808.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Greven S, Crainiceanu CM, Caffo BS, Reich D. Longitudinal functional principal component analysis. Electronic Journal of Statistics. 2010;4:1022–1054. doi: 10.1214/10-EJS575. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Greven S, Crainiceanu CM, Küchenhoff H, Peters A. Restricted likelihood ratio testing for zero variance components in linear mixed models. Journal of Computational and Graphical Statistics. 2008;17(4):870–891. [Google Scholar]
  29. Greven S, Kneib T. On the behaviour of marginal and conditional aic in linear mixed models. Biometrika. 2010;97(4):773–789. [Google Scholar]
  30. Gromenko O, Kokoszka P, Zhu L, Sojka J. Estimation and testing for spatially indexed curves with application to ionospheric and magnetic field trends. The Annals of Applied Statistics. 2012;6(2):669–696. [Google Scholar]
  31. Guo W. Functional mixed effects models. Biometrics. 2002;58(1):121–128. doi: 10.1111/j.0006-341x.2002.00121.x. [DOI] [PubMed] [Google Scholar]
  32. He G, Müller H-G, Wang JL. Extending correlation and regression from multivariate to functional data. In: Puri M, editor. Asymptotics in statistics and probability. VSP International Science Publishers; 2003. pp. 301–315. [Google Scholar]
  33. Herrick R. WFMM (Version 3.0 ed.) The University of Texas M.D. Anderson Cancer Center; 2013. [Google Scholar]
  34. Hörmann S, Kokoszka P. Weakly dependent functional data. The Annals of Statistics. 2010;38(3):1845–1884. [Google Scholar]
  35. Hörmann S, Kokoszka P. Consistency of the mean and the principal components of spatially distributed functional data. In: Ferraty F, editor. Recent Advances in Functional Data Analysis and Related Topics. Springer; 2011. pp. 169–175. [Google Scholar]
  36. Ivanescu AE, Staicu A-M, Scheipl F, Greven S. Penalized function-on-function regression. Technical Report 254. Johns Hopkins University, Dept. of Biostatistics Working Papers; 2012. [Google Scholar]
  37. James GM. Generalized linear models with functional predictors. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 2002;64(3):411–432. [Google Scholar]
  38. Krafty RT, Hall M, Guo W. Functional mixed effects spectral analysis. Biometrika. 2011;98(3):583–598. doi: 10.1093/biomet/asr032. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Krivobokova T, Kauermann G. A note on penalized spline smoothing with correlated errors. Journal of the American Statistical Association. 2007;102(480):1328–1337. [Google Scholar]
  40. Li Y, Wang N, Hong M, Turner ND, Lupton JR, Carroll RJ. Nonparametric estimation of correlation functions in longitudinal and spatial data, with application to colon carcinogenesis experiments. The Annals of Statistics. 2007;35(4):1608–1643. [Google Scholar]
  41. Malfait N, Ramsay J. The historical functional linear model. Canadian Journal of Statistics. 2003;31(2):115–128. [Google Scholar]
  42. Marra G, Wood SN. Practical variable selection for generalized additive models. Computational Statistics & Data Analysis. 2011;55(7):2372–2387. [Google Scholar]
  43. Marra G, Wood SN. Coverage properties of confidence intervals for generalized additive model components. Scandinavian Journal of Statistics. 2012;39(1):53–74. [Google Scholar]
  44. McLean MW, Hooker G, Staicu A-M, Scheipl F, Ruppert D. Functional generalized additive models. Journal of Computational and Graphical Statistics. 2012 doi: 10.1080/10618600.2012.729985. (to appear) [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Morris JS, Carroll RJ. Wavelet-based functional mixed models. Journal of the Royal Statistical Society. Series B. 2006;68(2):179–199. doi: 10.1111/j.1467-9868.2006.00539.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Morris JS, Vannucci M, Brown PJ, Carroll RJ. Wavelet-based nonparametric modeling of hierarchical functions in colon carcinogenesis. Journal of the American Statistical Association. 2003;98(463):573–583. [Google Scholar]
  47. Müller H-G, Yao F. Functional additive models. Journal of the American Statistical Association. 2008;103(484):1534–1544. [Google Scholar]
  48. Nerini D, Monestiez P, Manté C. Cokriging for spatial functional data. Journal of Multivariate Analysis. 2010;101(2):409–418. [Google Scholar]
  49. Nychka D. Confidence intervals for smoothing splines. Journal of the American Statistical Association. 1988;83:1134–1143. [Google Scholar]
  50. Panaretos VM, Tavakoli S. Cramér-Karhunen-Loève representation and harmonic principal component analysis of functional time series. Stochastic Processes and their Applications. 2013a;123(7):2279–2807. [Google Scholar]
  51. Panaretos VM, Tavakoli S. Fourier analysis of stationary time series in function space. The Annals of Statistics. 2013b;41(2):568–603. [Google Scholar]
  52. Prchal L, Sarda P. Spline estimator for functional linear regression with functional response. 2007 unpublished. [Google Scholar]
  53. Qin L, Guo W. Functional mixed-effects model for periodic data. Biostatistics. 2006;7(2):225–234. doi: 10.1093/biostatistics/kxj003. [DOI] [PubMed] [Google Scholar]
  54. Ramsay JO, Wickham H, Graves S, Hooker G. fda: Functional Data Analysis. R package version 2.2.7. 2011 [Google Scholar]
  55. R Development Core Team . R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing; Vienna, Austria: 2011. [Google Scholar]
  56. Reiss PT, Huang L, Mennes M. Fast function-on-scalar regression with penalized basis expansions. The International Journal of Biostatistics. 2010;6(1):28. doi: 10.2202/1557-4679.1246. [DOI] [PubMed] [Google Scholar]
  57. Reiss PT, Ogden T. Functional principal component regression and functional partial least squares. Journal of the American Statistical Association. 2007;102:984–996. [Google Scholar]
  58. Reiss PT, Ogden T. Smoothing parameter selection for a class of semiparametric linear models. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 2009;71(2):505–523. [Google Scholar]
  59. Ruppert D, Carroll RJ, Wand MP. Semiparametric Regression. Cambridge University Press; Cambridge, UK: 2003. [Google Scholar]
  60. Scarpa B, Dunson DB. Bayesian hierarchical functional data analysis via contaminated informative priors. Biometrics. 2009;65(3):772–780. doi: 10.1111/j.1541-0420.2008.01163.x. [DOI] [PubMed] [Google Scholar]
  61. Scheipl F, Greven S. Identifiability in penalized function-on-function regression models. Technical Report 125. LMU München; 2012. [Google Scholar]
  62. Scheipl F, Greven S, Küchenhoff H. Size and power of tests for a zero random effect variance or polynomial regression in additive and linear mixed models. Computational Statistics & Data Analysis. 2008;52(7):3283–3299. [Google Scholar]
  63. Staicu A-M, Crainiceanu CM, Carroll RJ. Fast methods for spatially correlated multilevel functional data. Biostatistics. 2010;11(2):177–194. doi: 10.1093/biostatistics/kxp058. [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Staicu A-M, Crainiceanu CM, Ruppert D, Reich D. Modeling functional data with spatially heterogeneous shape characteristics. Biometrics. 2011;68(2):331–343. doi: 10.1111/j.1541-0420.2011.01669.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Wood SN. Stable and efficient multiple smoothing parameter estimation for generalized additive models. Journal of the American Statistical Association. 2004;99(467):673–686. [Google Scholar]
  66. Wood SN. Generalized Additive Models: An Introduction with R. Chapman & Hall/CRC; 2006. [Google Scholar]
  67. Wood SN. Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models. Journal of the Royal Statistical Society: Series B. 2011;73(1):3–36. [Google Scholar]
  68. Wood SN. On p-values for smooth components of an extended generalized additive model. Biometrika. 2013;100(1):221–228. [Google Scholar]
  69. Wood SN, Scheipl F, Faraway JJ. Straightforward intermediate rank tensor product smoothing in mixed models. Statistics and Computing. 2013;23:341–360. [Google Scholar]
  70. Wu Y, Fan J, Müller H-G. Varying-coefficient functional linear regression. Bernoulli. 2010;16(3):730–758. [Google Scholar]
  71. Yao F, Müller H-G, Wang J-L. Functional data analysis for sparse longitudinal data. Journal of the American Statistical Association. 2005;100(470):577–590. [Google Scholar]
  72. Zhou L, Huang JZ, Martinez JG, Maity A, Baladandayuthapani V, Carroll RJ. Reduced rank mixed effects models for spatially correlated hierarchical functional data. Journal of the American Statistical Association. 2010;105(489):390–400. doi: 10.1198/jasa.2010.tm08737. [DOI] [PMC free article] [PubMed] [Google Scholar]
  73. Zhu H, Brown PJ, Morris JS. Robust, adaptive functional regression in functional mixed model framework. Journal of the American Statistical Association. 2011;106(495):1167–1179. doi: 10.1198/jasa.2011.tm10370. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Materials

RESOURCES