Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2011 Jul 6.
Published in final edited form as: J Stat Softw. 2010 Jan 1;32(11):i11.

Bayesian Functional Data Analysis Using WinBUGS

Ciprian M Crainiceanu 1, A Jeffrey Goldsmith 2
PMCID: PMC3130307  NIHMSID: NIHMS300123  PMID: 21743798

Abstract

We provide user friendly software for Bayesian analysis of functional data models using WinBUGS 1.4. The excellent properties of Bayesian analysis in this context are due to: (1) dimensionality reduction, which leads to low dimensional projection bases; (2) mixed model representation of functional models, which provides a modular approach to model extension; and (3) orthogonality of the principal component bases, which contributes to excellent chain convergence and mixing properties. Our paper provides one more, essential, reason for using Bayesian analysis for functional models: the existence of software.

Keywords: MCMC, mixed effects, covariance, smoothing

1. Introduction

Functional data analysis (FDA) is an area of research concerned with the statistical analysis of functions. It is closely related to multivariate data analysis because functions are highly multivariate objects. The main distinction between FDA and multivariate analysis is the intrinsic ordering of observations; for example in time for time series or space for images. FDA is under intense methodological development (Chiou, Müller, and Wang 2003; James 2002; James, Hastie, and Sugar 2001; Müller and Stadtmüller 2005; Ramsay and Silverman 2006; Yao, Müller, and Wang 2005b; Yao and Lee 2006, are just a few examples of fundamental contributions in this area). Two comprehensive monographs of FDA with applications to curve and image analysis are Ramsay and Silverman (2005, 2006).

FDA was extended to multilevel functional data (see e.g., Baladandayuthapani, Mallick, Hong, Lupton, Turner, and Carroll 2008; Di, Crainiceanu, Caffo, and Punjabi 2009; Guo 2002; Morris, Vanucci, Brown, and Carroll 2003; Morris and Carroll 2006; Crainiceanu, Staicu, and Di 2009b; Staicu, Crainiceanu, and Carroll 2009). In this paper we will use the Di et al. (2009) approach to modeling multilevel functional data. This approach uses functional principal component bases to reduce data dimensionality and accelerate the associated algorithms, which is especially useful in moderate and large data sets.

A close inspection of various types of functional models reveals that they can all be viewed as mixed effects models (James et al. 2001; James 2002; Morris and Carroll 2006; Di et al. 2009). Inferential methods for functional models range from using frequentist BLUP methods (Yao et al. 2005b; Yao and Lee 2006) to expectation maximization algorithms (James et al. 2001; James 2002) to Bayesian simulations (Morris et al. 2003; Morris and Carroll 2006; Crainiceanu et al. 2009b). However, Bayesian posterior inference has not become one of the standard inferential tools for functional data. Crainiceanu et al. (2009b) speculated that

Possible reasons for the current lack of Bayesian methodology in functional analysis could be: (1) the connection between functional models and joint mixed effects models was not known; and (2) the Bayesian inferential tools were perceived as unnecessarily complex and hard to implement.

In this paper we will further clarify the connection between functional and mixed effects models. We will also disprove (2) by providing a simple combination of R (R Development Core Team 2009) and WinBUGS 1.4 (Spiegelhalter, Thomas, Best, and Lunn 2003) programs that make Bayesian simulations relatively painless. Arguing the necessity for Bayesian methods in general exceeds the scope of our paper; we defer instead to some excellent monographs describing the methodological and computational research conducted over the last 10–20 years (see e.g., Carlin and Louis 2008; Congdon 2003; Gelman, Carlin, Stern, and Rubin 2003; Gilks, Richardson, and Spiegelhalter 1996, and the citations therein for a good overview). Instead, we will argue that the excellent properties of Bayesian analysis in the context of functional data analysis are due to: (1) dimensionality reduction, which leads to low dimensional projection bases; (2) mixed model representation of functional models, which provides a modular approach to model extension; and (3) orthogonality of the principal component bases, which contributes to excellent chain convergence and mixing properties.

The paper describes methods and software for a variety of functional models. Section 2 is focused on the functional principal component analysis of functional data observed at one visit. Section 3 is concerned with functional regression, that is regression where either the covariate or the outcome is a function. Section 4 is focused on the multilevel functional principal component analysis of data observed at multiple visits. Relevant features of the WinBUGS 1.4 programs are described throughout the paper, the entire WinBUGS 1.4 programs are provided in the Appendices. R and WinBUGS 1.4 programs are in the file Bayes_FDA.zip.

2. Single-level modeling of functional data

Consider, for illustration, data from the Sleep Heart Health Study (SHHS). SHHS is a multicenter study of sleep disrupted breathing, hypertension and cardiovascular disease (Quan, Howard, Kiley, Nieto, and O’Connor 1997). Here, we focus on modeling a particular characteristic of the spectrum of the Electroencephalograph (EEG) data, the proportion of δ-power. For more details on the definition and interpretation of δ-power (see e.g., Crainiceanu, Caffo, and Punjabi 2009a; Di et al. 2009; Crainiceanu et al. 2009b). In our context, it is sufficient to know that percent δ-power is a summary measure of the spectral representation of the EEG signal; in this paper we use percent δ-power calculated in 30-second intervals. Figure 1 displays the sleep EEG proportion of δ-power in each of 30-second interval for 3 subjects at the baseline visit. The x axis represents time in hours since sleep onset and the y axis represents the estimated proportion of δ-power. Observations are shown in adjacent 30-second intervals with missing observations indicating wake periods. For interpretation, δ-power corresponds to low-frequency neuronal firing and a higher value of an observation corresponds to lower frequency average neuron firing in a particular interval. Data are shown from sleep onset, when neuronal firing tends to contain higher frequencies, which corresponds to lower percent δ-power. As subjects go through deeper stages of sleep, neuronal firing slows down and contains more lower frequencies, which corresponds to higher values of percent δ-power. Please note the increase in average percent δ-power in the first 30–35 minutes of sleep. As time from sleep onset passes, the subject-specific percent δ-power functions become more and more de-synchronized; this results in a attening of the population average, even though the variability of the subject-specific curves around the population curve does not change. The SHHS contains more than 3, 000 subjects with baseline and visit 2 sleep EEG data.

Figure 1.

Figure 1

Percent δ-power since sleep onset (time = 0). Data shown are for the first 4 hours of sleep of 3 subjects (dotted lines) at the baseline visit. The average over 3, 040 subjects is the black solid line.

In this section, we describe Bayesian methods for the analysis of a sample of curves observed at one visit. The basic idea is to decompose each subject specific curve into a population average, a subject-specific deviation from the population average and measurement error. The subject-specific deviations are modeled by projecting them on a small number of eigenvectors of the covariance matrix of the sample of curves. Using the mixed model formulation of the underlying model we obtain the joint posterior distribution of all parameters given the data. The parameter space includes all subject-specific functions and their individual scores. Methods in this section can be applied to sparse or dense functional data.

2.1. Functional principal component analysis

We focus on the first hour of sleep EEG data for 500 subjects. Denote the observed EEG fraction of δ-power by Wi(t), for subject i = 1, …, I = 500, at the 30-second interval t = 1, …, T = 120. In this data set 14% of observations are missing because wake periods are removed. Wake periods are contiguous, have random lengths and appear at random times. Let Xi(t) be the true EEG fraction of δ-power and assume that Wi(t) is the functional proxy for Xi(t) and that they are related via the following functional measurement error model

Wi(t)=μ(t)+Xi(t)+εi(t) (1)

where εi(t) is a white noise process and Xi(t) is a realization of a mean-zero stochastic process with covariance operator KX(t, s) = cov{Xi(t),Xi(s)}. Using methods method of moments (MoM) and smoothing Di et al. (2009), we obtain a smooth estimator K̂X(t, s) and its corresponding eigenfunctions, ψk(·), k = 1, …, T. Table 1 provides the first 10 eigenvalues and associated levels of variance explained for K̂X(t, s) indicating that more than 95% of the observed functional variability is associated with the first 6 eigenfunctions. Figure 2 displays the first two eigenfunctions (left panels) and the deformations from the population mean in the positive and negative directions of the eigenfunctions.

Table 1.

Estimated eigenvalues for the percent δ power using data for the first hour of sleep during the baseline visit of 500 subjects from the SHHS.

Eigenvalues
1 2 3 4 5 6 7 8 9 10
Var 0.99 0.31 0.18 0.10 0.06 0.04 0.03 0.02 0.02 0.01
% Var 56.4 17.8 10.3 5.4 3.3 2.2 1.4 1.2 1.0 0.6
Sum % Var 56.4 74.2 84.5 90.0 93.3 95.5 96.9 98.1 99.1 99.8

Figure 2.

Figure 2

Left panels: First and second eigenvectors, ψ1(t) and ψ2(t), for the first hour of sleep percent δ power data on 500 subjects. Right panels: The positive (“+” sign) and negative (“−” sign) deformation from the population mean in the direction of the corresponding eigenfunction.

Staicu et al. (2009) and Crainiceanu et al. (2009b) point out that choosing the number of eigenfunctions corresponds to step-wise testing for zero variance components. They propose using a Restricted Likelihood Ratio Test (RLRT) for this zero variance. The null distribution can be easily approximated using methods introduced by Greven, Crainiceanu, Küchenhoff, and Peters (2008) based on the null distribution derived in Crainiceanu and Ruppert (2004) and Crainiceanu, Ruppert, Claeskens, and Wand (2005).

We will retain the first K = 10 eigenfunctions and consider the case when μ(t) = 0. This can be achieved by replacing Wi(t) by Wi(t) − μ̂(t), where μ^(t)=∑i=1IWi(t)/I. The functional model (1) becomes

{Wi(t)=∑k=1Kξikψk(t)+εi(t);ξik∼N(0,λk);εi(t)∼N(0,σε2), (2)

which is a mixed effects model. We now show how to implement this model in WinBUGS.

To completely specify the Bayesian model, one needs to provide prior distributions for all model parameters. We used independent Gamma(10−3, 10−3) priors for σε2,σk2 for k = 1, …, K = 10. The parameterization of the Gamma(a, b) distribution is chosen so that its mean is a/b = 1 and its variance is a/b2 = 103. These choices are reasonable as the MoM estimators of σε2 and λK=10 were 6.7 · 10−3 and 1.16 · 10−2, respectively (see e.g., Crainiceanu, Ruppert, Carroll, Adarsh, and Goodner 2007, for a thorough discussion on Gamma priors). In Bayesian models, the estimates of the variance components are known to be sensitive to the prior specification; see Gelman (2006). Alternative to gamma priors are discussed by, for example, Gelman (2006) and Natarajan and Kass (2000). These have the advantage of requiring less care in the choice of the hyperparameters. However, we find that with reasonable care, the conjugate gamma priors can be used in practice. Nonetheless, exploration of other prior families for functional data analysis would be well worthwhile, though beyond the scope of this paper.

2.2. WinBUGS program for the single-level exposure model

We now describe the WinBUGS program that follows closely the description of the Bayesian functional principal component analysis (FPCA) model (2). We provide the entire program in the Appendix A. While the program was designed for the SHHS data, it can be used for other FPCA with only minor adjustments. Many features of this program will be repeated in the other examples in this paper and changes will be described, as needed.

Model (2) describes the core components (likelihood and shrinkage assumptions) of FPCA and is specified in WinBUGS as follows

for (i in 1:N_subj)
     {for (t in 1:N_obs)
           {W[i,t]~dnorm(X[i,t],tau_eps)
            X[i,t]<-xi[i,1]*psi[t,1]+xi[i,2]*psi[t,2]+xi[i,3]*psi[t,3]+
                        xi[i,4]*psi[t,4]+xi[i,5]*psi[t,5]+xi[i,6]*psi[t,6]+
                        xi[i,7]*psi[t,7]+xi[i,8]*psi[t,8]+xi[i,9]*psi[t,9]+
                        xi[i,10]*psi[t,10]}
       for (k in 1:dim.space)
           {xi[i,k]~dnorm(0,tau_lambda[k])}
}

This part of the program describes a double loop over the subjects, for (i in 1:N_subj), and over the number of observations within subjects, for (t in 1:N_obs). The number of subjects, N_subj, is a constant in the program and is equal to 500. The number of observations within subject, N_obs, is also a constant and is equal to 120, which corresponds to the number of 30-second intervals in one hour. Note that, in general, the number of observations for each subject is smaller than 120 due to missing observations. However, missing observations are treated as random and are estimated like all the other unknowns in the model.

The first statement specifies that the Wi(t), the observed percent EEG δ-power, has a normal distribution with mean Xi(t), the true percent EEG δ-power, and precision τε=σε−2. The second statement provides the structure of the conditional mean function, Xi(t). Here psi[t,k] denotes the kth eigenfunction evaluated at time t, ψk(t). All eigenfunctions are obtained from the diagonalization procedure of the smooth estimator K̂X(t, s) of KX(t, s) and are treated as data. Also, xi[i,k] corresponds to ξik in model (2), which is the score of the subject i-specific function on the kth eigenfunction, ψk(t). Thus, xi[,] is a I × K dimensional matrix of random parameters, whose joint posterior distribution is the main target of inference. The third statement specifies that ξik for i = 1, …, I, the scores of subjects on principal component k, have a normal distribution with mean 0 and component k-specific precision τk,λ=λk−1. The matrices W[,] and psi[,] are I × T and T × K dimensional, respectively, are obtained outside WinBUGS and are entered as data. The software accompanying this paper contains an auxiliary R program that calculates these matrices and uses the R2WinBUGS package (Sturtz, Ligges, and Gelman 2005) to call WinBUGS from R. The formulae for X[i,t] could be shortened using the inner product function inprod. However, depending on the application, computation time can be 5 times longer when inprod is used. The model continues with the prior specifications for the precision parameters, τk=σk−2. These parameters were already estimated as the eigenvalues of the method of moments estimator K̂X(t, s) and could be embedded as constants. Here, we estimate them again by using uninformative gamma priors.

for (k in 1:dim.space)
     {tau_lambda[k]~dgamma(1.0E-3,1.0E-3)
       lambda[k]<-1/ll[k]}

2.3. Results

We obtained 1, 500 simulations, discarded the first 500 as burn-in and used the remainder 1, 000 for inference. For I = 500 subjects and T = 120 grid points for each function the total computation time was 4.8 minutes (Dual Core Processor 3GHz, 8Mb RAM PC). This number of simulations was enough for our purposes because convergence and mixing of the chains was excellent. Indeed, Figure 3 displays the un-thinned histories for 4 chains corresponding to two variance components and two subject-specific deviations from the population mean. The independence-like behavior of the chains is due to the orthogonality of the functional basis, ψk(·).

Figure 3.

Figure 3

1, 000 un-thinned draws from the posterior distributions for two variance components, λ1 and λ3, and two subject-specific deviations, X3(·) and X5(·), evaluated at two different time points, t = 10 and t = 50, respectively.

Chain properties, such as convergence and mixing are crucial in Bayesian analysis based on posterior simulations. Indeed, if the chain does not converge or converges very slowly to the target distribution then inferences based on these chains may be unreliable. Poor mixing may be due to a variety of factors, all of them undesirable: inadequate model parameterization, unidentifiable or nearly unidentifiable models, poor performance of simulation algorithms, wrong implementation, etc. Poor mixing is one of the most haunting problems of modern Bayesian computational problems, with hundreds of papers dedicated to improving mixing behavior. In practice, we found that convergence of chains is best illustrated by running multiple chains from over-dispersed initial values with respect to the target distribution and visually inspect when chains have converged. A more formal approach would be to monitor the Gelman and Rubin statistic Gelman and Rubin (1992). In our case convergence is basically instantaneous and simply removing the first 500 simulations as burn-in is more than enough. Mixing is typically assessed by visual inspection, by calculating the autocorrelation function or the Monte Carlo error. Visual inspection of chains indicated that our chains are very close to independence sampling.

The orthogonality of the principal component basis, the data reduction and the mixed model representation of functional models make Bayesian analysis particularly appealing for functional data analysis.

Bayesian posterior simulations provide the joint posterior distribution of all parameters given the data. Note that the subject-specific functional random effects, Xi(t)=∑k=1Kξikψk(t), are explicit functions of the model parameters, ξik; thus, one can obtain the joint posterior distribution of all Xi(t), for every t. Figure 4 displays the data (black dots) for 4 subjects together with the posterior means (solid black line) and 95% point-wise credible intervals (shaded areas) of the subject-specific mean, μ(t)+Xi(t). Constructing credible or confidence intervals is considered to be technically difficult but important; using the Bayesian methods in this paper this can be done relatively easily for data sets with a large number of subjects. We are not aware of any other published result that allows this.

Figure 4.

Figure 4

Estimated subject-specific means with 95% point-wise credible intervals for the EEG normalized δ power in the first hour of sleep of 4 subjects from the SHHS.

3. Functional regression models

In this section, we describe Bayesian analysis of three types of functional regression. First, we introduce the classical functional regression model where an outcome is regressed on a functional predictor. This is done by adding a regression model to the functional models introduced in Section 2, where the outcome is regressed on the functional scores and other covariates. The regression and the functional model are fitted jointly, which correctly accounts for the uncertainty of functional estimators. Second, we introduce the Bayesian penalized B-splines model to estimate the functional coefficient. This is an alternative to the classical functional regression model that uses a large number of principal components to estimate the subject-specific functions and a spline penalty to control the amount of smoothing of the functional parameter. This method avoids the typical discussions about choosing the right number of principal components. Third, we introduce the case when functional scores are regressed on other covariates. This is done by adding a regression model to the functional models introduced in Section 2, where the outcome is one of the functional scores and the regressors are other covariates. Models are fitted jointly to correctly incorporate the variability of the unknowns.

3.1. Classical functional regression model

A particularly useful class of models that describe associations between non-gaussian outcomes and functional data is the class of generalized functional linear models (GFLM, Müller and Stadtmüller 2005; Cardot, Ferraty, and Sarda 1999, 2003; Reiss and Ogden 2007; Ramsay and Silverman 2006, 2005). For the ith subject the data are [Yi,Zi, {Wi(tim), tim ∈ [0, 1]}], where Yi is the continuous or discrete outcome, Zi is a vector of covariates, and Wi(tim) is a random curve in L2[0, 1] observed at time tim, which is the mth observation, j = 1, …, Mi, for the ith subject, i = 1, …, n. We assume that Wi(t) is a proxy observation of the true underlying functional signal Xi(t) and that Wi(t) = μ(t) + Xi(t) + εi(t), where μ(t) is the population average and εi(t) is a mean zero white noise process with variance σε2. We also assume that the distribution of Yi is in the exponential family with linear predictor ϑi and dispersion parameter a, denoted here by EF(ϑi, a). The linear predictor is assumed to have the following form

ϑi=∫01Xi(t)β(t)dt+Zitγ, (3)

where β(·) ∈ L2[0, 1] is a functional parameter and the main target of inference. Note that if {ψk(·), k ≥ 1} is an orthonormal basis in L2[0, 1] then both Xi(·) and β(·) have unique representations Xi(t) = Σk≥1 ξikψk(t), β(t) = Σk≥1βkψk(t) and equation (3) can be rewritten as ϑi=∑k≥1ξikβk+Zitγ. Thus, conditional on the eigenfunctions ψk(t), k = 1, …, K, and on the number of eigenfunctions, K, the standard functional regression model can be re-written as the following mixed effects model

{Yi∼EF(ϑi,a);ϑi=∑k≥1ξikβk+Zitγ;Wi(t)=∑k=1Kξikψk(t)+εi(t);ξik∼N(0,λk);εi(t)∼N(0,σε2). (4)

The first line of model (4) describes the distribution of the outcome, where EF(ϑi, a) is an exponential family distribution with linear predictor ϑi and dispersion parameter a. The second line describes the structure of the linear predictor which contains the functional scores, ξik, and other covariates, Zi, as regressors. The scores ξik are not directly observable, but can be estimated from the functional exposure model described in the third line and the distributional assumptions in the fourth line. The difference between model (4) and a standard generalized linear mixed model (GLMM) is that in a GLMM the random effects are random parameters for known regressors. In model (4) the random effects are the regressors in the linear predictor ϑi.

Changes to the program

We provide the entire program in the Appendix B. For illustration, consider the case where a binary outcome is regressed on the functional scores on the first three eigenfunctions. That is, the functional predictor has the form

ϑi=μ+ξi1β1+ξi2β2+ξi3β3. (5)

This additional regression level is included in the WinBUGS program by simply adding the following code describing the logistic model for the outcome to the WinBUGS code described in Section 2.2

for (i in 1:N_subj)
     {Y[i]~dbern(pY[i])
     logit(pY[i])<-mu+beta[1]*xi[i,1]+beta[2]*xi[i,2]+beta[3]*xi[i,3]}

One also needs to specify the prior distributions for μ, β1, β2, β3. This is done as

for (j in 1:3){beta[j]~dnorm(0,1.0E-2)}
mu~dnorm(0,1.0E-2)

Extending this regression model to include more scores or other covariates is straightforward by simply adding terms to logit(pY[i]).

Results

We applied the functional regression model for regressing hypertension status, a binary variable, on the scores of the first three principal components. We used model the functional model (4) with the particular form of the linear predictor given in (5). We used the first hour of normalized EEG δ-power for 500 subjects as functional predictors. Table 2 provides the posterior mean and standard deviation of the parameters μ, β1, β2 and β3.

Table 2.

Estimated functional effects for sleep EEG normalized δ-power on Hypertension. Results shown for 500 subjects and first hour of sleep.

μ β1 β2 β3
Mean −0.25 0.14 −0.16 0.30
St. Dev. 0.09 0.09 0.16 0.21

3.2. Functional regression using penalized B-splines

An important approach to GFLMs uses a penalized spline expansion for the coefficient function (O’Sullivan 1986; Cardot et al. 2003; Cardot and Sarda 2005; Ruppert, Carroll, and Wand 2003; Wood 2006). As in Section 3.1, assume that for the ith subject we observe [Yi,Zi, {Wi(tim), tim ∈ [0, 1]}] where Wi(t) is a measured-with-error proxy for the true subject-specific function Xi(t), that Yi ~ EF(ϑi, a), and that the linear predictor ϑi has the form given in (3). Rather than expressing β(·) in terms of the eigenfunctions ψk(t), we use a cubic B-spline basis {φl(t): 1 ≤ l ≤ L} with equally spaced knots, so that β(t)=∑l=1Lβlφl(t); other spline bases could be used, but the parameters of the B-spline have good mixing properties in a Bayesian posterior simulation context. Given the truncation lag K and the identifiability constraint L ≤ K, the linear predictor has the form

ϑi=∫01{∑k=1Kξikψk(t)}{∑l=1Lβlφl(t)}dt+Ziγ=ξiJβ⊤+Ziγ,

where ξi = [ξi1, …, ξiK]⊤, β = [β1, …, βL]⊤ and J is a K × L dimensional matrix with the (k, l)th entry equal to ∫01ψk(t)φl(t)dt. These integrals are computed by numeric integration and are fixed for the purpose of this paper. WinBUGS loads the matrix J as data.

In this paper we use B-splines, which are a exible basis commonly used in smoothing applications; Figure 5 shows a cubic B-spline basis with 10 functions and 6 equally spaced knots. Smoothness on β(t) is typically induced by assuming a random walk prior on the βl, l = 1, …, L parameters (Brezger, Kneib, and Lang 2005; Lang and Brezger 2004). We use a first order random prior

Figure 5.

Figure 5

Cubic B-spline basis with 10 functions and 6 equally spaced knots, with knots marked by solid squares.

βl+1∼N(βl,σβ2),l=1,…,L−1,

where β1 is treated as a fixed unknown parameter. A second order random walk prior is

βl+2∼N(2βl+1−βl,σβ2),l=1,…,L−2,

where β1 and β2 are treated as a fixed unknown parameter. Both priors are easy to implement in WinBUGS and we choose to use the first order random walk prior.

Thus, the functional regression model using penalized B-splines has the mixed model representation

{Yi∼EF(ϑi,a);ϑi=ξiJβ⊤+Ziγ;Wi(t)=∑k=1Kξikψk(t)+εi(t);ξik∼N(0,λk);εi(t)∼N(0,σε2);βl+1∼N(β1,σβ2),l=1,…,L−1. (6)

Changes to the program

We provide the entire program in the Appendix C. We describe the changes in the WinBUGS program to account for the different parameterization of ϑi. Here, we choose K = L = 10; straightforward changes to the code below allow for larger values of both. First, we use the following code to define the functional predictor for each subject:

logit(pY[i])<-mu[i]
       mu[i]<-mp+eta[i]
       eta[i]<-xi[i,1]*gamma[1]+xi[i,2]*gamma[2]+xi[i,3]*gamma[3]+
             xi[i,4]*gamma[4]+xi[i,5]*gamma[5]+xi[i,6]*gamma[6]+
             xi[i,7]*gamma[7]+xi[i,8]*gamma[8]+xi[i,9]*gamma[9]+
             xi[i,10]*gamma[10]

Here, gamma is the L × 1 vector Jβ⊤. This vector is updated once per iteration, and is computed outside the loop over subjects in the following code:

for(l in 1:L){
      gamma[l]<-J[l,1]*beta[1]+J[l,2]*beta[2]+J[l,3]*beta[3]+
                 J[l,4]*beta[4]+J[l,5]*beta[5]+J[l,6]*beta[6]+
                 J[l,7]*beta[7]+J[l,8]*beta[8]+J[l,9]*beta[9]+
                 J[l,10]*beta[10]
                    }

The first order random walk prior is specified as

for (l in 2:L)
      {beta[l]~dnorm(beta[l-1],taubeta)}
        beta[1]~dnorm(0,1.0E-6)
        taubeta~dgamma(1.0E-3,1.0E-3)

Results

Figure 6 shows the estimate of the functional coefficient β(t) in the linear predictor (3) using the penalized B-spline approach (blue solid line); for reference, we also plot the estimate from the classical functional regression model presented in Section 3.1 (red solid line). In the Bayesian context obtaining credible intervals for the functional parameter is straightforward by simply monitoring β. Figure 6 displays the pointwise 95% credible interval for the estimated coefficient function, indicating that in this example there is not much evidence of statistical significance. Also, the classical and the penalized spline regression provide similar results, but this need not be the case in general (Goldsmith, Feder, and Crainiceanu 2010).

Figure 6.

Figure 6

The left panel shows two estimates of the functional coefficient β(t), one from the classical functional regression model (red solid line) and the other from the Penalized B-Spline approach (blue solid line). The right panel also shows the estimated functions and the pointwise 95% credible interval for β(t) using the Penalized B-spline approach.

3.3. Regression with functional scores as outcomes

In this section, we focus on models that regress functional scores on other covariates. This type of models are important to identify and quantify predictors of observed principal directions of functional variability. For simplicity, we focus on predictors of subject-specific scores, ξi,k0, on the k0th eigenfunction, φk0 (t). We regress ξik0 on the covariate vector Zi, which could include the scores on other eigenfunctions. The full model is

{ξik0∼N(Zitγ,λk0);Wi(t)=∑k=1Kξikψk(t)+εi(t);ξik∼N(0,λk)k≠k0;εi(t)∼N(0,σε2). (7)

The main difference between this model and model (2) is that the scores ξik0 are shrunk towards Zitγ instead of 0.

Changes to the program

We provide the entire program in the Appendix D. For illustration, consider the case where a the functional scores on the first eigenfunction is regressed on age and body mass index (BMI). That is,

Zitγ=μ+ageiγ1+BMIiγ2.

This additional level is included in the WinBUGS program by simply adding the following code describing the logistic model for the outcome to the WinBUGS code described in Section 2.2

for (i in 1:N_subj)
     {xi[i,1]~dnorm(m_xi[i],ll[1])
       m_xi[i]<-mu+gamma[1]*age[i]+gamma[2]*BMI[i]}

In this case there were 1 missing age and 41 missing BMI values. We added an imputation model for age and BMI by adding the following code

for (i in 1:N_subj)
     {age[i]~dnorm(mu_X[1],tau[1])
       BMI[i]~dnorm(mu_X[2],tau[2])}

This is the WinBUGS representation of the simple Gaussian imputation priors agei ~ N(μX,1, τ1) and BMIi ~ N(μX,2, τ2). More complex imputation priors could be set up, but this exceeds the scope of this paper.

One also needs to specify the prior distributions for μ, γ1, γ2, μX,1, μX,2, τ1, τ2. This is done as

mu~dnorm(0,1.0E-2)
for (l in 1:2)
     {gamma[l]~dnorm(0,1.0E-2)
       mu_X[l]~dnorm(m_prior[l],1.0E-3)
       tau[l]~dgamma(1.0E-3,1.0E-3)}

Here m_prior[l] is a 2-dimensional vector containing the sample means of the observed age and BMI values, respectively.

Results

We used the first hour of normalized EEG δ-power for 500 subjects as functional predictors and we regressed the first principal component scores on age and BMI. Table 3 provides the posterior mean and standard deviation of the parameters γ1 and γ2. Results indicate that higher age and BMI are positively associated with higher scores on the first principal component. A closer look at Figure 2 shows that the first principal component is negative and is, roughly, a vertical shift. We conclude that higher age and BMI are significantly associated with lower EEG percent δ-power in the first hour of sleep.

Table 3.

Estimated effects of age and BMI on the first principal component scores. Results shown for 500 subjects and first hour of sleep.

γ1 (age) γ2 (BMI)
Mean 0.016 0.021
St. Dev. 0.005 0.007

4. Multilevel modeling of functional data

The number of data sets where functional data is observed at multiple visits or within clusters is increasing. For example, the SHHS contains EEG data at two visits for thousands of subjects.

In this section, we describe Bayesian methods for the analysis of a sample of curves observed at multiple visits. The basic idea is to decompose each curve into a population average, a visit-specific deviation from the population average, a subject-specific deviation from the visit-specific mean, a subject/visit specific deviation from the subject-specific mean and measurement error. The subject-specific and the subject/visit specific deviations are modeled by projecting them on the eigenvectors of the covariance matrices of the subject- and subject/visit-specific processes, respectively. Using the mixed model formulation of the underlying model we obtain the joint posterior distribution of all parameters given the data. The parameter space includes all subject-specific and subject/visit-specific functions and their individual scores. Methods in this section can be applied to sparse or dense multilevel functional data.

4.1. Multilevel functional principal component analysis

For illustration consider the sleep EEG percent δ-power for the first hour of sleep of 500 subjects who have two visits, roughly 5 years apart. Denote the observed sleep EEG fraction of δ-power by Wij(t), for subject i = 1, …, I = 500 at visit j = 1, J = 2, at the 30-second interval t = 1, …, T = 120. In this data set 18% of observations are missing because wake periods are removed.

We assume that Wij(t) is a proxy observation of the true underlying subject-specific functional signal Xi(t), and that

Wij(t)=μ(t)+ηj(t)+Xi(t)+Uij(t)+εij(t).

Here μ(t) is the overall mean function, ηj(t) is the visit j specific shift from the overall mean function, Xi(t) is the subject i specific deviation from the visit specific mean function, and Uij(t) is the residual subject/visit specific deviation from the subject specific mean. To ensure identifiability we assume that Xi(t), Uij(t), and εij(t) are uncorrelated, that Σj ηj(t) = 0 and that εij(t) is a white noise process with variance σε2. Given the large sample size of the SHHS data, we can assume that μ(t) and ηj(t) are estimated with negligible error by W̄..(t) and W̄.j(t)−W̄.., respectively. Here W̄..(t) is the average over all subjects, i, and visits, j, of Wij(t) and W̄.j(t) is the average over all subjects, i, of observation at visit j of Wij(t). We can assume that these estimates have been subtracted from Wij(t), so that Wij(t) = Xi(t)+Uij(t)+ εij(t).

Eigenvalues and eigenvectors in MFPCA

We use MFPCA (Di et al. 2009) to obtain the parsimonious bases that capture most of the functional variability of the space spanned by Xi(t) and Uij(t), respectively. MFPCA is based on the spectral decomposition of the within- and between-visit functional variability covariance operators. We summarize here the main components of this methodology. Denote by KTW(s,t)=cov{Wij(s),Wij(t)} and KBW(s,t)=cov{Wij(s),Wik(t)} for j ≠ k the total and between covariance operator corresponding to the observed process, Wij(·), respectively. Denote by KX(t, s) = cov{Xi(t),Xi(s)} the covariance operator of the Xi(·) process and by KU(t, s) = cov{Uij(s),Uij(t) } the total covariance operator of the Uij(·) process. By definition, KBU(s,t)=cov{Uij(s),Uik(t)}=0 for j ≠ k. Moreover, KBW(s,t)=KX(s,t) and KTW(s,t)=KX(s,t)+KU(s,t)+σε2δts, where δts is equal to 1 when t = s and 0 otherwise. Thus, KX(s, t) can be estimated using a method of moments estimator of KBW(s,t), say K^BW(s,t). For t ≠ s a method of moment estimator of KTW(s,t)−KBW(s,t), say K̂U(s, t), can be used to estimate KU(s, t). To estimate K̂U(t, t) one predicts KU(t, t) using a bivariate thin-plate spline smoother of K̂U(s, t) for s ≠ t. This method was proposed for single-level FPCA (Yao and Lee 2006) and shown to work well in the MFPCA context (Di et al. 2009). Once consistent estimators of KX(s, t) and KU(s, t) are available, the spectral decomposition and functional regression proceed as in the single-level case. More precisely, Mercer’s theorem (see Indritz 1963, Chapter 4) provides the following convenient spectral decompositions KX(t,s)=∑k=1∞λk(1)ψk(1)(t)ψk(1)(s), where λ1(1)≥λ2(1)≥… are the ordered eigenvalues and ψk(1)(·) are the associated orthonormal eigenfunctions of KX(·, ·) in the L2 norm. Similarly, KU(t,s)=∑l=1∞λl(2)ψl(2)(t)ψl(2)(s), where λ1(2)≥λ2(2)≥… are the ordered eigenvalues and ψl(2)(·) are the associated orthonormal eigenfunctions of KU(·, ·) in the L2 norm. The Karhunen-Loève (KL) decomposition (Karhunen 1947; Loève 1945) provides the following infinite decompositions Xi(t)=∑k=1∞ξikψk(1)(t) and Uij(t)=∑l=1∞ζijlψl(2)(t) where ξik=∫01Xi(t)ψk(1)(t)dt,ζijl=∫01Uij(t)ψl(2)(t)dt are the principal component scores with E(ξik) = E(ζijl) = 0, Var(ξik)=λk(1),Var(ζijl)=λl(2). The zero-correlation assumption between the Xi(·) and Uij(·) processes is ensured by the assumption that cov(ξi, ζijl) = 0. These properties hold for every i, j, k, and l. For simplicity we will refer to ψk(1)(·),ψl(2)(·) and λk(1),λl(2) as the level 1 and 2 eigenfunctions and eigenvalues, respectively.

For the SHHS example the estimated eigenvalues of K̂X(·, ·) and K̂U(·, ·) are provided in Table 4. The Level 1 and 2 labels to refer to the X and U processes, respectively. Approximately 99% of the subject-specific process variability is explained by the first 3 principal components whereas as many as 12 components are necessary to explain the same percentage of variability for the subject-visit-specific variability. The within-subject between-visit correlation can be estimated as

Table 4.

Estimated eigenvalues for the percent δ power using data for the first hour of sleep during the baseline visit of 500 subjects from the SHHS.

Eigenvalues – level 1
1 2 3 4 5 6 7 8 9 10
Var 0.49 0.04 0.02
% Var 88.6 6.8 3.9
Sum % Var 88.6 95.4 99.3

Eigenvalues – level 2
Var 0.42 0.27 0.14 0.09 0.06 0.04 0.02 0.02 0.02 0.01
% Var 37.7 24.1 12.3 7.9 5.1 3.3 2.0 1.7 1.4 1.3
sum % Var 37.7 61.9 74.2 82.2 87.2 90.6 92.6 94.3 95.7 97.0
ρ^W=∑k≥1λ^k(1)∑k≥1λ^k(1)+∑l≥1λ^l(2).

In the SHHS example considered in this paper ρ̂W = 0.33.

Figure 7 displays the first two eigenvectors for the Level 1 (top panels) and 2 (bottom panels) processes. The first eigenvalues at both levels capture mainly vertical shifts. The second eigenvalue at the first level is roughly centered around zero. Subjects who are positively loaded on this component will tend to have: (1) a lower sleep EEG δ-power proportion than the population average in the first 10 and last 10 minutes of the first hour of sleep; and (2) a higher sleep EEG δ-power proportion than the population average between minutes 20 and 40. This eigenfunction is not smooth, most likely because of the sample size and small proportion of variation explained. One could easily smooth it using any reasonable scatterplot smoother.

Figure 7.

Figure 7

Top panels: First and second eigenvectors at level 1, ψ1(1)(t) and ψ2(1)(t). Bottom panels: First and second eigenvectors at level 1, ψ1(2)(t) and ψ2(2)(t).

Here, we work directly with the eigenvalue shown in Figure 7. The second eigenvalue at the second level is also roughly centered around zero. Subject-visits that are positively loaded on this component will tend to have: (1) a higher sleep EEG δ-power proportion than the subject-specific average in the first 25 minutes of the first hour of sleep; and (2) a lower sleep EEG δ-power proportion than the subject-specific average between in the last 10 minutes of the first hour of sleep.

Estimating the scores in the MFPCA

Conditional on the eigenfunctions and truncation lags K and L, the model for observed functional data can be written as a linear mixed model. Indeed, by assuming a normal shrinkage distribution for scores and errors, the model can be rewritten as

{Wij(t)=∑k=1Kξikψk(1)(t)+∑l=1Lζijlψl(2)(t)+εij(t);ξik∼N{0,λk(1)};ζijl∼N{0,λl(2)};εij(t)∼N(0,σε2). (8)

This model, like the other models in this paper, are especially well suited for Bayesian inference; the implementation in WinBUGS is simple, as we show in the following section. The properties of the mixing chains remain outstanding for the same reasons: orthogonality of the principal component bases, data reduction and mixed model representation of functional models.

4.2. WinBUGS program for the multilevel exposure model

We provide the entire program in the Appendix E. The WinBUGS program for the multilevel functional model has a similar structure with the one for the single level functional model presented in Section 2.2. Model (8) describes the core components (likelihood and shrinkage assumptions) of MFPCA and is specified in WinBUGS as follows

for (i in 1:N_subj)
     {for (t in 1:N_obs)
           {W_1[i,t]~dnorm(m_1[i,t],taueps)
             W_2[i,t]~dnorm(m_2[i,t],taueps)
             m_1[i,t]<-X[i,t]+U_1[i,t]
             m_2[i,t]<-X[i,t]+U_2[i,t]
             X[i,t]<-xi[i,1]*psi_1[t,1]+xi[i,2]*psi_1[t,2]+xi[i,3]*psi_1[t,3]
             U_1[i,t]<-zi[i,1,1]*psi_2[t,1]+zi[i,2,1]*psi_2[t,2]+
                    zi[i,3,1]*psi_2[t,3]+zi[i,4,1]*psi_2[t,4]+zi[i,5,1]*psi_2[t,5]+
                    zi[i,6,1]*psi_2[t,6]+zi[i,7,1]*psi_2[t,7]+zi[i,8,1]*psi_2[t,8]+
                    zi[i,9,1]*psi_2[t,9]+zi[i,10,1]*psi_2[t,10]
             U_2[i,t]<-zi[i,1,2]*psi_2[t,1]+zi[i,2,2]*psi_2[t,2]+
                    zi[i,3,2]*psi_2[t,3]+zi[i,4,2]*psi_2[t,4]+zi[i,5,2]*psi_2[t,5]+
                    zi[i,6,2]*psi_2[t,6]+zi[i,7,2]*psi_2[t,7]+zi[i,8,2]*psi_2[t,8]+
                    zi[i,9,2]*psi_2[t,9]+zi[i,10,2]*psi_2[t,10]}
       for (k in 1:dim.space_b)
           {xi[i,k]~dnorm(0,ll_b[k])}
       for (l in 1:dim.space_w)
           {zi[i,l,1]~dnorm(0,ll_w[l])
             zi[i,l,2]~dnorm(0,ll_w[l])}}

This part of the program describes a double loop over the subjects, for (i in 1:N_subj), and over the number of observations within subjects, for (t in 1:N_obs). The number of subjects, N_subj, is a constant in the program and is equal to 500. The number of observations within subject, N_obs, is also a constant and is equal to 120, which corresponds to the number of 30-second intervals in one hour. As in the single-level case, missing observations are treated as random and are estimated like all the other unknowns in the model. Even though N_obs is a constant in the program, the number of observations per subject varies. Observations that are missing are simply included as NA in the program.

The first four statements specify that the functions Wij(t), the observed percent sleep EEG δ-power, have a normal distribution with mean mij(t) = Xi(t) + Uij(t), the true percent EEG δ-power, and precision τε=σε−2. Here W_1[i,t] and W_2[i,t] are the WinBUGS representation of Wi1(t) and Wi2(t), respectively. Similarly, U_1[i,t] and U_2[i,t] are the WinBUGS representation of Ui1(t) and Ui2(t). A more compact representation of the U-processes could be achieved using triple-indexing. For example one could use U[1,i,t] and U[2,i,t] instead of U_1[i,t] and U_2[i,t]; this would be especially useful for more than two visits. However, our implementation works well and proved to be helpful during debugging.

The fifth statement provides the structure of the subject-specific process, Xi(t), where the kth level 1 eigenfunction evaluated at time t, ψk(1)(t), is psi_1[t,k]. In this case we only used the first three eigenfunctions because together they explain more than 99% of the functional variability at the subject-level. All eigenfunctions are obtained from the diagonalization procedure of the smooth estimator K̂X(t, s) of KX(t, s) and are treated as data; the first two eigenfunctions are displayed in the top panels of Figure 7. Also, xi[i,k] corresponds to ξik in model (8), which is the score of the subject i-specific mean function, Xi(t), on the kth level 1 eigenfunction, ψk(1)(t). Thus, xi[,] is a I × K dimensional matrix of random parameters, whose joint posterior distribution is one of the targets of inference. In our example, I × K = 500 × 3 = 1, 500.

The sixth and seventh statements specify the structure of the subject-visit-specific deviations, Ui1(t) and Ui2(t), from the subject specific mean, Xi(t). Here psi_2[t,k] denotes the kth level 2 eigenfunction evaluated at time t, ψk(2)(t). In this case we used the first ten eigenfunctions because together they explain more than 97% of the functional variability at the subject-visit level. All eigenfunctions are obtained from the diagonalization procedure of the smooth estimator K̂W(t, s) of KW(t, s) and are treated as data; the first two eigenfunctions are displayed in the bottom panels of Figure 7. Also, zi[i,l,j] corresponds to ζijl in model (8), which is the score of the subject i, visit j deviation, Uij(t), from the subject-specific mean, Xi(t), on the lth level 2 eigenfunction, ψk(2)(t). Thus, zi[,,] is a I×L×J dimensional matrix of random parameters, whose joint posterior distribution is one of the targets of inference. In our example, I × L × J = 500 × 10 × 2 = 10, 000.

The assumptions ξik∼N{0,λk(1)} in model (8) are specified as xi[i,k]~dnorm(0,ll_b[k]), where ll_b[k] are the precision parameters τk(1)=1/λk(1) and are estimated from the data. Here the dimension of the level 1 space is K = 3, is denoted as dim.space_b and is loaded as data in the program. The assumptions ζijl∼N{0,λl(2)} in model (8) are specified as zi[i,l,1]~dnorm(0,ll_w[l]) and zi[i,l,1]~dnorm(0,ll_w[l]), where ll_w[k] are the precision parameters τk(2)=1/λk(2) and is a parameter that is estimated from the data. Here the dimension of the level 2 space is L = 10, is denoted as dim.space_w and is loaded as data in the program. The parameters λk(1),λk(2)σε2 are jointly estimated with the other parameters of the model. Thus, we need to specify priors for the variance parameters as

for (i in 1:dim.space_b)
     {ll_b[i]~dgamma(1.0E-3,1.0E-3)
       lambda_b[i]<-1/ll_b[i]}
for (i in 1:dim.space_w)
     {ll_w[i]~dgamma(1.0E-3,1.0E-3)
       lambda_w[i]<-1/ll_w[i]}
taueps~dgamma(1.0E-3,1.0E-3)
sigma_sq_eps=1/taueps

An alternative would be to use the method of moment estimators of λk(1) and use this estimator as data. This is a robust approach that works well in practice, but is not used in this paper. The matrices W_1[,] and W_2[,] are I × T, psi_1[,] is T × K, and psi_2[,] is T × L dimensional, respectively; they are obtained outside WinBUGS and are entered as data.

4.3. Results

We obtained 1, 500 simulations, discarded the first 500 as burn-in and used the remainder 1, 000 for inference. For I = 500 subjects, J = 2 visits and T = 120 grid points for each function the total computation time was 10.2 minutes (Dual Core Processor 3GHz, 8Mb RAM PC). Figure 8 displays the posterior mean and 95% pointwise credible intervals for the same 4 subjects from Figure 4 at the first visit, using the multilevel model (8) instead of the single level model (2). For the subject in the bottom left panel of Figure 4 we display the more detailed inference permitted by the multilevel analysis. Figure 9 displays the sleep EEG percent δ-power for this subject at visit 1 and 2 (top two panels) together with the posterior mean and 95% pointwise credible intervals. The bottom left panel in Figure 9 displays the posterior mean and pointwise 95% credible intervals for the subject specific deviation from the visit-specific mean, Xi(t). The posterior mean of Xi(t) is positive everywhere indicating that this subject tends to have a higher sleep EEG percent δ-power than the average. However, none of these differences is statistically significant as all credible intervals cross zero. The bottom right panel in Figure 9 displays the posterior means of the subject-visit-specific deviations, Ui1(t) and Ui2(t), from the subject-specific mean, μ(t) + ηj(t) + Xi(t). The black solid line corresponds to the random functional effect at the first visit, Ui1, and the gray solid line corresponds to the random functional effect at the first visit, Ui2. Pointwise credible intervals are available for these processes, as well, but were omitted in the plot to avoid cluttering.

Figure 8.

Figure 8

Estimated subject-specific means with 95% pointwise credible intervals for the EEG normalized δ-power in the first hour of sleep of the first visit for 4 subjects from the SHHS. The model used was the multilevel model (8).

Figure 9.

Figure 9

Top panels: First and second visit EEG normalized δ-power data for the subject shown in the bottom left panel of Figure 8. Also shown are the posterior means and 95% credible intervals using the multilevel model (8). Bottom left panel: Posterior mean and 95% pointwise credible interval for the subject-specific deviation process, Xi(t). Bottom-right panel: Posterior means of the subject-visit-specific deviation processes, Ui1 (black solid line) and Ui2 (gray solid line).

5. Interface with and processing in R

We have used R to define the data, initial values and parameters and the R2WinBUGS package (Sturtz et al. 2005) to call and run the WinBUGS part of the program. R was also used to do output checking and processing as well as plotting. Once the WinBUGS code is written and debugged, the user can simply use the R interface to perform analyses or set up simulations. The software accompanying this paper contains the commented R interface.

6. Discussion

This paper is a compilation of examples of Bayesian functional data analysis implemented in WinBUGS. This provides a transparent, easy to use, reproducible alternative to frequentist software such as FDA (Ramsay, Wickham, Graves, and Hooker 2009) maintained by J. Ramsay, PACE (Yao, Müller, and Wang 2005a) maintained by H.G. Müller, and the R package nlmeODE (Tornoe 2007) maintained by C.W. Tornoe. There are at least three reasons for providing WinBUGS code for the Bayesian analysis of functional data. First, for regression models such as those introduced in Section 3, Bayesian analysis provides a natural platform for joint modeling. This can be especially useful when functional data are sparse or functional scores are predicted with sizeable error. Second, these examples could provide the starting point for the implementation of more complex models for realistic data; our WinBUGS programs would require only minimal changes to include random effects, smooth uni-or multi-variate components, and missing or miss-measured data. Third, these programs provide an alternative platform that could be used to confirm results of frequentist software. While, at this time, our programs are the only ones that can handle smooth penalized regression (Section 3.2) and multilevel FPCA (Section 4), this will probably change with new versions of frequentist software.

Acknowledgments

Crainiceanu’s research was supported by Award Number R01NS060910 from the National Institute Of Neurological Disorders And Stroke. The content is solely the responsibility of the author and does not necessarily represent the official views of the National Institute Of Neurological Disorders And Stroke or the National Institutes of Health.

A. Code for functional principal component analysis

See Section 2.1 for model description.

model {
         for (i in 1:N_subj)
        {
         for (t in 1:N_obs)
              {
              W[i,t]~dnorm(m[i,t],taueps)
              m[i,t]<-xi[i,1]*E[t,1]+xi[i,2]*E[t,2]+xi[i,3]*E[t,3]+
                     xi[i,4]*E[t,4]+xi[i,5]*E[t,5]+xi[i,6]*E[t,6]+
                     xi[i,7]*E[t,7]+xi[i,8]*E[t,8]+xi[i,9]*E[t,9]+
                     xi[i,10]*E[t,10]
              }
         for (k in 1:dim.space)
              {
               xi[i,k]~dnorm(0,ll[k])
               }
         }
for (k in 1:dim.space)
     {ll[k]~dgamma(1.0E-3,1.0E-3)
       lambda[k]<-1/ll[k]}
taueps~dgamma(1.0E-3,1.0E-3)
}
  1. N_subj is the number of subjects, in this case equal to 500, and is loaded as data

  2. N_obs is the number of observations within subjects, and is loaded as data. The program is set up to allow for different numbers of observations per subject with missing data being loaded as NA

  3. W[i,t] is the observed realization of the subject i specific function at the time t. The matrix W[,] is N_subj×N_obs, is loaded as data and may contain missing observations.

  4. m[i,t] is the smooth mean of W[i,t] and is an unknown whose joint distribution is simulated.

  5. taueps is the precision of the error process due to imperfect observation of W[i,t] around its smooth mean m[i,t]. This is a parameter of the model that is estimated.

  6. xi[i,k] is the score of the ith subject on the kth eigenfunction, E[,k]. The matrix xi[,] is N_subj×K dimensional and contains parameters whose posterior joint distribution is simulated. Here K is equal to 10,and is hard coded into the program.

  7. E[t,k] is the kth eigenfunction evaluated at the time t. The matrix E[,] is N_obs×K dimensional, does not contain any missing observation and is loaded as data.

  8. dim.space is the dimension of the eigenspace used for projection and is loaded as data. In this case dim.space is equal to 10

  9. ll[k] are the precisions of the scores xi[i,k], are unknown parameters that are estimated from the data.

  10. lambda[k] are the variances of the scores xi[i,k] and are equal to 1/ll[k]

  11. All precision priors are Gamma priors with mean 1 and variance 1000

B. Code for functional regression analysis

See Section 3.1 for model description.

model {
         for (i in 1:N_subj)
         {
         Y[i]~dbern(pY[i])
         logit(pY[i])<-mp+beta[1]*xi[i,1]+beta[2]*xi[i,2]+beta[3]*xi[i,3]
         for (t in 1:N_obs)
         {
               W[i,t]~dnorm(m[i,t],taueps)
               m[i,t]<-xi[i,1]*E[t,1]+xi[i,2]*E[t,2]+xi[i,3]*E[t,3]+
                      xi[i,4]*E[t,4]+xi[i,5]*E[t,5]+xi[i,6]*E[t,6]+
                      xi[i,7]*E[t,7]+xi[i,8]*E[t,8]+xi[i,9]*E[t,9]+
                      xi[i,10]*E[t,10]
          }
          for (j in 1:dim.space)
               {
               xi[i,j]~dnorm(0,ll[j])
               }
          }
for (i in 1:dim.space)
     {ll[i]~dgamma(1.0E-3,1.0E-3)
       lambda[i]<-1/ll[i]}
for (l in 1:3) {beta[l]~dnorm(0,1.0E-2)}
mp~dnorm(0,1.0E-2)
taueps~dgamma(1.0E-3,1.0E-3)
}

The definition of most variables is the same as the definition for WinBUGS code for functional principal component analysis. We describe only the new variables here.

  1. Y[i] is the Bernoulli outcome for subject i. The vector Y[] has length N_subj and is loaded as data. In the example in the paper the outcome is hypertension status YES/NO.

  2. pY[i] is the probability that subject i is hypertensive. The vector pY[] has length N_subj and is a vector of parameters that are estimated in the model.

  3. logit(pY[i]) is the logit transformation of the probabilities pY[i]. The probit transformation could also be used.

  4. beta[] is a vector of regression parameters of length 3 relating the logit(pY[i]) to the scores xi[,] of subject-specific functions on eigenfunctions, E[,]. The length 3 is not set in stone and other lengths can be accommodated when the outcome is regressed on a different set of eigenfunctions.

  5. mp is the intercept parameter of the regression of Y[] on the scores xi[,]. This is a parameter that will be estimated in the model.

  6. The priors for each beta[] parameter and mp are mutually independent normals with mean 0 and variance 100 (recall that WinBUGS specifies the precision, 0.01, instead of the variance, 100).

C. Code for penalized functional regression using B-splines

See Section 3.2 for model description.

model {
for (i in 1:N_subj)
      {
       Y[i]~dbern(pY[i])
       logit(pY[i])<-mp+eta[i]
       eta[i]<-xi[i,1]*gamma[1]+xi[i,2]*gamma[2]+xi[i,3]*gamma[3]+
              xi[i,4]*gamma[4]+xi[i,5]*gamma[5]+xi[i,6]*gamma[6]+
              xi[i,7]*gamma[7]+xi[i,8]*gamma[8]+xi[i,9]*gamma[9]+
              xi[i,10]*gamma[10]
              for (t in 1:N_obs)
              {
              W[i,t]~dnorm(m[i,t],taueps)
                      m[i,t]<-xi[i,1]*E[t,1]+xi[i,2]*E[t,2]+xi[i,3]*E[t,3]+
                              xi[i,4]*E[t,4]+xi[i,5]*E[t,5]+xi[i,6]*E[t,6]+
                              xi[i,7]*E[t,7]+xi[i,8]*E[t,8]+xi[i,9]*E[t,9]+
                              xi[i,10]*E[t,10]
                      }
              for (j in 1:dim.space)
                    {
                           xi[i,j]~dnorm(0,ll[j])
                           }
              }
for(l in 1:L){
      gamma[l]<-J[l,1]*beta[1]+J[l,2]*beta[2]+J[l,3]*beta[3]+
                  J[l,4]*beta[4]+J[l,5]*beta[5]+J[l,6]*beta[6]+
                  J[l,7]*beta[7]+J[l,8]*beta[8]+J[l,9]*beta[9]+
                  J[l,10]*beta[10]
}
for (i in 1:dim.space)
       {ll[i]~dgamma(1.0E-3,1.0E-3)
         lambda[i]<-1/ll[i]}
for (i in 2:L)
       {beta[l]~dnorm(beta[l-1],taubeta)}
         beta[1]~dnorm(0,1.0E-6)
         taubeta~dgamma(1.0E-3,1.0E-3)
mp~dnorm(0,1.0E-2)
taueps~dgamma(1.0E-3,1.0E-3)
}

The definition of most variables is the same as the definition for WinBUGS code for functional regression analysis with 10 eigenfunctions. We describe only the new variables here.

  1. eta[i] is the subject-specific component of the linear predictor. The vector eta[] has length N_subj and is the product of the parameters xi[i,] and b[], which are estimated in the model, and the matrix Jm[,], which is fixed.

  2. gamma[] is the vector containing the current product of the fixed matrix Jm[,] and the parameter vector b[], and has length L (here L is equal to 10). It is updated outside the loop over subjects.

  3. beta[i] is the coefficient of the ith B-spline basis function used to express the functional regressor. beta[]is the vector of regression parameters and has length L. Other values of L are possible with straightforward changes to the code presented here.

  4. J[,] is the K x L matrix containing the product of the eigenfunctions and the B-spline basis functions (here K = L = 10). It is loaded as data.

  5. taubeta is the precision parameter for beta[] and has a dgamma(1.0E-3,1.0E-3) prior distribution.

D. Code for regression with functional scores as outcomes

See Section 3.3 for model description.

model {
         for (i in 1:N_subj)
         {
         xi[i,1]~dnorm(m_xi[i],ll[1])
         m_xi[i]<-mu+gamma[1]*age[i]+gamma[2]*BMI[i]
         age[i]~dnorm(mu_X[1],tau[1])
         BMI[i]~dnorm(mu_X[2],tau[2])
         for (t in 1:N_obs)
              {
              W[i,t]~dnorm(m[i,t],taueps)
              m[i,t]<-xi[i,1]*E[t,1]+xi[i,2]*E[t,2]+xi[i,3]*E[t,3]+
                    xi[i,4]*E[t,4]+xi[i,5]*E[t,5]+xi[i,6]*E[t,6]+
                    xi[i,7]*E[t,7]+xi[i,8]*E[t,8]+xi[i,9]*E[t,9]+
                    xi[i,10]*E[t,10]
              }
         for (k in 2:dim.space)
              {
              xi[i,k]~dnorm(0,ll[k])
              }
         }
for (k in 1:dim.space)
     {ll[k]~dgamma(1.0E-3,1.0E-3)
       lambda[k]<-1/ll[k]}
for (l in 1:2)
     {
     gamma[l]~dnorm(0,1.0E-2)
     mu_X[l]~dnorm(m_prior[l],1.0E-3)
     tau[l]~dgamma(1.0E-3,1.0E-3)
     }
mu~dnorm(0,1.0E-2)
taueps~dgamma(1.0E-3,1.0E-3)
}

The definition of most variables is the same as the definition for WinBUGS code for functional principal component analysis and functional regression analysis. We describe only the new variables here.

  1. m_xi[i] is the mean of the of the subject-specific score xi[i,1] on the first eigenfunction. This is different from the classical functional regression analysis where xi[i,1] is shrunk towards zero. The vector of parameters m_xi[] has length N_subj and is estimated in the model.

  2. gamma[] is a vector of regression parameters of length 2 relating m_xi[i] to the covariates age[i] and BMI[i]. The length 2 is not set in stone and other lengths can be accommodated when the outcome is regressed on a different set of eigenfunctions.

  3. mu is the intercept parameter of the regression of xi[i,1] on the covariates age[i] and BMI[i]. This is a parameter that will be estimated in the model.

  4. age[] and BMI[] are N_subj vectors containing the age and BMI covariates. They are loaded as data in the program.

  5. mu_X[1] and mu_X[2] are the means of the covariates age[] and BMI[]. These are two parameters that are estimated from the data and are used because some age and BMI observations are missing.

  6. tau[1] and tau[2] are the precisions of the covariates age[] and BMI[]. These are two parameters that are estimated from the data and are used because some age and BMI observations are missing.

  7. The priors for each gamma[] parameter are mutually independent normals with mean 0 and variance 100.

  8. The priors for each of mu_X[l] parameter are mutually independent normals centered at the empirical mean of the observed covariates m_prior[l] and variance 1000.

  9. m_prior[l] are the empirical means of the observed covariates and are loaded as data in the program.

  10. The priors for each of tau[l] parameter are mutually independent gammas with mean 1 and variance 1000.

E. Code for multilevel functional modeling

See Section 4 for model description.

model {
         for (i in 1:N_subj)
         {
         for (t in 1:N_obs)
              {
               W_1[i,t]~dnorm(m_1[i,t],taueps)
                W_2[i,t]~dnorm(m_2[i,t],taueps)
                m_1[i,t]<-X[i,t]+U_1[i,t]
                m_2[i,t]<-X[i,t]+U_2[i,t]
                X[i,t]<-xi[i,1]*psi_1[t,1]+xi[i,2]*psi_1[t,2]+xi[i,3]*psi_1[t,3]
                U_1[i,t]<-zi[i,1,1]*psi_2[t,1]+zi[i,2,1]*psi_2[t,2]+
                       zi[i,3,1]*psi_2[t,3]+zi[i,4,1]*psi_2[t,4]+zi[i,5,1]*psi_2[t,5]+
                       zi[i,6,1]*psi_2[t,6]+zi[i,7,1]*psi_2[t,7]+zi[i,8,1]*psi_2[t,8]+
                       zi[i,9,1]*psi_2[t,9]+zi[i,10,1]*psi_2[t,10]
                 U_2[i,t]<-zi[i,1,2]*psi_2[t,1]+zi[i,2,2]*psi_2[t,2]+
                        zi[i,3,2]*psi_2[t,3]+zi[i,4,2]*psi_2[t,4]+zi[i,5,2]*psi_2[t,5]+
                        zi[i,6,2]*psi_2[t,6]+zi[i,7,2]*psi_2[t,7]+zi[i,8,2]*psi_2[t,8]+
                        zi[i,9,2]*psi_2[t,9]+zi[i,10,2]*psi_2[t,10]
             }
         for (k in 1:dim.space_b)
             {xi[i,k]~dnorm(0,ll_b[k])}
         for (l in 1:dim.space_w)
              {zi[i,l,1]~dnorm(0,ll_w[l])
                zi[i,l,2]~dnorm(0,ll_w[l])}
               }
for (k in 1:dim.space_b)
     {ll_b[k]~dgamma(1.0E-3,1.0E-3)
       lambda_b[k]<-1/ll_b[k]}
for (l in 1:dim.space_w)
     {ll_w[l]~dgamma(1.0E-3,1.0E-3)
       lambda_w[l]<-1/ll_w[l]}
taueps~dgamma(1.0E-3,1.0E-3)
sigma_sq_eps<-1/taueps
}

The definition of most variables is the same as the definition for WinBUGS code for functional principal component analysis and functional regression analysis. The structure is very similar to the one for functional principal component analysis with the exception that there are two levels of functional regression.

  1. W_1[i,t] and W_2[i,t] are the functional observations at visit 1 and 2, respectively for subject i at time t. Both matrices W_1[,] and W_2[,] are N_subj×N_obs, are loaded as data and may contain missing observations.

  2. m_1[i,t] and m_2[i,t] are the smooth means of W_1[i,t] and W_2[i,t], respectively, are unknown and their joint distribution is simulated.

  3. X[i,t] is the subject-specific mean process. X[,] is a N_subj×N_obs dimensional matrix of parameters that are estimated from the model.

  4. U_1[i,t] and U_2[i,t] are the subject/visit specific deviations from the subject-specific mean. U_1[,] and U_2[,] are N_subj×N_obs dimensional matrices of parameters that are estimated from the model.

  5. psi_1[t,k] and psi_2[t,l] are the level 1 and 2 eigenfunctions evaluated at the time t. The matrices psi_1[,] and psi_2[,] are N_obs×3 and N_obs×10 dimensional, respectively, do not contain any missing observation and are loaded as data.

  6. xi[i,k] are the scores for the subject i on the kth level 1 eigenfunction psi_1[t,k]. xi[,] is a N_subj×3 dimensional array of parameters that are estimated from the model.

  7. zi[i,l,1] and zi[i,l,2] are the scores for the subject i on the kth level 2 eigenfunction psi_2[t,k]. zi[,,] is an N_subj×10×2 dimensional array of parameters that are estimated from the model.

  8. ll_b[k] are the precisions for the distribution of the scores xi[i,k]. ll_b[] is a vector of parameters of size 3 that are estimated from the model.

  9. ll_w[l] are the precisions for the distribution of the scores zi[i,l,1] and zi[i,l,2], as the level 2 scores are assumed exchangeable. ll_w[] is a vector of parameters of size 10 that are estimated from the model.

Contributor Information

Ciprian M. Crainiceanu, Email: ccrainic@jhsph.edu, Department of Biostatistics, Johns Hopkins University, 615 N. Wolfe St. E3636, Baltimore, MD 21205, United States of America, URL: http://www.biostat.jhsph.edu/~ccrainic/

A. Jeffrey Goldsmith, Email: jgoldsmi@jhsph.edu, Department of Biostatistics, Johns Hopkins University, 615 N. Wolfe St. E3037, Baltimore, MD 21205, United States of America.

References

  1. Baladandayuthapani V, Mallick BK, Hong MY, Lupton JR, Turner ND, Carroll RJ. Bayesian Hierarchical Spatially Correlated Functional Data Analysis with Application to Colon Carcinogenesis. Biometrics. 2008;64(1):64–73. doi: 10.1111/j.1541-0420.2007.00846.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Brezger A, Kneib T, Lang S. BayesX: Analyzing Bayesian Structured Additive Regression Models. Journal of Statistical Software. 2005;14(11):1–22. URL http://www.jstatsoft.org/v14/i11/
  3. Cardot H, Ferraty F, Sarda P. Functional Linear Model. Statistics & Probability Letters. 1999;45(1):11–22. [Google Scholar]
  4. Cardot H, Ferraty F, Sarda P. Spline Estimators for the Functional Linear Model. Statistica Sinica. 2003;13(3):571–591. [Google Scholar]
  5. Cardot H, Sarda P. Estimation in Generalized Linear Models for Functional Data via Penalized Likelihood. Journal of Multivariate Analysis. 2005;92(1):24–41. [Google Scholar]
  6. Carlin BP, Louis TA. Bayesian Methods for Data Analysis. 3. Chapman & Hall/CRC; New York: 2008. [Google Scholar]
  7. Chiou JM, Müller HG, Wang JL. Functional Quasi-likelihood Regression Models with Smooth Random Effects. Journal of the Royal Statistical Society B. 2003;65(2):405–423. [Google Scholar]
  8. Congdon P. Applied Bayesian Modelling. John Wiley & Sons; Chichester, England: 2003. [Google Scholar]
  9. Crainiceanu CM, Caffo BS, Punjabi NM. Nonparametric Signal Extraction and Measurement Error in the Analysis of Electroencephalographic Activity During Sleep. Journal of the American Statistical Association. 2009a;104(486):541–555. doi: 10.1198/jasa.2009.0020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Crainiceanu CM, Ruppert D. Likelihood Ratio Tests in Linear Mixed Models with One Variance Component. Journal of the Royal Statistical Society B. 2004;66(1):165–185. [Google Scholar]
  11. Crainiceanu CM, Ruppert D, Carroll RJ, Adarsh J, Goodner B. Spatially Adaptive Penalized Splines with Heteroscedastic Errors. Journal of Computational and Graphical Statistics. 2007;16(2):265–288. [Google Scholar]
  12. Crainiceanu CM, Ruppert D, Claeskens G, Wand MP. Likelihood Ratio Tests of Polynomial Regression Against a General Nonparametric Alternative. Biometrika. 2005;92(1):91–103. [Google Scholar]
  13. Crainiceanu CM, Staicu AM, Di C. Generalized Multilevel Functional Regression. Journal of the American Statistical Association. 2009b doi: 10.1198/jasa.2009.tm08564. To appear. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Di C, Crainiceanu CM, Caffo BS, Punjabi NM. Multilevel Functional Principal Component Analysis. Annals of Applied Statistics. 2009;3(1):458–488. doi: 10.1214/08-AOAS206SUPP. Online access 2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Gelman A. Prior Distributions for Variance Parameters in Hierarchical Models. Bayesian Analysis. 2006;1(3):515–533. [Google Scholar]
  16. Gelman A, Carlin JB, Stern HA, Rubin DB. Bayesian Data Analysis. 2. Chapman & Hall/CRC; New York: 2003. [Google Scholar]
  17. Gelman A, Rubin DB. Inference from Iterative Simulation Using Multiple Sequences. Statistical Science. 1992;7(4):457–472. [Google Scholar]
  18. Gilks WR, Richardson S, Spiegelhalter DJ. Markov Chain Monte Carlo in Practice. Chapman & Hall/CRC; New York: 1996. [Google Scholar]
  19. Goldsmith AJ, Feder J, Crainiceanu CM. Penalized Functional Regression. 2010 Manuscript. [Google Scholar]
  20. Greven S, Crainiceanu CM, Küchenhoff H, Peters A. Restricted Likelihood Ratio Testing for Zero Variance Components in Linear Mixed Models. Journal of Computational and Graphical Statistics. 2008;17(4):870–891. [Google Scholar]
  21. Guo W. Functional Mixed Effects Models. Biometrics. 2002;58(1):121–128. doi: 10.1111/j.0006-341x.2002.00121.x. [DOI] [PubMed] [Google Scholar]
  22. Indritz J. Methods in Analysis. Macmillan & Collier-Macmillan; New York: 1963. [Google Scholar]
  23. James GM. Generalized Linear Models with Functional Predictors. Journal of the Royal Statistical Society B. 2002;64(3):411–432. [Google Scholar]
  24. James GM, Hastie TG, Sugar CA. Principal Component Models for Sparse Functional Data. Biometrika. 2001;87(3):587–602. [Google Scholar]
  25. Karhunen K. Über lineare Methoden in der Wahrscheinlichkeitsrechnung. An- nales Academiæ Scientiarum Fennicæ, Series A1: Mathematica-Physica, Suomalainen Tiedeakatemia. 1947;37:3–79. [Google Scholar]
  26. Lang S, Brezger A. Bayesian P-Splines. Journal of Computational and Graphical Statistics. 2004;13(1):183–212. [Google Scholar]
  27. Loève M. Functions Aleatoire de Second Ordre. Comptes Rendus de l’Académie des Sciences. 1945:220. [Google Scholar]
  28. Morris JS, Carroll RJ. Wavelet-Based Functional Mixed Models. Journal of the Royal Statistical Society, B. 2006;68(2):179–199. doi: 10.1111/j.1467-9868.2006.00539.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Morris JS, Vanucci M, Brown PJ, Carroll RJ. Wavelet-Based NonparametricModeling of Hierarchical Functions in Colon Carcinogenesis. Journal of the American Statistical Association. 2003;98(463):573–583. [Google Scholar]
  30. Müller HG, Stadtmüller U. Generalized Functional Linear Models. The Annals of Statististics. 2005;33(2):774–805. [Google Scholar]
  31. Natarajan R, Kass RE. Reference Bayesian Methods for Generalized Linear Mixed Models. Journal of the American Statistical Association. 2000;95(449):227–237. [Google Scholar]
  32. O’Sullivan F. A Statistical Perspective on Ill-Posed Inverse Problems. Statistical Science. 1986;1(4):505–527. [Google Scholar]
  33. Quan S, Howard B, Kiley CIJ, Nieto F, O’Connor GT. The Sleep Heart Health Study: Design, Rationale, and Methods. Sleep. 1997;20:1077–85. [PubMed] [Google Scholar]
  34. Ramsay JO, Silverman BW. Applied Functional Data Analysis. Springer-Verlag; New York: 2005. [Google Scholar]
  35. Ramsay JO, Silverman BW. Functional Data Analysis. Springer-Verlag; New York: 2006. [Google Scholar]
  36. Ramsay JO, Wickham H, Graves S, Hooker G. fda: Functional Data Analysis. R package version 2.1.3. 2009 URL http://CRAN.R-project.org/package=fda.
  37. R Development Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing; Vienna, Austria: 2009. URL http://www.R-project.org/ [Google Scholar]
  38. Reiss PT, Ogden RT. Functional Principal Component Regression and Functional Partial Least Squares. Journal of the American Statistical Association. 2007;102(479):984–996. [Google Scholar]
  39. Ruppert D, Carroll RJ, Wand MP. Semiparametric Regression. Cambridge University Press; Cambridge, UK: 2003. [Google Scholar]
  40. Spiegelhalter DJ, Thomas A, Best NG, Lunn D. WinBUGS Version 1.4 User Manual. MRC Biostatistics Unit; Cambridge: 2003. URL http://www.mrc-bsu.cam.ac.uk/bugs/ [Google Scholar]
  41. Staicu AM, Crainiceanu CM, Carroll RJ. Fast Methods for Spatially Correlated Multilevel Functional Data. 2009 doi: 10.1093/biostatistics/kxp058. Manuscript. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Sturtz S, Ligges U, Gelman A. R2WinBUGS: A Package for Running WinBUGS from R. Journal of Statistical Software. 2005;12(3):1–16. URL http://www.jstatsoft.org/v12/i03/
  43. Tornoe CW. nlmeODE: Non-Linear Mixed-Effects Modelling in nlme Using Differential Equations. R package version 0.3-1. 2007 URL http://CRAN.R-project.org/package=nlmeODE.
  44. Wood S. Generalized Additive Models: An Introduction with. R. Chapman & Hall/CRC; New York: 2006. [Google Scholar]
  45. Yao F, Lee TCM. Penalized Spline Models for Functional Principal Component Analysis. Journal of the Royal Statistical Society B. 2006;68(1):3–25. [Google Scholar]
  46. Yao F, Müller HG, Wang JL. Functional Data Analysis for Sparse Longitudinal Data. Journal of the American Statistical Association. 2005a;100(470):577–590. [Google Scholar]
  47. Yao F, Müller HG, Wang JL. Functional Linear Regression Analysis for Longitudinal Data. The Annals of Statistics. 2005b;33(6):2873–2903. [Google Scholar]

RESOURCES