Bayesian Semiparametric Functional Mixed Models for Serially Correlated Functional Data, with Application to Glaucoma Data

Abstract

Glaucoma, a leading cause of blindness, is characterized by optic nerve damage related to intraocular pressure (IOP), but its full etiology is unknown. Researchers at UAB have devised a custom device to measure scleral strain continuously around the eye under fixed levels of IOP, which here is used to assess how strain varies around the posterior pole, with IOP, and across glaucoma risk factors such as age. The hypothesis is that scleral strain decreases with age, which could alter biomechanics of the optic nerve head and cause damage that could eventually lead to glaucoma. To evaluate this hypothesis, we adapted Bayesian Functional Mixed Models to model these complex data consisting of correlated functions on spherical scleral surface, with nonparametric age effects allowed to vary in magnitude and smoothness across the scleral surface, multi-level random effect functions to capture within-subject correlation, and functional growth curve terms to capture serial correlation across IOPs that can vary around the scleral surface. Our method yields fully Bayesian inference on the scleral surface or any aggregation or transformation thereof, and reveals interesting insights into the biomechanical etiology of glaucoma. The general modeling framework described is very flexible and applicable to many complex, high-dimensional functional data.

1. INTRODUCTION

Glaucoma is one of the leading causes of blindness in the world. While its etiology is not fully understood, it is known to be caused by damage to the optic nerve head (ONH) that can be induced by intraocular pressure (IOP). Researchers hypothesized that the biomechanics of the peripapillary (PP) scleral region close to the ONH, shown to be an important determinant of ONH biomechanics, may play a major role in glaucoma pathogenesis and progression (Sigal et al., 2005; Burgoyne and Downs, 2008). Recently, novel custom instrumentation was developed that can induce a fixed level of IOP and precisely measure the mechanical strain in the posterior human sclera (Fazio, Bruno, Reynaud, Poggialini and Downs, 2012; Fazio, Grytz, Bruno, Girard, Gardiner, Girkin and Downs, 2012). A commercial laser speckle interferometer (ESPI) is used to measure the IOP-induced scleral displacement continuously around the posterior eye. The scleral displacement is processed to estimate the mechanical strain tensor on a grid of points on the scleral surface, and the largest eigenvalue of this strain tensor computed to yield the maximum principal strain (MPS), a scalar measurement for each location on the scleral surface that summarizes the magnitude of tensile strain at that location. Intuitively, scleral regions with higher MPS for a given level of IOP are more pliable, which in principle could either relieve IOP and reduce the potential for ONH damage or focus strain at the ONH and increase glaucoma damage. Age is a primary glaucoma risk factor and age-related stiffening occurs in other load-bearing soft tissues. Thus, the researchers hypothesized that age-related changes in scleral stiffness might contribute to the known age-related increase in glaucoma risk.

Utilizing this custom instrumentation, Fazio et al. (2014) conducted a study to test the hypothesis that scleral stiffness increases with age, with the expectation that future work will follow to elucidate the specific role of scleral stiffness in glaucoma. They obtained twenty pairs of eyes from normal human donors in the Lions Eye Bank of Oregon in Portland, Oregon, and the Alabama Eye Bank in Birmingham, Alabama. From each subject, the MPS was measured in the posterior globes of both left and right eyes on a partial spherical domain with 120 circumferential locations ϕ ∈ (0°, 360°) and 120 meridional locations θ ∈ (9°, 24°), where θ = 0° corresponds to the ONH. For each eye, MPS was measured under nine different IOP levels (7, 10, 15, 20, 25, 30, 35, 40, and 45 mmHg). Further scientific and technical details can be found in Section 5. Figure 1 plots a polar azimuthal projection of MPS functions for one subject under 45 mmHg IOP level. The center of each panel corresponds to the ONH of each eye. The hypothesis is that MPS will decrease with age, especially in scleral regions closer to the ONH.

Figure 1. Data plot.

Polar azimuthal projection of partial spherical MPS function for both left and right eyes from one subject of age 66yr under 45 mmHg of intraocular pressure.

The resulting data set is complex and high dimensional with many layers of structure. First, for each eye at each IOP level, the MPS measurements constitute a function on a two-dimensional partial spherical domain for which one needs to account for within-function (intrafunctional) correlations. Second, there are multiple sources of between-function (interfunctional) correlation. The measurements from the left and right eye from the same subject are expected to be correlated, and measurements from the same eye across the IOP levels should be serially correlated. The strength of these nested and serial correlations may potentially vary around the scleral surface. Based on preliminary looks at the data, it appears that the age effect on MPS may not be linear, and also appears to vary around the scleral surface. This data set is also enormous, with over 4.5 million measurements, which poses practical problems for model building and fitting.

Many researchers, when faced with such complexities, would use one of several strategies to simplify the data so they can analyze them. Some researchers would get rid of the complexities of the functional data by computing summaries and modeling only those, while discarding the original functional data. In this application area, some researchers create summaries in the peripapillary (PP) and mid-peripheral (MP) scleral regions by integrating over the region closest to the ONH (9° – 17°) and further out (17° – 24°), respectively, or over several circumferential sections. This practice would miss any scientific insights not captured by these summaries. Alternatively, some researchers would only model a subset of the data (e.g. only a single IOP or single eye per subject) to avoid having to deal with potentially complex interfunctional correlation, or ignore this correlation entirely by modeling IOP within eyes or eyes within subject as independent. Some would ignore intrafunctional correlation by modeling individual pixels on the image independently. Many researchers would also only consider parametric or linear effects for covariates such as age without considering whether more complex nonparametric covariate effects might be necessary to capture the true relationship. All of these simplification strategies have potential statistical downsides.

In our opinion, it would be far preferable to model this complex function data set in its entirety using a statistical model that is flexible enough to capture all of its potentially complex intrafunctional and interfunctional structure. However, to the best of our knowledge, no statistical model has been presented in existing literature that simultaneously handles all of this structure. In this paper, we use the Bayesian functional mixed model (BayesFMM) framework, introduced in Morris and Carroll (2006) and further developed in subsequent papers, to model these data, which has sufficient flexibility to capture nonparametric covariate effects, serial and nested interfunctional correlation, functions on a fine 2d partial spherical domain, and is computationally efficient enough to scale up to this enormous size and produce Bayesian inference for functional parameters as well as any desired summaries. This requires a careful description of how to utilize the BayesFMM framework to model smooth nonparametric effects and serial interfunctional correlation, which has not been done in any existing literature to date.

While motivated by and applied to the glaucoma scleral strain data, the BayesFMM framework we present here is more generally applicable to many types of complex, high dimensional functional data of modern interest including wearable computing data, genome-wide data, proteomics data, geospatial time series data and neuroimaging data. Our hope is that besides revealing scientific insights into glaucoma, this paper can serve as a template for performing a state-of-the-art functional response regression analysis for other complex functional data sets.

The rest of the paper is organized as follows. Section 2 contains a brief review of some relevant literature in functional regression, including existing methods for modeling nested or serial interfunctional correlation and nonparametric smooth covariate functional effects. Section 3 contains methods, overviewing the BayesFMM framework and providing methodological details for how to fit serially correlated functions and nonparametric smooth functional effects in this framework and discussing its properties. Section 4 describes a model selection heuristic that can be used to assess which of the various potential complex modeling structures are needed for the current data before having to run a full MCMC. Section 5 presents the details of our analysis of the glaucoma scleral strain data, including details on basis function and modeling components chosen, a summary of results, and various sensitivity analyses. Section 6 contains a discussion of general scientific conclusions from our analysis and an assessment of the strengths and weaknesses of the BayesFMM framework for this setting of complex functional regression models. Supplementary materials provide additional computational details and graphical results of the analysis, as well as code to fit the models contained in the paper.

2. LITERATURE REVIEW

There is a rich and rapidly expanding literature on methods to perform functional regression, which includes functional predictor regression (scalar-on-function), functional response regression (function-on-scalar), and function-on-function regression. For example, see Morris (2015) for an extensive review of work in this area, which has exploded in the past decade. In this paper, our goal is to regress the scleral strain MPS functions on predictors age and IOP, so we are interested in functional response regression. Here, we will summarize some key existing literature relevant to the structures present in the glaucoma scleral strain data set, including methods that account for nested and serial interfunctional correlation and smooth nonparametric covariate effects.

As summarized in Morris (2015), much of the work on functional response regression assumes independently sampled functions, but a number of published models can handle interfunctional correlation. Many are focused on nested or crossed sampling designs that induce compound symmetry covariance structures among functions sampled within the same cluster, including Brumback and Rice (1998), Morris et al. (2003), Berhane and Molitor (2008), Aston et al. (2010), and Goldsmith and Kitago (2016). There are relatively few that model serially correlated functions, for which functions are observed at multiple levels of some continuous variable for the same subject. Most commonly, the functions occur along a grid of time points, which could be called longitudinally correlated functional data, but can also occur over other continuous variables such as IOP for our glaucoma data. There are a number of papers that focus on functional predictor regression (Goldsmith et al., 2012; Gertheiss, Goldsmith, Crainiceanu and Greven, 2013; Kundu et al., 2016; Islam et al., 2016) or estimate multi-level principal components (Greven et al., 2010; Zipunnikov et al., 2011; Chen and Müller, 2012; Li and Guan, 2014; Zipunnikov et al., 2014; Park and Staicu, 2015; Shou et al., 2015; Hasenstab et al., 2017) for serially correlated functions, but these papers do not deal directly with functional response regression, i.e. do not regress the functions on covariates while accounting for this serial correlation in the error structure. Any methods built for a general functional mixed modeling framework containing multiple levels of general random effect functions, including the BayesFMM framework first introduced by Morris and Carroll (2006) and the functional additive mixed model (FAMM) framework first introduced by Scheipl et al. (2015), can be used for functional response regression while accounting for serial interfunctional correlation if an acceptable parametric form for the serial variable can be found. However, no specific models or examples in existing papers have presented such a case. In this paper, we will demonstrate in detail how to account for serial correlation within the BayesFMM framework.

Most functional response regression work, while modeling coefficients as nonparametric in the functional domain t, has focused on models linear in covariates x, e.g. with terms such as xβ(t). A number of methods allow functional coefficients that are smooth and nonparametric in covariate x as well, e.g. f(x, t). The last chapter of Wood (2006) describes additive mixed models (AMMs) that extend Laird and Ware (1982) to include nonparametric fixed effects and parametric random effects. After specifying a parametric form for t in fixed and random effects, this framework can fit terms f(x, t) that are nonparametric in x but parametric in t using mgcv in R. Scheipl et al. (2015) describe a generalization of AMMs to the functional regression settting, yielding functional additive mixed models (FAMM) that can fit functional response, functional predictor, or function-on-function regression models, and allowing terms f(x, t) that are nonparametric in both x and t, using mgcv to fit the underlying models. Initially designed for splines, this work was extended to allow the use of functional principal components (fPC) for the random effect functions to handle sparse, irregularly sampled outcomes in Cederbaum et al. (2015), to model generalized outcomes in Scheipl et al. (2016), and to utilize a new boosting-based fitting procedure for some models that leads to other computational and modeling benefits in Brockhaus et al. (2015). This series of work is summarized in a review article Greven and Scheipl (2017a). These works utilize the fact, going way back to Wahba (1978), that penalized splines can be represented using a linear mixed model framework. Using similar representations, the BayesFMM framework first introduced in Morris and Carroll (2006) can also be used to fit nonparametric terms f(x, t) with appropriate specification of the design matrices X and Z in the model, but this has not been done in any existing paper. In this paper, we will describe in detail how to accommodate smooth nonparametric terms f(x, t) in the BayesFMM framework.

To our knowledge, the only methods in existing literature with the flexibility to both fit smooth nonparametric functional covariate terms f(x, t) and simultaneously account for nested and serial correlation are the FAMM framework (Scheipl et al., 2015; Greven and Scheipl, 2017a) and the BayesFMM framework (Morris and Carroll, 2006). Both of these frameworks are extremely flexible, based on functional extensions of linear mixed models, but have important differences, which are discussed in detail in Morris (2017) and Greven and Scheipl (2017b). The FAMM framework has several limitations that prevent its application to our glaucoma scleral strain data, including the requirement of functions to be on 1d Euclidean domains, use of spline bases for fixed effects with L2 penalties, and use of a computational approach that may not scale up well to enormous data sets like this. Thus, to fit these data, we adapt the BayesFMM framework to include smooth nonparametric age effects and serial correlation across IOP, and demonstrate how we can use it to reveal insights into the biomechanical etiology of glaucoma.

3. METHODS

We first overview the BayesFMM framework in Section 3.1, and then for ease of exposition we build up the necessary components of our model separately before presenting our final general model. In Section 3.2, we demonstrate how to capture serial interfunctional correlation through functional growth curve models, and then demonstrate how to accommodate smooth nonparametric covariate functional effects in Section 3.3. Besides presenting the models, we also describe their properties and contrast with existing alternative methods. In Section 3.4 we present a general alternative form of the BayesFMM that includes smooth nonparametric terms that we use to fit our glaucoma scleral strain data.

3.1 Bayesian Functional Mixed Models (BayesFMM)

Suppose we have a sample of functions Y_i(t), i = 1, …, N observed on a common fine grid of size T on a domain 𝒯, which is potentially multi-dimensional and/or non-Euclidean (Morris et al., 2011). The FMM introduced by Morris and Carroll (2006) is a functional response regression model given by

$$Y_i(\mathbf{t})=\sum _{a=1}^A{X}_{ia}{B}_a(\mathbf{t})+\sum _{h=1}^H\sum _{m=1}^{M_h}{Z}_{\mathit{ihm}}{U}_{hm}(\mathbf{t})+E_i(\mathbf{t}).,$$ (1)

where B_a(t) are fixed effect functions that model the effect of covariate X_ia on the response Y at position t for each covariate a = 1,…, A, and the U_hm(t) are random effect functions at level h = 1, …, H corresponding to design matrix Z_ihm, with m = 1, …, M_h being the number of random effects at the respective levels. As in linear mixed models for scalar data, the fixed and random effect predictors can be discrete or continuous, and can involve individual covariates or interactions of multiple covariates. Although not explicitly portrayed in (1), this modeling framework also accommodates functional predictors to perform function-on-function regression (Meyer et al., 2015).

Distributional and Covariance Assumptions

Here, for simplicity, we first describe the Gaussian FMM with conditionally independent random effect and residual error functions, and then mention some other alternatives available in this framework. In this case, the random effect functions U_hm(t) are iid mean zero Gaussian Processes with intrafunctional covariance cov{U_hm(t₁), U_hm(t₂)} = Q_h(t₁, t₂) and the residual error functions E_i(t) are iid mean zero Gaussian Processes with intrafunctional covariance cov{E_i(t₁),E_i(t₂)} = S(t₁, t₂). Other extensions of this framework allow the option of conditional autoregressive (CAR) (Zhang et al., 2014) or Matern spatial covariance or AR(p) temporal interfunctional correlation structures in the residual errors (Zhu et al., 2014). Although focusing on Gaussian regression here, a robust version of this framework assuming heavier tailed distributions on the random effects or residuals is available (Zhu et al., 2011) if robustness to outliers is desired, and can also be utilized with any other features or modeling components in the BayesFMM framework.

Basis Transform Modeling Approach

A basis transform modeling approach is used to fit model (1). This first involves representing the observed functions with a basis expansion with a set of basis functions ψ_k(t), k = 1, …, K:

$$Y_i(\mathbf{t})=\sum _{k=1}^K{Y}_{ik}^{\ast }{\psi }_k(\mathbf{t})$$ (2)

While initially developed for wavelets (Morris and Carroll, 2006), as first discussed in Morris et al. (2011), the modeling approach can be used with any basis. It is meant to be used with lossless transforms with $Y_i(\mathbf{t})\equiv {\sum }_k{Y}_{ik}^{\ast }{\psi }_k(\mathbf{t})$ for all observed t, so that the basis coefficients { $Y_{ik}^{\ast }$; k = 1, …, K} contain all information within the observed functional data {Y_i(t); t = t₁, …, t_T }, or at least near-lossless with

$$\Vert Y_i(\mathbf{t})-\sum _{k=1}^K{Y}_{ik}^{\ast }{\psi }_k(\mathbf{t})\Vert <\epsilon \forall i=1,\dots ,N$$ (3)

for some small value ε and measure ||●||. This assures that the chosen basis is sufficiently rich such that for practical purposes it can recapitulate the observed functional data, and visual inspection of the raw functions and basis transformation should reveal virtually no difference. Any basis functions can be used, including commonly used choices splines, wavelets, Fourier bases, PCs or creatively constructed custom bases, and can be defined on multi-dimensional or non-Euclidean domains 𝒯.

Rather than including the bases in a design matrix and using scalar regression methods to fit the model, our approach is to transform the observed functions into the basis space to obtain the basis coefficients Y^*, a N ×K matrix for which element (i, k) contains the basis coefficient k for observed function i, fit a basis-space version of the FMM to these coefficients, and then transform results back to the data space model (1) for estimation and inference. With basis representation written in matrix form Y = Y^*Ψ with Ψ a K ×T matrix of basis functions evaluated on the observational grid with Ψ_kj = ψ_k(t_j ), the coefficients can be computed by Y^* = YΨ′ with Ψ′ = Ψ′(ΨΨ′)⁻¹ as long as rank(Ψ) = K, or for certain basis functions including wavelets and Fourier bases their special structure enables fast algorithms for computing these coefficients.

Basis Space Model

Thus, rather than fitting model (1) directly, the basis-space version of the model is fit for each basis coefficient k = 1, …, K:

$$Y_{ik}^{\ast }=\sum _{a=1}^A{X}_{ia}{B}_{ak}^{\ast }+\sum _{h=1}^H\sum _{m=1}^{M_h}{Z}_{\mathit{ihm}}{U}_{\mathit{hmk}}^{\ast }+E_{ik}^{\ast },$$ (4)

where $B_{ak}^{\ast },U_{\mathit{hmk}}^{\ast }$, and $E_{ik}^{\ast }$ are basis coefficients for the functional fixed effects $B_a(\mathbf{t})={\sum }_k{B}_{ak}^{\ast }{\psi }_k(\mathbf{t})$, functional random effects $U_{hm}(\mathbf{t})={\sum }_k{U}_{\mathit{hmk}}^{\ast }{\psi }_k(\mathbf{t})$, and functional residuals $E_i(\mathbf{t})={\sum }_k{E}_{ik}^{\ast }{\psi }_k(\mathbf{t})$, respectively. While in principle correlation across basis coefficients can be accommodated, for complex, high-dimensional functions, it may be beneficial to model these basis coefficients independently. For the Gaussian FMM with conditionally independent random effect and residual error functions, it is assumed $U_{\mathit{hmk}}^{\ast }~N(0,q_{hk})$ and $E_{ik}^{\ast }~N(0,s_k)$ with q_hk and s_k scalar variance components. Although modeling independently in the basis space, this structure induces intrafunctional correlation according to the chosen basis functions. For example, for the residual error functions, the induced T × T intrafunctional correlation S with S_ij = S(t_i, t_j) is given by:

$$\mathbf{S}={\mathbf{\Psi }}^{\prime }{\mathbf{S}}^{\ast }\mathbf{\Psi },$$ (5)

where S^* = diag{s_k; k = 1, …, K}, and Q_h defined likewise. For suitably chosen basis functions that effectively capture the characteristic structure of the observed functions Y_i(t), this can allow a flexible class of covariance structures indexed by K covariance parameters. One can assess the suitability of these assumptions by taking the basis-space variance components, computing (5), and plotting this covariance matrix to see if it appears to capture the salient structure. Figure 3 panels (e) and (f) plot the intrafunctional correlation structure induced by the chosen tensor wavelet basis for the scleral strain MPS data set for a particular scleral location for the eye-to-eye random intercepts and residual error functions, and the file intrafunctional_correlation.mp4 in the supplement shows a more extensive summary of all random levels across scleral locations.

Figure 3. Key Summaries of Fitted Model.

Shows key summaries at the scleral position marked by the open circle, including (a) the fitted MPS for a 90yr old with IOP=45mmHg, (b) the nonparametric MPS vs. age curve using the AUC to integrate over IOP, with blue line being posterior mean, dotted and solid red lines pointwise and joint credible bands, and the raw data (computing AUC for this scleral location for each eye) indicated by dots, (c) the degrees of freedom of the nonparametric age fit as a function of the scleral location, (d) the serial correlation across IOP induced by the model at this scleral position, and (e) and (f) being the intrafunctional correlation surface induced by our model and choice of tensor basis for the eye-to-eye random intercept and residual error levels, respectively, at this scleral position. The file combo_plot.mp4 in the supplement is a movie file showing how these summaries vary across scleral locations.

Shrinkage Priors for Regularization of Fixed Effects

If fitting this model using a frequentist approach, L1 or L2 penalties could be imposed on the basis space fixed effects to induce regularization/smoothing of the fixed effect functions B_a(t), as described in Morris (2015). However, this framework was designed to use a Bayesian modeling approach, in which case regularization of the fixed effect functions B_a(t) is accomplished through specification of shrinkage priors for the corresponding basis coefficients:

$$B_{ak}^{\ast }~g({\mathit{\gamma }}_{aj})$$ (6)

for some mean zero distribution g(●) with corresponding regularization parameters γ_aj indexed by j = 1, …, J that define a partitioning of the basis coefficients k = 1, …, K into regularization sets, which are subsets of basis coefficients sharing the same regularization parameters. Morris and Carroll (2006) used the spike-slab prior (George and McCulloch, 1993) for g(●) and we also use that here, but other alternatives include Gaussian, Laplace (Park and Casella, 2008), Horseshoe (Carvahlo et al., 2010), Normal-Gamma (Griffin and Brown, 2010), and Dirichlet-Laplace (Bhattachary et al., 2015). The regularization parameters γ_aj can be given hyperpriors or be estimated by empirical Bayes, which is described in detail for the spike-slab in Morris and Carroll (2006).

Model Fitting Approach

A fully Bayesian modeling approach is used to fit a Markov Chain Monte Carlo (MCMC) to the basis space model (4) for each k, and then transforming back to the data space, e.g. using $B_a(\mathbf{t})={\sum }_k{B}_{ak}^{\ast }{\psi }_k(\mathbf{t})$ to yield posterior samples for each parameter in the data-space FMM (1). This requires specification of priors for the variance components; a vague empirical Bayes approach is used that centers the prior on REML starting values of these parameters, minimally informative to be equivalent to two data points of information, as detailed in Section 5. BayesFMM fits a marginalized version of model (4) with all random effects U_* integrated out, which integrates over the interfunctional correlation induced by the random effect levels when updating the fixed effects, and speeds convergence of the chain and calculations since the random effect functions themselves need not be sampled. As detailed in the supplement, the fixed effect updates involve conjugate (spike-slab) Gibbs steps and variance components are updated via Metropolis-Hastings with proposal variances automatically computed based on the corresponding Fisher information. If desired, posterior samples for random effects can be obtained by sampling from conjugate Gaussians.

Bayesian Inference

Given these posterior samples, one can compute any desired posterior probabilities and pointwise or joint credible bands that can be used for Bayesian inference. Joint bands can be constructed using the approach described in Ruppert et al. (2003). These inferential summaries can be constructed in the data or basis space for any functional of model parameters, including contrasts (e.g. B₁(t) – B₂(t)), nonlinear transformations (e.g. exp{B_a(t)}), derivatives (e.g. ∂f(x_a, t)/∂x_a), or integrals (e.g. ∫_t∈𝒯0 B_a(t)dt for some 𝒯₀ ⊂ 𝒯) aggregating information across regions of t. We make use of these to produce inference on numerous scientifically interesting summaries for our scleral strain MPS data in Section 5.

Example FMM for Scleral Strain MPS Data

To illustrate this framework, suppose that we model MPS functions from both left and right eyes for each subject, but only for a specific IOP level, and suppose we are willing to assume a linear age effect. Let Y_i1(t) and Y_i2(t) be MPS functions for the left and right eyes respectively from the i-th subject, with t ∈ 𝒯 indexing the scleral domain, with t = (θ, ϕ) being spherical coordinates on the scleral surface. We could represent these data with the following FMM:

$$Y_{ij}(\mathbf{t})=B_0(\mathbf{t})+X_{\text{age},i}{B}_{\text{age}}(\mathbf{t})+U_i(\mathbf{t})+E_{ij}(\mathbf{t}),$$ (7)

with U_i(t) ~ N{0,Q(●)} and E_ij(t) ~ N{0, S(●)}. The U_i(t) induce a generalized compound symmetry covariance structure between the functions from the left and right eyes from the same subject, with cov{Y_ij(t), Y_ij′(t)} = Q(t, t) + S(t, t)/(j = j′). To simultaneously model data for all IOP, we would need to add a random effects level to capture the serial correlation across MPS functions for different IOP for the same eye. We next describe how this can be done.

3.2 Accommodating Serial Interfunctional Correlation via Functional Growth Curves

Recall that in our scleral strain MPS data set, for each eye we have MPS functions from each of a series of IOP levels ranging from 7 mmHg to 45 mmHg, which induces a serial correlation across scleral strain functions for the same eye according to IOP. It is important to account for this serial interfunctional correlation in some appropriate fashion in order to obtain efficient estimates and accurate inference for any fixed effect functions in the model. Here we demonstrate how functional random effects in a functional mixed modeling framework can be used to capture this serial correlation using a type of functional growth curve model, and discuss the properties of this strategy in the context of the BayesFMM framework using a basis transform modeling approach.

Basic Growth Curve Model for Scleral Strain MPS Data

For illustration, here we only consider the serial effect of IOP and omit any age effect and consider only the left eye for each subject. Suppose Y_ip(t) is the MPS function for the i^th subject, i = 1, …, N, after exposure to an IOP level of p. Note that Y_ip(t) is a function of t that varies across serial variable p. To capture the serial effect of p, we consider the following functional growth curve model:

$$Y_{ip}(\mathbf{t})=m(p,\mathbf{t})+u_i(p,\mathbf{t})+E_{ip}(\mathbf{t}),$$ (8)

where m(p, t) is the mean MPS for IOP level p and scleral location t, u_i(p, t) is a mean zero random effect for subject i that represents a subject-specific growth curve in p that is allowed to vary across scleral location t, and E_ip(t) are residual error functions assumed to be independent and identically distributed mean zero Gaussians with covariance S(●). If we can find a suitable parametric form for the serial effect of p with basis functions G_d(p), d = 0, 1, …, D, we assume $m(p,\mathbf{t})={\sum }_{d=0}^D{B}_d(\mathbf{t})G_d(p)$ and $u_i(p,\mathbf{t})={\sum }_{d=0}^D{U}_{i,d}(\mathbf{t})G_d(p)$ with cov{U_i,d(t₁), U_i,d(t₂)} = Q_d(t₁, t₂). In practice, we recommend using basis functions that are orthogonal across d, i.e. ∫ G_d(p)G_d′(p)dp = 0 for d ≠ d′ to obviate the need to have cross-covariance terms between U_i,d(t) and U_i,d′(t) to avoid additional computational complexity in the model.

The introduction of this eye-level random growth curve induces serial covariance across functions for the same eye at different levels of IOP, with $cov\{Y_{ip}(\mathbf{t}),Y_{{ip}^{\prime }}(\mathbf{t})\}={\sum }_{d=0}^D{G}_d(p)G_d(p^{\prime })Q_d(\mathbf{t},\mathbf{t})$. Indexed by t, the strength and shape of this serial covariance can vary across the scleral surface t. Figure 3 panel (d) contains the induced serial correlation across IOP for the marked scleral location, and the file IOP_corr.mp4 in the supplementary materials is a movie file that demonstrates how this correlation varies over the scleral surface. Note also that $cov\{Y_{ip}({\mathbf{t}}_1),Y_{{ip}^{\prime }}({\mathbf{t}}_2)\}={\sum }_{d=0}^D{G}_d(p)G_d(p^{\prime })Q_d({\mathbf{t}}_1,{\mathbf{t}}_2)$, meaning that this structure enables “borrowing of strength” from nearby t in determining the strength and shape of the serial covariance according to the intrafunctional covariance indicated by the off-diagonal elements of Q_d(●). If model (8) is marginalized with respect to the random effect functions, the resulting error terms can be seen to contain the induced serial correlation structure, which is subsequently accounted for in any estimation or inference of the fixed effect functions.

Incorporation into BayesFMM framework

It can be seen that model (8) can be written as a functional mixed model with D+1 fixed effect predictors and D+1 random effect levels, each with N subject-specific random effect functions. Thus, any FMM framework allowing multiple levels of random effect functions could be used to fit this model. In the BayesFMM framework, after transforming the observed functions Y_ip(t) into the basis space through $Y_{ip}(\mathbf{t})={\sum }_{k=1}^K{Y}_{\mathit{ipk}}^{\ast }{\psi }_k(\mathbf{t})$ as described in Section 3.1, the model for coefficient k would be given by:

$$Y_{\mathit{ipk}}^{\ast }=\sum _{d=0}^D{G}_d(p)B_{dk}^{\ast }+\sum _{d=0}^D{G}_d(p)U_{\mathit{idk}}^{\ast }+E_{\mathit{ipk}},$$ (9)

with $U_{\mathit{idk}}^{\ast }~N(0,q_{dk})$ and $E_{\mathit{ipk}}^{\ast }~N(0,s_k)$. This would induce serial covariance across p in the model for each basis coefficient with $cov(Y_{\mathit{ipk}}^{\ast },Y_{{ip}^{\prime }k}^{\ast })={\sum }_{d=0}^D{G}_d(p)G_d(p^{\prime })q_{dk}$, and the marginalized model that is fit with the $U_{\mathit{idk}}^{\ast }$ integrated out will contain this serial covariance in the error structure and explicitly account for it when updating the fixed effect coefficients. This basis space model induces the serial covariance structure described in the data space model (8), with $Q_d({\mathbf{t}}_1,{\mathbf{t}}_2)={\sum }_{k=1}^K{\psi }_k({\mathbf{t}}_1){\psi }_k({\mathbf{t}}_2)q_{dk}$. The heteroscedasticity of the variance components across basis functions k allows the strength and shape of the serial covariance to vary across the scleral surface t, and effectively borrows strength from nearby t through the chosen basis functions as determined by the induced off-diagonal elements of Q_d(t₁, t₂). Figure 3 panel (e) plots the intrafunctional correlation structure corresponding the random intercept Q₀(t₁, t₂) induced by the tensor wavelet basis chosen for the scleral strain MPS data set at the marked scleral location, and file intrafunctional_correlation.mp4 in the supplement presents the induced intrafunctional correlation for each of the Q_d(t₁, t₂), (d = 0, 1, 2) across all scleral locations.

This strategy can be used with any parametric model indicated by the G_d(p), d = 0, …, D, preferably orthogonalized. Section 5 demonstrates that the IOP effect in the scleral strain MPS data is hyperbolic, so we devise an orthogonalized hyperbolic model for these data. Also, note that while the serial variable IOP is sampled on a common grid across subjects for our data, this strategy can allow each the grid points for the serial variable to vary across subjects.

3.3 Smooth Nonparametric Covariate Functional Effects

One of the primary scientific goals in the scleral strain MPS data is to study the effect of age on MPS and assess how it varies around the scleral surface. Preliminary investigations of the data suggest that the age effect might not follow a simple parametric form, and a nonparametric representation might be appropriate. While the fixed effect functions in the BayesFMM framework are linear in the covariates, using the mixed model representation of penalized splines shown by Wahba (1978), it is possible to fit a semiparametric functional mixed model with a smooth nonparametric age effect using this framework, as we will demonstrate in this section.

Smooth Nonparametric Age Effect for Scleral Strain MPS data

For ease of exposition, in this section we just consider a single smooth nonparametric term with no other covariates or random effects. Thus, suppose for each subject i = 1, …, N we only model a single scleral strain function Y_i(t), say for left eye and IOP=45mmHg, with the model:

$$Y_i(\mathbf{t})=B_0(\mathbf{t})+f(X_{{\text{age}}_i},\mathbf{t})+E_i(\mathbf{t}),$$ (10)

where B₀(t) is a functional intercept and f(X_agei, t) represents a nonparametric effect of age on MPS at scleral location t, with ∫ f(x, t)dx = 0∀t and penalizing ∫{f″(x, t)}²dx to induce smoothness across x for each t. As we will demonstrate, it is possible to represent this smooth nonparametric term as a sum of a linear fixed effect function for age and spline random effect functions,

$$f(X_{\text{age},i},\mathbf{t})=X_{{\text{age}}_i}{B}_1(\mathbf{t})+\sum _{m=1}^{M+2}{Z}_{\mathcal{B},m}(X_{{\text{age}}_i})U_{\mathcal{S}m}(\mathbf{t})$$ (11)

for some suitably constructed random effect design matrix {Z_ℬ,m(x),m = 1, …, M + 2} based on Demmler-Reinsch basis functions (Demmler and Reinsch, 1975), with spline random effects U_𝒮m(t) following a mean zero Gaussian with cov{U_𝒮m(t₁), U_𝒮m(t₂)} = Q_𝒮(t₁, t₂). These model components can be incorporated within a FMM framework like BayesFMM, as we now describe.

Incorporation into BayesFMM framework

To fit model (10) using the BayesFMM framework, we fit separate penalized splines for each basis coefficient k, which induces correlated penalized spline fits for each scleral location t. Specifically, after transforming the observed functions Y_i(t) into the basis space according to the basis representation $Y_i(\mathbf{t})={\sum }_{k=1}^K{Y}_{ik}^{\ast }{\psi }_k(\mathbf{t})$ as described in Section 3.1, we specify the following model for each basis coefficient k = 1, …, K:

$$Y_{ik}^{\ast }=B_{0k}^{\ast }+f_k^{\ast }(X_{{\text{age}}_i})+E_{ik}^{\ast },$$ (12)

with $E_{ik}^{\ast }~N(0,s_k)$ and $f_k^{\ast }(x)$ a smooth nonparametric function of x for basis function k. We pull out the intercept $B_{0k}^{\ast }$ and constrain $\int f_k^{\ast }(x)dx=0$ to ensure identifiability in additive models that contain multiple smooth nonparametric terms. We represent $B_{0k}^{\ast }+f_k^{\ast }(x)$ using B-spline basis functions,

$$B_{0k}^{\ast }+f_k^{\ast }(x)=\sum _{m=1}^{M+4}{\mathcal{B}}_m(x){\nu }_{mk}^{\ast },$$ (13)

where ${\nu }_{mk}^{\ast }$ are B-spline coefficients and ℬ_m(x),m = 1, …, M + 4 are the cubic B-spline basis functions defined by the knots η₁, …, η_M+8 such that

$$a={\eta }_1={\eta }_2={\eta }_3={\eta }_4<{\eta }_5<\cdots <{\eta }_{M+4}={\eta }_{M+5}={\eta }_{M+6}={\eta }_{M+7}={\eta }_{M+8}=b,$$

and a and b are two boundary knots (Hastie et al., 2009). We can write model (12) in matrix form,

$${\mathbf{y}}_k^{\ast }=\mathcal{B}{\mathit{\nu }}_k^{\ast }+{\mathbf{e}}_k^{\ast },$$ (14)

where ${\mathbf{y}}_k^{\ast }={(Y_{1k}^{\ast },\dots ,Y_{Nk}^{\ast })}^{\prime }$, ℬ is the N × (M + 4) B-spline design matrix with the (i,m)-th entry being ℬ_m(X_age,i), ${\mathit{\nu }}_k^{\ast }={({\nu }_{1k}^{\ast },\dots ,{\nu }_{(M+4)k}^{\ast })}^{\prime }$, and ${\mathbf{e}}_k^{\ast }={(E_{1k}^{\ast },\dots ,E_{Nk})}^{\prime }~N(\mathbf{0},s_k{I}_N)$. Following Wand and Ormerod (2008), we assume the following prior distribution on the B-spline coefficients:

$${\mathit{\nu }}_k^{\ast }~\mathit{MVN}(\mathbf{0},q_{\mathcal{S}k}\mathbf{\Omega })$$ (15)

where Ω is a (M + 4) × (M + 4) matrix with ${\mathbf{\Omega }}_{m{m}^{\prime }}={\int }_a^b{\mathcal{B}}_m^{″}(x){\mathcal{B}}_{m^{\prime }}^{″}(x)dx$. The resulting posterior mean of the spline random effects is ${\widehat{\mathit{\nu }}}_k^{\ast }={({\mathcal{B}}^{\prime }\mathcal{B}+{\lambda }_k^{\ast }\mathrm{\Omega })}^{-1}{\mathcal{B}}^{\prime }{\mathbf{y}}_k^{\ast }$, where ${\lambda }_k^{\ast }=s_k/q_{\mathcal{S}k}$. It can be shown that $\mathcal{B}{\widehat{\mathit{\nu }}}_k^{\ast }$ corresponds to the O’Sullivan penalized spline estimator of $B_{0k}^{\ast }+f_k^{\ast }(x)$ with penalty term ${\lambda }_k^{\ast }{\int }_a^b{\{{f_k^{\ast }}^{″}(x)\}}^{2}dx$ (Wand and Ormerod, 2008), and if the knots are placed at each observed X_agei, then this corresponds to the cubic smoothing spline estimator.

The spectral decomposition of Ω allows us to reformulate this prior specification as a mixed model with independent random effects as follows. It is known that rank(Ω) = M + 2. Therefore, the spectral decomposition of Ω has the form of Ω = PDP′, where D = diag(0, 0, d₁, …, d_M+2) and P′P = I_M+4. Let P = (X_Ω, Z_Ω), where X_Ω is a (M +4) × 2 sub-matrix of P corresponding to the first two columns of P and Z_Ω is a (M +4) × (M + 2) sub-matrix of P corresponding to the other columns. Let ${\mathit{\beta }}_k^{\ast }$ be a two-dimensional vector, and ${\mathbf{u}}_{\mathcal{S}k}^{\ast }={(U_{\mathcal{S}1k}^{\ast },\dots ,U_{\mathcal{S}(M+2)k}^{\ast })}^{\prime }$ be an (M + 2)-dimensional random vector. It can be shown that ${\mathit{\nu }}_k^{\ast }=X_{\mathrm{\Omega }}{\mathit{\beta }}_k^{\ast }+Z_{\mathrm{\Omega }}\text{diag}(d_1^{-1/2},\dots ,d_{M+2}^{-1/2}){\mathbf{u}}_{\mathcal{S}k}^{\ast }$ with ${\mathit{\beta }}_k^{\ast }$ a fixed effect and ${\mathbf{u}}_{\mathcal{S}k}^{\ast }~\mathit{MVN}(\mathbf{0},q_{\mathcal{S}k}{I}_{M+2})$ Therefore, we have the following mixed model representation of (12),

$$\begin{gathered} {\mathbf{y}}_k^{\ast }=\mathcal{B}{\mathit{\nu }}_k^{\ast }+{\mathbf{e}}_k^{\ast } \\ =\mathcal{B}\{X_{\mathrm{\Omega }}{\mathit{\beta }}_k^{\ast }+Z_{\mathrm{\Omega }}\text{diag}(d_1^{-1/2},\dots ,d_{M+2}^{-1/2}){\mathbf{u}}_{\mathcal{S}k}^{\ast }\}+{\mathbf{e}}_k^{\ast } \\ =X_{\mathcal{B}}{\mathit{\beta }}_k^{\ast }+Z_{\mathcal{B}}{\mathbf{u}}_{\mathcal{S}k}^{\ast }+{\mathbf{e}}_k^{\ast }, \end{gathered}$$ (16)

where X_ℬ = ℬX_Ω and $Z_{\mathcal{B}}=\mathcal{B}{Z}_{\mathrm{\Omega }}\text{diag}(d_1^{-1/2},\dots ,d_{M+2}^{-1/2})$. The Z_ℬ are called the Demmler-Reinsch spline bases (Demmler and Reinsch, 1975). It can be shown that X_ℬ is a basis for the space of the straight line, so model can equivalently be rewritten as

$${\mathbf{y}}_k^{\ast }={\mathbf{1}}_N{B}_{0k}^{\ast }+{\mathbf{x}}_{\text{age}}{B}_{1k}^{\ast }+Z_{\mathbf{B}}{\mathbf{u}}_{\mathcal{S}k}^{\ast }+{\mathbf{e}}_k^{\ast },$$ (17)

where 1_N is an N-dimensional vector consisting of 1’s and x_age = (X_age,1, …, X_age,N )′. We see that this is the form of a linear mixed model with a level of spline random effects with design matrix Z_ℬ and random effects ${\mathbf{u}}_{\mathcal{S}k}^{\ast }~\mathit{MVN}(\mathbf{0},q_{\mathcal{S}k}{I}_{M+2})$. With the intercept term pulled out as implied by the $\int f_k^{\ast }(x)dx=0$ assumption, the term $f_k^{\ast }(X_{{\text{age}}_i})$ in (12) is given by $X_{{\text{age}}_i}{B}_{1k}^{\ast }+Z_{\mathcal{B}}(X_{{\text{age}}_i}){\mathbf{u}}_{\mathcal{S}k}^{\ast }$ and thus in the BayesFMM framework we can incorporate a nonparametric term $f_k^{\ast }(x)$ by simply including a linear fixed effect $x{B}_{1k}^{\ast }$ plus a level of random effects with the corresponding Demmler-Reinsch design matrix ${\sum }_{m=1}^{M+2}{Z}_{\mathcal{B}m}(x)U_{\mathcal{S}mk}^{\ast }$. These penalized splines for each basis k, when projected back to the function space, induce a smooth nonparametric functional effect f(X_agei, t) given by (11), with Q_𝒮(t₁, t₂) = Σ_k ψ_k(t₁)ψ_k(t₂)q_𝒮k. Based on these derivations, we can add any additional smooth nonparametric term f(z, t) to the FMM framework by simply adding a linear fixed effect function zB₂(t) and an additional level of spline random effects ${\sum }_{m=1}^{M_z+2}{Z}_{\mathcal{B}(z)}{U}_{{\mathcal{S}}_{z}m}(\mathbf{t})$ with U_𝒮zm(t) a mean zero Gaussian with covariance cov{U_𝒮zm(t₁), U_𝒮zm(t₂)} = Q_𝒮z (t₁, t₂).

A similar procedure could be followed to utilize other spline modeling approaches within this framework, e.g. P-splines (Eilers and Marx, 1986) with differencing penalties or truncated polynomial splines (Ruppert et al., 2003), but we prefer the O’Sullivan splines (Wand and Ormerod, 2008) given their natural second derivative penalty and formal connection to smoothing splines.

Intrafunctional Correlation of f(x, t) Across t

This framework allows the nonparametric smooth effect f(x, t) of x to vary over t, but is not the same as modeling independent splines for each t. Because the splines are fit in the basis space, the nonparametric fits are correlated intrafunctionally by:

$$cov\{f(x,{\mathbf{t}}_1),f(x,{\mathbf{t}}_2)\mid B_1({\mathbf{t}}_1),B_1({\mathbf{t}}_2),q_{\mathcal{S}m}\}=\sum _{k=1}^K\sum _{m=1}^{M+2}{\psi }_k({\mathbf{t}}_1){\psi }_k({\mathbf{t}}_2){\{Z_{{\mathcal{B}}_m(x)}\}}^2{q}_{\mathcal{S}m}.$$ (18)

This means that the spline fit for t₁ borrows strength from other functional locations t₂ according to the effective intrafunctional covariance structure Q_𝒮(t₁, t₂) that induces smoothing across t in the spline fits of f(x, t).

Smoothing Parameter of x, λ(t), is Nonstationary and Smooth Across t

In our BayesFMM implementation of the smooth nonparametric term f(x, t), we allow the penalized spline for each basis function k to have its own smoothing parameter λ_k = s_k/q_𝒮k. The basis space model induces a residual error covariance matrix cov{E_i(t₁),E_i(t₂)} = S_t1,t2 back in the data space, with diagonal elements s(t), and a spline random effect covariance matrix cov{U_𝒮m(t₁), U_𝒮m(t₂)} = Q_𝒮(t₁, t₂) back in the data space, with diagonal elements q_𝒮(t). Thus, the effective smoothing parameter for the induced spline fit f(x, t) at location t is given by

$$\lambda (\mathbf{t})=s(\mathbf{t})/q_{\mathcal{S}}(\mathbf{t}),$$ (19)

meaning that the smoothness in x is allowed to vary across t, enabling some parts of the function to be linear with large λ(t) and others to be nonlinear with small λ(t). Also, this smoothing parameter is not estimated independently for each t, but the off-diagonal elements of S and Q_𝒮 imply a dependency across t in λ(t), meaning that the model “borrows strength” across t leading to smoothness in λ(t) across t.

We believe this to be the first presentation of a model with such flexibility in the literature, i.e. with f(x, t) varying smoothly across t with the smoothing parameter in x, λ(t), also varying smoothly across t. The FAMM models of Scheipl et al. (2015) and Greven and Scheipl (2017a) estimate terms like f(x, t) that are smooth across both x and t, but utilize an additive penalty term involving marginal smoothing parameters in the x and t directions, λ_x and λ_t. This structure does not allow the type of nonstationarities enabled here, which in Section 5 of the supplement we demonstrate are necessary to accurately model the scleral strain MPS data. It may be possible in the FAMM framework to accommodate this type of flexibility by putting a spline on λ_x that varies smoothly across t, but this has not been done in any published paper to date, and it is not clear whether such an approach would be computationally feasible for large functional data sets.

Degrees of Freedom Function DF(t)

In the penalized spline literature with penalized spline estimator given by f̂(x) = ℬ(ℬ′ℬ+λΩ)′¹ℬ′y = X(λ)y, a standard summary of the nonlinearity of the fit is given by the dimensionality of the projection space given by DF = trace{X(λ)}, called the degrees of freedom of the fit. A DF = 2 indicates a linear model and DF ≫ 2 indicates significant nonlinearity. To assess how the degree of nonlinearity of the spline fit f(x, t) varies over t, we can compute the degrees of freedom function DF(t) marginally across t by

$$DF(\mathbf{t})=\text{trace}[X\{\lambda (\mathbf{t})\}]=\text{trace}[\mathcal{B}{\{{\mathcal{B}}^{\prime }\mathcal{B}+\lambda (\mathbf{t})\mathrm{\Omega }\}}^{-1}{\mathcal{B}}^{\prime }],$$ (20)

with λ(t) defined as in (19). In general semiparametric functional mixed models with other levels of random effects to account for interfunctional covariance according to model (21) below, as necessary for modeling our scleral strain MPS data, the derivation for DF(t) is more complex, and outlined in Section 2 of the supplementary materials. Panel (c) of Figure 3 presents DF(t) for the MPS data.

3.4 General Bayesian Semiparametric Functional Mixed Model

In order to model the highly structured scleral strain MPS data set, we need include all of the modeling structures described in the preceding sections, including random effects to capture nested and serial interfunctional correlation and smooth nonparametric smooth covariate effect functions, together in a common BayesFMM model. To highlight its ability to model smooth nonparametric structures as described in Section 3.3, it is useful to adapt the notation of the core BayesFMM model to explicitly include these terms. We term this version of the FMM a semiparametric functional mixed model since it includes both linear and smooth covariate effects.

Given a sample of functions Y_i(t); i = 1, …, N; t ∈ 𝒯, with covariates for fixed linear effects X_ial, a_l = 1, …, A_l, smooth nonparametric effects X_ian, a_n = 1, …, A_n, and H levels of random effect covariates Z_ihm, h = 1, …, H;m = 1, …, M_h, we have the following semiparametric FMM:

$$Y_i(\mathbf{t})=\sum _{a_l=1}^{A_l}{X}_{i{a}_l}{B}_{a_l}(\mathbf{t})+\sum _{a_n=1}^{A_n}f(X_{i{a}_n},\mathbf{t})+\sum _{h=1}^H\sum _{m=1}^{M_h}{Z}_{\mathit{ihm}}{U}_{hm}(\mathbf{t})+E_i(\mathbf{t}),$$ (21)

with U_hm(t) ~ GP(0,Q_h) and E_i(t) ~ GP(0, S) being mean zero Gaussian processes with covariance surfaces Q_h, h = 1, …, H and S defined on 𝒯 × 𝒯.

Using the structures defined in Section 3.3, this model can be directly fit by the BayesFMM software of Morris and Carroll (2006) using the following FMM:

$$Y_i(\mathbf{t})=\sum _{a_l=1}^{A_l}{X}_{i{a}_l}{B}_{a_l}(\mathbf{t})+\sum _{a_n=1}^{A_n}{X}_{i{a}_n}{B}_{a_n}(\mathbf{t})+\sum _{a_n=1}^{A_n}\sum _{m_{a_n}=1}^{M_{a_n}+2}{Z}_{\mathcal{B}{m}_{a_n}}(X_{i{a}_n})U_{\mathcal{S}{a}_n{m}_{a_n}}(\mathbf{t})+\sum _{h=1}^H\sum _{m=1}^{M_h}{Z}_{\mathit{ihm}}{U}_{hm}(\mathbf{t})+E_i(\mathbf{t}),$$ (22)

with M_an being the number of interior knots for the spline for X_ian, Z_ℬman (X_ian) the corresponding Demmler-Reinsch design matrix, U_𝒮anman (t) ~ GP(0,Q_𝒮an) the corresponding spline random effect functions, and U_hm(t) ~ GP(0,Q_h) and E_i(t) ~ GP(0, S) modeling the interfunction covariance structure. As described above, the model would be fit in the transformed basis space. We first fit the basis space model with all random effects integrated out, and then sample the spline random effects from their complete conditional distribution while integrating out the other H levels of random effects that capture any interfunctional covariance, and then project back to the function space in order to construct posterior samples of f(x_an, t) on any desired grid of t.

While omitted from (21) for ease of presentation, this model can also be easily made to include any desired parametric-nonparametric interaction terms, with interaction of parametrically modeled covariate X_ial and nonparametrically modeled covariate X_ian being represented by the term X_ialf_al(X_ian, t). For X_ial that are categorical dummy variables, this allows separate nonparametric fits of (X_ian, t) for different levels of the dummy variable. For continuous X_ial, this allows the corresponding slope to vary smoothly and nonparametrically with both X_ianand t. For example, in our scleral strain data, one may wish to include an interaction term to allow the nonparametric age effect to vary across IOP levels. If dummy variables were specified for each IOP level, this would allow separate independent nonparametric age effects for each IOP level. If IOP is modeled continuously via a parametric model like the hyperbolic model described in Section 5, this would allow the hyperbolic coefficients to vary smoothly by age and scleral position, which would be equivalent to nonparametric age effects that vary across IOP but borrow strength from nearby IOP according to the structure induced by the hyperbolic model. In either case, the fixed effect and random spline design matrices corresponding to the X_ialf_al(X_ian, t) would be given by X_ialX_ian and X_ialZ_𝒮m (X_ian), respectively, which are straightforwardly included in the FMM. As described in Section 5, we considered these interaction structures, but found they did not appear necessary for representing the scleral strain MPS so were not included in the final model.

4. MODEL SELECTION HEURISTIC FOR SEMIPARAMETRIC BAYESFMM

Given the extensive flexibility of the semiparametric BayesFMM framework, there are a large number of modeling decisions to make. For example, in our sclera strain MPS data set, should the age effect be linear or nonparametric? If nonparametric, should the smoothing parameter in age be allowed to vary across the scleral surface t, or is a common smoothing parameter across all scleral locations sufficient? Should the fixed IOP effect be linear, hyperbolic, or nonparametric? Should there be an interaction of age and IOP? Should there be a fixed left vs. right eye effect? For the random effect levels, is the subject-specific random effect necessary to account for correlation between right and left eyes for the same subject? Is the correlation across functions from multiple IOP for the same eye sufficiently handled by a compound symmetry structure assuming equal correlations, or is a structure allowing serial correlation necessary? Should this serial correlation be based on a linear, parabolic, or hyperbolic model? These decisions are challenging to make in a simple generalized additive mixed model framework with scalar responses, and become even more challenging in the current setting with complex, high-dimensional functional responses.

There are a some papers in the frequentist literature for performing variable selection in functional regression contexts (Scheipl et al., 2013; Gertheiss, Maity and Staicu, 2013; Brockhaus et al., 2015). However, there is a lack of functional regression model selection methods for MCMC-based fully Bayesian models such as the semiparametric BayesFMM, which present special challenges. One could split the data into training and validation data sets, fit separate MCMC for each prospective model in the training data, and then compute ratios of predicted marginal densities, integrating over MCMC posterior samples, for the validation data, as done in Zhu et al. (2014), for example. These predictive Bayes Factors would provide a rigorous model selection measure, or alternatively parallel MCMC could be run for each prospective model, and a multinomial random variable with Dirichlet prior used to select and perform Bayesian model averaging across models as the chains progress. These strategies might work fine for simple, low dimensional data sets or settings with only a few prospective models, but for the current setting with complex, high-dimensional data, they are impractical.

In this section, we present a model selection heuristic that we have developed that can explore a number of potential model structures to find which seem to be most appropriate for the given data without running any MCMC, and also provides ML and REML estimates that can be used as starting values for the parameters in the BayesFMM. This heuristic is admittedly ad hoc, but is based on standard methods and appears to perform well in simulations, and so we believe can be a useful tool for modelers to assess which structures to included in their semiparametric FMM.

Our overall approach is to fit linear mixed models (LMM) to each basis coefficient k using the lme function in R (Pinheiro et al., 2017) for each prospective model, and then use a weighted voting scheme based on importance weights for each basis and an adapted Bayesian Information Criterion (aBIC) to obtain probability scores for each prospective model. Here we outline the steps in detail.

1. Basis transform and importance weights: Transform the raw functions Y_i(t), i = 1, …, N to the basis space $Y_{ik}^{\ast }$, k = 1, …, K, and compute a series of weights w_k that measure the relative importance of each basis for representing the data set. These weights can be computed by $w_k={\sum }_i{Y}_{ik}^{\ast 2}/{\sum }_i{\sum }_k{Y}_{ik}^{\ast 2}$, with Σ_k w_k = 1. For orthogonal ψ_k(t), the w_k represent the relative percent energy captured by basis coefficient k.

2. Fit basis-specific LMM and compute aBIC scores: For each prospective model ℳ_c, c = 1, …, C, use lme in R (Pinheiro et al., 2017) to fit the corresponding LMM to the data for each basis coefficient k = 1, …, K, and compute an adapted version of the BIC (aBIC_ck), which we define as:

   $${\mathit{aBIC}}_{ck}=-2log-{\text{likelihood}}_{ck}+n_{\text{par},c}\text{log}(N),$$

   where the log-likelihood is the marginal likelihood of the fixed effect and variance components of the model with the non-spline random effects integrated out conditional on the data for basis k, N is the total number of observations in the dataset for basis k, and n_par,c is the total number of parameters of model c. As discussed by Vaida and Blanchard (2005) and Spiegelhalter et al. (2002), selection of the effective number of parameters for LMM or Bayesian hierarchical models is tricky and context dependent. If inference is desired on the random effects themselves, then counting only fixed effects and variance components as parameters is not appropriate. In our setting, we are not interested in the random effects at levels capturing interfunctional correlation as we work with the marginalized model, but for nonparametric terms f(x, t) we are clearly interested in the “random effects” corresponding to the spline coefficients. Thus, we count the number of parameters to be the sum of the number of fixed effects, the number of variance components and the estimated degrees of freedom for each nonparametric term. This last term adjusts appropriately for the extra parameters of the spline fits even thought they are captured as random effects in the LMM.

3. Use weighted voting scheme to rank models: We compute a probability weight P_c for each model ℳ_c; c = 1, …, C:

   $$P_c=\sum _{k=1}^K{w}_{k}I\{c=\underset{c^{\prime }}{argmin}{\mathit{aBIC}}_{c^{\prime }k}\}$$

This procedure is applied in two steps: first assessing different fixed effect models (including parametric and/or nonparametric effects), and second assessing various random effect structures for capturing interfunctional variability while conditioning on the best fixed effect model.

In principle, I{c = argmin_c′ aBIC_c′k} indicates whether the model ℳ_c is the best in terms of aBIC for the data set on the k^th basis. Therefore, the P_c is computed via a weighted voting scheme, an aggregated measure of proportion of timesℳ_c is the best model across the all basis coefficients, with basis coefficients weighted by w_k. In this way, the model fit for basis coefficients that account for a larger proportion of the total variability in the data count more towards the overall model selection. Empirically, we have found this weighted voting scheme seems to work well, as it is robust in the sense of not allowing any one basis function, especially one explaining a relatively low proportion of total energy for the given data set, to dominate the model selection because of an extreme aBIC score. This can also be applied using alternative measures (e.g. aAIC). We acknowledge that this strategy is ad hoc and more rigorous model selection methods for settings like this are needed, but we believe it can provide useful guidance for modelers and performs well in simulations, as described below.

A word of caution

This heuristic is meant for selecting among various different modeling structures for specified covariates as done for our case study, or perhaps could be used to select among a few covariates, but it is not intended for high-dimensional variable selection across many potential predictors. In such settings, consideration of a large number of models and only fitting the “best” one can dramatically inflate type I error rates, and post-selection inference as described in Berk et al. (2013) would need to be considered.

Simulation Study on Model Selection

We conducted a simple simulation study to investigate the performance of this model selection heuristic. We considered four different models:

• Model 1 (null model): Y_ij(t) = B₀(t) + E_ij(t),

• Model 2 (linear age effect): Y_ij(t) = B₀(t) + X_age,iB_age(t) + E_ij(t),

• Model 3 (nonparametric age effect): Y_i(t) = f(X_age,i, t) + E_ij(t), and

• Model 4 (linear age effect, random effect): Y_ij(t) = B₀(t) + X_age,iB_age(t) + U_j(t) + E_ij(t).

We fit each of these models to the scleral strain data and used the fitted model as the truth, and simulated 100 replicate data sets for each model. For each simulated data set, we fit each of these four models and performed the model selection procedure using P_c to select the best model. In all four scenarios, this procedure selected the correct model 100/100 times. Average values of P_c for each model can be found in Section 4 of the supplementary materials, and Section 6 of the supplement investigates issues that can arise in variable selection of GAMMs when considering nonparametric smooth terms of subject-specific covariates in models including subject-level random effects.

5. GLAUCOMA SCLERAL STRAIN MPS CASE STUDY

5.1 Overview of Glaucoma Scleral Strain MPS Data

As described in Section 1, glaucoma is characterized by ONH damage related to IOP but its etiology is not fully known. Researchers have hypothesized that biomechanics of the scleral region close to the ONH may modulate the effect of IOP on the ONH, and thus may play an important role in glaucoma. In particular, the scleral surface is elastic so deforms under pressure, which can partially relieve IOP-induced forces on the eye, including the ONH. Thus, studies of these properties could reveal insights into the etiology of glaucoma.

Recently, novel custom instrumentation was developed that can precisely measure the mechanical strain in the posterior human sclera at a fixed level of IOP (Fazio, Bruno, Reynaud, Poggialini and Downs, 2012; Fazio, Grytz, Bruno, Girard, Gardiner, Girkin and Downs, 2012). Briefly, the posterior 1/3 of the eye is clamped, sealed, and pressurized. Next, the eye is preconditioned, and then pressurized from 7 mmHg to 45 mmHg using an automated system with computer feedback control, while scleral surface displacements are measured by a laser speckle interferometer. This device measures a light interference distribution that is used to reconstruct the surface displacement field in three dimensions with nanometer-scale precision. These displacements were processed as described in Fazio, Bruno, Reynaud, Poggialini and Downs (2012) to compute the 3D strain tensor, a 3 × 3 matrix summarizing the displacement in the meridional, circumferential, and radial directions, continuously around the outer scleral surface. The leading eigenvalue of the strain tensor, called the maximum principal strain (MPS), was computed on a grid of scleral locations for 120 circumferential locations ϕ ∈ (0°, 360°) and 120 meridional locations θ ∈ (9°, 24°), where θ = 0° corresponds to the ONH. This yields MPS functions defined on a grid of 14,400 points on the scleral surface that comprises a partial spherical domain.

Using this custom instrumentation, Fazio et al. (2014) conducted a study to investigate age-related changes in the scleral surface strain. They obtained twenty pairs of eyes from normal human donors in the Lions Eye Bank of Oregon in Portland, OR and the Alabama Eye Bank in Birmingham, AL. For each subject, the MPS measurements were obtained as described above at nine different levels of IOP (7, 10, 15, 20, 25, 30, 35, 40, and 45 mmHg) for both left and right eyes. The data for both eyes from one subject failed a quality control check, so was excluded from analysis, as did one of the eyes from four other subjects. Thus, the data we analyzed consisted of 34 eyes from 19 subjects. With 14,400 measurements for each of 9 IOP levels × 34 eyes, this data set contained over 4.5 million measurements. Let Y_ijp(t) be the MPS for eye j for subject i under IOP level p at scleral location indexed by t = (θ, ϕ), which on the sampling grid can be written as a vector y_ijp of length 14, 400. The primary goals are to study MPS, assessing how it varies around the scleral surface, across IOP, and with age. The hypothesis is that MPS is greater near the ONH, which could confer a protective effect, and that MPS tends to decrease with age, which could contribute to increased stress on the ONH thus conferring increased glaucoma risk.

5.2 Model Specification

Basis Transform

Various criteria can be considered when choosing which basis to use within the BayesFMM framework, including sparse representation, fast calculation, richness for representing the functional parameters at the various levels of the models, ability to capture the key visual features of the observed functions, and flexibility for representing the intrafunctional correlation in the data. Multiresolution bases like wavelets have advantages for many of these considerations, so we constructed a custom rectangular wavelet basis defined on the cylindrical spherical projection of the partial scleral space t = (θ, ϕ), which is a tensor transform computed by successively applying 1D wavelet transforms to the meridional and circumferential directions.

Tensor Wavelets for Scleral Space

Specifically, we constructed ψ_k(t) = ψ_k(θ, ϕ) as a tensor wavelet, ${\psi }_k(\mathbf{t})={\psi }_{k1}^{\theta }(\theta )\otimes {\psi }_{k2}^{\varphi }(\varphi )$, with meriodonal wavelet ${\psi }_{k_1}^{\theta }(\theta )$ being a db3 wavelet basis with three vanishing moments, reflection boundary condition, 5 levels of decomposition and circumferential wavelet ${\psi }_{k_2}^{\varphi }(\varphi )$ being a db3 wavelet with three vanishing moments, 5 levels of decomposition and periodic boundary conditions since its domain is circular, covering the entire circumferential space. This transform yielded a basis {ψ_k(t); k = 1, …, K = 17, 185}. While single-indexed here for simplicity of presentation, these basis coefficients can be written as multiindexed by circumferential scale j₁ = 0, …, 5, meriodonal scale j₂ = 0, …, 5, circumferential locations k₁ = 1, …, K_1j1 and meriodonal locations k₂ = 1, …, K_2j2 with K = Σ_j1Σ_j2 K_1j1 *K_2j2. The levels j₁ = 0 and j₂ = 0 correspond to the father wavelet coefficients at the lowest level of decomposition, and the other j₁ and j₂ index the corresponding mother wavelets at increasing levels of scale. With y_ijp = Y_ijp(t) being the observed function for subject i, eye j, and IOP p on the scleral surface sampling grid of size T = 14, 400 written in vector form, this basis representation can be written as ${\mathbf{y}}_{\mathit{ijp}}={\mathbf{y}}_{\mathit{ijp}}^{\ast }\mathbf{\Psi }$, where Ψ is a K × T basis matrix with elements ψ_k(t) and ${\mathbf{y}}_{\mathit{ijp}}^{\ast }$ is a vector of K corresponding basis coefficients. Because of the structure of the tensor transform, if we unstack y_ijp into a (T₁ = 120) × (T₂ = 120) matrix Y_ijp with rows indexed by equally spaced meriodonal locations θ₁ = 9°, …, θ₁₂₀ = 24° and columns by equally spaced circumferential locations ϕ₁ = 0°, …, ϕ₁₂₀ = 360°, we could write ${\mathbf{y}}_{\mathit{ijp}}^{\ast }=\text{vec}({\mathbf{\Psi }}_{\theta }{\mathbf{Y}}_{\mathit{ijp}}{\mathbf{\Psi }}_{\varphi }^{\prime })$, where vec(●) is the column-stacking vectorizing operator, Ψ_θ is the K₁ × (T₁ = 120) basis matrix corresponding to the meriodonal wavelet ${\psi }_{k_1}^{\theta }(\theta )$ and Ψ_ϕ the K₂ × (T₂ = 120) basis matrix corresponding to the circumferential wavelet ${\psi }_{k_2}^{\varphi }(\varphi )$. In principle, spherical wavelets could be used as the transforming basis, but currently available software does not handle transforms for part of the sphere, and since we only model θ over a limited range of 9° – 24°, the distortion from using the basis on the projection and not the true spherical geodesic is not great.

Some eyes had technical processing artifacts that resulted in a spike of extremely high MPS at some local set of scleral locations, typically close to the boundary. Given the multiresolution nature of the wavelet transform, these artifacts were captured by wavelet coefficients at extremely high frequency scales. Given the relatively smooth nature of most of the MPS functions, these wavelet coefficients were essentially zero for all eyes except for those with the artifact which yielded very large coefficients. Thus, we removed these artifacts by filtering out any wavelet coefficients with extremely skewed distributions for which the mean across all samples was more than 100× the median. As seen in the supplementary file RawMPScurves.zip, with illustration in Supplemental Figure 26, this strategy effectively removed the outlying spikes without substantively affecting MPS values for other scleral locations. We also applied the joint wavelet compression strategy described in Morris et al. (2011) to obtain a reduced dimension near-lossless basis function to use, and found a subset of 269 wavelet coefficients that jointly preserved > 99.5% of the total signal energy for each eye, and an average of > 99.9%, leading to > 50: 1 compression. As shown in Supplemental Figure 26 and RawMPScurves.zip, the data projected into the basis is essentially identical to the raw data, demonstrating its near-lossless nature. We considered this basis for our model.

We modeled these basis coefficients using the basis transform modeling approach described in Section 3.1. Besides providing a relatively sparse representation and enabling the adaptive removal of spiky artifacts, this transform being a location-scale decomposition allowed nonstationary intra-scleral correlations and adaptive borrowing of strength across scleral locations. File intrafunctional_correlation.mp4 in the supplement contains a movie file demonstrating the form of the intrafunctional correlation structure induced by this choice, computed by constructing the basis and basis transform matrices Ψ and Ψ⁻, respectively, and applying (5) to the basis space covariances at the various random effect and residual error levels of the model. For illustration, panels (e) and (f) of Figure 3 contains plots of this surface at a particular scleral location for two of the random levels.

We also considered using principal components computed on the wavelet-transformed (and compressed) data, similar to the strategy used in Meyer et al. (2015), which implies applying a singular value decomposition to the wavelet-space data matrix, and then using the resulting eigenvectors to construct the empirical basis functions ψ_k(t) that are used for the BayesFMM modeling. In this case, we kept K = 29 basis functions that explained > 99.5% of the total variability in the data set according to the scree plot, which as estimated by four-fold cross validation retained a minimum of 96.7% of the total energy for each eye, so is somewhat near-lossless. We used the wavelets for our primary analysis given that it yielded a richer basis set for representing the various functional parameters at various levels of the models, but for sensitivity we also presented results using the BayesFMM using these wavelet-regularized principal components in Section 7 of the supplement, as well as other summaries including the induced intrafunctional correlation structures.

Model Selection

We applied the model selection heuristic described in Section 4 to help select the structures in the semiparametric FMM to include in the FMM. We first determined which fixed effect covariates to include and what their functional forms should be. Here we summarize the results, for which more details are provided in Section 3 of the supplementary materials. Three different fixed effects were considered: age, IOP, and eye (left vs. right). For the form of the age effect, we considered two possibilities: linear or nonparametric. For the form of the IOP effect, we considered three different possibilities: linear, hyperbola, or nonparametric. Models without the eye effect were also compared. As a result, we compared 12 different models for the fixed effect selection. It turned out that the model with the nonparametric age effect, the hyperbolic IOP effect, and no eye effect showed the highest P_c when using aBIC, and the model with nonparametric age effect, hyperbolic IOP effect, and a left vs. right eye effect had the highest P_c when using aAIC. For our primary analysis, we considered the model with no left vs. right eye effect, since there is no strong scientific rationale for such an effect, and present the other model as a sensitivity analysis in Section 7 of the supplementary materials. We also assessed whether the smoothing parameter for the nonparametric age effect should be constant or vary around the sclera, and found that the sclerally varying smoothing parameter was clearly necessary for good fit, as detailed in Section 5 of the supplement. Once we selected the main fixed effects, we assessed whether the interaction term between age and IOP was needed, and our model selection heuristic suggested the interaction was not necessary. Finally, with the selected fixed terms, we compared several different random effect distributions to capture the interfunctional covariance structure. Two different levels of random effects were considered: the subject-level random effect and the serial eye-level random effect. For the form of the eye-level random effect in terms of IOP as illustrated in Section 3.2, we considered three different forms: constant (compound symmetry), linear, or hyperbola. Our model selection heuristic selected the eye-level random effect with the hyperbolic IOP effect, but not the subject-level random effect.

Model

Thus, the final fitted semiparametric FMM was:

$$Y_{\mathit{ijp}}(\mathbf{t})=B_0(\mathbf{t})+B_1(\mathbf{t})G_1(p)+B_2(\mathbf{t})G_2(p)+f(X_{\text{age},i},\mathbf{t})+U_{ij}(\mathbf{t})+U_{ij1}(\mathbf{t})G_1(p)+U_{ij2}(\mathbf{t})G_2(p)+E_{\mathit{ijp}}(\mathbf{t}),$$ (23)

with X_age,i is the age for subject i, and G₁(p) and G₂(p) are the values of the orthogonalized hyperbolic basis corresponding to IOP = p as described below. This is equivalent to the FMM:

$$Y_{\mathit{ijp}}(\mathbf{t})=B_0(\mathbf{t})+B_1(\mathbf{t})G_1(p)+B_2(\mathbf{t})G_2(p)+B_3(\mathbf{t})X_{\text{age},i}+\sum _{m=1}^{M+2}{Z}_{\mathcal{B}m}(X_{\text{age},i})U_{\mathcal{S}m}(\mathbf{t})+U_{ij}(\mathbf{t})+U_{ij1}(\mathbf{t})G_1(p)+U_{ij2}(\mathbf{t})G_2(p)+E_{\mathit{ijp}}(\mathbf{t}),$$ (24)

with Z_ℬ,m(x) the Demmler-Reinsch basis functions corresponding toM interior knots on x, U_𝒮m(t) ~ GP{0,Q_𝒮}, U_ij(t) ~ GP{0,Q₀}, U_ij1(t) ~ GP{0,Q₁}, U_ij2(t) ~ GP{0,Q₂}, and E_i(t) ~ GP(0, S), and with Q_𝒮,Q₀,Q₁,Q₂, and S being covariance surfaces defined on 𝒯 ×𝒯. Following the guidelines suggested by Ruppert et al. (2003), we chose M = 5 equally spaced knots over X_age.

Parameterization of IOP effect

From a preliminary investigation in which we fit separate models to each scleral location t, we found that the serial IOP effects were well modeled by a hyperbola of special form Y = b₀ + b₁p + b₂p⁻¹, with an average R² of 0.98 across all eyes and scleral locations, and this form was also chosen by the model selection heuristic. To accommodate model fitting without having to include a covariance between b₁ and b₂, we utilized orthogonalized versions of these predictors: $G_1(p)=\sqrt{2}/2X_{1,p}-\sqrt{2}/2X_{2,p}$ and $G_2(p)=\sqrt{2}/2X_{1,p}+\sqrt{2}/2X_{2,p}$ where X_1,p is a standardized version of p and X_2,p is a standardized version of p⁻¹.

Basis Space Model

We used the fast rectangular 2D wavelet transform to compute the basis coefficients from the raw functions, equivalent to the matrix multiplication ${\mathbf{y}}_{\mathit{ijp}}^{\ast }={\mathbf{y}}_{\mathit{ijp}}{\mathbf{\Psi }}^{-}$ with Ψ⁻ = Ψ′(ΨΨ′)⁻¹ with ${\mathbf{y}}_{\mathit{ijp}}^{\ast }$ a vector of length K = 269 with elements $Y_{\mathit{ijpk}}^{\ast }$, k = 1, …, K. We then fit the basis-space version of model (24):

$$\begin{gathered} Y_{\mathit{ijpk}}^{\ast }=B_{0k}^{\ast }+B_{1k}^{\ast }{G}_1(p)+B_{2k}^{\ast }{G}_2(p)+B_{3k}^{\ast }{X}_{\text{age},i}+\sum _{m=1}^{M+2}{Z}_{\mathcal{B}m}(X_{\text{age},i})U_{\mathcal{S}mk}^{\ast }+U_{\mathit{ijk}}^{\ast }+U_{ij1k}^{\ast }{G}_1(p)+U_{ij2k}^{\ast }{G}_2(p)+E_{\mathit{ijpk}}^{\ast },\text{with} \\ {U}_{\mathcal{S}mk}^{\ast }~N(0,q_{\mathcal{S}k}),U_{\mathit{ijk}}^{\ast }~N(0,q_{0k}),U_{ij1k}^{\ast }~N(0,q_{1k}),U_{ij2k}^{\ast }~N(0,q_{2k}),\text{and}{E}_{\mathit{ijpk}}^{\ast }~N(0,s_k). \end{gathered}$$ (25)

Prior Specification

We specified vague conjugate inverse Gamma priors for each basis space variance component {q_𝒮k, q_0k, q_1k, q_2k, s_k} in the model, with prior mode being the REML starting values and with effective sample size of 2, e.g. s_k ~ InverseGamma(a_s, b_s) with a_s = 2 and b_s = 3 * ŝ_k where ŝ_k is the REML starting values for s_k. We used spike-slab priors for the basis-space fixed effects { $B_{ak}^{\ast }$, a = 0, …, 3}, with regularization parameters {π_aj, τ_aj} varying over predictor a = 0, …, 3, with regularization sets j = 1, …, J = 36 determined by the tensor wavelet scale levels, with j = 0 for j₁ = j₂ = 0, j = 1 for j₁ = 0, j₂ = 1, …, j = J = 36 for j₁ = j₂ = 5. We estimated the regularization parameters using the empirical Bayes algorithm specified in Morris and Carroll (2006). To assess sensitivity of results to these choices of regularization parameters, we also ran the model doing no additional regularization (beyond wavelet compression) by setting π_aj ≡ 1 and τ_aj ≡ 10⁶. Results are provided in Section 7 of the supplementary materials.

Model Fitting

We ran an MCMC to obtain posterior samples of the parameters of model (25) { $B_{●k}^{\ast }$, q_●k, s_k} from the marginalized version of this model with $U_{●k}^{\ast }$ all integrated out. We fit a total of 10,000 posterior samples after a burn-in of 5000, thinning by keeping every 10. We then sampled the spline random effects $U_{\mathcal{S}mk}^{\ast }$ from their complete conditional distributions with the other random effects still integrated out, which are conjugate multivariate normal Gibbs steps as detailed in Section 1 of the supplementary materials, from which posterior samples of $f_k^{\ast }(x)=x{B}_{3k}^{\ast }+{\sum }_m{Z}_{\mathcal{B}m}(x)U_{\mathcal{S}mk}^{\ast }$ were subsequently constructed for a grid of ages x of size 71 corresponding to ages 20–90. Let ${\mathbf{F}}_g^{\ast }$ be a 71 × (K = 269) matrix representing posterior sample g of the basis space nonparametric age effect, g = 1, …, 1000. We then transformed this back to the data space via ${\mathbf{F}}_g={\mathbf{F}}_g^{\ast }\mathrm{\Psi }$ to obtain the 71 × (T = 14, 400) matrix of posterior samples of the nonparametric age effect f(X_age, t) in model (23), and similarly transforming the other fixed effects back to the data space to get posterior samples of {B₀(t),B₁(t),B₂(t)} on the sampling grid of t and used for posterior inference.

On a laptop computer, the entire analysis took 7hr39min on a single core, with the basis transform taking 1m37s, model selection 22m48s, each MCMC iteration 0.77s with 15,000 iterations taking 3hr15m, and the postprocessing including inverse basis transform of posterior samples and key inferential summary calculations 4hr. Many other summaries and plots were computed for the purposes of this paper for sensitivity and illustration of the deep properties of the modeling framework at more computational expense, but these additional analyses are not necessary for analysis of the data. The Metropolis-Hastings acceptance probabilities (≈ 0.85 – 0.95) were reasonable, and Geweke convergence statistics (median 0.01, Q.₀₂₅ = −1.98,Q_0.975 = 1.94) across the many parameters in the model showed that most were within roughly 2 standard deviations of zero, and only 4.4% of the corresponding p-values were less than 0.05, so suggest reasonable MCMC convergence (see Section 9 of the supplementary materials for more details). We simulated virtual MPS functions for hypothetical subjects with specified age and IOP from the posterior predictive distribution of the data (see Section 10 of the supplementary materials), and found that the simulated MPS data are visually similar to the MPS data from real eyes, suggesting the BayesFMM model with the tensor wavelet bases was sufficiently flexible to capture the salient features of these data, and lending support to its use for inference. We share these pseudo data on github (https://github.com/MorrisStatLab/SemiparametricFMM), as well as the real data and scripts to perform all analyses.

5.3 Scientific Results

Nonparametric MPS function of age

First, we computed the posterior mean MPS curve as a function of age over the entire meridional and circumferential domain at each level of IOP. As a thorough summary of the model fit, we generated a plot of these fits for each scleral location, and joined these together to make a digital movie file MPSvAGE-wave.mp4 contained in the supplemental materials. Figure 2 shows a snapshot of the movie at the scleral location indicated by the white dot. Panel (a) depicts a polar azimuthal projection of the fitted MPS function for a left eye from a subject of age 90yr under 45 mmHg of IOP. Note that MPS is higher near the ONH, as expected as a protective effect. Panel (b) shows the posterior mean degrees of freedom (DF) of the nonparametric age effect. We see strong nonlinear age effects in scleral regions close to the ONH and towards the inferior and nasal regions of the sclera, while many other regions show linear or almost linear age effects. The right nine panels contain the fitted nonparametric age effect at nine different IOP levels at the scleral position indicated by the white dot. In each panel, the black dots are the raw data, the solid blue line is the estimated nonparametric mean MPS function of age, and the solid and dashed red lines correspond to joint and point-wise 95% credible bands of the mean MPS curve, with joint bands computed as described in Meyer et al. (2015). We see from this plot how the MPS increases with IOP, and that the hyperbolic model seems to capture the rate of increase very well. From this plot and the movie in the supplement showing results stepping across the scleral locations, we see this model fits the data for all IOP and scleral locations remarkably well in spite of the fact that independent splines were not fit to the data for each IOP and scleral location separately, but rather is the result of the complex joint unified model (23) that borrows strength from other IOP according to the modeled hyperbolic serial effect and from other scleral locations according to the basis functions. The model also borrows strength from other nearby scleral locations according to the basis functions in estimating the nonlinearity of the age effect, as seen in the local smoothness of the DF(t) plot.

Figure 2.

(a) Polar azimuthal projection of fitted MPS function for left eye from one subject of age 90yr under 45 mmHG of IOP. (b) posterior mean of degrees of freedom of nonparametric MPS fit of age. The right nine panels depict estimated nonparametric MPS fit of age for all nine IOP levels at the scleral location indicated by the white dot in (a) and (b), along with the raw data indicated by the black dots.

Induced Intrafunctional and Interfunctional Covariance Structures

To assess the intrafunctional covariance structures induced by our tensor wavelet bases, we used equation (5) to estimate the scleral space intrafunctional covariance matrices Q_d(t₁, t₂), d = 0, 1,2 and S(t₁, t₂). Supplementary Figure 22 plots the diagonals of these matrices representing the variances at the various hierarchical levels as a function of scleral location t. Note that the variance for the eye intercept Q₀(t, t) is an order of magnitude greater than that of the eye-level IOP coefficients Q₁(t, t) and Q₂(t, t), which are in turn an order of magnitude greater than the residual error variance S(t, t). Also, note that these covariances vary around the scleral surface, with locations near the ONH having greater levels of variability. The supplement also contains a movie file interfunctional_cor.mp4 that represents the corresponding intrafunctional correlation surfaces induced by our model by stepping around scleral locations and for each plotting a heatmap representing the correlation of the indicated location with all other scleral locations. For illustration, panels (e) and (f) of Figure 3 show the correlation of a specific scleral location t^* (indicated by the white dot) with all other scleral locations t at the eye intercept Q₀(t^*, t) and residual error S(t^*, t) levels, and Supplementary Figure 18 contains these plus those for the eye hyperbolic random effects Q₁(t^*, t) and Q₂(t^*, t) for this location, and Intrafunctional_correlations.mp4 contains a movie file showing all scleral locations. Note how our model captures local intrafunctional correlation, and the strength and tails of this correlation are allowed to vary by scleral location and hierarchical level. Supplementary Figure 19 includes equivalent plots for the wavelet-regularized principal component basis functions, and Intrafunctional_correlations-pc.mp4 contains a movie showing all scleral locations. Note that the PC basis functions, although global, induce intrafunctional correlation surfaces that are dominated by the local correlation among nearby scleral locations that is also captured by the tensor wavelet bases.

We also computed the induced interfunctional serial correlation across MPS curves for different IOP for the same eye using the formulas contained in Section 3.2. The supplement contains a movie file Intra_IOP_corr.mp4 that plots the variance, var{Y_ijp(t)| IOP = p, t}, as a function of IOP and scleral location and the serial correlation across IOP, corr{Y_ijp(t), Y_ijp′(t)}, as a function of IOP as it varies around the scleral surface t. Note the form of the serial correlation induced by the hyperbolic model, and how it is able to vary yet borrow strength across scleral locations. Panel (d) of Figure 3 portrays this serial correlation at a single scleral location.

Inference on functionals of the parameters

While Figure 2 and the accompanying movie file provide a thorough summary of the age effect on MPS estimated by our model, for interpretability it may be useful to aggregate results over IOP and/or scleral locations. One major advantage of our fully Bayesian approach is that we are able to compute posterior samples and obtain estimates and inference for any functional of the model parameters. First, to aggregate information across all IOP, we considered the area under the MPS vs. IOP curve, defined as follows:

$$\text{AUC}(X_{\text{age}},\mathbf{t})=f(X_{\text{age}},\mathbf{t})+{\int }_7^{45}\{B_1(\mathbf{t})G_1(p)+B_2(\mathbf{t})G_2(p)\}dp.$$

This integral summarizes the total MPS behavior over the range of IOP in the study, which covers the practical range of IOP values in this context. The integral is estimated numerically for each MCMC sample to yield posterior inference on AUC. We plotted the posterior mean AUC vs. age curve and corresponding posterior pointwise and joint credible bands for each scleral location, and assembled into a digital movie file AUCvAGE-wave.mp4 in the supplementary materials, and for illustration panel (b) of Figure 3 contains this plot for a single scleral location. Although our model was not fit to the AUC data, but rather the raw data for each IOP, note how the model provides a nonparametric smooth fit of AUC vs. age for each scleral location, and again these fits borrow strength across scleral locations. From these results, we see that that aggregating over IOP, the MPS tends to decrease with age at most scleral locations, especially near the ONH. With MPS decreasing with age, the eye seemingly becomes less elastic and less able to absorb IOP, potentially exposing the ONH to IOP-induced damage over time.

Given the nonlinearity of the age fit, it may be instructive to directly look at the rate of decline of MPS over age. We can do this by computing inference on the derivative of the AUC curve with respect to X_age. Given the lack of an IOP × age interaction in our final fitted model, it follows that ∂AUC(X_age, t)/∂X_age = ∂f(X_age, t)/∂X_age. Using the fact that f(X_age, t) = β₀(t) + X_ageβ₃(t) + Z_ℬ(X_age)u_𝒮(t) as described in Section 3.3, with $Z_{\mathcal{B}}(X_{\text{age}})={\{{\mathcal{B}}_1(X_{\text{age}}),\dots ,{\mathcal{B}}_{M+4}(X_{\text{age}})\}}^{\prime }{Z}_{\mathrm{\Omega }}\text{diag}(d_1^{-1/2},\dots ,d_{M+2}^{-1/2})$, it can be easily seen that

$$\frac{\partial \text{AUC}(X_{\text{age}},\mathbf{t})}{\partial X_{\text{age}}}={\beta }_3(\mathbf{t})+\frac{\partial Z_{\mathcal{B}}(X_{\text{age}})}{\partial X_{\text{age}}}{\mathbf{u}}_{\mathcal{S}}(\mathbf{t}).$$ (26)

Our R scripts contain calculations of $\frac{\partial Z_{\mathcal{B}}(X_{\text{age}})}{\partial X_{\text{age}}}$, which come from Wand and Ormerod (2008). By applying (26) to the posterior samples of β₃(t) and u_𝒮(t), we could obtain posterior samples of this derviative for each scleral location, although we do not present those results here. Since the nonparametric fits were done placing a smoothness penalty on the age effects themselves, the derivatives may appear slightly undersmoothed. If one had primary interest in estimating these derivatives smoothly, they could do so by simply choosing higher order penalties in the spline fits. A third order penalty would produce smoothness in the first derivative.

Aggregated summaries over functional regions

Although our model fits the entire data set over all scleral locations, for ease of interpretation researchers at times would like to look at estimates and inference for aggregated scleral regions. Given the hypothesis that scleral strain is most important near the ONH, we averaged results over all circumferential regions to obtain results as a continuous function of meridional distance from the optic nerve head. Figure 4 depicts the posterior mean AUC as a function of age and distance from the ONH, aggregating over circumferential regions. We can see how MPS is higher near the ONH and decreases moving away from the ONH, potentially providing a protective effect to the ONH. Younger individuals have high MPS levels at scleral locations extending well out from the ONH, while for middle age individuals the regions of high MPS does not extend out far from the ONH, and for older individuals the MPS is quite low even close to the ONH. This coincides with the increased glaucoma risk in older individuals.

Figure 4. Posterior mean AUC (of MPS) as a function of age and distance from ONH.

These results are obtained by aggregating posterior samples over all circumferential regions.

We also summarized the results aggregating over the innermost region closest to the optic nerve head, which is called peripapillary(PP) region and also within the adjacent region, called the mid-peripheral( MP) region. Here we present aggregated AUC results in both PP and MP regions. Figure 5 depicts the age effects aggregating AUC over the PP and MP regions, with the top two plots containing the posterior mean fits and the bottom containing the derivatives with respect to age. The blue line contains the posterior mean fit. The dotted and solid red lines indicating 95% pointwise and joint credible intervals, respectively. From this we can clearly see that the MPS is systematically higher in the PP region closest to the ONH than in the MP region further away, potentially conferring a protective effect. The MPS decreases with age in both regions, but the decrease is substantially steeper for the all-important PP region close to the ONH. This effect is nonlinear, with the rate of decrease accelerating throughout middle age (40–60 years old), an age at which glaucoma risk increases substantially.

Figure 5. Aggregated AUC summaries.

AUC aggregated over peripapillary (PP) region adjacent to the ONH and the mid-peripheral(MP) region just beyond PP. (a) and (b) show AUC summaries over the two regions. (c) and (d) show derivatives of aggregated AUC summaries presented in (a) and (b).

These analyses confirm the hypothesis that MPS is higher in scleral regions closer to the ONH, decreases with age, and this decrease with age is more pronounced near the ONH. This agrees with the notion that biomechanical changes in the sclera may contribute to increased glaucoma risk.

Sensitivity Analyses

Section 7 of the Supplement and Supplementary Figures 2–25 present extensive results for the alternative model with the left vs. eye effect, with the alternative values for the prior shrinkage hyperparameters {τ_aj, π_aj}, and using the wavelet-regularized principal components as the projected basis. Substantive results do not change, so we see our conclusions are not driven by these choices.

6. DISCUSSION

In this paper, we demonstrated how to adapt the BayesFMM modeling framework to account for serial interfunctional correlation and smooth nonparametric covariate functional effects and applied it to an innovative glaucoma study investigating MPS of scleral strain tensors. We found that MPS is maximized near the ONH. We also found that MPS decreases with age, especially in regions closest to the ONH, and the decrease in MPS accelerates throughout middle-age. This could contribute to the increased glaucoma risk seen in elderly. The age effect on MPS tends to be non-linear near the ONH, especially towards the inferior and nasal sides, while the other scleral regions show the age effect close to linear. Focal glaucoma damage is most often observed in the inferior quadrant of the ONH.

While motivated by the glaucoma data application, the BayesFMM framework presented here is extremely general, with the ability to be used with any near-lossless basis tranform and applicable to many types of complex, high dimensional functional data of modern interest including wearable computing data, genome-wide data, proteomics data, geospatial time series data, neuroimaging data, and many others. Using this framework, one can not only accommodate nonparametric functional fixed effects, but also model serially correlated functions through functional growth curve effects. Our model allows the nonparametric fits, smoothness, and interfunctional correlations to potentially vary over the functional domain, which is necessary for good fit to these data and likely many other complex functional data sets. In addition, we introduced a model selection heuristic that can be used to select among fixed and random effects and decide whether they should be linear, parametric, or nonparametric, and whether the fits or smoothness should be constant or vary around the functional domain before running the MCMC.

On our github page (https://github.com/MorrisStatLab/SemiparametricFMM), we share the Matlab files required to fit these models, including scripts to apply the model selection heuristic, links to automated software to perform the MCMC, and scripts to compute posterior samples and posterior inference for the nonparametric functional effects and all summary plots included in this paper. The model is set up using lmer style model statements (Bates et al., 2015). The software can be used to fit models with any fixed or random effect function covariates, with different basis functions, and for functions on domains of any dimension or measure given suitable basis functions for the corresponding space. The method is efficient enough to feasibly apply to very large data sets like our glaucoma data set here with over 4.5 million observations. While it took a relatively long time to run the MCMC using a single core, we do not think this is inordinate considering the cost and time to collect these data and the extensive inferential summaries provided by the model fits. This run time was orders of magnitude less than a conceptually simpler approach of applying lme to each scleral location, applying a 2d smooth, and using a bootstrap for inference, which by our calculations would take 10 weeks to compute using 1000 bootstrap samples on the same computer we used for the BayesFMM model. If further speed is desired for our method, the model fitting is highly parallelizable and the MCMC code has cluster computing capabilities so can be sped up using GPU or cluster computing resources. While to code we share only utilizes a single core, future updates of the software will enable distributed computing for faster calculations for big data sets. We are in the process of extending the package to fit these models and including many other features of the BayesFMM framework in other publications but not included in this paper. We anticipate this package, which will be available in Matlab and R, will greatly enhance the usability of the method, and expect this package to be completed and freely available in the near future.

One significant benefit of our fully Bayesian approach to fitting these semiparametric functional mixed models is that we can produce inference on any parameters in the model, or any functional or aggregation of these parameters, and this inference integrates over the various sources of interfunctional variability in the model and over the uncertainty in estimating the covariance parameters and level of nonlinearity of the spline fits. This allows us to perform a flexible, thorough analysis in the entire functional domain, yet produce inference and results for aggregated summaries that may be more interpretable to investigators. This raises the obvious question, “What is the advantage of fitting the entire functional model? Why not just compute the aggregated summaries and model those using standard tools?” The answer to this question is multi-faceted.

First, if one only looked at specific aggregated summaries, they may miss insights that could have been gleaned from their data but were not captured by these summaries. For example, if only looking at the PP and MP regions, one may miss out on different MPS behavior in the inferior and nasal regions of the sclera, which may be important. By modeling the scleral function in its entirety, we are able to examine the entire domain to ensure we are not missing out on any insights, and then we can still produce inference for any desired aggregated summaries for ease of interpretation, as well. This concept is also relevant in other application areas where summaries are used because of the complexity and high dimensionality of the raw functional data. This approach of flexible modeling followed by inferenct on summary measures can capture the best of both worlds, analyzing the entire function yet producing inference for summary measures interpretable for the scientific subject area.

Second, any aggregation draws arbitrary boundaries in the function space. MPS for scleral locations at the boundary of the PP and MP regions are highly correlated with each other, and yet PP and MP extracted summaries arbitrarily separate them from each other. The flexible functional modeling framework introduced in this paper models the entire functional space, yet captures and accounts for intrafunctional correlation through the chosen basis functions. This allows a smooth borrowing of strength from nearby locations, and if location-scale bases like wavelets are used, then this borrowing of strength can be adaptive, able to accommodate spatially heterogenous functions for which some functional regions are more correlated than others. In principle, accounting for this intrafunctional correlation leads to greater efficiency, as has been shown in various contexts.

While flexible, the BayesFMM framework presented here has some limitations and drawbacks, including the need to choose a common basis to use for transformation at all levels of the model, the independence in the basis space assumption that can limit certain types of intrafunctional covariance structure depending on the choice of basis, and the computational intensity from having to run a full MCMC to obtain estimates and inference for model quantities. It is designed for representing functional data sampled on a common fine grid, so is not suitable for sparsely sampled functional data or functional data for which the individual functions are sampled on wildly different sampling grids, a setting in which the Scheipl et al. (2015) and Greven and Scheipl (2017a) framework is well-suited. The model selection heuristic is ad hoc, and should not be used to select over a large number of variables.

In spite of these limitations and drawbacks, the BayesFMM modeling framework is very general, and this paper can serve as a template for how to utilize this modeling framework to model complex functions with various types of interfunctional correlation structures. It has the potential to impact many areas of science yielding complex functional data, and the analysis presented here illustrates a rigorous, thorough workflow to analyze the entire data set, extract many types of information from them, yet provide interpretable graphical summaries and inferential results desired by investigators.