Bayesian longitudinal low-rank regression models for imaging genetic data from longitudinal studies

Abstract

To perform a joint analysis of multivariate neuroimaging phenotypes and candidate genetic markers obtained from longitudinal studies, we develop a Bayesian longitudinal low-rank regression (L2R2) model. The L2R2 model integrates three key methodologies: a low-rank matrix for approximating the high-dimensional regression coefficient matrices corresponding to the genetic main effects and their interactions with time, penalized splines for characterizing the overall time effect, and a sparse factor analysis model coupled with random effects for capturing within-subject spatio-temporal correlations of longitudinal phenotypes. Posterior computation proceeds via an efficient Markov chainMonte Carlo algorithm. Simulations show that the L2R2 model outperforms several other competing methods. We apply the L2R2 model to investigate the effect of single nucleotide polymorphisms (SNPs) on the top 10 and top 40 previously reported Alzheimer disease-associated genes. We also identify associations between the interactions of these SNPs with patient age and the tissue volumes of 93 regions of interest from patients’ brain images obtained from the Alzheimer’s Disease Neuroimaging Initiative.

1. Introduction

Many longitudinal neuroimaging studies concomitantly collect genetic and recurrent imaging data to track individual changes in brain structure and function over time. Several neurodegenerative disorders, including Alzheimer disease (AD), are hypothesized to occur from abnormal development of the brain, which may be caused by the additive and/or interactive effects of various risk genes and environmental risk factors, each contributing small individual effects. Thus, recurrent neuroimaging measures may lead to discoveries of the genetic pathways and the causal genes associated with the specific brain changes underlying such neurodegenerative disorders (Scharinger et al., 2010; Paus, 2010; Peper et al., 2007; Chiang et al., 2011a,b; Saykin et al., 2015).

A standard statistical method used in longitudinal imaging and genetics studies is the massive marginal association (MMA) framework (Li et al., 2013; Zhang et al., 2014; Guillaume et al., 2014; Hibar and et al, 2011; Shen et al., 2010; Bernal-Rusiel et al., 2013; Zhang et al., 2014). This approach repeatedly fits a linear mixed effects model (or generalized estimating equations) for paired imaging phenotypes and genetic markers. Because the MMA framework entails numerous comparisons, it can detect only phenotype-marker pairs with extremely strong signals.

Several attempts have been made to more precisely investigate the effect of multiple genotypes on longitudinal phenotypes. Chen and Wang (2011) proposed functional mixed-effect models with penalized splines and varying coefficients, but they focused on small number of predictors and number of response variables in a low-dimensional setting. Wang et al. (2011) used a sparse multitask regression to examine the association between genetic markers and longitudinal neuroimaging phenotypes. However, their model focused on subjects with the same number of repeated measures and ignored the spatiotemporal correlations of imaging phenotypes. Therefore, the multitask regression model may lead to loss of statistical power to detect phenotype-marker pairs with moderate to weak signals. Vounou et al. (2011) and Silver et al. (2012) proposed that a sparse reduced-rank regression model using penalized regression can detect the main genetic effects on longitudinal phenotypes. They, however, did not account for the spatiotemporal association among the longitudinal phenotypes, which is important for estimation and prediction accuracy. Moreover, none of these studies explored SNP-age interactions, which can reveal dynamic genetic effects on phenotypes.

Several important statistical concerns are associated with the joint analysis of neuroimaging phenotypes and a set of candidate genotypes obtained from longitudinal imaging and genetic studies. First, the number of regression coefficients can be much larger than the sample size, denoted as N. Specifically, let d and p be the dimension of the responses and the number of covariates, respectively. Fitting a multivariate linear mixed effects model usually requires estimating a d × p matrix of regression coefficients, which can be much larger than N, even for moderately high d and p. Second, as illustrated in Fig. 1a, to improve prediction accuracy (Breiman and Friedman, 1997), it is critically important to account for unstructured, within-subject spatial correlations among multivariate neuroimaging phenotypes. Third, as illustrated in Fig. 2a, to improve both estimation and prediction accuracy, it is also important to account for within-subject temporal correlation. Fourth, as shown in Fig. 2b,c, the temporal growth pattern varies across regions of interest (ROIs) in the brain. Accounting for the overall longitudinal change of ROIs is required to increase the detection power of the genetic effects. Fifth, as shown in Fig. 2b, the genetic effects on ROI volumes can vary across time.

Fig. 1.

The characteristics of the ROIs and SNPs from the ADNI data set and their association.

Fig. 2.

The longitudinal characteristics of the ROI volumes in the ADNI data set. (a) The trajectories of the volumes of left lateral ventricle for all subjects. (b) and (c) The LOESS estimates of two volume trajectories based on the subjects with different SNP alleles.

Here, we have developed a Bayesian longitudinal low-rank regression (L2R2) model for the joint analysis of high-dimensional longitudinal responses and covariates. We integrated multiple robust methods to explicitly address the new challenges described previously. Our study has four major methodological contributions that were previously undescribed:

1. To the best of our knowledge, L2R2 is the first model of its kind for jointly analyzing high-dimensional longitudinal responses and covariates, although several approaches have been used for high-dimensional responses and covariates in cross-sectional studies (Rothman et al., 2010; Vounou et al., 2010; Zhu et al., 2014). The L2R2 model also provides a set of standard inference tools (e.g. standard deviation) for determining various unknown parameters. Zipunnikov et al. (2014) proposed a functional principle components analysis for high dimensional (> 10, 000) longitudinal responses, where the intercepts and slopes of time for all voxels were modeled by a few basis functions. However, their methods cannot handle high-dimensional responses and covariates simultaneously because the dimension of the covariance matrix that requires time-consuming spectral decomposition equals the product of the dimensions of the responses and covariates.

2. The temporal growth patterns of high dimensional responses were characterized by a penalized spline method. Although the spline method has been widely used in other studies (Eilers and Marx, 1996; Ruppert et al., 2003; Wood, 2006; Wu and Zhang, 2006; Lang and Brezger, 2004; Guo, 2002; Morris and Carroll, 2006; Greven et al., 2010), most of these studies focused on univariate responses. We included a new set of coefficients for the high-dimensional gene-age interactions to model the genetic effects on longitudinal trajectories of responses, which were different from previously reported coefficients corresponding to the main genetic effects that were time-invariant (Marttinen et al., 2014; Zhu et al., 2014). A low-rank regression model not only reduces the number of unknown coefficients, but also captures the low-rank structure of the regression coefficient matrix.

3. To efficiently model the correlation structure and increase the detection power of association between response and covariates, we used a sparse factor model (Bhattacharya and Dunson, 2011) to capture the spatial correlation and the structured random effects to model the temporal correlation of the longitudinal response variables.

4. We have developed a downloadable L2R2 package, which is available at http://odin.mdacc.tmc.edu/bigs2/.

2. Methods: L2R2 model formulation

Let y_ik(t_ij) be the j–th observation at time t_ij of the kth neuroimaging measurement (e.g. volume of ROI) for the ith subject for i = 1, … , n, j = 1, … , m_i and k = 1, … , d, x_i = (x_i1, · · · , x_ip)^T be a p × 1 vector of genetic predictors (e.g. p/2 genetic markers and p/2 gene-age interactions), and $N={\sum }_{i=1}^n{m}_i$ be the total number of observations. To address the challenges of concomitantly analyzing high-dimensional longitudinal responses and covariates, we proposed the following L2R2 model: For each k, we have

$$y_{ik}(t)={{\mathbf{x}}_i}^{\mathrm{T}}{\mathit{\beta }}_k+{\mu }_k(t)+{\mathbf{w}}_{2,i}{(t)}^{\mathrm{T}}{\mathit{\gamma }}_{2,k}+{\mathbf{z}}_i{(t)}^{\mathrm{T}}{\mathbf{b}}_{ik}+{\epsilon }_{ik}(t),$$ (1)

where β_k is a p×1 vector of coefficients for the genetic predictors, μ_k(t) is a nonparametric function of t that characterizes the overall trajectories of the kth measurement across all subjects, w_2,i(t_ij) = (w_2,i1(t_ij), · · · , w_2,iq2(t_ij))^T is a q₂ ×1 vector of time-variant or time-invariant prognostic factors (e.g. age, gender), and γ_2,k is a q₂ × 1 vector of coefficients for the prognostic factors. Moreover, b_ik is a p_b×1 vector of latent random effects that characterize the within-subject correlation of the ith subject for the kth measurement, and z_i(t) is the related p_b × 1 vector of covariates characterizing the correlation structure, and ε_ik(t) is an error term. Let b_i = (b_i1, … , b_id) such that

$$\mathit{vec}({\mathbf{b}}_i)~N(\mathbf{0},{\mathbf{\sum }}_b)\text{and}{\mathit{\epsilon }}_i(t)~N_d(\mathbf{0},{\mathbf{\sum }}_e={\mathbf{\Theta }}^{-1}),$$

where 0 is a vector of zero of appropriate dimension, and Σ_b and Σ_e are p_bd × p_bd and d × d covariance matrices, respectively. We assumed that b_i and ε_i(t) = (ε_i1(t), … , ε_id(t))^T are independent for all i and t, b_i and b_i′ are independent for i ≠ i′, and ε_i(t) and ε_i′(t′) are independent for i ≠ i′ or t ≠ t′.

To address the challenges described above, we first incorporated the gene-age interactions into x_i to model the dynamic genetic effects on individual brain development trajectories. The individual effect of a risk gene on brain development may vary across time. We used data published by the Alzheimer’s Disease Neuroimaging Initiative (ADNI) to show the estimated trajectories with local polynomial regression (LOESS, Cleveland and Devlin, 1988) of subjects with and without the minor alleles for two selected phenotype-single nucleotide polymorphism (SNP) pairs (Fig. 2b,c). There was a strong interaction effect of the SNP rs769451 and age on the right hippocampus. Ignoring such interaction in model (1) may inflate errors, decrease the detection power of important main genetic effects, and miss the age-dependent genetic effects on imaging phenotypes. In contrast, the interaction effect of SNP rs439401 and age on the left hippocampus was small. We need to efficiently distinguish the important interaction effects from others with negligible interaction effects.

We considered the p × d coefficient matrix B = [β₁ · · · β_d] that characterizes the time-dependent and main genetic effects on the longitudinal phenotypes across all subjects and time points. Given the large d and p, the number of elements in B could be much larger than that of N. Moreover, as shown in Fig. 1a,b, the elements in y_i(t) = (y_i1(t), … , y_id(t))^T and x_i were highly correlated because of the spatial pattern of brain structure and the linkage disequilibrium structure of SNPs, respectively. Therefore, the estimate of B was unstable and often exhibited horizontal- and vertical-banded structures (Fig. 1c). These structures were more apparent after setting unimportant (i.e. insignificant) elements to zero (Fig. 1d). Applying the characteristics of the coefficient matrix, we introduced a low-rank model for B as follows:

$$\mathbf{B}=\mathbf{U}\mathbf{\Delta }{\mathbf{V}}^{\mathrm{T}}=\sum _{l=1}^r{\mathbf{B}}_l=\sum _{l=1}^r{\delta }_l{\mathbf{u}}_l{{\mathbf{v}}_l}^{\mathrm{T}},$$ (2)

where r is the rank or number of layers of B; B_l = δ_lu_lv_l^T is the lth layer for l = 1, … , r; Δ = diag(δ₁, … , δ_r); U = [u₁, · · · , u_r] is a p × r matrix; and V = [v₁, · · · , v_r] is a d×r matrix. Moreover, u_l and v_l may be viewed as the coefficient of the linear combination of y_i(t) and x_i, respectively. Rather than investigating the correlation between all phenotype-SNP pairs, we examined the relation between important linear combinations of similar phenotypes and those of similar SNPs. Because only a few sets of genetic variants were expected to be associated with each longitudinal phenotype, a relatively small r ≪ min(p, d) and a sparse U were considered sufficient to represent the important structure of B. Model (2) considerably reduced the number of unknown parameters of B from pd to (d + p + 1)r and increased the power for detecting the important associations.

We used μ_k(t) to characterize the overall trajectory for the kth neuroimaging measure without consideration of any effects of genomic variables and prognostic factors. The solid curves in Fig. 2b,c depict the LOESS estimates of the development trajectories of the right and left hippocampal volumes, respectively. We used a penalized spline model (Eilers and Marx, 1996; Ruppert et al., 2003; Lang and Brezger, 2004) to model μ_k(t) with a polynomial of degree s given by

$${\mu }_k(t)={\gamma }_{k,0}{t}^0+\cdots +{\gamma }_{k,s}{t}^s+\sum _{m=1}^{q_1-s-1}{\gamma }_{k,s+m}{B}_m(t;{\mathbf{T}}_0)={\mathbf{w}}_1{(t)}^{\mathrm{T}}{\mathit{\gamma }}_{1,k},$$ (3)

where B_m(t;T₀) is a basis function of t; T₀ is a vector of predetermined knots over the range of the t_ij’s (e.g. the sample percentiles); γ_1,k = (γ_k,0, · · · , γ_k,q1−1)^T ; and w₁(t) = (1, t, · · · , t^s,B₁(t;T₀), · · · , B_q1−s−1(t;T₀))^T. Various spline basis functions may be considered for B_m(t;T₀), such as the B-spline basis (Eilers and Marx, 1996), truncated power basis (Ruppert et al., 2003), and wavelet basis (Morris and Carroll, 2006). Using μ_k(t) and the basis functions to flexibly capture the overall trajectories of the longitudinal ROIs measurements increases the detection power of the SNPs and SNP-age interactions and improves the prediction accuracy of responses at later time points.

Another issue in longitudinal studies that must be accounted for is the within-subject correlation among repeated measurements of each phenotype (Verbeke and Molenberghs, 2009; Hyun et al., 2016). Various correlation structures can be formulated through z_i(t)^Tb_ik and Σ_b. We examined the trajectories of all phenotypes across all subjects for the longitudinal data set obtained from the ADNI. The within-subject association mainly occurred from the subject-specific intercept. The trajectories of the left lateral ventricle volume across all subjects are depicted in Fig. 2a. To generate these trajectories, we used a univariate random effect for each ROI by setting p_b = 1, z_i(t) = 1, and Σ_b = τ_bI_d, where I_d is a d × d identity matrix. In this case, the random effects of different ROIs were assumed to be independent.

The correlation structure among the multivariate phenotypes was modeled with another component in (1). The covariance matrix of phenotypic measurements across all subjects and all time points is depicted in Fig. 1a. Phenotypic variables usually exhibit group correlation structures; therefore, we used latent factors to capture these correlation structures. Specifically, we considered an exploratory method of using the sparse factor model (Bhattacharya and Dunson, 2011) for ε_i(t) as follows:

$${\mathit{\epsilon }}_i(t)=\mathbf{\Lambda }{\mathit{\eta }}_i(t)+{\mathit{\xi }}_i(t),$$ (4)

where ∇ is a d × ∞ factor loading matrix, η_i(t) ~ N_∞(0, I_∞), and ξ_i(t) ~ N(0,Σ_ξ) with ${\mathbf{\sum }}_{\xi }=\text{diag}({\sigma }_{\xi ,1}^2,\dots ,{\sigma }_{\xi ,d}^2)$. The structure of ∇ and the number of factors were directly gleaned from the data.

The L2R2 in (1) with (2) – (4) incorporated can be rewritten in the matrix form

$$\mathbf{Y}=\mathbf{XU}\mathbf{\Delta }{\mathbf{V}}^{\mathrm{T}}+\mathbf{W}\mathbf{\Gamma }+\mathbf{Zb}+\mathbf{E}.$$ (5)

The notational definitions, prior distributions of the parameters and the technical details of the posterior computation were provided in the appendix.

3. Simulation study

3.1. Simulation setup

We used simulations to examine the finite-sample performance of the L2R2 model. We simulated Y according to model (5). We considered four cases with various dimensions and priors: (i) p = 50, d = 50; (ii) p = 100, d = 100; (iii) p = 200, d = 100; and (iv) p = 400, d = 100. The dimensions of these cases were comparable with the ADNI data set used in this study and with other related neuroimaging and genetic studies (Wang et al., 2011; Zhu et al., 2014; Marttinen et al., 2014). We chose the first n = 100 subjects from the ADNI data set and selected the top p/2 SNPs from the top 40 genes reported by AlzGene database (www.alzgene.org) as of June 10, 2010. Each subject was observed m_i times to replicate the ADNI data set, resulting in N = 422 records. We used the p/2 SNPs and their interactions with age to form the X in model (5). The Win (5) contains q = 15 columns, including the intercept, the standardized intracranial volume, sex, education, handedness, and the basis function of age. The basis function consists of age, age², age³, and the B-spline basis given the jth percentile, j = 0, 10, 20, … , 100. The elements of b were independently generated from a N(0, 1) distribution.

The following parameters in (5) were used in the simulation: The low-rank coefficient matrix B was generated with the true rank r₀ = 3 in a moderately sparse case with 65% zero elements and in an extremely sparse case with 95% zero elements. Specifically, we set B = UΔV, where U = (u_jl), Δ = diag(δ_ll) = diag(100, 80, 60), and V = (v_lk) as p × 3, 3 × 3, and 3 × d matrices, respectively. Moreover, we generated all elements u_jl and v_lk independently from a N(0, 1)×binomial(1, p₁) distribution and then normalized the columns of U and V to have zero mean and unit variance. The value of p₁ was tuned so that approximately 65% of the elements of B were zeros. Each element of Γ was independently generated as a γ_jk ~ N(0, 1) × binomial(1, 0.5). We simulated ε_i(t) ~ N_d(0,Σ_e), where the precision matrix Σ_e was first generated by a d × d matrix A = (a_jj′) with a_jj = 1 and a_jj′ = uniform(0, 1) × binomial(1, p₂) for j ≠ j′, setting Σ_e = AA^T, and standardizing Σ_e into a correlation matrix. The value of p₂ was tuned so that approximately 20% of the elements in Σ_e were zeros, yielding the mean of the absolute correlations of Σ_e at approximately 0.40.

For each case, 100 simulated data sets were generated. For each simulated data set, we ran the Gibbs sampler for 10, 000 iterations after 5000 burn-in iterations. We chose noninformative priors for the hyperparameters and set a₀ = b₀ = c₀ = d₀ = e₀ = f₀ = 10⁻⁶. For the covariance parameter Σ_e, we chose moderately informative priors to impose the positive-definite constraint.

Besides modeling the trajectories of the response variables to improve detection power of the genetic variables, the splines in W could also be used to predict the response variables at later time points. We calculated the mean of the response variables X_predUΔV^T + W_predΓ + Z_predb for every the subjects at one year after their last observation, where X_pred, W_pred, and Z_pred contain the corresponding predictors for these subjects at the later time points. The prediction was based on X_pred Û Δ̂ V̂^T +W_predΓ̂ + Z_predb̂, where the hat represents the posterior means of the parameters and random effects. We compared the means and predicted means to check the prediction accuracy using the splines in W.

3.2. Comparison with four other state-of-the-art methods

We compared the L2R2 model with four state-of-the-art methods that were developed to establish the association between high-dimensional responses and predictors. First, we considered MMA analysis between the high-dimensional response variables and predictors. Each phenotype was regressed on the covariate matrix W and the residue was associated with each SNP in X with regression. We used the lme4 R package (Bates et al., 2015) for MMA.

Second, we considered the group-sparse multitask regression and feature-selection (G-SMuRFS, Wang et al., 2011) method, which is a representation of associating high dimensional imaging and genetic variables with regularization methods. The longitudinal volumes of ROIs were used as the response variables, and the X and W in L2R2 were combined as the matrix of predictors in G-SMuRFS. The spline basis in W modeled the overall longitudinal trajectories of the ROI volumes, and the SNP-age interactions in X represented the SNP effects on the longitudinal trajectories of ROIs. The large matrix of coefficients was estimated with penalty functions and optimization. Each covariate, SNP and SNP-age interaction formed a group in the penalty function. We used exp(−7), exp(−6), … , exp(2) as possible values of the tuning parameters in the G-SMuRFS method and performed a five-fold cross validation to choose the tuning parameters. G-SMuRFS did not make any assumption on the distribution and correlation structure of the response variables. In our longitudinal study, G-SMuRFS method did not explicitly account for the within-subject temporal correlations and spatial correlations. Also, G-SMuRFS did not use dimension reduction for the coefficient matrix like the low-rank representation of B in L2R2.

Third, we considered the Bayesian canonical correlation analysis (BCCA Klami et al., 2013) to obtain linear combinations of the response variables Y and the predictor X. The coefficients of the linear combinations were conceptually similar to U and V in (5), and we denoted their estimates by Ũ and Ṽ, respectively. The coefficient matrix B was estimated by Ũ Ṽ^T. Gamma mixture multivariate normal distributions were assigned to U and V, and the correlation structure of the response variables and the SNPs were characterized by a factor model. We used the default setting of the companion R package and used eight components in the BCCA method. The R package cannot run a continuous predictor; therefore, we excluded the SNP-age interactions in X. Also, the package does not process covariates like W. Therefore, we regressed Y on W and applied the BCCA method to the residue and X. Unlike the L2R2 model, the BCCA method does not account for within-subject structural correlation or the interactions between SNPs and age.

Fourth, we considered the Bayesian reduced-rank regression (BRRR) method developed by Marttinen et al. (2014). Because the BRRR software package does not allow continuous predictors, we excluded the SNP-age interaction terms in X. We also set the rank to three and ran all of the other program parameters at their default settings. The BRRR method can process W directly, so we did not need to regress Y on W beforehand. In contrast with the L2R2 model, the BRRR method does not account for SNP-age interactions or within-subject structural correlation.

3.3. Evaluation

To evaluate the finite-sample classification performances of all five of the methods, we calculated the sensitivity and specificity scores of selecting the nonzero elements in the coefficient matrix B for each method. The following criteria were used for the selection of the coefficients in each method. As the G-SMuRFS method does not provide standard error estimate for each coefficient, the importance of the association between each pair of ROI and SNP/SNP-age interaction was determined by the absolute values of the coefficient estimates. For all other methods, the coefficients were selected if the absolute values of their standardized estimates (coefficient estimate over its standard error commonly used to account for the estimation uncertainty) were larger than a threshold. The sensitivity and specificity scores for a given threshold T₀ were defined as

$$\text{Se}(T_0)=\frac{\text{TP}(T_0)}{\text{TP}(T_0)+\text{FN}(T_0)}\text{and}\text{Sp}(T_0)=\frac{\text{TN}(T_0)}{\text{TN}(T_0)+\text{FP}(T_0)},$$

where TP(T₀), FN(T₀), TN(T₀), and FP(T₀) were the numbers of true positives, false negatives, true negatives, and false positives, respectively. Changing the threshold T₀ produced different sensitivity and specificity scores, generating the receiver operating characteristic (ROC) curves. In each ROC curve, the sensitivity was plotted against one minus specificity.

3.4. Results

The ROC curves of all four methods and the L2R2 model are depicted in Fig. 3. ROC curves that are closer to the upper-left corner are more likely to identify true positives and control for false positives.

Fig. 3.

Simulation results depicting the estimated ROC curves of the true positive rates and false positive rates for B. The four columns correspond to the settings (i) p = 50, d = 50; (ii) p = 100, d = 100; (iii) p = 200, d = 100; (iv) p = 400, d = 100. The first row shows the coefficients of the SNPs and the second row shows the coefficients of the interactions between the SNPs and subject age. The lines represent the L2R2 (black solid), BRRR (red dotted), BCCA (purple dashed), G-SMuRFS (blue dot-dash), and MMA (green dashed).

In all simulation settings, the L2R2 model outperformed the four competing methods because the L2R2 model was designed to specifically address the challenges described above. Among the four previously developed methods, the G-SMuRFS method had the best performance. However, the G-SMuRFS method did not characterize the correlation structure among the response variables like the L2R2 model did. Therefore, the G-SMuRFS method potentially increased the number of false positives. Consequently, when the number of phenotypic variables increased, the difference between the L2R2 model and the G-SMuRFS method for d = 100 was larger in terms of main effects and interactions than that for d = 50.

The BRRR and BCCA methods cannot process SNP-age interactions because of the limitations of their companion R packages. Therefore, they increase modeling error and reduce the power of detecting important main genetic effects in the presence of the SNP-age interactions. These limitations are worsened as the number of SNPs increases. Although the model setup of the BRRR method was similar to that of the L2R2 model, the prior setting for the BRRR method was more subtle and appeared to be more suitable for small numbers of responses and predictors. Thus, the BRRR method is very sensitive to the number of responses and predictors. The MMA method performed the worst, as it did not account for the correlation structure of responses and predictors or the spatio-temporal correlation structure of the longitudinal phenotypes.

Selected spline functions in the true coefficients (Γ) and their corresponding estimates obtained from the L2R2 model and the G-SMuRFS method are shown in Fig. 4a–b. It is apparent from these splines that the bias of the L2R2 model was smaller than that of the G-SMuRFS method. The estimated trajectories with smaller bias enabled us to better predict the response variables of later time points. A scatter plot of the true means at later time points versus the predicted means across all subjects and response variables in a randomly selected data set is shown in Fig. 4c. The true and predicted means were close to each other, indicating that model (5) can accurately predict longitudinal responses.

Fig. 4.

Simulation results from one randomly selected replication with p = 50, d = 50, and moderately sparse B. (a) and (b) The true and estimated splines for a randomly selected standardized ROI volume. (c) The scatterplot showing the true mean versus the predicted mean at one year after the subject’s last observation using model (5). (d) The Gelman and Rubin’s shrink factors at different iterations for all the parameters in B.

We examined the Markov chainMonte Carlo (MCMC) convergence through calculation of Gelman and Rubin’s shrink factors for all the parameter estimates obtained from three MCMC chains with different start values of parameters (Fig. 4d). The MCMC chains appeared to converge after 2000 iterations, as the shrink factors for all the parameters were less than 1.2. Generating 4000 MCMC samples after discarding 4000 burn-in samples for the L2R2 model to fit the four cases (i.e. p=50, d=50; p=100, d=100; p=200, d=100; and p=400, d=100) took approximately 6, 14, 16, and 22 min, respectively, with a single core of an Intel Xeon E5520 CPU. In comparison, the G-SMuRFS method took 8, 18, 30, and 90 s to fit the four cases, respectively. However, the MCMC samples of the L2R2 model allowed us to make statistical inferences on all parameter estimates, whereas this could not be performed with the G-SMuRFS method. It is important to note that statistical inferences made under a high-dimensional setting require some resampling and splitting procedures (Meinshausen and Bühlmann, 2010). In this case, the G-SMuRFS method would require processing of thousands of subsamples, which may be computationally demanding.

4. Real data analysis

4.1. ADNI data description

The ADNI was launched in 2003 as a 5-year public-private partnership by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, the Food and Drug Administration, private pharmaceutical companies, and nonprofit organizations. The primary goal of the ADNI is to test whether serial magnetic resonance imaging (MRI), positron emission tomography, other biological markers, and clinical and neuropsychological assessments can be combined to measure the progression of mild cognitive impairment and early AD. Determination of sensitive and specific markers of very early AD progression may aid researchers and clinicians in developing new treatments and monitoring their effectiveness, as well as reduce the time and cost of clinical trials.

4.2. Preprocessing

The MRI data were collected across a variety of MRI scanners with individualized protocols for each scanner to obtain standard T1-weighted images, volumetric 3-dimensional sagittal magnetization prepared gradient-echo sequences, or equivalent images with various resolutions. The typical imaging protocol included inversion time=1000 ms, repetition time=2400 ms, flip angle=8°, and field of view=24 cm with a 256×256×170 acquisition matrix in the x–, y–, and z–dimensions, yielding a voxel size of 1.25 × 1.26 × 1.2 mm³. Standard steps such as anterior commissure and posterior commissure correction, skull stripping, cerebellum removing, intensity inhomogeneity correction, segmentation, and registration (Shen and Davatzikos, 2004) were used to preprocess the MRI data. We then carried out automatic regional labeling for the template and transferring the labels after the deformable registration of subject images. After labeling 93 ROIs, we computed the ROI volumes for each subject.

The Human 610-Quad BeadChip (Illumina, Inc., San Diego, CA) was used to genotype the subjects whose images in the ADNI database we analyzed, resulting in a set of 620,901 SNP and copy number variation markers. Because the Apolipoprotein E (APOE) SNPs rs429358 and rs7412 are not included in the Human 610-Quad Bead-Chip, they were genotyped separately. These two SNPs define a three allele haplotype, namely the ε2, ε3, and ε4 variants, and the presence of each variant was available in the ADNI database for all subjects. We used EIGENSTRAT software (package 3.0) to calculate the population stratification coefficients of all subjects. To reduce population stratification effects, we used images from only white subjects who had at least one imaging sample available (749 of 818 subjects in the ADNI data set).

We also performed quality control on this initial set of genotypes (Wang et al., 2011). To input the missing genotypes into our analysis, we used MACH4 software (version 1.0.16) with default parameters to infer the haplotype phase. We also included the APOE-ε₄ variant, coded as the number of observed ε₄ variants. We removed SNPs with more than 5% missing values and entered the mode for the missing SNPs. In the final quality-controlled genotype data, we removed SNPs with a minor allele frequency smaller than 0.1 and a Hardy-Weinberg p-value< 10⁻⁶.

4.3. Data analysis

The aim of our analysis was to apply the L2R2 model to establish an association between the SNPs in the top AD-risk genes reported by AlzGene (http://www.alzgene.org/) and the volumes of 93 brain ROIs collected by the ADNI. We used the L2R2 model to carry out formal statistical inferences, such as the identification of significant SNPs and SNP-age interactions. In this data analysis, we included data from n = 749 subjects and N = 2817 MRI measurements, resulting in an unbalanced data set. Among the subjects, 41 had only one observation and another 67 had only two observations. A random effect was used to account for the subject-specific intercept. For the age effect, we used a quadratic penalized spline with 11 knots that were based on the percentiles of standardized age. We also included intracranial volume, sex, education, and handedness as covariates in W to adjust the ROI volumes.

We considered two sets of top AD-risk genes. First, we used the 114 SNPs in the top 10 genes reported by the AlzGene database. After we performed quality control to the data set, 87 SNPs, APOE-ε4, and their interaction with age were included in the model (1). Second, we selected the 1224 SNPs in the top 40 AD-risk genes by the AlzGene database. After applying quality control to the data, we included 1072 SNPs and their interactions with age in our analysis. A map of the linkage disequilibrium among the 1072 SNPs revealed a clear clustering pattern of SNPs (Fig. 1b). Specifically, the correlations among SNPs within each gene were large (> 0.7), whereas those among SNPs in different genes were relatively small.

We fitted the L2R2 model (1) to the ADNI data as follows: To determine the rank of B, the L2R2 model was run up to r = 10 layers. By comparing the five different selection criteria in Zhu et al. (2014), we chose r = 3 layers as the optimal rank for the final data analysis. We ran the Gibbs sampler for 20, 000 iterations after 20, 000 burn-in iterations. For the G-SMuRFS method, we used the same Y matrix, combined the W and X matrices into a single predictor matrix, and then used the five-fold cross validation to choose the optimal penalty.

On the basis of the MCMC samples, we calculated the posterior median and median absolute deviations of U, V, and B, and used 1.4826 × median absolute deviations to compute the robust standard errors for each element of B. To determine the important coefficients, we used the standardized estimates B_std, which were the absolute values of the estimates over their standard errors. To validate the low-rank structure of B, we used the results of the top 10 genes and set the elements of B to zero if their absolute standardized estimates were larger than a threshold (Fig. 5). The sparsely distributed points along the horizontal and vertical directions indicated that the low-rank model was suitable for characterizing the association between the neuroimaging and genetic variables in the ADNI data set. We used a pragmatic approach to calculate the threshold for the Bayesian estimates, which resembled the Bonferroni correction in a frequentist approach. The threshold was the F(1−0.025/(pr/2+dr+r)), and F was the quantile function of standard normal distribution, where the denominator was the total number of parameters in the low-rank structure U_int, Δ, and V. U_int was the matrix of coefficients of interaction terms. The thresholds for the top 10 genes and top 40 genes were 3.912 and 4.339, respectively. We referred the SNPs that passed the thresholds as significant SNPs though the meaning is different from frequentist significance.

Fig. 5.

The thresholded estimates of the main genetic effects and the SNP-age interaction effects on ROIs based on the ADNI data set with the top 10 genes.

4.4. Results

We determined the longitudinal trajectories of the neuroimaging ROI volumes, the effects of the significant SNPs on such longitudinal trajectories, and the dynamic genetic effects on the trajectories. We estimated the trajectories of all standardized ROI volumes on the basis of Γ̂ and W (Fig. 6). ROIs with decreasing volumes occurred in many regions of the brain, including the left and middle temporal gyri, the left and superior temporal gyri, and the left and right amygdala. ROIs with increasing volumes occurred in the hollow areas of the brain and areas of white matter and included the left and right lateral ventricles, the left and right frontal lobe white matter, and the left and right temporal lobes. Moreover, the trajectories of most ROIs demonstrated structural symmetry.

Fig. 6.

The estimated spline functions for (a) and (b) all the ROIs, (c) and (d) selected ROIs with declining volumes, and (e) and (f) selected ROIs with increasing volumes based on the SNPs from the ADNI data set with the top 10 genes and top 40 genes, respectively.

Because hippocampal atrophy and ventricular enlargement are consistent findings in patients with AD and mild cognitive impairment (Apostolova et al., 2012), we depicted the effects of some SNPs on the longitudinal trajectories of the left and right lateral ventricles and the left and right hippocampi (Fig. 7). Let A and a denote the major and minor alleles of a specific SNP, respectively. For each ROI, we depicted the mean trajectory associated with a selected SNP (black solid curves) and the mean trajectory corresponding to the three allelic combinations (i.e. AA, Aa, and aa) of the SNP (Fig. 7). The minor allele of SNP rs10501608 increased the rate of hippocampal atrophy, whereas the SNP rs1354106 minor allele had the opposite effect. Moreover, the minor allele of SNP rs10501608 increased the rate of ventricular enlargement, yet the SNP rs10501604 minor allele had the opposite effect.

Fig. 7.

The effects of SNP alleles on the trajectories of two ROIs in the ADNI analysis. Black solid line represents the overall effects characterized by the basis functions and coefficients in Γ. The blue dashed, magenta dot-dash, and red dotted lines are the trajectories of the ROIs altered by the SNP with AA, Aa, and aa, respectively. ‘A’ represents the major allele and ‘a’ represents the minor allele of a SNP.

We plotted the primary SNP-age interaction effects, whose absolute standardized estimates were larger than the predetermined threshold, on the longitudinal trajectories corresponding to the top 10 genes (upper row) and the top 40 genes (lower row) (Fig. 8). The plots revealed a generally symmetric pattern for the left and right hemispheres, which indicated the genetic effects on the longitudinal change of ROI volumes were symmetric. Some asymmetric longitudinal genetic effects also occurred. For example, in the analysis of the SNPs from the top 40 genes, only the left nucleus accumbens and the right subthalamic nucleus were associated with SNP rs2273684, whereas their contralateral counterparts were not substantially correlated with any SNP. With exception of the left and right lateral ventricles, the genetic effects on most ROIs in the left and right hemispheres occurred in the same direction. Because both the number of parameters and the threshold for significance increased with the number of top genes, the number of significant SNP-age interactions decreased with the number of top genes. However, the effects of significant SNP-age interaction detected for the top 40 genes were more substantial.

Fig. 8.

Heatmaps of the standardized estimates of the SNP-age interaction coefficients in the (a) and (c) left and (b) and (d) right brain hemispheres. (a) and (b) represent the estimates with the top 10 genes, and (c) and (d) represent the estimates with the top 40 genes. Standardized estimates with absolute value smaller than the threshold were set to zero.

The significant SNP-age interaction effects of the ROIs depicted in Fig. 8 formed SNP-specific networks that we visualized with BrainNet Viewer (Xia et al., 2013) in Fig. 9 and Fig. 10. Among all the networks, each SNP-specific network demonstrated spatially adjacent patterns. Some SNPs were associated with many ROIs, although other SNPs were associated only with ROIs in a specific brain area. The networks could be considerably different for different SNPs in a gene. In each network the sizes of genetic effects on various ROIs was relatively homogeneous, whereas the those across different networks may be substantially different. The most significant SNPs were close to the ventricular system. Some ROIs in the frontal lobe were also associated with SNPs that exhibited small growth effects, but ROIs near the prefrontal cortex were not strongly associated with the SNPs in the top 10 and top 40 genes.

Fig. 9.

The SNP-specific networks of ROIs based on the ADNI data set with the top 10 genes. The first row depicts the locations of ROIs that are correlated with the SNPs rs10501604, rs475639, rs677909, and rs10501608 in gene PICALM; the second row shows those of rs3826656, rs3865444, rs988337, rs1354106 in gene CD33; the third row shows those of rs3752237 and rs3752240 in gene ABCA7 and rs12734030 and rs650877 in gene CR1; the last row shows those of APOE34 (APOE), rs880436 (BIN1) and rs6458573 (CD2AP). The sizes of the dots represent the absolute magnitudes of the regression coefficients. Warmer and cooler colors represent ROIs located in superior and inferior anatomical sites of the brain, respectively.

Fig. 10.

The SNP-specific networks of ROIs based on the ADNI data set with the top 40 genes, which show the locations of ROIs that are correlated with SNPs rs4513489 (CCR2), rs2273684 (PRNP) and rs6088662 (PRNP). The sizes of the dots represent the absolute magnitudes of the regression coefficients. Warmer and cooler colors represent ROIs located in superior and inferior anatomical sites of the brain, respectively.

We used MMA based on linear mixed effects models where each SNP, its interaction with age, and the covariates in W were regressed on each ROI volume. The p-values of the significant coefficients are smaller than 0.05/(pr/2 + dr + r) (Bonferroni correction with the number of parameter in the L2R2), even though the true number of regression parameter is much larger (pd). The number of significant coefficients is smaller if the true number of parameters is used for correction. We compared the important estimated associations with L2R2. The two ROIs identified in the case with top 10 genes were the left and right lateral ventricles. Twelve ROIs were found in the case with top 40 genes, ten of which were found by L2R2. The number of significant associations identified by MMA were much less than that of L2R2. L2R2 provided more power in identifying important pairs of ROIs and SNP-age interactions and revealed more SNPs that may alter the longitudinal trajectories of ROIs.

The results of G-SMuRFS were not directly comparable to those of L2R2 and MMA because G-SMuRFS does not provide standard error estimates or a cut-off for all pairs of association as in L2R2 and MMA. The importance of the SNPs and SNP-age interactions was ranked by the sum of the absolute values of the estimated coefficients across all ROIs. Nonetheless, we found the top ROIs and SNP-age interactions, and some of them overlap with those from L2R2 in the case with top 40 genes.

We built the networks of ROIs with two criteria to study the structure of the longitudinal volumes of ROIs. Let B_int be the part of B corresponding to the SNP-age interactions, and B_int,std be the standardized B_int. Firstly, we selected the ROIs corresponding to the large diagonal elements of ${\mathbf{B}}_{\mathit{int},\mathit{std}}^T{\mathbf{B}}_{\mathit{int},\mathit{std}}$ to form a network. The change of each ROI in the network was heavily associated with the SNPs. The network was a collection of important ROIs which were affected by the time-dependent SNP effects. Secondly, we constructed networks based on the absolute values of every columns of the standardized V_std, which is the standardized V. Each network was an important combination of the longitudinal ROI measurements, and was strongly associated with the joint effect of a weighted combination of SNPs and age-SNP interactions quantified by the corresponding column of U. The top 10 ROIs in the networks were listed in Table 1 and Table 2 for the top 10 and 40 genes, respectively. Such networks were mostly symmetric for the left and right hemispheres, which also illustrated the genetic effects on the longitudinal volumes of ROIs are largely symmetric.

Table 1.

Top-ranked ROIs based on the diagonal of $B_{\mathit{bin}}^T{B}_{\mathit{bin}}$ and columns of V based on the top 10 genes.

$B_{\mathit{int},\mathit{std}}^T{B}_{\mathit{int},\mathit{std}}$	V1	V2	V3
sup.t.gy.R	mid.t.gy.R	caud.neuc.L	prec.L
sup.t.gy.L	mid.t.gy.L	thal.L	per.cort.L
mid.t.gy.L	sup.t.gy.R	caud.neuc.R	tmp.pl.L
mid.t.gy.R	hiopp.R	glob.pal.L	per.cort.R
inf.t.gy.L	amyg.R	post.limb.R	ent.cort.R
parah.gy.L	unc.L	putamen.L	mid.f.gy.R
inf.f.gy.R	unc.R	nuc.acc.R	ling.gy.R
lat.f-o.gy.L	inf.t.gy.R	post.limb.L	sup.f.gy.L
inf.f.gy.L	sup.t.gy.L	insula.R	pec.R
unc.R	inf.t.gy.L	subtha.nuc.L	sup.p.lb.L

Table 2.

Top-ranked ROIs based on the diagonal of $B_{\mathit{bin}}^T{B}_{\mathit{bin}}$ and columns of V based on the top 40 genes.

$B_{\mathit{int},\mathit{std}}^T{B}_{\mathit{int},\mathit{std}}$	V1	V2	V3
insula.R	hiopp.R	per.cort.R	glob.pal.R
insula.L	hiopp.L	sup.p.lb.L	post.limb.R
amyg.R	inf.t.gy.R	per.cort.L	thal.L
hiopp.L	sup.t.gy.R	tmp.pl.R	caud.neuc.R
sup.t.gy.R	unc.L	sup.p.lb.R	caud.neuc.L
unc.L	mid.t.gy.R	me.f.gy.L	nuc.acc.R
unc.R	insula.R	lat.ve.R	ant.caps.R
mid.t.gy.L	amyg.R	pstc.gy.L	glob.pal.L
hiopp.R	mid.t.gy.L	prec.L	subtha.nuc.L
mid.t.gy.R	inf.o.gy.L	tmp.pl.L	subtha.nuc.R

The imaging phenotypes provided quantitative measurements of the morphometric changes of the ROIs, which may characterize the underlying degenerative process of AD and capture information mirroring patients’ diagnostic status. We used a generalized linear-mixed model with logit link to determine the associations between longitudinal diagnostic status and the ROI volume. We found that 70 of 93 ROIs and 69 ROI and age interactions were significantly correlated with longitudinal diagnostic status after applying the false discovery rate correction with the significant level 0.05. Tissues with ROIs that correlated with longitudinal diagnostic status included the hippocampus, cingulate cortex, amygdala, and lateral ventricles, in addition to many other areas of the brain. Moreover, most of the ROIs identified by the L2R2 model in Figure 8 were among the 69 ROIs that demonstrated age-dependent interactions. The SNPs used in the L2R2 model were previously reported in a genome-wide association study of the diagnostic status of patients with AD. Our findings show that the longitudinal trajectories of many ROIs are highly associated with the longitudinal patterns of diagnostic status; therefore, these ROIs may serve as consistent surrogates for measuring AD progression.

The L2R2 model elucidated some ROI-SNP pairs that were reported in previous genome-wide association studies on the basis of diagnostic status, cognitive scores, and imaging measurements. For example, the SNP rs1354106, which occurs in the CD33 gene, was associated with a slower declining rate of the AD assessment scale cognitive score (Sherva et al., 2014). Biffi et al. (2010) reported that SNP rs1408077, which occurs in the CR1 gene, was associated with disease status. Moreover, the SNP rs1408077 was related to the entorhinal cortex thickness (Biffi et al., 2010) and hippocampal atrophy (Lazaris et al., 2015). The SNP rs6088662, which occurs in the PRNP gene, was associated with reduced hippocampal volume and cognitive performance in healthy individuals (Li et al., 2016).

Multivariate imaging measurements may provide additional information compared with univariate measurements, such as diagnostic status. Some ROIs that were not associated with diagnostic status in the generalized linear mixed model were associated with certain SNPs in the L2R2 model, such as the left and right thalamus, right medial frontal gyrus, and right lateral front-orbital gyrus. More ROI measurements yielded additional variation in the SNPs compared with that of diagnostic status. Thus, integrating multivariate imaging measurements with genetic variables and diagnostic status may provide a clearer picture of the biological processes occurring in AD (Zhao and Castellanos, 2016).

4.5. Random effects

We examined the usefulness of incorporating random effects into our model by investigating the distribution of the estimated random effects over all subjects for each ROI. We generated histograms and quantile-quantile plots of the random effects obtained from two randomly selected ROIs from the L2R2 model that were fitted to the top 10 genes (Fig. 11). The variances of the estimated random intercepts were substantial compared with the scale of the response variables, which coincided with the subject-specific intercepts of the neuroimaging measurements (Fig. 2a). Moreover, these distributions did not severely deviate from the normal distribution.

Fig. 11.

Histograms and QQ plots of the posterior means of the random effects for two random selected ROIs over all subjects in the analysis of ADNI data set with the top 10 genes.

To statistically test the presence of the subject-specific random effects Zb, we refitted the data with a restricted L2R2 model without the presence of Zb and then compared the log likelihood ratio of the full and restricted L2R2 models. The inclusion of random effects decreased the log likelihood function by 14,000 (or 5%), and the number of parameters increased by 93. Therefore, incorporating random effects into the L2R2 model was useful for characterizing the correlation structure of the imaging phenotypes and improved the detection of the association between imaging phenotypes and genetic variants.

5. Discussion

We have developed a Bayesian L2R2 model to determine the association between longitudinal imaging responses and covariates with applications in imaging genetic data. We used this model to approximate the large association matrix. We combined a sparse latent factor model and random effects to flexibly capture the complex spatiotemporal correlation structure. We incorporated splines to capture the effect of aging and combined a traditional coefficient estimation with a low-rank approach. The L2R2 model dramatically reduced the number of parameters to be sampled and tested, leading to a remarkably faster sampling scheme and efficient inference. The simulation studies demonstrated that the L2R2 model has higher power in detecting important time-dependent and main genetic effects. Age-dependent genetic effects are useful for characterizing age-related degenerative disorders like AD. When strong genetic effects modify changes in brain morphology over time, it is critically important to account for SNP-age interaction effects and to improve the detection of time-invariant main genetic effects. Although our findings confirmed the important role of well-known genes, such as APOE-ε4, in the pathology of AD, they also elucidated other potential candidate genes that warrant further investigation.

The L2R2 model effectively established associations between phenotypic markers present in neuroimaging data and the presence of high-risk AD alleles. However, many considerations still merit further research. First, it is important to consider the joint effect of genetic markers and environmental factors on high-dimensional imaging phenotypes (Thomas, 2010). Second, it is of great interest to incorporate rare variant genetic markers (Bansal et al., 2010) with the L2R2 model. Third, the key features of the L2R2 model may be adapted to more complex data structures (e.g. twin and family sequencing studies) and other parametric and semiparametric models. Fourth, the L2R2 model may be extended to combine different imaging phenotypes calculated from other imaging modalities (e.g. diffusion tensor imaging, functional MRI, and electroencephalography) in concomitant imaging and genetic studies. Fifth, group structures among SNPs (i.e. gene-based grouping)could be incorporated to improve the efficiency of modeling-correlated SNPs by choosing group priors for U.

One important feature of L2R2 is the inclusion of SNP-age interactions to model the genetic effects on the longitudinal change of ROIs. The interactions can also be viewed as the age-dependent genetic effects. The total age-varying genetic effects of the jth SNP x_ij is β_j + β_2jt_ij, which assumes that the effect is a linear function of age. This assumption can be relaxed by modeling the effects as more flexible functions of age, e.g., polynomial functions and nonparametric functions, which is known as the varying-coefficient models (Hastie and Tibshirani, 1993; Fan and Zhang, 1999, 2008). Varying-coefficient models have been used to study the time-dependent genetic effects and gene-environment interactions (Gong and Zou, 2012; Wu and Cui, 2013; Li et al., 2015). Compared to L2R2, these studies only modeled univariate response variables. Extending L2R2 to incorporate more flexible varying genetic effects is helpful to better understand the genetic impact on the complex diseases at different age and under various states of environment factors.

It is worth noting that in Fig. 2b, the volume of right hippocampal formation appeared to increase in the three-year visit for subjects with minor allele of rs768451. This phenomenon is less expected in studies of Alzheimer’s disease but it may be explained by the following reasons. We included healthy subjects, MCI and AD patients from the ADNI data set in our analysis. The volumes for the healthy subjects may be relatively stable. Moreover, several studies in the literature (Erickson et al., 2011; Leavitt et al., 2014; Niemann et al., 2014) reported that exercises were associated with the increase of hippocampal volume in older adults. Also, the number of subjects in the three-year visit was smaller than those in the previous visits, and the average of the hippocampal volumes for these subjects was larger than the average for all subjects at the initial visit. The dropout during the last visit may be informative. These factors warrant further investigation in the future studies.

7. Appendix: technical details

7.1. Matrix formulation of L2R2

The L2R2 can be rewritten in the matrix form (5)

$$\mathbf{Y}=\mathbf{XU}\mathbf{\Delta }{\mathbf{V}}^{\mathrm{T}}+\mathbf{W}\mathbf{\Gamma }+\mathbf{Zb}+\mathbf{E},$$

where y_i(t) = (y_i1(t), . . . , y_id(t))^T, y_i = (y_i(t_i1), . . . , y_i(t_imi))^T, Y = (y₁^T, . . . , y_n^T)^T, 1_mi be a m_i × 1 vector of ones, and t_mi = (t_i1, . . . , t_i,mi)^T. Let ${\mathbf{x}}_i^{\ast }$ be the SNPs for the ith subject, ${\mathbf{X}}_{\mathit{main}}={({\mathbf{x}}_1^{\ast }\otimes {{\mathbf{1}}_{m_1}}^{\mathrm{T}},\dots ,{\mathbf{x}}_n^{\ast }\otimes {{\mathbf{1}}_{m_n}}^{\mathrm{T}})}^{\mathrm{T}},{\mathbf{X}}_{\mathit{int}}={({\mathbf{x}}_1^{\ast }\otimes {{\mathbf{t}}_{m_1}}^{\mathrm{T}},\dots ,{\mathbf{x}}_n^{\ast }\otimes {{\mathbf{t}}_{m_n}}^{\mathrm{T}})}^{\mathrm{T}}$, and X = (X_main,X_int). Moreover, let Z_i = (z_i(t_i1), . . . , z_i(t_imi))^T, Z be an N × np_b block diagonal matrix with diagonal elements Z₁, . . . ,Z_n, and b = (b_ik) be an np_b × d block matrix with elements b_ik. We defined w_i(t) = (w₁(t)^T,w_2,i(t)^T)^T, W_i = (w_i(1), . . .w_i(t_imi))^T, and W = (W₁^T, . . . ,W_n^T)^T. Let ε_i(t) = (ε_i1(t), . . . , ε_id(t))^T, E_i = (ε_i(t_i1), . . . , ε_i(t_imi))^T, and E = (E₁^T, . . . ,E_n^T)^T. Consequently, Y is an N × d matrix of imaging measurements; X is an N × p matrix of genetic predictors; W is an N × q matrix of covariates with fixed effects; Z is an N × np_b matrix of covariates with random effects; and E is an N × d matrix of error terms, where q = q₁ + q₂. Γ is a q × d matrix and its first q₁ row consists of Γ₁ = (γ_1,1, . . . , γ_1,d), and its last q₂ columns consist of Γ₂ = (γ_2,1, . . . , γ_2,d). In model (5), we primarily made statistical inferences from B (equivalently U,Δ,V) and Γ.

7.2. Priors

We consider priors on the elements of B. Let Ga(a, b) be a gamma distribution with scale a and shape b. We choose L₂ priors on the parameters at each layer B_l as follows:

$${\delta }_l~N(0,{\tau }_{\delta }^{-1}),{\tau }_{\delta }~\text{Ga}(a_0,b_0),{\mathbf{u}}_l~N_p(0,p^{-1}{I}_p),\text{and}{\mathbf{v}}_l~N_d(0,d^{-1}{I}_d),$$

where a₀ and b₀ are prespecified hyperparameters. Although, we have used the same precision parameter for each element of u_l, group information can be incorporated by choosing separate precision parameters for each group. The number of predictors p (or d) is included in the hyperprior of u_l (or v_l) to have a positive definite covariance matrix of high dimensional u_l (or v_l). Moreover, this data driven approach for the priors of u_l and v_l requires no additional hyperparameters to choose. because we standardize all predictors to have zero mean and unit variance, a single prior suffices for all elements of u_l. Moreover, because we rescale all of the responses, we use the same dispersion for all components of v_l. Since we focus on exploiting the potential two-way correlations among the estimated coefficients, we choose the L₂ priors, which tend to borrow strength from correlated neighbors and force the coefficients towards each other to produce two highly correlated coefficients. Moreover, the posterior computations are simpler and faster under the L₂ priors.

We consider priors on the elements of Γ = (γ_jk). We also choose the L₂ prior on the γ_jks as follows:

$${\gamma }_{jk}~N(0,{\tau }_{\gamma }^{-1})\text{and}{\tau }_{\gamma }~\text{Ga}(c_0,d_0),$$

where c₀ and d₀ are hyperparameters. For the subject-specific random coefficients, we also choose independent and identically distributed normal priors as

$${\tau }_b~\text{Ga}(e_0,f_0),$$

where e₀ and f₀ are hyperparameters.

For the sparse factor model (4), we assign the multiplicative gamma process shrinkage prior (Bhattacharya and Dunson, 2011) on Λ to automatically determine the dimension of the factors needed to characterize the error covariance structure. The shrinkage prior increasingly shrinks the factor loadings towards zero with the column index and avoids identifiability of the order of the factor in η_i(t). Specifically, these priors are summarized as follows:

$$\begin{array}{lll}\hfill \mathbf{\Lambda }& =\hfill & \{{\lambda }_{kh}\},k=1,\dots ,d,h=1,\dots ,\infty ,\hfill \\ \hfill {\lambda }_{kh}\mid {\varphi }_{kh},{\tau }_{\lambda h}& ~\hfill & N(0,{\varphi }_{kh}^{-1}{\tau }_{\lambda h}^{-1}),{\varphi }_{kh}~\text{Ga}(v/2,v/2),{\tau }_{\lambda h}=\prod _{l=1}^h{\psi }_l,\hfill \\ \hfill {\psi }_1& ~\hfill & \text{Ga}(a_1,1),{\psi }_l~\text{Ga}(a_2,1),l\ge 2,{\sigma }_{\xi ,k}^{-2}~\text{Ga}(a_{\sigma k},b_{\sigma k}),\hfill \end{array}$$

where v, a₁, a₂, a_σk, and b_σk are prefixed hyperparameters, τ_λh is a global shrinkage parameter for the h-th column, and the ϕ_khs are local shrinkage parameters for the elements in the h-th column. When a₂ > 1, the τ_λhs increase stochastically with the column index h, which shrinks the elements of the loading matrix to zero as the column index progresses and determines the effective dimension of Λ.

7.3. Posterior computation

The joint posterior for the L2R2 model with the above priors can be written as

$$\begin{array}{l}p(\mathbf{U},\mathbf{\Delta },\mathbf{V},\mathbf{\Gamma },\mathbf{b},\mathbf{\Theta }\mid \text{data})\propto {\tau }_{\delta }^{a_0+\frac{1}{2}r-1}{\tau }_{\gamma }^{c_0+\frac{1}{2}qd-1}{\tau }_b^{e_0+\frac{1}{2}n{p}_{b}d-1}{e}^{-b_{0\tau \delta }-d_0{\tau }_{\gamma }-f_0{\tau }_b}\\ exp\left\{-\frac{1}{2}\sum _{l=1}^r\left({\tau }_{\delta }{\delta }_l^2+d{{\mathbf{v}}_l}^{\mathrm{T}}{\mathbf{v}}_l+p{{\mathbf{u}}_l}^{\mathrm{T}}{\mathbf{u}}_l\right)\right\}+exp\left\{-\frac{1}{2}\text{Tr}({\tau }_{\gamma }\mathbf{\Gamma }{\mathbf{\Gamma }}^{\mathrm{T}}+{\tau }_b\mathbf{b}{\mathbf{b}}^{\mathrm{T}})\right\}\\ exp\left\{-\frac{1}{2}\text{Tr}(\mathbf{Y}-\mathbf{XU}\mathbf{\Delta }\mathbf{V}-\mathbf{W}\mathbf{\Gamma }-\mathbf{Zb})\mathbf{\Theta }{(\mathbf{Y}-\mathbf{XU}\mathbf{\Delta }\mathbf{V}-\mathbf{W}\mathbf{\Gamma }-\mathbf{Zb})}^{\mathrm{T}}\right\},\end{array}$$ (6)

where Tr(·) is the trace of the matrix.

We propose a straightforward Gibbs sampler for posterior computation, which converges rapidly. Starting from the initiation step, the Gibbs sampler at each iteration proceeds as follows:

1. For l = 1, . . . , r, update u_l from the full conditional distributions

   $$p({\mathbf{u}}_l\mid -)~N_p({\delta }_l{\mathbf{\sum }}_{{\mathbf{u}}_l}{\mathbf{X}}^{\mathrm{T}}{\mathbf{Y}}_{B,l}\mathbf{\Theta }{\mathbf{v}}_l,{\mathrm{\sum }}_{{\mathbf{u}}_l}),$$

   where ${\mathbf{Y}}_{B,l}=\mathbf{Y}-\mathbf{X}{\sum }_{l^{\prime }\ne l}^r{\mathbf{B}}_l-\mathbf{W}\mathbf{\Gamma }-\mathbf{Zb}$ and ${\mathbf{\sum }}_{{\mathbf{u}}_l}={\{p{\mathbf{I}}_p+{\delta }_l^2({{\mathbf{v}}_l}^{\mathrm{T}}\mathbf{\Theta }{\mathbf{v}}_l){\mathbf{X}}^{\mathrm{T}}\mathbf{X}\}}^{-1}$.

2. Update v_l from

   $$p({\mathbf{v}}_l\mid -)~N_d({\delta }_l{\mathbf{\sum }}_{{\mathbf{v}}_l}\mathbf{\Theta }{{\mathbf{Y}}_{B,l}}^{\mathrm{T}}\mathbf{X}{\mathbf{u}}_l,{\mathbf{\sum }}_{{\mathbf{v}}_l}),$$

   where ${\mathbf{\sum }}_{{\mathbf{v}}_l}={\{d{\mathbf{I}}_d+{\delta }_l^2({{\mathbf{u}}_l}^{\mathrm{T}}{\mathbf{X}}^{\mathrm{T}}\mathbf{X}{\mathbf{u}}_l)\mathbf{\Theta }\}}^{-1}$.

3. Update δ_l from

   $$p({\delta }_l\mid -)~N\left({\sigma }_{{\delta }_l}^2{{\mathbf{u}}_l}^{\mathrm{T}}{\mathbf{X}}^{\mathrm{T}}{\mathbf{Y}}_{B,l}\mathbf{\Theta }{\mathbf{v}}_l,{\sigma }_{{\delta }_l}^2\right),$$

   where ${\sigma }_{{\delta }_l}^2={\{{\tau }_{\delta }+({\mathbf{v}}_l{\mathbf{\Theta }}^{\mathrm{T}}{\mathbf{v}}_l)({{\mathbf{u}}_l}^{\mathrm{T}}{\mathbf{X}}^{\mathrm{T}}\mathbf{X}{\mathbf{u}}_l)\}}^{-1}$.

4. Update τ_δ from

   $$p({\tau }_{\delta }\mid -)~\text{Ga}(a_0+0.5r,b_0+0.5\sum _{l=1}^r{\delta }_l^2).$$
5. Update γ_k (i.e. the kth column of Γ) from

   $$p({\mathit{\gamma }}_k\mid -)~N_d({\mathbf{\sum }}_{{\gamma }_k}{\mathbf{W}}^{\mathrm{T}}\{{\mathbf{y}}_{\mathrm{\Gamma },k}{\theta }_{kk}+({\mathbf{Y}}_{\mathrm{\Gamma },-k}-\mathbf{W}{\mathbf{\Gamma }}_{-k}){\mathit{\theta }}_k,{\mathbf{\sum }}_{{\gamma }_k}),$$

   where Σ_Γk = {θ_kkW^TW+ τ_γI_q}⁻¹. y_Γ,k is the kth column of Y_Γ = Y − XB − Zb; Y_Γ,−k is the matrix after dropping the kth column of Y_Γ; θ_kk is the element for the kth row and kth column of Θ; θ_k is the kth column of Θ after dropping θ_kk; and Θ_−kk is the matrix after dropping kth row and kth column of Θ.

6. Update τ_γ from

   $$p({\tau }_{\gamma }\mid -)~\text{Ga}(c_0+0.5qd,d_0+0.5\sum _{j=1}^q\sum _{k=1}^d{\gamma }_{jk}^2).$$
7. Let b̃_k be the kth column of b. Update b̃_k from

   $$p({\stackrel{\sim }{\mathbf{b}}}_k\mid -)~N({\mathbf{\sum }}_{b_k}{\mathbf{Z}}^{\mathrm{T}}\{{\mathbf{y}}_{b,k}-({\mathbf{Y}}_{b,-k}-\mathbf{Z}{\mathbf{b}}_{-k}){\mathit{\theta }}_k\},{\mathbf{\sum }}_{b_k}^2),$$

   where Σ_bk = (θ_kkZ^TZ + τ_bI_npb)⁻¹. y_b,k is the kth column of Y_b = Y–XB–WΓ, and Y_b,−k is the matrix after dropping the kth column of Y_b.

8. Update τ_b from

   $$p({\tau }_b\mid -)~\text{Ga}(e_o+0.5n{p}_{b}d,f_0+0.5\sum _{i=1}^n\sum _{j=1}^{p_b}\sum _{k=1}^d{b}_{\mathit{ijk}}^2).$$
9. Let k_* be a integer large enough to approximate the number of factors in the factor model (4), and ${\mathit{\epsilon }}_{it}={\mathbf{\Lambda }}^{\ast }{\mathit{\eta }}_{it}^{\ast }+{\mathit{\xi }}_{it}$ be the corresponding approximation. Update ${\mathbf{\Lambda }}_k^{\ast }$, which is the kth row of Λ_*, from its full conditional distribution

   $$p({\mathbf{\Lambda }}_k^{\ast }\mid -)~\mathbf{N}({({\sigma }_{\xi ,k}^{-2}{\mathbf{\Omega }}^{\mathrm{T}}\mathbf{\Omega }+{\mathbf{D}}_k^{-1})}^{-1}{\mathbf{\Omega }}^{\mathrm{T}}{\sigma }_{\xi ,k}^{-2}{\mathbf{E}}_k,{({\sigma }_{\xi ,k}^{-2}{\mathbf{\Omega }}^{\mathrm{T}}\mathbf{\Omega }+{\mathbf{D}}_k^{-1})}^{-1}),$$

   where Ω = (η₁₁, . . . , η_1m1 . . . , η_nmn)^T, E_k is the kth column of E = Y–XB–WΓ−Zb, and $D_k=\text{diag}({\varphi }_{k1}^{-1}{\tau }_{\lambda 1}^{-1},\dots ,{\varphi }_{k{k}_{\ast }}^{-1}{\tau }_{\lambda k_{\ast }}^{-1})$ for k = 1, . . . , d.

10. Update ϕ_kh from its full conditional distribution

    $$p({\varphi }_{kh}\mid -)~\text{Ga}(\frac{v+1}{2},\frac{v+{\lambda }_{kh}^2{\tau }_{\lambda h}}{2}).$$
11. Update ${\sigma }_{\xi ,k}^{-2}$, k = 1, . . . , d, from its full conditional distribution

    $$p({\sigma }_{\xi ,k}^{-2}\mid -)~\text{Ga}(a_{\sigma k}+\frac{n}{2},b_{\sigma k}+\frac{1}{2}\sum _{i=1}^n\sum _{t=1}^{m_i}{({\epsilon }_{\mathit{itk}}-{{\mathbf{\Lambda }}_k^{\ast }}^{\mathrm{T}}{\mathit{\eta }}_{it}^{\ast })}^2),$$

    where ε_itk is the element in E for the ith subject at time t and kth response variable.

12. Update ψ ₁ from its full conditional distribution

    $$p({\psi }_1\mid -)~\text{Ga}(a_1+\frac{1}{2}d{k}_{\ast },1+\frac{1}{2}\sum _{g=1}^{k_{\ast }}{\tau }_{\lambda g}^{(1)}\sum _{k=1}^d{\varphi }_{kg}{\lambda }_{kg}^2),$$

    where ${\tau }_{\lambda g}^{(1)}={\prod }_{t=2}^g{\psi }_t$ for h = 1, . . . , k_*.

13. Update ψ _h, h ≥ 2 from its full conditional distribution

    $$p({\psi }_h\mid -)~\text{Ga}(a_2+\frac{1}{2}d(k_{\ast }-h+1),1+\frac{1}{2}\sum _{g=h}^{k^{\ast }}{\tau }_{\lambda g}^{(h)}\sum _{k=1}^d{\varphi }_{kg}{\lambda }_{kg}^2),$$

    where ${\tau }_{\lambda g}^{(h)}={\prod }_{t=1,t\ne h}^g{\psi }_t$ for h = 1, . . . , k_*.

14. Update η_it for t = 1, . . . , m_i, i = 1, . . . , n from the conditionally independent posteriors

    $$p({\mathit{\eta }}_{it}\mid -)~N({(I_{k_{\ast }}+{\mathbf{\Lambda }}^{\mathrm{T}}{\mathbf{\sum }}_{\xi }^{-1}\mathbf{\Lambda })}^{-1}{{\mathbf{\Lambda }}_{k_{\ast }}}^{\mathrm{T}}{\mathbf{\sum }}_{\xi }^{-1}{\mathit{\epsilon }}_{it},{(I_{k_{\ast }}+{\mathbf{\Lambda }}^{\mathrm{T}}{\mathbf{\sum }}_{\xi }^{-1}\mathbf{\Lambda })}^{-1}),$$

    where ε_it is the row of E corresponding to the ith subject at time t.

7.4. Full conditional distributions of L2R2

7.4.1. Full conditional distribution for Δ

From equation (6), we can write

$$\begin{array}{l}p({\delta }_l\mid -)\propto exp\left(-\frac{1}{2}{\tau }_{\delta }{\delta }_l^2\right)\text{etr}\left\{({\mathbf{Y}}_{B,l}-{\delta }_l\mathbf{X}{\mathbf{u}}_l{\mathbf{v}}_l)\mathbf{\Theta }{({\mathbf{Y}}_{b,l}-{\delta }_l\mathbf{X}{\mathbf{u}}_l{\mathbf{v}}_l)}^{\mathrm{T}}\right\}\\ \propto exp\left({\delta }_l{{\mathbf{u}}_l}^{\mathrm{T}}{\mathbf{X}}^{\mathrm{T}}{\mathbf{Y}}_{B,l}\mathbf{\Theta }{\mathbf{v}}_l-\frac{1}{2}{\delta }_l^2({\tau }_{\delta }+{\mathbf{v}}_l{\mathbf{\Theta }}^{\mathrm{T}}{\mathbf{v}}_l)({{\mathbf{u}}_l}^{\mathrm{T}}{\mathbf{X}}^{\mathrm{T}}\mathbf{X}{\mathbf{u}}_l)\right);\\ p({\tau }_{\delta }\mid -)\propto {\tau }_{\delta }^{a_0+\frac{1}{2}r-1}exp\left(-(b_0+\frac{1}{2}\sum _{l=1}^r{\tau }_{\delta }{\delta }_l^2)\right).\end{array}$$

This implies

$$p({\delta }_l\mid -)~N({\sigma }_{{\delta }_l}^2{{\mathbf{u}}_l}^{\mathrm{T}}{\mathbf{X}}^{\mathrm{T}}{\mathbf{Y}}_{B,l}\mathbf{\Theta }{\mathbf{v}}_l,{\sigma }_{{\delta }_l}^2)\text{and}p({\tau }_{\delta }\mid -)~\text{Ga}(a_0+\frac{1}{2}r,b_0+\frac{1}{2}\sum _{l=1}^r{\delta }_l^2),$$

where ${\sigma }_{{\delta }_l}^2={\{T_{\delta }+({\mathbf{v}}_l{\mathbf{\Theta }}^{\mathrm{T}}{\mathbf{v}}_l)({{\mathbf{u}}_l}^{\mathrm{T}}{\mathbf{X}}^{\mathrm{T}}\mathbf{X}{\mathbf{u}}_l)\}}^{-1}$ and Y_B,l = Y − WΓ − Zb − Σ_l′≠l δ_l′Xu_l′v_l′^T.

7.4.2. Full conditional distribution for U

From equation (6), we can write

$$\begin{array}{l}p({\mathbf{u}}_l\mid -)\propto exp\left(-\frac{1}{2}p{{\mathbf{u}}_l}^{\mathrm{T}}{\mathbf{u}}_l\right)\text{etr}\left\{({\mathbf{Y}}_{B,l}-{\delta }_l\mathbf{X}{\mathbf{u}}_l{\mathbf{v}}_l)\mathbf{\Theta }{({\mathbf{Y}}_{B,l}-{\delta }_l\mathbf{X}{\mathbf{u}}_l{\mathbf{v}}_l)}^{\mathrm{T}}\right\}\\ \propto \text{etr}\left({{\mathbf{u}}_l}^{\mathrm{T}}{\delta }_l{\mathbf{X}}^{\mathrm{T}}{\mathbf{Y}}_{B,l}-\frac{1}{2}{{\mathbf{u}}_l}^{\mathrm{T}}(p{\mathbf{I}}_p+{\delta }_l^2{{\mathbf{v}}_l}^{\mathrm{T}}\mathbf{\Theta }{\mathbf{v}}_l{\mathbf{X}}^{\mathrm{T}}\mathbf{X}){\mathbf{u}}_l\right).\end{array}$$

This implies

$$p({\mathbf{u}}_l\mid -)~N_p({\delta }_l{\mathbf{\sum }}_{{\mathbf{u}}_l}{\mathbf{X}}^{\mathrm{T}}{\mathbf{Y}}_{B,l}\mathbf{\Theta }{\mathbf{v}}_l,{\mathbf{\sum }}_{{\mathbf{u}}_l}),$$

where ${\mathbf{\sum }}_{{\mathbf{u}}_l}={\{p{I}_p+{\delta }_l^2({{\mathbf{v}}_l}^{\mathrm{T}}\mathbf{\Theta }{\mathbf{v}}_l){\mathbf{X}}^{\mathrm{T}}\mathbf{X}\}}^{-1}$.

7.4.3. Full conditional distribution for V

From equation (6), we can write

$$\begin{array}{l}p({\mathbf{v}}_l\mid -)\propto exp\left(-\frac{1}{2}{{\mathbf{v}}_l}^{\mathrm{T}}(d{\mathbf{I}}_d){\mathbf{v}}_l\right)\text{etr}(({\mathbf{Y}}_{B,l}-{\delta }_l\mathbf{X}{\mathbf{u}}_l{\mathbf{v}}_l)\mathbf{\Theta }{({\mathbf{Y}}_{B,l}-{\delta }_l\mathbf{X}{\mathbf{u}}_l{\mathbf{v}}_l)}^{\mathrm{T}})\\ \propto exp\left({{\mathbf{v}}_l}^{\mathrm{T}}{\delta }_l\mathbf{\Theta }{Y_{B,l}}^{\mathrm{T}}\mathbf{X}{\mathbf{u}}_l-\frac{1}{2}{{\mathbf{v}}_l}^{\mathrm{T}}(p{\mathbf{I}}_p+{\delta }_l^2{{\mathbf{u}}_l}^{\mathrm{T}}{\mathbf{X}}^{\mathrm{T}}\mathbf{X}{\mathbf{u}}_l{\mathbf{v}}_l{\mathbf{X}}^{\mathrm{T}}\mathbf{X}){\mathbf{v}}_l\right).\end{array}$$

This gives

$$p({\mathbf{v}}_l\mid -)~N_d({\delta }_l{\mathbf{\sum }}_{{\mathbf{v}}_l}\mathbf{\Theta }{{\mathbf{Y}}_{B,l}}^{\mathrm{T}}\mathbf{X}{\mathbf{u}}_l,{\mathbf{\sum }}_{{\mathbf{v}}_l}),$$

where ${\mathbf{\sum }}_{{\mathbf{v}}_l}={\{d{\mathbf{I}}_d+{\delta }_l^2({{\mathbf{u}}_l}^{\mathrm{T}}{\mathbf{X}}^{\mathrm{T}}\mathbf{X}{\mathbf{u}}_l)\mathbf{\Theta }\}}^{-1}$.

7.4.4. Sampling Γ by columns

Sampling the coefficient matrix one element at a time would be time consuming and less attractive in high-dimensional settings. Another approach is to convert the whole matrix into a vector and sample the vector at once, but this requires a covariance matrix with dimension qd × qd, which can be quite large and requires considerable memory making it infeasible. We propose a columnwise sampling scheme, which allows for computationally efficient sampling with a feasible dimension of the conditional covariance matrix (Khondker et al., 2013). Let Y_Γ = Y − XUΔV − Zb, we can write Y_Γ = WΓ+ E. For k = 1, ⋯, d, we can reorder and partition

$${\mathbf{Y}}_{\mathrm{\Gamma }}=({\mathbf{y}}_{\mathrm{\Gamma },k}{Y}_{\mathrm{\Gamma },-k}),\mathrm{\Gamma }=({\mathit{\gamma }}_k{\mathbf{\Gamma }}_{-k}),\text{and}\mathbf{\Theta }=\left(\begin{array}{cc}{\theta }_{kk}& {\mathit{\theta }}_k\\ {{\mathit{\theta }}_k}^{\mathrm{T}}& {\mathbf{\Theta }}_{-kk}\end{array}\right).$$

In the above partition, y_Γ,k is the kth column of Y_Γ, and Y_Γ,−k and Γ_−k are the matrices after dropping the kth column of Y_Γ and Γ, respectively. θ_kk is the element for the kth row and kth column of Θ; θ_k is the kth column of Θ after dropping θ_kk; and Θ_−kk is the matrix after dropping the kth row and kth column of Θ. We can write

$$\begin{array}{l}p({\mathit{\gamma }}_k\mid -)\propto \text{etr}\left\{-\frac{1}{2}{({\mathbf{Y}}_{\mathrm{\Gamma }}-\mathbf{W}\mathbf{\Gamma })}^{\mathrm{T}}({\mathbf{Y}}_{\mathrm{\Gamma }}-\mathbf{W}\mathbf{\Gamma })\mathbf{\Theta }\right\}\mathit{exp}\left(-\frac{1}{2}{\tau }_{\gamma }{{\mathit{\gamma }}_k}^{\mathrm{T}}{\mathbf{I}}_q{\mathit{\gamma }}_k\right)\\ \propto \text{etr}\left(-\frac{1}{2}{({\mathbf{y}}_{\mathrm{\Gamma },k}-\mathbf{W}{\mathit{\gamma }}_k)}^{\mathrm{T}}({\mathbf{y}}_{\mathrm{\Gamma },k}-\mathbf{W}{\mathit{\gamma }}_k){\mathit{\theta }}_{kk}\right)\times \text{etr}({({\mathbf{y}}_{\mathrm{\Gamma },k}-\mathbf{W}{\mathit{\gamma }}_k)}^{\mathrm{T}}({\mathbf{Y}}_{\mathrm{\Gamma },-k}-\mathbf{W}{\mathbf{\Gamma }}_{-k}){\mathit{\theta }}_k)exp\left(-\frac{1}{2}{{\mathit{\gamma }}_k}^{\mathrm{T}}({\tau }_{\gamma }{\mathbf{I}}_q){\mathit{\gamma }}_k\right)\\ \propto exp\left({{\mathit{\gamma }}_k}^{\mathrm{T}}{\mathbf{W}}^{\mathrm{T}}\{{\mathbf{y}}_{\mathrm{\Gamma },k}{\theta }_{kk}+({\mathbf{Y}}_{\mathrm{\Gamma },-k}-\mathbf{W}{\mathbf{\Gamma }}_{-k}){\mathit{\theta }}_k\}-\frac{1}{2}{{\mathit{\gamma }}_k}^{\mathrm{T}}({\mathit{\theta }}_{kk}{\mathbf{W}}^{\mathrm{T}}\mathbf{W}+({\tau }_{\gamma }{\mathbf{I}}_q)){\mathit{\gamma }}_k\right).\end{array}$$

This gives us

$$p({\mathit{\gamma }}_k\mid -)~N_d({\mathbf{\sum }}_{{\gamma }_k}{\mathbf{W}}^{\mathrm{T}}\{{\mathbf{y}}_{\mathrm{\Gamma },k}{\theta }_{kk}+({\mathbf{Y}}_{\mathrm{\Gamma },-k}-\mathbf{W}{\mathbf{\Gamma }}_{-k}){\mathit{\theta }}_k,{\mathbf{\sum }}_{{\gamma }_k}),$$

where Σ_γk = {θ_kkW^TW+ (τ_γI_q)}⁻¹.

7.4.5. Sampling b

Given that vec(b_i) ∼ N(0,Σ_b) with Σ_b = τ_bI_d, the full conditionals for b can be derived in a similar way to Γ. Let b̃_k be the kth column of b. Update b̃_k from

$$p({\stackrel{\sim }{\mathbf{b}}}_k\mid -)~N({\mathbf{\sum }}_{b_k}{\mathbf{Z}}^{\mathrm{T}}\{{\mathbf{y}}_{b,k}-({\mathbf{Y}}_{b,-k}-\mathbf{Z}{\mathbf{b}}_{-k}){\mathit{\theta }}_k\},{\mathbf{\sum }}_{b_k}^2),$$

where Σ_bk = (θ_kkZ^TZ+τ_bI_npb)⁻¹. y_b,k is the kth column of Y_b = Y–XB–WΓ; and Y_b,−k and b_k is the matrix after dropping the kth column of Y_b and b, respectively.

Full conditionals for all other parameters are straightforward.