Generalized reduced rank latent factor regression for high dimensional tensor fields, and neuroimaging-genetic applications

Abstract

We propose a generalized reduced rank latent factor regression model (GRRLF) for the analysis of tensor field responses and high dimensional covariates. The model is motivated by the need from imaging-genetic studies to identify genetic variants that are associated with brain imaging phenotypes, often in the form of high dimensional tensor fields. GRRLF identifies from the structure in the data the effective dimensionality of the data, and then jointly performs dimension reduction of the covariates, dynamic identification of latent factors, and nonparametric estimation of both covariate and latent response fields. After accounting for the latent and covariate effects, GRLLF performs a nonparametric test on the remaining factor of interest. GRRLF provides a better factorization of the signals compared with common solutions, and is less susceptible to overfitting because it exploits the effective dimensionality. The generality and the flexibility of GRRLF also allow various statistical models to be handled in a unified framework and solutions can be efficiently computed. Within the field of neuroimaging, it improves the sensitivity for weak signals and is a promising alternative to existing approaches. The operation of the framework is demonstrated with both synthetic datasets and a real-world neuroimaging example in which the effects of a set of genes on the structure of the brain at the voxel level were measured, and the results compared favorably with those from existing approaches.

1. Introduction

The past decade has witnessed the dawn of the big data era. Advances in technologies in areas such as genomics and medical imaging, among others, have presented us with an unprecedentedly large volume of data characterized by high dimensionality. This not only brings opportunities but also poses new challenges to scientific research. Neuroimaging-genetics, one of the burgeoning interdisciplinary fields emerging in this new era, aims at understanding how the genetic makeup affects the structure and function of the human brain and has received increasing interest in recent years.

Starting with candidate gene and candidate phenotype studies, imaging-genetic methods have made significant progress over the years (Thompson et al., 2013; Liu and Calhoun, 2014; Poline et al., 2015). Different strategies have been implemented to combine the genetic and neuroimaging information, producing many promising results (Hibar et al., 2015; Richiardi et al., 2015; Jia et al., 2016). Using a few summary variables of the brain features is the most popular approach in the literature (Joyner et al., 2009; Potkin et al., 2009; Vounou et al., 2010); voxel-wise and genome-wide association approaches offer a more holistic perspective and are used in exploratory studies (Hibar et al., 2011; Vounou et al., 2012); multivariate analyses have also been used to capture the epistatic and pleiotropic interactions, therefore boosting the overall sensitivity (Hardoon et al., 2009; Ge at al., 2015a,b). Apart from the population studies, family-based studies offer additional insights on the genetic heritability (Ganjgahi et al., 2015). Recently, a few probabilistic approaches have been proposed to jointly model the interactions between genetic factors, brain endophenotypes and behavior phenotypes (Batmanghelich et al., 2013, Stingo et al., 2013), and some Bayesian methods originally developed for eQTL studies can also be applied to imaging-genetic problems (Zhang and Liu, 2007; Jiang and Liu, 2015).

The trend in imaging-genetics is to embrace brain-wide genome-wide association studies with multivariate predictors and responses, but this is challenged by the combinatorial complexity of the problem. For example, the probabilistic formulations do not scale well with dimensionality; and standard brute force massive univariate approaches (Stein et al., 2010a; Vounou et al., 2012) treat each voxel and predictor as independent units and compute pairwise significance, and the loss of spatial information and the colossal multiple comparison corrections involved have high costs in terms of sensitivity (Hua et al., 2015). Various attempts have been made to remedy this. Some approaches involve dimension reduction techniques, which either first embed genetic factors onto some lower dimensional space using methods such as principal component analysis (PCA) before subsequent analyses (Hibar et al., 2011), or jointly project genetic factors and imaging traits by methods such as parallel independent component analysis (pICA), canonical correlation analysis (CCA) and partial least square (PLS) (Liu et al., 2009; Le Floch et al., 2012, 2013). These methods often lack model interpretability. Other popular approaches enforce penalties or constraints to regularize the solutions, for example (group) sparsity or rank constraints (Wang et al., 2012a,b; Vounou et al., 2012; Lin et al., 2015; Huang et al., 2015). But they are usually difficult to compute and the significance of the findings cannot be directly evaluated.

One path towards more efficient estimation for brain-wide association, both in the statistical and computational sense, is to exploit the inherent spatial structure from the neuroimaging data. Two prominent examples in this direction are random field theory based methods (Worsley et al., 1996, Penny et al., 2011; Ge et al., 2012) and functional based methods (Wahba, 1990; Ramsay and Silverman, 2005; Reiss and Ogden, 2010) where the smoothness of the data is considered. Random field methods are established as the core inferential tool in neuroimaging studies. These methods correct the statistical thresholds based on the smoothness estimated from the images, resulting in increased sensitivity. Functional based methods explicitly use smooth fields parametrized by smooth basis functions in the model, thereby regularizing the solution and simplifying the estimation at the same time. Related to functional methods are tensor-based methods (Zhou et al., 2013; Li, 2014) and wavelet-based methods (Van De Ville et al., 2007; Wang et al., 2014), where either low rank tensor factorization or a wavelet basis is used to approximate the spatial field of interest.

Long overlooked in neuroimaging studies, including imaging-genetics, is the influence from unobservable latent factors (Bhattacharya and Dunson et al., 2011; Montagna et al., 2012). An illustrative cartoon is presented in Fig. 1 for a typical neuroimaging-genetic case, in which the effect of interest is usually small compared with the total variance. This is known as low signal to noise ratio (SNR). Large-scale multi-center collaborations have become a common practice in the neuroimaging community (Jack et al., 2008; Michael et al., 2012; Van Essen et al., 2013; Thompson et al., 2014) and increasing numbers of researchers are starting to pool data from different sources. The heterogeneity of the data introduces large unexplained variance originating from population stratification or cryptic relatedness, for example genetic background, medical history, traumatic experiences and environmental impacts. Such variance aggregates the SNR issue and confuses the estimation procedures if unaccounted for. However these confounding factors are usually difficult or costly to quantify, and therefore they are hidden from the data analysis in most, if not all, studies.

Fig. 1.

An illustrative cartoon for latent influence in imaging-genetic studies. Low variance genetic effects could be dominated by large variance latent effects. (For simplicity we omit the fixed effect term from the covariates in this illustrative cartoon.)

To see how the latent factor-induced variance undermines the power of statistical procedures, let us take the most commonly used least squares regression as an example. Assume the model Y = Xβ + L + E, where Y is the response, X is the predictor of interest, β is the regression coefficient, L is the unobservable latent factor and E is the noise term. In the absence of knowledge regarding L, the alternative model $Y=X\stackrel{\sim }{\beta }+\stackrel{\sim }{E}$ is estimated instead, where $\stackrel{\sim }{E}=L+E$. Assuming independence between X, L and E, we have $\text{var}[\stackrel{\sim }{E}]=\text{var}[L]+\text{var}[E]$, where var[·] measures the variance. Denote $\widehat{\beta }$ the oracle estimator where the true model is fit with the knowledge of L and $\widehat{\stackrel{\sim }{\beta }}$ the estimator for the alternative model, the asymptotic theory of least square estimator tells us $\widehat{\beta }\sim \mathcal{N}(\beta ,\mathrm{var}\left[E\right]{(X\prime X)}^{-1})$ and $\widehat{\stackrel{\sim }{\beta }}\sim \mathcal{N}(\beta ,\mathrm{var}\left[\stackrel{\sim }{E}\right]{(X\prime X)}^{-1})$ as the sample size goes to infinity, that is to say $\widehat{\stackrel{\sim }{\beta }}$ is more variable than $\widehat{\beta }$ and converges slowly to the population mean. See Fig. 2 for a graphical illustration.

Fig. 2.

An illustrative example for how the latent factor induced variance undermines the statistical efficiency of least squares estimator. The color coded region are the distribution of the oracle estimator $\widehat{\beta }$ (red) and the alternative estimator $\widehat{\stackrel{\sim }{\beta }}$ (purple) under different sample sizes, with the nonzero population mean β. The oracle estimator requires smaller sample size to achieve the desired sensitivity. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this paper.)

Solutions have been proposed to alleviate the loss of statistical efficiency caused by latent factors. In Zhu et al. (2014) the authors propose to dynamically estimate the latent factors from the observed data. However this approach is based on Markov chain Monte-Carlo (MCMC) sampling, and therefore the computational cost is prohibitive for high dimensional tensor field applications. In the eQTL literature, several methods that explicitly account for the hidden determinants have been developed. Following a Bayesian formulation, Stegle et al. (2010) factors out the hidden effect; Fusi et al. (2012), however, computes the ML estimate of hidden factors by marginalizing out the regression coefficients and then using the estimated hidden factors to construct certain covariance matrices for subsequent analyses. These studies are not concerned with the spatial structure and the inherent dimensionality of the model, and the results depend on the choice of parameters for the prior distributions. Additionally, these studies consider latent effect as “variance of no interest”, but as we will see in later sections, the latent structure also contains vital information and therefore should not be simply disregarded as unwanted variance.

In this article, we formulate a new generalized reduced rank latent factor regression model (GRRLF) for high dimensional tensor fields. Our method exploits the spatial structure of the neuroimaging data and the low rank structure of the regression coefficient matrix, which computes the effective covariate space, improves the generalization performance and leads to efficient estimation. The model works for general tensor field responses which include a wide range of imaging modalities, i. e. MRI, EEG, PET, etc. Although motivated by imaging-genetic applications, the proposed GRRLF is thus widely applicable to almost all types of neuroimaging studies. The estimation is carried out via minimizing a properly defined loss function, which includes maximum likelihood estimation (MLE) and penalized likelihood estimation (PLE) as special cases.

The contributions of this paper are four-fold. Firstly, we introduce field-constrained latent factor estimation for high dimensional tensor field regression analysis. It efficiently explains the covariance structure in the data caused by the hidden structures. Secondly, our model integrates dimension reduction, that not only improves the statistical efficiency but also facilitates model interpretability. Thirdly, we provide several implementations to efficiently compute the solution under constraints, including Riemannian manifold optimization (Absil et al., 2009) and nuclear norm regularization which are both based on manifold optimization. We highlight the flexibility of using manifold optimization to formulate neuroimaging problems, which can lead to further interesting applications. Lastly, we present an efficient kernel approach for brain-wide genome-wide association studies under the GRRLF framework and apply it to the ADNI dataset. Empirical results provide evidence that the kernel GRRLF approach is capable of capturing the interactions that can be missed in conventional studies.

The rest of the paper is organized as follows. In Section 2, we detail the model formulation and estimation. In Section 3, the proposed method is evaluated with both synthetic and real-world examples and compared with other conventional approaches. Finally we conclude this paper with a summary and future prospects in Section 4. The real-world data used and detailed preprocessing steps are described in Appendix A. MATLAB scripts for GRRLF are available online from http://github.com/chenyang-tao/grrlf/.

2. Materials and methods

2.1. Model formulation

Denote the Ω as the spatial domain of the brain and v as its spatial index, $\mathcal{X}$, $\mathcal{Y}$ are the random vectors/fields of covariates and responses, we denote $\mathit{X}={\left\{{\mathit{x}}_i\right\}}_{i=1}^n$ and $\mathit{Y}=\{{\mathit{y}}_{i,\mathit{v}}|i=1,\cdots ,n,\mathit{v}\in \Omega \}$ the respective empirical sample where ${\mathit{x}}_i\in {ℝ}^p$, ${\mathit{y}}_{i,\mathit{v}}\in {ℝ}^q$ and n is the sample size. Here p is the dimension of covariates and q is the number of image modalities (for example, y_i,_v is the 3×3 diffusion tensor from DTI imaging, the 3×1 tissue composition (WM, GM, CSF) from VBM analysis or the time series of a task response). All $\mathit{B}\in {ℝ}^p{ }^{\times d}$ orthonormal matrices, i.e. B^⊤B = I_d, form a Riemannian manifold known as the Stiefel manifold and denoted as ${\mathcal{S}}_d\left({ℝ}^p\right)$ while a less restrictive manifold requiring only diag(B^⊤B) = I_d is called the oblique manifold with the notation ${\mathcal{O}}_d\left({ℝ}^p\right)$. We call d the effective dimension of $\mathcal{X}$ w.r.t to $\mathcal{Y}$ if $\mathcal{X}⫫\mathcal{Y}|B^{\top }\mathcal{X}$ for some projection matrix $\mathit{B}\in {\mathcal{S}}_d\left({ℝ}^p\right)$ where ⫫ stands for independence and ·|· is the conditioning operator. The voxel-wise model writes

$${\mathit{y}}_{i,\mathit{v}}={\stackrel{\sim }{\mathbf{\Phi }}}_{\mathit{v}}{\mathit{B}}^{\top }{\mathit{x}}_i+{\stackrel{\sim }{\mathbf{\Gamma }}}_{\mathit{v}}{\mathit{l}}_i+{\mathit{\xi }}_{i,\mathit{v}},$$ (1)

where x is the covariate term, $\mathit{l}\in {ℝ}^t$ is the latent factor, ${\mathit{\xi }}_{\mathit{v}}\in {ℝ}^q$ is the noise, ${\stackrel{\sim }{\mathbf{\Phi }}}_{\mathit{v}}\in {ℝ}^{q\times d}$ is the covariate regression coefficient matrix and ${\stackrel{\sim }{\mathbf{\Gamma }}}_{\mathit{v}}\in {ℝ}^{q\times t}$ the latent factor loading matrix.

To understand model (1), let us consider a concrete example. Say for example, a researcher is interested in how substance abuse alters brain morphometry. The researcher has collected voxel-wise gray matter and white matter volumes (response ${\mathit{y}}_{\mathit{v}}\in {ℝ}^2$), and various evaluation scores related to substance abuse, including the Alcohol Use Disorders Identification Test (AUDIT) (Saunders et al., 1993), Fagerstrom Test for Nicotine Dependence (FTND) (Heatherton, 1991) and Substance Use Risk Profile Scale (SURPS) (Woicik et al., 2009) for a group of subjects. Each of these evaluations has several sub-scores and altogether the researcher has a 14 dimensional feature vector for each subject (covariate $\mathit{x}\in {ℝ}^{14}$). These features are correlated, and it is expected that a low dimensional summary (effective covariate $\stackrel{\sim }{\mathit{x}}={\mathit{B}}^{\top }\mathit{x}\in {ℝ}^d$, d ∈ [1,⋯,3]) is sufficient to explain the variations in brain morphometry caused by substance abuse. The researcher also collects covariates of no interest, such as age, gender and race, that correlate with the imaging features and will be modeled to remove their effect. The researcher is aware that population stratification and subjects’ medical history can affect brain tissue volumes, but unfortunately, the subjects are not genotyped and their individual files do not cover medical records therefore such information is unavailable (latent status l).

For notational simplicity hereafter we assume q=1 so that we can write the brain-wide model in matrix form. Denote N_vox the number of voxels within Ω, then with $\mathit{Y}\in {ℝ}^{n\times N_{\text{vox}}}$ the observation matrix, $\mathit{X}\in {ℝ}^{n\times p}$ the covariate matrix, $\stackrel{\sim }{\mathbf{\Phi }}\in {ℝ}^{p\times N_{\text{vox}}}$ the covariate effect, $\mathit{L}\in {ℝ}^{n\times t}$ the latent status matrix, $\stackrel{\sim }{\mathbf{\Gamma }}\in {ℝ}^{t\times N_{\text{vox}}}$ the latent response and $\mathit{E}\in {ℝ}^{n\times N_{\text{vox}}}$ the noise term, we have the matrix form of the brain-wide model

$$\mathit{Y}=\mathit{X}\mathit{B}\stackrel{\sim }{\mathbf{\Phi }}+\mathit{L}\stackrel{\sim }{\mathbf{\Gamma }}+\mathit{E},$$ (2)

In the case {ξ_v} are i.i.d Gaussian variables, the maximal likelihood solution of GRRLF is

$$\left\{\widehat{\stackrel{\sim }{\mathbf{\Phi }}},\widehat{\mathit{B}},\widehat{\stackrel{\sim }{\mathbf{\Gamma }}},\widehat{\mathit{L}}\right\}=\underset{B,\stackrel{\sim }{\Phi },\stackrel{\sim }{\Gamma },L}{{\displaystyle \arg \min }}{\Vert \mathit{Y}-\mathit{X}\mathit{B}\stackrel{\sim }{\mathbf{\Phi }}-\mathit{L}\stackrel{\sim }{\mathbf{\Gamma }}\Vert }_{\mathrm{F}}^2\text{subject}\text{to}\mathit{B}\in {\mathcal{S}}_d\left({ℝ}^p\right)\text{and}\mathit{L}\in {\mathcal{O}}_t\left({ℝ}^n\right),$$ (3)

where ‖·‖_Fro is the Frobenius norm. We note that the restriction on L is simply a normalization and $(\mathit{B},\stackrel{\sim }{\mathbf{\Phi }})$ is an equivalent class under orthogonal transformations, i.e. if (B, Φ) is a solution then (BQ, Q^⊤Φ) is also a solution for all unitary matrices $\mathit{Q}\in {ℝ}^d{ }^{\times d}$. More generally, GRRLF can be formulated as

$$\left\{\widehat{\stackrel{\sim }{\mathbf{\Phi }}},\widehat{\mathit{B}},\widehat{\stackrel{\sim }{\mathbf{\Gamma }}},\widehat{\mathit{L}}\right\}=\underset{B,\stackrel{\sim }{\Phi },\stackrel{\sim }{\Gamma },L}{{\displaystyle \mathrm{argmin}}}\ell \left(\mathit{X},\mathit{Y}|\mathit{B},\stackrel{\sim }{\mathbf{\Phi }},\mathit{L},\stackrel{\sim }{\mathbf{\Gamma }}\right)\text{subject}\text{to}\mathit{B}\in {ℳ}_1,\mathit{L}\in {ℳ}_2,$$ (4)

where ℓ is some loss function and ${\left\{{ℳ}_i\right\}}_{i=1}^2$ are some Riemannian manifolds to constrain the solution.

2.2. Smoothing the tensor fields

To more effectively exploit the spatial structures, further constraints can be enforced. For example, it is natural to assume the smoothness of tensor fields $\stackrel{\sim }{\mathbf{\Phi }}$ and $\stackrel{\sim }{\mathbf{\Gamma }}$. In this work, we assume $\stackrel{\sim }{\mathbf{\Phi }}$ and $\stackrel{\sim }{\mathbf{\Gamma }}$ can be approximated by linear combinations of some (smooth) basis functions as

$${\stackrel{\sim }{\mathbf{\Phi }}}_{\mathit{v}}={\displaystyle \sum _{b=1}^{N_{\text{knot}}}{h}_b\left(\mathit{v}\right){\mathbf{\Phi }}_b},{\stackrel{\sim }{\mathbf{\Gamma }}}_{\mathit{v}}={\displaystyle \sum _{b=1}^{N_{\text{knot}}}{h}_b\left(\mathit{v}\right){\mathbf{\Gamma }}_b},$$

where ${\left\{h_b(\cdot )\right\}}_{b=1}^{N_{\text{knot}}}$ is the set of basis functions, $\{{\mathbf{\Phi }}_b\in {ℝ}^{q\times d}\}$ and ${\mathbf{\Gamma }}_b\in {ℝ}^{q\times t}$ are the coefficients, and here we have assumed both tensor fields have the same “smoothness” for notational clarity. Similar to model (2) the smoothed model can be written in the matrix form as

$$\mathit{Y}=\mathit{X}\mathit{B}\mathbf{\Phi }\mathit{H}+\mathit{L}\mathbf{\Gamma }\mathit{H}+\mathit{E},$$ (5)

where $\mathbf{\Phi }\in {ℝ}^{d\times N_{\text{knot}}}$ and $\mathbf{\Gamma }\in {ℝ}^{t\times N_{\text{knot}}}$ are the coefficient matrices, and we call $\mathit{H}\in {ℝ}^{N_{\text{knot}}\times N_{\text{vox}}}$ the smoothing matrix. $\stackrel{\sim }{\mathbf{\Phi }}=\mathbf{\Phi }\mathit{H}$ and $\stackrel{\sim }{\mathbf{\Gamma }}=\mathbf{\Gamma }\mathit{H}$ are respectively referred to as the covariate response field and the latent response field, $\stackrel{\sim }{\mathit{B}}=\mathit{B}\mathbf{\Phi }\in {ℝ}^{p\times N_{\text{knot}}}$ as the covariate effect matrix and $\stackrel{\sim }{\mathit{L}}=\mathit{L}\mathbf{\Gamma }\in {ℝ}^{n\times N_{\text{knot}}}$ as the latent effect matrix. Since N_knot < < N_vox, the smoothing operation can significantly reduce the number of parameters to be optimized. In this study we have used Gaussian radial basis functions (RBF) ${\{h_b(\mathit{v})=\text{exp}(-{\Vert \mathit{v}-{\mathit{v}}_b\Vert }_2^2/2{\sigma }^2)\}}_{b=1}^{N_{\text{knot}}}$ as basis functions, where v_b ∈ Ω is the bth knot and σ² is the bandwidth parameter. Other non-smooth basis functions can also be used if they are well justified for the application.

Note that (4) is a very general formulation that encompasses many statistical models as special cases. In Table A1 we provide an inexhaustive list of loss function ℓ that lead to commonly used statistical models.

2.3. Generalized cross-validation procedure

GRRLF needs constraint parameters Θ to regularize its solution, therefore a parameter selection procedure is necessary to ensure good generalization performance. In the literature, a cross-validation (CV) procedure is often used to assess the generalizing performance of the parameters, by evaluating the loss with the validation set and the parameters estimated from the training set. However, for GRRLF the conventional CV procedure cannot be used, because the latent parameters are unique to the validation set, and as such, they cannot be estimated from the training set. Here we propose a generalized cross-validation (GCV) procedure to resolve this dilemma.

Assuming that the training and validation sets are drawn from the same distribution, we know that for the latent component the latent response field $\stackrel{\sim }{\mathbf{\Gamma }}$ is shared by both sets while the latent status variables L are different. Therefore given $\{\widehat{\mathit{B}},\widehat{\mathbf{\Phi }},\widehat{\mathbf{\Gamma }}\}$_, we can estimate the latent status L of the validation set by minimizing the residual error ${\Vert {\mathit{Y}}_{\text{test}}-{\mathit{X}}_{\text{test}}\widehat{\mathit{B}}\widehat{\mathbf{\Phi }}\mathit{H}-\mathit{L}\widehat{\mathbf{\Gamma }}\mathit{H}\Vert }_F^2$ of the validation set and using the minimal residual error as the generalizing performance score. The pseudo code for GCV is given in Algorithm 1.

Algorithm 1.

Generalized cross-validation procedure for GRRLF.

Input: Xeval, Yeval, Xtext, Ytext, H, Θ
Output: ErrCV
$\left(\widehat{\stackrel{\sim }{\mathit{B}}},\widehat{\stackrel{\sim }{\mathit{L}}}\right):=\text{GRRLF}\left({\mathit{X}}_{\text{eval}},{\mathit{Y}}_{\text{eval}},\mathit{H},\mathbf{\Theta }\right)$;
$\left[\mathit{U},\mathbf{\sum },\mathit{V}\right]:=\text{SVD}\left(\widehat{\stackrel{\sim }{\mathit{L}}}\right),\widehat{\mathbf{\Gamma }}:={\mathit{V}}^{\top }$;	$/\ast \widehat{\stackrel{\sim }{\mathit{L}}}:=\mathbf{U}\sum {\mathit{V}}^{\top }\ast /$
${\widehat{\stackrel{\sim }{\mathit{R}}}}_{\text{test}}:={\mathit{Y}}_{\text{test}}-{\mathit{X}}_{\text{test}}{\widehat{\stackrel{\sim }{\mathit{B}}}}_{\text{eval}}\mathit{H}$;
${\widehat{\mathit{L}}}_{\text{test}}:=\arg {\min }_L{\Vert {\widehat{\stackrel{\sim }{\mathit{R}}}}_{\text{test}}-\mathit{L}\widehat{\mathbf{\Gamma }}\mathit{H}\Vert }_F^2$;	/* Least squares */
${\text{Err}}_{\text{CV}}:={\Vert {\widehat{\stackrel{\sim }{\mathit{R}}}}_{\text{test}}-{\widehat{\mathit{L}}}_{\text{test}}\widehat{\mathbf{\Gamma }}\mathit{H}\Vert }_F^2$;

2.4. Estimation based on Riemannian manifold optimization

Since (4) is a nonlinear optimization problem constrained on Riemannian manifolds it is difficult to optimize directly. A key observation is that individually optimizing {B, Φ, L, Γ} reduces to a linear problem, which suggests the use of the so-called block relaxation algorithm (De Leeuw, 1994; Lange, 2010) to alternately update {B, Φ, L, Γ} at each iteration until convergence. In this work, we use the manopt toolbox (Boumal et al., 2014) to efficiently solve the manifold optimization problem (4). Manopt provides a general framework for solving Riemannian manifold optimization problems, which grants the modeler the freedom of specifying the constraints for the model without worrying about the implementation details and still use an efficient solver. We remark that while the general purpose solver relieves the burden from the modeler, the computational efficiency can be significantly improved by using a customized solver w.r.t the loss ℓ. Here we detail the implementation details of our customized solver for (3).

The key idea of GRRLF is to improve the estimation of weak signals by accounting for the strong signals. Therefore, if the covariate signal or the latent signal is of interest, and there is no prior knowledge of which signal is “dominating”, a choice on which component is used to start the iteration should be made. For example, if the covariate effect is dominating but the latent effect is estimated first, then part of the covariate effect might be erroneously interpreted as latent effect. Here we propose to base our decision on the generalizing performance from GCV. If the ‘latent first’ strategy is favored by GCV, we further test for the association between covariates and estimated latent status variables, using dependency tests such as CCA or more general Hilbert–Schmidt independence criteria (HSIC) (Gretton et al., 2007). If a significant association is detected between the covariates and the latent status variables, the previous decision is overruled and, instead, we estimate the covariate effect first. The complete estimation procedure is summarized in Algorithm 2, and hereafter we refer to it as the general manifold GRRLF implementation (GM-GRRLF). More sophisticated procedures, that control for the dependency between covariates and latent components, are discussed in later sections.

Algorithm 2.

GM-GRRLF.

2.5. Model selection

The performance of GRRLF depends on the parameters (d,t), denoting the effective dimension of the covariates and latent factors. In the absence of prior knowledge of (d,t), we can use the Akaike information criterion (AIC) (Akaike, 1974), the Bayesian information criterion (BIC) (Schwarz et al., 1978) or generalized cross-validation described above to dynamically determine these two parameters. Likelihood models can use *IC (and possibly also a χ²-test) to select (d,t) while for other more general models the GCV approach is preferred. For (3), fast determination of (d,t) can be achieved by combining RRR and PCA. The sequence of RRR and PCA is determined based on generalized cross validation. Both RRR and PCA involve solving an eigenvalue problem and the magnitude of the eigenvalues provides information about the inherent structural dimensionality of the data. Assuming the eigenvalues estimated are sorted in descending order, the ‘elbow’ or ‘jump point’ of the eigenvalue curve is used as an estimate of the rank/structural dimensionality.

2.6. Constrained nuclear norm formulation of GRRLF

In this section we present an alternative formulation of GRRLF using the nuclear norm regularization (NNR), which has a global optima and can be solved with convex optimization techniques. NNR is a powerful tool restoring the low rank structure of matrices with noisy or incomplete observations and is widely used in machine learning applications (Yuan et al., 2007; Koren et al., 2009; Candès and Tao, 2010).

Notice that solving (5) is a nonlinear optimization problem, and its solutions can be easily trapped in local optima. However, solving model

$$\mathit{Y}=\mathit{X}\stackrel{\sim }{\mathit{B}}\mathit{H}+\stackrel{\sim }{\mathit{L}}\mathit{H}+\mathit{E}$$ (6)

for convex loss function f(·) with respect to $\stackrel{\sim }{\mathit{B}}$ and $\stackrel{\sim }{\mathit{L}}$ is easy because there exists a global minimum that can be easily approached with standard optimization tools. But this nice property is no longer valid with the rank constraints applied, because (1) the manifold has changed and the geodesics is different, so f (·) may no longer be convex; (2) the feasible solution domain may no longer be a convex set. To overcome such difficulties, an alternative formulation that produces effective low rank solutions while keeping the convexity of the problem is desired. NNR fulfills such needs.

For a matrix $\mathit{A}\in {ℝ}^n{ }^{\times m}$, its nuclear norm $\left(\text{NN}\right){\Vert \mathit{A}\Vert }_{\ast }$ is defined as the ℓ₁ norm of its singular values ${\left\{{\sigma }_h\right\}}_{h=1}^{\min \left(n,m\right)}$, or equivalently as ${\Vert \mathit{A}\Vert }_{\ast }=\text{tr}(\sqrt{\mathit{A}{\mathit{A}}^{\top }})$ thus also known as the trace norm. It can be shown for matrix completion problems, penalizing the nuclear norm of the solution matrices is equivalent to thresholding the singular values thus producing low rank results (Chen et al., 2013). Thus for GRRLF, we can similarly write down its NNR formulation as

$$(\widehat{\stackrel{\sim }{\mathit{B}}},\widehat{\stackrel{\sim }{\mathit{L}}})=\arg {\min }_{\stackrel{\sim }{\mathit{B}},\stackrel{\sim }{\mathit{L}}}{\Vert \mathit{Y}-\mathit{X}\stackrel{\sim }{\mathit{B}}\mathit{H}-\stackrel{\sim }{\mathit{L}}\mathit{H}\Vert }_F^2+{\lambda }_1{\Vert \stackrel{\sim }{\mathit{B}}\Vert }_{\ast }+{\lambda }_2{\Vert \stackrel{\sim }{\mathit{L}}\Vert }_{\ast },$$ (7)

where λ₁ and λ₂ are regularization parameters. Since ‖·‖_* does not admit an analytical expression, in this study we optimize the following alternative form

$$(\widehat{\stackrel{\sim }{\mathit{B}}},\widehat{\stackrel{\sim }{\mathit{L}}})=\arg {\min }_{\Vert \stackrel{\sim }{\mathit{B}}\Vert \le t_1,\Vert \stackrel{\sim }{\mathit{L}}\Vert \le t_2}{\Vert \mathit{Y}-\mathit{X}\stackrel{\sim }{\mathit{B}}\mathit{H}-\stackrel{\sim }{\mathit{L}}\mathit{H}\Vert }_F^2,$$ (8)

where t₁ and t₂ are the NN constraints, therefore (8) becomes a constrained optimization problem. By extending the results of Jaggi et al. (2010) we prove that (8) is equivalent to a convex optimization problem on the domain of fixed-trace positive semi-definite (PSD) matrices and present the pseudo code for estimation in Algorithm 3, where we have used notation $A_{a:b}^{c:d}$ for extracting the sub-matrix of A with row a to b and column to c to d. The details are provided in Appendix C.

Algorithm 3.

NNR-GRRLF.

2.7. An efficient kernel GWAS extension

The model developed so far focuses on the candidate approach, i.e. a set of variables of interest are grouped into the candidate predictor x and then we proceed with the model estimation. However, in modern neuroimaging-genetic studies, a genome wide association study (GWAS) is often performed, which means testing the association with the imaging phenotypes for a colossal number of candidate genes/SNPs (typically anywhere from thousands to millions). Estimating the complete model for each candidates incurs a heavy computational burden, a practice often to be avoided even in conventional univariate GWAS studies (Eu-ahsunthornwattana et al., 2014). Also an accurate yet efficient statistical testing procedure is required to assign the significance level to the observed association.

Inspired by the works of Liu et al. (2007) and Ge et al. (2012), we propose to address the challenge of an efficient GRRLF-GWAS implementation by integrating the powerful least squares kernel machines (LSKM) under the small effect assumption. For convenience we will refer to the genetic candidates as ‘genes’ in the following exposition. Specifically, the model writes

$${\mathit{y}}_{i,\mathit{v}}=h_{\mathit{v}}^{\left(k\right)}({\mathit{g}}_i^{\left(k\right)})+{\mathbf{\Phi }}_{\mathit{v}}{\mathit{B}}^{\top }{\mathit{x}}_i+{\mathit{\Gamma }}_{\mathit{v}}{\mathit{l}}_i+{\mathit{\xi }}_{i,\mathit{v}}^{\left(k\right)},$$

where k ∈ {1,…,p_g} is the index for the genes, g^(k) is the data for the kth gene, h^(k)(·) is the nonparametric function defined on the kth genetic data in some function space ${\mathscr{H}}_k$, ${\mathit{\xi }}_{i,\mathit{v}}^{\left(k\right)}$ is the gene-specific residual component and the rest follows the previous definitions, except that we have shifted our interest from x to g. We will suppress the index i, k and v for clarity whenever the context is clear. The function space $\mathscr{H}$ is determined by the semi-positive definite kernel function κ(g, g′) defined on the genetic data and we call the matrix K defined by K_ij = κ(g_i, g_j) the kernel matrix. The small effect assumption basically states that the gene induced variance var[h (g)] is small compared with the gene-specific residual variance var[ξ] and ignoring it does not significantly bias the estimation for non-genetic parameters in the model. So, instead of estimating the complete model for each gene, the non-genetic parameters are estimated once under the full null model where h^(k) equals zero for all k. Then a score test is performed using the empirical kernel matrix K^(k) and the estimated residual component $\widehat{\mathit{\xi }}$ for each gene k, for example, in the case of univariate response,

$$Q^{\left(k\right)}:=\frac{1}{2{\widehat{\sigma }}^2}{\widehat{\mathit{\xi }}}^{\top }{\mathit{K}}^{\left(k\right)}\widehat{\mathit{\xi }},$$

where $Q^{\left(k\right)}$ is the test score following a mixed chi-square distribution under the null hypothesis with some mild conditions and ${\widehat{\sigma }}^2$ is the estimated variance of the residual ξ. The mixed Chi-square is approximated by a scaled Chi-square with moment matching and the significance level is assigned based on the parametric approximation (Hua and Ghosh, 2014). Note however that the validity of using the parametric approximation hinges on its closeness to the null distribution, which should always be examined in practice. If the approximation deviates from the empirical null, the later should be used. Statistical correction procedures should be invoked after the computation of significance maps to control for the false positives. For example, Bonferroni or FDR can be used for the gene-wise correction, and the peak inference or cluster size inference for the spatial correction. Consult Appendix H for detailed discussions.

2.8. Independence between the covariate effect and the latent effect

In some applications the independence between the covariate effect and the latent effect is assumed. In the simplest case of two zero mean Gaussian variables ξ and ζ, independence is equivalent to vanishing covariance between the variables, i. e. cov[ξ, ζ] = 0. For their empirical samples $\mathit{\xi },\mathit{\zeta }\in {ℝ}^n$ and ζ, this means the asymptotic orthogonality lim_n→∞n⁻¹ξ^⊤ζ = 0. Now let us assume covariate variable $\mathit{X}\in {ℝ}^p$ and latent status $\mathit{L}\in {ℝ}^t$ are jointly zero mean Gaussian variables and their covariance matrices are of full rank. Then for their empirical sample $\mathit{X}\in {ℝ}^{n\times p}$ and $\mathit{L}\in {ℝ}^{n\times t}$, the orthogonality condition writes X^⊤L =O and L^⊤1_n = 0, where the columns of X have already been centralized. This brings (p + 1) × t linear equality constraints to L so it can be reparameterized to $\mathit{L}\prime \in {ℝ}^{(n-p-1)\times t}$, then we restrict Γ instead of L′ to some bounded manifold (for example the oblique manifold) and carry out the GM-GRRLF estimation.

For more general cases, for example non-Gaussian state variables, we propose to encourage the independence by penalizing the loss function (likelihood in most cases) with a measure of dependency ϒ(·,·) between the covariate variable X and latent status L, which generalizes the concept of “orthogonality” in the Gaussian case. More specifically, we optimize the model

$$\ell \left(\mathit{B},\mathbf{\Phi },\mathit{L},\mathbf{\Gamma }|\mathit{X},\mathit{Y}\right)+\lambda \mathit{\Upsilon }\left(\mathit{X},\mathit{L}\right),$$ (9)

where λ is the regularization parameter that balances the trade off. A good candidate for ϒ(·,·) is the square loss mutual information (Karasuyama and Sugiyama, 2012). We note however, ϒ(·,·) usually has its own parameter to be optimized, and solving (9) can be extremely expensive.

3. Results

3.1. Synthetic examples

For clarity, we use a 1-D synthetic example to illustrate the proposed method.² The synthetic data are generated as follows: N_knot = 10 knots and N_vox = 100 artificial voxels are placed uniformly on interval Ω = [0, 1] and kernel bandwidth set to σ = 0.1, we set p = 10, q = 1, d = 2, t = 2, B = [I₂; O] (so only the first two dimensions of the covariate are contributing), $X\sim \mathcal{N}(\mathbf{0},{\mathit{I}}_p)$, $\Phi \sim \mathcal{N}(\mathbf{0},{\mathit{I}}_{q\times d\times N_{\text{knot}}})$, $\Gamma \sim \mathcal{N}(\mathbf{0},{\mathit{I}}_{q\times t\times N_{\text{knot}}})$, $L\sim \mathcal{N}(\mathbf{0},{\mathit{I}}_t)$ and ${\mathit{\xi }}_{\mathit{v}}\sim \mathcal{N}(\mathbf{0},{\mathit{I}}_q)$ independent from other voxels unless otherwise specified. For each simulation n=100 samples are drawn. We use nonparametric permutations to obtain the p-values for the sensitivity studies. Specifically, the sum of squared error (sse) is used as the test statistic and the empirical p-value is determined by p_emp = max(#{sse_b ≤ sse₀}, 1)/m_perm where #{·} denotes the counting measure, m_perm the number of permutation runs, b = 1,…,m_perm the permutation index, ${\text{sse}}_b={\displaystyle {\sum }_{i,\mathit{v}}{\Vert {\widehat{e}}_{i,\mathit{v}}^b\Vert }^2}$, ${\widehat{e}}_{i,\mathit{v}}^b$ denote the residual estimated at voxel v for sample i with the bth permuted X and b=0 refers to the original X.

We first experiment with the NNR implementation of GRRLF. We set the candidate parameter set for nuclear norm constraints t_i to {2⁰, 2¹, …,2¹⁵}, and we stop the iteration when either of the following criteria is satisfied:

1. the number of iterations reaches k=3000;

2. the improvement of the current iteration is less than 10⁻⁵ compared with the average of previous 10 iterations.

The performance is evaluated by the relative mean square error (RMSE) defined by

$$\text{RMSE}=\frac{{\Vert \mathit{A}-\widehat{\mathit{A}}\Vert }_F}{{\Vert \mathit{A}\Vert }_F}.$$

Fig. 3(a and b) respectively visualizes the optimization procedure and the regularization path of the solution matrices’ nuclear norm, and only the results for parameter pairs (t₁, t₂) satisfying t₁ = t₂ are shown. For tight constraints (with small t_i), the solutions converge rapidly and the optimal solutions are achieved on the boundary of the feasible domain. Slow convergence is observed for larger t_i, and as the constraints are relaxed the solutions move away from the boundary.

Fig. 3.

NNR-GRRLF model estimation with Jaggi–Hazan algorithm. (a) Convergence curve of the normalized mean square error for different constraints. (b) Nuclear norm constraint vs. nuclear norm of the solution matrices, blue solid line for the covariate coefficient matrix, green solid line for the latent coefficient matrix and red dash line is the nuclear norm upper bound with respect to the constraint. Here we have fixed t₁ = t₂. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this paper.)

Fig. 4 gives an example of the regularization paths of the leading singular values of the NNR-GRRLF solution matrices. To facilitate visualization we have used the normalized SVs defined by ${\stackrel{\sim }{\sigma }}_h={\sigma }_h/{\displaystyle {\sum }_{h\prime }{\sigma }_{h^{\prime }}}$, where {σ_h} are the original SVs. Under the nuclear norm constraints, the solution matrices show sparsity with respect to their SVs. We call the number of SVs that are bounded away from zero as the “effective rank” (ER) of the matrix; as the nuclear norm constraints are relaxed, ER grows.

Fig. 4.

Regularization path for the (normalized) leading singular values with respect to the nuclear norm constraint. (a) Regularization path for t₁ with t₂ fixed. (b) Regularization path for t₂ with t₁ fixed. The X-axis spans the six leading singular values and the Y-axis indicates their (normalized) magnitude; different regularization parameters are color coded. It can be seen the effective rank of the solution grows with the nuclear norm constraint. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this paper.)

Fig. 5 gives an example of a GCV RMSE heatmap for parameter selection. The RMSE on the training sample drops as the NN constraints are relaxed, as more flexibilities are allowed for the model. Interestingly for a wide range of parameter settings the RMSE on the validation sample is smaller than that on the training sample, which seems contradictory for CV procedures. This is because with our modified CV procedure,

1. nuclear norm of the latent coefficient matrix is no longer bounded;

2. the latent response field $\stackrel{\sim }{\mathbf{\Gamma }}=\mathbf{\Gamma }\mathit{H}$ is well approximated, although $\stackrel{\sim }{\mathit{L}}=\mathit{L}\mathbf{\Gamma }$ is not because of the nuclear norm constraint.

Fig. 5.

Residual heatmaps for nuclear norm regularization parameter selection. (a) Relative mean square error for the training sample. (b) Relative mean square error for the validation sample. The Y-axis corresponds to the covariate coefficient constraint t₁ and the X-axis corresponds to the latent coefficient constraint t₂. The green arrow points at the parameter pair with minimal validation error. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this paper.)

In practice a relatively large region of the parameter space can show similar good generalization performance (for example see Fig. 5(b)). This is because the framework is robust to a small level of over relaxation, and the latent part of the model can compensate for the modeling error from the covariate part, to some extent. In the spirit of Occam’s razor, we want to keep the simplest model. This means that the model with the tightest constraints (smallest t_i, with the latent constraint t₂ is prioritized) should be preferred when the validation RMSE is tied.

For GM-GRRLF, we compare AIC, BIC and RRR-PCA for automatic model selection. We perform experiments on the selection of coefficient rank d and latent dimension t. All combinations in {(d′, t′)|d′, t′ = 1,…,4} are tested with all experiments repeated for m=100 times and the results are presented in Fig. 6. In Fig. 6(a), the mean raw score and mean rank of AIC and BIC are shown. AIC gives more confusing results, as it is difficult to choose between (1, 3) and (2, 2). In such ties we opt for the model with the larger coefficient rank because in the absence of predictive information, the latent factor part of the model will try to interpret the signal as a latent contribution. AIC also tends to favor models that are larger than the original model. BIC seems to be a better choice as it successfully identifies the true structural dimensionality at its minimum value. As can be seen in Fig. 6(b), RRR-PCA also performs well in that it successfully identifies t and narrows down the choice of d to 2 or 3. Taking into account that RRR-PCA is much more computationally efficient than *IC based model selection methods, it is therefore favorable in neuroimaging studies. One can also use the GCV procedure to identify the appropriate model order.

Fig. 6.

(a) Mean raw and rank map for AIC and BIC. (b) Box plot of eigenvalues from RRR-PCA. *IC procedures identifies model order with lowest score as optimal while RRR-PCA bases its decision on the jumping point of eigenvalues. AIC slightly over estimates the model order and BIC makes the right decision, RRR-PCA gives an fair estimate with much less cost compared with *IC methods.

We now compare the two different implementations of GRRLF (GM and NNR) with voxel-wise least-square regression (LSR) and whole field reduced rank regression (RRR). LSR corresponds to the massive univariate approaches most commonly used in neuroimaging studies, and RRR corresponds to those methods that only consider spatial correlations. For GM-GRRLF and RRR the regression coefficient rank, latent factor number and kernel bandwidth are set to the ground truth. Fig. 7 presents an illustrative example: the upper panel gives the smooth response curves corresponding to the effective covariate space and latent space while the lower left figure visualizes three noisy field realizations. In Fig. 7(d), the estimated covariate response curves for an unseen sample using different methods are shown. As can be seen, LSR gave the most noisy estimate as it disregards all spatial information while RRR gave a much smoother estimate by considering the covariance structure. However both of them were susceptible to the influence of latent responses, which drove their estimates away from the true response. Overall GRRLF methods showed more robustness against the latent influences, and GM-GRRLF gives the best result. The inferior performance of NNR-GRRLF compared with GM-GRRLF can be caused by (1) the regularization parameter setting needs further refining; (2) part of the covariate signal may have been misinterpreted as the latent signal.

Fig. 7.

A 1-D example of GRRLF model. (a) True covariate response fields $\stackrel{\sim }{\mathbf{\Phi }}$. (b) True latent factor response fields $\stackrel{\sim }{\mathbf{\Gamma }}$. (c) Observed responses for three randomly selected samples. (d) Estimated response using GM-GRRLF, NNR-GRRLF, RRR and LSR together with the ground truth expected response for an unseen sample. (dotted: ground truth, red: GM-GRRLF, brown: NNR-GRRLF, green: RRR, purple: LSR) This demonstrates GRRLF is robust to the latent influences while common solutions fail. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this paper.)

In Table 1 we present the computational cost for the above methods. We notice that despite NNR having a much more elegant formulation, it is computationally much more costly than the other alternatives (it takes roughly six CPU hours while all others take less than 1.5 s). This is because there is no direct correspondence between the rank and nuclear norm, thus one has to traverse the parameter space to identify the optimal parameter setting, via the costly GCV procedure. Smarter parameter space traversing strategies may significantly cut the cost, but it still takes tens of seconds to compute the generalization error for a fixed parameter pair—still more expensive than other methods.³ The redundant parameterization of NNR-GRRLF also drags its efficiency and makes it less scalable than GM-GRRLF. We note that there are a few nuclear norm regularization optimization algorithms that are more efficient compared with the Jaggi–Hazan algorithm (Avron et al., 2012, Zhang et al., 2012b; Mishra et al., 2013; Chen et al., 2013; Hsieh and Olsen, 2014); however, these algorithms are mostly specific to certain problems and thus cannot be easily extended to solve GRRLF. We therefore leave the topic of more efficient NNR-GRRLF optimization for future research, and we present some discussions on a few possible directions in Appendix D. In the following experiments, we will exclude NNR-GRRLF due to its excessive computational burden.

Table 1.

Computation time for different methods.

Method	LSR	RRR	GM-GRRLF	NNR-GRRLF
Time	< 0.01 s	< 0.01 s	1.20 s	2.09 × 104 2 s

To better see how this robustness can improve the estimate and in turn boost sensitivity, we varied the intensity of the covariate response and the latent response. For the covariate response experiment, we benchmarked the performance under the null, low SNR and high SNR cases, where B is scaled by 0, 0.1 and 1 respectively while fixing other settings. For the latent response experiment, we similarly test the none, weak and strong latent influence via scaling V by 0, 0.5 and 2, which accounts for 0%, 16.7% and 73.3% of the total variance respectively. All experiments were repeated for m=500 times to ensure stability and for the sensitivity study we ran m_perm = 100 permutations to empirically estimate p-values, further details are provided in the Appendix. The results are summarized in Fig. 8. Fig. 8(a–d) gives the p-value distributions from the sensitivity experiment. The distribution of p-values from all three methods fall within expected region for the null case in Fig. 8(a), confirming the validity of the permutation procedure. Fig. 8(c and d) show that GRRLF significantly improves the sensitivity over RRR and LSR. Fig. 8(b) provides a box plot of the squared difference between the estimated response and expected response on a log scale for different latent response intensities. In all cases GRRLF gives the best estimate followed by RRR. It is interesting to observe that while RRR gives a better parameter estimate compared with LSR, the latter appears to be more sensitive in our experiments.

Fig. 8.

(a) log₁₀ P–P plot of p-values under null model, shaded region corresponds to the 95% confidence interval under null. (b) Box plot of estimation error with different latent intensity. (c) Histogram of p-values and box plot of estimation error for low SNR case. (d) Histogram of p-values and box plot of estimation error for high SNR case. GRRLF demonstrates improved sensitivity and reduced estimation error compared to its commonly used alternatives under various experimental setups.

3.2. Real-world data

736 Caucasian subjects with both genetic and tensor based morphometry (TBM) data from ADNI1 (http://adni.loni.usc.edu) are used in the current analysis. Similar to previous investigations (Stein et al., 2010a; Hibar et al., 2011; Ge et al., 2012), only age and gender are included as covariates. We use LS-PCA to estimate the dimensionality of the latent space and then alternate between least square and PCA to decompose the image Y into the covariate component C, the latent component L and the residual component R, i.e. Y = C + L + R. We call J = R + L the joint component. We have chosen the LS-PCA implementation to demonstrate because this is the simplest form of GRRLF, computationally efficient and there is no parameter to be tuned, which makes it more likely to be used in practice compared with other more sophisticated implementations. Then we apply the LSKM to estimate the gene-wise genetic effect on J, L and R respectively for each voxel. A total of 26, 664 genes and 29, 479 voxels enter the study. We thresholded the significance image with threshold p < 10⁻³ and use the largest cluster size (in RESEL units) as the test statistic. All p-values, including those of the voxel-level LSKM test score and the largest cluster size statistics, were determined via nonparametric permutations. As a post hoc validation step, we searched the Genevisible database (Nebion, 2014; Hruz et al., 2008) for the top genes identified in each category to examine whether they are highly expressed in neuron-related tissues (HENT).⁴ Consult Appendix F for more details on the study sample, data preprocessing and statistical analyses. The latent factor identification results are visualized in Fig. 9 and the GWAS results are tabulated in Table 2.

Fig. 9.

(a) Eigenvalues from PCA with or without the covariates. (b) Spatial maps of the first three latent factors. First three eigencomponents encode significantly more variances compares with other eigencomponents thus being identified as the latent components.

Table 2.

GWAS results.

Chr.	Gene name	SNPs	Cluster size (in RESEL)	puc	qfdr	Nearby genes	HENT	Related functions
Joint component J
1	PGM1	16	1.89	7.69E−05	7.66E−01		NO	AD
11	CCDC34	8	1.72	1.20E−04	7.66E−01	BDNF	NO	Psychiatric disorder risk factora
14	CTD-2555O16.1	3	1.59	1.56E−04	7.66E−01		N/A
14	TEX21P	5	1.51	2.16E−04	7.66E−01		N/A
6	EFHC1	16	1.5	2.21E−04	7.66E−01		NO	Neuroblasts migration, epilepsy
12	RP11-421F16.3	2	1.42	3.45E−04	7.66E−01	TM7SF3	NO
2	CHRNA1	3	1.42	3.49E−04	7.66E−01		NO	Autism
3	MARK2P14	2	1.42	3.51E−04	7.66E−01		NO
16	RP11-488I20.8	1	1.37	4.31E−04	7.66E−01	LINC01566	YES
15	ISLR	1	1.36	4.31E−04	7.66E−01		NO
Latent component L
12	GRIN2B	179	3.99	7.50E−06	2.00E−01		YES	Learning, memory, AD, etc.
7	AC074389.7	1	3.09	2.63E−05	3.50E−01	ELFN1	NO	Seizure, ADHD
8	HPYR1	1	2.26	4.50E−05	4.00E−01		YES
7	ELFN1	6	1.62	6.75E−05	4.40E−01		NO	Seizure, ADHD
12	RSRC2	4	1.38	8.25E−05	4.40E−01		NO
2	CHRNA1	3	1.06	1.14E−04	4.93E−01		NO	Autism
17	TAC4	2	0.85	1.29E−04	4.93E−01		NO
11	RP11-872D17.8	9	0.4	2.63E−04	7.81E−01	PRG2/3,SLC43A3	NO
1	EMC1	8	0.35	4.63E−04	7.81E−01		YES
17	AP2B1	27	0.35	4.63E−04	7.81E−01		NO	Schizophrenia
Residual component R
12	RP5-1096D14.6	2	2.51	9.38E−06	1.25E−01	CACNA1C	YES	Psychiatric disease risk factora
12	DCP1B	12	2.48	9.38E−06	1.25E−01	CACNA1C	NO	Psychiatric disease risk factora
15	PYGO1	10	1.79	8.06E−05	5.36E−01		NO	Wnt signaling pathway, AD
11	DOC2GP	1	1.72	1.05E−04	5.36E−01		N/A
6	GLYATL3	7	1.71	1.05E−04	5.36E−01		N/A
2	MREG	36	1.63	1.41E−04	5.36E−01		NO
4	ZGRF1	9	1.62	1.41E−04	5.36E−01		NO
12	FAM216A	2	1.58	1.73E−04	5.75E−01		YES	Neurodegenerative disease
10	CEP164P1	16	1.52	2.44E−04	7.22E−01		N/A
10	RP11-285G1.15	22	1.47	3.13E−04	8.28E−01	RSU1P2	YES	Substance addictionb

GWAS results, showing the distinct findings between joint, latent and residual component. “Nearby Genes”, those genes that lie in close vicinity (within a few hundred KB) of the primary gene that has showed significant association, in some cases the genes are co-located so the nearby genes can also be regarded as the primary gene; “HENT”, the primary gene (or the nearby gene if such information is not available for the primary gene) is highly expressed in neuron-related tissues (see main text for detailed definition).

^aThe function is related to the nearby gene(s).

^bthe function is related to the functioning gene. We have highlighted genes that are statistical significant after multiple comparison and underlined genes of particular interest.

Fig. 9(a) indicates that the first three eigencomponents are the dominant parts of J, and thus we identify them as the latent components, i.e. t=3. Fig. 9(b) gives the spatial maps of the decomposed latent components, and interestingly they seem to respectively correspond to white matter, ventricles and gray matter. For the GWAS analysis, smaller p-values are obtained for the top hits in factorized analyses. While no gene from the above three analyses survived stringent Bonferroni correction, three of the genes, all from the factorized GWAS analyses, survived the FDR significance level q=0.2 suggested by Efron (2010). More than half of the top entries identified in the factorized analyses have been reported to be relevant in neuronal researches, indicating that the results from the factorized analyses are biologically relevant.

The top hit in Table 2 is CACNA1C (overlapping with DCP1B), an L-type voltage gated calcium channel subunit gene well known for its psychiatric disease susceptibility (PGC et al., 2013). The significance map between CACNA1C and the TBM map is overlaid on the population template in Fig. 10, and it can be seen that the voxels susceptible to this influence are clustered within the orbitofrontal cortex, and overlapping gyrus rectus and olfactory regions, which include the caudal orbitofrontal cortex Brodmann area 13 (Öngür et al., 2003). We further conducted SNP-wise association for all the imputed SNPs within 500 KB of CACNA1C’s coding region. Only SNPs that have a minor allele frequency over 0.1 are included. The result is presented in Fig. 11. The peak association is achieved at SNP rs2470446 (maf=0.47), which is imputed. For the genotyped SNPs, rs2240610 (maf=0.49) yields the largest association. No association is observed between rs2240610 and the Alzheimer or dementia diagnostic state of the subjects (all p > 0.05). In the following we use DCP1B as a surrogate for CACNA1C as the majority of CACNA1C SNPs lie outside the genetic hot spot. We extract the first eigen-component of the largest voxel cluster associated with DCP1B (CACNA1C) and plot them against the genotype of SNP rs2240610. Subjects with genotype ‘AA’ have significantly different responses compared with the other two genotypes (t-test, p = 8.55 × 10⁻¹⁰), which have similar responses compared with each other (t-test, p=0.50). This result suggests that the recessive model is appropriate for the genetic effect. A similar distribution is observed for the mean response of the cluster.

Fig. 10.

Significance map of CACNA1C, color coded with −log₁₀(p). Voxels susceptible to the genetic influence from CACNA1C are clustered within the orbitofrontal cortex. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this paper.)

Fig. 11.

Regional SNP-wise cluster size analyses for CACNA1C. Inset: distribution of first principal coordinate of the voxels within the largest cluster according to the genotype of SNP rs2240610.

4. Discussion

In this paper, we propose a general framework of reduced rank latent factor regression for neuroimaging applications. In summary, we (1) reduce the variance of the covariate effect estimate by simultaneously (a) projecting the predictors onto a lower dimensional effective subspace and (b) conditioning on the latent components that are dynamically estimated; (2) we use additional constraints such as smoothness of the response field to regularize the solution; (3) we recast the problem into a sequence of block-manifold optimization problems and effectively solve them by Riemannian manifold optimization; (4) we present an alternative nuclear norm regularization based formulation of GRRLF with which the global optimum can be achieved; (5) we present a least squares kernel machines based procedure for brain-wide GWAS conditioning on the latent factors.

Our method exploits the structured nature of the imaging data to better factorize the signal observed. The application of our method to a real-world dataset suggests that this factorization improves upon the sensitivity over existing brain-wide GWAS methods and gives biologically plausible results. The most significant gene identified, CACNA1C, is a widely recognized multi-spectrum psychiatric risk factor and has been intensively studied. Our result lends further evidence for the pleiotropic role it plays. Most of the top genes that we identified are found to be either relevant to psychiatric diseases or highly expressed in neuronal tissues, lending plausibility to our framework.

4.1. Methodology assessment

Our method reports two genes surviving the FDR threshold at q=0.2 while previous work has not fund any (Hibar et al., 2011). We note our imaging-genetic solution is closely related to Ge et al. (2012) where they use analytical approximation of LSKM statistics, extreme value theory (EVT) and random field theory (RFT) to make inferences. Different from Ge et al. (2012), our null simulations fail to support the use of these analytical approximations, so only permutation-based results are reported in the current study. None of the top genes identified from the residual component study have been reported in Ge et al. (2012) and Hibar et al. (2011) while there are a few overlaps for the genes from the joint and latent component study. This suggests that conditioning for the hidden variables might be important to reveal certain otherwise buried signals.

While GRRLF can be implemented in various forms, the key idea underlying our framework is three-fold: (1) using the structure of brain imaging to estimate the latent components; (2) conditioning on the latent component to reduce the variance of covariate effect of interest; (3) estimating the effective dimensionality of the covariate further reduces variance. An interesting comparison can be made with the linear mixed model (LMM) which has recently gained popularity in GWAS studies (Eu-ahsunthornwattana et al., 2014), where a kinship matrix, estimated from either pedigree or genome sequences, is used to structure the covariance matrix for genetic random effects. LMM deals with univariate response so it can only look into the kinship matrix for structured unexplained variance, while for neuroimaging data the richness of the structural information allows further decomposition of the observed signals. Current large scale multi-center neuroimaging collaborations often use comprehensive survey to capture as much population variance as possible and researchers are compelled to include more predictors in their model to factor out the variances in the data. However, the price paid is the degrees of freedom (DOF) and therefore more uncertainty in estimating the effect of interest. Enforcing proper regularizations, in our case constraining the effective dimensions of the predictors, serves to balance the trade-off between the explained variance and DOF.

The three sets of results just presented show that each decomposition scheme has its advantages and that they are complementary to each other. The residual component approach is more sensitive to weak signals that would otherwise be dominated by a large latent component effect. The latent component approach has the advantage that it acts to reduce noise, but may not detect local effects. The joint component approach is useful if there are contributions of both global and local effects. We therefore suggest that the results with all three approaches should be compared with each dataset analyzed.

4.2. Biological significance

CACNA1C is known as one of the risk genes for a wide spectrum of psychiatric disorders including bipolar disease, schizophrenia, and major depression and autism. Its association with susceptibility to psychiatric disorders has been consistently confirmed by several large-scale genome-wide association studies (PGC et al., 2013) thus making it one of the most replicable results in psychiatric genetics. A series of human brain imaging and behavioral studies have shown morphological and functional alterations in individuals carrying the CACNA1C risk allele on a macroscopic level (Bigos et al., 2010; Franke et al., 2010; Zhang et al., 2012a; Tesli et al., 2013; Erk et al., 2014), and it has been experimentally confirmed that the risk variant will also affect cellular level electrophysiology using induced human neuron cell lines (Yoshimizu et al., 2014). An Australian twin study has previously reported CACNA1C to be significantly associated with white matter integrity and function as a hub in the expression network belonging to the enriched gene ontology category “synapse” (Chiang et al., 2012). Previous studies on the ADNI dataset have also reported significant genetic interactions for CACNA1C using Positron Emission Tomography (PET) imaging (Koran et al., 2014) and LASSO screening with candidate phenotypes (Yang et al., 2015), which all involve certain ‘conditioning’ for the contribution from CACNA1C to be detected. Animal AD models have confirmed several results from human studies (Hopp et al., 2014) and related pathways have been identified as a therapeutic target (Liang and Wei, 2015) for AD. Interestingly, a recent multi-site large-scale voxel level functional connectivity study, which included 939 subjects, has revealed that functional connectivity patterns in the orbitofrontal cortex region are significantly altered in depression patients (Cheng et al., 2016). Also gray matter volume reductions are reported in the same area in depression patients (Ballmaier et al., 2014). The results from these studies are consistent with the assumption that CACNA1C affects depression susceptibility through the orbitofrontal region, a hypothesis to be tested in future studies.

ELFN1 has been implicated to be associated with seizures and ADHD in both human clinical samples and animal models (Tomioka et al., 2014; Dolan and Mitchell, 2013). The expression of ELFN1 localizes mostly to excitatory postsynaptic sites (Sylwestrak and Ghosh, 2012) and recent studies show that the ELFN1 gene specifically controls short-term plasticity, which denotes changes in synaptic strength that last up to tens of seconds, at some synapse types (Blackman et al., 2013). Data from the Allen’s Brain Atlas (Hawrylycz et al., 2012); http://human.brain-map.org) also show that ELFN1 is highly expressed in the cortical regions (Fig. 12(a)), consistent with the ELFN1 significance map we obtained from the ADNI dataset (Fig. 12(b)).

Fig. 12.

(a) ELFN1 expression profile from Allen Brain Atlas. (b) Significance map of ELFN1, color coded with −log₁₀(p). Cortical regions show elevated ELFN1 expression and they are also under the genetic influence of the same gene. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this paper.)

GRIN2B encodes the N-methyl-D-aspartate (NMDA) glutamate receptor NR2B subunit and is well known to be involved in learning and memory (Tang et al., 1999), structural plasticity of the brain (Lamprecht and LeDoux, 2004) and excitotoxic cell death (Parsons et al., 2007), and has age-dependent prevalence in the synapse (Yashiro and Philpot, 2008). Therefore, the relationship between NR2B subunit gene GRIN2B variants and AD has attracted a large amount of attention and interest. Many studies have confirmed that the NR2B subunit is down-regulated significantly in susceptible regions of AD brains (Bi and Sze, 2002; Farber et al., 1997; Hynd et al., 2004). Actually GRIN2B is already a therapeutic target in Alzheimer’s disease and has also been indicated by several studies in the literature (Jiang and Jia, 2009; Stein et al., 2010b).

The gene PGM1 encodes the protein Phosphoglucomutase-1. The level of this enzyme was found to be significantly altered in the hippocampus of patients who suffer from AD compared with control hippocampus using two-dimensional gel electrophoresis and mass spectrometry techniques (Boyd-Kimball et al., 2007). Down-regulation of this gene might have an effect on memory and cognitive functions in human brains.

The pygopus gene of Drosophila encodes an essential component of the Armadillo (β-catenin) transcription factor complex of canonical wnt signaling (Schwab et al., 2007). The wnt signaling pathway has been implicated in a wide spectrum of physiological processes during the development of the central nervous system and assumes some roles in mature synapses that could cause cognitive deficiencies (Oliva et al., 2013). A recent study has pointed out that aberrant wnt signaling pathway function is associated with medial temporal lobe structures of Alzheimer’s disease and PYGO1 is differentially expressed in an AD population using post-mortem brain samples (Riise et al., 2015).

FAM216A has been reported to be a risk gene for neurodegenerative diseases from an integrated multi-cohort transcriptional meta-analysis study using 1, 270 post-mortem central nervous system tissue samples (Li et al., 2014). AP2B1 is reported to be differentially expressed in a rat model for schizophrenia (Zhou et al., 2010). CHRNA family genes have been implicated as susceptible targets in autism spectrum disorders (Lee et al., 2012). EFHC1 mutations are known to cause juvenile myoclonic epilepsy (Suzuki et al., 2004; Stogmann et al., 2006; Noebels et al., 2012). CCDC34 has been previously reported to be associated with ADHD and autism (Shinawi et al., 2011) and it locates next to the gene BDNF which is known to be a risk factor for psychiatric disorders (Petryshen et al., 2010). RSU1P2 is the pseudogene of RSU1, i.e. a DNA sequence that is similar to the functioning gene RSU1 but nonetheless unable to produce functional protein products and only assuming regulatory roles. It is reported that RSU1 has a conserved role regulating reward related phenotypes such as ethanol consumption, ranging from Drosophila to humans (Ojelade et al., 2015).

4.3. Future directions

The current study provides for advances in a number of directions. On the biological side, a few interesting assumptions have been made combining the results from the ADNI dataset and existing studies. These assumptions can be checked on the data from other phases of the ADNI project, for example ADNI GO and ADNI2, or other population samples. The proposed method can also be applied to longitudinal recordings from the ADNI dataset, where brain-wide genome-wide imaging-genetic investigations are rare due to the fact that the observed phenotype is a function of time, i.e. a one-dimensional tensor, thus unsuited for most neuroimaging-genetic solutions.

On the methodological side, many aspects can be further improved in the future. For example, we do not deal with the identifiability issue of the model to maximize the generality of the formulation. More stringent constraints are expected to theoretically ensure the identifiability under certain assumptions, which is left for future investigations. Pragmatically it will be interesting to compare the empirical performance of GRRLF with latent factors estimated from different models, say ICA. More computationally efficient estimation procedures, and more sensitive yet less expensive statistical tests are also important topics for future exploration.

Appendix A. Connection to other models

Different choices of loss function ℓ, covariate dimension d and latent dimension t of (4) give different commonly used statistical models. In Table A1 ‖·‖_Fro denotes the Frobenius norm, η(·), τ (·)and α(·) are the functions charactering the exponential family distributions (McCullagh and Nelder, 1989), ϒ(·) certain dependency measure (Suzuki and Sugiyama, 2013; Gretton et al., 2005), ϑ(·) certain independence measure (Bach and Jordan, 2003; Hyvärinen et al., 2004; Smith et al., 2012), Ψ(·) the basis functions of some functional space (Ramsay and Silverman, 2005) and {λ_i} the regularization parameters (Zou and Hastie, 2005).

Table A1.

Connection to other models.

ℓ	d	t	Statistical model
$\Vert Y-XB\Phi {\Vert }_{\text{Fro}}^2$	<p	0	Reduced rank regression
−ϒ (Y, XBΦ)	<p	0	Supervised dimension reduction
−η(Φ)·τ (Y, X) + α (Φ)	p	0	GLM (exponential family)
$\Vert Y-X\Phi {\Vert }_{\text{Fro}}^2+{\lambda }_1\Vert \Phi {\Vert }_1+{\lambda }_2\Vert \Phi {\Vert }_2^2$	p	0	Lasso/elastic net
$\Vert Y-\Psi (X)B\Phi {\Vert }_{\text{Fro}}^2$	<|Ψ|	0	Functional PCA
$\Vert Y-L\Gamma {\Vert }_{\text{Fro}}^2$	0	Any	Generalized PCA
−ϑ(YΓ⊤)	0	Any	ICA

Appendix B. Reduced rank regression

Here we detail the implementation of reduced rank regression (Izenman, 1975). Assume that X and Y have been demeaned. For Y ∈ ℝ^q, X ∈ ℝ^p, rank-d reduced rank regression intends to find the A ∈ ℝ^q×d, B ∈ ℝ^d×p that minimizes $\mathbb{E}[\Vert Y-\mathit{ABX}{\Vert }_2^2]$. Denote Σ_XX the covariance matrix of X and Σ_YX the cross-covariance matrix between Y and X. Denote ${\displaystyle \mathit{\sum }={\displaystyle {\mathit{\sum }}_{YX}{\displaystyle {\mathit{\sum }}_{XX}^{-1}{\displaystyle {\mathit{\sum }}_{YX}^{\top } }}}}$ and Σ = VΛV^⊤ its eigen-decomposition, where V is a unitary matrix and Λ a diagonal matrix with non-negative entries in descending order. Denote V_d ∈ ℝ^q×d the first d columns of V, then the solution of reduced-rank regression with rank d is

$$A^{\ast }=V_d,B^{\ast }=V_d^{\top }{\displaystyle {\mathit{\sum }}_{YX}{\displaystyle {\mathit{\sum }}_{XX}^{-1}.}}$$

Appendix C. Nuclear norm regularized GRRLF

First we prove solving (8) is a convex optimization problem.

Lemma 1

(Jaggi et al., 2010 Lemma 1). For any nonzero matrix X ∈ ℝ^n×m and t ∈ ℝ, $\Vert \mathit{X}{\Vert }_{\ast }\le \frac{t}{2}$ iff there exists $\mathit{A}\in {\mathbb{S}}_{\text{PSD}}^n$ and $\mathit{B}\in {\mathbb{S}}_{\text{PSD}}^n$, such that $\left(\begin{array}{cc}\mathit{A}& \mathit{X}\\ {\mathit{X}}^{\top }& \mathit{B}\end{array}\right)\succcurlyeq 0$ and tr(A) + tr(B) = t.

Lemma 2

(Laurent and Vallentin, 2012). ${\mathbb{S}}_{\text{PSD}}^n$ is a cone.

Lemma 3

Define ${\mathbb{S}}_{\text{PSD}}^n(t):=\{\mathit{A}\in {\mathbb{S}}_{\text{PSD}}^n|\text{tr}(\mathit{A})=t\}$, then ${\mathbb{S}}_{\text{PSD}}^n(t)$ is a convex set.

Proof

For A, $\mathit{B}\in {\mathbb{S}}_{\text{PSD}}^n(t)$ and ω ∈ [0, 1], for C = ωA + (1 − ω)B, we have $\text{tr}(\mathit{C})=\text{tr}(\omega \mathit{A}+(1-\omega )\mathit{B})=\omega \text{tr}(\mathit{A})+(1-\omega )\text{tr}(\mathit{B})=t,$

therefore $\mathit{C}\in {\mathbb{S}}_{\text{PSD}}^n(t)$, which concludes the proof.□

Theorem 1

f(X₁,…,X_K) is a convex function, where ${\mathit{X}}_k\in {ℝ}^{n_k\times m_k}$, k ∈ [K]. let {t_k|k = [K]} be a set of positive real numbers, then the nuclear norm regularized problem

$$\left({\mathit{X}}_1^{\ast },\dots ,{\mathit{X}}_K^{\ast }\right)=\underset{\Vert {\mathit{X}}_k{\Vert }_{\ast }\le t_k/2}{\text{argmin}}f\left({\mathit{X}}_1,\dots ,{\mathit{X}}_K\right)$$ (C.1)

is equivalent to the convex problem

$$({\mathit{Z}}_1^{\ast },\dots ,{\mathit{Z}}_K^{\ast })=\underset{{\mathit{Z}}_k\in {\mathbb{S}}_{\text{PSD}}^{n_k+m_k}(t_k)}{\text{argmin}}\stackrel{\sim }{f}({\mathit{Z}}_1,\dots ,{\mathit{Z}}_K),$$ (C.2)

where ${\mathit{Z}}_k=\left(\begin{array}{cc}\mathit{A}& \mathit{X}\\ {\mathit{X}}^{\top }& \mathit{B}\end{array}\right)$ and $\stackrel{\sim }{f}({\mathit{Z}}_1,\dots ,{\mathit{Z}}_K):=f({\mathit{X}}_1,\dots ,{\mathit{X}}_K)$.

Proof

Since the Cartesian product of convex sets is also a convex set, so by Lemma 3 we know ${\displaystyle {\prod }_{\otimes }{\mathbb{S}}_{\text{PSD}}^{n_k\times m_k}(t_k)}$ is a convex set. And the convexity of $\stackrel{\sim }{f}$ is inherited from f. The proof is completed by applying Lemma 1 to obtain the bound of ‖X_k‖_*, k ∈ [K].□Setting K=2 in Theorem 1 proves the convexity of (8).

Now we elaborate how to efficiently compute the solutions given the NN constraint t₁ and t₂. First we extend $\stackrel{\sim }{\mathit{B}}$ and $\stackrel{\sim }{\mathit{L}}$ to ${\mathit{Z}}_B=\left(\begin{array}{cc}\mathit{A}& \stackrel{\sim }{\mathit{B}}\\ {\stackrel{\sim }{\mathit{B}}}^{\top }& \mathit{C}\end{array}\right)$ and ${\mathit{Z}}_L=\left(\begin{array}{cc}\mathit{C}& \stackrel{\sim }{\mathit{L}}\\ {\stackrel{\sim }{\mathit{L}}}^{\top }& \mathit{D}\end{array}\right)$, where ${\mathit{Z}}_B\in {\mathbb{S}}_{\text{PSD}}^{(p+m)}(t_1)$, ${\mathit{Z}}_L\in {\mathbb{S}}_{\text{PSD}}^{n+m}(t_2)$. Let ${\stackrel{\sim }{\mathit{Z}}}_B=t_1^{-1}{\mathit{Z}}_B$, ${\stackrel{\sim }{\mathit{Z}}}_L=t_2^{-1}{\mathit{Z}}_L$, X_t = t₁X, ${\mathit{B}}_t=t_1^{-1}\stackrel{\sim }{\mathit{B}}$, ${\mathit{L}}_t=t_2^{-1}\stackrel{\sim }{\mathit{L}}$, define $\stackrel{\sim }{f}({\stackrel{\sim }{\mathit{Z}}}_B,{\stackrel{\sim }{\mathit{Z}}}_L)=f_t({\mathit{B}}_t,{\mathit{L}}_t,t_2)=\Vert \mathit{Y}-{\mathit{X}}_t{\mathit{B}}_t\mathit{H}-t_2{\mathit{L}}_t\mathit{H}{\Vert }_F^2=f(\mathit{B},\mathit{L}).$

Notice $\text{tr}({\stackrel{\sim }{\mathit{Z}}}_B)=\text{tr}({\stackrel{\sim }{\mathit{Z}}}_L)=1$, so we can use the Hanzan algorithm (Hazan, 2008) to optimize $\stackrel{\sim }{f}({\stackrel{\sim }{\mathit{Z}}}_B,{\stackrel{\sim }{\mathit{Z}}}_L)$ over ${\mathbb{S}}_{BL}:={\mathbb{S}}_{\text{PSD}}^{p+m}(1)\times {\mathbb{S}}_{\text{PSD}}^{n+m}(1)$. Let R_t = Y − XB_tH − t₂L_tH, we have ${\nabla }_{{\mathit{B}}_t}{f}_t=2{\mathit{X}}^{{ }^{\top }}{\mathit{R}}_t{\mathit{H}}^{{ }^{\top }}$, ${\nabla }_{{\mathit{L}}_t}{f}_t=2t_2{\mathit{R}}_t{\mathit{H}}^{{ }^{\top }}$. Let ${\mathit{v}}_B=\text{MaxEV}(-{\nabla }_{{\mathit{B}}_t}{f}_t)$ and ${\mathit{v}}_L=\text{MaxEV}({\nabla }_{{\mathit{L}}_t})$, where MaxEV(A) computes the eigenvector corresponds to the maximal eigenvalue of $\mathit{A}\in {\mathbb{S}}_{\text{PSD}}^n$; by Hazan algorithm $\mathbf{\Delta }{\mathit{B}}_t:={\mathit{v}}_B{\mathit{v}}_B^{{ }^{\top }}-{\mathit{B}}_t$ and $\mathit{\Delta }{\mathit{L}}_t:={\mathit{v}}_L{\mathit{v}}_L^{{ }^{\top }}-{\mathit{L}}_t$ are the search directions for ${\stackrel{\sim }{\mathit{Z}}}_B$ and ${\stackrel{\sim }{\mathit{Z}}}_L$ respectively. By minimizing $\stackrel{\sim }{f_t}({\mathit{B}}_t,{\mathit{L}}_t,\alpha ):=f({\mathit{B}}_t+\alpha \mathbf{\Delta }{\mathit{B}}_t,{\mathit{L}}_t+\alpha \mathbf{\Delta }{\mathit{L}}_t)$ with respect to learning rate alpha, we have the optimal learning rate given by

$${\alpha }^{\ast }=\frac{{\langle {\mathit{M}}_B,\mathbf{\Delta }{\mathit{B}}_t\rangle }_F+{\langle {\mathit{M}}_L,\mathbf{\Delta }{\mathit{L}}_t\rangle }_F}{{\langle {\mathit{N}}_B,\mathbf{\Delta }{\mathit{B}}_t\rangle }_F+{\langle {\mathit{N}}_L,\mathbf{\Delta }{\mathit{L}}_t\rangle }_F},$$ (C.3)

where

$${\mathit{D}}_{BL}=({\mathit{X}}_t\mathit{\Delta }{\mathit{B}}_t+t_2\mathbf{\Delta }{\mathit{L}}_t)\mathbf{\Phi },{\mathit{M}}_B={\mathit{X}}_t^{\top }{\mathit{R}}_t{\mathbf{\Phi }}^{\top },{\mathit{M}}_L=t_2{\mathit{R}}_t{\mathbf{\Phi }}^{\top },{\mathit{N}}_B={\mathit{X}}_t^{\top }{\mathit{D}}_{BL}{\mathbf{\Phi }}^{\top },{\mathit{N}}_L=t_2{\mathit{D}}_{BL}{\mathbf{\Phi }}^{\top }.$$

To further speed up the computation we adopt the “hot start” strategy, that the solution of $(t_1^{(1)},t_2^{(1)})$ is used to initialize the optimization for nearby parameter pair $(t_1^{(2)},t_2^{(2)})$.

Notice ${\nabla }_{{\mathit{Z}}_B}\stackrel{\sim }{f}$ and ${\nabla }_{{\mathit{Z}}_L}\stackrel{\sim }{f}$ are always symmetric matrices of the block form $\left(\begin{array}{cc}\mathbf{0}& \mathit{G}\\ {\mathit{G}}^{\top }& \mathbf{0}\end{array}\right)$, of which the eigenvectors are also symmetric: whenever (v^⊤, w^⊤)^⊤ is the eigenvector for eigenvalue λ, (v^⊤, −w^⊤)^⊤ is the eigenvector for eigenvalue −λ. Also it is easy to see v and w are the eigenvectors for GG^⊤ and G^⊤G respectively, with the eigenvalue λ². These factor will break down the computation of MaxEV(−Δ_□ f_t) into computing the principal eigenvectors5 for two lower order matrices GG^⊤ and G^⊤G respectively. This can be easily achieved via Lanczos method, Ritz approximation or simply the power iteration. Further speedup can be achieved for “tall” matrices by squaring the lower order matrix product.

Appendix D. Further discussions on NNR implementations

In this section we present some discussions on the NNR implementation to motivate more efficient algorithms. First we define the notation of SVD-thresholding operators. Consider the singular value decomposition of Y ∈ ℝ^p×q,

$$\mathit{Y}=\mathit{U}\mathit{D}{\mathit{V}}^T,$$ (D.1)

where U and V are respectively p×h and q×h orthonormal matrices with h = min(p, q) known as the left and right singular vectors, and diagonal matrix D consists of non-increasing non-negative diagonal elements d known as the singular values of Y. For any λ ≥ 0, the hard SVD-thresholding operator is defined as

$${\mathscr{H}}_{\lambda }(\mathit{Y})=\mathit{U}{\mathscr{H}}_{\lambda }(\mathit{D}){\mathit{V}}^T,{\mathscr{H}}_{\lambda }(\mathit{D})=\text{diag}(\{d_i{I}_{d_i>\lambda }\}),$$ (D.2)

where I_· is the indicator function, and the soft SVD-thresholding operator

$${\mathcal{S}}_{\lambda }(\mathit{Y})=\mathit{U}{\mathcal{S}}_{\lambda }(\mathit{D}){\mathit{V}}^T,{\mathcal{S}}_{\lambda }(\mathit{D})=\text{diag}(\{{(d_i-\lambda )}_{+}\}),$$ (D.3)

where x₊ = max (0, x) denotes the non-negative part of x. The following theorem shows that a connection can be established between the SVD-thresholding operation and solving the simplest form of NNR optimization.

Theorem 2

(Proposition 2.1, Chen et al., 2013). For any λ ≥ 0 and Y ∈ ℝ^p×q, the hard/soft SVD-thresholding operators can be characterized as

$${\mathscr{H}}_{\lambda }(\mathit{Y})=\underset{C}{\text{argmin}}\{\Vert \mathit{Y}-\mathit{C}{\Vert }_F^2+{\lambda }^{2}r(\mathit{C})\},$$ (D.4)

$${\mathcal{S}}_{\lambda }(\mathit{Y})=\underset{C}{\text{argmin}}\{\Vert \mathit{Y}-\mathit{C}{\Vert }_F^2+\lambda \Vert \mathit{C}{\Vert }_{\ast }\},$$ (D.5)

where r(C) denotes the rank of matrix C. This theorem suggests that instead of taking the slow gradient descend, one can benefit from applying the soft SVD-thresholding operator to the observation matrix Y to obtain the exact solution, which involves solving the full SVD of Y. Unfortunately this one-step solution cannot be directly applied to NNR-GRRLF. This is because: (1) there are two matrices instead of one that are involved in the objective function, with different norm constraints; (2) we have also an additional covariate matrix X and a smoothing matrix H that complicates the objective function.

However, with some reformulations to the problem we can decouple the two matrices involved and apply the above SVD scheme in a step-wise fashion. Using the trick developed in Ji and Ye (2009), we can reduce the convergence rate from Jaggi–Hanzan’s $\mathcal{O}(1/k)$ to the optimal rate of $\mathcal{O}(1/k^2)$. But this improved convergence rate does not necessarily imply faster computation in practice because it invokes higher per-iteration cost. We now present the details below.

First consider the minimization of the smooth loss function without the trace norm regularization:

$$\underset{W}{\min }f(\mathit{W}).$$ (D.6)

Let α_k = 1/β_k be the step size for iteration k, the gradient step for solving the smooth problem

$${\mathit{W}}_k={\mathit{W}}_{k-1}-\frac{1}{{\beta }_k}\nabla f({\mathit{W}}_{k-1})$$ (D.7)

can be reformulated equivalently as a proximal regularization of the linearized function of f(W) at W_k−1 as

$${\mathit{W}}_k={\text{argmin}}_{\mathit{W}}{\mathit{P}}_{{\beta }_k}(\mathit{W},{\mathit{W}}_{k-1}),$$ (D.8)

where

$${\mathit{P}}_{{\beta }_k}(\mathit{W},{\mathit{W}}_{k-1})=f({\mathit{W}}_{k-1})+\langle \mathit{W}-{\mathit{W}}_{k-1},\nabla f({\mathit{W}}_{k-1})\rangle +\frac{{\beta }_k}{2}\Vert \mathit{W}-{\mathit{W}}_{k-1}{\Vert }_F^2,$$

and 〈A, B〉 = tr(A^TB) denotes the matrix inner product. The above ${\mathit{P}}_{{\beta }_k}$ can be considered as a linear approximation of the function f at point W_k−1 regularized by a quadratic proximal term. Based on this equivalence, the optimization problem

$$\underset{W}{\min }\left(f\left(\mathit{W}\right)+\lambda {\Vert \mathit{W}\Vert }_{\ast }\right)$$ (D.9)

for λ≥0 can be solved with the following iterative step:

$${\mathit{W}}_k=\underset{W}{\mathrm{argmin}}{P}_{{\beta }_k}\left(\mathit{W},{\mathit{W}}_{k-1}\right)+\lambda {\Vert \mathit{W}\Vert }_{\ast }$$ (D.10)

By ignoring W-independent terms, we arrive at a new objective

$$Q_{{\beta }_k}\left(\mathit{W},{\mathit{W}}_{k-1}\right)=\frac{{\beta }_k}{2}{\Vert \mathit{W}-\left({\mathit{W}}_{k-1}-\frac{1}{{\beta }_k}\nabla f\left({\mathit{W}}_{t-1}\right)\right)\Vert }_F^2+\lambda {\Vert \mathit{W}\Vert }_{\ast },$$ (D.11)

and

$${\mathit{W}}_k=\underset{W}{\arg \min }{Q}_{{\beta }_k}\left(\mathit{W},{\mathit{W}}_{k-1}\right).$$

$${\mathit{W}}_k=\underset{W}{\arg \min }{Q}_{{\beta }_k}\left(\mathit{W},{\mathit{W}}_{k-1}\right).$$ (D.12)

The key idea behind the above formulation is that by exploiting the structure of the trace norm solution (that can be computed exactly), it can be proven the convergence rate of the regularized objective is the same as that of the gradient descend of f(W). The Nesterov gradient approach can be further exploited to achieve the optimal convergence rate of $\mathcal{O}(1/k^2)$. Readers are referred to Ji and Ye (2009) for details.

Back to GRRLF, let us denote $G\left(\stackrel{\sim }{\mathit{B}},\stackrel{\sim }{\mathit{L}}\right)={\Vert \mathit{Y}-\mathit{X}\stackrel{\sim }{\mathit{B}}\mathit{H}-\stackrel{\sim }{\mathit{L}}\mathit{H}\Vert }_F^2$ then using the idea above we can decouple the term $\stackrel{\sim }{\mathit{B}}$ and $\stackrel{\sim }{\mathit{L}}$ in the objective function as

$${\stackrel{\sim }{P}}_{{\beta }_k}\left(\stackrel{\sim }{\mathit{B}},\stackrel{\sim }{\mathit{L}},{\stackrel{\sim }{\mathit{B}}}_{k-1},{\stackrel{\sim }{\mathit{L}}}_{k-1}\right)=\frac{{\beta }_k}{2}\left({\Vert \stackrel{\sim }{\mathit{B}}-\left({\stackrel{\sim }{\mathit{B}}}_{k-1}-\frac{1}{{\beta }_k}{\nabla }_{B}G\left({\stackrel{\sim }{\mathit{B}}}_{k-1},{\stackrel{\sim }{\mathit{L}}}_{k-1}\right)\right)\Vert }_F^2+{\Vert \stackrel{\sim }{\mathit{L}}-\left({\stackrel{\sim }{\mathit{L}}}_{k-1}-\frac{1}{{\beta }_k}{\nabla }_{L}G\left({\stackrel{\sim }{\mathit{B}}}_{k-1},{\stackrel{\sim }{\mathit{L}}}_{k-1}\right)\right)\Vert }_F^2\right),$$ (D.13)

and define a new surrogate objective function at each iteration as

$${\stackrel{\sim }{Q}}_{{\beta }_k}\left(\stackrel{\sim }{\mathit{B}},\stackrel{\sim }{\mathit{L}},{\stackrel{\sim }{\mathit{B}}}_{k-1},{\stackrel{\sim }{\mathit{L}}}_{k-1}\right)={\stackrel{\sim }{P}}_{{\beta }_k}+{\lambda }_1{\Vert \stackrel{\sim }{\mathit{B}}\Vert }_{\ast }+{\lambda }_2{\Vert \stackrel{\sim }{\mathit{L}}\Vert }_{\ast }={\stackrel{\sim }{Q}}_{{\beta }_k}^B+{\stackrel{\sim }{Q}}_{{\beta }_k}^L,$$ (D.14)

where

$${\stackrel{\sim }{Q}}_{{\beta }_k}^B={\Vert \stackrel{\sim }{\mathit{B}}-\left({\stackrel{\sim }{\mathit{B}}}_{k-1}-\frac{1}{{\beta }_k}{\nabla }_{B}G\left({\stackrel{\sim }{\mathit{B}}}_{k-1},{\stackrel{\sim }{\mathit{L}}}_{k-1}\right)\right)\Vert }_F^2+\frac{2{\lambda }_1}{{\beta }_k}{\Vert \stackrel{\sim }{\mathit{B}}\Vert }_{\ast },$$ (D.15)

$${\stackrel{\sim }{Q}}_{{\beta }_k}^L={\Vert \stackrel{\sim }{\mathit{L}}-\left({\stackrel{\sim }{\mathit{L}}}_{k-1}-\frac{1}{{\beta }_k}{\nabla }_{L}G\left({\stackrel{\sim }{\mathit{B}}}_{k-1},{\stackrel{\sim }{\mathit{L}}}_{k-1}\right)\right)\Vert }_F^2+\frac{2{\lambda }_2}{{\beta }_k}{\Vert \stackrel{\sim }{\mathit{L}}\Vert }_{\ast }.$$ (D.16)

Therefore solving for (D.14) reduces to the application of soft SVD thresholding to (D.15) and (D.16) independently.

The gain in convergence rate is however not for free. In each iteration a full SVD needs to be solved instead of a partial SVD that is required by the JH algorithm. Additionally, we can no longer compute an optimal step size for each iteration as that in JH. To summarize, while we can expect a theoretically optimal solver for NNR-GRRLF, the best implementation is application dependent and relies on careful tuning.

Appendix E. GM-GRRLF specifications for the synthetic experiment

While we used the GCV procedure to decide which component is estimated first in the estimation error experiment, we replaced the costly GCV with a simpler heuristic thresholding strategy when computing the empirical p-values in the sensitivity experiment to save time. We first estimate the percentage of variance contributed by the covariates with voxel-wise least squares, if the covariate signal proportion exceeds a specified threshold, the covariate effect will be estimated first in the iterative GM-GRRLF. We set the variance threshold to 20% in this experiment, which gave very similar estimation error distribution compared with that of GCV (not shown). We used the HSIC to test for the association between covariates and estimated latent components, and set the association significance threshold to p_thres = 10⁻³.

Appendix F. ADNI study design and subjects

The data used in the preparation of this article were obtained from the Alzheimers Disease Neuroimaging Initiative (ADNI) database (http://www.adni.loni.usc.edu). The ADNI was launched in 2003 as a public–private partnership, led by Principal Investigator Michael W. Weiner, MD. The primary goal of ADNI has been to test whether serial magnetic resonance imaging (MRI), positron emission tomography (PET), other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of mild cognitive impairment (MCI) and early Alzheimer’s disease (AD). Determination of sensitive and specific markers of very early AD progression is intended to aid researchers and clinicians to develop new treatments and monitor their effectiveness, as well as lessen the time and cost of clinical trials. The Principal Investigator of this initiative is Michael W. Weiner, MD, VA Medical Center and University of California—San Francisco. ADNI is the result of efforts of many co-investigators from a broad range of academic institutions and private corporations, and subjects have been recruited from over 50 sites across the U. S. and Canada. The initial goal of ADNI was to recruit 800 adults, ages 55–90, to participate in the research, approximately 200 cognitively normal older individuals to be followed for 3 years, 400 people with MCI to be followed for 3 years and 200 people with early AD to be followed for 2 years. For up-to-date information, see http://www.adni-info.org. 818 subjects were genotyped as part of the ADNI study. However, only 736 unrelated Caucasian subjects identified by self-report and confirmed by MDS analysis (Stein et al., 2010a) were included to reduce population stratification effects. Volumetric brain differences were assessed in 176 AD patients (80 female/96 male; 75.47 ± 7.54 years old), 356 MCI subjects (126 female/230 male; 75.03 ± 7.25 years old), and 204 healthy elderly subjects (94 female/110 male; 76.04 ± 4.98 years old).

Appendix G. Data preprocessing

G.1. MRI images

High-resolution structural brain MRI scans were acquired at 58 ADNI sites with 1.5 T MRI scanners using a sagittal 3D MP-RAGE sequence developed for consistency across sites (Jack et al., 2008) (TR=2400 ms, TE=1000 ms, flip angle=8°, field of view=24 cm, final reconstructed voxel resolution=0.9375 × 0.9375 × 1.2 mm³). Images were calibrated with phantom-based geometric corrections to ensure consistency across scanners. Additional image corrections included (Jack et al., 2008): (1) correction of geometric distortions due to gradient nonlinearity, (2) adjustment for image intensity inhomogeneity due to B1 field non-uniformity using calibration scans, (3) reducing residual intensity inhomogeneity, and (4) geometric scaling according to a phantom scan acquired for each subject to adjust for scanner- and session-specific calibration errors. Images were linearly registered with 9 parameters to the International Consortium for Brain Imaging template (ICBM-53) (Mazziotta et al., 2001) to adjust for differences in brain position and scaling.

For TBM analysis, a minimal deformation template was first created for the healthy elderly group to serve as an unbiased average template image to which all other images were warped using a nonlinear inverse-consistent elastic intensity-based registration algorithm (Leow et al., 2005). Volumetric tissue differences were assessed at each voxel in all individuals by calculating the determinant of the Jacobian matrix of the deformation, which encodes local volume excess or deficit relative to the mean template image. The maps of volumetric tissue differences were then down-sampled using trilinear interpolation to 4 × 4 × 4 mm³ isotropic voxel resolution for computational efficiency. After resampling, 29, 479 voxels remained in the brain mask. The percentage volumetric difference relative to a population-based brain template at each voxel served as a quantitative measure of brain tissue volume difference for genome-wide association.

G.2. Genetic data

Genome-wide genotype data were collected at 620,901 markers on the Human610-Quad BeadChip (Illumina, Inc., San Diego, CA). For details on how genetic data were processed, please see Saykin et al. (2010) and Stein et al. (2010a). Different types of markers were genotyped (including copy number probes), but only SNPs were used in this analysis. Due to the filtering based on Illumina GenCall quality control measures, individual subjects have some residual missing genotypes at random SNPs throughout the dataset. We performed imputation using the software, Mach (version 1.0), to infer the haplotype phase and automatically impute the missing genotype data (Li et al., 2009). The genetic tags are translated into corresponding Reference SNP cluster ID (rsid) with a dictionary used in imputation. Chromosome positions of the rsids are mapped according to the GRCh38.p2 reference assembly. We use the gene annotations from Ensembl release 79 (Cunningham et al., 2015), which also mapped to GRCh38.p2 reference assembly, to define the start and end position of the genes. All SNPs fall into the same gene region are considered as belonging to the same gene. We use only the SNPs that have been physically genotyped on the 22 autosomes for the gene grouping and after that, a total of n_gene = 26, 664 genes were left for analysis. Only SNPs with imputed minor allele frequency (MAF) ≥0.1 are used for the single-locus experiment on the target gene.

Appendix H. Statistical methods for ADNI data analysis

We use a modified version of the LSKM-based vGWAS proposed in Ge et al. (2012) in the ADNI data analysis, which is detailed below.

H.1. Fitted model and choice of kernel

Since only gender and age are supplied as covariates, the dimension reduction on covariates is unnecessary in this particular case. So we fit the following simplified null model for the GWAS analysis on ADNI data

$$y_{i,\mathit{v}}={\mathit{x}}_i^{\top }{\mathit{\beta }}_{\mathit{v}}+l_{i,\mathit{v}}+{\xi }_{i,\mathit{v}},$$

where i = 1,…,n is the subject index, v ∈ Ω is the voxel index, y is the image phenotype, x is the covariates, l the latent effect and ξ the residual component. We use a generalized identity by state (IBS) function as the kernel function in this study, which is defined as

$$\kappa \left({\mathit{g}}_1,{\mathit{g}}_2\right)=1-{\Vert {\mathit{g}}_1-{\mathit{g}}_2\Vert }_1/(2n_{\mathit{g}}),$$

where ${\mathit{g}}_i\in {\left[0,2\right]}^{n_{\mathit{g}}}$ for i=1,2 is the genetic data and n_g is the number of SNPs on gene g. To expedite the computation, we use incomplete Cholesky decomposition (ICL) (Bach and Jordan, 2003) to give low rank approximation LL^⊤ of the kernel matrix K. We restrict the maximum allowed rank to r=50 and the results are similar to those using original kernel matrix (data not shown).

H.2. Null distribution of the LSKM test score

The test score Q of LSKM follows a mixed Chi-square distribution under certain assumptions (see Liu et al., 2007 for details). With the Satterthwaite method (matching the first two moments), the distribution of the test score Q can be approximated by equating the mean and variance of the scaled chi-square variable $\kappa {\chi }_{\nu }^2$. Specifically, $\kappa ={\stackrel{\sim }{I}}_{\tau \tau }/2\stackrel{\sim }{e}$, $\nu =2{\stackrel{\sim }{e}}^2/{\stackrel{\sim }{I}}_{\tau \tau }$, where ${\stackrel{\sim }{I}}_{\tau \tau }=I_{\tau \tau }-I_{\tau {\sigma }^2}{I}_{{\sigma }^2{\sigma }^2}^{-1}{I}_{\tau {\sigma }^2}$, $I_{\tau \tau }=\text{tr}({\left(P_{0}K\right)}^2)$, $I_{\tau {\sigma }^2}=\text{tr}\left(P_{0}K{P}_0\right)/2$, $I_{{\sigma }^2{\sigma }^2}=\text{tr}(P_0^2)/2$ and $\stackrel{\sim }{e}=\text{tr}\left(P_{0}K\right)/2$ and P₀ is the residual forming matrix defined as

$${\mathit{P}}_0=\mathit{I}-\mathit{X}{\left({\mathit{X}}^{\top }\mathit{X}\right)}^{-1}{\mathit{X}}^{\top }.$$

We note however, these assumptions, such as the normality of the residuals, can be easily violated in practice. Also the fitness of the approximation, especially at the tail, depends on how well the moment matched distribution in the scaled Chi-square family resembles the originally mixed Chi-square. Thus researchers need to check the validity of the use of parametric approximation with their data. Unfortunately our null simulations suggest that with the kernel matrices derived from empirical genetic data, the p-values evaluated using the approximated scaled Chi-square is severely inflated at the tail, see Fig. 13(a) for the distribution of p-values using empirical kernel matrices from the ADNI1 dataset and i.i.d standard Gaussian variables as responses. To correct for the inflation, we use nonparametric permutation to evaluate the p-value under the null instead of the scaled Chi-square approximation. The subject index of ${\widehat{\mathit{\xi }}}_{\mathit{v}}$ is independently shuffled for each voxel v, then the test score $Q_{\mathit{v}}^{\text{null}}$ is calculated using the shuffled ${\widehat{\mathit{\xi }}}_{\mathit{v}}$ for each voxel $\mathit{v}\cdot {\{Q_{\mathit{v}}^{\text{null}}\}}_{\mathit{v}\in \Omega }$ is considered as N_vox independent test scores under the null hypothesis thus giving the empirical null distribution. Then it is used to calculate the empirical p-values for the test scores as

$$p_{\text{emp}}\left(Q\right)=\max \left\{\frac{\#\left\{Q_{\mathit{v}}^{\text{null}}\ge Q\right\}}{N_{\text{vox}}},\frac{1}{N_{\text{vox}}}\right\}.$$

We further used generalized Pareto distribution (GPD) (Coles et al., 2001; Knijnenburg et al., 2009) to approximate the tail of the empirical distribution. The largest 1% of $\{Q_{\mathit{v}}^{\text{null}}\}$ is used for the maximum likelihood estimation of GPD parameters and then the p-values for the tail statistics are evaluated using the estimated parameters. The results of the GPD approximated p-values are presented in Fig. 13(b). The GPD approximated tail p-values are also prone to inflation when they are smaller than 10⁻⁴. In this regard, no peak inference is conducted in this study, as the results are unreliable. We report only the result of cluster-size based inference with cluster-forming threshold set to p_thres = 10⁻³, where the inflation is negligible.

Fig. 13.

Expected and observed distribution of p-values in log₁₀ scale. (a) Uncorrected p-values. (b) Corrected p-values. Black solid: expected p-value, black dash: expected 95% confidence interval, blue solid: median of the observed p-value, blue dash: observed 95% interval. Uncorrected p-values from the LKSM Satterthwaite approximation gives much more false positives than expected thus cannot be directly reported. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this paper).

H.3. Cluster size based inference

In this study, the maximum cluster size S in RESEL for each gene is used as the test statistics. RESEL stands for RESolution ELement (Worsley et al., 1992), which represents a virtual voxel with size [FWHM_X, FWHM_Y, FWHM_Z]. In the stationary case, RESEL count R is the number of such virtual voxels that fit into the search volume V

$$R=\frac{V}{{\displaystyle {\prod }_{u\in X,Y,Z}{\text{FWHM}}_u}}.$$

In the nonstationary case (Hayasaka et al., 2004), voxel-wise Resels Per Voxel (RPV) statistics is defined as

$${\text{RPV}}_{\mathit{v}}={\displaystyle \sum _{\mathit{v}\in \Omega }\frac{{\left|\Omega \right|}^{-1}V}{{\displaystyle {\prod }_{u\in X,Y,Z}{\text{FWHM}}_u^{\left(\mathit{v}\right)}}}},$$

where |Ω| is the voxel count and $R_n={\displaystyle {\sum }_{\mathit{v}}{\text{RPV}}_{\mathit{v}}}$ generalizes RESEL count R in stationary case. Simply put, RESEL count is a measure of volume normalized by the smoothness of image. Specifically, we use SPM’s spm_est_smoothness function in SPM 8 to estimate the RPV image. Then we construct all clusters the using spm_bwlabel function with the connectivity pattern criterion set to ‘edge’. The cluster size is calculated by integrating RPV for each cluster. For each gene, the maximum cluster size is reported. To construct the null distribution of the maximum cluster size, we shuffled the subject index and then permute the rows and columns of the kernel matrices accordingly. For each gene, 20 null statistics were calculated. Then the $M_{\text{perm}}=20N_{\text{gene}}$ null statistics were pooled together to give an empirical null distribution ${\{S_b^{\text{null}}\}}_{b=1}^{M_{\text{perm}}}$. The empirical p-value of the cluster size S is given as

$$p_{\text{emp}}^{\text{clu}}\left(S\right)=\max \left\{\frac{\#\left\{S_b^{\text{null}}\ge S\right\}}{M_{\text{perm}}},\frac{1}{M_{\text{perm}}}\right\}.$$

We found that the number of permutations we ran is unable to give sufficient samples for the estimation of tail distribution of maximum cluster size using GPD (data not shown), so only the empirical p-value is reported.