Joint Sparse Collaborative Regression on Imaging Genetics Study of Schizophrenia

Abstract

The imaging genetics approach generates large amount of high dimensional and multi-modal data, providing complementary information for comprehensive study of Schizophrenia, a complex mental disease. However, at the same time, the variety of these data in structures, resolutions, and formats makes their integrative study a forbidding task. In this paper, we propose a novel model called Joint Sparse Collaborative Regression (JSCoReg), which can extract class-specific features from different health conditions/disease classes. We first evaluate the performance of feature selection in terms of Receiver operating characteristic (ROC) curve and the area under the ROC curve (AUC) in the simulation experiment. We demonstrate that the JSCoReg model can achieve higher accuracy compared with similar models including Joint Sparse Collaborative Regression (JSCCA) and Sparse Collaborative Regression (SCoReg). We then applied the JSCoReg model to the analysis of schizophrenia dataset collected from the Mind Clinical Imaging Consortium. The JSCoReg enables us to better identify biomarkers associated with schizophrenia, which are verified to be both biologically and statistically significant.

1. Introduction

Schizophrenia (SZ) is a complex mental disorder potentially caused by the interaction between genetic factors and environmental influences [1]. Genetic variants are considered as possible risk factors of SZ, and a large number of studies have focused on exploring critical genes susceptible to the SZ [2, 3]. Besides genetic studies, brain imaging study has been largely used for the research of SZ to identify brain function disorders [4, 5]. Although unimodal (genetic or brain imaging) studies can discern SZ’s biomarkers from a certain point of view, it is difficult to capture potential links between genetic variants and brain imaging in the study of SZ[6, 7]. Multi-view study has many advantages over unimodal study for identifying meaningful biomarkers [8]. Therefore, it has become a significant and challenging problem to combine genetic and imaging data using multivariate approaches to explore biomarkers [9].

Despite being an emerging field, many advanced multivariate methods have been proposed and successfully used by the research community. Early studies based on Canonical Correlation Analysis (CCA) model performed pairwise analysis between genetic and brain imaging data. For example, Sparse Canonical Correlation Analysis (SCCA), a model penalized on conventional CCA by a sparse regularization term, has been widely used to extract signification features through sparse canonical variables [10–12]. In order to incorporate the biological prior knowledge, Group SCCA was developed by Lin et al. [13] to exploit structural information, which can further enhance the feature selection. Considering that some prior knowledge is uncertain or unavailable, structured SCCA model was proposed by Du et al. [14] with absolute graph net regularizer, and the group effect was evaluated by an upper bound. Joint SCCA (JSCCA) model was built by Fang et al. [15] to analyze the differential dependency across multiple classes (e.g., different disease-status). In recent studies, Collaborative Regression (CoReg) model, a intuitive combination of regression and CCA, has attracted extensive attention. CoReg is a useful model for extracting features that are common to both genetic imaging data and phenotype, and it leads to promising results in the context of imaging genomics [16]. For example, Pascal et al. [17] intuitive combine sparse regression and CCA to extract the features that associated with phenotypes while displaying a significant level of co-expression. Based on the CoReg model, Bai et al. [18] identify biomarkers of SZ through integrating fMRI and Epigenetics.

As we all know, the genetic imaging data usually are collected from different classes (e.g., the SZ patients and healthy controls). And the conventional analysis methods usually select the same biomarkers from different classes. However, some diseases usually caused by the genetic variants, so that the biomarkers usually show the class-specific structure. For example, certain single nucleotide polymorphisms (SNPs) differ in their contribution to the genetic-modality latent variable across classes, whereas other SNPs show similar contribution across classes. Nevertheless, the class-specific biomarkers has been mostly overlooked in genetic imaging study, although it is useful to facilitate the understanding of SZ. In this paper, based on sparse regression and CCA model, we propose a Joint Sparse Collaborative Regression (JSCoReg) model to extract both shared and different features from different health conditions / disease classes. Specifically, the fused lasso penalty is applied to perform the fusion of different classes by encouraging canonical loadings from different class to share a similar structure. For imaging genetics data, our method can identify common brain regions and class-related genes. Fig. 1 is a schematic illustration of JSCoReg model.

Fig. 1.

An illustration of the JSCoReg model for feature selection from multimodal datasets.

The contribution of this paper is twofold. On one hand, we propose a JSCoReg model and derive the corresponding optimization algorithm for the solution. JSCoReg extracts the features associate with a given phenotypes while exhibiting class-specific patterns. A modified block coordinate descent algorithm is used to solve this model in three steps: fusion step, sparsity step, and normalization step. For validation, the simulation experiment is first conducted to evaluate the performance of JSCoReg. Then JSCoReg is applied to real Mind (Mental Illness and Neuroscience Discovery Institute) Clinical Imaging Consortium (MCIC) data [19], pinpointing the potential genes and brain regions related to schizophrenia, which can in turn inform of clinical decision making.

The rest of this paper is organized as follows. In Section 2, we review some relevant models and then propose our model followed by the optimization algorithm. The performance of our model is evaluated on both simulated and real MCIC data in Section 3 and Section 4, respectively. Followed by conclusions and discussions in Section 5.

2. The novel model

2.1. Sparse regression model

Assuming that we have a set of measurements $X=\left[x_1;x_2;\cdots ;x_n\right]\in R^{n\times p}$ and the corresponding observations $Y={\left[y_1,y_2,\cdots y_n\right]}^{\mathrm{T}}$. We assume that the $y_i$ are conditionally independent given its measurement $x_i(i=1,\cdots ,n)$. In order to extract a linear link between $Y$ and $X$, we consider the following linear regression model

$$\underset{\omega }{\mathrm{m}\mathrm{i}\mathrm{n}}\parallel Y-X\omega {\parallel }_2^2$$ (1)

where $\omega \in R^p$ is the vector of objective regression coefficients. Phenotype $Y$ represents class labels (that is, $Y={\left[y_1,y_2,\cdots y_n\right]}^{\mathrm{T}},y_i\in \left\{-1,1\right\},i=1,\cdots ,n$). It is noted that, although we only discussed the possibility of $Y$ being a class label, other options exist as well. For instance, $Y$ could be an influence factor on SZ (e.g., DNA methylation) or a quantitative outcome (e.g., clinical assessment score).

In practice, the number of variables (or features) is much larger than that of observations, i.e., $n\ll p$, which causes Eq. (1) to be underdetermined. A useful tool to perform this underdetermined problem is the least absolute shrinkage and selection operator (LASSO) regression model that proposed in [21]

$$\underset{\omega }{\mathrm{m}\mathrm{i}\mathrm{n}}\parallel Y-X\omega {\parallel }_2^2+\lambda \parallel \omega {\parallel }_1$$

where $\lambda$ is hyper-parameter, and the $l_1$-norm induce the sparsity of the regression coefficient $\omega$.

Lasso regression established the relationship between only a few features and phenotypes. And it is a remarkable model to perform variable selection. Next, we review the SCCA model for extracting the links between different modalities.

2.2. SCCA model and SCoReg model

We consider two datasets $X=\left[X_1;X_2;\cdots ;X_n\right]$ and $Z=\left[Z_1;Z_2;\cdots ;Z_n\right]$ on the same $n$ subjects, where $X$ has $p$ features and $Z$ has $q$ features. In previous studies on multimodal datasets, both CCA and penalized CCA have been used for feature-extraction and selection. CCA finds the most correlated canonical variables (the linear combination of features with canonical loadings). A form of CCA is expressed as follows:

$$\begin{array}{r}\underset{w,v}{\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}}F(w,v)=\parallel Xw-Zv{\parallel }_2^2,\\ \mathit{s.t.}{w}^T{X}^{T}Xw=1,v^T{Z}^{T}Zv=1,\end{array}$$

where $w$ and $v$ are the canonical loadings to be determined. However, in many biomedical applications, a common problem is that the dimension of the data is much bigger than the sample size, which leads to overfitting problems or curse of dimensionality. To circumvent these issue, SCCA selects the key or significant features by imposing the $l_1$-penalty on the canonical loadings. Specifically, SCCA model can be described as

$$\begin{array}{r}\underset{w,v}{\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}}F(w,v)=\parallel Xw-Zv{\parallel }_2^2+{\lambda }_1\parallel w{\parallel }_1+{\lambda }_2\parallel v{\parallel }_1,\\ s.t.w^T{X}^{T}Xw=1,v^T{Z}^{T}Zv=1,\end{array}$$

where ${\lambda }_1$ and ${\lambda }_2$ are hyper-parameters to tune the weight of each regularization term. While SCCA is a useful tool to find the information shared by both $X$ and $Z$, it overlooks the relationship between the extracted features and phenotypes. To address these issue, SCoReg combines both regression and SCCA into a unified framework with the $l_1$-penalty to choose a subset of co-expressed discriminative features. Given phenotype $Y$, SCoReg[16] can be expressed as follows:

$$\begin{array}{r}\underset{w,v}{\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}}F(w,v)=\parallel Xw-Zv{\parallel }_2^2+\parallel Y-Xw{\parallel }_2^2\\ +\parallel Y-Zv{\parallel }_2^2+{\lambda }_1\parallel w{\parallel }_1+{\lambda }_2\parallel v{\parallel }_1,\\ s.t.w^T{X}^{T}Xw=1,v^T{Z}^{T}Zv=1.\end{array}$$

In this SCoReg model, the extracted features are assumed to come from the same class or group (e.g., healthy controls and schizophrenia patients). However, in real imaging genetics studies, these different classes may contain common features and/or ones specific to each class. To this end, we propose a novel model, namely JSCoReg, to discover both the class-specific and class-shared information, which is detailed in the following.

2.3. Joint Sparse Collaborative Regression Model (JSCoReg)

As discussed in the previous sections, numerous methods have been proposed to select the features that are common to different measurements while are associated with phenotype. In this section, we consider the scenario where exist the class-specific features in a certain measurement. We aim to propose a new model to select class-specific features associated with their phenotypes.

Dividing the data into $K$ classes, the two sets of data can be denoted as $X=\left[X_1;X_2;\cdots ;X_K\right]$, and $Z=\left[Z_1;Z_2;\cdots ;Z_K\right]$, where $X_k\in R^{n_k\times p},$ $Z_k\in R^{n_k\times q}$ $(k=1,2,\cdots ,K)$, and $n_k$ is the number of subjects belonging to $k$-th class. JSCoReg estimates the features’ coefficients $w$ and $v_k$ by minimizing the following objective function:

$$\begin{array}{r}\hfill F\left(w,v_k\right)=\parallel Y-Xw{\parallel }_2^2+\sum _{k=1}^K\parallel Y_k-Z_k{v}_k{\parallel }_2^2\\ \hfill +\sum _{k=1}^K{\Vert X_{k}w-Z_k{v}_k\Vert }_2^2+{\lambda }_1\parallel w{\parallel }_1+\sum _{k=1}^K{\lambda }_2\parallel v_k{\parallel }_1\\ \hfill +\sum _{k<k^{\prime }}a\parallel v_k-v_{k^{\prime }}{\parallel }_1,\\ \hfill s.t.w^T{X}_k^T{X}_{k}w=1,v_k^T{Z}_k^T{Z}_k{v}_k=1.\end{array}$$ (2)

where the columns of $Z_k$ and $X$ are standardized separately to have zero means and unit variances; $w\in R^{p\times 1}$ and $v_k\in R^{q\times 1}(k=1,2,\cdots ,K)$ are the canonical loadings of $X$ and $Z_k$ respectively; $a,$ ${\lambda }_1$, and ${\lambda }_2$ are hyper-parameters to tune the weight of each regularization term.

The $l_1$-penalties on $w$ and each $v_k$ are equipped to control their sparsity, and the fused lasso penalty on canonical loadings $v_k(k=1,2,\cdots ,K)$ encourages them to both share a similar structure and have the class-specific patterns. The hyper-parameter a plays an important role in adjusting the degree of fusion: if $a=0$, there is no fusion across the canonical loadings and our model reduces to SCoReg for each single class, and we define this case as SCoReg(Single); if $a\to \mathrm{\infty }$, canonical loadings $v_{k}s(k=1,2,\cdots ,K)$ are required to be identical to each other and our model reduces to SCoReg where all extracted features belong the same class, and we define this case as SCoReg (Combined).

2.4. Optimization Algorithm

By expanding the $l_2$-penalty in (2) and then removing the constant terms in the expansion, the optimization problem is transformed into the following form:

$$\begin{array}{r}\hfill \underset{\omega ,v_k}{\text{min}}-\sum _{k=1}^{K}w{X}_k^T{Z}_k{v}_k-\sum _{k=1}^K{Y}_k^T{Z}_k{v}_k-Y^{T}Xw\\ \hfill +{\lambda }_1\parallel w{\parallel }_1+\sum _{k=1}^K{{\lambda }_2\Vert v_k\Vert }_1+\sum _{k<k^{\prime }}a\parallel v_k-v_{k^{\prime }}{\parallel }_1,\\ \hfill s.t\text{.}{w}^T{X}_k^T{X}_{k}w=1,v_k^T{Z}_k^T{Z}_k{v}_k=1,\\ \hfill w^T{X}^{T}Xw=1.\end{array}$$ (3)

It has been proven that in high-dimensional cases, treating covariance matrices (e.g.,$X^{T}X$, $X_k^T{X}_k$ and $Z_k^T{Z}_k$) as diagonal can lead to good results [22, 23]. Inspired by the work of [11], we substitute the covariance matrices (e.g., $X^{T}X$, $X_k^T{X}_k$ and $Z_k^T{Z}_k$) with identity matrices to ensure the validity of the model and simplify the optimization problem (3), and obtain the following “diagonal penalized CCA” form:

$$\begin{array}{r}\hfill \underset{\omega ,v_k}{\text{min}}-\sum _{k=1}^K{w}^T{X}_k^T{Z}_k{v}_k-\sum _{k=1}^K{Y}_k^T{Z}_k{v}_k-Y^{T}Xw\\ \hfill +{\lambda }_1\parallel w{\parallel }_1+\sum _{k=1}^K{\lambda }_2\parallel v_k{\parallel }_1+\sum _{k<k^{\prime }}a\parallel v_k-v_{k^{\prime }}{\parallel }_1,\\ \hfill s.t\text{.}\parallel w{\parallel }_2^2=1,\parallel v_k{\parallel }_F^2=1.\end{array}$$ (4)

Note that $Y_k^T{Z}_k=0$ as the columns of $X_k$ and $Z_k$ are standardized with zero mean and unit variance. Thus, we rewrite the optimization problem into the following form:

$$\begin{array}{r}\hfill \underset{w,v_k}{\text{min}}-\sum _{k=1}^K{w}^T{X}_k^T{Z}_k{v}_k-Y^{T}Xw+{\lambda }_1\parallel w{\parallel }_1\\ \hfill +\sum _{k=1}^K{\lambda }_2\parallel v_k{\parallel }_1+\sum _{k<k^{\prime }}a\parallel v_k-v_{k^{\prime }}{\parallel }_1,\\ \hfill s.t.\parallel w{\parallel }_2^2=1,\parallel v_k{\parallel }_F^2=1.\end{array}$$ (5)

Especially, considering the scenario where $K=2$, we code the phenotype of two classes ($n_1$ and $n_2$ observations, respectively) as $\frac{c}{n_1}$ and $-\frac{c}{n_2}$ respectively, where $c$ is a non-negative parameter [20]. Notice that $X$, $Y$, $w$, $n_1$, and $n_2$ satisfy

$$Y^{T}Xw=c\left(\overline{X_{class1}}-\overline{X_{class2}}\right)w=c\tilde{X}w$$ (6)

where $\overline{X_{classi}}=\left[\overline{x_{classi1}},\overline{x_{classi2}},\cdots ,\overline{x_{classip}}\right]\in R^p(i=1,2)$ is the mean vector of the observations in $X$ that belong to class $i$, and $\tilde{X}=\overline{X_{class1}}-\overline{X_{class2}}=\left[{\tilde{x}}_1,{\tilde{x}}_1,\cdots ,{\tilde{x}}_1\right]\in R^p(i=1,2)$.

Therefore we can rewrite (5) as

$$\begin{array}{r}\hfill \underset{w,v_k}{\text{min}}-\sum _{k=1}^K{w}^T{X}_k^T{Z}_k{v}_k-c\tilde{X}w+{\lambda }_1\parallel w{\parallel }_1\\ \hfill +\sum _{k=1}^K{\lambda }_2\parallel v_k{\parallel }_1+\sum _{k<k^{\prime }}a\parallel v_k-v_{k^{\prime }}{\parallel }_1,\\ \hfill s.t\text{.}\parallel w{\parallel }_2^2=1,\parallel v_k{\parallel }_F^2=1.\end{array}$$ (7)

In (7), the parameter $c$ has a great impact on large $\left|{\tilde{x}}_i\right|\left(\right|\cdot |$ is the absolute value, $i=1,2,\cdots ,p)$ in $\tilde{X}$. That is, when $c$ increases, the elements of $w$ corresponding to the larger $\left|{\tilde{x}}_i\right|$ will be selected with a priority. Thus, finding an appropriate $c$ is critical for feature selection. A special case is that, when $c=0$, this model reduces to a regular JSCCA. When there are only two classes in the data, namely $K=2$, we will work on model (7), which is a more flexible and feasible form of (5).

To summarize, our JSCoReg model can be realized by solving the optimization problem (5) including its special case (7). The objective function (5) is biconvex, that is, it is convex with respect to the decision variable $w$ with given $v_k$ and vice versa. Thus, the block coordinate descent is applied here and the iteration procedure mainly contains the following two steps:

$$\begin{array}{r}\hfill \underset{\Vert w{\Vert }_2=1}{\min }-(\sum _{k=1}^K{v}_k{Z}_k^T{X}_k+Y^{T}X)w+{\lambda }_1\Vert w{\Vert }_1\\ \hfill (Y^{T}X=c\tilde{X},\text{when}K=2),\end{array}$$ (8)

$$\begin{array}{r}\hfill \underset{\Vert V{\Vert }_F=1}{\min }-\sum _{k=1}^K{w}^T{X}_k^T{Z}_k{v}_k+\sum _{k=1}^K{\Vert {\lambda }_2{v}_k\Vert }_1\\ \hfill +\sum _{k<k^{\prime }}{\Vert a{v}_k-v_{k^{\prime }}\Vert }_1,\end{array}$$ (9)

where $V={\left[v_1,v_2,\cdots v_K\right]}^T$.

In detail, when $V={\left[v_1,v_2,\cdots v_K\right]}^T$ is fixed, a soft-thresholding operator as shown in Lemma 1 is applied to solve (8) to obtain $w$.

Lemma 1[11]

Consider the following optimization problem

$$\underset{u}{\mathrm{m}\mathrm{i}\mathrm{n}}-u^{T}a\text{s.t.}\parallel u{\parallel }_2^2\le 1,\parallel u{\parallel }_1\le \lambda .$$

The solution satisfies $u=\frac{s(a,\lambda )}{\parallel s(a,\lambda ){\parallel }_2}$, where $\lambda$ is a positive constant, $a\in R^{n\times 1}$, $s(a,\lambda )={\left[s_1,s_2,\dots ,s_n\right]}^T$, and $s_i=\mathrm{s}\mathrm{g}\mathrm{n}\left(a_i\right)\mathrm{m}\mathrm{a}\mathrm{x}\left(\left|a_i\right|-\lambda ,0\right)(i=1,2,\dots ,n)$.

When $w$ is fixed, as shown in Lemma 2, $V$ can be derived by referring to the method from Section 2.3 in [15].

Lemma 2[11, 15]

The solution of

$$\begin{array}{r}\hfill \underset{\parallel V{\parallel }_F=1}{\text{min}}-\sum _{k=1}^K{R}_k{v}_k+\sum _{k=1}^K{\lambda }_2\parallel v_k{\parallel }_1\\ \hfill +\sum _{k<k^{\prime }}a\parallel v_k-v_{k^{\prime }}{\parallel }_1,\end{array}$$ (10)

is given by $\hat{V}/\parallel \hat{V}{\parallel }_{F^{’}}$, and $\hat{V}$ is the optimum of

$$\begin{array}{r}\hfill \underset{V}{\min }\sum _{k=1}^K{‖v_k-R_k^T‖}_2^2+\sum _{k=1}^K{\lambda }_2{‖v_k‖}_1\\ \hfill +\sum _{k<k^{\prime }}a{‖v_k-v_{k^{\prime }}‖}_1,\end{array}$$ (11)

where $R_k=w^{\mathrm{T}}{X}_k^T{Z}_k$.

Problem (11) is a special case of the fused lasso model [24], and can be solved in three steps [25, 26]. Firstly, $\bar{V}$ is obtained by a fusion step, which fuses the features without significant difference (depending on $a$). The $i$-th element of variable $v$ (corresponding to the $i$-th feature of $Z$) will be fused between the $k\text{th}$ and the $k^{\prime }\text{th}$ classes if $\left|v_{ki}-v_{k^{\prime }i}\right|\le a$. Secondly, $\hat{V}=\left[{\hat{v}}_1,{\hat{v}}_2,\cdots ,{\hat{v}}_K\right]$ is derived through the soft-thresholding method; that is, ${\hat{v}}_i=S\left({\bar{v}}_i,{\lambda }_2\right)(i=1,2,\dots ,K)$. Finally, $V$ is obtained by a normalization step, $\hat{V}/\parallel \hat{V}{\parallel }_F$. The detailed algorithm to solve JSCoReg is given in Algorithm 1.

Algorithm 1.

Iterative Algorithm for JSCoReg

Input: Standardize data $X\in R^{n\times p},X_k\in R^{n_k\times p}$, $Z_k\in R^{n_k\times q},Y\in R^{n\times 1}\left(k=1,2,\cdots K\right)$.
Output: Canonical loadings $w$ and $v_k(k=1,2,\cdots K)$
1:	Initialize $w\in R^{p\times 1},v_k\in R^{q\times 1},k=1,2,\cdots K$
2:	Repeat
3:	$\overline{w}=\sum _{k=1}^K{v}_k^T{Z}_k^T{X}_k+X^{T}Y$
	$(X^{T}Y=c{\tilde{X}}^T,\text{when}K=2)$
4:	$\hat{w}=S\left(\bar{w},{\lambda }_1\right)$
5:	$w=\hat{w}/\parallel \hat{w}{\parallel }_2$
6:	$\begin{array}{l}\left[{\overline{v}}_1,{\overline{v}}_2,\cdots ,{\overline{v}}_K\right]=\arg \underset{\nu }{\min }\sum _{k=1}^K{‖v_k-{\left(w^T{X}_k{Z}_k\right)}^T‖}_2^2\\ +\sum _{k<k^{\prime }}a{‖v_k-v_{k^{\prime }}‖}_1\end{array}$
7:	for $k=1\cdots K$
8:	${\hat{v}}_k=S\left({\bar{v}}_k,{\lambda }_2\right)$
9:	end for
10:	$V=\left[{\hat{v}}_1,{\hat{v}}_2,\cdots ,{\hat{v}}_K\right]/{∥\left[{\hat{v}}_1,{\hat{v}}_2,\cdots ,{\hat{v}}_K\right]∥}_F$
11:	until Convergence

The step3 and step6 is the core of the Algorithm 1. The main method to solve step3 and step 6 have been presented in Lemma 1 and Lemma 2, respectively.

2.5. Parameters selection

There are four hyper-parameters ${\lambda }_1$, ${\lambda }_2$, $a$ and $c$ (when $K=2$) in the proposed model. ${\lambda }_1$ and ${\lambda }_2$ tune the sparsity degree of canonical loadings, $a$ controls the similarity between different genomic canonical loadings, and $c$ is the weight of phenotype(when $K=2$).

Although cross-validation is a widely used method in hyper-parameters’ selecting, our practical data may have a small sample size and is not suitable [27, 28]. As an alternative, we adopt a two-step stability selection method. In the first step, we use the sparsity level of the solution to guide the selection of ${\lambda }_1$ and ${\lambda }_2$ [29]. For large ${\lambda }_1$ and ${\lambda }_2$, the term ${\lambda }_1\Vert w{\Vert }_1+\sum _{k=1}^K{\lambda }_2{‖v_k‖}_1$ will dominate the objective function and be penalized severely during the optimization process. For small ${\lambda }_1$ and ${\lambda }_2$, the term ${\lambda }_1\Vert w{\Vert }_1+\sum _{k=1}^K{\lambda }_2{‖v_k‖}_1$ will play a subordinate role, and the solutions may not be sparse. In [15], Fang et al. tuned ${\lambda }_1$ and ${\lambda }_2$ based on the sample size $n$, yielding a much less sensitive searching procedure. In particular, we set ${\lambda }_1$ based on $k_w$, the number of nonzero elements in $w$. In each iteration, we adjust ${\lambda }_1$ through the following formulation:

$${\lambda }_1\in \left[|w{|}_{k_w+1},|w{|}_{k_w}\right],$$

where $|w{|}_{k_w}$ is the $k_w$-th largest element of $w$ in absolute value. Meanwhile, $k_v$ is used to determine ${\lambda }_2$ by using the same scheme.

In the second step, by fixing ${\lambda }_1$ and ${\lambda }_2$, we employ Bootstrapping method to determine $a$ and $c$. Parameter $a$ adjusts the fusion degree between multiple pairs of canonical loadings to encourage them to present significant class-specific structure. The fusion is realized by $l_1$ norm penalty on the prediction-error parameter between $v_k$ and $v_{k^{\prime }}$ (i.e., $a{\left|v_k-v_{k^{\prime }}\right|}_1$). By the $l_1$-norm penalty, the common and highly similar features between different classes are fused into an identical pattern, and the remaining features are the significant class-specific pattern. When $a=0$, there is no fusion across the canonical loadings, and our model reduces to SCoReg which is for a single class. When $a$ increases from 0 to $\mathrm{\infty }$, more features with less similarity are gradually fused into a same pattern. When $a=\mathrm{\infty }{v}_k$ and $v_{k^{\prime }}$ are completely fused into an identical pattern, and our model reduces to SCoReg in which all extracted features belong to a same class.

Given a data set $S$ with $n$ observations, a new data set $S^{\prime }$ is generated using Bootstrapping sampling, and then $S^{\prime }$ is treated as the training set and $S^{\prime c}$ is the testing set ($S^{\prime c}$, which contains the observations belonging to $S$ but not presented in $S^{\prime }$). Repeating Bootstrapping sampling for $T$ times, we obtain $T$ pairs of training and testing sets. Afterwards, we fix ${\lambda }_1$ and ${\lambda }_2$, and $a$ and $c$ are selected from a candidate set [10⁻², 10⁻¹, 10⁰, 10¹, 10²] based on the following optimization problem

$$\arg {\min }_{a,c}{\mathrm{\Delta }}_{corr}=\frac{1}{T}\sum _{t=1}^T\mid cor{r}_{train}^t-cor{r}_{test}^t\mid ,$$ (12)

where ${corr}_{train}^t$ is the Pearson correlation between the training sets of the two modalities and ${corr}_{test}^t$ is that of the testing sets. At the same time, the canonical loadings $w^t$ and $v_k^t$ consistent with the optimal parameters are collected. We calculate the probability of having voxels and SNPs by the following equation:

$$p_{wi}=\frac{1}{T}\sum _{t}I\left(\left|w_i^t\right|>0\right),$$ (13)

where $I$ is the indicator function. Then we select the important (a high probability of occurrence) voxels by

$$S_{wi}=\left\{i:p_{wi}>c_1\right\},$$ (14)

where $c_1$ is a given threshold. Similarly, we identify the important SNPs with a given threshold $c_2$.

3. Simulation

3.1. Simulation Setup

To evaluate the performance of JSCoReg model, we generate a dataset with $K$ classes. Each class consists of $n_k(k=1,2,\cdots ,K)$ samples containing fMRI data $X$ with $p$ voxels and SNP data $Z$ with $q$ SNPs. A latent method similar to the one used in [11,13] is applied here to simulate the correlation between $X$ and $Z$.

The dataset generation process consists of the following three steps. The first step constructs the latent variables. A latent variable $h_k~N(0,\delta )\left(h_k\in R^{n_k\times 1},k=1,2,\cdots K)\right.$ with normal distribution puts similar effect on the associated $X$ and $Z$ (here $n_k$ is the number of observations in the $k$-th class). Then $\alpha \in R^{1\times p}$ and ${\beta }_k\in R^{1\times q}$ are set as the canonical loadings of $X$ and $Z$ respectively. There are $p_1$ non-zero entries in $\alpha$ and $q_k$ non-zero entries in ${\beta }_k$. Dataset $X$ and $Z$ are generated using $h_k\alpha$ and $h_k{\beta }_k(k=1,2,\cdots ,K)$. Finally, Gaussian noise is introduced for each dataset, and we set $\frac{c_k}{n_k}$ ($\pm \frac{c}{n_k}$, when $K=2$) as phenotype for the $k$–th class.

In this simulation, we consider the data belonging to two classes. For each class, we generate 100 observations $\left(n_k=100,k=1,2\right)$, and each of observation has 500 features.

3.2. Simulation Results

To assess the performance of the proposed JSCoReg model, we calculated the true positive rate (TPR) and false positive rate (FPR), and compare the performance of our model (JSCoReg) with JSCCA, SCoReg(Combined), and SCoReg(Single). Fig. 2 displays the TPR against FPR on the selected features (x axis corresponds to the FPR values and y axis corresponds to the TPR values), and the area under the ROC curve (AUC) is presented in Table 1. Fig. 2(a) displays the ROC of the selected voxels. The ROC curve and AUC indicate that the JSCoReg model outperforms the JSCCA, SCoReg(Combined), and SCoReg(Single) model on the detection of canonical voxels. The reason is that the phenotypic variables effectively improves the accuracy of feature selection. Fig. 2(b)–(c) display the ROC of the selected case and healthy canonical SNPs, respectively. The ROC curve of our model is slightly higher than that of other models, and the value of AUC indicates that our model selects SNPs with higher accuracy.

Fig. 2.

A comparison of JSCoReg, JSCCA, SCoReg in selecting the features. The x axis corresponds to FPR value, and the y axis is the TPR value. The closer the curve to the top left, the better. (a) The ROC curve on the detection of canonical voxels. (b) The ROC curves on the detection of canonical SNPs from case class. (c) The ROC curves on the detection of canonical SNPs from healthy class.

Table 1.

A comparison of AUC of JSCoReg, JSCCA, SCoReg in selecting the features.

Method	AUC
	voxels	SNPs (case/healthy class)
JSCoReg	0.9915	0.9541/0.9543
JSCCA	0.9445	0.9133/0.9422
SCoReg(Combined)	0.9880	0.8753/0.9079
SCoReg(Single)	0.9907	0.9091/0.9122

By comparing the AUC of using extracted features, we can clearly draw a conclusion that our method has successfully identified significant features.

Then, we test the robustness of the proposed JSCoReg model under different noise levels. Normal distribution random noise is added to standardized dataset as

$$\hat{D}=\left(1+\sigma \mu \right)D,$$ (15)

where $\hat{D}$ is the noisy dataset, $D$ is the noise-free dataset, $\sigma$ is the noise level, and $\mu$ follows a standard normal distribution vector with zero mean and unit standard deviation. In simulation studies, two kinds of noise level $(\sigma =10\mathrm{\%}$ and $\sigma =20\mathrm{\%})$ are considered, and the experimental results are shown in Fig. 3 and Table 2. The experimental results illustrate that our method has good stability under two different noise level.

Fig. 3.

A comparison of the precision under different noise levels. (a) ROC curves on the detection of canonical voxels. (b) ROC curves on the detection of canonical SNPs from case class. (c) ROC curves on the detection of canonical SNPs from healthy class.

Table 2.

A comparison of AUC under different noise levels.

Noise level	AUC
	voxels	SNPs (case/healthy class)
noise-free	0.9915	0.9541/0.9543
noise1	0.9268	8983/0.9283
noise2	0.8775	0.8862/0.9035

Finally, we evaluate the parameter sensitivity of JSCoReg, and the results are presented in Fig. 4. In Fig. 4(a), parameters ${\lambda }_1$, ${\lambda }_2$ and $a$ are preset at their optimal values, and the influence of parameter $c$ is evaluated for the feature selection on synthetic dataset $X$. It is obvious that $c=30$ and $c=40$ outperform $c=0$ and $c=150$. As the value of $c$ increases from 0 to 40, the TPR increases synchronously. This indicates the importance of linking phenotype on improving the accuracy of feature selection. However, when $c=150$, the precision drops, due to the inappropriate incorporation of phenotype data. In short, Fig. 4 (a) indicates that the right use of phenotype is critical for feature selection.

Fig. 4.

A comparison of the precision under different parameter value. (a) ROC curves on the detection of canonical voxels with different parameter $c$. (b) ROC curves on the detection of SNPs from case class. (c) ROC curves on the detection of SNPs from healthy class.

Given the optimal values of parameters ${\lambda }_1$, ${\lambda }_2$ and $c$, we evaluate the influence of $a$ on the synthetic dataset $X$. The results are presented in Fig. 4(b) and Fig. 4(c). It is obvious that the results of using $a=10$ and $a=20$ outperform $a=0$ and $a=1000$. This is due to the fact that JSCoReg has no fusion effect on the canonical loadings of different classes when $a=0$, and when $a=1000$ JSCoReg model degenerates into a SCoReg model for one class. This indicates that the combination of fused lasso and $l_1$-norm has the ability to select class-related features.

4. Application to Schizophrenia Data Set

4.1. Data preparation and preprocessing

After validation via simulation, we apply JSCoReg to real data to find biomarkers associated with SZ. We test on the data from the Mind (Mental Illness and Neuroscience Discovery Institute) Clinical Imaging Consortium (MCIC) [19], which includes fMRI (containing information of 41,236 voxels) and SNP (including 777,635 SNPs) data of 184 subjects. Among these participants, 80 are SZ patients (34 ± 11 years old, 20 females) diagnosed based on the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) and the other 104 are healthy controls (32 ± 11 years old, 38 females). In this study, the phenotype $Y$ indicates case or controls samples. $(Y={\left[y_1,y_2,\cdots y_n\right]}^{\mathrm{T}},y_i\in \left\{-1,1\right\},i=1,\cdots ,n)$

The fMRI data were collected while subjects were performing a sensor motion task. The images were acquired using a Siemens3T Trio Scanner and 1.5T Sonata with echo-planar imaging (EPI) with parameters (TR = 2000ms, TE = 30 ms (3.0T)/40 ms(1.5T), field of view = 22cm, slice thickness = 4mm, 1mm skip, 27 slices, acquisition matrix = 64 × 64, flip angle = 90°). The data were pre-processed using SPM5 software, and then were realignment, spatial normalization, and smoothing. Next, data were analyzed by multiple regression considering the stimulus and their temporal derivatives plus an intercept term as repressors. After these steps, the dimension of fMRI data is 53*63*46. Then we select 41236 voxels from 116 brain regions based on the automated anatomical labeling (AAL) [30] brain atlas.

The SNPs data was obtained from each subject’s blood sample. Genotyping for all participants was performed at the Mind Research Network using the Illumina Infinium HumanOmni1-Quad assay covering 1,140,419 SNP loci. Then, the PLINK software package (http://pngu.mgh.harvard.edu/purcell/plink) was used to perform a series of standard quality control procedures, resulting in the final dataset spanning 777,635 SNP loci. Each SNP was categorized into three clusters and Genotypes BB (non-minor allele), AB(one minor allele), and AA(two minor alleles) were coded as 0, 1, and 2, respectively [43].

4.2. Results

We apply JSCoReg model to study fMRI and SNP datasets. The parameters selection method shown in “parameters selection” section was used to determine optimal parameters. Parameters $k_w$ and $k_v$ are set to 100, and $T$ is set to 200. The formulation (13) and (14) are used to select SNPs and voxels, and parameters $c_1$ and $c_1$ was preset as 0.2 and 0.15, respectively. We use AAL template to divide the brain into 116 Regions of Interests (ROIs), and present the identified ROIs and genes in Table 3 and Table 4.

Table 3.

Brain regions identified by JSCoReg

Model	Roi ID	Number of Voxel	Brain region
JSCoReg	77	3	Thalamus-L
	78	35	Thalamus-R
	91	12	Cerebellum-Crus1-L
	93	6	Cerebellum-Crus2-L
	94	4	Cerebellum-Crus1-R
	99	3	Cerebellum-6-L
	112	4	Vermis-6
	113	13	Vermis-7
	114	16	Vermis-8
	115	15	Vermis-9

Table 4.

Genes Identified from the SNP data

Number of Snp	Gene ID
34	CNOT4	TPD52	TNFSF18	ADAMTS5	DNM3	AKAP6
	DCLK2	MGST1	LOC644578	DOCK5	BAIAP2L1	HNF1B
	C8orf84	HLA-DPB2	KSRI	LHX2	LPL	FAM19A5
	MARCH1	FER	FAM136A	CDH2	NEURL1B	BSX
	LOC642340	LOC100289521	MYADML	LOC643563	DCC	MYO18B
	LOC100286951	PRPS1L1

Next, we use the significance test method to calculate the P-value of identified ROIs and genes. Assume $F=\left[f_1,f_2,\cdots ,f_{m1}\right]$ and $S=\left[s_1,s_2,\cdots s_{m2}\right]$ represent the selected voxels and SNPs using JSCoReg. The square mean correlation between these two types of datasets can be expressed as follows:

$${\rho }^{\text{*}}=\frac{1}{m_1{m}_2}\sum _{g=1}^{m_1}\sum _{h=1}^{m_2}{\left(\rho \left(f_g,s_h\right)\right)}^2$$ (16)

For a given data set $S$, we randomly change the order of the row vectors of the $S$, and repeat the process $\mathrm{\Delta }$ times. For each permutation, ${\rho }^{\mathrm{*}}$ is used as the null hypothesis of square correlation, and $\bar{\rho }$ is the new square correlation coefficient calculated by Eq. 14. The significance of the P-value can be estimated by

$$\text{P}-\text{value}=\frac{1}{\text{Δ}}\sum _{t}I\left(\overline{{\rho }_t}\ge {\rho }^{\text{*}}\right)$$ (17)

Finally, the P-value of the case and control group were calculated as 0.0083 and 0.0062, respectively. Therefore, we consider the selected voxels and SNPs are significant.

As shown in Table 3, 111 voxels in 10 ROIs are identified by JSCoReg. ROIs which have been reported to be associated with SZ are presented in bold fonts. For example, Thalamus (right) has been reported to play a crucial role in harmonizing the pass of information among brain regions [31, 32]. Many studies have shown the relationship between SZ and dysfunction of Thalamus [33, 34]. ROIs 91, 93, 94, and 99 are located at the declive of the Cerebellum, and the Culmen is related with many genes that have been reported to be related to SZ [35, 36]. The Cerebellum has been found associated with SZ by other independent studies [37].

As shown in Table 4. JSCoReg selects 34 SNPs located within 32 genes. Genes that have been reported in other studies to be relevant to SZ are shown in bold fonts. The gene DCC (Deleted in Colorectal Carcinoma, Ensemble id: ENSG00000187323) is connected to behavioral abnormalities, and it is demanded for mesocorticolimbic dopamine development (e.g., schizophrenia) [38, 39]. BSX and LPL are found related to the risk of developing SZ. The LPL gene is expressed in the brain regions related to cognitive functions and has been reported to be related to SZ [40]. BSX is expressed in the embryonic period and is a pivotal gene for brain structure development in the early phase [41].

To validate the correlation between identified voxels and case SNPs, we compute the Pearson correlation between the selected voxels and case SNPs by the JSCoReg model. The results are visualized in Fig. 5 for both SZ patients (left) and health controls (right) (the same y-axis is shown in both heat maps of Fig. 5). It is clear that the heatmap calculated from SZ patients has much higher absolute values compared to the one calculated from health controls, demonstrating that these class-related SNPs have a strong correlation with the obtained brain regions. It is worth noting that two heatmaps are generated using the SNPs selected from the case class, which indicates that JSCoReg model is an effective tool for class-specific feature selection.

Fig. 5.

The correlation heatmaps between selected voxels and SNPs. Left: the SZ patients; Right: the health controls. Y-axis displays the corresponding brain regions defined by AAL template; and X-axis displays the corresponding gene names. It is clear that the heatmap calculated from health controls has much higher absolute values compared to the one calculated from SZ patients.

Moreover, we use ConsensusPathDB [42] to test the biological significance of the selected risk SNPs. The Gene Ontology terms related to neural and brain activity enriched with P-value less than 0.01 are summarized in table 5. In table 5, ‘brain development’ and ‘cerebral cortex development’ are related to the brain activity, and ‘brain development’ has been pointed out in [15] to be related to brain disorders. ‘Neuron development’, ‘neuron to neuron synapse’ and ‘central nervous system development’ are related to neural activity. Through the above analysis, the biological significance of the risk genes selected by the JSCoReg model from SNPs has been distinctly demonstrated.

Table 5.

The enriched gene ontology terms that are related to the neural activity

term name	p-value
brain development	0.00983
cerebral cortex development	0.0081
neuron development	0.00815
neuron to neuron synapse	0.00756
central nervous system development	0.00495

4.3. Comparison with JSCCA and Group SCCA

JSCoReg, JSCCA, and Group SCCA are effective multivariate modeling tools in imaging genetics, and they provide data modeling ideas from different perspectives. By comparison, JSCoReg is a more generalized model. For example, JSCCA can be regarded as a special case of JSCoReg (phenotype is zero vector), and JSCoReg can be transformed into the JSCCA or SCoReg model by adjusting parameters. In practical application, according to the available phenotype and class difference information, it can be transformed to satisfy the actual application scenario and improve the accuracy of feature selection.

In the simulation experiment, we have verified the advantages of our model over the JSCCA and SCoReg in feature selection. More complex problems will be encountered in real application. As show in Fig. 6, we employed Venn diagram to display the results of these three methods. From the results of brain region selection, 18 brain regions were selected in group SCCA model, 4 brain regions were selected in the JSCCA model, and 10 brain regions were selected in our model. The bilateral Thalamus and Cerebellum related brain regions are selected in both our model and the Group SCCA model. From the results of gene selection, 25 genes were selected in the Group SCCA model, 14 genes in the Joint SCCA model, and 32 genes in our model. DCC genes were selected in both our model and the JSCCA model. Some of the selected biomarkers in our model have been reported by other similar studies. All these results show the reliability the advantage of our method.

Fig. 6.

The Venn diagram to compare the identified biomarkers. (a) The brain regions identified by JSCoReg and Group SCCA. (b) The genes identified by JSCoReg and JSCCA.

5. Conclusion and discussion

In this paper, we have proposed a novel JSCoReg model with a modified block coordinate descent algorithm for its solution. It has improved both JSCCA and SCoReg models by considering both shared and class-specific features among multiple classes (e.g., health and case groups). The effectiveness and reliability of our model have been illustrated in simulation in terms of False Positive Rate and Receiver operating characteristic curve. The result has shown that the JSCoReg model can achieve higher accuracy compared with JSCCA and SCoReg. For the real schizophrenia dataset, JSCoReg model can identify more significant biological and statistical biomarkers.

In regards to biomarker recognition, multi-view learning has many advantages over single view learning. Inspired by the CoReg model and JSCCA model, this work proposed a JSCoReg model to extract and select features (in this case, brain regions and SNPs) associate with a given phenotype while displaying class-specific information to facilitate the understanding of SZ. At the same time, many potential problems still need further study in application. On the one hand, a definite and robust parameter selection for the JSCoReg model remains an open problem. It is necessary to propose a robust parameter-selection-method to provide a optimal parameter set. We will continue to work on improving this. On the other hand, although the model is valid to extract features associate with a given phenotype, with the increase of model complexity, the possibility of overfitting also increases greatly. In future work, we will pursue a more concise penalty term or other sparse optimization models to extract class-specific information.