Preference Matrix Guided Sparse Canonical Correlation Analysis for Mining Brain Imaging Genetic Associations in Alzheimer’s Disease

Abstract

Investigating the relationship between genetic variation and phenotypic traits is a key issue in quantitative genetics. Specifically for Alzheimer’s disease, the association between genetic markers and quantitative traits remains vague while, once identified, will provide valuable guidance for the study and development of genetics-based treatment approaches. Currently, to analyze the association of two modalities, sparse canonical correlation analysis (SCCA) is commonly used to compute one sparse linear combination of the variable features for each modality, giving a pair of linear combination vectors in total that maximizes the cross-correlation between the analyzed modalities. One drawback of the plain SCCA model is that the existing findings and knowledge cannot be integrated into the model as priors to help extract interesting correlations as well as identify biologically meaningful genetic and phenotypic markers. To bridge this gap, we introduce preference matrix guided SCCA (PM-SCCA) that not only takes priors encoded as a preference matrix but also maintains computational simplicity. A simulation study and a real-data experiment are conducted to investigate the effectiveness of the model. Both experiments demonstrate that the proposed PM-SCCA model can capture not only genotype-phenotype correlation but also relevant features effectively.

1. Introduction

Investigating the relationship between genetic variation and phenotypic traits is a key issue in quantitative genetics. Specifically for Alzheimer’s disease, the association between genetic markers and quantitative traits (e.g., molecular, imaging and cognitive measures) remains vague while, once identified, will provide valuable guidance for the study and development of genetics-based treatment approaches.

However, considering the vast number of genetic markers such as single nucleotide polymorphisms (SNPs) and multimodal quantitative traits (QTs), it would be computationally expensive and less powerful to perform univariate testing on all possible SNP-QT pairs. Therefore, as an alternative method, canonical correlation analysis (CCA) is commonly used to detect multi-SNP-multi-QT associations between genotype and quantitative phenotype data in a single model [18, 19, 31, 49, 62]. A plain CCA computes two canonical weight vectors for each modality as linear combinations of feature variables in two modalities that maximize their cross-correlation. Despite all the advantages of CCA, its result is usually not informative: on one hand, the dimensions of SNP and QT datasets are usually extremely high, allowing infinitely many solutions to exist; on the other hand, the low signal-to-noise ratio in most SNP and QT datasets is directly reflected in the result weight vectors, thereby lowering the result interpretability in practice. To counter these drawbacks, sparse CCA (SCCA) is later proposed to reduce the number of variables selected in the canonical weight vectors [26]. While the results by SCCA are much more robust and interpretable in practice, the direct computation of SCCA is expensive, which makes it challenging to perform analysis given limited computational resource. For computational efficiency, in practice, a simplified version of SCCA (equivalent to sparse partial least square regression or SPLS) is often widely used through maximizing the cross-covariance of two canonical variables [50]. This simplification makes some additional assumptions on the problem that SCCA solves but transforms the fractional optimization problem into a constrained optimization problem with closed-form solutions for each subproblem, which reduces the computational cost.

One common disadvantage of these models is that the existing findings from past biomedical research cannot be integrated into the procedure of these models as priors. Nonetheless, the priors about relations between SNPs and QTs are valuable in variable selection and will offer more practical meaning and interpretability in the result by SCCA. In this work, we propose a new SCCA model, named as “preference matrix guided SCCA (PM-SCCA)”, which not only takes priors encoded as a preference matrix but also keeps computational simplicity by transforming it to the simplified SPLS version described above. The discussion about CCA models in the rest of the paper all refer to this simplified version. The preference matrix contains the numeric values of how preferred one SNP is deemed as associated with one QT as priors based on existing biological knowledge. A simulation study and a real-data experiment are conducted to investigate the effectiveness of the model. Both experiments demonstrate that the proposed PM-SCCA model is capable of capturing not only multi-SNP-multi-QT associations but also relevant genetic and phenotypic features effectively.

The subsequent sections of this article are structured as follows. Section 2 outlines the problem formation of the proposed PM-SCCA. In Section 3, we elaborate on the methodology employed for performing PM-SCCA. In Section 4, we present the results from the simulation study for assessing models’ performance as well as the results from the real-data experiment along with ensuing discussion over their biological validity. In Section 5, we summarize and conclude the paper.

2. Problem formulation

Let $\mathit{X}\in {ℝ}^{n\times p}$ be the genotyping SNP data and $\mathit{Y}\in {ℝ}^{n\times q}$ be the quantitative trait (QT) data (e.g., molecular, imaging, fluid and/or cognitive measures), where $n$, $p$ and $q$ are the numbers of participants, SNPs and QTs, respectively. Assume the features variables (i.e., columns) in $\mathit{X}$, $\mathit{Y}$ have been centered at zeros.

Canonical correlation analysis (CCA) aims to find a linear combination of features in $\mathit{X}$ and $\mathit{Y}$ to maximize the correlation between $\mathit{X}\mathit{u}$ and $\mathit{Y}\mathit{v}$. Mathematically, it finds the canonical weight vectors $\mathit{u}$, $\mathit{v}$ that satisfy:

$$\underset{\mathit{u}\text{,}\hspace{0.17em}\mathit{v}}{\text{maximize}}\frac{{\mathit{u}}^{\top }{\mathit{X}}^{\top }\mathit{Y}\mathit{v}}{\parallel \mathit{X}\mathit{u}{\parallel }_2\parallel \mathit{Y}\mathit{v}{\parallel }_2}\equiv \underset{\mathit{u}\text{,}\hspace{0.17em}\mathit{v}}{\text{maximize}}\frac{{\mathit{u}}^{\top }{\text{Σ}}_{\mathit{X}\mathit{Y}}\mathit{v}}{\sqrt{{\mathit{u}}^{\top }{\text{Σ}}_{\mathit{X}\mathit{X}}\mathit{u}{\mathit{v}}^{\top }{\text{Σ}}_{\mathit{Y}\mathit{Y}}\mathit{v}}}$$ (1)

It is customary to interpret ${\sum }_{\mathit{X}\mathit{X}}$ and ${\sum }_{\mathit{Y}\mathit{Y}}$ as diagonal matrices since the SNP and QT data typically exhibit both high dimensionality and insignificant correlation among features [42, 58, 59]. This simplification allows the fractional optimization problem to be reduced to a constrained optimization problem. Additionally, since sparsity on the loadings is desired for feature selection and practical interpretation in biological settings, we employ a simplified sparse CCA (SCCA), equivalent to sparse PLS (SPLS), for the following problem:

$$\begin{array}{c}\underset{\mathit{u},\mathit{v}}{\text{maximize}}\\ \text{subject}\hspace{0.17em}\text{to}\end{array}\begin{array}{l}\frac{1}{n-1}{\mathit{u}}^{\top }{\mathit{X}}^{\top }\mathit{Y}\mathit{v}\hfill \\ \begin{array}{l}\parallel \mathit{u}{\parallel }_2^2\le 1\text{,}\parallel \mathit{u}{\parallel }_1\le c_1\\ \parallel \mathit{v}{\parallel }_2^2\le 1\text{,}\parallel \mathit{v}{\parallel }_1\le c_2\end{array}\hfill \end{array}$$ (2)

where $c_1$, $c_2>0$ are the regularization parameters for sparsity and the objective function is often referred to as canonical covariance. In this paper, we introduce a new SCCA model, “preference matrix guided SCCA (PM-SCCA)”, and develop an algorithm based on the above computationally simplified model (i.e., Equation (2)). Thus PM-SCCA is formulated as

$$\begin{array}{c}\underset{\mathit{u}\text{,}\hspace{0.17em}\mathit{v}}{\text{maximize}}\\ \text{subject}\hspace{0.17em}\text{to}\end{array}\begin{array}{l}\text{(}1-\lambda \text{)}\cdot \frac{1}{n-1}{\mathit{u}}^{\top }{\mathit{X}}^{\top }\mathit{Y}\mathit{v}+\lambda \text{|}\mathit{u}{\text{|}}^{\top }E\text{|}\mathit{v}\text{|}\hfill \\ \begin{array}{l}\parallel \mathit{u}{\parallel }_2^2\le 1\text{,}\parallel \mathit{u}{\parallel }_1\le c_1\\ \parallel \mathit{v}{\parallel }_2^2\le 1\text{,}\parallel \mathit{v}{\parallel }_1\le c_2\end{array}\hfill \end{array}$$ (3)

where $c_1$, $c_2>0$ and $\lambda \in \text{[}0\text{,}\hspace{0.17em}1\text{)}$ are the regularization parameters, $\mathit{E}$ is the preference matrix with ${\mathit{E}}_{i\text{,}\hspace{0.17em}j}\ge 0\forall i$, $j$, and $|\cdot |$ stands for the elementwise absolute value. Each element in the preference matrix $\mathit{E}$ reflects user-specified preference level (based on prior biological knowledge) on the corresponding SNP-QT association, and therefore regularizes the magnitude of the elements of both canonical weight vectors, $\mathit{u}$ and $\mathit{v}$.

When $c_1=c_2=+\hspace{0.17em}\infty$, the sparsity constraints will not be active. In this case, the methods are referred to as CCA and PM-CCA respectively. In our empirical study, we will compare the proposed PM-SCCA and PM-CCA with SCCA and CCA using both simulation and read world data.

3. Methods

We employ an alternating optimization method to solve Equation (2). Alternating optimization is an iterative procedure that proceeds in two alternating steps: update of $\mathit{u}$ while holding $\mathit{v}$ fixed and update of $\mathit{v}$ while holding $\mathit{u}$ fixed.

3.1. Update of $\mathit{u}$ with $\mathit{v}$ fixed

Denote

$$a=\text{(}1-\lambda \text{)}\cdot \frac{1}{n-1}{\mathit{X}}^{\top }\mathit{Y}\mathit{v}\in {ℝ}^{p\times 1}\text{,}\hspace{1em}b=\lambda \mathit{E}\text{|}\mathit{v}\text{|}\in {ℝ}_{+}^{p\times 1}.$$

The optimization of Equation (3) with respect to $\mathit{u}$ can be written as

$$\begin{array}{ll}\underset{\mathit{u}}{\text{maximize}}\hfill & {\sum }_{i=1}^p\left(a_i{u}_i+b_i\left|u_i\right|\right)\hfill \\ \text{subject to}\hfill & \parallel \mathit{u}{\parallel }_2^2\le 1\text{,}\parallel \mathit{u}{\parallel }_1\le c_1.\hfill \end{array}$$ (4)

Notice that, since $b_i\ge 0$, it can be shown that the optimal $u_i$ should have the same sign as $a_i$. Otherwise, by reversing the sign of $u_i$, the objective value can always be increased. Thus, with substitution $\left|u_i\right|=\text{sign}\left(a_i\right)u_i$, the following problem is equivalent to Equation (4):

$$\begin{array}{ll}\underset{\mathit{u}}{\text{maximize}}\hfill & {\sum }_{i=1}^p\left(a_i+b_i\hspace{0.17em}\text{sign}\left(a_i\right)\right)u_i\hfill \\ \text{subject}\hspace{0.17em}\text{to}\hfill & \parallel \mathit{u}{\parallel }_2^2\le 1\text{,}\parallel \mathit{u}{\parallel }_1\le c_1.\hfill \end{array}$$ (5)

With

$$\mathit{w}=\left(\begin{array}{c}{a}_1+b_1\hspace{0.17em}\text{sign}\left(a_1\right)\\ a_2+b_2\hspace{0.17em}\text{sign}\left(a_2\right)\\ ⋮\\ a_p+b_p\hspace{0.17em}\text{sign}\left(a_p\right)\end{array}\right).$$

defined, Equation (5) can be expressed in a more compact form as

$$\underset{\mathit{u}}{\text{maximize}}\hspace{0.17em}{\mathit{w}}^{\top }\mathit{u}\hspace{1em}\text{subject}\hspace{0.17em}\text{to}\hspace{0.33em}\parallel \mathit{u}{\parallel }_2^2\le 1\text{,}\parallel \mathit{u}{\parallel }_1\le c_1.$$ (6)

Equation (6) can be solved by first using a soft-thresholding operator to shrink the elements in $w$ toward zero by a non-negative constant $\text{Λ}$ and then normalizing it to the unit ${\ell }_2$ ball. Specifically, let $\text{S(}a\text{,}\hspace{0.17em}\Lambda \text{)}$ be the soft-thresholding operator:

$$\text{S(}a,\hspace{0.17em}\Lambda \text{)}=\left\{\begin{array}{ll}a-\Lambda ,\hfill & a>\Lambda \hfill \\ a+\Lambda ,\hfill & a<-\Lambda \hfill \\ 0,\hfill & -\Lambda \le a\le \Lambda ,\hfill \end{array}\right.$$

where $\Lambda$ is a non-negative constant. Note that if a vector $a$ is input, then the soft-thresholding operator shall be applied with the same shrinking constant $\Lambda$ on every element of $a$. With this definition, the solution to Equation (6) can be written as

$${\mathit{u}}^{\ast }=\frac{S\left(w\text{,}\hspace{0.17em}{\Lambda }_1\right)}{{‖S\left(w\text{,}\hspace{0.17em}{\Lambda }_1\right)‖}_2}$$

3.2. Update of $\mathit{v}$ with $\mathit{u}$ fixed

Solving $\mathit{v}$ with $\mathit{u}$ fixed is the same as solving $\mathit{u}$ with $\mathit{v}$ fixed. Specifically, denote

$$\alpha =\text{(}1-\lambda \text{)}\cdot \frac{1}{n-1}{\mathit{Y}}^{\top }\mathit{X}\mathit{u}\in {ℝ}^{q\times 1}\text{,}\hspace{1em}\beta =\lambda {\mathit{E}}^{\top }\text{|}\mathit{u}\text{|}\in {ℝ}_{+}^{q\times 1}$$

The optimization of Eq. (3) with respect to $\mathit{v}$ can be written as

$$\begin{array}{cc}\underset{\mathit{v}}{\text{maximize}}& {\sum }_{j=1}^q\left({\alpha }_j{v}_j+{\beta }_j\left|v_j\right|\right)\\ \text{subject to}& \parallel \mathit{v}{\parallel }_2^2\le 1\text{,}\parallel \mathit{v}{\parallel }_1\le c_2.\end{array}$$ (7)

Since ${\beta }_j\ge 0$, the optimal ${\mathit{v}}_j$ should the same sign as ${\alpha }_j$ with the same reason as stated for $\mathit{u}$.

Define

$$\gamma =\left(\begin{array}{c}{\alpha }_1+{\beta }_1\hspace{0.17em}\text{sign}\left({\alpha }_1\right)\\ {\alpha }_2+{\beta }_2\hspace{0.17em}\text{sign}\left({\alpha }_2\right)\\ ⋮\\ {\alpha }_q+{\beta }_q\hspace{0.17em}\text{sign}\left({\alpha }_q\right)\end{array}\right).$$

Then, with similar techniques, the optimization problem to update $\mathit{v}$ with $\mathit{u}$ fixed can be boiled down to solving the following problems:

$$\underset{\mathit{v}}{\text{maximize}}\hspace{0.17em}{\mathbf{\gamma }}^{\top }\mathit{v}\hspace{1em}\text{subject to}\parallel \mathit{v}{\parallel }_2^2\le 1\text{,}\parallel \mathit{v}{\parallel }_1\le c_2\text{. }$$ (8)

Similarly, the solution to Equation (8) can be written as

$${\mathit{v}}^{\ast }=\frac{S\left(\gamma \text{,}\hspace{0.17em}{\Lambda }_2\right)}{\parallel S\left(\gamma \text{,}\hspace{0.17em}{\Lambda }_2\right){\parallel }_2}$$

The original Equation (3) has been reduced to two quadratically constrained linear programming optimization problems, Equation (6) and Equation (8). Notice that, with the ${\ell }_1$ constraint, the solution is generally not unique with the nonzero elements possible to assume any non-negative number that satisfies the ${\ell }_1$ constraint. Given an initial estimate for $\mathit{u}$ and $\mathit{v}$, the algorithm for the PM-SCCA alternately updates $\mathit{u}$ and $\mathit{v}$ iteratively until convergence, as outlined in Algorithm 1. Note that Algorithm 1 calls a Quadratically Constrained Linear Programming (QCLP) Solver [37], which is outlined in Algorithm 2.

For soft-thresholding operator used in QCLP solver (outlined in Algorithm 2), the parameter ${\Lambda }_1$ and ${\Lambda }_2$ can be chosen flexibly. If ${‖\frac{\text{S(}a\text{,}\hspace{0.17em}\text{Λ)}}{\parallel \text{S(}a\text{,}\hspace{0.17em}\text{Λ)}{\parallel }_2}‖}_1\le c$, then choose $\Lambda =0$. Otherwise, choose ${\Lambda }_1$ and ${\Lambda }_2$ according to the desired sparsity of the solution that fits within the ${\ell }_1$ upper bounds $c_1$ and $c_2$. Despite this common practice, we also allow $\Lambda$ in Algorithm 2 to be an optional input due to the possibility of tuning the parameter $c$ faster and more accurately without using a prespecified parameter grid of $c$ in certain cases.

Additionally, we see the preference matrix $\mathit{E}$ as well-bounded if

$$\frac{\text{max}\left\{{\sigma }_{\text{min}}^{\prime }\text{(}\mathit{X}\text{)}\parallel \mathit{Y}{\parallel }_2\text{,}{\sigma }_{\text{min}}^{\prime }\text{(}\mathit{Y}\text{)}\parallel \mathit{X}{\parallel }_2\right\}}{n-1}\le \parallel \mathit{E}\parallel \le \frac{\parallel \mathit{X}{\parallel }_2\parallel \mathit{Y}{\parallel }_2}{n-1}$$

If $\mathit{E}$ is not well-bounded, then it is advised to perform magnitude alignment as

$$\mathit{E}\leftarrow \left\{\begin{array}{ll}\frac{\mathit{E}\parallel {\mathit{X}}^{\top }\mathit{Y}{\parallel }_2}{\text{(}n-1\text{)}\parallel \mathit{E}{\parallel }_2}\hfill & \text{(A)}\hfill \\ \frac{\mathit{E}\left(\parallel \mathit{X}{\parallel }_2\parallel \mathit{Y}{\parallel }_2+\text{max}\left\{{\sigma }_{\text{min}}^{\prime }\text{(}\mathit{X}\text{)}\parallel \mathit{Y}{\parallel }_2\text{,}{\sigma }_{\text{min}}^{\prime }\text{(}\mathit{Y}\text{)}\parallel \mathit{X}{\parallel }_2\right\}\right)}{2\text{(}n-1\text{)}\parallel \mathit{E}{\parallel }_2}\hfill & \text{(B)}\hfill \end{array}\right.$$

where (A) is when ${\mathit{X}}^{\top }\mathit{Y}$ can be computed and (B) is otherwise. (B) is considered because ${\mathit{X}}^{\top }\mathit{Y}$ is computationally expensive and approximations are sometimes required. Here, ${\sigma }_{\text{min}}^{\prime }\left(A\right)$ denotes the smallest non-trivial spectral radius of $A$. It is specifically defined this way because the matrix is easily ill-conditioned and the minimum spectral radius to be used can be filtered with a deliberately selected cutoff.

In order to determine the optimal sets of hyperparameters $c_1$, $c_2$, $\lambda$, it is recommended to employ cross-validation. Depending on the specific task, readers may opt to utilize either canonical covariance or canonical correlation as the cross-validation score metric. For general purposes, we suggest using canonical covariance for cross-validation as it is employed as a fundamental part of the objective function for computation in the simplified CCA models. Furthermore, we conducted additional simulation experiments to compare the models’ performance between these two metrics, and we observed that the use of canonical covariance for cross-validation typically results in the selection of hyperparameter sets that produce testing canonical correlation values closer to the theoretical values. In the next subsection, we provide the convergence analysis.

Alternating minimization is one of the most commonly used strategies to solve these biconvex optimization problems. In practice, this is a balanced choice because it guarantees descent. Some other variant approaches can be explored (ADMM) for this objective function with guaranteed descent, although our algorithm addresses the problem while bearing simplicity. Some other commonly used approaches for both vanilla CCA and some CCA variants usually involve matrix inversion, which is computationally expensive in many settings with hyperparameter tuning that genetics problems usually involve and therefore is less practical.

$$\overline{\underset{¯}{\begin{array}{l}\underset{¯}{\mathbf{Algorithm}\mathbf{1}\hspace{0.33em}\text{PM-SCCA}\hspace{0.33em}\text{algorithm}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}}\\ \mathbf{Input}:\text{ Genotype}\hspace{0.17em}\text{data: }\mathit{X}\in {ℝ}^{n\times p}\text{; Imaging}\hspace{0.17em}\text{phenotype}\hspace{0.17em}\text{data: }\\ \hspace{1em}\hspace{1em}\mathit{Y}\in {ℝ}^{n\times q}\text{; Preference}\hspace{0.17em}\text{matrix}\hspace{0.33em}\mathit{E}\in {ℝ}_{+}^{p\times q}\text{, where }n\text{,}\hspace{0.17em}p\\ \hspace{1em}\hspace{1em}\text{and}\hspace{0.17em}q\hspace{0.17em}\text{are}\hspace{0.17em}\text{the}\hspace{0.17em}\text{numbers}\hspace{0.17em}\text{of}\hspace{0.17em}\text{observations,}\hspace{0.17em}\text{SNPs,}\hspace{0.17em}\text{and}\hspace{0.17em}\text{QTs,}\\ \hspace{1em}\hspace{1em}\text{respectively}.\\ \mathbf{Output}:\text{Loading}\hspace{0.17em}\text{weight}\hspace{0.17em}\text{vectors:}\hspace{0.17em}\widehat{\mathit{u}}\text{,}\hspace{0.17em}\widehat{\mathit{v}}\\ \hspace{1em}\text{1:}\hspace{0.17em}\text{Subtract the column mean from the data}\hspace{0.17em}\mathit{X}\text{,}\hspace{0.17em}\mathit{Y}\hspace{0.17em}\text{for}\hspace{0.17em}\text{feature}\\ \hspace{1em}\hspace{1em}\text{mean}\hspace{0.17em}\text{centering:}\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\mathit{X}\leftarrow \mathit{X}-\frac{1}{n}{\mathbf{1}}_n{\mathbf{1}}_n^{\top }\mathit{X}\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.33em}\mathit{Y}\leftarrow \mathit{Y}-\frac{1}{n}{\mathbf{1}}_{\mathit{n}}{\mathbf{1}}_{\mathit{n}}^{\top }\mathit{Y}\\ \hspace{1em}\hspace{1em}\hspace{1em}\text{{or}\hspace{0.17em}\text{use}\hspace{0.17em}\text{column-wise}\hspace{0.17em}\text{standardization}\hspace{0.17em}\text{as}\hspace{0.17em}\text{necessary}}\\ \hspace{1em}\text{2:}\hspace{0.17em}\text{Perform}\hspace{0.17em}\text{magnitude}\hspace{0.17em}\text{alignment}\hspace{0.17em}\text{to}\hspace{0.17em}\mathit{E}\hspace{0.17em}\text{if}\hspace{0.17em}\text{not}\hspace{0.17em}\text{well-bounded;}\\ \hspace{1em}\text{3:}\hspace{0.17em}\text{Select}\hspace{0.17em}\text{the}\hspace{0.17em}\text{hyperparameters}\hspace{0.17em}{c}_1\text{,}\hspace{0.17em}{c}_2\text{,}\hspace{0.17em}\lambda \hspace{0.17em}\text{via}\hspace{0.17em}\text{cross-validation;}\\ \hspace{1em}\text{4:}\hspace{0.17em}\text{Initialize}\hspace{0.17em}\widehat{\mathit{u}}\text{,}\hspace{0.17em}\widehat{\mathit{v}}\text{,}\hspace{0.17em}\mathit{w}\text{,}\hspace{0.17em}\gamma \text{;}\\ \hspace{1em}\text{5:}\hspace{0.17em}\mathbf{repeat}\\ \hspace{1em}\text{6:}\hspace{1em}\hspace{1em}\text{Update}\hspace{0.17em}\widehat{\mathit{u}}\hspace{0.17em}\text{with}\hspace{0.17em}\text{QCLP}\hspace{0.17em}\text{solver}\left(\mathit{w}\text{,}\hspace{0.17em}{c}_1{\text{,Λ}}_1\right)\text{; }\\ \hspace{1em}\text{7:}\hspace{1em}\hspace{1em}\text{Update}\hspace{0.17em}\widehat{\mathit{v}}\hspace{0.17em}\text{with}\hspace{0.17em}\text{QCLP}\hspace{0.17em}\text{solver}\left(\gamma \text{,}\hspace{0.17em}{c}_2\text{,}\hspace{0.17em}{\Lambda }_2\right)\text{;}\\ \hspace{1em}\text{8:}\hspace{0.17em}\mathbf{until}\hspace{0.17em}\text{convergence}\end{array}}}$$

3.3. Convergence Analysis

The main algorithm (Algorithm 1) is an alternating iterative algorithm that updates $\mathit{u}$ to its optimal solution in the iteration where $\mathit{v}$ is fixed and $\mathit{v}$ to its optimal solution in the iteration where $\mathit{u}$ is fixed. This strategy solves the biconvex problem Equation (3). Since the QCLP solver (Algorithm 2) relies on the partial subgradients of Equation (3), we show that our algorithm monotonously increases the value of the objective function to maximize within each iteration and therefore converges to a local extreme by proving that Equation (3) is upper bounded.

Proof:

Note that the following discussion holds within the domain $\parallel \mathit{u}{\parallel }_2^2\le 1$ and $\parallel \mathit{v}{\parallel }_2^2\le 1$. First, by bilinearity it is true

$$\overline{\underset{¯}{\begin{array}{l}\underset{¯}{\begin{array}{l}\mathbf{Algorithm}\mathbf{2}\hspace{0.33em}\text{Quadratically}\hspace{0.33em}\text{Constrained}\hspace{0.33em}\text{Linear}\hspace{0.33em}\text{Program-}\\ \text{ming}\hspace{0.17em}\text{(QCLP)}\hspace{0.17em}\text{Solver}\hspace{0.17em}\text{[37]}\end{array}}\\ \mathbf{Input}:\text{ Coefficient vector: }a\text{;}\hspace{1em}{\ell }_1\hspace{0.33em}\text{constraint:}\hspace{1em}c\text{; soft-}\\ \hspace{1em}\hspace{0.33em}\hspace{0.33em}\text{thresholding parameter: Λ}\\ \mathbf{Output}:\hspace{0.33em}\underset{x}{\text{argmax}\hspace{0.17em}}{a}^{\top }x\hspace{1em}\text{subject to}\parallel x{\parallel }_2^2\le 1\text{,}\parallel x{\parallel }_2\le c\text{. }\\ \hspace{0.33em}\hspace{0.33em}1\text{:}\hspace{0.33em}\mathcal{S}\leftarrow \left\{i\hspace{0.33em}\text{:}\hspace{0.33em}{\text{argmax}}_j\left|a_j\right|\right\}\\ \hspace{0.33em}\hspace{0.33em}\text{2:}\hspace{0.33em}\mathbf{if}\hspace{0.33em}0\le c<1\hspace{0.33em}\mathbf{then}\\ \hspace{0.33em}\hspace{0.33em}\text{3:}\hspace{1em}\text{return empty set;}\\ \hspace{0.33em}\hspace{0.33em}\text{4:}\hspace{0.33em}\mathbf{else}\mathbf{if}1<c<\sqrt{\text{|}\mathcal{S}\text{|}}\mathbf{then}\\ \hspace{0.33em}\hspace{0.33em}5\text{:}\\ \hspace{1em}\hspace{1em}\hspace{1em}{\left[{\mathbf{\text{x}}}^{\ast }\right]}_i\leftarrow \left\{\begin{array}{ll}\frac{c}{\text{|}\mathcal{S}\text{|}}\text{sign}\left(a_i\right)\text{,}\hfill & i\in \mathcal{S}\hfill \\ 0\text{,}\hfill & i\notin \mathcal{S}\hfill \end{array}\hspace{1em}\hspace{1em}\text{(9)}\right.\\ \hspace{0.33em}\hspace{0.33em}\text{6:}\hspace{0.33em}\mathbf{else}\\ \hspace{0.33em}\hspace{0.33em}7\text{:}\\ \hspace{1em}\hspace{1em}\hspace{1em}{\mathbf{\text{x}}}^{\ast }\leftarrow \frac{\text{S(}a\text{,}\hspace{0.17em}\text{Λ)}}{\parallel \text{S(}a\text{,}\hspace{0.17em}\text{Λ)}{\parallel }_2}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\text{(}10\text{)}\\ \hspace{0.33em}\hspace{0.33em}\text{8:}\hspace{0.33em}\hspace{0.33em}\mathbf{end}\hspace{0.33em}\mathbf{if}\end{array}}}$$

that ${\mathit{u}}^{\top }{\mathit{X}}^{\top }\mathit{Y}\mathit{v}\le {‖{\mathit{X}}^{\top }\mathit{Y}‖}_2<\infty$. Then, for any $\mathit{X}\in {ℝ}^{p\times q}$, $\text{|}\mathit{u}{\text{|}}^{\top }\mathit{X} \text{|}\mathit{v}\text{|}\le \parallel \text{|}\mathit{u}\text{|}{\parallel }_2\parallel \mathit{X} \text{|}\mathit{v}\text{|}{\parallel }_2\le \parallel \mathit{X}{\parallel }_2$.

• For equation (3) specifically, $\mathit{E}$ will only contain finite values. Thus, $\exists M\in ℝ$ such that $\parallel \mathit{E}{\parallel }_2\le \parallel \mathit{E}{\parallel }_F\le M<\infty$.

• If magnitude alignment on $\mathit{E}$ is performed for $\widehat{\mathit{E}}$ suggested, then it follows that

$$\begin{array}{l}\hspace{1em}\parallel \widehat{\mathit{E}}{\parallel }_2\\ \le \text{max}\left\{\frac{\parallel {\mathit{X}}^{\top }\mathit{Y}{\parallel }_2}{n-1}\text{,}\right.\\ \left.\hspace{1em}\hspace{1em}\frac{\mathit{E}\left(\parallel \mathit{X}{\parallel }_2\parallel \mathit{Y}{\parallel }_2+\text{max}\left\{{\sigma }_{\text{min}}^{\prime }\text{(}\mathit{X}\text{)}\parallel \mathit{Y}{\parallel }_2\text{,}\hspace{0.17em}{\sigma }_{\text{min}}^{\prime }\text{(}\mathit{Y}\text{)}\parallel \mathit{X}{\parallel }_2\right\}\right)}{2\text{(}n-1\text{)}\parallel \mathit{E}{\parallel }_2}\right\}\\ \le \frac{\parallel \mathit{X}{\parallel }_2\parallel \mathit{Y}{\parallel }_2}{n-1}<\infty \end{array}$$

Since $x$, $y<\infty$ if and only if $\lambda x+\text{(}1-\lambda \text{)}y<\infty$, $\forall \lambda \in \text{[}0\text{,}\hspace{0.17em}1\text{]}$, $x$, $y\in ℝ$, we proved Equation (3) is upper bounded.

Note that a biconvex problem is generally not solvable within polynomial time. Therefore, it is not practical to expect a global optimum to be found here unless an informed initialization is made. One common approach is to initialize with the left- and right-singular vectors of ${\mathit{X}}^{\top }\mathit{Y}$. This enables the algorithm to start from somewhere close to the desired optimum, facilitating the convergence within a lower number of iterations. These general principles remain applicable in our specific case. Whether a local optimum can be as effective as a global optimum depends on the particular task being addressed through this mathematical formulation. In the context of the imaging-genetics scenario explored in our paper, a local optimum can be equally efficient if the key variables are highly pronounced, while the trivial variables carry little to no weight. Our algorithm achieves this, as demonstrated by the verification through simulation experiments shown later.

4. Experimental results and discussions

4.1. Simulation study on synthetic data

We perform the simulation analyses to demonstrate that PM-SCCA can outperform SCCA in terms of prediction accuracy and CCA in terms of feature selection performance.

4.1.1. Generation of synthetic data

We assume the data $\mathit{X}\in {ℝ}^{n_0\times p}$ and $\mathit{Y}\in {ℝ}^{n_0\times q}$ contain $n_0$ i.i.d. observations of random vectors $x\in {ℝ}^p$ and $y\in {ℝ}^q$ respectively, with $n_0=2500$, $p=200$, and $q=100$.

The probabilistic generative model for tensor CCA (TCCA) can be adapted by treating a matrix as a first-order tensor to generate synthetic data. The generative model for the first-order TCCA takes canonical weights $\mathit{u}$, $\mathit{v}$, the targeted canonical correlation $\rho \ge 0$, the noise levels ${\eta }_x\ge 0$, ${\eta }_y\ge 0$, the means of the features ${\mu }_x$, ${\mu }_y$, and the number of observations $n_0$ to generate synthetic data. The model adapted to our simulation study is described in detail as follows. [41]

Given $\mathit{u}\in {ℝ}^p$ and $\mathit{v}\in {ℝ}^q$ specified as ground truth canonical weights for $\mathit{X}$ and $\mathit{Y}$, we compute

$$\left\{\begin{array}{l}\tilde{\mathit{u}}=\frac{\mathit{u}}{\parallel \mathit{u}{\parallel }_2^2}\hfill \\ \tilde{\mathit{v}}=\frac{\mathit{v}}{\parallel \mathit{v}{\parallel }_2^2}\hfill \end{array}\to \left\{\begin{array}{l}{\text{Σ}}_x=\tilde{\mathit{u}}{\tilde{\mathit{u}}}^{\top }+{\eta }_x\left(I_p-\parallel \mathit{u}{\parallel }_2^2\tilde{\mathit{u}}{\tilde{\mathit{u}}}^{\top }\right)\hfill \\ {\text{Σ}}_y=\tilde{\mathit{v}}{\tilde{\mathit{v}}}^{\top }+{\eta }_y\left(I_q-\parallel \mathit{v}{\parallel }_2^2\tilde{\mathit{v}}{\tilde{\mathit{v}}}^{\top }\right)\hfill \\ {\text{Σ}}_{xy}=\rho \tilde{\mathit{u}}{\tilde{\mathit{v}}}^{\top }\hfill \end{array}\right.\right.$$

Then, we generate $\mathit{X}$, $\mathit{Y}$ by row-stacking $n_0$ draws from

$$\left[x_1\text{,}\dots \text{,}\hspace{0.17em}{x}_p\text{,}\hspace{0.17em}{y}_1\text{,}\dots \text{,}\hspace{0.17em}{y}_q\right]\sim \mathcal{N}\left(\left[{\mathit{\mu }}_x\text{,}\hspace{0.17em}{\mathit{\mu }}_y\right]\text{,}\left[\begin{array}{cc}{\mathbf{\text{Σ}}}_x& {\mathbf{\text{Σ}}}_{xy}\\ {\mathbf{\text{Σ}}}_{xy}^{\top }& {\mathbf{\text{Σ}}}_y\end{array}\right]\right)$$

and separate the matrix as

$$\text{[}X\hspace{0.33em}\text{|}\hspace{0.33em}Y\text{]}=\left[\begin{array}{c}{x}_{1\text{,}1}\\ ⋮\\ x_{n_0\text{,}\hspace{0.17em}1}\end{array}\begin{array}{c}\cdots \\ \ddots \\ \cdots \end{array}\begin{array}{c}{x}_{1\text{,}\hspace{0.17em}p}\\ ⋮\\ x_{n_0\text{,}\hspace{0.17em}p}\end{array}\left.\begin{array}{l}\\ \\ \\ \end{array}\right|\begin{array}{c}{y}_{1\text{,}\hspace{0.17em}1}\\ ⋮\\ y_{n_0\text{,}\hspace{0.17em}1}\end{array}\begin{array}{c}\cdots \\ \ddots \\ \cdots \end{array}\begin{array}{c}{y}_{1\text{,}\hspace{0.17em}q}\\ ⋮\\ y_{n_0\text{,}\hspace{0.17em}q}\end{array}\right]$$

In our simulation study, we set the targeted canonical correlation as $\rho =\text{{}0.1\text{,}\hspace{0.17em}0.3\text{,}\hspace{0.17em}0.5\text{,}\hspace{0.17em}0.7\text{,}\hspace{0.17em}0.9\text{}}$, the noise levels as ${\eta }_x={\eta }_y=\text{{}0.001\text{,}\hspace{0.17em}0.005\text{,}\hspace{0.17em}0.01\text{,}\hspace{0.17em}0.05\text{,}\hspace{0.17em}0.1\text{,}\hspace{0.17em}0.5\text{,}\hspace{0.17em}1\text{}}$ and the means of the features ${\mu }_x={\mathbf{0}}_p$ and ${\mu }_y={\mathbf{0}}_q$. Note that in the rest of the article, when $\eta$ is used without subscript, it indicates $\eta ={\eta }_x={\eta }_y$ is set to the same value and therefore should represent the same as both appearing simultaneously.

We generate the preference matrix by, first, choosing which elements to be nonzero by random sampling and, then, drawing from a truncated normal $\hspace{0.17em}\psi \left({\mu }_{\mathit{E}}=1\text{,}\hspace{0.17em}{\sigma }_{\mathit{E}}^2={0.5}^2\text{,}\hspace{0.17em}0\text{,}+\infty \right)$ for the blocks between correlated features and the all other parts between uncorrelated feature respectively. As two weight groups in $\mathit{u}$ and two weight groups in $\mathit{v}$ are defined as non-trivial (Fig. 2, Theoretical), 2 × 2 = 4 nontrivial blocks in the covariance matrix of signals without noises between $\mathit{X}$ and $\mathit{Y}$ are expected. We use this pattern as the correlated blocks and the uncorrelated parts for the preference matrix simulation. In this study, 30% ∼ 40% of the elements in the correlated block (60% ∼ 70% false negative rate allowed) and 5% of the elements in the uncorrelated parts are randomly chosen to be nontrivial in the simulated preference matrix. This two-stage process allows not only different levels of quantitative confidence in feature associativity between modalities but also random false-positive and false-negative prior beliefs to exist, which brings the simulation closer to real situations.

Figure 2:

Estimated canonical weights of $\mathit{X}$ features ($\widehat{\mathit{u}}$, left) and $\mathit{Y}$ features ($\widehat{\mathit{v}}$, right) by PM-SCCA, PM-CCA, SCCA, and CCA on simulated data $\rho =0.3$, $\eta =0.01$.

4.1.2. Simulation results

With $n_0=2500$ observations in total, we use $n=2000$ for training-validation and $n_t=500$ for testing for the simulation study. For $\lambda$, we include 19 values ranging from 0.09 to 0.95 in increments of 0.05. For $c_1$, we include 10 values equally spaced between 1 and $\sqrt{200}$, and for $c_1$, we include 10 values equally spaced between 1 and $\sqrt{100}$. For each model, we first use 5-fold cross-validation for maximal validation canonical covariance to select optimal hyperparameters $c_1$, $c_2$, $\lambda$ on the validation dataset and, with the selected hyperparameters, re-train the models on the training-validation subset to obtain $\widehat{\mathit{u}}$, $\widehat{\mathit{v}}$, the estimated canonical weights of the features in $\mathit{X}$ and $\mathit{Y}$. Then, we calculate the canonical correlation (the expression in Eq (2)) both on the test subset ((C)Cor@Test) and on the full dataset ((C)Cor@Full). Additionally, we collect the canonical covariance on the validation datasets calculated during hyperparameter selection with cross-validation ((C)Cov@Val). We use canonical covariance from cross-validation because the simplified CCA models uses it as a part of the objective function for computation. This provides another robust rule on models’ generalizability. For each model, we execute this process for 100 times with 100 random seeds used in train-validation-test splitting and report the summary of the key metrics for model evaluation.

The proposed model has two primary objectives. First, a satisfactory level of similarity in feature weight distribution between theoretical weight vectors $\mathit{u}$, $\mathit{v}$ and weight vectors estimated $\widehat{\mathit{u}}$, $\widehat{\mathit{v}}$ is expected. This can be met if and only if the weight vectors estimated by our models demonstrate sufficient robustness against the datasets synthesized with theoretical weight vectors. We evaluate with CCov@Val and CCor@Test, of which higher values indicate better performance. Second, we aim to achieve a high level of proximity between $\widehat{\rho }=\text{CCorr}\left(\widehat{\mathit{u}}\text{,}\hspace{0.17em}\widehat{\mathit{v}}\text{,}\hspace{0.17em}{\mathit{X}}_{\text{Full}}\text{,}{\mathit{Y}}_{\text{Full}}\right)$ and $\rho =\text{CCorr}\left(\mathit{u}\text{,}\hspace{0.17em}\mathit{v}\text{,}\hspace{0.17em}{\mathit{X}}_{\text{Full}}\text{,}\hspace{0.17em}{\mathit{Y}}_{\text{Full}}\right)$. Here, note the actual synthetic datasets may have their canonical correlations, $\tilde{\rho }$, that are calculated by using the theoretical weight vectors on the actual synthetic datasets. Due to the randomness of the noises, $\tilde{\rho }$ may bear values that differ from but are close to $\rho$. Given our interest in obtaining the weight vectors that closely resemble the theoretical weight vectors, we regard $\tilde{\rho }$ as the optimal canonical correlation that can be recovered in this instance. Therefore, we use $\text{|}D\text{|}=\text{|}\tilde{\rho }-\widehat{\rho }\text{|}$ to calculate the absolute canonical correlation difference, where a lower value denotes superior performance.

Figure 1 shows the three metrics employed to assess the models’ performance for various combinations of $\rho$ and $\eta$. The PM-guided models, specifically PM-SCCA, are shown to achieve significantly lower absolute canonical correlation difference than non-PM-guided models in almost all cases. This indicates that PM-SCCA exhibits superior performance in recovering the theoretical canonical correlation between two datasets. For CCov@Val and CCor@Test, all four models behave similarly when the signal-noise ratio is high. However, as the noise parameter, $\eta$, increases, we find that both PM-guided models yield higher values on both CCov@Val and CCor@Test than the non-PM-guided models, with PM-SCCA leading the performance among four models. Thus, PM-SCCA demonstrates the best generalizability, followed by PM-CCA, provided the new datasets come from the similar distribution as the training datasets.

Figure 1:

The canonical covariance on the validation datasets (in the order of magnitude 1e-4), the absolute difference between the recovered canonical correlation and the actual canonical correlation on each synthetic datasets (mean and standard deviation), and the canonical correlation on the test datasets (mean) for PM-SCCA, PM-CCA, SCCA, and CCA with synthetic datasets generated with $\rho =\text{{}0.1\text{,}\hspace{0.17em}0.3\text{,}\hspace{0.17em}0.5\text{,}\hspace{0.17em}0.7\text{,}\hspace{0.17em}0.9\text{}}$ and $\eta =\text{{}0.001\text{,}\hspace{0.17em}0.005\text{,}\hspace{0.17em}0.01\text{}}$.

Tables 1 and 2 present the complete summary data used for Figure 1. For $\eta =0.01$, the estimation can only yield information when the signal-noise ratio is reasonably controlled. Specifically, we observe a change point for CCor@Test to assume non-trivial positive values at around $\rho =0.3$. Notably, only a subset of results from the experiments conducted are included in Figure 1, Table 1, and Table 2 because the analyses indicate that none of the four models correctly estimate weight vectors when $\eta \ge 0.05$, as evidenced by CCor@Test values all close to 0. Therefore, all results featuring $\eta >0.05$ are omitted.

Table 1.

The target canonical correlation, $\rho$, the noise parameters, $\eta$, and the actual canonical correlation, $\tilde{\rho }$, of the synthetic datasets used in the simulation study. The canonical covariance on the validation datasets (mean and standard deviation, in the order of magnitude 1e-4), the absolute difference between the recovered canonical correlations and the actual canonical correlation on each synthetic datasets (mean and standard deviation), and the canonical correlation on the test datasets (mean), given by PM-guided models.

PM-SCCA
$\rho$	$\eta$	$\tilde{\rho }$	${\widehat{\mu }}_{\text{Cov@Val}}$	${\widehat{\sigma }}_{\text{Cov@Val}}$	${\widehat{\mu }}_{\text{|D|}}$	${\widehat{\sigma }}_{\text{|D|}}$	${\widehat{\mu }}_{\text{Cor@Test}}$
0.1	0.001	0.0866	4.8237	1.1157	0.1188	0.0316	0.0672
0.1	0.005	0.1171	8.1206	1.5506	0.1230	0.0348	0.0742
0.1	0.01	0.0947	7.8513	1.2883	0.1109	0.0673	−0.0020
0.1	0.05	0.1047	39.1858	7.0949	0.1099	0.0637	0.0014
0.3	0.001	0.3259	25.8502	0.9592	0.0169	0.0020	0.3222
0.3	0.005	0.2995	21.2462	1.5817	0.0452	0.0167	0.2424
0.3	0.01	0.3022	13.4311	2.7865	0.0323	0.0255	0.0958
0.3	0.05	0.3106	38.6897	7.3042	0.1126	0.0745	0.0069
0.5	0.001	0.5191	44.1314	1.0696	0.0147	0.0009	0.5167
0.5	0.005	0.5030	39.8898	1.4643	0.0106	0.0047	0.4530
0.5	0.01	0.4938	34.2092	2.0177	0.0412	0.0144	0.3345
0.5	0.05	0.5023	38.3398	7.1229	0.3032	0.0783	−0.0024
0.7	0.001	0.6857	58.0269	1.1134	0.0089	0.0006	0.6861
0.7	0.005	0.7103	55.5742	1.0871	0.0102	0.0042	0.6637
0.7	0.01	0.7105	55.1928	1.7614	0.0616	0.0114	0.5880
0.7	0.05	0.7040	39.3355	6.7596	0.5292	0.0883	−0.0067
0.9	0.001	0.9016	74.9956	1.1269	0.0014	0.0002	0.8983
0.9	0.005	0.9006	75.7257	1.2390	0.0185	0.0032	0.8688
0.9	0.01	0.8956	71.4985	1.4331	0.0691	0.0082	0.7996
0.9	0.05	0.9040	38.0900	6.4746	0.7087	0.0830	−0.0018
PM-CCA
$\rho$	$\eta$	$\tilde{\rho }$	${\widehat{\mu }}_{\text{Cov@Val}}$	${\widehat{\sigma }}_{\text{Cov@Val}}$	${\widehat{\mu }}_{\text{|D|}}$	${\widehat{\sigma }}_{\text{|D|}}$	${\widehat{\mu }}_{\text{Cor@Test}}$
0.1	0.001	0.0866	4.6201	1.1718	0.1372	0.0232	0.0780
0.1	0.005	0.1171	5.8433	1.5535	0.1580	0.0357	0.0629
0.1	0.01	0.0947	4.1458	1.6818	0.2061	0.0613	−0.0042
0.1	0.05	0.1047	18.3390	7.7987	0.1889	0.0638	−0.0124
0.3	0.001	0.3259	24.7222	0.9418	0.0349	0.0019	0.3197
0.3	0.005	0.2995	20.1022	1.7429	0.0575	0.0146	0.2451
0.3	0.01	0.3022	9.5375	3.0880	0.0332	0.0194	0.0905
0.3	0.05	0.3106	20.0623	9.0711	0.0528	0.0283	0.0084
0.5	0.001	0.5191	43.1007	1.1098	0.0220	0.0010	0.5130
0.5	0.005	0.5030	36.4425	1.5393	0.0175	0.0059	0.4241
0.5	0.01	0.4938	30.4494	2.4106	0.0483	0.0203	0.3001
0.5	0.05	0.5023	17.7948	7.5620	0.2161	0.0664	0.0065
0.7	0.001	0.6857	57.1741	1.1266	0.0128	0.0007	0.6840
0.7	0.005	0.7103	53.1532	1.1317	0.0134	0.0045	0.6361
0.7	0.01	0.7105	51.1109	2.0419	0.0840	0.0140	0.5328
0.7	0.05	0.7040	21.0020	8.7994	0.4186	0.0627	−0.0119
0.9	0.001	0.9016	74.3326	1.1352	0.0013	0.0003	0.8968
0.9	0.005	0.9006	73.5624	1.3117	0.0359	0.0030	0.8400
0.9	0.01	0.8956	66.3297	1.6879	0.1053	0.0074	0.7333
0.9	0.05	0.9040	20.6521	8.7334	0.6105	0.0629	−0.0187

Table 2.

The target canonical correlation, $\rho$, the noise parameters, $\eta$, and the actual canonical correlation, $\tilde{\rho }$, of the synthetic datasets used in the simulation study. The canonical covariance on the validation datasets (mean and standard deviation, in the order of magnitude 1e-4), the absolute difference between the recovered canonical correlations and the actual canonical correlation on each synthetic datasets (mean and standard deviation), and the canonical correlation on the test datasets (mean), given by vanilla models.

SCCA
$\rho$	$\eta$	$\tilde{\rho }$	${\widehat{\mu }}_{\text{Cov@Val}}$	${\widehat{\sigma }}_{\text{Cov@Val}}$	${\widehat{\mu }}_{\text{|D|}}$	${\widehat{\sigma }}_{\text{|D|}}$	${\widehat{\mu }}_{\text{Cor@Test}}$
0.1	0.001	0.0866	4.4106	1.1160	0.1595	0.0131	0.0813
0.1	0.005	0.1171	5.8460	1.5001	0.1564	0.0467	0.0516
0.1	0.01	0.0947	5.8212	1.4679	0.1777	0.0717	−0.0146
0.1	0.05	0.1047	26.3407	6.4901	0.1703	0.0725	−0.0092
0.3	0.001	0.3259	24.4279	1.0466	0.0319	0.0039	0.3142
0.3	0.005	0.2995	15.4644	1.7130	0.0880	0.0043	0.1875
0.3	0.01	0.3022	7.4557	2.3430	0.0574	0.0314	0.0335
0.3	0.05	0.3106	25.6368	5.5683	0.0765	0.0682	−0.0061
0.5	0.001	0.5191	43.6410	1.0976	0.0190	0.0009	0.5153
0.5	0.005	0.5030	35.9321	1.6572	0.0145	0.0040	0.4160
0.5	0.01	0.4938	24.4337	2.5970	0.0463	0.0102	0.2240
0.5	0.05	0.5023	27.6384	6.1344	0.2529	0.0798	−0.0083
0.7	0.001	0.6857	57.7397	1.1227	0.0105	0.0005	0.6853
0.7	0.005	0.7103	52.0602	1.2847	0.0186	0.0037	0.6217
0.7	0.01	0.7105	46.1880	2.2393	0.1041	0.0096	0.4621
0.7	0.05	0.7040	27.5815	6.1522	0.4336	0.0788	−0.0061
0.9	0.001	0.9016	74.8590	1.1327	0.0014	0.0002	0.8978
0.9	0.005	0.9006	73.6035	1.3854	0.0381	0.0041	0.8375
0.9	0.01	0.8956	63.7250	1.8347	0.1323	0.0084	0.6866
0.9	0.05	0.9040	27.4097	7.2420	0.6370	0.0877	−0.0069
CCA
$\rho$	$\eta$	$\tilde{\rho }$	${\widehat{\mu }}_{\text{Cov@Val}}$	${\widehat{\sigma }}_{\text{Cov@Val}}$	${\widehat{\mu }}_{\text{|D|}}$	${\widehat{\sigma }}_{\text{|D|}}$	${\widehat{\mu }}_{\text{Cor@Test}}$
0.1	0.001	0.0866	4.3501	1.1402	0.1625	0.0083	0.0858
0.1	0.005	0.1171	4.5906	1.6026	0.1998	0.0051	0.0654
0.1	0.01	0.0947	2.8408	1.8024	0.2702	0.0117	−0.0227
0.1	0.05	0.1047	12.0889	8.7378	0.2709	0.0084	−0.0146
0.3	0.001	0.3259	23.8874	1.0227	0.0424	0.0017	0.3142
0.3	0.005	0.2995	15.2703	1.6853	0.0921	0.0029	0.1899
0.3	0.01	0.3022	4.9386	3.1409	0.0752	0.0077	0.0592
0.3	0.05	0.3106	11.3707	8.3313	0.0678	0.0073	−0.0045
0.5	0.001	0.5191	42.9288	1.1270	0.0234	0.0010	0.5124
0.5	0.005	0.5030	34.1525	1.6561	0.0223	0.0033	0.4001
0.5	0.01	0.4938	23.0776	2.6115	0.0375	0.0056	0.2258
0.5	0.05	0.5023	10.2840	8.2069	0.1236	0.0072	−0.0022
0.7	0.001	0.6857	57.1092	1.1292	0.0130	0.0007	0.6836
0.7	0.005	0.7103	51.0726	1.2596	0.0200	0.0033	0.6107
0.7	0.01	0.7105	44.3358	2.4120	0.1051	0.0068	0.4493
0.7	0.05	0.7040	13.2813	9.0978	0.3335	0.0091	−0.0309
0.9	0.001	0.9016	74.3255	1.1378	0.0013	0.0003	0.8967
0.9	0.005	0.9006	72.4614	1.4056	0.0462	0.0034	0.8237
0.9	0.01	0.8956	61.0916	1.9863	0.1435	0.0066	0.6554
0.9	0.05	0.9040	13.2115	9.3816	0.5322	0.0086	−0.0244

Figure 2 shows the feature selection performance with $\widehat{\mathit{u}}$ and $\widehat{\mathit{v}}$ by PM-SCCA, PM-CCA, SCCA, CCA with their respective optimal sets of hyperparameters $c_1$, $c_2$, $\lambda$ under the highest noise parameter $\eta =0.01$ and the lowest target canonical correlation $\rho =0.3$ with which all four models have non-trivial testing canonical correlation. Both PM-guided methods correctly assigned large weights to the non-trivial features and small to zero weights to the trivial features in ground truth vectors $\mathit{u}$, $\mathit{v}$. In contrast, the features SCCA selected scarcely agree with either trivial or non-trivial features in the theoretical ground truth. The results by CCA contain excessive noises and therefore provide very limited information for interpretation in practice. Taking into account what Figure 1 presents, we may assert that PM-guided models demonstrate superior performance in mitigating overfitting and therefore finding generalizable results when compared to non-PM-guided models.

4.2. Analysis of ADNI data

To demonstrate the usefulness of our proposed method in real data analysis, we aimed to mine the group-level associations between the genetic determinants and the brain region-level amyloid-beta deposition with the incorporation of biological prior knowledge. We conducted real data analysis using genotyping and positron emission tomography (PET) imaging with (18)F-AV-45 (florbetapir) obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) biobank database. To evaludate the performance of our proposed PM-SCCA model, we also conducted the same analyses using CCA, SCCA, PM-CCA as the benchmark methods.

4.2.1. Data description and preprocessing

The genotyping, PET imaging, and demographic data used in the preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (http://adni.loni.usc.edu) [44, 47, 48, 55, 56]. The ADNI was launched in 2003 as a public-private partnership, led by Principal Investigator Michael W. Weiner, MD. The primary goal of ADNI has been to test whether serial magnetic resonance imaging (MRI), positron emission tomography (PET), other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of mild cognitive impairment (MCI) and early Alzheimer’s disease (AD). Up-to-date information about the ADNI is available at www.adni-info.org.

We downloaded genotyping data from ADNI 1, GO, 2, and 3 studies. We aligned and integrated the downloaded data using Homo sapiens (human) genome assembly NCBI37 (hg19) genome builder. We performed the strand alignment according to 1000 Genome phase 3 [1] using McCarthy Group Tools (https://www.well.ox.ac.uk/~wrayner/tools/). We imputed the genotyping data using the Michigan Imputation Server [17] with 1000 Genome phase 3 reference panel of European ancestry. We annotated our imputed genotyping data using ANNOVAR [53]. After alignment and imputation, we performed the quality control (QC) using the following criteria: genotyping call rate > 98%, minor allele frequency > 0.1%, Hardy-Weinberg Equilibrium > 1e-6, missingness per individual < 5%. The QC was performed using PLINK 1.9 [11]. To validate our PM-SCCA model, we prioritized 87 single nucleotide polymorphisms (SNPs) based on their associations with Alzheimer’s disease (AD). Specifically, we downloaded the genome-wide association study (GWAS) summary statistics of the major AD genetics landmark study, Bellenguez et al. [8], from NHGRI-EBI GWAS Catalog [10]. We prioritized 47 AD-related SNPs with the smallest GWAS association $p‐\mathit{v}al\mathit{u}e\text{s }$ and 40 AD-unrelated SNPs with the largest association $p‐\mathit{v}al\mathit{u}e\text{s }$, totaling 87 SNP markers.

The PET imaging with (18)F-AV-45 (florbetapir) regional endophenotype quantitative traits (QTs) were pre-processed by the ADNI PET core [28] and directly downloaded from the LONI website. Specifically, the original florbetapir data were downloaded from the LONI³ in the most fully pre-processed format. Each florbetapir image is co-registered using SPM8 [3] to that subject’s magnetic resonance imaging (MRI) image that was closest in time. FreeSurfer [20] processing was carried out to skull-strip, segment, and delineate cortical and subcortical regions in all MRI scans. Then, the volume-weighted florbetapir means were extracted from a cortical summary region that includes frontal, anterior/posterior cingulate, lateral parietal, and lateral temporal regions according to their FreeSurfer naming convention. Standardized uptake value ratios (SUVRs) [38, 52] are provided. In our study, we further normalized the summary SUVR based on the whole cerebellum reference region [30, 13] as recommended for the cross-sectional studies [35, 9, 12]. We extracted the first visit of each subject for the subsequent analysis. After preprocessing, our (18)F-AV-45 (florbetapir) endophenotype consists of 68 SUVRs of the global cortical target (CTX) regions and 955 subjects. Among 68 regions, 24 regions were found to be AD-related by previous studies [21].

We constructed a binary preference matrix for our PM-SCCA model based on the relevance of the corresponding features to AD. Specifically, we employed a 68-by-87 dimensional binary matrix in which “1” indicates the corresponding row (regional CTX SUVR (18)F-AV-45 florbetapir imaging QT) and column (genetic determinants) were identified to be influential to AD.

We downloaded the demographic information from an Alzheimer’s Disease Modelling Challenge, named QT-PAD project (http://www.pi4cs.org/qt-pad-challenge). We summarized the population characteristics in Table 3

Table 3. Participant characteristics.

We summarized the number of subjects in each diagnosis group along with their age and sex information. In particular, we reported the mean ± standard deviation (mean ± sd) for the age and the number of females/males (F/M) of all subjects within each diagnosis group. Abbreviations: CN: cognitive normal; SMC: subjective memory complaint; EMCI: early mild cognitive impairment; LMCI: late mild cognitive impairment; AD: Alzheimer’s disease; sd: standard deviation; F/M: females/males.

Diagnosis	CN	SMC	EMCI	LMCI	AD	Overall
Number	237	96	272	225	125	955
Age (mean ± sd)	74.61 ± 5.53	72.26 ± 5.76	71.10 ± 7.42	72.93 ± 7.45	74.48 ± 8.40	72.96 ± 7.12
Sex (F/M)	108∕129	58∕38	118∕154	89∕136	51∕74	424∕531

4.2.2. Results and discussion

AD is a complex and multifaceted neurodegenerative disorder [45, 46, 57] while its underlying neuropathological mechanism remains largely unknown. Despite many studies investigating the case/control status of AD relying on a single modality [32, 33, 34], studying AD from the imaging-genetics perspective may enhance the statistical power [5] and may provide new insights into the phenotypic characteristics and genetic mechanisms of the brain [7, 6]. Borrowing the power of the amyloid pathogenesis hypothesis of AD [21, 27], one promising approach to understanding the mechanism of AD is to identify group-level associations between regional amyloid-beta deposition and genetic markers. To facilitate this, we constructed two datasets containing AD-related and AD-unrelated features, respectively. Using a preference matrix, we employed the PM-SCCA model and identified pairs of features that exhibited significant AD-relatedness, enabling us to search for group-level imaging-genetics associations (Figure 3). To evaluate our model, we also employed the baseline CCA, SCCA, and PM-CCA models for comparison.

Figure 3: Framework of analysis of ADNI data.

The study of AD is complex and multifaceted [57]. One promising approach to understanding the mechanism of AD is the discovery of group-level associations between regional amyloid-beta deposition and genetic markers. To facilitate this, we constructed two datasets containing AD-related and AD-unrelated features, respectively. Using a preference matrix, we identified pairs of features that exhibited significant AD-relatedness, enabling us to search for group-level imaging-genetics associations. Abbreviations: AD: Alzheimer’s disease; SUVR: standardized uptake value ratios; CTX: global cortical target; SNPs: single nucleotide polymorphisms.

To obtain robust and replicable group-level imaging-genetics associations, we employed the cross-validation and hyperparameter tuning process. Specifically, we equally divided 955 subjects into three folds where two folds were treated as a training-validation dataset and the third fold was treated as an independent testing set. To tune the hyperparameters $c_1$, $c_2$, and $\lambda$, we employed a three-fold cross-validation procedure within the training-validation dataset. For $c_1$, we tested ten candidate values equally spaced between 1 and $\sqrt{68}$; For $c_2$, we tested ten values equally spaced between 1 and $\sqrt{84}$; and For $\lambda$, we tested values ranging from 0.05 to 0.95 in increments of 0.05. We trained our models on two folds of the training-validation dataset and then validated them by applying the weights to the remaining fold of data and calculating the absolute value of the canonical correlation. We repeated this process for all possible combinations of the three folds and selected the hyperparameter set that produced the highest average absolute validation canonical correlation. We then retrained our models with the best-tuned hyperparameters on the entire training-validation dataset and reported the resulting canonical correlation as the training correlation. To evaluate the generalizability of our models, we applied the weights from the training model to the independent testing dataset to obtain the testing canonical correlation. We repeated this entire process 100 times using 100 different random seeds to split the subjects into folds. Finally, we reported the mean and standard error of the training and testing canonical correlations across the 100 random splits.

In summary, our PM-SCCA model outperforms the other baseline models (Table 4). The preference matrix guided models demonstrate lower training canonical correlations and higher testing canonical correlations compared to the model without the incorporation of the preference matrix, indicating that the preference matrix can aid models in reducing overfitting. Furthermore, when comparing the PM-SCCA with PM-CCA, the incorporation of sparsity enables the model to produce more robust results (with lower standard error) and even higher training and testing canonical correlations.

Table 4. Training and testing canonical correlations in the analysis of ADNI data.

We used cross-validation and hyperparameter tuning to obtain reliable imaging-genetics associations. We tuned the hyperparameters using three-fold cross-validation. We conducted the training-testing process 100 times and reported mean and standard error of the training and testing canonical correlations across the 100 random splits. Abbreviations: CCA: canonical correlation analysis; SCCA: sparse canonical correlation analysis; PM-CCA: preference matrix guided canonical correlation analysis; PM-SCCA: preference matrix guided sparse canonical correlation analysis.

Methods	CCA	SCCA	PM-CCA	PM-SCCA
Training	0.2737 (±0.0016)	0.2702 (±0.0018)	0.2277 (±0.0021)	0.2424 (±0.0022)
Testing	0.1676 (±0.0041)	0.1676 (±0.0042)	0.1759 (±0.0046)	0.1799 (±0.0044)

We compared the association matrices reconstructed from the weights learned by the four CCA models that were used to project the regional (18)F-AV-45 QTs and SNP markers to obtain the canonical correlations (Figure 4). For each of the four models, the association matrix was calculated by averaging the corresponding matrices across the 100 random splits. Our PM-SCCA model achieved the best biological interpretability. Specifically, on one hand, the CCA and SCCA models selected multiple AD-unrelated features. On the other hand, although the PM-CCA was able to select AD-related genetic markers and do further prioritization, the algorithm cannot prioritize the regional (18)F-AV-45 imaging QTs to relevant genetic features. Our PM-SCCA achieves the two-dimensional feature selections by leveraging the information from the preference matrix.

Figure 4: Comparison of group-level imaging-genetics associations.

The group-level imaging-genetics associations can provide a more comprehensive understanding of the pathological mechanisms for AD [61]. We compared the association matrices reconstructed from the weights learned by the four CCA models that were used to project the regional (18)F-AV-45 QTs and SNP markers to obtain the canonical correlations. For each of the four models, the association matrix was calculated by averaging the corresponding matrices across the 100 random splits. Abbreviations: CCA: canonical correlation analysis; SCCA: sparse canonical correlation analysis; PM-CCA: preference matrix guided canonical correlation analysis; PM-SCCA: preference matrix guided sparse canonical correlation analysis; SNP: single nucleotide polymorphisms; QTs: quantitative traits.

We annotated brain regions that are highly associated with AD genetic markers to investigate the brain functions that are mostly associated with genetics. The absolute value of the group-level imaging-genetics association matrix was summed up across all SNPs and resulted in a vector of length 68, where each element represents an aggregated genetic relatedness of the corresponding imaging QT (Figure 5). We then used the resulting vector to functionally annotate brain regions through Neurosynth [64] and NeuroVault [23]. Our brain functional annotation results indicate that the prefrontal cortex area is most closely correlated with the AD genetic marker prioritized regions. In the previous study involving 17 normal subjects engaging in a transitive inference task with determinate and indeterminate relations, the prefrontal cortex is suggested to play an essential role in resolving indeterminate relations [22]. Additionally, a functional magnetic resonance imaging (MRI) study indicates the neural representations of different dimensions of emotion (valence, intensity, and recognition) are closely related to different regions in the prefrontal cortex [24]. These results provide biological validation for our identified group-level imaging-genetics associations, showing that the functions associated with imaging ROIs mostly associated with genetic markers are significantly linked to AD.

Figure 5: Genetic relatedness of 68 imaging QTs summarized from group-level imaging-genetics association matrix.

We constructed the region-level genetic relatedness of 68 (18)F-AV-45 imaging QTs by summing up the absolute value of the group-level imaging-genetics association matrix across all SNPs. The darker red color represents that the corresponding regions are more genetically associated. (A) Genetic relatedness of 68 imaging QTs on the left hemisphere; (B) The axial view of genetic relatedness of 68 imaging QTs on the whole brain; (C) Genetic relatedness of 68 imaging QTs on the right hemisphere. Abbreviations: SNP: single nucleotide polymorphisms; QTs: quantitative traits.

We further validated our group-level imaging-genetics associations from the genetics perspective. We summed up the absolute value of the group-level imaging-genetics association matrix across the (18)F-AV-45 imaging dimension, resulting in a vector of aggregated imaging endophenotype relatedness of corresponding genetic markers with length 87. We then filtered out the genetic markers with low aggregated imaging relatedness. After the filtering, we obtain 47 genetic markers and all of them are AD-related. Next, we further prioritized 16 SNPs with aggregated imaging relatedness greater than one standard deviation of the mean aggregated imaging relatedness and mapped them to their nearest genes using ANNOVAR [53]. We conducted the gene set enrichment analysis using Gene Ontology [14, 16, 4] and PANTHER [15, 40, 39]. As a result, we found two pathways that have been significantly enriched: the apoptosis signaling pathway (with fold enrichment 27.45) and the epidermal growth factor receptor signaling pathway (with fold enrichment 24.17). From the previous experimental study, we found that the amyloid-beta is able to activate the caspase signaling pathway and induce significant apoptosis in primary cultured cerebral cortical neurons [25], whose mechanism is later shown to be preceded by selective alterations in the expression of the apoptosis-related genes [63]. On the other hand, the epidermal growth factor receptor signaling pathway has been found to be a preferred target for treating amyloid-beta–induced memory loss [54] and preventing APOE4 and amyloid-beta-induced cognitive and cerebrovascular deficits in female mice [51]. These findings support the biological validity of the genetic markers selected by our PM-SCCA model.

5. Conclusion

In summary, we have proposed a preference matrix guided sparse canonical correlation analysis (PM-SCCA) model to investigate the relationship between genetic variations such as SNPs and imaging quantitative traits (QTs). Our model can detect biologically meaningful multi-SNP-mulit-QT associations and has the potential to provide new insights into AD. The incorporation of the preference matrix can facilitate the feature selection, where features proven to be related to AD are encouraged to be selected. Moreover, sparsity regularization can further achieve feature selections within the AD-related feature group. Our simulation study demonstrated that our models could both select the nontrivial features and recover the theoretical canonical correlation without overfitting. Our real data analysis demonstrated that our model could identify biologically interpretable features.

Our PM-SCCA is a useful statistical method to understand the relationship between two sets of variables without lossing biological interpretability. Our PM-SCCA model can encourage the selection of features that are in the same pathways and can further reveal the intricate interplay between selected features. As such, prioritizing the features that share the strong correlations between regional brain Amyloid-beta deposition and genetics is just one application of our PM-SCCA model. However, the utility of PM-SCCA extends beyond this. It can identify the gene expression profiles that are associated to different subtypes of the diffuse large B-cell lymphoma [2, 36, 43, 60], and discover robust associations between blood cell counts and protein abundance using proteomics and methylomics data across multiple cohorts [29].

Our PM-SCCA study has potential limitations. First, our study fixed 60% − 70% false negative rate in the preference matrix and designed the experiments to investigate how the target canonical correlation and noise level in the datasets may affect the models. The model performance with PM false negative rate, target CC, and noise level all varying can be further investigated in the future. Second, our PM-SCCA model can only leverage the information from one preference matrix at a time. the SCCA models that can simultaneously borrow information from multiple levels of prior knowledge are needed to mine biologically interpretable multi-SNP-multi-QT associations from multiple perspectives.

Highlights.

• Canonical correlation analysis guided by a matrix with confidence in feature linkages.

• PM-guided sparse CCA overpowers prototypes in both generalizability and CC recovery.

• PM-SCCA achieves bimodal feature selection; computationally feasible while accurate.

• Detects PET-gene associations, biologically validated by functional annotation & GSEA.