aNy-way Independent Component Analysis

Abstract

Multimodal data fusion is a topic of great interest. Several fusion methods have been proposed to investigate coherent patterns and corresponding linkages across modalities, such as joint independent component analysis (jICA), multiset canonical correlation analysis (mCCA), mCCA+jICA, disjoint subspace using ICA (DS-ICA) and parallel ICA. JICA exploits source independence but assumes shared loading parameters. MCCA maximizes correlation linkage across modalities directly but is limited to orthogonal features. While there is no theoretical limit to the number of modalities analyzed together by jICA, mCCA, or the two-step approach mCCA+jICA, these approaches can only extract common features and require the same number of sources/components for all modalities. On the other hand, DS-ICA and parallel ICA can identify both common and distinct features but are limited to two modalities. DS-ICA assumes shared loading parameters among common features, which works well when links are strong. Parallel ICA simultaneously maximizes correlation between modalities and independence of sources, while allowing different number of sources for each modality. However, only a very limited number of modalities and linkage pairs can be optimized. To overcome these limitations, we propose aNy-way ICA, a new model to simultaneously maximize the independence of sources and correlations across modalities. aNy-way ICA combines infomax ICA and Gaussian independent vector analysis (IVA-G) via a shared weight matrix model without orthogonality constraints. Simulation results demonstrate that aNy-way ICA not only accurately recovers sources and loadings, but also the true covariance/linkage patterns, whether different modalities have the same or different number of sources. Moreover, aNy-way ICA outperforms mCCA and mCCA+jICA in terms of source and loading recovery accuracy, especially under noisy conditions.

I. Introduction

With the fast advancement of technology, growing medical data from different imaging modalities and omics (e.g. genomics, transcriptomics, epigenomics, microbiomics) can be collected from the same participants for a variety of diseases. Data from different modalities/omics provide unique and complementary features for understanding diseases. Thus, integrative/fusion methods are essential to identify unique and coherent biomarkers across modalities.

Several fusion approaches have been proposed to explore the latent shared structures in multimodal datasets in order to discover and establish links among modalities. To that end, joint independent component analysis (jICA) [1] has focused on identifying shared loadings across modalities. Meanwhile, mCCA+jICA [2] proposed a multiset canonical correlation analysis (mCCA) preprocessing in order to improve the correspondence over modalities and more closely satisfy the assumption of shared identical loadings prior to jICA, enabling N-way multimodal fusion. With similar motivation, disjoint subspace using ICA (DS-ICA) [3] proposed to apply jICA on the common subspace extracted with principal component analysis-CCA (PCA-CCA) [4], and separate ICAs on the remaining (distinct) subspace. Parallel ICA [5], on the other hand, proposed to interweave separate (parallel) ICAs, one per modality, with simultaneous maximization of correlation between specific multimodal loading pairs. This is attractive because both independence and linkage are optimized together rather than in two steps. However, Parallel ICA (and 3-way Parallel ICA) [5, 6] is limited in the number of modalities and linked components it can robustly detect since it optimizes individual correlation pairs rather than the entire underlying linkage structure.

Here, we propose a new model called aNy-way ICA, which overcomes these limitations. aNy-way ICA optimizes the entire loadings correlation structure of linked components via Gaussian independent vector analysis (IVA-G) [7] and simultaneously optimizes independence via separate (parallel) ICAs. Therefore, aNy-way ICA is capable of detecting multiple linked sources over any number of modalities without requiring orthogonality constraints on sources and permitting different number of sources for different modalities. The remainder of the paper is organized as follows. Section II introduces the aNy-way ICA model. Section III describes simulation scenarios. Results and conclusions are shown in sections IV and V, respectively.

II. METHOD

aNy-way ICA aims to simultaneously maximize the independence of sources s while minimizing the mutual information between subspace component vectors (SCV) a_k. As shown in Fig. 1, without loss of generality, we consider three modalities: structural MRI (sMRI), functional MRI (fMRI) and electroencephalography (EEG), each with 5, 4, and 6 sources, respectively. While the total number of sources C_m is different per modality, at most K = 4 SCVs (maximum number of one-to-one correspondences across all modalities) can be extracted. For each modality, ICA decomposes the data X_m (dimension: N × L_m; m = 1,2,3 modality indices; N is the number of subjects; L_m is the feature dimensionality) into a source matrix S_m (dimension: C_m × L_m) with corresponding loading matrices A_m (dimension: N × C_m), i.e., X_m = A_mS_m.

Figure 1.

Diagram of aNy-way ICA. Sources S_m are estimated by maximization of independence in each modality separately. Corresponding loadings A_m are then organized into SCVs ak, followed by minimization of their mutual information with IVA-G, which amounts to maximization of the correlation structure within the SCV without orthogonality constraints [8]. The model is optimized with stochastic gradient descent until convergence.

The source matrix is estimated by the product of a weight matrix W_m and data as S_m = W_mUX_m, where U is a fixed projection into a shared variance-adjusted space (see below). The loading matrix is then reconstructed as the product of data and pseudo-inverse of the source matrix $\left(A_m=X_m{S}_m^{-}\right)$, a key strategy of this work. SCVs are then formed by concatenation of linked loadings across the three modalities, where the total number of SCVs K is the minimum component number among the three modalities. Assuming each SCV follows a multivariate Gaussian distribution, mutual information among SCVs is minimized utilizing IVA-G since that has been shown to correspond to mCCA’s generalized variance (GENVAR) approach without orthogonality constraints [8].

The cost function of aNy-way ICA is thus defined as:

$${\mathit{\text{max}}}_w{\sum }_{m=1}^M\text{H}\left(y_m\right)-\lambda \cdot \text{I}\left(a_1,\cdots ,a_K\right).$$ (1)

where M is the total number of modalities, H(y_m) represents the entropy of y_m (the same as Infomax ICA cost [9]), y_m is a nonlinear transformation of the bias-adjusted source u_m ($y_m=\frac{1}{1+e^{-u_m}}$, u_m = s_m + b_m, b_m a bias), I(a₁, ⋯, a_K) is the mutual information among SCVs a_k, λ is a regularizer to balance between independence (ICA) and correlation (IVA-G) maximization. The entropy of y_m [9] and mutual information among SCVs a_k [7] are defined in (2) and (3), respectively:

$$\text{H}\left(y_m\right)=-\mathbb{E}\left[\text{ln}{p}_{y_m}\left(y_m\right)\right].$$ (2)

$$\text{I}\left(a_1,\cdots ,a_K\right)={\sum }_{k=1}^k\text{H}\left(a_k\right)-\text{H}\left(a_1,\cdots ,a_K\right).$$ (3)

where $p_{y_m}\left(y_m\right)$ is the probability density function of the vector y_m. The k-th SCV a_k can be formed as $a_k=A{P}_k^{\top }$, where $P_k^{\top }$ is akin to a permutation matrix that selects the loadings from the k-th component of each modality, and A = [A₁, ⋯, A_M] is a concatenation of loadings from all modalities.

H(a_k) represents the marginal entropy of the k-th SCV and $\text{H}\left(a_1,\cdots ,a_K\right)\approx {\sum }_m^M\text{log}|\text{det}\left(S_m^{-}{P}_m^{\top }\right)|$ is the joint entropy of all SCVs, which simplifies to a determinant function on the sources after noting the entropy of the data is constant. The proposed aNy-way ICA is then solved by stochastic gradient descent (SGD), utilizing the relative gradients [10] of (2) and (3) with respect to W, which are given, respectively, as:

$$\nabla W_m^H=\left(I+\left(1-2y_m\right)u_m^{\top }\right)W_m.$$ (4)

$$\nabla W_m^I=-W_m^{-\top }{K}_m{V}_{1m}^{-}{\dot{S}}_m^{-}{W}_m-W_m^{-\top }{F}_m^{\top }{T}_m^{-\top }{Q}_m^{\top }{W}_m.$$ (5)

where I is an identity matrix, 1 is a column vector of ones, ⊤ is the matrix transpose, $K_m=\text{diag}\left(\text{std}\left(V_{1m}^{\top }\right)\right)$ is a diagonal matrix, and std $\left(V_{1m}^{\top }\right)$ is a vector with the standard deviations from each row of $V_{1m}^{\top }$. $V_{1m}^{\top }=U^{\top }{X}_m$ is based on the eigenvalue decomposition (EVD) of $\Sigma \stackrel{\text{EVD}}{=}U\Lambda U^{\top }$, with $\Sigma =\frac{1}{M}{\sum }_{m=1}^M\frac{{\Sigma }_m}{\text{trace}\left({\Sigma }_m\right)}$ being the average of normalized sample covariance matrices from each modality ${\Sigma }_{\text{m}}=\frac{x_m{x}_m^{\top }}{L_{\text{m}}-1}$ (X_m has zero mean rows and columns). ${\dot{S}}_m^{-}=\left[{\dot{S}}_{m1}^{-}\cdots {\dot{S}}_{mK}^{-}\right]P_m$ is an intermediate gradient for the pseudo-inverse of the source matrix of the m-th modality, and ${\dot{S}}_{mk}^{-}=-\frac{1}{N-1}{X}_m^{\top }{z}_{mk}$ with $z_{mk}=a_k{\left(a_k^{\top }{a}_k\right)}^{-1}{P}_{1m}^{\top }$. Note that $P_{1m}^{\top }$ selects the m-th column from the matrix to its left, and P_m expands the concatenation of ${\dot{S}}_{mk}^{-}$ vectors from L_m × K to V_m × C_m (effectively inserting zero columns as needed). Finally, approximating the determinants in the SCV joint entropy term as the product of singular values of $S_m^{-}{P}_m^{\top }\stackrel{\text{SVD}}{=}{U}_{2m}{D}_m{V}_{2m}^{\top }$ (via singular value decomposition (SVD)) and collecting constant terms into $F_m=U_{2m}^{\top }{V}_{1m}^{-\top }{K}_m$ and $Q_m=P_m^{\top }{V}_{2m}$, yields the final form in (5), where $T_m=F_m{W}_m^{-1}{Q}_m$.

Since the source/loading ordering from separate ICAs per modality would be arbitrary (not linked), it is important not to over-emphasize independence early during the optimization. Rather, we propose to project the multimodal data into a shared variance-adjusted space, i.e., along the shared basis U defined above, which provides a good (linked) starting point for the optimization. Also, we observed improved loading (SCV) linkage and source alignment by initially optimizing only IVA-G for a few steps/epochs (here, 5) before optimizing (1). Moreover, the regularizer parameter λ is initialized with 1 and adaptively adjusted by annealing it according to the entropy trend during the optimization: whenever entropy decreases with a slope larger than a threshold (here, 10⁻⁴), this triggers λ to be reduced by a factor (here, 0.8).

III. SIMULATION

Simulated sMRI, fMRI and EEG data were employed to validate the proposed aNy-way ICA, where sMRI and fMRI sources were simulated with the SimTB toolbox [11]. For EEG sources, we used a subset of simulated independent Laplacian timeseries in [12], which had an autocorrelation around 0.85 between timepoint t and t − 1, and 0.2 between t and t − 10. Each SCV was sampled from a zero mean m-dimensional multivariate Gaussian distribution with unique covariance structures (correlations’ range: [0.2,0.8]). SCVs were then organized into loading matrices A_m. If applicable, extra loadings for non-linked sources were sampled from a standard Normal distribution and padded to A_m. Finally, the data was generated as the product of the loading and source matrices as X_m = A_mS_m. The following scenarios were then examined:

• Scenario 1:All 3 modalities had the same number of sources C_m = 7. The number of subjects was fixed at N = 500. Feature dimensions (L_m) for sMRI, fMRI and EEG data were 31064 (voxels), 17420 (voxels), 4444 (time points), respectively (same L_m used for scenario 2). Comparisons were performed against both mCCA and mCCA+jICA.

• Scenario 2:sMRI, fMRI, and EEG data had different number of sources. In part (a), we assessed the effect of varying the number of linked SCVs (K = [3,5,7,9,11]) while keeping N = 500 fixed. The number of sources C_m for sMRI, fMRI and EEG data were set as K + 1, K, and K + 2, respectively, except for K = 11 where C₁ = C₂ = 11, and C₃ = 12. In part (b), the number of sources C_m for sMRI, fMRI and EEG data were fixed at 8, 7, 9, respectively, while the subject number N varied from 100 to 1000. For each subject number, 10 sets of SCV/loadings were generated and run for each modality. Finally, in part (c), we assessed the effect of additive Gaussian white noise with C_m fixed at 8, 7, 9 for sMRI, fMRI and EEG, respectively, and N fixed at 500. The signal-to-noise ratio (SNR) varied from 0db to 30dB with a step size of 5dB. For each SNR level, the noise samples E_im were generated 5 times, yielding noisy data $X_{\mathit{\text{im}}}^{\mathit{\text{Noisy}}}=X_m+E_{im}$, i = 1, …, 5 for modality m. Note: scenarios 1, 2(a), and 2(b) were noise-free.

The following component-wise performance measures were computed per modality for aNy-way ICA, mCCA and mCCA+jICA: correlations between recovered sources and ground-truth, as well as correlations between recovered loadings and ground-truth. To examine SCV recovery, we also calculated the MSE value between the recovered and groundtruth SCV cross-correlations (linkages), considering the diagonal blocks and off-diagonal portions separately, as well as together (whole matrix). For mCCA and mCCA+jICA, we estimated both $K=\underset{\text{m}}{\text{min}}{C}_m$ and $K=\underset{\text{m}}{\text{max}}{C}_m$ for all modalities (two separate runs). For scenario 2(a), correlations were averaged across matched components per modality, while for scenarios 2(b) and 2(c), correlations were averaged across runs and matched components. MSE values were averaged across runs.

IV. RESULT

Fig. 2 illustrates the seven sources for (a) sMRI, (b) fMRI, and (c) EEG data in scenario 1. Fig. 3 (a)–(d) shows their recovery accuracies: under noise-free conditions, aNy-way ICA recovered all designed SCV structures very well (the MSE value was 2.8E-4). Both mCCA and mCCA+jICA failed to recover, as indicated by the large values on off-diagonal positions, likely due to overfitting of correlations and inability to estimate non-orthogonal sources in mCCA, plus potential residual source misalignments from mCCA leaking into jICA for mCCA+jICA. Fig. 3 (e)–(f) demonstrates aNy-way ICA accurately recovered both sources and loading coefficients for all three modalities, contrary to mCCA and mCCA+jICA.

Figure 2.

Scenario 1 sources for (a) sMRI, (b) fMRI, and (c) EEG data.

Figure 3.

Scenario 1. Cross-correlation of SCVs from (a) groundtruth, (b) aNy-way ICA, (c) mCCA+jICA, and (d) mCCA. Correlation coefficients between (e) recovered sources and ground-truth, and between (f) recovered loadings and ground-truth for each independent component (IC): sMRI (downward triangle), fMRI (star), and EEG data (upward triangle) from aNy-way ICA (red), mCCA+jICA(green) and mCCA(blue). Note that shapes and colors are consistent throughout the paper.

Performance measures for scenario 2(a) are shown in Fig. 4. aNy-way ICA accurately recovered components and loadings for sMRI and fMRI data, with slightly reduced recovery accuracy for EEG data (especially for 9 and 11 SCVs). The inter-symbol interference [13] for the recovered loading matrix and minimum distance indices [14] for the recovered source matrix corroborate these observations and, thus, are omitted here. Lower accuracies for EEG data are likely due to its relatively small feature dimensionality (L₃ = 4444) and weaker designed correlation linkages with other modalities within each SCV. Overall, aNy-way ICA outperformed mCCA+jICA and mCCA in terms of source and loading recovery. mCCA+jICA and mCCA decompositions with the minimum component number of the three modalities had higher accuracies compared to those obtained with the maximum component number. As the SCV number increased (e.g., 9 or 11 SCVs), source and loading accuracies from both mCCA and mCCA+jICA dropped. Table I lists MSE values for aNy-way ICA’s recovered covariance structure. aNy-way ICA fully recovered the designed covariance structure when SCV number was 3, 5, and 7, and largely recovered it when SCV number was 9 and 11 (actually, sMRI and fMRI were aligned well but not sMRI-EEG and fMRI-EEG).

Figure 4.

Scenario 2(a), where the number of the SCVs varied from 3 to 11. Recovered source accuracies for (a) sMRI, (c) fMRI, and (e) EEG data, and loading accuracies for (b) sMRI, (d) fMRI, and (f) EEG data using aNy-way ICA (red), mCCA+jICA with maximum (solid green) or minimum component number (dashed green), mCCA with maximum (solid blue) or minimum component number (dashed blue). Values are averaged across matched components per modality.

TABLE I.

MSE Values Between the Recovered and Ground-Truth SCV Cross-Correlations by aNy-Way ICA while Varying the Number of SCVs

MSE	2(a) Varying Number of SCVs (K)
	3	5	7	9	11
All*	2×10−4	5×10−4	6×10−4	2×10−2	6×10−3
Diag*	3×10−5	2×10−5	2×10−4	8×10−2	3×10−2
Off*	3×10−4	7×10−4	6×10−4	9×10−3	3×10−3

^*Note: ‘All’ denotes MSE of whole matrix, ‘Diag’ denotes MSE of diagonal blocks only, ‘Off’ denotes MSE of off-diagonal blocks only.

Fig. 5 demonstrates performance measures for scenario 2(b). As number of subjects increased, the recovery accuracies of sources and loadings for fMRI and EEG, as well as the covariance structure of SCVs, increased, although there were some variations when the subject number was less than 400. Variation of accuracies of EEG data from aNy-way ICA were probably due to weak covariance structures generated from few samples. MSE values for the whole matrix, diagonal blocks, and off-diagonal blocks of the recovered SCV cross-correlation are listed in Table II 2(b). As the number of subjects increased, the MSE values decreased (except for N = 400 and N = 1000).

Figure 5.

Scenario 2(b), where the number of subjects N varied from 100 to 1000. Recovered source accuracies for (a) sMRI, (c) fMRI, and (e) EEG data, and loading accuracies for (b) sMRI, (d) fMRI, and (f) EEG data. Values are averaged across runs and matched components per modality.

TABLE II.

MSE Values Between Recovered and Ground-Truth SCV Cross-Correlations BY aNy-Way ICA while Varying the Number of Subjects or SNR.

MSE
2(b) Varying Number of Subjects (N)				2(c) Varying SNR
N	All*	Diag*	Off*	SNR	All*	Diag*	Off*
100	2×10−2	5×10−2	1×10−2	0	4×10−2	2×10−1	1×10−2
200	1×10−2	4×10−2	8×10−3	5	3×10−2	2×10−1	4×10−3
300	8×10−3	2×10−2	5×10−3	10	3×10−2	1×10−1	1×10−2
400	1×10−2	3×10−2	6×10−3	15	1×10−2	7×10−2	2×10−3
500	8×10−3	3×10−2	5×10−3	20	8×10−3	3×10−2	4×10−3
700	5×10−3	1×10−2	4×10−3	25	2×10−3	5×10−3	1×10−3
1000	6×10−3	2×10−2	4×10−3	30	9×10−4	4×10−4	1×10−3

^*Note: ‘All’ denotes MSE of whole matrix, ‘Diag’ denotes MSE of diagonal blocks only, ‘Off’ denotes MSE of off-diagonal blocks only.

Performance measures for scenario 2(c) are plotted in Fig. 6. Increasing the strength of noise superimposed on data, recovery accuracies of sources and loadings dropped. aNy-way ICA outperformed mCCA and mCCA+jICA when SNR ≥ 5dB. When SNR = 0db, aNy-way ICA, mCCA, as well as mCCA+jICA failed to recover the sources and loadings. Table II 2(c) summarizes MSE values of the whole matrix, diagonal blocks and off-diagonal blocks of the recovered SCV cross-correlation, reflecting larger MSE values for smaller SNR (i.e. heavier noise).

Figure 6.

Scenario 2(c), where SNR varied from 30db to 0db. Recovered source accuracies for (a) sMRI, (c) fMRI, and (e) EEG data, and loading accuracies for (b) sMRI, (d) fMRI, and (f) EEG data. Values are averaged across runs and matched components per modality.

V. CONCLUSION

The proposed aNy-way ICA simultaneously optimizes independence of sources and correlations across modalities without orthogonality constraints. It can be flexibly applied to N-way multimodal fusion with the same or different number of sources per modality. Simulation results support that aNy-way ICA can recover sources and loadings, as well as the true covariance patterns, with improved accuracy compared to mCCA and mCCA+jICA, especially under noisy conditions.

Clinical Relevance—

This establishes a model for N-way data fusion of any number of modalities and linkage pairs, allowing different number of non-orthogonal sources for different modalities.