Fair Multi-modal Canonical Correlation Analysis: A Neuroimaging Study of Alzheimer’s Disease

Abstract

This study addresses fairness concerns in Multi-modal Canonical Correlation Analysis (MCCA), a technique for analyzing relationships across multiple datasets. We introduce Fair MCCA (F-MCCA), which mitigates bias by optimizing for both correlation performance and demographic fairness. Our method quantifies disparities using Correlation Disparity Error (CDE) and employs a multi-objective optimization framework to derive projection matrices that achieve consistent correlation levels across sensitive groups. We validate F-MCCA on neuroimaging data from the Alzheimer’s Disease Neuroimaging Initiative using sex as the sensitive attribute. Experiments demonstrate that F-MCCA substantially improves fairness metrics with minimal correlation performance sacrifice. In downstream classification tasks, F-MCCA reduces demographic parity difference and equalized odds difference while maintaining comparable accuracy to standard approaches. Results confirm that our method effectively balances analytical performance with fairness considerations, supporting more unbiased healthcare applications of multi-modal data analysis.

Introduction

Multi-modal Canonical Correlation Analysis (MCCA) is a powerful statistical method that identifies latent relationships across datasets from different sources or modalities. In healthcare and biomedical research, MCCA has become increasingly valuable for integrating heterogeneous data types such as imaging, genomics, and clinical measures to improve understanding of complex diseases.^{1, 2} MCCA achieves this integration by maximizing the sum of squared correlations between the datasets’ linear projections in a common space, facilitating comparison and analysis across diverse data sources. Specifically, given M centered modalities ${\left\{X_m\right\}}_{m=1}^M$, MCCA seeks the R-dimensional sub-spaces that optimize the correlation of these linear projections ${\{X_m\hspace{0.17em}{U}_m\}}_{m=1}^M$ with a group configuration Y, subject to certain constraints,^{3, 4} which can be formulated as:

$$\underset{Y,{\left\{U_m\right\}}_{m=1}^M}{\max }\hspace{0.17em}{\Vert Y\hspace{0.17em}-\hspace{0.17em}{X}_m\hspace{0.17em}{U}_m\Vert }_{\text{F}}^2\hspace{0.17em}\text{subj}.\hspace{0.17em}\text{to}\hspace{0.17em}Y\hspace{0.17em}\in \hspace{0.17em}\mathcal{Y}\hspace{0.17em}:=\hspace{0.17em}\left\{Y\right|Y^{\top }Y\hspace{0.17em}=\hspace{0.17em}{I}_R\}.$$ (MCCA)

Analytically, the group configuration is solved via the eigendecomposition of $({\displaystyle {\sum }_{m=1}^M{X}_m{(X_m^{\top }{X}_m)}^{-1}}{X}_m^{\top })Y\hspace{0.17em}=\hspace{0.17em}Y\hspace{0.17em}\Lambda$ and the canonical weights are calculated through $U_m\hspace{0.17em}=\hspace{0.17em}{(X_m^{\top }{X}_m)}^{-1}{X}_{m}Y\hspace{0.17em}$ for m = 1, …, M.

MCCA has been instrumental in improving machine learning models’ performance across various domains, including cross-modal retrieval,⁵ joint representation learning,⁶ sentiment analysis,⁷ human-computer interaction, and more.^{8, 9} Its application is also notable in genetics,^{10, 11} health care data analytics,¹² biomedical engineering,^{13, 14, 15, 16} and precision medicine initiatives,¹⁷ demonstrating its utility in extracting complex inter-modal relationships. Such extensive applications highlight the method’s potential for integrating and analyzing multi-sourced data, leading to improved generalization and prediction.

However, as healthcare systems increasingly rely on these computational methods for clinical decision support and research, ensuring fairness across various patient populations has become a critical concern. Standard MCCA approaches may inadvertently perpetuate or amplify biases present in training data, particularly affecting underrepresented demographic groups.^{18, 19} For example, in neurological disorder analysis, diagnostic models may perform differently across sex and ethnic groups due to underlying differences in disease presentation or data representation.^{20, 21} If not adequately addressed, these disparities can lead to biased healthcare delivery and research outcomes.

While fairness in machine learning for healthcare has gained attention, existing approaches have mainly concentrated on supervised learning contexts or limited data modalities. Our previous work introduced Fair CCA²² for dual-modality scenarios; however, extending fairness principles to multi-modal settings remains unexplored. To fill this gap, we introduce Fairness-aware MCCA (F-MCCA), a novel algorithm designed to ensure unbiased representation of all sensitive groups in multi-modal analysis. Unlike standard MCCA, which may disproportionately reflect patterns from dominant groups, F-MCCA equalizes the contribution of each demographic group to the overall correlation structure. This approach is particularly relevant for healthcare applications where balanced representation is essential for developing unbiased diagnostic and prognostic models.

Figure 1 illustrates the conceptual difference between standard MCCA and our proposed F-MCCA approach. In traditional MCCA (Figure 1g), the resulting projections may favor patterns from dominant groups while overlooking important correlations in minority groups. In contrast, F-MCCA (Figure 1h) balances the contribution of each demographic group, ensuring fair representation in the shared projection space. While this fairness-oriented approach necessarily involves some trade-off with the original correlation objective, our experiments demonstrate that this compromise is modest compared to the fairness gains achieved.

Figure 1:

Illustration of MCCA and F-MCCA with three data modalities. The subfigures (a-d) demonstrate the MCCA framework: data from three modalities is projected to minimize distance to a common group configuration. The resting subfigures compare fairness implications: (e) shows MCCA optimized for Group 1 only, (f) shows MCCA optimized for Group 2 only, (g) illustrates standard MCCA where Group 1 receives preferential correlation performance, and (h) presents our F-MCCA approach, which equalizes projection distances across demographic groups.

We validate our approach using neuroimaging data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI), with sex as the sensitive attribute. Through comprehensive evaluations from both unsupervised correlation analysis and supervised classification perspectives, we demonstrate that F-MCCA significantly improves fairness metrics while maintaining competitive performance in disease characterization tasks. These findings highlight the potential of fairness-aware multi-modal analysis to advance unbiased healthcare research and clinical applications.

Method

Preliminary. The set of real numbers is denoted by ℝ, with ℝ₊ indicating non-negative numbers and ℝ₊₊ for positive numbers. Vectors are in bold lowercase (e.g., a) and matrices in bold uppercase (e.g., A), with elements represented as a_i and a_ij, respectively. Sets are denoted by calligraphic letters (e.g., $\mathcal{S}$), and set cardinality by | · | (e.g., |$\mathcal{S}$|). For vectors x, y ∈ ℝ^m, x ≺ y signifies all elements of y − x are positive, while $\text{x}\hspace{0.17em}\preccurlyeq \hspace{0.17em}\text{y}$ means they are non-negative. Λ represents a diagonal matrix with a matrix’s eigenvalues.

Correlation Disparity Error

Traditional MCCA approaches applied to heterogeneous datasets can inadvertently introduce bias by over-representing dominant demographic groups and marginalizing smaller populations. To address this fairness concern in healthcare applications, we introduce Fairness-aware MCCA (F-MCCA), a novel framework that ensures unbiased representation across sensitive demographic attributes.

F-MCCA operates through a two-step process: first analyzing data stratified by sensitive attributes (such as sex, race, or socioeconomic status), then systematically integrating these group-specific insights into a globally fair representation. This approach ensures that all demographic groups maintain proportionate influence on the dimensional reduction process, particularly critical in clinical contexts where underrepresented populations must not be overlooked.

To formalize this concept, consider datasets ${\{X_m\in {ℝ}^{N\times D_m}\}}_{m=1}^M$ comprising N observations across M modalities. These observations are partitioned into K sensitive groups ${\left\{X_m^k\right\}}_{m=1,k=1}^{M,K}$, where $\left\{X_m^k\right\}\in \hspace{0.17em}{ℝ}^{N_k\hspace{0.17em}\times \hspace{0.17em}{D}_m}$ ∈ ℝ^{N_k × D_m} represents the kth group’s observations in the mth modality. To quantify the disparate impacts of MCCA across these groups, we introduce:

Definition 1 (Correlation Disparity Error (CDE)). For datasets ${\left\{X_m\right\}}_{m=1}^M$ partitioned into K sensitive groups with data matrices ${\left\{X_m^k\right\}}_{m=1,k=1}^{M,k}$, the Correlation Disparity Error for group k ∈ [K] under projection parameters ${\left\{U_m\right\}}_{m=1}^M$ and a group configuration Y is:

$$E^k(Y,{\left\{U_m\right\}}_{m=1}^M)\hspace{0.17em}:=\hspace{0.17em}{L}_k\hspace{0.17em}(Y^{k,\ast },{\left\{U_m^{k,\ast }\right\}}_{m=1}^M)\hspace{0.17em}-\hspace{0.17em}{L}_k(Y^k,{\left\{U_m\right\}}_{m=1}^M),$$

where $L_k\hspace{0.17em}:=\hspace{0.17em}\frac{{\displaystyle {\sum }_{k=1}^K{N}_k}}{M{N}_k}{\displaystyle {\sum }_{m=1}^M{\Vert Y^k\hspace{0.17em}-\hspace{0.17em}{X}_m^k\hspace{0.17em}{U}_m^k\Vert }_F^2}$ represents a size-adjusted correlation measure , and $(Y^{k,*},\{U_m^{k,*}{\}}_{m=1}^M)$ denotes the optimal group-specific MCCA projections. The term Y^k refers to the group configuration for group k, which is the subset of Y corresponding to observations in group k.

This metric quantifies how much a global projection deviates from optimal group-specific projections, with larger values indicating greater disparity. Using this definition, we formulate our fairness criterion:

Definition 2 (F-MCCA). A multi-modal canonical correlation analysis solution $({\text{Y}}^{\ast },{\left\{{\text{U}}_m^{*}\right\}}_{m=1}^M)$ is considered fair if it equalizes the Correlation Disparity Error across all demographic groups:

$$\begin{array}{cc}{E}^k(Y*,{\left\{U_m^{*}\right\}}_{m=1}^M)\hspace{0.17em}=\hspace{0.17em}{E}^s(Y*,{\left\{U_m^{*}\right\}}_{m=1}^M)& \forall k\ne s.\end{array}$$

The fairness criterion defined in Definition 2 builds upon our prior FairCCA framework, which in turn extends the pairwise fairness formulation originally proposed in Fair PCA by Kamani et al. (2022).²³ While Fair PCA focuses on minimizing the variance gap across sensitive groups in a single-modality setting, our method generalizes this principle to the multi-modal correlation setting. Specifically, the use of Correlation Disparity Error (CDE) and pairwise CDE aligns conceptually with Kamani’s pairwise reconstruction error, but is adapted to measure inter-group differences in canonical correlation performance across multiple modalities. We retain the pairwise penalty structure to preserve theoretical consistency and ensure meaningful fairness guarantees in the extended multi-modal framework.

Definition 3 (Pairwise CDE). The pairwise CDE between groups k and s for any global projection $(Y,{\left\{U_m\right\}}_{m=1}^M)$ and group-specific projections ${\{({\text{Y}}^{k,\ast },{\left\{{\text{U}}_m^{k,\ast }\right\}}_{m=1}^M)\}}_{k=1}^K$ is:

$$\begin{array}{cc}{\text{Δ}}^{k,s}(Y,{\left\{U_m\right\}}_{m=1}^M)\hspace{0.17em}:=\hspace{0.17em}\varphi (E^k(Y*,{\left\{U_m^{*}\right\}}_{m=1}^M\hspace{0.17em}-\hspace{0.17em}{E}^s(Y*,{\left\{U_m^{*}\right\}}_{m=1}^M)),& \forall k\ne s,\end{array}$$

where ϕ : ℝ → ℝ₊ is a convex penalty function, commonly implemented as ϕ(x) = exp(x), ϕ(x) = x².

This pairwise formulation allows us to directly minimize disparities between demographic groups, creating a projection that balances overall correlation performance with unbiased representation—a crucial consideration in healthcare applications where algorithmic fairness directly affects clinical outcomes and health fairness.

Algorithm 1.

A Multi-Objective Gradient Method for F-MCCA

A Multi-Objective Framework for F-MCCA

Optimizing both correlation performance and fairness involves inherent trade-offs in multimodal analysis. To tackle this challenge, we formulate F-MCCA as a multi-objective optimization problem that systematically balances the competing goals of maximizing correlation while minimizing disparities between demographic groups.

We define our objective functions as follows: let $h_1\left(Y,{\left\{U_m\right\}}_{m=1}^M\right):={\displaystyle {\sum }_{m=1}^M\left|\right|Y-X_m{U}_m|{|}_F^2}$ represent the standard MCCA correlation objective, and $h_{k+1}(Y,{\left\{U_m\right\}}_{m=1}^M):={\displaystyle {\sum }_{s=1,s\ne k}^K{\Delta }^{k,s}}(Y,{\left\{U_m\right\}}_{m=1}^M)$ capture the pairwise disparities for group k for k = 1, …, K. The complete multi-objective optimization problem is then formulated as:

$$\underset{Y,{\left\{U_m\right\}}_{m=1}^M}{minimize}\hspace{0.17em}H(Y,{\left\{U_m\right\}}_{m=1}^M)\hspace{0.17em}:=\hspace{0.17em}[h_1(Y,{\left\{U_m\right\}}_{m=1}^M),h_2(Y,{\left\{U_m\right\}}_{m=1}^M),\hspace{0.17em}\dots ,\hspace{0.17em}{h}_{K+1}(Y,{\left\{U_m\right\}}_{m=1}^M)],\hspace{0.17em}\hspace{0.17em}\text{subject}\hspace{0.17em}\text{to}\hspace{0.17em}Y\hspace{0.17em}\in \hspace{0.17em}\mathcal{Y}\hspace{0.17em}:=\hspace{0.17em}\left\{Y\right|Y^{\top }Y\hspace{0.17em}=\hspace{0.17em}{I}_R\}.$$ (1)

A solution $(Y^{*},{\left\{U_m^{*}\right\}}_{m=1}^M)$ is considered weakly Pareto optimal if no alternative feasible solution exists that simultaneously improves all objectives—formally, if no $(Y,{\left\{U_m\right\}}_{m=1}^M)$ exists such that

$$H(Y,{\left\{U_m\right\}}_{m=1}^M)\prec H(Y*,{\left\{U_m^{*}\right\}}_{m=1}^M).$$

This formulation acknowledges the inherent trade-offs between correlation performance and fairness, seeking the best possible compromises among them.

Optimization Algorithm. To solve this multi-objective problem efficiently, we developed a gradient-based approach (Algorithm 1) that converges to a weakly Pareto optimal solution. The algorithm operates in two phases:

1. Compute group-specific optimal projections $\left(\text{Y}*\hspace{0.17em},\hspace{0.17em}{\left\{{\text{U}}_m^{*}\right\}}_{m\hspace{0.17em}=\hspace{0.17em}1}^M\right)$ to establish fairness benchmarks

2. Iteratively optimize the global projections through a descent direction that balances all objectives

At each iteration, the algorithm determines the optimal descent direction $(D_Y^{(t)},{\left\{D_{{\text{U}}_m}^{(t)}\right\}}_{m=1}^M)$ by solving the following minimax problem²⁴ iteratively concerning the gradient with respect to ${\text{Y}}^{\left(t\right)},\hspace{0.17em}{\text{U}}_1^{\left(t\right)},\hspace{0.17em}\dots \hspace{0.17em},\hspace{0.17em}{\text{U}}_M^{\left(t\right)}$ :

$$D^{(t)}\hspace{0.17em}=\hspace{0.17em}\arg \hspace{0.17em}\underset{D}{\min }\left\{\underset{k=1,\dots ,K+1}{\max }\text{trace}(D^{\top }\nabla h_k(Y^{(t)},{\left\{U_m^{(t)}\right\}}_{m=1}^M))\hspace{0.17em}+\hspace{0.17em}\frac{1}{2}{\Vert D\Vert }_{\text{F}}^2\right\}.$$ (2)

This approach ensures the algorithm progresses on the most challenging objective at each step, effectively balancing improvement across all optimization criteria. The following lemma characterizes the descent direction:

Lemma 1. The solution to the minimax problem in Equation (2) is either D^(t) = 0 (indicating a Pareto stationary point), or takes the form ${\text{D}}^{(t)}=-{\sum }_{k=1}^{K+1}{\text{λ}}_k^{(t)}\nabla h_k({\text{Y}}^{(t)},{\left\{{\text{U}}_m^{(t)}\right\}}_{m=1}^M)$, where ${\sum }_{k\hspace{0.17em}=1}^{K\hspace{0.17em}+\hspace{0.17em}1}\hspace{0.17em}{\text{λ}}_k^{\left(t\right)}\hspace{0.17em}=\hspace{0.17em}1$ and ${\text{λ}}_k^{\left(t\right)}\hspace{0.17em}\ge \hspace{0.17em}0,$ representing a convex combination of gradient directions.

To maintain the orthogonality constraint on Y throughout optimization, we employ a transformation $\mathbb{T}$ that projects the updated Y back onto the Stiefel manifold. Specifically, if Y^(t) has eigenvalue decomposition with eigenvalues Λ(t) and corresonding eigenvectors Q^(t), then $\mathbb{T}\hspace{0.17em}\left({\text{Y}}^{\left(t\right)}\right)\hspace{0.17em}=\hspace{0.17em}{\text{Y}}^{\left(t\right)}\hspace{0.17em}{\text{Q}}^{\left(t\right)}\hspace{0.17em}{\Lambda }^{-\hspace{0.17em}\frac{1}{2}}\hspace{0.17em}{\text{Q}}^{{\left(t\right)}^{\top }}$ ensures the constraint is satisfied.

Convergence Analysis. The convergence properties of our algorithm are established through the following theoretical results:

Assumption 1. For a subset $\mathcal{S}$ of the feasible space $\mathcal{Y}\hspace{0.17em}\times \hspace{0.17em}{ℝ}^{D_1\hspace{0.17em}\times \hspace{0.17em}R}\hspace{0.17em}\times \hspace{0.17em}\cdots \hspace{0.17em}\times \hspace{0.17em}{ℝ}^{D_M\hspace{0.17em}\times \hspace{0.17em}R}$, there exists a Lipschitz constant L_H such that for all W, V ∈ $\mathcal{S}$:

$$H(W)\hspace{0.17em}\preccurlyeq \hspace{0.17em}H(V)\hspace{0.17em}+\hspace{0.17em}\nabla \hspace{0.17em}+\hspace{0.17em}\frac{L_H}{2}{\Vert W\hspace{0.17em}-\hspace{0.17em}V\Vert }_F^2{1}_{K+1},$$

where ∇_k := 〈∇h_k(V), (W − V)〉, ∇ := [∇1, …, ∇_K+1] ∈ ℝ^K+1.

Theorem 1. Under Assumption 1, if ${\eta }_Y^{\left(t\right)}\hspace{0.17em}=\hspace{0.17em}{\eta }_{U_1}^{\left(t\right)}\hspace{0.17em}=\hspace{0.17em}\cdots \hspace{0.17em}=\hspace{0.17em}{\eta }_{U_M}^{\left(t\right)}\hspace{0.17em}=\hspace{0.17em}\eta \hspace{0.17em}\le \hspace{0.17em}1/L_H$ for all iterations t ∈ {0, …, T − 1}, then the sequence generated by Algorithm 1 satisfies:

$$\min \left\{{\Vert D^{(t)}\Vert }_{\text{F}}\hspace{0.17em}:\hspace{0.17em}t\hspace{0.17em}=\hspace{0.17em}0,\dots ,T\hspace{0.17em}-\hspace{0.17em}1\right\}\hspace{0.17em}\le \hspace{0.17em}\frac{2}{\eta }{\left[\frac{h_{k_{*}}(Y^{(0)},{\left\{U_m^{(0)}\right\}}_{m=1}^M)\hspace{0.17em}-\hspace{0.17em}{h}_{k_{*}}^{*}}{T}\right]}^{\frac{1}{2}}$$

where $h_{k_{*}}\left(Y^{\left(0\right)},{\left\{U_m^{\left(0\right)}\right\}}_{m=1}^M\right)-h_{k_{*}}^{*}\hspace{0.17em}:=\hspace{0.17em}\min \hspace{0.17em}\left\{h_k\left(Y^{\left(0\right)},{\left\{{\text{U}}_m^{\left(0\right)}\right\}}_{m=1}^M\right)-h_k^{*}\hspace{0.17em}:\hspace{0.17em}k\hspace{0.17em}\in \left[K+1\right]\right\}$ and $h_k^{*}$ is the infimum value of objective h_k, defined as $h_k^{*}:=\inf \left\{h_k\left(Y,{\left\{U_m\right\}}_{m=1}^M\right):\left(Y,{\left\{U_m\right\}}_{m=1}^M\right)\in \mathcal{Y}\times {ℝ}^{D_1\times R}\times \cdots \times {ℝ}^{D_M\times R}\right\}$.

This theorem guarantees that the algorithm makes steady progress toward a Pareto stationary point, with the minimum gradient norm decreasing at a rate of O(1/$\sqrt{T}$). The complete proofs for Lemma 1 and Theorem 1 follow from our previous work on fair CCA.22

Experiment

This study evaluated our F-MCCA method from unsupervised and supervised learning perspectives, specifically applied to neuroimaging data in Alzheimer’s disease research. Our evaluation framework consisted of two complementary approaches. First, in the unsupervised learning context, we implemented F-MCCA on multi-modal neuroimaging data to quantify improvements in fairness by reducing CDE while simultaneously preserving strong inter-modality correlations. Second, we examined the downstream impact of these fairer representations on supervised classification tasks by comparing classification performance using features derived from F-MCCA against those from standard MCCA and unprocessed features.

Our experiments focused on the three-class classification problem in Alzheimer’s disease progression (Healthy Control, Mild Cognitive Impairment, and Alzheimer’s Disease) using multi-modal brain imaging data from the ADNI dataset. We paid particular attention to sex-related fairness disparities, as previous research had documented significant differences in AD manifestation between male and female populations. Sex-based differences in AD presentation included variations in cognitive decline patterns, biomarker profiles, and structural brain changes that could influence diagnostic accuracy when not properly addressed in analytical methods.^{25, 26, 27} Ensuring fairness in AD classification models was particularly important given these known biological variations, as biased models could have led to delayed diagnosis, inappropriate treatment recommendations, or research conclusions that failed to generalize across demographic groups.

Through this comprehensive evaluation, we aimed to demonstrate that F-MCCA not only improved fairness metrics but also maintained or enhanced the clinical utility of the resulting models for AD classification and staging, ultimately supporting more unbiased healthcare research and decision-making.

Data Acquisition and Preprocessing

Our study utilized multi-modal neuroimaging data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database, a landmark longitudinal study designed to develop clinical, imaging, genetic, and biochemical biomarkers for the early detection and tracking of Alzheimer’s disease (AD).^{28, 29} We integrated three complementary imaging modalities that capture different aspects of AD pathophysiology: structural MRI for brain morphometry, 18F-Florbetapir PET (AV45-PET) for amyloid plaque deposition, and 18F-Flortaucipir PET (AV1451-PET) for tau tangle burden.

Following rigorous spatial normalization and co-registration of these modalities, we established a cohort of 375 participants (182 males, 193 females) with complete imaging data across all three modalities. From the structural MRI, we extracted 66 region-specific cortical thickness measurements using FreeSurfer’s automated parcellation, reflecting neurodegeneration patterns. From both PET modalities, we derived 68 cortical Standardized Uptake Value Ratio (SUVR) measurements per modality, normalized to the cerebellum as the reference region, quantifying the regional burden of pathological protein deposits.

The study population represented the spectrum of AD progression: 30 participants met the clinical criteria for Alzheimer’s Disease (AD), 121 were diagnosed with Mild Cognitive Impairment (MCI), and 224 were classified as cognitively normal Healthy Controls (HC). This distribution reflects the typical prevalence pattern observed in memory clinic populations.

Experimental Setup

Evaluation Criteria. Let ẑ represent the predicted label and z the true label. This study considers two sensitive groups, G₁ and G₂, with C = {c₁, c₂, c₃} representing the three classes used in the classification task. The evaluation metrics are detailed in Table 1. Specifically, for unsupervised methods MCCA and F-MCCA, correlation maximization is assessed using correlation (ρ), and fairness improvement is evaluated using mean pairwise CDE (Δ). In supervised classification, accuracy measures performance, while fairness is evaluated using demographic parity difference (DPD) and equalized odds difference (EOD), two common fairness metrics.

Table 1:

Evaluation metrics for both unsupervised and supervised models. We consider two sensitive groups (male and female) and three diagnostic classes (HC, MCI, AD) in this study.

Metric	Mathematical expression
Correlation	$\rho ={\Vert Y-X_m{U}_m\Vert }_F^2$
Mean Pairwise CDE	$\text{Δ}=\frac{1}{K}{\displaystyle {\sum }_{k,s\in \left[K\right],k\ne s}{\text{Δ}}^{k,s}}$
Accuracy	$\mathbb{P}\left(\widehat{z}=z\right)$
demographic parity difference (DPD)30, 31	$\frac{1}{\left|\mathcal{C}\right|}{\displaystyle {\sum }_{c\in \mathcal{C}}|\mathbb{P}\left(\widehat{z}=c|{\mathcal{G}}_1\right)-\mathbb{P}\left(\widehat{z}=c|{\mathcal{G}}_2\right)|}$
equalized odds difference (EOD)30, 32	$\frac{1}{\left|\mathcal{C}\right|}{\displaystyle {\sum }_{i\in \left|\mathcal{C}\right|}^{\left|\mathcal{C}\right|}{\displaystyle {\sum }_{j\in \left|\mathcal{C}\right|}^{\left|\mathcal{C}\right|}|\mathbb{P}\left(\widehat{z}=c_j|z=c_i,{\mathcal{G}}_1\right)-\mathbb{P}\left(\widehat{z}=c_j|z=c_i,{\mathcal{G}}_2\right)|}}$

Baseline. We benchmarked our proposed F-MCCA against standard MCCA for unsupervised representation learning, while employing Support Vector Machine (SVM)³³ for downstream classification tasks. In the supervised setting, we systematically compared three feature extraction strategies: (1) F-MCCA projections, (2) standard MCCA projections, and (3) original features without dimensionality reduction. Each strategy was evaluated across three neuroimaging modalities (AV1451-PET, AV45-PET, and MRI), analyzed both individually and in combination, to comprehensively assess improvements in both classification performance and fairness metrics.

Experimental Setup and Parameter Configuration. For unsupervised analysis, we utilized the complete dataset of 375 subjects, initializing $\left(Y^{\left(0\right)},{\left\{U_m^{\left(0\right)}\right\}}_{m=1}^M\right)$ with standard MCCA results on the entire dataset. The algorithm ran for a maximum of 1000 iterations with learning rates ${\eta }_Y^{\left(t\right)}$ and ${\left\{{\eta }_{U_m}^{\left(t\right)}\right\}}_{m=1}^M$ set to 5e-5, applying an adaptive line search at each iteration (decay rate 0.1) to ensure convergence.

For supervised learning, the experimental procedure occurred in the following steps:

1. Randomly partitioned the dataset into training (70%) and testing (30%) sets

2. Applied F-MCCA or MCCA to the training data for dimensionality reduction (dimension = 50), or alternatively retained all original features for baseline comparison

3. Trained SVM classifiers on the processed training data using grid search optimization with hyperparameters {C: [1e-3, 1e-2, …, 10], kernel: [‘rbf’, ‘poly’, ‘sigmoid’, ‘linear’], gamma: [‘scale’, ‘auto’, 1e-2, …, 100]} and 5-fold cross-validation to maximize accuracy

4. Processed the testing data using the same dimensionality reduction approach (projecting to 50 dimensions with trained F-MCCA/MCCA models or keeping all original features)

5. Evaluated classification performance on the testing set using three metrics: accuracy (overall performance), DPD and EOD (fairness measures)

To ensure robust results, we repeated this process across 20 random seeds, using the first 10 iterations for parameter tuning of F-MCCA and the remaining 10 for independent validation, thus minimizing potential optimization bias.

Results and Discussion

Unsupervised MCCA. Table 2 presents the comparative performance of the standard MCCA and our proposed F-MCCA across multiple dimensions. To facilitate interpretation, we quantify the percentage changes in correlation and mean pairwise CDE, respectively, as follows:

$$\begin{array}{c}\%\rho \hspace{0.17em}=\hspace{0.17em}-(\rho \hspace{0.17em}\text{of}\hspace{0.17em}\text{F}-\text{MCCA}\hspace{0.17em}-\hspace{0.17em}\rho \hspace{0.17em}\text{of}\hspace{0.17em}\text{MCCA})\hspace{0.17em}/\hspace{0.17em}\rho \hspace{0.17em}\text{of}\hspace{0.17em}\text{MCCA},\\ \%\text{Δ}\hspace{0.17em}=\hspace{0.17em}-(\text{Δ}\hspace{0.17em}\text{of}\hspace{0.17em}\text{F}-\text{MCCA}\hspace{0.17em}-\hspace{0.17em}\text{Δ}\hspace{0.17em}\text{of}\hspace{0.17em}\text{MCCA})\hspace{0.17em}/\hspace{0.17em}\text{Δ}\hspace{0.17em}\text{of}\hspace{0.17em}\text{MCCA}.\end{array}$$

Table 2:

Results from 10 repeated experiments on an Apple M2 Pro processor are summarized with the objective function value (ρ), mean pairwise CDE (Δ), and average computation time in seconds (± standard deviation). Ten different projection dimensions (R) are considered in this experiment. Lower values (↓) are better, with the best results in bold. Given the input data matrix, both ρ and Δ are deterministic.

R	ρ ↓		%ρ	Δ ↓		%Δ	Runtime ↓
	MCCA	F-MCCA		MCCA	F-MCCA		MCCA	F-MCCA
1	1.1828	1.1833	-0.04%	0.0005	0.0002	46.63%	1.44 (± 0.2)	56.8 (± 4.0)
2	1.7517	1.7525	-0.04%	0.0002	0.0001	52.14%	1.38 (± 0.2)	55.4 (± 2.4)
3	2.1970	2.1973	-0.01%	0.0001	0.0000	52.59%	1.32 (± 0.1)	58.0 (± 4.5)
4	2.5958	2.5960	-0.01%	0.0001	0.0000	55.83%	1.48 (± 0.1)	58.4 (± 2.8)
10	4.4520	4.4794	-0.61%	0.0057	0.0025	56.06%	1.89 (± 0.1)	62.4 (± 2.0)
20	6.8537	6.9728	-1.74%	0.0371	0.0082	77.88%	1.97 (± 0.2)	71.1 (± 3.2)
30	8.9092	9.0545	-1.63%	0.0526	0.0065	87.73%	1.87 (± 0.2)	89.9 (± 6.0)
40	10.806	10.977	-1.58%	0.0737	0.0055	92.57%	1.74 (± 0.2)	94.5 (± 7.7)
50	12.587	12.818	-1.84%	0.1128	0.0058	94.86%	1.66 (± 0.3)	86.6 (± 7.8)
60	14.284	14.544	-1.82%	0.1478	0.0057	96.14%	1.52 (± 0.1)	83.9 (± 4.5)

F-MCCA achieves substantial improvements in fairness (46.63% to 96.14% reduction in mean pairwise CDE) with minimal correlation performance loss (0.01% to 1.84%), with the most pronounced gains at higher dimensions. This favorable trade-off highlights the central tension in fairness-aware representation learning: improving equity often requires controlled compromises in statistical fit. In our case, F-MCCA introduces fairness objectives that regularize the projection space, slightly constraining the optimization of canonical correlation. Nevertheless, this constraint yields representations that are more demographically balanced—an essential property in clinical applications where fairness is not only a technical consideration but also a requirement for trust, generalizability, and regulatory compliance. The ability to enforce fairness with minimal impact on core performance metrics underscores the practical value of our approach.

Supervised Classification. Figure 2 and Table 3 present three classification-related measures: accuracy, DPD, and EOD. The figure demonstrates that MCCA improves classification fairness but at the expense of some accuracy. Building on this, F-MCCA preserves MCCA’s accuracy while further enhancing fairness. This trend holds consistently across classification tasks for individual modalities and their combinations, validating our fairness-enhancing strategy. However, the inherent linearity of standard MCCA imposes limitations, leading to information loss during dimensionality reduction. This lost information is likely modality-specific and could be crucial for distinguishing between the three classes, resulting in lower accuracy compared to applying SVM directly to the original data.

Figure 2:

Results from 20 repeated experiments comparing accuracy (top), demographic parity difference (middle), and equalized odds difference (bottom) across three methods: F-MCCA + SVM, where F-MCCA reduces dimensions before SVM; MCCA + SVM, applying MCCA for dimension reduction then SVM; and All-SVM, which directly uses SVM on all original features. Each bar corresponds to one imaging modality (AV1451, AV45, MRI) or the combined set. The projection dimension is fixed at 50. X-axis labels indicate the evaluation metric. F-MCCA consistently achieves the lowest DPD and EOD across all modalities, with only a slight reduction in accuracy, demonstrating its effectiveness in enhancing fairness with minimal performance loss.

Table 3:

Results from 20 repeated experiments are summarized with accuracy, DPD, EOD (mean ± standard deviation). ↓ means that lower values are better, while ↑ means that higher values are better. The best results are in bold.

Modality	Method	Accuracy ↑	DPD ↓	EOD ↓
AV1451	F-MCCA+SVM	0.613 (± 0.04)	0.012 (± 0.02)	0.029 (± 0.05)
	MCCA+SVM	0.615 (± 0.04)	0.015 (± 0.02)	0.032 (± 0.05)
	SVM	0.679 (± 0.03)	0.053 (± 0.04)	0.112 (± 0.05)
AV45	F-MCCA+SVM	0.603 (± 0.05)	0.016 (± 0.02)	0.042 (± 0.06)
	MCCA+SVM	0.603 (± 0.05)	0.023 (± 0.02)	0.065 (± 0.06)
	SVM	0.632 (± 0.04)	0.050 (± 0.04)	0.110 (± 0.05)
MRI	F-MCCA+SVM	0.603 (± 0.03)	0.014 (± 0.02)	0.031 (± 0.04)
	MCCA+SVM	0.588 (± 0.04)	0.020 (± 0.03)	0.051 (± 0.05)
	SVM	0.631 (± 0.04)	0.072 (± 0.04)	0.131 (± 0.05)
Combined Modalities	F-MCCA+SVM	0.614 (± 0.04)	0.008 (± 0.01)	0.015 (± 0.03)
	MCCA+SVM	0.600 (± 0.04)	0.020 (± 0.03)	0.045 (± 0.05)
	SVM	0.665 (± 0.03)	0.078 (± 0.05)	0.108 (± 0.06)

Conclusion

This paper introduces F-MCCA, a fairness-aware extension of multi-modal canonical correlation analysis that mitigates bias in multi-modal data integration. F-MCCA formalizes correlation disparity error and employs a multi-objective optimization framework to ensure unbiased representations across demographic groups while preserving analytical utility. Experiments on ADNI neuroimaging data show that F-MCCA significantly reduces fairness disparities with minimal performance trade-offs in both unsupervised and supervised tasks. Its balance between fairness and performance makes it valuable for healthcare applications, promoting more inclusive biomedical modeling and improving diagnostic accuracy. Future work could explore non-linear extensions of F-MCCA and validate its effectiveness across additional healthcare domains and sensitive attributes beyond sex, further advancing algorithmic fairness in precision medicine.