Cascaded Multi-view Canonical Correlation (CaMCCo) for Early Diagnosis of Alzheimer’s Disease via Fusion of Clinical, Imaging and Omic Features

Abstract

The introduction of mild cognitive impairment (MCI) as a diagnostic category adds to the challenges of diagnosing Alzheimer’s Disease (AD). No single marker has been proven to accurately categorize patients into their respective diagnostic groups. Thus, previous studies have attempted to develop fused predictors of AD and MCI. These studies have two main limitations. Most do not simultaneously consider all diagnostic categories and provide suboptimal fused representations using the same set of modalities for prediction of all classes. In this work, we present a combined framework, cascaded multiview canonical correlation (CaMCCo), for fusion and cascaded classification that incorporates all diagnostic categories and optimizes classification by selectively combining a subset of modalities at each level of the cascade. CaMCCo is evaluated on a data cohort comprising 149 patients for whom neurophysiological, neuroimaging, proteomic and genomic data were available. Results suggest that fusion of select modalities for each classification task outperforms (mean AUC = 0.92) fusion of all modalities (mean AUC = 0.54) and individual modalities (mean AUC = 0.90, 0.53, 0.71, 0.73, 0.62, 0.68). In addition, CaMCCo outperforms all other multi-class classification methods for MCI prediction (PPV: 0.80 vs. 0.67, 0.63).

Introduction

Alzheimer’s Disease (AD) is the most prevalent type of dementia in the US, and is primarily characterized by irreversible cognitive decline associated with neurodegeneration¹. On account of an increasing aging population in the US, the annual incidence of AD is expected to double by 2050². However, studies have shown that the number of cases in 2050 can be reduced by 50% if the average age at the onset of the disease could be delayed by 5 years³. This may be achieved by early diagnosis and intervention with treatments that delay disease progression.

The original diagnostic criteria, known as NINCDS-ADRDA criteria, qualitatively combined information from medical history, clinical examination, neurophysiological testing and laboratory assessments to provide a sensitivity of 81% and a specificity of 70% for AD diagnosis⁴. In an attempt to diagnose AD earlier, the revised criteria now include two major changes: (i) addition of an intermediate diagnostic group, mild cognitive impairment (MCI) as well as (ii) guidelines for interpretation of imaging and molecular markers¹. The intermediate diagnostic category, MCI, comprises of a heterogeneous group of patients who present early symptoms of cognitive impairment which do not interrupt daily life. While some MCI patients progress to AD over time, some remain stable while a few even regress back to healthy states. Given that MCI patients are at a greater risk for AD, there is an opportunity for early diagnosis of AD by identifying the subpopulation of MCI patients who progress to AD. However, to move toward this opportunity, the most immediate challenge is to accurately distinguish MCI from both HC and AD. The new diagnostic criteria therefore includes recommendations for incorporation of alternate biomarkers that have previously shown promise in predicting presymptomatic disease⁵.

Alongside recent developments in imaging and molecular diagnostic technologies, several studies have sought to identify biomarkers of AD. Cerebrospinal fluid (CSF) markers in particular have been extensively studied^6–8 on account of their direct relationship with pathological characteristics of the disease such as amyloid burden and neuronal degeneration. On the genetic front, apolipoprotein E (ApoE) has been established as an indicator of risk for AD⁹. Structural information on 1.5 Tesla T1w Magnetic Resonance Imaging (MRI)¹⁰ such as hippocampal volume and functional information on [18F] fluorodeoxyglucose uptake (FDG-PET)¹¹ such as changes in glucose metabolism have previously shown to be predictive of AD. Availability of multiple, complementary markers and data streams now presents an opportunity to combine different sources of information in order to potentially improve the ability to predict AD early, prior to its onset. However, qualitatively combining the vast amount of information is challenging and likely to result in subjective interpretations. On the other hand, quantitative approaches to identification of fused biomarkers is challenged by differences in data dimensionality, small sample size of most biomedical datasets and by the increase in data dimensionality associated with combining multiscale data^12–15.

Several methods have previously been developed and explored to quantitatively combine multiscale biomedical data. Most data fusion approaches can generally be categorized based on the level at which information is combined: (i) raw data level (low level fusion), (ii) feature level (intermediate level fusion) or (iii) decision level (high level fusion)¹⁶. Data integration at the raw data level is limited to homogeneous data sources and is thus not directly applicable for fusion of multiscale, biomedical data. Alternatively, decision level strategies¹⁷ bypass challenges associated with fusion of heterogeneous data types by combining independently derived decisions from each data source. In doing so, relationships between the different data channels remain largely unexploited^{14, 15}.

Most previous work in prediction of AD employ feature level integration where raw data is first converted into quantitative feature representations which are then combined using concatenation-based^{18, 19}, kernel-based^{20, 21}, manifold-based²² and most recently deep learning-based^{23, 24} methods. A brief summary of select related previous work is provided in Table 1. While feature concatenation^{18, 19} provides a simple method for investigating the added predictive value of each modality, it is sub-optimal for combining modalities with significantly different dimensionalities as modalities with larger feature sets are likely to dominate the joint-representation and hence the fused predictor¹². Kernel-based and manifold-based methods^20–22 alternatively transform raw data from the original space to a high dimensional embedding space where the different data types are more homogeneously represented, thereby making them more amenable for fusion. However, such methods are prone to overfitting^{25, 26} particularly given the small sample sizes of most biomedical datasets and the noise associated with each of the biomedical data sources which, if unaccounted for, may drown the increase in signal achievable by fusion. Suk et al.²³ and Liu et al.²⁴ presented deep learning based fusion approaches which seek to learn integrated structural and functional feature representations from MRI and PET. However, the method is limited to fusion of spatially aligned imaging data. In addition, deep learning methods generally require very large datasets in order to model complex non-linear relationships via several hidden layers. This could very easily result in overfitting on datasets with small sample size, especially in the presence of noise.

Table 1.

Summary of related previous work.

Previous Work	Modalities	Methods	N	Classes and Performance
Gray et al. (NeuroImage, 2013)22	Baseline T1w MRI, FDG PET, CSF	Joint embedding of manifolds constructed using random forest based similarity measure	147	AD/HC (Acc: 89% +/−0.7), MCI/HC (Acc: 74.6% +/−0.8), pMCI/sMCI (Acc: 58% +/−0.9)
Zhang et al. (NeuroImage, 2011)20	Baseline T1w MRI, FDG PET, CSF	Kernel combination method embedded with support vector machine classifier	202	AD/HC (Acc: 93.2%, Sen: 93%, Spec: 93.3%), MCI/HC (Acc: 76.4%, Sen: 81.8%, Spec: 66%; 91.5% pMCI and 73.4% sMCI classified as MCI)
Hinrichs et al. (NeuroImage, 2011)21	Baseline and longitudinal T1w MRI, FDG PET, cognitive measures; Baseline CSF, ApoE	Multi-kernel learning framework with support vector machine classifier	233	AD/HC (Acc: 92.4%, Sen: 86.7%, Spec: 96.6%, AUC: 0.977), pMCI/rMCI (AUC: 0.97), pMCI/sMCI (AUC: 0.77)
Westman et al. (NeuroImage, 2012)39	Baseline T1w MRI, CSF	Orthogonal partial least squares (OPLS)	369	AD/HC (Acc: 91.8%, Sen: 88.5%, Spec: 94.6%, AUC: 0.958), MCI/HC (Acc: 77.6%, Sen: 72.8%, Spec: 84.7%, AUC: 0.876), pMCI/sMCI using AD/HC model (Acc: 58.6%, 65.8%, 66.4%, 66.1%, AUC: 0.594, 0.647, 0.610, 0.578 for conversion within 12, 18, 24 and 36 months, respectively)
Da et al. (NeuroImage, 2014)18	Baseline T1w MRI, Cognitive scores, CSF, ApoE	SVM Classification of concatenated features	432, 381	AD/HC (T1w MRI AUC: 0.98), sMCI/pMCI Kalpan Meier analysis
Davatzikos et al. (Neurobiology, 2011)19	Baseline T1w MRI (SPARE-AD), CSF	SVM Classification of concatenated features; pMCI and sMCI categorization based on global CDR score change at follow-up (6–36 months)	239	sMCI/pMCI (T1w MRI AUC: 0.734, T1w MRI + CSF AUC: 0.671)
Suk et al. (NeuroImage, 2014)23	Baseline T1w MRI, FDG PET	Joint feature representation of image patches using Deep Boltzman Machine (DBM)	194, 305, 204	AD/HC (Acc: 95.35%, AUC: 0.9877), MCI/HC (Acc: 85.67%, AUC: 0.88), pMCI/sMCI (Acc: 75.92%, AUC: 0.747)
Zhu et al. (NeuroImage, 2014)40	T1w MRI, FDG PET, CSF	Feature selection method and regression to predict clinical variables in addition to class labels	202	AD/HC (Acc: 95.9%, AUC: 98.8), MCI/HC (Acc: 82.0%, AUC: 87.0), sMCI/pMCI (Acc: 72.6%, AUC: 78.8%)
Liu et al. (IEEE TMI, 2015)24	Baseline T1w MRI, FDG PET	Fused data representation of image patches using stacked autoencoder for multiclass classification	331	AD/HC (Multiclass Precision: 59.1 +/− 19.7, 52.2 +/− 11.8, 40.2 +/− 14.4, 64.1 +/− 15.24 for HC, sMCI, pMCI and AD, Acc: 53.8 +/− 4.8, Sen: 52.1 +/− 11.8, Spe:87 +/− 9.6)

Regardless of the fusion strategy employed, most previous studies evaluate their methods by simplifying the multiclass problem (HC vs. MCI vs. AD) into the following binary classification tasks – AD vs. HC, MCI vs. HC. Recent work²⁴ showed that multiclass classification resulted in significantly lower predictive performance as compared to that of the aforementioned binary classification tasks, suggesting that all the diagnostic classes must be considered to estimate the performance of a proposed model in a clinical setting. Generally, there are three common methods for multiclass classification – one vs. another, one vs. all (OVA) and one shot classification (OSC). For classification task with $\mathcal{C}=\{c_1,c_2,\dots c_n\}$ classes, the one vs. another classifier attempts to independently solve binary class problems c _i vs. c _j, $i\ne j$ arising from all pairwise combinations of classes. With this strategy, it is unclear how to combine results from the multiple binary problems in order to then determine the overall classifier performance. Alternatively, OVA seeks to solve c _i vs {c _j}, j = 1 … n, $j\ne i$, while OSC attempts to simultaneously solve c ₁ vs. c ₂ vs. … vs. c _n. OVA may not be able to appropriately classify intermediate classes such as MCI where the ‘all’ category comprises of data points that lie on either extrema of disease spectrum (i.e. healthy and AD). While OSC, which classifies multiple classes at once, overcomes the aforementioned limitations of the other two strategies, it assumes that the same set of modalities are optimal for separating all classes. When addressing multiclass problem in the context of data fusion, it may not be realistic to expect the same combination of modalities to be the most informative for all the various classes. In addition, some classification tasks may require information from fewer modalities to provide sufficiently accurate information while other, more challenging tasks may require additional information.

In this work, we introduce the cascaded multiview canonical correlation (CaMCCo) framework which brings together three different unique ideas; data fusion approach, modality selection concept and a cascaded classification scheme. The CaMCCo approach is employed in this paper for the problem of AD diagnosis. CaMCCo seeks to fuse a subset of modalities from T1w MRI, FDG PET, ApoE, CSF, plasma proteomics and neurophysiological exam scores in order to optimize classifier performance at each level of the cascade (Fig. 1). For data fusion, CaMCCo employs supervised multiview canonical correlation analysis (sMVCCA)^{27, 28} which provides a common, low dimensional representation that is discriminative of classes and allows for combining any number of heterogeneous forms of multidimensional, multimodal data. The fusion scheme operates under the assumption that information overlap increases with increasing number of data sources or ‘views’ as all views fundamentally capture information pertaining to the same object. As such, it seeks to maximize correlations between modalities and with class labels.

Figure 1.

The cascade and the modalities for fusion at each level of the cascade were determined on training set and validated on independent testing set. Neurophysiological test scores (ADAS-Cog) are fused with CSF proteomics and APOE at the first level of the cascade to identify healthy controls (HC). At the second level, ADAS-Cog scores are combined with PET to distinguish between patients with Alzheimer’s Disease (AD) and mild cognitive impairment (MCI).

Previous work^{27, 28} has shown the application of sMVCCA in the context of predicting prostate cancer prognosis where sMVCCA based fusion of histologic and proteomic features was found to be more discriminative of classes as compared to individual modalities as well as several fused representations including LDA, CCA, MVCCA, PCA, regularized CCA (RCCA), supervised regularized CCA (SRCCA), and generalized embedding concatenation (GEC). Although sMVCCA is able to fuse any number of modalities, the practicality of its application in a clinical setting where the trade-off between added improvement in performance and increased burden of additional tests must be leveraged²⁹. CaMCCo therefore extends on previous work to address clinical challenges associated with AD diagnosis by employing the fusion methodology within cascaded classification framework where only a subset of modality(ies) that maximize the performance for each classification task are fused at each level of cascade. Unlike most prior applications of multiclass classification methods to AD diagnosis, CaMCCo simultaneously considers all diagnostic classes via its cascaded classification approach. Unlike most prior applications of data fusion methods for AD diagnosis, CaMCCo seeks to identify, selectively retain and combine only the most informative data source(s) for each class label. As shown in Fig. 1, each patient is first classified as being healthy or cognitively impaired (CI) using ADAS-Cog score, CSF and APOE. If classified as CI, the ADAS-Cog and PET are used to distinguish between MCI and AD cases.

Methods

Supervised Multiview Canonical Correlation Analysis for Data Fusion

We apply supervised Multiview Canonical Correlation Analysis (sMVCCA)^{27, 28}, an extension of canonical correlation analysis (CCA) and multiview canonical correlation analysis (MVCCA)³⁰, to obtain a low-dimensional, shared representation of the modalities of interest. CCA³¹ is a linear dimensionality reduction method commonly used for data fusion as it accounts for relationships between two sets of input variables. MVCCA generalizes CCA by finding the linear subspace where pairwise correlations between multiple (more than two) modalities can be maximized. However, both CCA and MVCCA are unsupervised and therefore do not guarantee a subspace that is optimal for class separation. sMVCCA is a supervised form of MVCCA where class labels are embedded as one of the variable sets. Additional details and formulations for CCA and MVCCA are provided in the appendix and the theoretical framework for sMVCCA is provided below. Table 2 provides a summary of notations used in this section.

Table 2.

Summary of Notations.

Symbol	Description
n, N	subjects, total number of subjects
k, K	modalities, total number of modalities; x k, k ∈ {1, …, k}
m, M k	features, total number of features in each modality; m ∈ {1, …, M k}
M	total number of features over all modalities; M = ∑k  M k
x k	data matrix containing features from modality k for all subjects, ${ℝ}^{n\times M_k}$
X	concatenated data matrix containing all features from all modalities $\left[{\mathbf{x}}_1,\dots ,{\mathbf{x}}_K\right]$, ${ℝ}^{n\times (M_1+\dots +M_K)}$
w k	weight vector for modality k, ${ℝ}^{M_k\times 1}$
W k	weight matrix for modality k, ${ℝ}^{M_k\times n}$
w	concatenated weight vector over all modalities ${\left[{\mathbf{w}}_1^T,{\mathbf{w}}_2^T,\dots ,{\mathbf{w}}_K^T\right]}^T$, ${ℝ}^{M\times 1}$
W x	weight matrix for all modalities $\left[{\mathbf{W}}_1,{\mathbf{W}}_2,\dots ,{\mathbf{W}}_M\right]$, ${ℝ}^{M\times n}$
Y	label matrix ${ℝ}^{n\times G}$
g, G	classes, total number of classes
W y	notation used in sMVCCA to denote W for all labels ${ℝ}^{g\times n}$
i	data vector of selected modalities $i\subseteq k$
p	total number of features over modalities in i, $p={\sum }_i{M}_i$
X i	concatenated data matrix containing all features from a subset of modalities [x i], ${ℝ}^{n\times p}$
d	dimensionality of the fused data subspace

Consider a multimodal dataset $\mathbf{X}\in \left\{x_1,\dots ,x_k,\dots ,x_K\right\}$ in ${ℝ}^{n\times M}$, where n is the number of subjects, K is the number of modalities and x _k in ${ℝ}^{n\times M_k}$ refers to the feature matrix of modality k containing M _k features. Additionally, X has a corresponding binary class label matrix Y in ${ℝ}^{n\times G}$, where G is the total number of classes. sMVCCA seeks to maximize correlation within the modalities in X and between X and Y as shown below

$$\begin{array}{ll}\underset{{\mathbf{w}}_1,\mathrm{...},{\mathbf{w}}_k,\mathrm{...},{\mathbf{w}}_K,{\mathbf{w}}_Y}{\text{arg}\max }\hfill & \sum _{k\ne 1}\sum {\mathbf{w}}_k^T{\mathbf{x}}_k^T{\mathbf{x}}_j{\mathbf{w}}_j+\sum _k{\mathbf{w}}_k^T{\mathbf{x}}_k^T\mathbf{Y}{\mathbf{w}}_Y\hfill \\ s\mathrm{.}t\mathrm{.}\hfill & {\mathbf{w}}_1^T{\mathbf{x}}_1^T{\mathbf{x}}_1{\mathbf{w}}_1=\mathrm{1,}\dots ,{\mathbf{w}}_K^T{\mathbf{x}}_K^T{\mathbf{x}}_K{\mathbf{w}}_K=1,{\mathbf{w}}_Y^T{\mathbf{x}}_Y^T{\mathbf{x}}_Y{\mathbf{w}}_Y=1\hfill \end{array}$$ 1

$$s\mathrm{.}t\mathrm{.}{\mathbf{w}}_1^T{\mathbf{x}}_1^T{\mathbf{x}}_1{\mathbf{w}}_1=1,\dots ,{\mathbf{w}}_K^T{\mathbf{x}}_K^T{\mathbf{x}}_K{\mathbf{w}}_K=1,{\mathbf{w}}_Y^T{\mathbf{x}}_Y^T{\mathbf{x}}_Y{\mathbf{w}}_Y=1$$ 2

This can be expressed in a compact matrix form as follows:

$$\begin{array}{l}\underset{{\mathbf{W}}_x,{\mathbf{W}}_y}{\arg \max }trace({\mathbf{W}}_x^T\overline{\mathbf{C}}{\mathbf{W}}_x)+2\times trace({\mathbf{W}}_x^T{\mathbf{X}}^T\mathbf{Y}{\mathbf{W}}_y)\hfill \\ =trace\left(\left[\begin{array}{cc}{\mathbf{W}}_x^T& {\mathbf{W}}_y^T\end{array}\right]\left[\begin{array}{ll}\overline{\mathbf{C}}\hfill & {\mathbf{X}}^T\mathbf{Y}\hfill \\ {\mathbf{Y}}^T\mathbf{X}\hfill & \mathbf{0}\hfill \end{array}\right]\left[\begin{array}{c}{\mathbf{W}}_x\\ {\mathbf{W}}_y\end{array}\right]\right)\hfill \\ =trace({\stackrel{\mathbf{ˆ}}{\mathbf{W}}}^T\stackrel{\mathbf{ˆ}}{\mathbf{C}}\stackrel{\mathbf{ˆ}}{\mathbf{W}})\hfill \\ s.t.\hfill \end{array}$$

$$\begin{array}{lll}\hfill \left[\begin{array}{ll}{\mathbf{W}}_x^T\hfill & {\mathbf{W}}_y^T\hfill \end{array}\right]\left[\begin{array}{cc}{\overline{\mathbf{C}}}_d& \mathbf{0}\\ \mathbf{0}& {\mathbf{Y}}^T\mathbf{Y}\end{array}\right]\left[\begin{array}{c}{\mathbf{W}}_x\\ {\mathbf{W}}_y\end{array}\right]& \hfill =\hfill & \mathbf{I}\hfill \\ \hfill \iff {\stackrel{\mathbf{ˆ}}{\mathbf{W}}}^T{\stackrel{\mathbf{ˆ}}{\mathbf{C}}}_d\stackrel{\mathbf{ˆ}}{\mathbf{W}}& \hfill =\hfill & \mathbf{I}\hfill \end{array}$$ 3

$${\mathbf{W}}_1^T{\mathbf{C}}_{11}{\mathbf{W}}_1=\cdots ={\mathbf{W}}_K^T{\mathbf{C}}_{KK}{\mathbf{W}}_K={\mathbf{W}}_y^T{\mathbf{Y}}^T\mathbf{Y}{\mathbf{W}}_y.$$ 4

where Y is a matrix in which class labels are encoded using Soft-1-of-Class strategy³².

Solving Equation 3 consists of two steps: (i) Ignoring the constraint in (4) leaves us with a quadratic programming problem, whose W* corresponds to eigenvectors of the n-largest eigenvalues of a generalized eigenvalue system: ${\mathbf{C}}_{\mathbf{xy}}\mathbf{W}=\lambda {\mathbf{C}}_{{\mathbf{d}}_{\mathbf{xy}}}\mathbf{W}$; (ii) Imposing constraint (4) upon obtaining the optimal eigenvectors W* by normalizing the corresponding section of each modality: ${\mathbf{W}}_{\mathbf{j}}^{⁎⁎}={\mathbf{W}}_{\mathbf{j}}^{⁎}{({\mathbf{W}}_{\mathbf{j}}^{⁎\mathbf{T}}{\mathbf{C}}_{\mathbf{jj}}{\mathbf{W}}_{\mathbf{j}}^{⁎})}^{-\frac{1}{2}},j=1,\dots ,k$.

Cascaded Multi-view Canonical Correlation Analysis (CaMCCo)

As shown in Fig. 2, CaMCCo divides the classification task for a multiclass, multimodal dataset into a cascade of multiple, sequential binary classification tasks, for each of which the optimal fused representation is independently determined and provided as input to the classifier. For the multimodal dataset X in ${ℝ}^{n\times M}$, consider a label matrix Y in ${ℝ}^{n\times G}$. A subset modalities suitable for classifying class g from all input samples can be denoted as i where $i\subseteq k$. Features from modalities in i are concatenated to generate ${\mathbf{X}}_{\mathbf{i}}^{\prime }$ in ${ℝ}^{n\times p}$, where $p={\sum }_i{M}_i$ and $p\le M$. The i modalities are fused via sMVCCA to reduce the dimensionality from p to d, where $d\ll p$, resulting in ${\mathbf{X}}_{\mathbf{i}}^{″}$. Subsequently, ${\mathbf{X}}_{\mathbf{i}}^{″}$ serves as the input to a classifier which predicts if each sample does or does not belong to class g, ${\hat{y}}_g=1$ and ${\hat{y}}_g=0$, respectively. The multimodal dataset consisting of only samples classified as $\hat{y}=0$ subsequently serves as the input for the next level of cascade where the modality selection, data fusion and classification steps are repeated for another class in Y.

Figure 2.

Cascaded multiview canonical correlation analysis (CaMCCo) algorithm for constructing the joint multimodal data fusion and multiclass classification framework.

Therefore, designing the cascaded classifier for CaMCCo requires determination of (a) the sequence of classification tasks that provide the best overall classifier performance, as well as the respective (b) number and (c) type of modalities to combine at each level of the cascade. In this work, these parameters were determined experimentally on the training cohort as described in Section 3.5.

Experimental Design

Dataset Description

Data used in the preparation of this work were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (www.loni.ucla.edu/ADNI). The ADNI was launched in 2003 with the primary goal of testing whether serial magnetic resonance imaging (MRI), positron emission tomography (PET), other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of mild cognitive impairment (MCI) and early Alzheimer’s disease (AD). The initial goal of ADNI was to recruit 800 adults, ages 55 to 90, to participate in the research – approximately 200 cognitively normal older individuals to be followed for 3 years, 400 people with MCI to be followed for 3 years, and 200 people with early AD to be followed for 2 years (see www.adni-info.org for up-to-date information). The research protocol was approved by each local institutional review board and written informed consent. In addition to raw data, the ADNI database contains several post-processed and individually evaluated biomarkers.

In this work, we consider a subset of cases for which the following was available in the database (i) pre-computed features from T1w MRI and FDG PET, (ii) neurocognitive ADAS-cog score, (iii) complete record of CSF Proteomics, Plasma Proteomics, ApoE and (iv) clinical diagnosis at baseline. 149 ADNI participants who fulfilled the criteria were included, of which 52 were diagnosed with Alzheimer’s Disease (AD), 71 were diagnosed with mild cognitive impairment (MCI), and 26 were healthy controls (HC).

Table 3 provides the clinical and demographic details of the population considered in this study as per their diagnosis at baseline. The unique ADNI database provided RID of all patients considered in this study is provided in the Appendix.

Table 3.

Clinical and demographic information of the 149 ADNI subjects considered in this study, selected based on the availability of imaging, non-imaging and clinical metrics at baseline.

Diagnosis	N(F/M)	Age	MMSE Score
AD	52 (16/24)	75.1 +/− 8.1	23.8 +/− 2.0
MCI	71 (17/37)	74.1 +/− 7.2	27.1 +/− 1.7
HC	26 (10/14)	74.9 +/− 7.3	28.6 +/− 1.4
Total	149 (43/75)	74.2 +/− 7.2	26.3 +/− 2.6

The dataset was split into independent training set with 60 cases (40%) and a holdout validation set with 89 cases (60%). Note that gender information was unavailable for a subset of the data, as a result of which N does not equal to the sum of females (F) and males (M).

Feature Description

Table 4 summarizes the number and types of features considered in this study for each modality. From imaging data, we consider volumetric features extracted from T1w MRI and measures of hippocampal glucose metabolism^{33, 34} from FDG PET. Considered molecular markers include proteomic measurements from cerebrospinal fluid (CSF), plasma from the biomarker consortium and geneotype ApoE data. We additionally included a neurophysiological test score as such tests serve as the primary means for diagnosis in the current clinical setting. Although the Mini-Mental State Examination (MMSE) is the most commonly performed clinical test, we avoid using MMSE scores as they were used to determine the “ground truth” labels on which we train and test CaMCCo. Therefore, we use an alternate test score, modified Alzheimer’s Disease Assessment Scale - Cognition (ADAS-Cog) which has been used to assess the effects of experimental treatments for AD in clinical trials³⁵.

Table 4.

Summary of features considered in this study from each modality.

Modality	Features	Description	Number
Neurophysiologic Exam	Modified ADAS-Cog score41	Score based on cognitive test assessing memory, praxis, orientation, word recall and recognition	1
T1w MRI	Volumetric Measurements	Volumetric measures of atlas based segmented brain regions	327
FDG PET	Hippocampal Glucose Metabolism33, 34	Pons normalized left and right hippocampal glucose metabolism	2
CSF Proteomics	t-tau, Aβ 1–42, p-tau181	Markers of neuronal degeneration, plaque formation and tau hyperphosphorylation42	3
Plasma Proteomics	Adiponectin, Insulin, Fibrinogen etc43	Concentrations of signaling proteins in blood, measured by multiplex immunoassay panel	146
ApoE Genotype	ApoE alleles 1 & 2	Combination of allele forms ε 2, ${\epsilon }_3$, ${\epsilon }_4$	1

Classification Model

The dataset was split into training and a holdout validation set with each comprising 40% and 60% of the data, respectively. Classification and fusion parameters were determined on the training set using 10 iterations of 5-fold stratified cross validation, upon which the optimized classifier trained on the full training set was applied to the independent validation set. Naive Bayes classifier³⁶ was used to evaluate the various fused and individual modality representations. Naive Bayes is a widely used, well-established probabilistic classifier that is known to perform well on small datasets.

Evaluation metrics

Performance measures used to evaluate each classification task include: accuracy (ACC), balanced accuracy (BACC)³⁷, area under the receiver operating characteristic curve (AUC)³⁸, sensitivity (SEN), specificity (SPE) and positive predictive value (PPV). The definitions and descriptions of each of these metrics are provided in Table 8 in the Appendix.

CaMCCO Model

Class groupings and modalities selected for fusion at each level of the cascaded classification design employed by CaMCCo (Fig. 1) was determined experimentally on the training set. One-vs-all (AD vs. all, MCI vs. all, HC vs. all) classifiers were constructed and evaluated independently for each considered modality. The task that most consistently resulted in the highest AUC across all modalities served as the first level of the cascade so as to reduce error propagation. Among AD, MCI and HC, the remaining classes were assigned to the second level of the cascade.

For every classification task within the cascade, each modality was ranked based on the AUC it achieved across iterations and cross validation folds within the training set. The n highest performing modalities were fused via sMVCCA, where n was varied from 2 to 6 (total number of considered modalities). The n modalities, which in combination, provided the highest training AUC were selected.

Comparative Strategies

CaMCCo represents a framework ($\mathscr{H}$) composed of multiple modules corresponding to modality selection ($\mathcal{K}$), multimodal data fusion ($ℱ$), and multiclass classification ($\mathcal{C}$). Accordingly, the comparative strategies against which we evaluate CaMCCo involve systematically replacing the method used for one or more of these modules with an alternative strategy. Table 5 lists the notation for each of these strategies and provides a short description.

Table 5.

Summary of notations used to refer to comparative strategies evaluated in this work.

Symbol	Description
Classification Methods
${\mathcal{C}}_{CAS}$	Cascaded Classifier
${\mathcal{C}}_{OSC}$	One Shot Classifier
${\mathcal{C}}_{OVA}$	One-vs-All Classifier
${\mathcal{C}}_{BIN}$	Binary Classifier
Data Fusion Methods
${ℱ}_{SMVCCA}$	Supervised Multiview CCA data fusion approach
${ℱ}_{PCA}$	Principal component analysis of concatenated features as baseline fusion approach
Modality Selection
${\mathcal{K}}_{ALL}$	Multimodal dataset comprising all modalities considered in this study
${\mathcal{K}}_{SEL}$	Multimodal dataset comprising select subset of all available modalities
${\mathcal{K}}_{MRI}$	Unimodal dataset containing quantitative attributes extracted from MRI
${\mathcal{K}}_{PET}$	Unimodal dataset containing quantitative attributes extracted from PET
${\mathcal{K}}_{CSF}$	Unimodal dataset containing proteomic measurements from CSF
${\mathcal{K}}_{PP}$	Unimodal dataset containing plasma proteomic data
${\mathcal{K}}_{APOE}$	Unimodal dataset containing APOE data
${\mathcal{K}}_{ADAS}$	Unimodal dataset containing ADAS-Cog scores
Comparative Methods
${\mathscr{H}}_{MRI}$ (${\mathcal{C}}_{CAS}$ + ${\mathcal{K}}_{MRI}$)	Cascaded classification of single modality MR data
${\mathscr{H}}_{PET}$ (${\mathcal{C}}_{CAS}$ + ${\mathcal{K}}_{PET}$)	Cascaded classification of single modality PET data
${\mathscr{H}}_{CSF}$ (${\mathcal{C}}_{CAS}$ + ${\mathcal{K}}_{CSF}$)	Cascaded classification of single modality CSF data
${\mathscr{H}}_{PP}$ (${\mathcal{C}}_{CAS}$ + ${\mathcal{K}}_{PP}$)	Cascaded classification of single modality Plasma Proteomics data
${\mathscr{H}}_{APOE}$ (${\mathcal{C}}_{CAS}$ + ${\mathcal{K}}_{APOE}$)	Cascaded classification of single modality APOE data
${\mathscr{H}}_{ADAS}$ (${\mathcal{C}}_{CAS}$ + ${\mathcal{K}}_{ADAS}$)	Cascaded classification of single modality ADAS-Cog data
${\mathscr{H}}_{CAMCCO}$ (${\mathcal{C}}_{CAS}$ + ${ℱ}_{SMVCCA}$ + ${\mathcal{K}}_{SEL}$)	Classifier resulting from CaMCCo framework, which is comprised of cascaded classifier, sMVCCA data fusion and modality selection
${\mathscr{H}}_{ALL}$ (${\mathcal{C}}_{CAS}$ + ${ℱ}_{SMVCCA}$ + ${\mathcal{K}}_{ALL}$)	Cascaded classifier in combination with sMVCCA based data fusion method to combine all modalities
${\mathscr{H}}_{PCAL}$ (${\mathcal{C}}_{CAS}$ + ${ℱ}_{PCA}$ + ${\mathcal{K}}_{ALL}$)	Cascaded classifier with PCA reduced representation of data concatenated from all modalities
${\mathscr{H}}_{PCA}$ (${\mathcal{C}}_{CAS}$ + ${ℱ}_{PCA}$ + ${\mathcal{K}}_{SEL}$)	Cascaded classifier with PCA reduced representation of data concatenated from selected subset of modalities
${\mathscr{H}}_{OVA}$ (${\mathcal{C}}_{OVA}$ + ${ℱ}_{SMVCCA}$ + ${\mathcal{K}}_{SEL}$)	One-vs-all classifier constructed from sMVCCA fused data from selected modalities.
${\mathscr{H}}_{OSC}$ (${\mathcal{C}}_{OSC}$ + ${ℱ}_{SMVCCA}$ + ${\mathcal{K}}_{SEL}$)	One shot classifier constructed from sMVCCA fused data from selected modalities.
${\mathscr{H}}_{BIN}$ (${\mathcal{C}}_{BIN}$ + ${ℱ}_{SMVCCA}$ + ${\mathcal{K}}_{SEL}$)	Binary classifier constructed from sMVCCA fused data from selected modalities.

Single Modality and Multimodality Approaches

Each modality was evaluated using a single modality framework (${\mathscr{H}}_{MRI}$, ${\mathscr{H}}_{PET}$, ${\mathscr{H}}_{CSF}$, ${\mathscr{H}}_{PP}$, ${\mathscr{H}}_{APOE}$, ${\mathscr{H}}_{ADAS}$) consisting of cascaded classification (${\mathcal{C}}_{CAS}$) to ensure fair comparison with CaMCCo. In addition, we compared classification performance of CaMCCo with that of a cascaded classification model where all modalities were fused at each level of the cascade (${\mathscr{H}}_{ALL}$).

Principal Component Analysis for Data Fusion

Principal Component Analysis (PCA) is a dimensionality reduction method which projects input data onto an alternate subspace defined by orthogonal basis vectors which capture the direction of variance in the data. Consider a high dimensional, concatenated multimodal data matrix $\mathbf{X}=[x_1,\dots ,x_k,\dots x_K]\in {ℝ}^{n\times M_1+\dots +M_k+\dots +M_K}$ where K refers to the number of modalities, n refers to the number of subjects, and M _k refers to the number of features in modality k.

$$[\mathbf{U},\mathbf{S},\mathbf{V}]=\mathrm{SVD}(\overline{\mathbf{X}}{\overline{\mathbf{X}}}^T)$$

Singular value decomposition is applied to mean centered data matrix, $\overline{\mathbf{X}}$, which results in U, S, V. The columns of $\mathbf{V}\in {ℝ}^{(M_1+\dots +M_k+\dots +M_K)\times (M_1+\dots +M_k+\dots +M_K)}$ are the principal components of $\overline{\mathbf{X}}$ or the orthogonal basis vectors, ordered decreasingly by the amount of variance in the dataset explained by each component. $\mathbf{U}\in {ℝ}^{n\times M_1+\dots +M_k+\dots +M_K}$ contains the projections of $\overline{\mathbf{X}}$ on the subspace defined by V. $\mathbf{S}\in {ℝ}^{(M_1+\dots +M_k+\dots +M_K)\times (M_1+\dots +M_k+\dots +M_K)}$ is a diagonal matrix. To reduce data dimensionality, the top d principal components containing most of the variance in the data are retained, onto which the data is projected.

Multiclass Classification

For a classification task with G classes, One-vs-All (OVA) method constructs G classifiers, each tailored to separate one class from the rest. One Shot Classification (OSC) generates a single classifier designed to simultaneously distinguish between all the classes. The last comparative strategy involves the following binary classification tasks, AD vs. HC and MCI vs. HC.

Experiment 1: Single Modality and Multi-Modality Cascaded Classification

The objective of this experiment is to examine (i) classification performance achieved by combining multiple modalities as compared to any single modality for all classification tasks within the cascade design. In addition, the experiment seeks to determine if (ii) combining subsets of modalities tailored to optimize classification at each level of the cascade provides comparable and/or improved performance as compared to combining all the modalities for all tasks. Finally, it also evaluates (iii) the impact of the chosen fusion method on the findings for (i) and (ii).

To meet these objectives, we compare the data fusion ($ℱ$) and modality selection ($\mathcal{K}$) modules in CaMCCo (${\mathscr{H}}_{CAMCCO}={\mathcal{C}}_{CAS}+{ℱ}_{SMVCCA}+{\mathcal{K}}_{SEL}$) with other fusion (${ℱ}_{PCA}$), and modality selection (${\mathcal{K}}_{ALL}$) approaches, including the simple single modality (${\mathcal{K}}_{MRI}$, ${\mathcal{K}}_{PET}$, ${\mathcal{K}}_{CSF}$, ${\mathcal{K}}_{PP}$, ${\mathcal{K}}_{APOE}$, ${\mathcal{K}}_{ADAS}$) classification. For individual modality experiments, PCA was applied to the experiments where the number of features were larger than the number of samples to avoid curse of dimensionality. For fused and concatenated classifiers, the number of reduced dimensions was optimized on the training set. Therefore, we consider all the combinations of modalities and fusion methods listed below:

$$\begin{array}{lll}\hfill {\mathscr{H}}_{ALL}& \hfill =\hfill & {\mathcal{C}}_{CAS}+{ℱ}_{SMVCCA}+{\mathcal{K}}_{ALL}\hfill \\ \hfill {\mathscr{H}}_{PCA}& \hfill =\hfill & {\mathcal{C}}_{CAS}+{ℱ}_{PCA}+{\mathcal{K}}_{SEL}\hfill \\ \hfill {\mathscr{H}}_{PCAL}& \hfill =\hfill & {\mathcal{C}}_{CAS}+{ℱ}_{PCA}+{\mathcal{K}}_{ALL}\hfill \\ \hfill {\mathscr{H}}_{MRI}& \hfill =\hfill & {\mathcal{C}}_{CAS}+{\mathcal{K}}_{MRI}\hfill \\ \hfill {\mathscr{H}}_{PET}& \hfill =\hfill & {\mathcal{C}}_{CAS}+{\mathcal{K}}_{PET}\hfill \\ \hfill {\mathscr{H}}_{CSF}& \hfill =\hfill & {\mathcal{C}}_{CAS}+{\mathcal{K}}_{CSF}\hfill \\ \hfill {\mathscr{H}}_{PP}& \hfill =\hfill & {\mathcal{C}}_{CAS}+{\mathcal{K}}_{PP}\hfill \\ \hfill {\mathscr{H}}_{APOE}& \hfill =\hfill & {\mathcal{C}}_{CAS}+{\mathcal{K}}_{APOE}\hfill \\ \hfill {\mathscr{H}}_{ADAS}& \hfill =\hfill & {\mathcal{C}}_{CAS}+{\mathcal{K}}_{ADAS}\hfill \end{array}$$

Experiment 2: Comparison of Multi-Class Classification Strategies for Fused Predictors

The objective of this experiment is to compare the cascaded classification method ${\mathcal{C}}_{CAS}$ used in CaMCCo (${\mathscr{H}}_{CAMCCO}={\mathcal{C}}_{CAS}+{ℱ}_{SMVCCA}+{\mathcal{K}}_{SEL}$) with other multiclass classification methods including OVA (${\mathcal{C}}_{OVA}$) and OSC (${\mathcal{C}}_{OSC}$). To ensure that only the classification module of the CaMCCo framework is evaluated, comparative classification strategies are combined with the same data fusion (${ℱ}_{SMVCCA}$) and modality selection method (${\mathcal{K}}_{SEL}$) as CaMCCo. Therefore,${\mathscr{H}}_{OVA}={\mathcal{C}}_{OVA}+{ℱ}_{SMVCCA}+{\mathcal{K}}_{SEL}$, and ${\mathscr{H}}_{OSC}={\mathcal{C}}_{OSC}+{ℱ}_{SMVCCA}+{\mathcal{K}}_{SEL}$. As with CaMCCo, the optimal set of modalities to combine for each classification task associated with OVA and OSC are determined experimentally from the training set.

Experiment 3: Evaluation of Fused Representation on Binary Classification Tasks

We perform binary classification (${\mathcal{C}}_{BIN}$) for the following two sets of classes, HC vs. AD and MCI vs. HC, in order to allow for direct comparison of the performance of fusion approach used in CaMCCo (${ℱ}_{SMVCCA}$ + ${\mathcal{K}}_{SEL}$) with that reported in literature. As with CaMCCo, the optimal set of modalities to combine for each classification task are determined experimentally from the training set. In addition, we also report binary classification results achieved by individual modalities to examine the effect of fusion for these classification tasks and also to gain insight into the differences in classifier performance on account of the data cohort used in this study as compared to those in other studies.

Results and Discussion

Experiment 1: Single Modality and Multi-Modality Cascaded Classification

Figure 3 shows the performance of cascaded classifier when applied to (i) single modalities (${\mathscr{H}}_{MRI}$, ${\mathscr{H}}_{PET}$, ${\mathscr{H}}_{CSF}$, ${\mathscr{H}}_{PP}$, ${\mathscr{H}}_{APOE}$, ${\mathscr{H}}_{ADAS}$), (ii) fusion of all modalities (${\mathscr{H}}_{ALL}$, ${\mathscr{H}}_{PCAL}$) and (iii) fusion of selected modalities with multiple fusion methods (${\mathscr{H}}_{CAMCCO}$, ${\mathscr{H}}_{PCA}$) for prediction of HC, MCI and AD on the testing cohort. As shown in Fig. 1, ADAS-Cog, CSF and APOE were combined at the first cascade level (HC vs. All) and ADAS-Cog, and PET were combined at the second level (AD vs. MCI) in both ${\mathscr{H}}_{CAMCCO}$ and ${\mathscr{H}}_{PCA}$. For HC vs. all, ${\mathscr{H}}_{CAMCCO}$ shows higher performance (AUC = 0.97) as compared to all individual modalities (max AUC = 0.93), ${\mathscr{H}}_{PCA}$ (AUC = 0.94), ${\mathscr{H}}_{PCAL}$ and ${\mathscr{H}}_{ALL}$ (AUC = 0.52). Among the individual modalities, ${\mathscr{H}}_{ADAS}$, ${\mathscr{H}}_{CSF}$ and ${\mathscr{H}}_{APOE}$ provided the top 3 classification AUCs, which was consistent with observations in the training set which led to these three modalities being selected for fusion in CaMCCo.

Figure 3.

Performance of single and multi modality cascaded classifiers. Area under the ROC curve (AUC) for prediction of (a) healthy control (HC) from all cognitive impairments, and (b) mild cognitive impairment (MCI) from Alzheimer’s Disease (AD).

For AD vs. MCI, ${\mathscr{H}}_{CAMCCO}$, ${\mathscr{H}}_{PCA}$ and ${\mathscr{H}}_{ADAS}$ showed similar performances (AUC = 0.89). This may, in part, be on account of the lack of orthogonality in the features being fused. A correlation test between the ADAS and PET features showed correlation coefficients between −0.49 and −0.51 with p-value < 0.01. In fact, the modality selection strategy employed in this work is limited in that it does not account for relationships between modalities to identify those that optimize the performance when fused. Instead, the selection of modalities are simplified and are based on their individual performances. However, it is interesting to note that despite the significantly poorer performance of ${\mathscr{H}}_{PET}$ as compared to ${\mathscr{H}}_{ADAS}$, combining PET with ADAS-Cog does not degrade the performance. We note that, similar to HC vs. All, ${\mathscr{H}}_{CAMCCO}$ significantly outperforms ${\mathscr{H}}_{ALL}$.

Experiment 2: Comparison of Multi-Class Classification Strategies for Fused Predictors

Table 6 shows the performance of ${\mathscr{H}}_{CAMCCO}$, ${\mathscr{H}}_{OVA}$ and ${\mathscr{H}}_{OSC}$ for prediction of HC, MCI and AD. Across all 3 classes, it is evident that ${\mathscr{H}}_{OVA}$ and ${\mathscr{H}}_{CAMCCO}$ outperform ${\mathscr{H}}_{OSC}$. ${\mathscr{H}}_{OVA}$ and ${\mathscr{H}}_{CAMCCO}$ show comparable AUCs for AD classification, although ${\mathscr{H}}_{OVA}$ shows incrementally higher accuracy, sensitivity and specificity as compared to ${\mathscr{H}}_{CAMCCO}$. For MCI classification however, ${\mathscr{H}}_{CAMCCO}$ significantly outperforms ${\mathscr{H}}_{OVA}$ in terms of all metrics (OVA AUC = 0.78 vs. CaMCCo AUC = 0.88). This is on account of the lower classification performance of MCI vs. all which is a challenging task provided the heterogeneity of the ‘all’ category which consists of both AD and HC patients. Therefore, CaMCCo provides the most optimal performance overall, across all 3 classes.

Table 6.

Performance of multiclass classification strategies – one shot classifier (OSC), one vs. all (OVA), cascaded classifier in CaMCCo – upon fusion of modalities chosen from training set for each classification task.

		ACC	BACC	AUC	SEN	SPEC	PPV
CN	${\mathscr{H}}_{OSC}$	0.69	0.63	0.96	0.34	0.92	0.75
	${\mathscr{H}}_{OVA}$	0.89	0.77	0.97	0.59	0.96	0.77
	${\mathscr{H}}_{CAMCCO}$	0.89	0.77	0.97	0.59	0.96	0.77
MCI	${\mathscr{H}}_{OSC}$	0.69	0.69	0.77	0.68	0.70	0.67
	${\mathscr{H}}_{OVA}$	0.68	0.68	0.77	0.78	0.59	0.63
	${\mathscr{H}}_{CAMCCO}$	0.80	0.78	0.89	0.88	0.69	0.80
AD	${\mathscr{H}}_{OSC}$	0.69	0.67	0.84	0.53	0.82	0.69
	${\mathscr{H}}_{OVA}$	0.85	0.82	0.90	0.72	0.91	0.81
	${\mathscr{H}}_{CAMCCO}$	0.80	0.78	0.89	0.69	0.88	0.80

The highest accuracy (ACC), balanced accuracy (BACC), area under the ROC curve (AUC), sensitivity (SEN), specificity (SPE) and positive predictive value (PPV) achieved for each class are shown in bold. These results indicate that although the performance of both ${\mathscr{H}}_{OVA}$ and ${\mathscr{H}}_{CAMCCO}$ are comparable for CN and AD classification, ${\mathscr{H}}_{CAMCCO}$ outperforms all other methods for MCI classification.

Experiment 3: Evaluation of Fused Representation on Binary Classification Tasks

Table 7 shows ${\mathscr{H}}_{BIN}$ results obtained by combining select few modalities (${\mathcal{K}}_{SEL}$) via sMVCCA (${ℱ}_{SMVCCA}$) for the following binary classification tasks: (i) AD vs. HC and (ii) MCI vs. HC. On the training set, the fusion of ADAS-Cog and CSF provided the best classification AUC for AD vs. HC whereas the fusion of ADAS-Cog, CSF and PET provided the best classification for MCI vs. HC. Thus, ${\mathscr{H}}_{BIN}$ for the two classification tasks fused the respective, aforementioned modalities for the test set. For the former classification task, the performance of ${\mathscr{H}}_{BIN}$ was similar to that of the best performing individual modality, ADAS-Cog, which already provided near perfect AUC leaving little scope for improvement. According to other evaluation metrics, ADAS-Cog outperforms the fused representation. For the more challenging MCI vs. HC classification task however, ${\mathscr{H}}_{BIN}$ improves classifier performance slightly in terms of AUC (0.92 vs. 0.93) but more significantly in terms of BACC (0.77 vs. 0.82) and SPEC (0.65 vs. 0.71). In comparison to most previous work, our individual modality and fused modality results appear to be slightly higher possibly on account of the features that were considered in this work, all of which were quality controlled and independently proven to provide good performance previously. In addition, this work considers a neurocognitive score (ADAS-Cog), a measure that is mostly either used as a response variable or unconsidered in many fusion studies. The ADAS score appears to be strongly predictive of all classification tasks, possibly on account of a strong correlation with the MMSE scores, which were used to derive the ground truth class labels. Therefore, most gains in classification accuracy appears to be only slightly incremental. A correlation test between ADAS scores and MMSE scores across 118 patients, who had the latter data available, showed that the two were indeed highly correlated with a coefficient of −0.65 and p-value < 0.01.

Table 7.

Performance of the combined fusion (${ℱ}_{SMVCCA}$) and modality selection (${\mathcal{K}}_{SEL}$) modules of CaMCCo for binary classification (${\mathscr{H}}_{BIN}$).

		ACC	BACC	AUC	SEN	SPEC	PPV
AD vs. HC	ADAS - Cog	0.93	0.92	0.98	0.97	0.88	0.93
	T1w MRI	0.63	0.55	0.65	0.86	0.24	0.66
	PET	0.80	0.78	0.87	0.86	0.71	0.83
	CSF	0.91	0.89	0.97	0.97	0.82	0.90
	PP	0.59	0.47	0.51	0.93	0.00	0.61
	APOE	0.76	0.70	0.86	0.93	0.47	0.75
	${\mathscr{H}}_{BIN}$	0.87	0.85	0.98	0.93	0.76	0.87
MCI vs. HC	ADAS-Cog	0.79	0.75	0.84	0.85	0.65	0.85
	T1w MRI	0.69	0.49	0.53	0.98	0.00	0.70
	PET	0.72	0.67	0.76	0.80	0.53	0.80
	CSF	0.83	0.77	0.92	0.90	0.65	0.86
	PP	0.71	0.52	0.54	0.98	0.06	0.71
	APOE	0.83	0.71	0.84	1.00	0.41	0.80
	${\mathscr{H}}_{BIN}$	0.86	0.82	0.93	0.93	0.71	0.88

${\mathscr{H}}_{BIN}$ shows improvement over individual modality classifiers for MCI vs. HC, particularly in terms of achieving the both high sensitivity and specificity. For AD vs. HC, several individual modalities have sufficiently high classification performance and thereby leaving no room for further improvement with ${\mathscr{H}}_{BIN}$.

Conclusion

In this work, we present a joint cascaded classification and radio-omics data fusion framework, called Cascaded Multiview Canonical Correlation (CaMCCo), for early diagnosis of Alzheimer’s disease. CaMCCo employs a unique strategy as compared to most previous approaches in that it accounts for multiclass classification while attempting to optimize classification accuracy by fusing a select subset of modalities for prediction of each class. As a framework, CaMCCo is comprised of three modules: (i) data fusion, (ii) modality selection and (iii) multiclass classification. Experiments were designed to investigate the choice of methods used for each CaMCCo module independently. In the first experiment, for instance, classification method was held constant while the data fusion and modality selection modules were varied and compared with that of CaMCCo. In the second experiment, the modality selection and data fusion methods were held constant and the cascaded classifier in CaMCCo was compared against other multiclass classification methods. Experimental findings on the ADNI dataset, comprising imaging, proteomics, genomics and neurophysiological data, consistently indicated that fusion of select multi-scale data channels, as in CaMCCo, outperforms fusion of all available modalities. In addition, the results showed that cascaded classification used in CaMCCo is better suited than other multi-class classification methods for MCI prediction. Finally, CaMCCo was compared against individual modalities for the two most commonly investigated binary classification tasks in most related studies, AD vs. HC and MCI vs. HC. While AD vs. HC was a simpler task well resolved by a single modality in our study, MCI vs. HC was a more challenging task where the application of CaMCCo appeared to improve classification, most significantly in terms of specificity. CaMCCo appears to be better able to distinguish between MCI and HC as compared to most previous studies, some of which are listed in Table 5.

However, the work presented in this paper is limited mainly by the method with which the modalities to be combined at each level of the cascade is determined. We only combine the modalities that independently provide the best accuracies on the training set, which may not be complementary. Nonetheless, we found that considering a subset of modalities provides improved performance over fusing all modalities. These findings indicate that incorporation of a more advanced modality selection method and additionally a feature selection method into the framework may provide further improvement in performance. Another limitation of the proposed strategy is the propagation of error from one level of the cascade to the next. To minimize this error, we therefore begin the cascade with the one-vs-all classification providing the least error. Despite these limitations, current findings indicate that the presented framework provides a promising platform for fusion of multiscale, multimodal data for early diagnosis of Alzheimer’s Disease.

Author Contributions

A.S. designed the study, conducted the experiments and prepared the manuscript. H.W. provided expert advice and contributed to methods development. A.M. provided supervision. All authors reviewed the manuscript. Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (www.loni.ucla.edu/ADNI). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at: http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf.

Competing Interests

Conflict of interest disclosures for Anant Madabhushi: Inspirata-Stock Options/Consultant/Scientific Advisory Board Member, Elucid Bioimaging Inc.-Stock Options, Siemens, GE-NIH Academic Industrial Partnership.