Joint Pairing and Structured Mapping of Convolutional Brain Morphological Multiplexes for Early Dementia Diagnosis

Abstract

Diagnosis of brain dementia, particularly early mild cognitive impairment (eMCI), is critical for early intervention to prevent the onset of Alzheimer's disease, where cognitive decline is severe and irreversible. There is a large body of machine-learning-based research investigating how dementia alters brain connectivity, mainly using structural (derived from diffusion magnetic resonance imaging [MRI]) and functional (derived from resting-state functional MRI) brain connectomic data. However, how early dementia affects cortical brain connections in morphology remains largely unexplored. To fill this gap, we propose a joint morphological brain multiplexes pairing and mapping strategy for eMCI detection, where a brain multiplex not only encodes the relationship in morphology between pairs of brain regions but also a pair of brain morphological networks. Experimental results confirm that the proposed framework outperforms in classification accuracy several state-of-the-art methods. More importantly, we unprecedentedly identified most discriminative brain morphological networks between eMCI and normal control (NC), which included the paired views derived from maximum principal curvature and the sulcal depth for the left hemisphere, and sulcal depth and the average curvature for the right hemisphere. We also identified the most highly correlated morphological brain connections in our cohort, which included the pericalcarine cortex and insula cortex on the maximum principal curvature view, entorhinal cortex and insula cortex on the mean sulcal depth view, and entorhinal cortex and pericalcarine cortex on the mean average curvature view for both hemispheres. These highly correlated morphological connections might serve as biomarkers for eMCI diagnosis.

Introduction

Early diagnosis of brain dementia, specifically mild cognitive impairment (MCI), which might progress toward Alzheimer's disease (AD), might help prevent the onset of AD through early efficient intervention. There is evidence that MCI alters brain morphology, including cortical thinning (Querbes et al., 2009). For instance, previous research (Im et al., 2008) found that sulci in brains of MCI patients were characterized by reduced curvature, with sulcal widening observed and sulcal depth reduced compared with controls. Liu et al. (2012) demonstrated decreased global sulcal index and increased widths of nearly all individual sulci in MCI, whereas Hamelin et al. (2014) found cortical thickness, the hippocampal volume, and the sulcal width to be the best markers for distinguishing MCI from normal control (NC). This is of great clinical value as it might help individualize early intervention to effectively alleviate the symptoms of the disease (Prince et al., 2013).

This early stage of AD was shown to affect functional and structural brain connectivities (obtained from functional magnetic resonance imaging [fMRI] and diffusion-weighted MRI [dMRI]) (Bullmore and Sporns, 2009), thereby causing cognitive decline not fulfilling the AD criteria but greater than expected of their age and educational level (Mucke, 2009). Recently, more research focused on accurate detection of early MCI (eMCI), which is essential for slowing down potential conversion to AD. For instance, Prasad et al. (2015) investigated the predictive power of various combinations of connectomic features, such as pairwise connectivity and maximum flow between two brain regions, extracted from dMRI images for eMCI and NC classification problem. More recently, Wee et al. (2016) computed sparse temporal networks using sliding-window approach over a time series of resting-state functional MRI. Chen et al. (2016) extended this work by additionally considering the high-order correlation between different pairs of brain regions. By combining low-order and high-order brain networks, they further improved the classification accuracy of eMCI/NC patients.

Other studies showed that properties of networks, constructed by volumetric (Yao et al., 2010) and geometric morphological measures (Li et al., 2016), were affected by MCI. However, all these connectomic studies relied on using functional or diffusion-based MRI, which are much more difficult to acquire as they are time consuming, costly, and prone to noise, and are not conventionally used in the diagnostic routine. According to the tension theory of cerebral cortex morphogenesis, network changes in the morphological attributes (surface) of the brain reflect the underlying changes in the structural and functional connectivity (Van Essen, 1997), and can be studied without the need for costly and time-consuming imaging of patients. Hence, we recently introduced the use of morphological brain network for eMCI diagnosis (Lisowska and Rekik, 2017). Specifically, we devised an ensemble classifier architecture leveraging a novel representation of multilayer morphological cortical networks for dementia onset identification.

Previous research on dementia state classification showed that using multilayer networks (i.e., stacking different networks) improved the prediction accuracy for disease identification when compared with using single-view networks (Crofts et al., 2016; Giuliano Zippo and Castiglioni, 2016; La Rocca et al., 2017). However, none of these multilayer network-based methods explored the relationship between two consecutive layers in the network or cortical morphology (Brown and Hamarneh, 2016).

To fill this gap, we proposed a multilayer network (multiplex), consisting of multiple morphological brain network views (Lisowska and Rekik, 2017). We note that a simple concatenation of multiple networks hinders the investigation of potentially complex changes in cortical regions, which might vary jointly or independently across different brain views as they become affected by dementia onset. Hence, we introduce interlayers into a multiplex structure to capture the relationship between different brain views. Since each multiplex is not invariant to the ordering of the intralayers, in our previous research (Lisowska and Rekik, 2017), we generated multiple multiplexes for each subject while considering all possible combinations of intralayers, thereby capturing all relationships between different brain views. However, this resulted in highly correlated data and many redundant features. To address this limitation, in this work we propose a new shallow multiplex structure, each consisting of two morphological views with a single interlayer between them.

Next, to leverage complementary information from different brain multiplexes, we previously used canonical correlation analysis (CCA) to map two sets of multiplex features into a shared space where they become more comparable (Haghighat et al., 2016; Lisowska and Rekik, 2017; Zhu et al., 2016). However, one of the main limitations of CCA is that it lacks biological interpretability, as it does not perform feature selection (Chen et al., 2013). Sparse CCA was shown to solve this issue by computing the relationship between modalities using much less features (Parkhomenko et al., 2009). Since connections in the brain tend to be affected jointly by a disease, we propose to embed the recently developed structured sparse CCA (SS-CCA) (Du et al., 2017) for morphological multiplex fusion into our eMCI/NC classification framework.

Leveraging the strengths of ensemble classifier learning (Džeroski and Ženko, 2004; Quan et al., 2016), we propose structured ensemble classifier learning using multiple sets of shallow brain multiplexes, where each pair of multiplex sets is mapped onto a SS-CCA space then fused. Ultimately, we use the fused multiplex features to train a linear classifier in each spanned SS-CCA space for eMCI identification.

Materials and Methods

Proposed joint pairing and structured mapping strategy

Figure 1 displays the key steps of the proposed framework, where shallow brain multiplexes illustrated in Figure 2 are used to train an ensemble classifier architecture through a data pairing strategy. First, each subject is represented by a set of brain multiplexes. Each colored cube denotes a unique multiplex constructed from two brain networks (e.g., derived from cortical thickness and sulcal depth). Second, we use principal component analysis (PCA) to reduce the dimensionality of all training multiplexes. Third, we map each pair of training multiplexes onto a space where their correlation is maximized. Next, each mapped pair from two multiplex sets will serve as a training sample to learn a support vector machine (SVM) classifier. Last, using a weighted majority voting by all classifiers we identify the label of a new testing subject.

FIG. 1.

Pipeline of the proposed pairing-based structured ensemble classifier learning using fused shallow convolutional brain multiplexes. We use principal component analysis (PCA) to first reduce the dimensionality of each training multiplex set, then apply graph-guided pairwise group lasso-based sparse canonical correlation analysis (GGL-SCCA) (Du et al., 2017) to map a pair of PCA-projected multiplex sets onto a common space where pairs of multiplexes are most correlated. Then, we fuse the selected highly correlated features (given highest canonical weights) from paired shallow multiplexes to train a linear support vector machine (SVM) classifier. The decisions from all the individual classifiers in the ensemble are combined for the final classification decision.

FIG. 2.

Proposed shallow (two-layer) morphological brain multiplex construction for a single subject. For M views (or networks), we generate brain multiplexes by enumerating the number of pairs of views.

Convolutional morphological brain multiplexes

In this section, we introduce the concept of a d-layer (d ≥2) convolutional brain multiplex and present our novel structured mapping strategy using paired sets of brain multiplexes. Matrices are denoted by boldface capital letters, for example, , and scalars are denoted by lowercase letters, for example, x. We denote the transpose operator and the trace operator as and , respectively. For easy reference and enhancing the readability, we have summarized the major mathematical notations in Table 1. We illustrate in Figure 2 the proposed framework for shallow (two-layer) brain multiplex construction and in Figure 1 pairing-based ensemble classifier learning using SS-CCA mapping of sets of brain multiplexes.

Table 1.

Major Mathematical Notations Used In This Paper

Mathematical notations	Definition
	Brain network (single view) in
	Brain multiplex composed of intralayers and convolutional interlayers
	Convolutional intralayer between consecutive brain network views and in
	Subject-specific brain multiplexes with different orderings of intralayers
	Matrix in containing the d multiplex features for all Ns training samples from multiplex
	Paired multiplex feature matrices derived from two training multiplexes in
	Graph modeling the relationship between features extracted from multiplexes and
U	Learned sparse SS-CCA weighting vector for -derived features
V	Learned sparse SS-CCA weighting vector for -derived features

SS-CCA, structured sparse canonical correlation analysis.

Single-view morphological network construction

In line with the work of Mahjoub (et al.) (2018) and Soussia and Rekik (2017), we define morphological brain networks as follows: For each cortical attribute (e.g., cortical thickness), we construct a single-view network for each subject. Such network comprises a set of nodes (anatomical brain regions) and a collection of edges interconnecting the nodes (representing the dissimilarity between the two brain regions in morphology). The average value of a cortical attribute was calculated for each anatomical region of interest (ROI). For each cortical attribute, the strength of each network edge connecting two ROIs is then computed as the absolute difference between their average values, thereby quantifying their dissimilarity (Fig. 3). The same procedure was followed to obtain the connectivity matrices from different cortical attributes (e.g., sulcal depth, curvature). We note that a morphological brain connectivity models dissimilarity in morphology between anatomical brain regions (similarly to functional connectivity, which models correlation between firing neurons), rather than being a real physical connection (like structural connectivity). We believe that both functional and morphological connections mediate “real” connections, as there is a relationship between brain function, morphology, and structure (Van Essen, 1997).

FIG. 3.

Morphological brain network estimation using different cortical attributes. Color images are available online.

Convolutional brain multiplex construction

In a generic way, we define a brain multiplex using a set of M intralayers , each representing a single view of the brain morphology (i.e., cortical attribute), where between two consecutive intralayers and we slide an interlayer , which is defined by convolving two consecutive intralayers. The convolutional interlayer models the relationship between two layers. More specifically, the convolution “blends” two layers together and is the expression of the amount of overlap of one layer as it is shifted over another. Each element in row a and column b within the convolutional interlayer matrix between views and is defined as follows: . The multiplex architecture allows us to explore not only how different brain views get altered by a specific disorder but also how their relationship might get affected. Since the morphological brain connectivity matrices are symmetric (Fig. 3), we extract features from each multiplex by directly concatenating the off-diagonal weights of all connections in each triangular matrix. For each network of size , we extract a feature vector of size . Previously, in Lisowska and Rekik (2017), we introduced the generalized multiplex architecture:

. We note that, for a specific multiplex, we were only allowed to explore similarities between consecutive layers. To explore the inter-relationship between all possible combinations of intralayers, we generated for each subject N multiplexes through simply reordering the intralayer networks, thereby generating ensemble multiplexes (Fig. 2). However, this approach resulted in many highly correlated multiplex features used for the ensemble classifier learning, which may somewhat mislead CCA mapping. To minimize the correlation between different multiplexes, we propose a shallow (i.e., two-layer) convolutional multiplex structure. We define a shallow multiplex using two intralayers and and an interlayer encoding the relationship between and , slid in between them (Fig. 2). We note that each subject-specific brain multiplex in captures unique similarities between two different morphological brain network views (e.g., sulcal depth network and cortical thickness network) that are not present in a different shallow multiplex.

Proposed canonical correlational mapping of brain multiplex sets

Since each multiplex captures a unique and complex relationship between different brain network views, one needs to examine all morphological brain multiplexes in the ensemble . This will provide us with a more holistic understanding of how explicit morphological brain connections can be altered by dementia onset as well as how their implicit high-order (a connection of connections) relationship can be affected. To make use of all the information available from different multiplexes, we seek a feature fusion method that would extract the most relevant features for the classification task, minimize the modality-specific noise, and reduce data dimensionality. CCA was shown to be efficient in analyzing and fusing associations between two sets of variables (Haghighat et al., 2016; Zhu et al., 2016) by identifying the structure common to the two views and creating a subspace that is robust to noise from different modalities. Since CCA aims to find such subspace where correlations between projected features are maximized, the noise present in either modality that is uncorrelated with the other modality is suppressed in the projected subspace (Singanamallia et al., 2014). However, due to the high dimensionality of our multiplex data, which might lead to high-dimensional covariance singularity, CCA cannot be used without applying a prior dimensionality reduction technique to the data (Chen et al., 2013). Therefore, we first apply PCA to our data before CCA-based fusion step. PCA was shown to positively affect the classification performance when using morphological features for MCI diagnosis (Park et al., 2012).

Proposed SS-CCA mapping of pairs of shallow brain multiplexes

Compared with conventional CCA mapping, the SS-CCA mapping brings two main advantages: First, it performs feature selection by imposing a sparsity constraint on the linear coefficients (Parkhomenko et al., 2009), which means the correlation between the modalities is computed using much less features, and without the need to previously apply a dimensionality reduction. Second, it imposes sparsity in a structure-aware manner to capture the high-level structure information (Chen et al., 2013), which is of great importance when extracting features from brain network data. In this article, we particularly use graph-guided pairwise group lasso (GGL)-based sparse canonical correlation analysis model (GGL-SCCA), recently introduced by Du et al. (2017), as it has many advantages compared with methods using the group lasso or the graph/network-guided fused lasso penalty to find the group structure. First, GGL-SCCA can recover the structure information from the input data in an unsupervised manner and without any a priori knowledge. Second, it has a strong upper bound for the grouping effect of correlated variables independently of sample correlation sign. Third, the GGL-SCCA finds stronger, more stable canonical correlations and cleaner canonical loading patterns compared with other state-of-the-art SS-CCA methods (Chen and Liu, 2012; Du et al., 2016). Specifically, we model the relationship between features extracted from two multiplexes and using a graph . We then identify the most correlated features through estimating the weighting vector , which sparsifies the edge set in in through solving the following GGL:

GGL computes the Euclidian distance between features and encourages correlated features to be assigned similar canonical weights. This way GGL-SCCA overcomes the limitations of sparse CCA, which follows an assumption that all the features within the same view are independent of one another. Because GGL uses sample correlation to define the group constraint, unlike previous SS-CCA methods, it does not require prior knowledge about the group structure, which may be unavailable or incomplete for the biomedical data. Ultimately, we formulate GGL-SCCA using pairs of brain multiplexes as follows:

and can be solved using the Lagrange method as in Du et al. (2017):

where the closed form solution is for each iteration t:

where denotes the diagonal matrix with the k₁-th element being and denotes the diagonal matrix with the k₂-th element being , α and λ are tuning parameters. Both p and q values denote the upper bound for parameters k₁ and k₂, respectively.

The grouping effect of GGL-SCCA ensures that correlated features are grouped together and given similar canonical weights, whether they are positively or negatively correlated, which gives the GGL-SCCA an advantage over previous SS-CCA methods. On an example of two features from one view, the grouping effect follows:

where is the solution to the SS-CCA problem. u_i and u_j are features connected in the graph. denotes sample correlation between connected features.

Pairing-based structured ensemble classifier learning

Next, for each pair of multiplexes, we concatenate highly correlated features sparsely selected from each multiplex to train a linear SVM classifier (Fig. 1). Noting that for each training subject we have N estimated multiplexes, we perform mappings of each pair of multiplexes in . Subsequently, a linear SVM classifier is learned for each pair of multiplexes. In the testing stage, for a specific pair of multiplexes, we use the weights and learned for this pair to respectively select features from each testing multiplex, which are then inputted to the pair-specific trained SVM classifier. Finally, we combine the decisions from all individual SVM classifiers in the ensemble through a weighted voting strategy (i.e., by averaging all prediction scores) to predict the label of the testing subject.

Identification of highly correlated morphological connections

Since we need to use PCA for multiplex dimensionality reduction before GGL-SCCA mapping to avoid the covariance matrix singularity, we are not able to directly identify the selected multiplex morphological features. However, this projection step can be avoided when using pairs of views, as they have a much smaller feature vector size than that of multiplexes. Hence, for morphological connectional features' identification, we performed the GGL-SCCA mapping directly on the pairs of views without previous application of any dimensionality reduction technique. To do so, for each GGL-SCCA mapped morphological view, the canonical weights of each feature were averaged across subjects and when paired with other views. Ultimately, top K features with highest average canonical weights were identified. GGL-SCCA is first performed on the training data. The features were given canonical weights. Based on these weights, the most relevant features were identified. For each feature, we take the average of its canonical weights obtained across different subjects to identify features with the highest average weight.

Materials

Evaluation dataset

We used 42 eMCI (average age 70.4 ±7.5) and 42 NC (average age 74.1 ± 6.7) age- and gender-matched subjects from ADNI GO public dataset (http://adni.loni.usc.edu), each with structural T1-w MR image (Mueller et al., 2005). We used FreeSurfer analysis suite (https://surfer.nmr.mgh.harvard.edu) to reconstruct both right and left cortical hemispheres for each subject from T1-w MRI. The processing included skull stripping, motion correction, two T1-w images averaging, intensity normalization, topology correction, and segmentation of the subcortical white matter (WM) and deep gray matter (GM) volumetric structures to identify GM/WM and GM/cerebrospinal fluid boundaries, as in Dale et al. (1999). Then, we parcellated each cortical hemisphere into 35 cortical regions using Desikan-Killiany atlas.

Proposed brain multiplexes

We defined shallow multiplexes, each using two cortical network views. For each cortical attribute, we compute the strength of the morphological network connection linking i^th ROI to the j^th ROI as the absolute difference between the averaged attribute values in both ROIs. Multiplex includes cortical attribute views , includes , includes , includes , includes , and includes . For each cortical region, denotes the maximum principal curvature brain view, denotes the mean cortical thickness brain view, denotes the mean sulcal depth brain view, and denotes the mean of average curvature brain view.

Remark

The morphological networks and multiplexes are constructed separately for the left and the right hemispheres, and they are studied independently, as we do not want our morphological connections to be “biased” by brain hemispheric asymmetry. It also prevents loss of insightful information on how eMCI affects each hemisphere independently.

Results

Experimental setup

We used leave-one-out cross-validation strategy with SVM parameter (strength of penalty imposed on missclassified data) set at default , which aids the training efficiently, while avoiding overfitting of the model to the training data. Since the grouping effect of GGL-SCCA is controlled oppositely by and and it is more sensitive to than to , we fix at a moderate value; , as indicated in Du et al. (2017). Then, we fine-tuned and using fivefold nested cross-validation on the training data. If is too small, the GGL-SCCA would reduce to CCA, and too large leads to overpenalization of the results. Therefore, we set the range for the grid search of between 0.1 and 0.5, which gives the desired number of features and accuracy level for the classification. Exhaustive hyperparameter tuning is not feasible due to high computational power requirements. However, our experiments show that additional tuning of and does not significantly affect the performance. The range for tuning was established experimentally for left and right hemispheres.

For the classification task using paired GGL-SCCA mapping, due to very high dimensionality of multiplex structures, the pairs of multiplexes were first projected onto a lower dimensional space using PCA. For a fair comparison of performance, we also projected the pairs of views onto a lower dimensional space. According to Park et al. (2012), the PCA might aid in improving the classification performance of early demented patients from healthy controls using morphological brain features. Since the performance of classifiers heavily depends on the number of input features, for the SS-CCA we chose a range of input features given the highest canonical weights. The ensemble SVM classification performance was evaluated using the top features selected from each view/multiplex in the pair. For the final SS-CCA classification performance, we report the average classification accuracy for different feature numbers in .

Comparison methods

To the best of our knowledge, no other study used morphological brain networks for eMCI diagnosis. Therefore, we benchmark our proposed framework against other similar approaches using our morphological data. For our eMCI/NC classification task, we benchmarked our pairing-based ensemble classifier strategy against the following: (1) using single SVM trained on the concatenated views, (2) ensemble SVM classifiers (without the pairing or any mapping strategies), (3) ensemble paired SVM classifiers (without CCA mapping), and (4) ensemble paired SVM classifiers with CCA mappings (Lisowska and Rekik, 2017). For each of these methods, we generated four classification results using (1) features from brain views, (2) features from correlational four-layer multiplexes (interlayer computed using Pearson's correlation), (3) features from convolutional four-layer multiplexes (interlayer computed using 2D convolution), and (4) features from shallow (two-layer) convolutional multiplexes. For evaluation, we report in Table 2 the classification accuracy, the area under the receiver operating characteristic (ROC) curve, the sensitivity and specificity of the eMCI/NC classification task. In Figure 4, we specifically show the comparison of classification accuracy for ensemble, pairing-based ensemble (without CCA or GGL-SCCA mapping), pairing-based ensemble with CCA-mapping, and pairing-based ensemble with GGL-SCCA mapping classification based on (1) concatenated views, (2) correlational deep multiplexes, (3) convolutional deep multiplexes (Lisowska and Rekik, 2017), and (4) proposed shallow convolutional multiplexes.

Table 2.

Early Mild Cognitive Impairment/Normal Control Classification Performance Using Our Method and Different Comparison Methods

		Left hemisphere				Right hemisphere
Classifier	Method	Accuracy (%)	AUC	Sensitivity (%)	Specificity (%)	Accuracy (%)	AUC	Sensitivity (%)	Specificity (%)
Ensemble classifiers	Views	53.57	54.31	57.14	50.00	57.14	62.24	52.38	61.90
	Deep correlation	65.48	67.18	66.67	64.29	65.48	70.52	66.67	64.29
	Deep convolution	64.29	73.81	69.05	59.52	72.62	74.21	71.43	73.81
	Shallow convolution	69.05	72.85	73.81	64.29	73.81	76.59	73.81	73.81
Ensemble paired classifiers	Views	61.90	73.58	64.29	59.52	70.24	73.53	71.43	69.05
	Deep correlation	66.67	67.86	69.05	64.29	64.29	70.52	66.67	61.90
	Deep convolution	67.86	72.68	71.43	64.29	67.86	71.20	66.67	69.05
	Shallow convolution	67.86	73.07	71.43	64.29	75.00	75.96	73.81	76.19
Ensemble CCA paired classifiers	Views	63.10	66.89	66.67	59.52	70.24	76.87	69.05	71.43
	Deep correlation	64.29	67.01	69.05	59.52	57.14	63.21	64.29	50.00
	Deep convolution	65.48	64.40	66.67	64.29	72.62	78.40	76.19	69.05
	Shallow convolution	63.10	67.06	64.29	61.90	77.38	79.20	78.57	76.19
Ensemble GGL-SCCA paired classifier	Views	64.15	72.06	61.11	67.20	68.92	73.50	69.58	68.25
	Deep correlation	63.62	67.11	61.11	66.14	65.21	70.67	68.25	62.17
	Deep convolution	67.59	74.29	65.08	70.11	68.25	70.93	69.58	66.93
	Shallow convolution	72.88	76.06	66.93	78.84	70.90	76.52	70.11	71.69

Bold indicates best classification performance.

AUC, area under the receiver operating characteristic curve; CCA, canonical correlation analysis; GGL, graph-guided pairwise group lasso; GGL-SCCA, GGL-based sparse canonical correlation analysis.

FIG. 4.

Classification accuracies for our proposed joint pairing and structured GGL-SCCA mapping of brain features, and comparison with other ensemble classifier methods. Views: morphological brain views. Correlation: correlational brain multiplexes. Convolution: convolutional brain multiplexes. Two-layer: shallow convolutional multiplex. Ensemble classifiers: one SVM trained for each view (or multiplex) without any pairing strategy or mapping. Ensemble paired classifiers: pairing different views (or multiplexes) without any mapping. Ensemble CCA-paired classifiers: pairing different views (or multiplexes) with CCA mapping, Ensemble GGL-SCCA paired classifiers: pairing different views (or multiplexes) with GGL-SCCA mapping (average accuracy across different number of features from 10 to 50, with an incremental step of 10 features). For brevity, we shortened GGL-SCCA to SCCA. Color images are available online.

The best accuracy, area under the ROC curve, sensitivity, and specificity were always obtained when using the proposed shallow convolutional multiplexes (Table 2). Our proposed ensemble GGL-SCCA paired classifier framework outperformed all comparison methods (72.9%) when using shallow convolutional multiplexes of the left hemisphere (Fig. 4 and Table 2). In contrast, the best classification result (77.4%) for the right hemisphere was obtained using ensemble CCA paired classifier, introduced in Lisowska and Rekik (2017). This might indicate that the group effect introduced by GGL-SCCA may better capture a morphological connectional structure in the left hemisphere that is not prevalent in the right hemisphere. It might also imply different levels of complexity in morphological cortical disease progression across both hemispheres.

Identification of highly correlated morphological connections

For each morphological view, we identified the top 10 features with the highest average GGL-SCCA canonical weights across subjects, and when paired with other views for both left and right hemispheres. Figure 5 displays circular graphs with top 10 features identified by GGL-SCCA for each cortical attribute and each hemisphere. We display in Figure 6 the index of each anatomical cortical brain region referred to in the circular graph together with their corresponding names.

FIG. 5.

The top 10 connections given the highest canonical weights by the GGL-SCCA visualized on the circular graphs for the left and the right hemisphere. (A) The maximum principal curvature brain view. (B) The mean cortical thickness brain view. (C) The mean sulcal depth brain view. (D) The mean of average curvature. Color images are available online.

FIG. 6.

Cortical regions of interest index. Each cortical hemisphere is parcellated using Desikan Cortical atlas. We display the cortical region names and their respective identification numbers. Color images are available online.

For the three cortical attributes (the maximum principal curvature, the mean sulcal depth, and the mean of average curvature), one very strongly correlated connection was present (Fig. 5A, C, D), whereas for mean cortical thickness more highly correlated connections were identified (Fig. 5B). The connection given the highest canonical weight in the maximum principal curvature brain view was the connection between the transverse temporal cortex and the pericalcarine cortex, which seems to act as a “hub” region for many highly correlated connections. This is consistent for both cortical hemispheres. In addition, the transverse temporal cortex seems to serve as a hub region in the right hemisphere (Fig. 5A). For the mean sulcal depth brain view (Fig. 5C), the strongest connection was between the entorhinal cortex and the insula cortex, which is identified as a hub region with many connections extending from it in the left and right hemispheres. In contrast, for the mean of average curvature brain view (Fig. 5D), we note a distinctive strong connection between the entorhinal cortex and the pericalcarine cortex, which acts as a hub for other morphological connections in the left hemisphere, as well as the right hemisphere. For the mean cortical thickness brain view (Fig. 5B), many highly correlated connections were spread out across the frontal pole, the temporal pole, the insula cortex, the bank of superior temporal sulcus, the corpus callosum, and the entorhinal cortex.

Having identified the underlying structure of important connections between brain regions, we wanted to identify which connections are altered by eMCI and could serve as early disease marker. To do so, we plotted a graph of classification accuracies obtained from each GGL-SCCA mapped pair of views in the ensemble (Fig. 7) to identify the most discriminative features between NC and eMCI patients. These are obtained from pairs of views that gave the highest classification accuracy.

FIG. 7.

Classification accuracy obtained from different pairs of morphological views in the ensemble paired GGL-SCCA mapping setting. (1) The maximum principal curvature brain view. (2) The mean cortical thickness brain view. (3) The mean sulcal depth brain view. (4) The mean of average curvature.

Influence of PCA on classification performance

We also compared the performance of ensemble paired views with GGL-SCCA mapping with and without prior PCA (Fig. 7). The PCA-projected morphological views gave higher classification accuracy for the left and the right hemispheres as compared with using raw morphological views across all the different number of features used for classification using the ensemble GGL-SCCA classifier.

Ensemble classifier learning strategy for eMCI diagnosis

The ensemble GGL-SCCA paired classifier performed better than using any single combination of multiplexes when using the shallow convolutional brain multiplexes as input features (Fig. 8), indicating that ensemble classifier is an effective strategy in combining information from different morphological brain properties to improve early dementia diagnosis.

FIG. 8.

Classification accuracy using different pairings of our six shallow (two-layer) convolutional multiplexes in the ensemble paired GGL-SCCA mapping setting. Numbers displayed along the horizontal axis denote the multiplex index (e.g., 1 for ). The multiplex pair () of the left hemisphere produced the best classification accuracy among individual networks, whereas multiplex () of the right hemisphere produced the best classification accuracy. For both hemispheres, ensemble multiplexes outperformed individual multiplexes.

Discussion

We proposed a joint pairing and structured mapping strategy of morphological brain multiplexes for early dementia diagnosis. Specifically, we presented an individual-specific representation of brain connectivity based on shallow convolutional brain multiplexes, each capturing a unique relationship between two different cortical networks. To combine the information from these various multiplexes, we introduced a pairing-based ensemble classifier, where we fused features sparsely selected from a pair of multiplexes using GGL-SCCA mapping, trained a linear SVM classifier for each of the mapped features, and combined the decisions from all the classifiers in the ensemble by weighted voting to distinguish eMCI patients from NC subjects.

Our proposed ensemble GGL-SCCA paired classifier outperformed other methods when using shallow convolutional multiplexes based on morphological brain features from the left hemisphere, while the best classification result for the right hemisphere was obtained using CCA-mapped shallow convolutional multiplexes (Fig. 4 and Table 2). These results may suggest that early dementia exhibits different morphological progression patterns across the two cortical hemispheres, possibly affecting the main communication centers (hubs) predominantly in the left hemisphere, while affecting the right hemisphere in a less structured manner. It also shows that for the right hemisphere, the correlation between different morphological attributes was most discriminative between the eMCI and NC patient, while the structure information introduced by SS-CCA may be more prevalent in the left hemisphere.

PCA dimensionality reduction

We chose to use PCA to project our data onto a lower dimensional space, since it was shown to improve the discriminative power of brain morphological features in classifying MCI and NC subjects (Park et al., 2012). In our work, we compared the classification accuracy of the ensemble GGL-SCCA paired classifier using morphological brain views with and without previous application of PCA, and confirmed that reducing the dimensionality of morphological data using PCA improves the eMCI/NC classification performance (Fig. 7). However, PCA as an unsupervised method does not take into account the data class labels, and it does not ensure good separability of the data from the two different classes in the new space. Furthermore, projecting the morphological features onto a lower dimensional space hinders their biological interpretability, making it impossible to identify important multiplex biomarkers for early dementia diagnosis.

Identification of highly correlated morphological features

Despite these drawbacks and thanks to the sparsity constraint of the GGL-SCCA, we were able to apply GGL-SCCA to pairs of morphological brain networks and identify features that are highly correlated between different cortical attributes. We found that the most discriminative features for the eMCI diagnosis belonged to maximum principal curvature and average curvature views (Fig. 7). These produced the highest classification accuracies when paired with sulcal depth brain view, thereby indicating correlated brain changes in sulcal depth and curvature in early dementia. Further, GGL-SCCA allowed us to identify underlying structure of the brain networks for different cortical attributes. Specifically, hub regions, which were characterized by highly correlated morphological connections to many other cortical regions, were identified in the entorhinal cortex and the insula cortex (Fig. 5). These regions and their morphological relationships may serve as important biomarkers for eMCI diagnosis. The insula cortex is involved in emotion processing and regulation, introspective awareness and integration of multimodal inputs, and the reduction in the insular volume was associated with the occurrence of neuropsychiatric symptoms, such as agitation and apathy in patients with AD (Moon et al., 2014). The entorhinal cortex plays an important role in navigation and formation, and consolidation of spatial and declarative memory. Various research works, in line with our findings, have shown that the pathological changes in the neuronal structure and function of the entorhinal cortex occur before the onset of any AD symptoms (Zhou et al., 2016), making it a ROI for early dementia diagnosis.

To obtain further insights into which of the identified features are most discriminative between the NC and eMCI patients, we plotted a graph of classification accuracies obtained from each GGL-SCCA mapped pair of views in the ensemble (Fig. 9). The highest accuracy of NC/eMCI classification (69.97%) using cortical attributes from the left hemisphere was obtained by a classifier trained on a pair of views consisting of the maximum principal curvature and the mean sulcal depth. This might indicate that the connection between the transverse temporal cortex and the pericalcarine cortex and the connection between the entorhinal cortex and the insula cortex may be a likely target of morphological changes in the early stages of dementia. Interestingly, the pairs that contained the maximum principal curvature as one of the views consistently performed best in case of the left hemisphere, which shows that this cortical attribute fingerprints “morphological” dementia progression. For the right hemisphere, the best classification performance was also achieved when using the sulcal depth and the mean of average curvature, giving 64.95% accuracy (Fig. 9). The most notable connections are between the entorhinal cortex and the pericalcarine cortex and the entorhinal cortex and the insula cortex. Together, these results suggest that the curvature of the sulci is also altered by the early dementia and could provide insight into the progression of the disease. Further, the most likely candidates of hub regions affected by the eMCI are the entorhinal cortex and the pericalcarine cortex.

FIG. 9.

Classification accuracies using the pairing-based ensemble classifier learning using GGL-SCCA-mapped brain view features with and without dimensionality reduction. Left views: morphological brain views from the left hemisphere. Right views: morphological brain views from the right hemisphere. Left views + PCA: PCA-projected brain views from the left hemisphere. Right views + PCA: PCA-projected brain views from the right hemisphere. Color images available online at www.liebertpub.com/brain

Our findings are consistent with those of previous research on MCI patients, which reported brain atrophy in regions such as the entorhinal cortex, hippocampus, medial temporal lobe, insula cortex, and temporal lobe (Fan et al., 2008). In addition, Li et al. (2016) showed that networks constructed by sulcal depth exhibit disrupted connectivity in amnestic MCI patients, implying loss of efficiency in communication between different brain regions. Recently, Cai et al. (2017), which used morphological properties for MCI diagnosis, showed that the most discriminative features were found in the hippocampus, amygdala, entorhinal cortex, and precuneus. However, none of the previous research explored the relationship between morphological networks constructed from different cortical attributes as shown in this study.

Shallow versus deep multiplex structure

Our research showed that introducing the relationship between different cortical networks (convolutional interlayers exploring the relationship between brain views), which we modeled using a multiplex structure, improved the classification accuracy of eMCI/NC in comparison with using different brain networks alone (Fig. 8). The new two-layer multiplex structure reduced the correlation between individual classifiers in the ensemble and resulted in a better ensemble classifier performance compared with the ensemble classifier using previous convolutional multiplex structure (Fig. 4 and Table 2). Furthermore, we show that the ensemble classifier combining prediction scores from linear SVM classifiers using shallow multiplexes better distinguishes eMCI patients from NC subjects than any of the individual classifiers used independently for both left and right hemispheres (Fig. 8).

Limitations and future directions

With increasing availability of data and computational power, more advanced machine-learning-based classifiers, for example, structured random forests and neural networks, can be used as base classifiers to define our ensemble to improve early dementia diagnosis. However, one advantage of using linear classifiers as individual classifiers in the ensemble is that they generally give stable results—that is, the output from the ensemble does not significantly change in response to small changes in the training data (Dietterich et al., 2000).

Although we showed that the relationship between different cortical brain networks brings novel insights into early dementia and cortex morphology from multiple views, we were unable to pinpoint the specific relationship between brain connections from different morphological brain networks that are most discriminative between the eMCI and NC classes due to the necessity of applying previous dimensionality reduction method before the GGL-SCCA mapping. In fact, our method cannot directly identify the informative connectional features; however, it achieves the best classification accuracy. For this reason, we used the paired views to identify the most discriminative connectional features. We also note that the devised architecture of pairing morphological brain views in our ensemble learning framework is novel. Ideally, our multiplex-based classification method would be able to achieve both tasks (high classification accuracy and identifying discriminative features). However, this was not possible as a dimensionality reduction had to be applied to the data prior to SCCA mapping and training of the classifier, which hinders the tracking of the original features. This is a typical drawback of classification methods that make use of projections such as conventional CCA and linear discriminant analysis. Future work should seek a mapping method that would be less sensitive to the high dimensionality of the data or a dimensionality reduction technique (i.e., feature selection) that would not adversely hinder the interpretability of the projected multiplex features.

In addition to investigating how dementia alters the relationship between connections from different cortical networks, we expect that by jointly exploring changes in the brain morphological, structural, and functional networks, we could provide a more holistic connectomic understanding of early dementia onset and further improve the classification performance of our model for a more reliable early dementia diagnosis. If this study is further extended and improved to match the performance of eMCI diagnosis based on structural/functional connectivity, the time and cost of eMCI diagnosis would be significantly reduced, and more patients could benefit from early dementia diagnosis.

Conclusion

In this work, we proposed a joint pairing and mapping strategy for early dementia diagnosis using morphological convolutional brain multiplexes. This allowed us to investigate multiview changes in brain cortical morphology on a connectional level. Our pairing-based ensemble classifier learning strategy produced the best classification performance in comparison with state-of-the-art methods. More importantly, we identified the most discriminative pairs of morphological brain views distinguishing eMCI from NC subjects as well as the most highly correlated morphological brain connections in our cohort. In our future work, we will focus on further improving the data fusion method (i.e., by incorporating supervised multiview mapping or applying more complex learning models) and integrating multimodal connectomic data (e.g., functional and structural networks) to complement our morphological connectivity data for more holistic investigation of early dementia.

Acknowledgments

Data collection and sharing for this project were funded by the Alzheimer's Disease Neuroimaging Initiative (ADNI; National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions from the following: AbbVie, Alzheimer's Association; Alzheimer's Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate; Eisai, Inc.; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; EuroImmun; F. Hoffmann-La Roche Ltd. and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Lumosity; Lundbeck; Merck & Co., Inc.; Meso Scale Diagnostics, LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer, Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health (www.fnih.org). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer's Therapeutic Research Institute at the University of Southern California. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California.

Author Disclosure Statement

No competing financial interests exist.