Interpretation of Brain Morphology in Association to Alzheimer’s Disease Dementia Classification Using Graph Convolutional Networks on Triangulated Meshes

Abstract

We propose a mesh-based technique to aid in the classification of Alzheimer’s disease dementia (ADD) using mesh representations of the cortex and subcortical structures. Deep learning methods for classification tasks that utilize structural neuroimaging often require extensive learning parameters to optimize. Frequently, these approaches for automated medical diagnosis also lack visual interpretability for areas in the brain involved in making a diagnosis. This work: (a) analyzes brain shape using surface information of the cortex and subcortical structures, (b) proposes a residual learning framework for state-of-the-art graph convolutional networks which offer a significant reduction in learnable parameters, and (c) offers visual interpretability of the network via class-specific gradient information that localizes important regions of interest in our inputs. With our proposed method leveraging the use of cortical and subcortical surface information, we outperform other machine learning methods with a 96.35% testing accuracy for the ADD vs. healthy control problem. We confirm the validity of our model by observing its performance in a 25-trial Monte Carlo cross-validation. The generated visualization maps in our study show correspondences with current knowledge regarding the structural localization of pathological changes in the brain associated to dementia of the Alzheimer’s type.

1. Introduction

Alzheimer’s disease dementia (ADD) is a clinical syndrome characterized by progressive amnestic multidomain cognitive impairment [27]. The causative underlying pathology is Alzheimers disease (AD), defined as the co-occurrence of neurofibrillary tangles and amyloid-beta plaques. Globally, the number of individuals living with AD is expected to reach 1 out of 85 people by the year 2050 [4]. Automated methods for the computer-aided clinical diagnosis of ADD has been an area of interest in the medical imaging community for the development of assistive tools aiding in the visual inspection of structural information captured by magnetic resonance imaging (MRI).

Previous studies in the neuroanatomical pathologies of AD have demonstrated correlations in cortical folding pattern [5] and different neurodegenerative pathologies. Specific patterns of atrophy in the cortex and subcortical structures have been linked to AD [21, 25]. For example, [5] discusses a potential to focus on high variability in association cortices like the intermediate sulcus of Jensen. As [28] also points out, widespread cortical thinning and a greater rate of atrophy is present in temporal lobe regions, primarily the left parahippocampal gyrus, for subjects with AD. Furthermore, Jong et al. [6] discuss irregularities like reduced putamen and thalamus volumes for subjects with AD. In studies such as ADNI, it is common to find bias towards more left-sided atrophy because of the verbal language tests given to assess memory function [8]. For example, if asymmetrical atrophy of the language network is more prominent, subjects may perform worse on verbal tests and be diagnosed with dementia earlier.

Machine learning (ML) methods have been a growing area of interest in the automated clinical diagnosis for ADD. [2, 24, 38] discuss the use of support vector machines (SVMs) in unimodal and multimodal imaging pipelines for the automated classification of ADD using MRI, PET, and cerebrospinal fluid (CSF). In [23, 30], the use of MRI and PET imaging in multimodal convolutional neural networks (CNNs) for ADD diagnosis is discussed. SVM-based approaches, like those used in [2, 24, 38], have historically been hard to interpret, expensive to train, and often serve as the logical choice only when there is enough domain expertise to construct meaningful kernels. Multimodal volumetric CNNs like [30], often require a lot of memory and frequently are limited to smaller-batch operations or using lower resolution 3D volumes.

Motivated by 3D object detection via surfaces [26], cortical and subcortical irregularities correlated with ADD, our work uses mesh manifolds of the cortex and subcortical structures in the diagnosis of ADD. Our technique leverages a reduction in computational complexity offered by [7]. In [29], Parisot et al. leverage this work from [7] to make similar predictions for Alzheimer’s disease and Autism using graph convolutional networks (GCNs) on ADNI/ABIDE subject population graphs. In [31], their convolutional mesh autoencoder (CoMA) framework uses the same GCN basis from [7] on human face surface meshes to generate new meshes from a learned distribution conditioned on facial expression labels. Their network is also able to reconstruct input meshes from compressed 8-dimensional representations with a 50% reduction in reconstruction error, while using 75% fewer parameters than volumetric models that operate on voxels.

The interpretability of results from ML models has remained an open issue in highlighting regions of interest (ROI) in relation to classification decisions. In this paper we demonstrate that it is possible to (1) extract meaningful surface meshes of the cortex and subcortical structures, (2) achieve accurate predictions for the clinical binary classification of ADD using meshes, (3) extract class-discriminative localization maps for interpretable ROI, and (4) reduce the number of learnable parameters.

2. Methods

Data used in the preparation of this article were obtained from the Alzheimers Disease Neuroimaging Initiative (ADNI) database (https://adni.loni.usc.edu). The ADNI was launched in 2003 as a public-private partnership, led by Principal Investigator Michael W. Weiner, MD. The primary goal of ADNI has been to test whether serial magnetic resonance imaging (MRI), positron emission tomography (PET), other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of mild cognitive impairment (MCI) and early Alzheimers disease (AD).

2.1. Localized Spectral Filtering on Graphs

Spectral-based graph convolution methods inherit ideas from a graph signal processing (GSP) perspective as described by [37]. Like [7], our work focuses on using undirected graphs defined by a finite set of vertices, $\mathcal{V}$, with $N=\left|\mathcal{V}\right|$ vertices, and a corresponding set of edges, $\mathcal{E}$, with scalar edge weights, $e_{ij}=e_{ji}\in \mathcal{E}$, which are stored in the i^th rows and j^th columns of the adjacency matrix, $A\in {ℝ}^{N\times N}$. A graph’s node attributes are defined using the node feature matrix $\text{X}\in {ℝ}^{N\times F}$ where each column, $x_i\in {ℝ}^N$, represents the feature vector for a particular shared feature across each of the vertices, $v_i\in \mathcal{V}$.

A great emphasis in GSP is placed on the normalized graph Laplacian, L = I_N − D^−1/2AD^−1/2, where I_N is the identity matrix and D_ii = ∑_j A_ij is the diagonal matrix of node degrees. L can be factored via the eigendecomposition: L = UΛU^T, where $U\in {ℝ}^{N\times N}$ is the complete set of orthonormal eigenvectors for L and $\Lambda =\mathit{\text{diag}}\left(\right[{\lambda }_0,\dots ,{\lambda }_{N-1}\left]\right)\in {ℝ}^{N\times N}$ is the corresponding set of eigenvalues. Given a spectral filter, g_θ, defined in the graph’s Fourier space [34] as a polynomial of the Laplacian, L, and U’s orthonormality, we can filter x via multiplication s.t.

$$g_{\theta }{*}_{\mathcal{G}}x=g_{\theta }(L)x=g_{\theta }\left(U\Lambda U^T\right)x=U{g}_{\theta }(\Lambda )U^{T}x,$$ (1)

where $\theta \in {ℝ}^N$ are the parameters of the filter g_θ and $*\mathcal{G}$ is the spectral convolution operator notation borrowed from [7]. Furthermore, U^Tx is the graph Fourier transform (GFT) of the graph signal x, g_θ(Λ) is a filter defined using the spectrum (eigenvalues) of the normalized Laplacian, L, and the left-sided multiplication with U is the inverse-GFT (IGFT). In this context, convolution is implicitly performed by using the duality property of the Fourier transform s.t. a spectral filter is first multiplied with the GFT of a signal, and then the IGFT of their product is determined.

Our approach uses Chebyshev polynomials of the first kind [1, 7] to approximate g_θ using the graph’s spectrum s.t.

$$g_{\theta }(\tilde{L})=\sum _{k=0}^{K-1}{\theta }_k{T}_k(\tilde{L}),$$ (2)

for the scaled Laplacian $\tilde{L}=\frac{2L}{{\lambda }_{\mathit{\text{max}}}}-I_N$, where λ_max is the largest eigenvalue in Λ, and K can be interpreted as the kernel size. Chebyshev polynomials of the first kind are defined by the recurrence relation, $T_k(\tilde{L})=2\tilde{L}{T}_{k-1}(\tilde{L})-T_{k-2}(\tilde{L})$ where $T_0(\tilde{L})=I$ and $T_1(\tilde{L})=\tilde{L}$ as shown in [7].

2.2. Mesh Extractions of Cortical & Subcortical Structures

Using FreeSurfer v6.0 [10], all MRIs were denoised followed by field inhomogeneity correction, and intensity and spatial normalization. Inner cortical surfaces (interface between gray and white matter) and outer cortical surfaces (CSF/gray matter interface) were extracted and automatically corrected for topological defects. Additionally, seven subcortical structures per hemisphere were segmented (amygdala, nucleus accumbens, caudate, hippocampus, pallidum, putamen, thalamus) and modeled into surface meshes using SPHARM-PDM (https://www.nitrc.org/projects/spharm-pdm).

Surfaces were inflated, parameterized to a sphere, and registered to a corresponding spherical surface template using a rigid-body registration to preserve the cortical [10] and subcortical [3] anatomy. Surface templates were converted to meshes using their triangulation schemes. A scalar edge weight, e_ij, was assigned to connect vertices v_i and v_j using their geodesic distance, ψ_ij, s.t.

$$e_{ij}=e_{ji}=\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{1}{2}{\left(\frac{{\psi }_{ij}}{\sigma }\right)}^2}.$$ (3)

Surface templates were parcellated using a hierarchical bipartite partitioning of their corresponding mesh. Starting with their initial mesh representation of densely triangulated surfaces, spectral clustering was used to define two partitions. These two groups were then each separated yielding four child partitions, and this process was repeated until the average distance across neighbor partitions was below 2.5 mm. For each partition, the central node was defined as the node whose centrality was highest and the distance across two partitions was defined as the geodesic distance (in mm) across the central vertices. Two partitions were neighbors if at least one node in each partition were connected. Finally, partitions were numbered so that partitions 2i and 2i+1 at level L had the same parent partition i at level L − 1. Therefore, for each level a graph was obtained s.t. the vertices of the graph were the central vertices of the partitions and the edges across neighboring vertices were weighted as in Eq. 3. This serves as an improvement upon [7] to ensure that no singleton is ever produced by pooling operations for the cortex and subcortical structures. At the finest level, meshes had a total of 47, 616 vertices: 32, 768 vertices for the cortex and 14, 848 vertices to represent the subcortical structures.

Vertex features were defined as the Cartesian coordinates of the surface vertices in the subjects native space registered to the surface templates. This can create issues if the original scans are not registered to the same template, as was also done by Ranjan et al. in [31]. Similar studies, like that of Gutirrez-Becker and Wachinger [15], implement “rotation network” modules as the first few layers of their neural network (NN) architecture to aid in correcting and aligning their samples to a common template. Performing our template registration as an additional preprocessing step reduces the complexity of our NN architecture and eliminates the need of incorporating an “alignment” term to our cost function to optimize later, as was needed in [15].

Cortical vertices were assigned 6 features: the x, y, and z coordinates of both the white matter (WM) and gray matter (GM) vertices in the native space. This was decided because vertices on these surfaces use the same edge weights and therefore the same “faces” with different coordinates for the vertices of the respective triangles. Similar to the cortex, subcortical vertices had 3 features: their corresponding x, y, and z coordinates in the native space as well. To maintain the same number of features for all vertices per scan, the corresponding cortical and subcortical feature matrices were block-diagonalized into a single node feature matrix per scan s.t. $\text{X}\in {ℝ}^{47,616\times 9}$. Sample meshes extracted from a randomly selected HC and one with ADD are demonstrated in Figure 1.

Fig.1.

Cortical meshes from a randomly selected HC subject (blue) and meshes of the subcortical structures from a randomly selected ADD subject (yellow). Presented are lateral views (a-b) of the HC’s left hemisphere (LH) and right hemisphere (RH) cortical meshes respectively. Medial views of the ADD subject’s LH and RH subcortical structure meshes are also presented (c-d).

2.3. Residual Network Architecture

Inspired by the work of He et al. in [16], we propose an improvement upon ChebNet [7] using residual connections within GCNs, which have been shown in prior work to address the common “vanishing gradient” problem and improve the performance of deep NNs. Typically, these types of residual networks (ResNets) are implemented by using batch normalization (BN) [19] before a ReLU activation function, and followed by convolution as seen in Fig. 2. Using ResBlocks (Fig. 2), max-pooling operations as described by Defferrard et al. [7], and a standard fully connected (FC) layer [32], the total architecture used in our study is defined in Fig. 3. An additional ResBlock, which we refer to as a “post-ResBlock,” was introduced prior to the FC layer as a linear mapping tool to match the number of FC units.

Fig.2.

Single ResBlock in the GCN architecture used in this study. Linear mapping of F_in to F_out channels is implemented using a convolutional layer, $*\mathcal{G}$. This is done to match the number of input features to the number of desired feature maps.

Fig.3.

Residual GCN used for the binary classification of ADD. In this study, max-pooling operations are used to downsample the vertex dimension by a factor of 2.

2.4. Grad-CAM Mesh Adaptation

Interpretability of CNNs was addressed by [33] via their gradient-weighted class activation map (Grad-CAM) approach. In their work, images are fed to CNNs and gradients for each class score (logits prior to softmax) are extracted at the last convolutional layer. Using these gradients, they perform a global average pooling (GAP) operation for each feature map per class to extract “neuron importance weights,” ${\alpha }_c^{(k)}\in {ℝ}^{c\times k}$, whose formulation we readapt for meshes s.t.

$${\alpha }_c^{(k)}=\frac{1}{N}\sum _n\frac{\partial y_c}{\partial A_n^{(k)}},$$ (4)

where y_c corresponds to the class score of class c, and $A_n^{(k)}$ refers to the value at vertex n for the k-th feature map $A^{(k)}\in {ℝ}^N$. A set of neuron importance weights, ${\alpha }_c^{(k)}$, is extracted for each k-th feature map, A^(k), and projected onto them to get the class activation maps (CAMs) s.t.

$$M_c=\text{ReLU}\left(\sum _k{\alpha }_c^{(k)}{A}^{(k)}\right)\in {ℝ}^N.$$ (5)

As a consequence of pooling, CAMs are upsampled to the same number of nodes as the input mesh for a direct “overlay” using a trivial interpolation by going backward along the hierarchical tree used by the pooling operations.

3. Experimental Design

3.1. Dataset & Preprocessing

T1-weighted MRIs from ADNI [20] were selected with ADD/HC diagnosis labels given up to 2 months after the corresponding scan. This was taken as a precaution to ensure that each diagnosis had clinical justification. The dataset in our study consisted of 1,191 different scans for 435 unique subjects. Section 3.2 outlines our stratified data splitting strategy to ensure no data leakage occurs at the subject level across the training, validation, and testing sets [12].

Meshes for each MRI were extracted following the process described in Section 2.2. The spatial standard deviation from Eq. 3, σ, was set to 2 ad-hoc. The visual quality for each mesh was assessed manually via a direct overlay over slices of the corresponding MRI. Laplacians for the cortex and each subcortical structure were block-diagonalized to create one overall L representing a single mesh with multiple connected components. Extracted feature matrices for each sample were min-max normalized per feature to the interval [−1, 1] prior to feeding batches of data into the networks. The added zeros during block-diagonalization (as discussed in Section 2.2) were ignored during each normalization step.

3.2. Network Architecture & Training

Extra care was taken in the shuffling of samples to avoid bias from subject overlap in our cross-validation [12]. A custom dataset splitting function was implemented s.t. the distribution of labels was preserved amongst each set while also ensuring to avoid subject overlap. 20% of the samples were selected at random for the testing set. Of the remaining 80%, 20% of those were withheld as the validation set, while the remaining belonged to the training set. A 25-trial Monte Carlo cross-validation was performed using this data split scheme.

The architecture in Fig. 3 was implemented using 16 kernels per convolutional layer (not including the post-ResBlock), Chebyshev polynomials of order K = 3, and pooling windows of size p = 2. Four alternating ResBlock and pooling layers were cascaded as shown in Fig. 3 prior to the post-ResBlock. The number of units at the post-ResBlock and FC layer was 128. Our GCN was optimized by minimizing a standard binary cross-entropy loss function

$$\mathcal{L}=-\frac{1}{N}\sum _{n=1}^N{y}_n\text{log}\left({\widehat{y}}_n\right)+\left(1-y_n\right)\text{log}\left(1-{\widehat{y}}_n\right),$$ (6)

where ${\widehat{y}}_n$ is the predicted class for the n^th sample of N total samples and y_n is the ground truth label for the same sample index, n.

Networks were trained using batches of 32 samples per step for 100 epochs in each Monte Carlo trial. The Adam [22] optimizer was used with a learning rate of 5 × 10⁻⁴ and a learning rate decay of 0.999. Experiments were implemented in Python 3.6 using Tensorflow 1.13.4 using an NVIDIA GeForce GTX TITAN Z GPU in a Dell Precision Tower 7910 with Linux Mint 19.2.

4. Results & Discussion

4.1. ADD vs. HC Classification

Our cross-validation includes the same multilayer perceptron (MLP) classifier architecture, ridge classifier, and a 100-estimator random forest classifier set up by Parisot et al. in [29], where a similar graph approach is also used on the classification of ADD based on population graphs. The MLP designed was synonymous to the design in [29] s.t. the number of hidden layers and parameters was the same as our GCNs. Demonstrated in Figure 4, our GCN outperformed other standard classifiers not limited to graph methods on our dataset split.

Fig.4.

Monte Carlo cross-validation accuracy results for GCN and baseline model architectures from [29] used on brain meshes.

The results in Table 1 highlight comparable metrics of our model versus other studies that operate on voxels from full 3D MRI volumes, including [30]. In their work, Punjabi et al. train a multi-modal CNN using both volumetric MRI and FDG-PET imaging for the same task, which we outperform while training and evaluating on a smaller subset of their subject population. Furthermore, volumetric models like those in [30] rely on 3D CNNs with far more learned parameters, e.g. [30]’s 200,194,502 weights (×2 for fusion model), in comparison to our GCN’s 497,522 learned parameters needed for comparable results. Like [31], we also achieve comparable results with far less learning parameters by working on meshes and focusing on brain shape instead of working on raw voxels obtained from MRIs and using voxel-based approaches.

Table 1.

Model comparison to classifiers in studies not limited to surface methods.

Study	Data	ADD/HC	Acc. (%)	Sens. (%)	Spec. (%)	AUC (%)
[30]	MRI	−/− (723)	73.76	–	–	–
[30]	MRI+PETamyloid	−/− (723)	92.34	–	–	–
[24]	MRI+PETFDG	51/52	94.37	94.71	94.04	97.24
[36]	MRI+CSF	96/111	91.80	88.50	94.60	95.80
[18]	MRI	228/188	84.13	82.45	85.63	90.00
[2]	MRI	92/94	93.01	89.13	96.80	93.51
[17]	MRI	70/70	97.60	–	–	–
[14]	MRI	200/232	94.74	95.24	94.26	–
Ours	MRI	167/265	96.35	92.37	96.74	96.84

4.2. Class Activation Map Visualization

By employing Grad-CAM on our best GCN, an average CAM was generated for true positive (TP) predictions (Fig. 5). We project our CAM onto the cortical template [11] provided by FreeSurfer [10] and the homemade subcortical structure templates detailed in [3]. The color scale highlights areas from least-to-most influential in TP predictions. The patterns in the CAM match previously described distributions of cortical and subcortical atrophy [9, 21]. One reason we may observe a mismatch between the CAM and expected atrophy in the inferior parietal lobule could be the degree of variability in highly folded association cortex, e.g., the intermediate sulcus of Jensen is found only in some individuals [5, 35]. The slightly more left lateralized pattern in the CAM aligns with previous reports that propose greater pathologic burden and neurodegeneration of the language network which leads to worsening on verbal-based neuropsychological measures of memory resulting in a diagnosis for ADD [8].

Fig.5.

Average TP CAMs on the cortical template from [10, 11] (top) and subcortical structures from [3] (bottom) including: (a-b, e-f) lateral-medial views of the LH respectively, (c-d, g-h) medial-lateral views of the RH respectively.

5. Conclusion & Future Work

In this work, we demonstrated the effectiveness of using cortical and subcortical surface meshes in the context of binary ADD clinical diagnosis and ROI visualization in TP predictions. Furthermore, we compared the cross-validation results of our model for the same ADD vs. HC problem using other ML models on our data. Additionally, our final results were comparable to the results of other studies that use traditional neuroimaging modalities as inputs. When compared to the performance of the multimodal approach used in [30], our model outperforms their approach, thus potentially indicating the reliability of leveraging shape information represented as meshes to perform the same binary classification task.

Natural extensions of this work could be to (1) expand our classification problem to include a third class from ADNI, mild cognitive impairment (MCI), (2) increase the population in our study to include those in ADNI3 [20], (3) work on longitudinal predictions, and (4) compare our model’s performance in using only the cortex, subcortical structures, or both. Additionally, having a 3D-volume-to-mesh dataset offers the potential for developing generative networks, as in [13], for performing the graph extraction preprocessing step described in Section 2.2. This will provide more autonomy and limit the need for the manual quality assessment (QA) of meshes as a part of our pipeline.