MULTIMODAL CLASSIFICATION OF DEMENTIA USING FUNCTIONAL DATA, ANATOMICAL FEATURES AND 3D INVARIANT SHAPE DESCRIPTORS

Abstract

Multimodality classification of Alzheimer’s disease (AD) and its prodromal stage, Mild Cognitive Impairment (MCI), is of interest to the medical community. We improve on prior classification frameworks by incorporating multiple features from MRI and PET data obtained with multiple radioligands, fluorodeoxyglucose (FDG) and Pittsburg compound B (PIB). We also introduce a new MRI feature, invariant shape descriptors based on 3D Zernike moments applied to the hippocampus region. Classification performance is evaluated on data from 17 healthy controls (CTR), 22 MCI, and 17 AD subjects. Zernike significantly outperforms volume, accuracy (Zernike to volume): CTR/AD (90.7% to 71.6%), CTR/MCI (76.2% to 60.0%), MCI/AD (84.3% to 65.5%). Zernike also provides comparable and complementary performance to PET. Optimal accuracy is achieved when Zernike and PET features are combined (accuracy, specificity, sensitivity), CTR/AD (98.8%, 99.5%, 98.1%), CTR/MCI (84.3%, 82.9%, 85.9%) and MCI/AD (93.3%, 93.6%, 93.3%).

1. INTRODUCTION

Neuroimaging tools such as Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET) provide valuable diagnostic information. Recently there has been a lot of interest in PET radioligands fluorodeoxyglucose (FDG-PET) and Pittsburg compound-B (PIB-PET). Moreover, there are reports that MRI and PET biomarkers may provide complementary information that can be used to improve diagnostic accuracy [3]. The question remains as to what are optimal feature combinations of each modality.

Recently, a “kernel combination” method was developed for optimally combining MRI and FDG-PET measures. MRI gray matter volume and FDG-PET average intensity from 93 regions were used as features in a Support Vector Machine (SVM) classifier [4]. This approach yielded good performance for CTR/AD and modest CTR/MCI discrimination. Most importantly, they demonstrated that utilizing multiple biomarkers improves accuracy by as much as 10–20%. While the results are admirable the features derived from MRI and PET can be improved.

First, hippocampal volume alone, when controlled for intracranial volume (i.e. head size) is a significant predictor of MCI and AD, and it is complementary to other clinical markers [5]. Second, PET average intensity is influenced by overall brain uptake of the radioligand, which can be accounted for by normalizing to average intensity in the cerebellum. Selecting AD specific regions further improves accuracy [6]. Third, combining MRI with multiple PET radioligands has yet to be evaluated. Finally, utilizing more advanced features from the MRI (other than volume) has been suggested. Hippocampal shape and morphometric changes in the hippocampal sub-regions appear to be related to progression from MCI to AD [7]. Several studies demonstrated that hippocampus shape analysis from an MRI image could differentiate between MCI subjects that do not progress to AD [8, 9]. However, there is no agreement about the optimal way to analyze hippocampus shape and new approaches could be valuable.

In this paper we evaluate the combined utility of multiple features and improve on the features reported in other MCI/AD classification literature. Specifically, we explore the combination of three modalities MRI, FDG-PET and PIB-PET. Most notably, we introduce the first implementation of 3D Zernike moments to neuroimaging regional shape analysis and use it to derive invariant shape descriptors of the hippocampus. After demonstrating the utility of Zernike over the commonly used hippocampus volume measurement, we explore the classification performance with different combinations of MRI, PET, 3D Zernike features. To our knowledge this is the first report combining shape features with PET data towards multimodality classification in neuroimaging.

2. ZERNIKE MOMENTS

This section briefly describes the theory behind 3D Zernike moments. Its application in the context of shape analysis is developed in [10].

2.1 Moments

Moments are projections of the object function f ∈ L² onto a set of functions φ = {φ_k}, k ∈ ℕ in the domain D ∈ ℝ³. This projection is computed as a dot product in L², where x = (x, y, z)^T and $\overline{{\phi }_k(\mathit{x})}$ represents the complex conjugate, it yields [10]:

$${\mathit{\alpha }}_{\mathit{k}}=\langle \mathit{f},{\mathit{\phi }}_{\mathit{k}}\rangle ={{\displaystyle \int }}_{\mathit{D}}\mathit{f}(\mathit{x})·\overline{{\mathit{\phi }}_{\mathit{k}}(\mathit{x})}\mathit{dx}$$

The desirable characteristics of shape descriptors are invariance, orthonormality and completeness. The 3D Zernike functions, in spherical coordinates, are defined as:

$$Z_{\mathit{nl}}^m(\mathit{x})=R_{\mathit{nl}}(r)Y_l^m(\theta ,\varphi )$$

where $Y_l^m(\theta ,\varphi )$ are the spherical harmonics and R_nl(r) a polynomial in the radial direction, both specified in [10]. They represent different harmonics on the unit sphere at different radii taking the form of polynomials in Cartesian coordinates. Therefore, for a function f(x) defined on the unit sphere, Zernike moments are derived as:

$${\Psi }_{\mathit{nl}}^m=\frac{3}{4\pi }{{\displaystyle \int }}_{\left|\mathit{x}\right|\le \mathbf{1}}f(\mathit{x})\overline{Z_{\mathit{nl}}^m(\mathit{x})}d\mathit{x}$$

where n ∈ [0, N], l ∈ [0, n] such that n − l is even and m ∈ [−l, l]. It is important to note the symmetry relation ${\Psi }_{\mathit{nl}}^{-m}={(-1)}^m\overline{{\Psi }_{\mathit{nl}}^m}$, thus we only need to compute the Zernike moments for m ≥ 0.

2.2 3D Zernike Descriptors

Zernike moments in three dimensions are not invariant under rotations. In order to achieve this property and use them as invariant descriptors we can construct the following vector: ${\Psi }_{\mathit{nl}}=({\Psi }_{\mathit{nl}}^l,{\Psi }_{\mathit{nl}}^{l-1},\dots ,{\Psi }_{\mathit{nl}}^{-l+1},{\Psi }_{\mathit{nl}}^{-l})$ of (2l + 1) dimensionality. Therefore, the rotationally invariant descriptors for three dimensions are [10]:

$${\Theta }_{\mathit{nl}}=\Vert {\mathrm{\Psi }}_{\mathit{nl}}\Vert$$

2.3 Efficient Algorithm for 3D Zernike

We utilize the efficient algorithm proposed in [11] for unstructured surface meshes of triangles. As we have a triangle mesh, it allows us to decompose the integral, involving geometric moments, in different tetrahedra. The latter permits the algorithm to run through each facet sequentially and independently, allowing parallelization. Subsequently, Zernike moments are derived from them.

3. MULTIMODAL CLASSIFIER

The SVM approach uses a vector of features and class labels to train a classifier to predict the class given a novel vector of features. Let x_i be a vector of features (e.g regional volume) for the i^th subject, and its class label y_i ∈ {−1..1} (e.g. −1 implying CTR and 1 implying MCI or AD). SVM solves the following problem:

$$\mathit{min}\frac{1}{2}{\Vert \mathit{w}\Vert }^2+C{\sum }_{i=1}^n{\epsilon }_i$$

subject to y_i(w^T φ(x_i) + b) ≥ 1 − ε_i, ε_i ≥ 0, i = 1, …, n. Where w^T and φ denote the normal vector of the hyperplane, the kernel based mapping function. The equations can be represented in the dual form as:

$$\begin{array}{c}\mathit{max}\\ \alpha \end{array}{\sum }_{i=1}^n{\alpha }_i-\frac{1}{2}{\sum }_{\mathit{ij}}{\alpha }_i{\alpha }_j{y}_i{y}_{j}k({\mathit{x}}_i,{\mathit{x}}_j)$$

subject to Σ_ij α_iy_i = 0, 0 ≤ α_i ≤ C, i = 1, …, n, where C is the maximum penalty for any point in feature space being on the wrong side of the hyperplane that separates the two classes. In our analysis we used a linear kernel, so $k({\mathit{x}}_i,{\mathit{x}}_j)={\mathit{x}}_i^T{\mathit{x}}_j$ is the kernel function for feature vectors from the i^th and j^th samples. Once the dual form is solved the class of a new sample can be derived from its feature vector, x_s, by:

$$\mathit{\text{class}}=\mathit{\text{sign}}({\displaystyle \sum }_{i=1}^n{y}_i{\alpha }_{i}k({\mathit{x}}_i,{\mathit{x}}_s)+b)$$

To combine multiple features we concatenate them into x_i such that x_i = [f_i1, f_i2,.., f_im], with m number of features f from all modalities. Finally, the feature vectors are centered and normalized to their standard deviation.

4. CLASSIFICATION OF SUBJECTS

We assess the performance of 3D Zernike descriptors, MRI volume, FDG-PET, and PIB-PET modalities both individually and in combination; evaluating the performance on CTR/MCI, CTR/AD and MCI/AD patient groups.

4.1. Subjects and data

Detailed description of the patients, MRI and PET acquisition and processing has been previously described [6]. Briefly, patients were recruited in the Memory Disorders Center at New York State Psychiatric Institute/Columbia University, based on consensus diagnosis. A total of 17 CTR, 22 MCI and 17 AD patients completed the previous study. All patients had structural MRI scan, FDG-PET, and PIB-PET. MRI was acquired from either a 1.5T or 3T scanner, 1.5T image size was 256×256, 1.5×0.86×0.86 mm³, 3T was size of 256 × 256, 1×1×1mm³. Regions of interest (ROIs) were manually drawn on the MRI for prefrontal (PFC), cingulate (CIN), parietal cortex (PAR), hippocampus (HIP), parrahippocampal gyrus (PIP), and precuneus (PCN). PET images were acquired on an ECAT EXACT HR+ (Seimens/CTI, Knoxville Tenn.). PIB and FDG data were collected separately in 3D mode for 90- and 60- minutes, respectively. They were reconstructed to a size 128×128×63, 2.5×2.5×5.1mm³. MRI data were de-skulled, segmented and co-registered to motion-corrected PET data. Co-registered PET data were time averaged, 40–60min for FDG and 30–90min for PIB.

4.2. Features extraction

The following features were calculated from MRI and PET data: (1) FDG- and PIB- PET region to cerebellum average intensity ratio for each of the seven ROIs, (2) hippocampus volume normalized to intracranial volume, and (3) hippocampus 3D Zernike descriptors. Zernike moments were calculated for Order 20 as recommended in [10]. This resulted in 121 descriptors for the left and right hippocampus that were averaged, yielding a total of 121 descriptors per subject. Due to our small sample size we could not utilize all 121 descriptors at the risk of over fitting. Instead we searched for the 2 descriptors with the highest discriminatory power and used those. First, a single descriptor with the highest area under the curve was determined from logistic regression within each CTR/AD, CTR/MCI and CTR/AD patient groups. This descriptor was paired with the remaining 120; AUC was determined for each case. The pair (2 descriptors) with the highest AUC was selected for analysis. Forcing HIP Zernike to 2 features also made it more comparable to HIP Volume (left and right), which are 2 features. A similar feature selection procedure was applied to the ROI data. Single regions or pairs with the highest AUC from a logistic regression were used in the analysis. Our small sample size rendered the SVM unstable with more than 5 features, so the number of features in each modality was limited to 1 or 2, maximum of 5 when modalities were combined.

4.3. Classification and bootstrapping

With an SVM classifier, one needs to select a value for the tuning parameter, C. Typically this is done by splitting the data into training and testing samples, where optimal value for C estimated using training data. Our small sample sizes (CTR and AD groups had < 20 subjects) prohibited reliable estimation of C. Instead, we performed a grid search over C ranging 0.1–100, with a step of 0.1. For each value of C, the model was bootstrapped 100 times to obtain robust estimates of accuracy, specificity, sensitivity and their 95% confidence intervals (CI). We report on the model(s) with the “best” bootstrapped results on features individually and in combination.

5. RESULTS

Feature selection with logistic regression yielded pairs {} of Zernike descriptors, Ψ_nl (n,l), FDG-PET and BTA-PET regions respectively: {(0,0), (19,9)}, PAR and PFC for CTR/AD; {(6,0), (16,0)}, CIN and {CIN, PAR} for CTR/MCI; {(10,0), (11,3)}, PAR and PCN for MCI/AD.

In order to assess the utility of 3D Zernike descriptors, we compared its performance against HIP volume in three patient groups (Table 1). Bootstrapped mean performance was almost always higher for Zernike than volume (Zernike vs. volume): CTR/AD accuracy (90.7% to 71.6%), specificity (93.8% vs. 72.8%), sensitivity (88.2 to 74.0%); CTR/MCI accuracy (76.2% vs. 60.0%), specificity (75.7% vs. 68.3%), sensitivity (76.1% vs. 50.0%); MCI/AD accuracy (84.3% vs. 65.5%), specificity (87.3% vs. 87.2%), sensitivity (80.9% vs. 39.4%). The combined performance of volume and Zernike tended to be the same or worse than Zernike alone, so volume was not included in subsequent analysis.

Table 1.

Individual and combined classification performance using hippocampus volume and shape (3D invariant Zernike descriptors) features in different patient groups. Data are bootstrapped average accuracy (ACC), specificity (SPEC), sensitivity (SENS) and 95% confidence interval (2.5%, 97.5%).

		HIP Volume	Zernike	Combined
CTR vs. AD	ACC	71.6 (60.6, 75.8)	90.7 (84.7, 97.0)	90.9 (78.8, 97.0)
	SPEC	72.8 (35.4, 94.1)	93.8 (81.3, 93.8)	91.0 (70.6, 100)
	SENS	74.0 (43.8, 93.8)	88.2 (76.5, 100)	91.1 (75.0, 93.8)
CTR vs. MCI	ACC	60.0 (44.3, 75.1)	76.2 (59.7, 88.0)	74.2 (58.2, 86.2)
	SPEC	68.3 (30.1, 81.7)	75.7 (45.9, 92.6)	78.1 (40.3, 94.2)
	SENS	50.0 (20.4, 90.3)	76.1 (47.8, 92.0)	71.8 (61.9, 96.9)
MCI vs. AD	ACC	65.5 (52.6, 76.3)	84.3 (64.6, 95.2)	81.3 (67.9, 90.1)
	SPEC	87.2 (27.3, 100)	87.3 (58.2, 98.5)	83.7 (59.2, 98.6)
	SENS	39.4 (17.4, 87.5)	80.9 (51.8, 95.7)	78.0 (53.3, 98.0)

Zernike generally outperformed PET in all cases except CTR/AD where PIB-PET had the highest accuracy, specificity and sensitivity (Table 2). CTR/MCI and MCI/AD performance was mediocre, ranging between 50–85% for all modalities. However, the combination of Zernike and PET features yielded the highest performance in all patient groups, CTR/AD, CTR/MCI and MCI/AD accuracy of 98.8%, 84.3% and 93.3%, respectively. This suggests that Zernike is comparable and complementary to PET.

Table 2.

Individual and combined classification performance using HIP shape (3D invariant Zernike descriptors) and PIB-PET and FDG-PET features in different patient groups. Data are bootstrapped average accuracy (ACC), specificity (SPEC), sensitivity (SENS) and 95% confidence interval (2.5%, 97.5%).

		Zernike	PIB-PET	FDG-PET	Combined
CTR vs. AD	ACC	90.7 (84.7, 97.0)	96.7 (90.9, 97.0)	84.2 (72.7, 84.9)	98.8 (97.0, 100)
	SPEC	93.8 (81.3, 93.8)	93.9 (81.3, 93.8)	80.9 (58.8, 94.1)	99.5 (94.1, 100)
	SENS	88.2 (76.5, 100)	99.4 (94.1, 100)	88.9 (62.5, 87.0)	98.1 (93.4, 100)
CTR vs. MCI	ACC	76.2 (59.7, 87.9)	72.3 (58.9, 82.6)	53.8 (43.6, 66.8)	84.3 (70.8, 96.1)
	SPEC	75.7 (45.9, 92.6)	64.4 (27.8, 94.7)	64.3 (31.6, 84.9)	82.9 (54.7, 98.2)
	SENS	76.1 (47.8, 92.0)	80.0 (42.4, 100)	45.7 (22.8, 76.4)	85.9 (59.6, 100)
MCI vs. AD	ACC	84.3 (64.6, 95.2)	80.0 (73.7, 84.2)	81.5 (79.0, 84.2)	93.3 (84.2, 97.7)
	SPEC	87.3 (58.2, 98.5)	83.7 (72.7, 90.9)	86.5 (81.8, 90.9)	93.6 (77.3, 95.5)
	SENS	80.9 (51.8, 95.7)	76.3 (56.3, 93.8)	75.3 (62.5, 87.5)	93.3 (81.3, 100)

6. DISCUSSION AND FUTURE WORK

We introduced a novel application of invariant 3D Zernike descriptors (e.g. 3D moment invariants) as anatomical region shape descriptors in a neuroimaging application. Zernike was applied to the hippocampus of MRI data where it showed significantly improved performance over using hippocampal volume. Zernike was also comparable complementary to PET biomarkers. The most notable finding was that classification performance was greatest when features from multiple modalities were combined.

This proof of concept study was meant to demonstrate the potential benefit of using Zernike. Further work is necessary to address the limitations herein. Zernike was calculated for Order=20 yielding 121 shape descriptors. Due to our small sample size limitation we selected 2 descriptors with the highest performance for bootstrap analysis. The issue of multiple was comparisons were not addressed here. Dimension reduction may be appropriate via principle components analysis, or feature selection done in independent cohorts of patients.

Hippocampus ROIs were not drawn to volumetric standards reported in other studies [12]. So, volume results could yet be improved. It is likely that Zernike would also benefit from having a more detailed labeling so results of this study can still be used as a benchmark for comparing the two features. Additional studies, using a larger cohort of patients with manually labeled volumetric hippocampus should be done. It is interesting that despite these significant limitations, optimal performance of Zernike was quite remarkable, especially for CTR/AD and MCI/AD patient groups. Automatically segmented hippocampus tends to be much less accurate than manual, hence Zernike maybe able to extract relevant information from these rough delineations and integrated into a fully automated diagnostic platform. We are pursuing a more extensive validation of 3D Zernike descriptors using a significantly larger cohort of patients with volumetric hippocampal labeling.