Multimodal analysis of functional and structural disconnection in Alzheimer's disease using multiple kernel SVM

Abstract

Alzheimer's disease (AD) patients exhibit alterations in the functional connectivity between spatially segregated brain regions which may be related to both local gray matter (GM) atrophy as well as a decline in the fiber integrity of the underlying white matter tracts. Machine learning algorithms are able to automatically detect the patterns of the disease in image data, and therefore, constitute a suitable basis for automated image diagnostic systems. The question of which magnetic resonance imaging (MRI) modalities are most useful in a clinical context is as yet unresolved. We examined multimodal MRI data acquired from 28 subjects with clinically probable AD and 25 healthy controls. Specifically, we used fiber tract integrity as measured by diffusion tensor imaging (DTI), GM volume derived from structural MRI, and the graph‐theoretical measures ‘local clustering coefficient’ and ‘shortest path length’ derived from resting‐state functional MRI (rs‐fMRI) to evaluate the utility of the three imaging methods in automated multimodal image diagnostics, to assess their individual performance, and the level of concordance between them. We ran the support vector machine (SVM) algorithm and validated the results using leave‐one‐out cross‐validation. For the single imaging modalities, we obtained an area under the curve (AUC) of 80% for rs‐fMRI, 87% for DTI, and 86% for GM volume. When it came to the multimodal SVM, we obtained an AUC of 82% using all three modalities, and 89% using only DTI measures and GM volume. Combined multimodal imaging data did not significantly improve classification accuracy compared to the best single measures alone. Hum Brain Mapp 36:2118–2131, 2015. © 2015 Wiley Periodicals, Inc.

INTRODUCTION

Alzheimer's disease (AD) is characterized by regionally specific neuronal degeneration which can be estimated in vivo by use of structural MRI scans [Braak and Braak, 1991; Whitwell et al., 2008a]. In addition, postmortem histologic studies have reported a significant decline in intracortical projecting fiber tract integrity in AD [Brun and Englund, 1986]. Using diffusion tensor imaging (DTI) as a surrogate in vivo marker for fiber tract integrity [Le Bihan et al., 1992], imaging studies have revealed alterations in DTI‐derived measures in AD dementia patients or even prodromal AD stages compared to healthy controls (HC) [Acosta‐Cabronero et al., 2012; Agosta et al., 2011; Bozzali, 2002; Canu et al., 2010; Cui et al., 2012; Dyrba et al., 2013; Haller et al., 2010; Medina et al., 2006; Mesrob et al., 2012; O'Dwyer et al., 2012; Scola et al., 2010; Teipel et al., 2010b, 2007]. For resting‐state functional MRI (rs‐fMRI) measures of functional connectivity, which is defined as the correlation between the signal time courses of spatially segregated brain regions [Cordes et al., 2001; Haughton and Biswal, 1998], have been shown to be different in AD patients than in healthy subjects [Brier et al., 2012; Chen et al., 2011; Koch et al., 2012; Petrella et al., 2011; Sanz‐Arigita et al., 2010; Soldner et al., 2012; Sorg et al., 2007; Supekar et al., 2008]. Both DTI and rs‐fMRI have been recommended by the National Institute on Aging—Alzheimer's Association as experimental biomarkers for the diagnosis of mild cognitive impairment (MCI) due to AD [Oishi et al., 2011]. Surprisingly, only very few studies have assessed DTI, functional MRI and anatomical MRI together to assess the level of concordance between them [Filippi and Agosta, 2011; Honey et al., 2009; Soldner et al., 2012; Teipel et al., 2010a; Wee et al., 2012; Zhu et al., 2013].

Multivariate machine learning approaches are particularly sensitive to distributed disease‐specific changes. One such approach is the support vector machine (SVM) classifier [Cortes and Vapnik, 1995], a method which has been used successfully in several AD and MCI imaging studies involving gray matter (GM) volume [Abdulkadir et al., 2011; Cuingnet et al., 2011; Klöppel et al., 2008; Plant et al., 2010], DTI measures [Cui et al., 2012; Dyrba et al., 2013; Graña et al., 2011; Haller et al., 2010; O'Dwyer et al., 2012; Wee et al., 2012], and rs‐fMRI [Abdulkadir et al., 2011; Cuingnet et al., 2011; He et al., 2012; Li et al., 2013; Wee et al., 2012]. The multiple kernel SVM (MK‐SVM) classifier [Dyrba et al., 2012; Sonnenburg et al., 2006; Wee et al., 2012] is an extension of the conventional SVM algorithm and permits the combined analysis of different imaging modalities. In contrast to simpler strategies, such as concatenating features (images/regions) or stacking classifiers (e.g., using voting), the MK‐SVM enables the contribution of each modality to the classification result to be controlled more closely and potentially improves the power of the SVM algorithm to use complementary information provided by the modalities within its model [Dyrba et al., 2012]. So far, studies using the MK‐SVM have focused on MRI and fluorodeoxyglucose positron emission tomography (FDG‐PET) [Hinrichs et al., 2011; Young et al., 2013; Zhang et al., 2011], or on DTI and rs‐fMRI [Wee et al., 2012]. These studies found the multimodal MK‐SVM to be more accurate than a single modality SVM. In contrast, we observed no increase in accuracy for the multimodal data in comparison to the best single modality using DTI and MRI [Dyrba et al., 2012]. Together, the literature suggests that using multimodal imaging data and MK‐SVM is most promising when functional or metabolic data (e.g., rs‐fMRI, FDG‐PET) are included in the analysis as well as structural data (DTI, MRI) [Dyrba et al., 2012].

We wanted to assess the usefulness of machine learning approaches in automated diagnosis based on different imaging modalities. The primary aim of this study, then, was to compare the overall classification performance of the SVM machine learning algorithm for each single imaging modality with the performance of the combined multimodal imaging data using the MK‐SVM approach. Second, the study was also carried out to assess the information provided by the different imaging modalities with respect to the contribution to the group separation and the concordance between them. Therefore, indices of similarity/concordance between pairs of modality‐specific classification results were determined. Finally, the information which regions contribute most to the group separation was assessed allowing the interpretation of the different types of discriminative features with respect to the underlying neurobiology of AD.

MATERIALS AND METHODS

Subjects

The subjects for this study were selected from the German Center for Neurodegenerative Diseases (DZNE) Rostock database and included 28 AD patients and 25 HC matched for age, gender, and education. The study was approved by the ethics committee of the University Medical Center Rostock. Written informed consent was provided by all participants or their representatives. All members of the patients group were diagnosed as having clinically probable AD according to NINCDS‐ADRCA criteria [McKhann et al., 1984] and had a positive test for AD cerebrospinal fluid (CSF) neurodegeneration markers according to the recommendations of the National Institute on Aging‐Alzheimer's Association (NIA‐AA) workgroups on diagnostic guidelines for AD [McKhann et al., 2011]. More precisely, they had a decreased amyloid‐β 1‐42 level and/or a decreased amyloid‐β 1‐42/amyloid‐β 1‐40 ratio. Twenty‐five of the 28 AD patients (89%) additionally had an increased phosphorylated tau level and of those, 15 patients were also pathologic for total tau. The temporal offset between MRI and CSF examinations was 25 (SD 12) days on average, not including the two AD patients whose CSF examination took place 13 months prior to the MRI scan. All participants were free of any significant neurological, psychiatric, or other medical condition (except AD in the patients group), in particular cerebrovascular apoplexy, vascular dementia, depression, subclinical hypothyroidism, or substance abuse. Several participants were taking medication with known neurological effects: three AD patients were taking antidepressants/hypnotics (when needed), one control subject was taking pregabalin for lower limb paresthesia, half of the subjects were taking beta blockers (across both groups), four AD patients were taking antidepressants, nine AD patients were taking antidementives (cholinesterase inhibitors), and five AD patients were taking ginkgo. HC were required to have no cognitive complaints and scored within one standard deviation of the age and education‐adjusted norm in all subtests of the Consortium to Establish a Registry of Alzheimer's Disease (CERAD) cognitive battery [Morris et al., 1989]. Demographic details and participants' scores on the Mini Mental State Examination (MMSE) are summarized in Table 1.

Table 1.

Group characteristics and subject demographics

	AD	HC
No. of subjects (female/male)	14/14	13/12
Age (SD) in years	72 (7)	73 (6)
Years of education (SD)	13 (3)	13 (2)
MMSE (SD)	24 (3)a	28 (1)

Abbreviations: AD—Alzheimer's disease, HC—healthy controls, SD—standard deviation

^aStatistical difference using Mann–Whitney‐U = 75, P < 0.001

Image Data Acquisition

MRI scanning was performed with a 3T Siemens Magnetom VERIO scanner (Erlangen, Germany) using a 32‐channel head coil. Initially, the field‐of‐view (FOV) was orientated to be in plane with the anterior–posterior commissure line covering the whole brain. Participants were instructed to rest with their eyes closed but not to fall asleep during scanning. Functional MRI was based on echo‐planar imaging using a 96 × 96 image matrix with 45 axial slices (spacing 2.6 mm, thickness 2 mm, gap 30%) and interleaved acquisition. The FOV was 192 × 192 × 117 mm, voxel size 2.0 × 2.0 × 2.6 mm³, echo time 30 ms, repetition time 3,000 ms, flip angle 90°, and parallel imaging acceleration factor 2. The sequence took 6 min 11 s. DTI was based on echo‐planar imaging using a 128 × 128 image matrix with 64 axial slices (spacing 2.4 mm, thickness 2 mm, gap 20%). The FOV was 250 × 250 × 154 mm, voxel size 1.0 × 1.0 × 2.4 mm³ (interpolated), echo time 93 ms, repetition time 8,200 ms, flip angle 90°, and parallel imaging acceleration factor 3. Each scan was run with three repetitions, each of which consisted of one nondiffusion‐weighted b ₀ image followed by 20 gradients with a b‐value of 1,000 s/mm². The sequence took 9 min 19 s. During the same scanning session, a high‐resolution T1‐weighted anatomical image was also acquired for each participant using the magnetization‐prepared rapid gradient echo sequence with the following parameters: 256 × 256 image matrix with 176 sagittal slices, FOV 250 × 250 × 176 mm, voxel size 1 × 1 × 1mm³, echo time 2.52 ms, repetition time 1,900 ms, flip angle 9°, and parallel imaging acceleration factor 2. The duration of the sequence was 4 min 18 s.

Image Processing

DTI data were preprocessed using the diffusion toolbox in FSL (version 5.0.4, FMRIB, Oxford, UK) [Smith et al., 2004]. Corrections were made for eddy currents and head motion. The skull was stripped using Brain Extraction Tool and the diffusion tensors were fitted to the data using DTIfit. Fractional anisotropy (FA), mean diffusivity (MD), and mode of anisotropy (MO) maps were then computed using the standard FSL protocol. Functional MRI data processing was carried out using Data Processing Assistant for Resting‐State fMRI Advanced (DPARSFA, version 2.2, State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, China) [Chao‐Gan and Yu‐Feng, 2010]. After the removal of the first six images, the rs‐fMRI data was slice time corrected and realigned to the time series mean image. Nuisance covariates to regress out included head movement and the mean time courses for the global brain signal, the white matter segment signal, and the CSF segment signal. The images were band‐pass filtered using five frequency sub‐bands: between 0.025 and 0.03929 Hz, between 0.03929 and 0.05357 Hz, between 0.05357 and 0.06786 Hz, between 0.06786 and 0.08214 Hz, and between 0.08214 and 0.1 [Wee et al., 2012]. The anatomical T₁‐weighted image for each participant was coregistered to the mean functional image and subsequently segmented into GM, white matter, and CSF partitions using the New Segment toolbox in Statistical Parametric Mapping (SPM8, release 5236, Wellcome Department of Imaging Neuroscience, London, UK) [Friston et al., 2007]. In SPM8, the Diffeomorphic Anatomical Registration Through Exponentiated Lie (DARTEL) algebra algorithm [Ashburner, 2007] was used to create a customized template and to normalize the T₁‐weighted images to the Montreal Neurological Institute (MNI) reference coordinate system. The quality of normalized T₁‐weighted images was inspected before moving to the next steps. The preprocessed diffusion scans were aligned to the T₁‐weighted images using the brain‐extracted nondiffusion weighted B₀ images as the source and the corresponding T₁‐weighted scans as the target for the coregistration algorithm, with the estimated transformations applied also to the FA, MD, and MO maps. Finally, we used the inverse deformation fields generated by DARTEL to project the brain atlases from the MNI reference space into each subjects' native image space, in which all subsequent analyses were performed.

Feature Extraction

This study followed a region‐of‐interest (ROI)‐based approach. For the T₁‐weighted anatomical scans, the most commonly used AAL atlas [Tzourio‐Mazoyer et al., 2002] was selected, containing 115 cortical and subcortical anatomical structures. For the DTI scans, the Johns Hopkins University–International Consortium for Brain Mapping (JHU‐ICBM) DTI atlas was selected, containing 48 WM fiber tract labels [Mori et al., 2008]. For the rs‐fMRI data, Craddock's functional atlas [Craddock et al., 2012] was selected, containing 200 clusters obtained from 41 healthy subjects (age: 18–55; mean 31.2) using independent component analysis [Craddock et al., 2012]. A threshold of 0.5 was applied to the GM segment of our brain template and this mask then applied to the functional atlas to restrict the subsequent fMRI analysis to voxels within the GM areas only.

As described above, each atlas from MNI was projected into the subjects' native image space using the inverse DARTEL deformation fields. The average GM volume for each region was determined from the GM segments and normalized by total intracranial volume (TIV) using the proportion of ROI GM volume and TIV [Chan et al., 2001; Jenkins et al., 2000; Müller et al., 2007]. For the DTI data, the mean tract intensity for FA, MD, and MO was calculated. For the rs‐fMRI data, the mean signal time course was extracted for each region using the corrected and band‐pass filtered images. Pairwise Pearson correlation coefficients were then calculated to generate functional connectivity matrices for each subject. From these matrices, the weighted local clustering coefficient (WLCC) [Brier et al., 2012; Onnela et al., 2005] and the average shortest weighted path length (SWPL) [Brier et al., 2012; Rubinov and Sporns, 2010] were calculated using the Brain Connectivity Toolbox (http://www.brain-connectivity-toolbox.net) [Rubinov and Sporns, 2010]. The WLCC measures the interconnectedness of a node in a graph by summing up and normalizing the weighted triangular connections with each of two neighbor nodes. In our case, a node is a brain region and a weighted connection is the correlation between the rs‐fMRI time signals of two brain regions. The clustering coefficient has been reported to be useful in the automated discrimination of MCI from HC [Wee et al., 2012], and, when used in combination with DTI data, found to increase accuracy when compared to single modalities [Wee et al., 2012]. The SWPL measures the efficiency of a graph, that is the shortest way from one node to another [Rubinov and Sporns, 2010]. The procedure used to calculate the WLCC and the SWPL is illustrated in Supporting Information Figure S1. First of all, the autocorrelation entries were removed by filling the diagonal of the functional connectivity matrices with zeroes. Negative correlation values were then inverted as the WLCC is not defined for negative weights. Subsequently, a correlation significance threshold was applied to exclude nonsignificant entries and Fisher's z‐transformation [Fisher, 1915] used to adjust the distribution of correlation coefficients to be normal: z = 1/2 ln [(1 + r)/(1 − r)], a requirement for classical linear models. Finally, the WLCC was calculated using the formula WLCC_i = 2 ∑_j,l (w_ij w_jl w_il)^1/3/(k_i(k_i − 1)), with k_i being the number of connections of node i, and w_ij, w_il, and w_jl being the weight of the connection between nodes i and j, i and l, or j and l, respectively [Onnela et al., 2005]. The average SWPL was calculated as SWPL_i = 1/n ∑_j d_ij, with n being the number of nodes (regions), d_ij being the distance between node i and j, being defined as d_ij = ∑1/w_u,v∈g_i,j, with 1/w_u,v being the inverse weight (cost) for the path from node u to v, and g_i,j defining the shortest or ‘cheapest’ path from node i to j. The shortest path g_i,j was determined automatically from the inverted functional connectivity matrix using the Dijkstra algorithm [Rubinov and Sporns, 2010]. Using this procedure, we obtained 10 features for each region: the WLCC and the SWPL for each of the five frequency sub‐bands. All of them were combined as feature vector for the machine learning analysis. As the optimal correlation significance threshold was not known in advance, we selected the three commonly used values 0.05, 0.01, and 0.001 and included them in the parameter estimation procedure as described below.

Classification Methods

The learning and classification process involves five steps: (i) dividing the subjects into a training set and an evaluation set, (ii) reducing the effect of covariates on the data, (iii) selecting discriminative regions, (iv) training the SVM classifier model using the training data, and (v) evaluating the performance of the SVM model using the evaluation data (Fig. 1). The whole process was conducted using the LibSVM MATLAB library (version 3.17, Department of Computer Science and Information Engineering, National Taiwan University, Taiwan) [Chang and Lin, 2011] and custom scripts implemented in MATLAB (release 2013a, The MathWorks, Natick, MA).

Figure 1.

Machine learning analysis flow chart. [Color figure can be viewed in the online issue, which is available at http://wileyonlinelibrary.com.]

To determine the general performance of the SVM classifier, a leave‐one‐out cross‐validation approach was taken. Every subject was selected once as the evaluation dataset, with the remaining subjects forming the training dataset (Step i). Steps (ii)–(iv) were performed blind to the evaluation subject, with the parameters for the transformations (for ii) or feature selection operations (for iii) estimated from the training dataset. To reduce the effect of covariates on the data (Step ii), we used linear regression models to estimate the effect of the covariates age, gender, and years of education on each feature of the training sample [Hastie et al., 2013, section 3.2]: y_ij = β _{0 j} + β _{1 j} age_i + β _{2 j} gender_i + β _{3 j} edu_i + e_ij, with y_ij being the mean volume, intensity, WLCC, or SWPL of region j for subject i; β _{0 j},…,β _{3 j} being the estimated regression parameters for region j; age_i, gender_i, edu_i being the covariates for subject i; and e_ij being the residual for region j and subject i. The β _{0 j},…,β _{3 j} parameters were estimated using the least square error criterion which optimized the model fit and minimized the sum of squared residuals [Hastie et al., 2013, section 3.2]. The residual e_ij for each feature y_ij was used for each of the subsequent steps. The residual e_ij was calculated for the evaluation data using the β _{0 j},…,β _{3 j} parameters obtained for the training data: e_ij = y_ij − (β _{0 j} + β _{1 j} age_i + β _{2 j} gender_i + β _{3 j} edu_i). Next, a t‐test was applied to the training data for feature selection (Step iii). All the regions that showed a group difference above a given threshold were included as input features in the SVM. To determine the optimal threshold, we tested 0.01, 0.05, 0.1, 0.2, 0.3, 0.4, and 0.5 as possible p‐values and used the optimal value selected by the parameter estimation procedure. We did not include 0.001 because this value was too conservative and led to the exclusion of all regions for some iterations of the cross‐validation. Before applying the SVM algorithm to the data, all training data was rescaled so that the values of every feature ranged between zero and one. We then selected the SVM parameters C (margin width) and γ (radial basis function kernel width) together with the p‐threshold for initial feature selection using a grid search in the range of C = 2⁻⁵, 2⁻⁴,…, 2¹⁵ and γ = 2⁻¹⁵, 2⁻¹⁴,…, 2³ and an internal tenfold cross‐validation procedure which further divided the training dataset into tenfolds to determine the optimal parameters. The set of parameters which performed best across the internal cross‐validations was selected for each imaging modality and used to train the SVM models (Step iv). Our final step (Step v) was to evaluate the performance of the SVM model in conjunction with the evaluation data.

As measures of performance, we report the mean accuracy, sensitivity, specificity, positive and negative predictive values, and the area under the receiver operating characteristic curve (AUC) derived from the SVM decision values for the evaluation data. The 95% confidence intervals of these measures were computed using the efficient‐score method [Newcombe, 1998], and in particular Newcombe's fourth method, which is commonly referred to as the Wilson procedure with continuity correction. We additionally report the mean ratio of support vectors, calculated as the number of support vectors divided by the number of training subjects [Cortes and Vapnik, 1995]. This measure indicates the degree of complexity of the SVM model. A ratio of 100% indicates that the model is highly complex and overfitted to the training data, while smaller values indicate less complex models.

We used contingency tables to compare the coherence of the SVM models across the different imaging modalities. These 2‐by‐2 tables permit a pairwise comparison of prediction error: T = [[m ₀₀, m ₀₁]; [m ₁₀, m ₁₁]]. They contain the number of subjects incorrectly classified by both modalities m ₀₀, the number of subjects correctly classified by both modalities m ₁₁, and the number of subjects correctly classified by the first modality m ₁₀ or the second modality m ₀₁, respectively. To measure the similarity of the SVM predictions, we calculated the Jaccard similarity coefficient [Jaccard, 1901] J = m ₁₁/(m ₀₁ + m ₁₀ + m ₁₁) and the Spearman rank correlation coefficient ρ.

Multimodal Analysis

For multimodal analysis, a MK‐SVM algorithm [Dyrba et al., 2012; Sonnenburg et al., 2006; Wee et al., 2012] was used. The multimodal SVM kernel k(x ₁, x ₂), sometimes referred to as the ‘mixed kernel,’ is calculated as the weighted sum of single modality kernels [Dyrba et al., 2012; Wee et al., 2012]: k(x ₁, x ₂) = (∑_{m ∈ M} β_mk^(m)( ${\mathit{x}}_1^{(m)}$, ${\mathit{x}}_2^{(m)}$))/(∑_{m ∈ M} β_m) with the modality weights β_m ≥ 0, the modalities M = {rs‐fMRI, DTI, GM vol}, and the single modality kernels k^(m)( ${\mathit{x}}_1^{(m)}$, ${\mathit{x}}_2^{(m)}$). The parameters for our analysis were selected as follows: the p‐thresholds used for feature selection were obtained from the single modality analysis and γ was chosen to be optimal for every single modality. The optimal β_m kernel weights and the margin width C were determined using the grid search method described above, with β_m ∈ {0.5, 0.75, 1, 1.25, 1.5} and C = 2⁰, 2¹, …, 2⁷.

Visualization

To assess which regions contributed most to the separation of the data we used a heuristic method known as sensitivity analysis [Rasmussen et al., 2011; Smith et al., 2007], which assesses the relative importance of a single feature for classification, and thus provides a relative measure of how much the value of a particular region influences the outcome of the learned SVM model [Rasmussen et al., 2011]. To calculate the sensitivity values, we used the MATLAB script provided by Rasmussen et al. [2011]. We first applied the logarithm to the raw sensitivity values to reshape the distribution of values from exponential to more uniform, and then rescaled the values to range between zero and one. For the purposes of visualization, the original atlases were used and each region colored in accordance with its sensitivity.

RESULTS

In the automated separation of AD from HC, we obtained a cross‐validated accuracy of 74% for rs‐fMRI, 85% for DTI, and 81% for GM volume, and an AUC of 80% for rs‐fMRI, 87% for DTI, and 86% for GM volume (Table 2). In the multimodal analysis, we obtained an accuracy of 79% and an AUC of 82% (Table 2), with the highest contribution coming from the DTI data and GM volume. The multimodal results did not differ significantly from the results of the single modalities (McNemar's test). The mean ratio of support vectors was 67% for rs‐fMRI, 99% for DTI, 52% for GM volume, and 66% for the MK‐SVM (Table 2). We observed that accuracy and AUC ranged between 52 and 80% for the rs‐fMRI data when different p‐value thresholds were used, but were less variable for the DTI data and the GM volume.

Table 2.

SVM performance and ratio of support vectors for the different modalities and the MK‐SVM

	Accuracy (%)	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)	AUC (%)	Ratio of SV (%)
rs‐fMRI	74 (60, 85)	82 (69, 91)	64 (50, 76)	72 (58, 83)	76 (62, 86)	80 (66, 89)	67
DTI	85 (72, 93)	86 (73, 93)	84 (71, 92)	86 (73, 93)	84 (71, 92)	87 (74, 94)	99
GM volume	81 (67, 90)	82 (69, 91)	80 (66, 89)	82 (69, 91)	80 (66, 89)	86 (73, 93)	52
MK‐SVM1	79 (65, 88)	82 (69, 91)	76 (62, 86)	79 (65, 88)	79 (65, 88)	82 (69, 91)	66
MK‐SVM2	85 (72, 93)	79 (65, 88)	92 (80, 97)	92 (80, 97)	79 (65, 88)	89 (77, 95)	79

MK‐SVM1: multiple kernel SVM classifier using rs‐fMRI functional connectivity measures weighted local clustering coefficient and average weighted path length, the DTI data, and GM volume as input; MK‐SVM2: multiple kernel SVM classifier based on DTI data and GM volume

Abbreviations: PPV—positive predictive value, NPV—negative predictive value, AUC—area under the receiver operating characteristic curve, SV—support vectors, rs‐fMRI—resting state functional magnetic resonance imaging, DTI—diffusion tensor imaging, GM—gray matter, MK‐SVM—multiple kernel support vector machine

The sensitivity maps are shown in Figure 2, with each ROI colored in accordance with its sensitivity value. Table 3 contains the list of the corresponding anatomical structures. For each map, a relative threshold was applied to restrict the list to those regions within the highest 10% of sensitivity values only. Several regions of the frontal, parietal, and temporal lobe contributed information to the group separation using the rs‐fMRI clustering coefficients and shortest path length measures. The sensitivity analysis of the DTI data highlighted the fornix (MD and FA) and the ventral part of the cingulum (MD only). The MO did not show sensitivity values above the threshold but revealed a similar pattern of regions as obtained for FA. The most informative regions for GM volume included the left and right caudate nucleus and the left amygdala, as well as the right precuneus and frontal areas.

Figure 2.

The SVM sensitivity maps indicate the relative importance of each region of interest for the SVM prediction. [Color figure can be viewed in the online issue, which is available at http://wileyonlinelibrary.com.]

Table 3.

Anatomical regions derived from the SVM sensitivity maps

	Sensitivity value	Side	Region	ROI index
rs‐fMRI	1,00	R	Insula + inferior frontal gyrusSWPL,e	71
	0,81	L+R	Caudate nucleusSWPL,b	130
	0,77	R	Superior + middle temporal gyrusSWPL,a	153
	0,75	L+R	PrecuneusSWPL,b	174
	0,74	R	Precentral + postcentral gyrusSWPL,d	60
	0,73	R	Inferior temporal + occipital gyrusSWPL,a	26
	0,70	R	CerebellumSWPL,c	41
	0,65	R	Precuneus + cuneusSWPL,a	3
	0,58	L	PrecuneusSWPL,b	136
	0,58	L	Lingual gyrusSWPL,b	177
DTI	1.00	–	MD fornix (body)	6
	0.96	L	MD hippocampal and posterior cingulum	38
	0.88	R	MD hippocampal and posterior cingulum	37
	0.81	L	FA superior fronto‐occipital fasciculus	44
	0.80	L	FA tapetum	48
	0.78	–	FA fornix (body)	6
GM volume	1.00	L	Caudate nucleus (Cau)	71
	0.96	L	Amygdala (Amy)	41
	0.81	R	Caudate nucleus (Cau)	72
	0.82	L	Inferior frontal gyrus (IF)	11
	0.71	R	Precuneus (Prec)	68
	0.70	R	Middle frontal gyrus (MF)	10

The SVM sensitivity values were obtained by calculating the mean sensitivity value for all cross‐validation iterations. A relative threshold was applied in each modality to restrict the list to those regions within the highest 10% of sensitivity values only. The ROI index is given for the following atlases: rs‐fMRI—Craddock's 200 clusters functional atlas, DTI—JHU‐ICBM DTI atlas, GM volume—AAL atlas (references are given in the text). In the case of the rs‐fMRI regions, the approximate anatomical locations of the clusters obtained from the AAL atlas were added.

rs‐fMRI: SWPL—sensitivity for the functional connectivity measure shortest weighted path length, for the measure weighted local clustering coefficient no region was among those with the highest sensitivity; sensitivity for the frequency sub‐band: a—0.025‐0.03929 Hz, b—0.03929‐0.05357 Hz, c—0.05357‐0.06786 Hz, d—0.06786‐0.08214 Hz, e—0.08214‐0.1 Hz

Abbreviations: rs‐fMRI—resting state functional magnetic resonance imaging, DTI—diffusion tensor imaging, GM—gray matter, R—right, L—left, ROI—region of interest, MD—mean diffusivity, FA—fractional anisotropy, WM—white matter

The contingency tables given in Table 4 provide an estimate of the concordance between the SVM predictions using the different modalities. The Jaccard indices and correlations for the pairwise comparison of modalities are given in Table 5. We observed a relatively high level of concordance between the SVM predictions based on rs‐fMRI, DTI data, GM volume, and multimodal MK‐SVM, with Jaccard indices J ≥ 0.74 and Spearman correlations ρ ≥ 0.66. Using only DTI data and GM volume as input for the MK‐SVM neither significantly change the results nor did it improve the accuracy or AUC above those obtained for the DTI data or the GM volume individually.

Table 4.

Contingency tables for the pair‐wise comparison of correctly classified subjects

	DTI incorrect	DTI correct		GM vol incorrect	GM vol correct
rs‐fMRI incorrect	5	9	rs‐fMRI incorrect	6	8
rs‐fMRI correct	3	36	rs‐fMRI correct	4	35
	GM vol incorrect	GM vol correct
DTI incorrect	3	5
DTI correct	7	38
	MK‐SVM1 incorrect	MK‐SVM1 correct		MK‐SVM1 incorrect	MK‐SVM1 correct
DTI incorrect	5	3	GM vol incorrect	6	4
DTI correct	6	39	GM vol correct	5	38
	MK‐SVM2 incorrect	MK‐SVM2 correct		MK‐SVM2 incorrect	MK‐SVM2 correct
DTI incorrect	3	5	GM vol incorrect	5	5
DTI correct	5	40	GM vol correct	3	40

MK‐SVM₁: multiple kernel SVM classifier using rs‐fMRI weighted local clustering coefficient and shortest weighted path length measures, DTI data, and GM volume as input; MK‐SVM₂: MK‐SVM based on DTI data and GM volume

Abbreviations: rs‐fMRI—resting state functional magnetic resonance imaging, DTI—diffusion tensor imaging, GM vol—gray matter volume, MK‐SVM—multiple kernel support vector machine

Table 5.

Jaccard index J and Spearman correlation ρ for the pair‐wise comparison of SVM predictions

	J	ρ
rs‐fMRI vs. DTI	0.75	0.69
rs‐fMRI vs. GM vol	0.74	0.69
rs‐fMRI vs. MK‐SVM1	0.84	0.82
DTI vs. GM vol	0.76	0.66
DTI vs. MK‐SVM1	0.81	0.76
GM vol. vs. MK‐SVM1	0.81	0.76
DTI vs. MK‐SVM2	0.80	0.74
GM vol. vs. MK‐SVM2	0.83	0.78

MK‐SVM₁: multiple kernel SVM classifier using rs‐fMRI weighted local clustering coefficient and shortest weighted path length measures, DTI data, and GM volume as input; MK‐SVM₂: multiple kernel SVM classifier based on DTI data and GM volume.

Abbreviations: vs.—versus, rs‐fMRI—resting state functional magnetic resonance imaging, DTI—diffusion tensor imaging, GM vol—gray

DISCUSSION

Classifier Performance and Coherence

The group separation obtained for our dataset was accurate, with an AUC of 86% for GM volume and 87% for DTI measures. Previous studies have reported a similar AUC, with accuracy levels ranging from 72 to 96% for GM volume [Abdulkadir et al., 2011; Cuingnet et al., 2011; Klöppel et al., 2008; Plant et al., 2010], or from 71 to 98% for DTI data [Cui et al., 2012; Dyrba et al., 2013; Graña et al., 2011; Haller et al., 2010; O'Dwyer et al., 2012; Wee et al., 2012]. The AUC of 80% we obtained for rs‐fMRI data compares favorably with the values reported by Wee et al. [2012], who obtained an AUC of 79% when separating MCI patients from healthy subjects on the basis of local clustering coefficient measures. In contrast to Wee et al., we used the two measures WLCC and SWPL. When using only the WLCC, the accuracy decreased to 64% (Supporting Information Table 1). In our study, performance using the rs‐fMRI data was highly variable and depended on the thresholds tested in the parameter estimation step. The high level of variation in SVM accuracy (between 52 and 74%) indicate that the rs‐fMRI WLCC and SWPL measures contain a large amount of noise that interferes with the accurate detection of group difference patterns, probably causing the SVM models to overfit and the prediction rate to degrade.

In contrast to Wee et al. [2012] and other studies [Hinrichs et al., 2011; Young et al., 2013; Zhang et al., 2011], we did not observe a significant increase in the accuracy of the multimodal analysis when the MK‐SVM algorithm was used rather than the single modality analyses. When only two of the three modalities were used, for instance DTI data and GM volume, the results obtained were essentially the same. In contrast, Wee et al. reported an accuracy increase of 7% when rs‐fMRI and DTI data were assessed in combination, while others [Hinrichs et al., 2011; Young et al., 2013; Zhang et al., 2011] reported a slight increase in accuracy of between 3 and 5% when GM volume and FDG‐PET were combined. However, our results match those obtained in our previous studies [Dyrba et al., 2012] and [Dyrba et al., in press] using independent multicenter datasets, where the multimodal combination of DTI measures and GM volume did not increase the accuracy with which AD [Dyrba et al., 2012] or MCI [Dyrba et al., in press] was detected compared to the single modality analyses. These findings are in line with previous studies that found independent patterns of group differences using DTI data and tissue volume [Canu et al., 2010, 2011; Yoon et al., 2011], but also high correlation between MD and volume in medial temporal lobe areas such as the hippocampus [Canu et al., 2010, 2011; Cherubini et al., 2010; O'Dwyer et al., 2011]. This suggests that there is probably a ceiling effect in the detection of manifest AD using structural imaging methods such as GM volume or DTI measures which makes it impossible to improve the already high diagnostic accuracy of approximately 90%. However, with these highly desirable accuracies, both modalities are well suited to use in automated image diagnostic systems. Future studies need to assess whether multimodal imaging, including functional (or metabolic) imaging methods, provides additional diagnostic accuracy for prodromal AD.

We observed a relatively high level of concordance between the SVM predictions based on rs‐fMRI, DTI data, GM volume, and multimodal MK‐SVM respectively, with Jaccard indices J ≥ 0.74 and Spearman correlations ρ ≥ 0.66. Interestingly, DTI data and GM volume reached highly desirable AUC values of approximately 90% with neither performing with clear superiority. However, the lower ratio of support vectors (52%) for GM volume indicated that the SVM model generalized well in this particular case, while the ratio of support vectors for the DTI data (99%) clearly indicated that the SVM models here were more complex and overfitted the data.

Visualization

A sensitivity analysis of the SVM models identified (bilateral) regions in the caudate nucleus, and the left amygdala as being most informative for group separation based on GM volume (Table 3 and Fig. 2). The amygdala has been reported to be significantly atrophied in AD both in several histological studies [Scott et al., 1992; Unger et al., 1991; Vereecken et al., 1994] and in MRI studies [Basso et al., 2006; Fan, 2011; Laakso et al., 1995; Poulin et al., 2011]. The same applies to the caudate nucleus (histological studies: [Mann, 1991; Pearce et al., 1984], MRI studies: [Barber, 2002; Jiji et al., 2013; Madsen et al., 2010]). The hippocampal region made a moderate contribution to group separation (left hippocampus: sensitivity value of 0.18; right hippocampus: sensitivity of 0.05). When both hippocampus regions were removed from the data, accuracy decreased to 77% (a reduction of 4%) while the AUC remained unchanged at 86%. This may be related to the large size of this region in the AAL atlas compared to the directly adjacent but smaller amygdala, in that calculating the GM volume for the whole AAL hippocampus region may have lowered the region's statistical power. The discrepancy of the minor contribution of the hippocampus in comparison to other studies [Cuingnet et al., 2011; Klöppel et al., 2008; Whitwell et al., 2008a,b; Zhang et al., 2011] may also be an artifact of the segmentation/normalization approach [SPM8 New Segment + DARTEL]. We repeated the GM volume analysis using the Voxel‐Based Morphometry toolbox (VBM8, Release 433) [Gaser et al., 1999; VBM8 segmentation + DARTEL] and obtained a higher sensitivity value for the left hippocampus than for the left amygdala (left hippocampus: sensitivity value of 0.45; right hippocampus: 0.23; left amygdala: 0.27; see Supporting Information Table 2) while the accuracy/AUC remained nearly unchanged (both decreased by 2%; see Supporting Information Table 1). The sensitivity analysis also highlighted frontal regions that might be related to the deficit in executive function displayed by AD patients, as our sample showed a statistically significant group difference in the trail‐making test between AD patients and HC (P < 0.001 for a t‐test).

With reference to the DTI data, the most informative regions were the body of the fornix and the ventral cingulum, both areas that have been reported to be altered in AD in several previous studies [Acosta‐Cabronero et al., 2010; Agosta et al., 2011; Dyrba et al., 2013; Fischer et al., 2012; Nakata et al., 2009; Teipel et al., 2007, 2012; Zhou et al., 2008]. The corpus callosum also contributed information to the group separation, but only to a moderate extent: 0.5 for the genu of the corpus callosum (MD) and 0.3 for the body part (FA and MD). The MO, which reflects differences in the shape of the diffusion tensor, has been reported to provide information which is useful in discriminating cognitively stable MCI subjects from MCI subjects that convert to AD [Douaud et al., 2011]. This measure turned out to be moderately important in the group separation, especially in the left crus of the fornix (0.6). Generally, the regions highlighted by the sensitivity analysis were similar in the MO and the FA. Both measures were associated with a Pearson correlation coefficient of r = 0.61.

The functionally defined regions obtained from the sensitivity analysis of the rs‐fMRI WLCC and SWPL measures included frontal, temporal, and parietal association areas, as well as subcortical regions. In the main, these regions matched areas that we obtained from the sensitivity analysis of GM volume, in particular in the precuneus and cuneus (bilateral), right inferior temporal and occipital gyrus, caudate nucleus (bilateral), and in the caudal part of the left thalamus/hippocampus. In contrast, the clusters in the right insula and inferior frontal gyrus, in the right superior and middle temporal gyrus, and in the right precentral and postcentral gyrus did not coincide with high sensitivity in the GM volume of the corresponding AAL regions. Nearly all the functional regions highlighted were located directly adjacent to fiber tracts that exhibited high or moderate sensitivity. Most of these regions have been reported to show altered functional connectivity measures in AD or MCI [Buckner et al., 2009; Minati et al., 2014; Sorg et al., 2007; Tijms et al., 2013; Wee et al., 2012]. Remarkably, the regions of the default mode network , which have been the focus of several studies regarding AD and functional connectivity [Koch et al., 2012; Pievani et al., 2011; Teipel et al., 2010a], were not shown, with the exception of the posterior cingulate/precuneus and medial prefrontal cortex (both bilateral), to make relevant contributions to group separation. This may be due to the fact that we determined the clustering coefficient and shortest path length across all regions of the brain as previously suggested by Wee et al. [2012], possibly missing high inter‐regional connectivity between small groups of regions. Moreover, a large‐scale analysis of functional connectivity alterations across multiple networks in predementia and mild AD stages revealed similar or even higher effect sizes for functional connectivity disruptions within several other networks, most notably a sensory‐motor network, as compared to the DMN [Brier et al., 2012].

Limitations and Future Work

In this study, we used a hypothesis‐free atlas‐based approach which can be implemented easily in clinical expert systems. The advantages of this approach are that it is more stable and less vulnerable to noise artifacts than a voxel‐based analysis, in particular for functional MRI studies [Poldrack, 2006], and that it lowers the number of multiple comparisons [Poldrack, 2006]. The drawback is that some atlas areas might be too large or too unspecific to detect group differences [Giuliani et al., 2005; Snook et al., 2007], as was possibly the case for the AAL hippocampus region. In the past, most atlas‐based studies assessing functional connectivity [Sanz‐Arigita et al., 2010; Wee et al., 2012] have used the AAL atlas based on cortical and subcortical anatomical structures [Tzourio‐Mazoyer et al., 2002]. These anatomically defined regions may contain distinct functional subregions with different time signal patterns [Craddock et al., 2012; Poldrack, 2006]. To avoid this problem, we used Craddock's functionally defined atlas [Craddock et al., 2012] to provide the parcellation for the rs‐fMRI analysis.

The data used in our study were from AD patients in mild to moderate stages of the disease, but patients were not further stratified by disease severity. Future studies should investigate the spatial and temporal relationship/concordance between the various imaging modalities at different stages of the disease. Furthermore, future studies need to confirm whether or not multimodal imaging provides additional diagnostic accuracy in prodromal stages of AD or in differential diagnosis between different types of dementia.

CONCLUSION

This study compared functional and structural connectivity patterns and volumetric alterations using the imaging modalities rs‐fMRI, DTI, and anatomical MRI. The multivariate SVM algorithm was used to separate CSF‐positive AD patients from matched healthy subjects. We obtained highly accurate results for DTI measures of major WM fiber tracts, GM volume, and, in a multimodal MK‐SVM analysis, the two combined. Whole brain functional connectivity, measured by the clustering coefficient and shortest path length, did not contribute additional information to the multimodal classification model.

Conflict of Interest Disclosure

No current or potential conflict of interest exists in relation to this article.

Supporting information

Supplementary Information