Performance of FDG PET for Detection of Alzheimer's Disease in Two Independent Multicentre Samples (NEST-DD and ADNI)

Abstract

Aim:

We investigated the performance of FDG PET using an automated procedure for discrimination between Alzheimer's disease (AD) and controls, and studied the influence of demographic and technical factors.

Methods:

FDG PET data were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) [102 controls (76.0 ± 4.9 years) and 89 AD patients (75.7 ± 7.6 years, MMSE 23.5 ± 2.1) and the Network for Standardisation of Dementia Diagnosis (NEST-DD) [36 controls (62.2 ± 5.0 years) and 237 AD patients (70.8 ± 8.3 years, MMSE 20.9 ± 4.4). The procedure created t-maps of abnormal voxels. The sum of t-values in predefined areas that are typically affected by AD (AD t-sum) provided a measure of scan abnormality associated with a preset threshold for discrimination between patients and controls.

Results:

AD patients had much higher AD t-sum scores compared to controls (p < 0.01), which were significantly related to dementia severity (ADNI: r = −0.62, p < 0.01; NEST-DD: r = −0.59, p < 0.01). Early-onset AD patients had significantly higher AD t-sum scores than late-onset AD patients (p < 0.01). Differences between databases were mainly due to different age distributions. The predefined AD t-sum threshold yielded a sensitivity and specificity of 83 and 78% in ADNI and 78 and 94% in NEST-DD, respectively.

Conclusion:

The automated FDG PET analysis procedure provided good discrimination power, and was most accurate for early-onset AD.

Introduction

Alzheimer's disease (AD) is one of the most common neuropsychiatric disorders in late life that is characterized by deficits in cognitive and behavioural function, personality changes and impaired activities of daily living [1]. Increasingly, the importance of improving the accuracy of diagnosis at an early stage of the disease when clinical symptoms may still be ambiguous is being recognized, and imaging techniques are being considered as promising tools to achieve this [2].

Data used in the preparation of this article were obtained from the Network for Standardisation of Dementia Diagnosis (NEST-DD) and the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (www.loni.ucla.edu/ADNI). As such, the investigators within NEST-DD and ADNI contributed to the design and implementation of those studies and/or provided data, but did not participate in the analysis or writing of this report, except K.H. and W.J.J. (complete list of ADNI investigators available at: www.loni.ucla.edu/ADNI/About/About_InvestigatorsTable.shtml; NEST-DD principal investigators were D. Perani, Milan, S. Sorbi, Florence, E. Salmon, Liege, V. Holthoff, Dresden, and J.C. Baron, Caen).

Positron emission tomography (PET) using the tracer [¹⁸F]fluorodeoxyglucose (FDG) is widely applied for measuring the regional cerebral metabolic rate of glucose consumption, which provides information about the distribution and functionality of neurons and synapses in vivo [3]. AD is characterized by regional impairment of cerebral glucose metabolism in neocortical association areas (posterior cingulate, temporoparietal and frontal multimodal association cortex) [4, 5, 6] thought to reflect loss of synaptic activity and density. These changes are already present in those patients with mild cognitive impairment (MCI) who will progress to dementia within 1–2 years [6, 7, 8, 9]. Cross-sectional and longitudinal studies have demonstrated that regional metabolic impairment is closely related to dementia severity and cognitive impairment [3]. Good concordance between parieto-temporal hypometabolism and histopathological diagnosis of AD has also been shown, with values for sensitivity ranging from 90 to 94% and for specificity ranging from 65 to 73% when FDG PET scans, evaluated as ‘positive' or ‘negative' for AD-like patterns, were compared with postmortem pathologic validation [10, 11]. ¹⁸F-FDG PET is therefore seen as a valuable ancillary technique for diagnosing AD [12], and for evaluating the efficacy of drugs that aim at modifying the progression of AD [4].

Especially in the mild stages of AD, metabolic changes can be subtle and difficult to detect and discriminate from normal age-related changes in healthy elderly people by visual inspection. Computer-assisted procedures for detection of metabolic changes that are typical for AD have been developed in recent years to assist visual scan interpretation [13, 14].

The present analysis is based on a discrimination procedure that has been developed by the European Network for Standardisation of Dementia Diagnosis (NEST-DD) [13]. The technique was originally validated in 395 AD patients and 110 controls. It highlights abnormal brain regions and calculates the sum of abnormal t-values in voxels that are located within brain areas that are typically affected by AD (AD t-sum) as a global measure of scan abnormality. The procedure was implemented as the Alzheimer's discrimination tool (PALZ tool) as part of the commercially available software package PMOD (PMOD Technologies, Switzerland). An abnormal AD t-sum is a sensitive indicator of metabolic abnormalities and, in conjunction with clinical symptoms, supports the diagnosis of AD. It does, however, not provide discrimination of AD from other disorders.

The intention of our present project was to examine the performance of this automated discrimination procedure in 2 independent samples of controls and AD patients. The samples were taken from the second part of the NEST-DD data project, in which additional data that had not been included in the set-up of the diagnostic procedure were collected prospectively, and from the American Alzheimer's Disease Neuroimaging (ADNI) project.

Material and Methods

Subjects

Our data were from the ADNI database (www.adni-info.org) and the NEST-DD databases. The ADNI is a partnership of the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, the Food and Drug Administration, private pharmaceutical companies and non-profit organizations. From ADNI, we included 102 normal controls (age 76.0 ± 4.9 years) and 89 patients with mild AD (age 75.8 ± 7.6 years, MMSE 23.5 ± 2.1) (table 1). From the NEST-DD database, only data that had not been part of a previous analysis of diagnostic accuracy were used [13], and were collected prospectively. They comprised 36 normal controls (age 62.2 ± 5.0 years) and 237 patients with mild AD (age 70.8 ± 8.3 years, MMSE 20.9 ± 4.4) (table 1).

Table 1.

Demographic overview of controls and AD patients from ADNI and NEST-DD databases

	ADNI		NEST-DD
	Controls	AD	Controls	AD
n	102	89	36	237
Sex (F/M)	40/62	36/53	21/15	155/82
Age, years	76.0 ± 4.9	75.8 ± 7.6	62.2 ± 5.0	70.8 ± 8.3
Age range, years	62–86	55–88	49–74	49–86
MMSE score	28.9 ± 1.1	23.5 ± 2.1	29.6 ± 0.7	20.9 ± 4.4
Education, years	16.0 ± 3.1	14.7 ± 3.3	13.7 ± 3.2	9.1 ± 4.2

Where indicated, data presented as means 8 SD.

In both databases, all subjects and/or authorized representatives gave written informed consent to participate in this study, which was approved by the responsible ethics committee. Subjects received an extensive screening and diagnostic battery consisting of medical, neurologic, psychiatric, neuropsychological and MRI examinations, as well as laboratory tests and blood samples for genetic analysis. Patients with mild AD met the NINDS/ADRDA criteria [15]. Dementia severity was assessed with the Mini Mental State Examination (MMSE) [16].

Exclusion criteria in both databases were any significant neurologic disease other than AD or history of head trauma followed by persistent neurologic deficits or structural brain abnormalities. Presence or history of psychiatric disorders as well as psychotic features, agitation or behavioural problems within the last 3 months also led to exclusion from the study. Further exclusion criteria were alcohol and substance abuse or dependence within the past 2 years, current use of specific psychoactive medications, and any significant systemic illness or an unstable medical condition.

AD patients and controls from the ADNI database were comparable with respect to age (p = 0.826), gender (χ² = 0.03, p = 0.862) and education (p = 0.081). In the NEST-DD database, AD patients were comparable regarding gender (χ² = 0.681, p = 0.409), but significantly older (p < 0.01) and less educated (p < 0.01) than healthy controls. Both ADNI groups were significantly older than controls and AD patients from NEST-DD (all p < 0.01). NEST-DD controls were less educated than ADNI controls (p < 0.01), but no difference was found for ADNI AD patients (p = 0.557). NEST-DD AD patients had significantly less years of education compared to ADNI controls and ADNI AD patients (both p < 0.01). ADNI AD patients had significantly lower MMSE scores than ADNI controls (p < 0.01). MMSE values were lower for NEST-DD AD patients compared to NEST-DD controls, ADNI controls and ADNI AD patients (all p < 0.01). ADNI controls had significantly lower MMSE scores than NEST-DD controls (p < 0.01).

¹⁸F-FDG PET Scanning Procedures

ADNI PET Scanning

All ADNI subjects underwent PET scanning procedures between January 2005 and December 2007 to study cerebral glucose metabolism (for used scanner types, please see online suppl. table E1, www.karger.com/doi/10.1159/000241879). In total, 186 subjects were injected with a dose of 159–241 MBq of ¹⁸F-FDG, and 5 subjects with a dose of 407–580 MBq, in a resting state in a quiet darkened room (for all subjects: 197 ± 47 MBq). According to the ADNI protocol, PET scans were recorded with eyes open, otherwise the procedures were closely comparable. On average, scans were started 27 ± 12 min after injection. All sites performed 3D data acquisition, provided images corrected for Compton scatter, and measured attenuation correction based upon ‘transmission' and ‘blank' scans for those systems having rod sources, or by CT scan for those sites having a PET/CT scanner. Raw PET data were finally converted to DICOM file format.

NEST-DD PET Scanning

All PET scans were obtained from 6 PET centres between January 2001 and March 2003 after intravenous injection of FDG in a resting state with eyes closed and ears unplugged (for used scanner types, please see online suppl. table E2). Injected doses were not available. On average, scans were started 41 ± 11 min after injection and scan duration was 19 ± 2 min. All centres performed 3D data acquisition, and images were reconstructed using filtered backprojection, including correction for attenuation (measured by transmission scan) and scatter. Raw images were converted to Analyze file format.

Discrimination Analysis

Discrimination analysis of ADNI and NEST-DD PET data of healthy controls and AD patients was done using the PMOD Alzheimer's Discrimination Tool (PALZ), which ran the following procedures in a fully automatic way in accordance to the methodology described in Herholz et al. [13]. Firstly, the images were spatially normalized using the methodology and the standard PET template of SPM99 [17], and then smoothed with a Gaussian filter of 12 mm. Normalized voxel values were obtained by division of the image voxels through the mean calculated in a mask representing voxels with AD-preserved activity. This mask had been defined in the initial NEST-DD sample. In each image voxel, the predicted activity was calculated based on the subject's age and the voxel-dependent age-regression parameters. The normalized voxel values were transferred into t-values with reference to the predicted values, which resulted in t-maps. A mask was then applied, comprising those voxels that had shown a significant decline in glucose metabolism, which included major parts of the temporal and parietal (including the precuneus and posterior cingulate) association cortices, as well as some lateral prefrontal areas (see fig. 1 in Herholz et al. [13]). The sum of all t-values of voxels with FDG uptake below the 95% age-adjusted prediction limit within this mask (AD t-sum) was calculated for each individual, providing a measure of scan abnormality. A fixed threshold (AD t-sum >11,089), as provided by PALZ and defined on the initial NEST-DD sample, was used for discrimination between AD patients and controls. This threshold provides the basis for a statistical test of the null hypothesis: ‘the AD t-sum is normal'. If the AD t-sum is within the normal range, it is unlikely that a tested subject has AD. In contrast, an AD t-sum outside the 95% prediction limit would require rejecting the null hypothesis and support the diagnosis of AD in conjunction with clinical symptoms.

All images were visually checked for smoothing and normalization outcome by using the fusion methods provided by PALZ, which allowed an overlay of the normalized images with the PET template image, and by applying contours to this overlay to verify correct image processing, especially for small brain regions and regions localized at the brain border. Out of the 464 images we processed, we had to exclude 7 images due to normalization failure, which means 1.5% of all images could not be processed further.

Statistical Analysis

SPSS (version 15.0) was used for statistical analysis. Distribution of age, MMSE and AD t-sum values in all groups were analysed with the Kolmogorov-Smirnov test. In the following analyses, we used parametric tests for normally distributed data and the Mann-Whitney U test for data that was not normally distributed. All reported p values are two-tailed, and we defined statistical significance at the 5% level. Diagnostic accuracy was analysed by receiver operating characteristic (ROC) analysis and compared between samples by z-statistics.

Results

Healthy Controls from ADNI and NEST-DD Databases

ADNI controls had significantly higher AD t-sum scores compared to NEST-DD controls (8,111 ± 12,841 vs. 4,408 ± 4,389, p < 0.05), which could be due to significantly older ADNI controls (76.0 ± 4.9 vs. 62.2 ± 5.0 years, p < 0.01). Therefore, we selected an ADNI subgroup of 36 subjects (MMSE 28.8 ± 1.3) with a mean age of 70.9 ± 2.6 years that was as close as possible to the NEST-DD sample. For this group, there was no significant difference in the AD t-sums between the ADNI and NEST-DD databases (4,687 ± 4,768 vs. 4,408 ± 4,389, p = 0.797), demonstrating that the difference had in fact been due to age.

AD Patients from ADNI and NEST-DD Databases

There was no significant difference in AD t-sums between patients from both databases (ADNI 38,709 ± 31,970 vs. NEST-DD 39,977 ± 35,203, p = 0.757) in spite of differences in age (75.8 ± 7.6 vs. 70.8 ± 8.3 years, p < 0.01) and dementia severity (MMSE 23.5 ± 2.1 vs. 20.9 ± 4.4, p < 0.01), which possibly compensated each other.

Early-Onset AD Patients versus Late-Onset AD Patients

As expected, late-onset AD (LOAD) patients were significantly older than early-onset AD patients (EOAD) in both databases (p < 0.01), but no difference regarding dementia severity (MMSE) was found (ADNI p = 0.628, NEST-DD p = 0.960) (table 2). In both databases, patients with EOAD had significantly higher AD t-sum scores than patients with LOAD (p < 0.01).

Table 2.

Demographic data, AD t-sums (means ± SD) and values for AUC and 95% CI in ADNI and NEST-DD databases

	ADNI			NEST-DD
	all patients	EOAD	LOAD	all patients	EOAD	LOAD
n	89	15	74	237	73	163
Age	75.8 ± 7.6	64.2 ± 5.1	78.2 ± 5.6	70.8 ± 8.3	60.8 ± 5.8	75.3 ± 4.2
MMSE	23.5 ± 2.1	23.3 ± 2.2	23.5 ± 2.1	20.9 ± 4.4	20.9 ± 4.6	21.0 ± 4.3
AD t-sum	38,709 ± 31,970	76,025 ± 41,884	31,145 ± 23,485	39,977 ± 35,203	58,622 ± 46,544	31,794 ± 24,778
AUC	0.896	0.976	0.879	0.932	0.937	0.929
CI	0.850–0.941	0.947–1.000	0.825–0.932	0.897–0.966	0.892–0.982	0.890–0.968

NEST-DD: classification of onset type was missing for 1 patient.

Comparison of AD t-Sums between Healthy Controls and AD Patients

ADNI AD patients had much higher AD t-sum scores compared to ADNI controls (38,709 ± 31,970 vs. 8,111 ± 12,841, p < 0.01) and their AD t-sum scores were significantly related to dementia severity (Spearman r = −0.308, p < 0.01); this relationship was also significant for the combined group of AD patients and controls (Spearman r = −0.615, p < 0.01).

In NEST-DD, AD t-sum values for AD patients were also significantly higher than for controls (39,977 ± 35,203 vs. 4,408 ± 4,389, p < 0.01) and their AD t-sum scores were also significantly related to dementia severity (Spearman r = −0.431, p < 0.01); for the combined group of AD patients and controls, the relationship was also significant (Spearman r = −0.588, p < 0.01).

Discrimination Results

Discrimination between AD patients and controls was analysed primarily by ROC (fig. 1, 2; table 2). The area under the ROC curve (AUC) tended to be smaller for ADNI data (0.896) than for NEST-DD data (0.932), but the difference was not significant. Neither the ROC curves for EOAD nor LOAD differed significantly between the 2 studies (fig. 2). The only significant difference was observed between the higher AUC in EOAD than in LOAD (0.976 vs. 0.879, z = 3.18, p = 0.0014) in the ADNI study.

Fig. 2.

ROC curves for diagnostic performance of FDG PET data in ADNI and NEST-DD database for EOAD and LOAD patients (true positives = sensitivity; false positives = 1 − specificity).

For further analysis, the preset threshold of 11,089, as obtained in the publication by Herholz et al. [13], was applied. It represents the age-adjusted 95% prediction limit of AD t-sum distribution in normal controls. In our data, it yielded a sensitivity of 83.2% (73.7–90.3%, 95% binomial confidence limits, CL) and a specificity of 78.4% (69.1–85.9% CL) for discrimination of ADNI AD patients from ADNI controls. In NEST-DD sensitivity (77.6%; 71.8–82.8% CL) tended to be lower, while specificity was very high (94.4%; 81.3–99.3% CL). This indicates that the preset threshold operated at a very conservative level in the NEST-DD sample, and that adjustment of that threshold could yield more balanced values for sensitivity and specificity.

At a threshold set at 7,100 in the NEST-DD sample, both sensitivity and specificity were 86%. In the ADNI sample, balanced sensitivity and specificity of 79 and 78%, respectively, was reached at a threshold of 12,600. Thus, the automated analysis produced acceptable results using the preset threshold, but also provides room for adjustment of sensitivity and specificity to diagnostic needs in individual samples.

Discussion

The present study provides a validation of the unmodified and user-independent data analysis procedure for FDG PET scans provided by PALZ in a completely independent large patient sample which, to our knowledge, has not yet been accomplished for any other automated procedure. Obviously, this achievement does not obviate the need for standard visual quality control of scans, and is not intended to replace or supersede individual descriptive reporting of brain scans. Only the latter can exclude artefacts and address the multitude of possible findings in patients with brain disorders. To emphasize the importance of visual inspection of image data and its interpretation in conjunction with clinical symptoms, as well as to show the different influence of diverse brain regions on the AD t-sum, we included normalized image examples for controls, EOAD patients and LOAD patients from ADNI and NEST-DD as supplementary material (online suppl. appendix E2).

The main results of our study confirm previous reports of high accuracy of FDG PET for discrimination between AD patients and normal controls [for review, see 18]. It also confirms the difference in discrimination accuracy between EOAD and LOAD that has been observed in a single-centre study, which used another automated procedure based on the related concept of z-sums derived from parietal volumes of interest that had been defined on stereotactic surface projections [19]. In that study, ROC AUC values (0.967 and 0.878 in EOAD and LOAD, respectively) were very similar to those observed in our study in the ADNI sample. This was the case even though the volume of interest for sampling of the z-sum had been defined separately for EOAD and LOAD. Our procedure instead included a correction for an age effect in normals by linear regression.

Aging is associated with a tendency for global reduction in cerebral glucose metabolism. This mostly affects frontal brain areas, and is related to microvascular changes and atrophy [20]. Indirect evidence of those was apparent in the voxel-wise comparison of NEST-DD and ADNI controls (online suppl. appendix E3 incl. fig. E1), where the latter showed reduced periventricular glucose metabolism. The reduction in temporoparietal metabolism is the same in vascular and in Alzheimer's dementia when measured in absolute units, even though the distribution pattern is more global in the former and more regional in the latter [21]. Thus, LOAD may be confounded by microvascular changes that impair cognitive function and reduce glucose metabolism globally, but global metabolic impairment will remain undetected in the present approach that does not include absolute quantitation of cerebral glucose metabolism.

Greater memory impairment is often found in LOAD patients, whereas EOAD patients present greater deficits in language and praxis abilities [22]. These non-memory-domain cognitive deficits probably correspond to the observation that EOAD patients often show severer hypometabolism in the temporoparietal cortex, cingulate cortex and precuneus than LOAD [23, 24]. Additionally, more cerebral atrophy, more pronounced perfusion reductions as well as higher plaque and neurofibrillary tangle density are demonstrated in EOAD patients [25, 26, 27],

Values for the AD t-sum were significantly higher in ADNI controls compared to NEST-DD controls, apparently corresponding to the higher age of ADNI controls. This corresponded to a reduction in specificity in ADNI compared to NEST-DD, which could not be explained solely on the basis that the NEST-DD normal sample was considerably smaller. The relevance of age was also confirmed by selection of an ADNI subgroup consisting of 36 controls with a mean age comparable to that of the NEST-DD control group, which removed the difference in AD t-sums. As there is little overlap between the brain regions from which the AD t-sum is calculated and the frontal brain regions that are affected by age [28] and the procedure provides reference values for all voxels that are corrected for age effects, this difference probably indicates a higher prevalence of latent AD in the older ADNI control sample than in the NEST-DD sample. The criticality of control sample characteristics has also been demonstrated by a recent study comparing a cross-sectional sample of elderly normal controls with a group of age-matched normal controls who remained normal for a further 4 years [29]. The former showed significantly reduced metabolism in the posterior cingulate and temporo-parietal regions, which would increase their AD t-sums. Older normal controls are expected to have a higher prevalence of asymptomatic AD and also show a higher prevalence of asymptomatic amyloid deposition [30], which explains the difference in AD t-sums observed in the present study.

The correlation between AD t-sum and dementia severity that was measured with the MMSE was found to be significant in both databases, reflecting the link between cerebral glucose metabolism and cognitive performance [3]. However, the association appeared to be only moderate, especially in the ADNI AD group (r = −0.308). This might be due to the small range of obtained MMSE values in this group (MMSE 18–27) which comprised only mild AD patients. In contrast, the NEST-DD AD group presented with MMSE values between 3 and 28 resulting in a stronger association between the AD t-sum and the dementia severity (r = −0.431). In both databases, the correlation was higher in the combined groups of controls and AD patients which included a broader range of the test values (r = −0.615 in ADNI and r = −0.588 in NEST-DD). Nevertheless, the relationship between clinical manifestations of dementia and underlying neuropathological indexes seems to be variable depending on the age of the patients. Savva et al. [31] reported strong associations between dementia severity and neuritic plaques as well as neurofibrillary tangles in the hippocampus and neocortex of AD patients who died at 75 years of age, whereas this association was less pronounced in patients who died at 95 years of age. Therefore, the interaction of age at onset with the relation between neuropathological, neurometabolic and neuropsychological findings needs further investigation.

Reduction in cerebral glucose metabolism can be present in clinically healthy elderly subjects years before they progress to MCI or AD [32, 33]. In our study, we used a reference database consisting of PET scans of healthy controls recruited on a cross-sectional basis without data on cognitive status in subsequent years. Mosconi et al. [29] showed increased diagnostic sensitivity in identifying MCI and AD patients by using a reference database derived from stable healthy elderly compared to a mixed reference group that included also normal controls who converted to MCI after 4 ± 1 years. It is therefore likely that use of a prospectively stable reference group would increase the diagnostic sensitivity of our procedure but, as the diagnostic target is AD rather than MCI, probably at the expense of specificity.

ADNI and NEST-DD used somewhat different examination conditions. Subjects were studied with eyes open in ADNI and eyes closed in NEST-DD. Voxel-wise comparison between controls from the 2 databases demonstrated the expected higher occipital glucose metabolism in the eyes open condition (online suppl. material). However, we did not find a difference in AD t-sums in age-matched controls and in AD patients, suggesting that the difference in occipital glucose metabolism does not have a major effect on AD t-sums. This is consistent with the fact that occipital glucose metabolism only contributes a minor part to the reference regions which provided intensity scaling, and is not part of the regions that contribute to the AD t-sum.

The proposed data analysis procedure condenses regionally spread abnormalities on the PET images into a single figure, which in effect is a single laboratory value indicating the degree of metabolic abnormality in brain regions that are relevant to AD. The procedure would not provide discrimination between different types of dementia, such as fronto-temporal dementia, dementia with Lewy bodies or vascular dementia, as all of them are associated with some degree of metabolic impairment in the association cortices on which the AD t-sum is based. There are, however, studies demonstrating that the regional spread and progression of functional changes in AD could be separated from other dementia types [34, 35]. Such advanced discrimination is likely to be achieved more efficiently by multivariate techniques, such as principal components analysis [36, 37, 38], which may yield multi-dimensional discriminant functions. These techniques, however, still await further validation by applying them without modification to completely independent samples. Future studies will also show whether combination with amyloid imaging, parameters derived from magnetic resonance scanning (e.g. measures of hippocampal atrophy), or CSF and plasma markers can improve diagnostic accuracy and provide biomarkers for assessment of disease progression [39].

Conclusion

The results from both databases were comparable when adjusted for differences in age distribution. The predefined automated FDG PET analysis procedure provided good discrimination power between AD patients and normal controls, and was most accurate for early-onset AD.

Fig. 1.

ROC curves for diagnostic performance of FDG PET data in ADNI and NEST-DD database (true positives = sensitivity; false positives = 1 - specificity).