Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Apr 1.
Published in final edited form as: Alzheimers Dement. 2014 May 22;11(4):415–424. doi: 10.1016/j.jalz.2014.03.005

Susceptibility of the conventional criteria for MCI to false positive diagnostic errors

Emily C Edmonds 1, Lisa Delano-Wood 2,1, Lindsay R Clark 4, Amy J Jak 2,1, Daniel A Nation 5, Carrie R McDonald 1, David J Libon 6, Rhoda Au 7, Douglas Galasko 3,2,1, David P Salmon 3, Mark W Bondi 2,1, for the Alzheimer’s Disease Neuroimaging Initiative
PMCID: PMC4241187  NIHMSID: NIHMS585654  PMID: 24857234

Abstract

Background

We assessed whether MCI subtypes could be empirically derived within the Alzheimer’s Disease Neuroimaging Initiative (ADNI) MCI cohort and examined associated biomarkers and clinical outcomes.

Methods

Cluster analysis was performed on neuropsychological data from 825 MCI ADNI participants.

Results

Four subtypes emerged: 1) Dysnomic (n=153), 2) Dysexecutive (n=102), 3) Amnestic (n=288), and 4) Cluster-Derived Normal (n=282) who performed within normal limits on cognitive testing. The Cluster-Derived Normal group had significantly fewer APOE-ε4 carriers and fewer who progressed to dementia compared to the other subtypes; they also evidenced cerebrospinal fluid AD biomarker profiles that did not differ from the normative reference group.

Conclusions

Identification of empirically-derived MCI subtypes demonstrates heterogeneity in MCI cognitive profiles that is not captured by conventional criteria. The large Cluster-Derived Normal group suggests that conventional diagnostic criteria are susceptible to false positive errors, with the result that prior MCI studies may be diluting important biomarker relationships.

Keywords: Mild Cognitive Impairment, MCI, Alzheimer’s Disease, Dementia, Neuropsychology, Misdiagnosis, Misclassification, Cluster analysis

1. Introduction

Mild Cognitive Impairment (MCI), conceptualized as a transitional state between normal aging and dementia, is defined by objective evidence for cognitive impairment along with a subjective memory complaint in the context of preserved global cognition and activities of daily living [1–3]. Objective impairment is typically operationalized as 1.5 standard deviations (SDs) or more below normative means on at least one measure in a neuropsychological battery. MCI has been further divided as “amnestic” and “non-amnestic,” which involves deficits in other cognitive domains such as executive functions or language. However, recent research using cluster analytic techniques has demonstrated that individuals with MCI can be grouped based on similarities in their neuropsychological profiles, providing an actuarial method of describing MCI subtypes without being confined to the amnestic/non-amnestic distinction [4–6].

One critical finding from a recent cluster analytic study was the identification of a large subgroup who performed within normal limits on neuropsychological testing despite their MCI diagnosis [4]. This Cluster-Derived Normal group did not differ from a normal control group in terms of cognition or imaging measures of cortical thickness in areas usually affected in MCI or Alzheimer’s disease (AD). These results suggest that the conventional diagnosis of MCI may be highly susceptible to false positive diagnostic errors, which is consistent with previous reports of high reversion rates or lack of progression in those with MCI [7–12].

To replicate and extend our previous findings to a large cohort with longitudinal clinical outcomes, we assessed whether distinct MCI subtypes could be empirically derived within the Alzheimer’s Disease Neuroimaging Initiative (ADNI) MCI cohort, and if present examined associated clinical characteristics, biological markers, and longitudinal outcomes.

2. Methods

Data were obtained from the ADNI database (adni.loni.usc.edu). The primary goal of ADNI is to test whether neuroimaging, other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of MCI and early AD. ADNI is the result of efforts of many coinvestigators from a range of academic institutions and private corporations, and subjects have been recruited from over 50 sites across the U.S. and Canada. Participants are recruited via newsletters, web-based communication, direct mail, and press releases. Inclusion criteria include age 55–90, permitted medications stable for 4 weeks, study partner who can accompany participant to visits, Geriatric Depression Scale less than 6, Hachinski Ischemic Score of less than or equal to 4, adequate visual and auditory acuity, good general health, 6 grades of education or work history equivalent, and ability to speak English or Spanish fluently. Exclusion criteria for cognitively normal and MCI participants include any significant neurologic disease or history of significant head trauma. For more information, see www.adni-info.org.

2.1. Participants

Participants were 1,109 ADNI participants who completed a neuropsychological evaluation: 825 diagnosed as MCI at their initial screening evaluation based on ADNI diagnostic criteria [2,13], and 284 classified as cognitively normal. Nearly all of the 825 MCI participants were classified as “amnestic MCI” by ADNI, with only 2 being coded as “non-amnestic MCI.” Criteria for MCI were: 1) subjective memory complaint reported by participant or study partner; 2) Mini-Mental State Examination (MMSE) scores between 24–30 (inclusive); 3) global Clinical Dementia Rating Scale (CDR) score of 0.5; 4) abnormal memory function documented by scoring below education-adjusted cutoffs for delayed free recall on Story A of the Wechsler Memory Scale-Revised (WMS-R) Logical Memory II subtest [14], and 5) general cognition and functional performance sufficiently preserved to an extent that they could not qualify for a diagnosis of Alzheimer’s disease. Importantly, we retained in the Normal Control group all participants who had at least one year of follow-up data and who remained classified as normal for the duration of their participation in the study (range of 1–7 years of follow-up). The Normal Control group of 284 participants did not differ from the MCI group in terms of age, education, or gender (p-values > .05).

2.2. Materials and procedure

Neuropsychological battery

Cognitive measures consisted of six scores from each participant’s baseline neuropsychological evaluation: 1) Animal Fluency; total score, 2) 30-item Boston Naming Test (BNT) total score; 3) Trail Making Test (TMT), Part A; time to completion, 4) TMT, Part B; time to completion, 5) Rey Auditory Verbal Learning Test (AVLT) 30-minute delayed free recall; number of words recalled, and 6) AVLT recognition; number of words correctly recognized. These variables were selected because they were administered to all participants and they assessed three different domains of cognitive ability – language (Animal Fluency, BNT), attention/executive function (TMT, Parts A & B), and memory (AVLT recall & recognition).

Cerebrospinal fluid (CSF) and genetic biomarkers

Biological markers included CSF concentrations of hyperphosphorylated tau (p-tau181p), beta-amyloid (Aβ1-42), and the ratio of p-tau181p/Aβ1-42 [15]; and frequency of the apolipoprotein E (APOE) e4 allele [16–18].

2.3. Statistical analyses

Raw neuropsychological scores for each MCI participant were converted into age- and education-adjusted z-scores based on regression coefficients derived from the Normal Control group. A hierarchical cluster analysis was performed on the z-scores using Ward’s method, consistent with previous MCI studies [4,5]. A discriminant function analysis (DFA) was conducted to in order to more quantitatively examine the ability of the six neuropsychological measures to discriminate the cluster subgroups. The stability of the cluster solution was also examined using leave-one-out cross-validation procedure, a method that reduces the potential bias of using the same individuals to develop the classification matrix and to compute the discriminant function. Following these analyses, differences between groups (i.e., cluster and normal control groups) were examined using a series of ANOVA/ANCOVAs with post-hoc t-tests, and chi-squares. Bonferroni correction was used to adjust for multiple comparisons. Survival curves and Cox regression explored progression and reversion rates.

3. Results

3.1. Cluster and discriminant function analyses

A cluster analysis of the neuropsychological scores from 825 MCI participants resulted in four distinct subgroups based on the mean performance for each group (see Fig. 1): 1) Dysnomic MCI (n = 153; 18.5%) with a significant deficit in naming; 2) Dysexecutive MCI (n = 102; 12.4%) with a significant deficit in executive function, as well as impairments in attention, naming, and memory; 3) Amnestic MCI (n = 288; 34.9%) with isolated memory impairment; and 4) a Cluster-Derived Normal group (n = 282; 34.2%) that performed within normal limits on cognitive testing.

Fig. 1.

Fig. 1

Open in a new tab

Neuropsychological performance for the cluster groups. Error bars denote standard deviations. The horizontal dotted line indicates the typical cutoff for impairment (−1.5 SDs).

DFA using the six neuropsychological measures to predict group membership in the four cluster groups identified three discriminant functions: the first accounted for 74.0% of the variance between groups, the second for 17.1%, and the third for 8.8%. The full predictive model accurately classified 88.0% of participants, and cross-validation of the four-cluster solution using the leave-one-out method showed only mild expected reduction in correct classification (87.3%). A four cluster solution was determined to be optimal relative to a three cluster solution that combined the Dysnomic and Amnestic groups into one group (as this did not allow us to examine how the traditional “amnestic MCI” subtype compared to other cognitive phenotypes), or a five cluster solution that produced unbalanced groups (i.e., one group had only 10 participants). Notably, all the cluster solutions produced an invariant Cluster-Derived Normal group of 282 participants.

3.2. Clinical characteristics of the cluster and normal control groups

Demographic characteristics

As shown in Table 1, the five groups differed in terms of age and education (ps ≤ .001). For age, the Amnestic group was significantly younger than the Dysnomic, Dysexecutive, and Normal Control groups; and the Cluster-Derived Normal group was younger than the Dysnomic group. For education, the Dysexeuctive group was significantly less educated than the all other groups. There was no gender difference between groups (p > .05). Therefore, all further analyses used age and education as covariates.

Table 1.

Demographic, neuropsychological, biomarker, and clinical outcome characteristics of the cluster groups and normal control group

Dysnomic
MCI
(n=153)		 Dysexecutive
MCI
(n=102)		 Amnestic
MCI
(n=288)		 Cluster-
Derived
Normal
(n=282)		 Normal
Control
(n=284)		 F or X2 Sig. Effect Size
Demographics*
Age (years) 75.5 (6.8) 74.7 (7.3) 72.5 (6.9) 73.1 (7.8) 74.2 (5.2) F =6.4 p < .001 ηp2 = .02
Education (years) 16.1 (2.9) 15.0 (3.3) 16.1 (2.6) 16.2 (2.6) 16.3 (2.7) F =4.8 p = .001 ηp2 = .02
Gender (% male) 59.5% 56.9% 62.2% 54.6% 51.8% X2=7.3 p > .05 φc = .08
Cognitive Measures (raw)*
Language
 Animal Fluency 14.5 (3.9) 12.6 (4.2) 16.8 (4.4) 20.3 (4.7) 21.0 (5.6) F=103.7 p < .001 ηp2 = .27
 BNT 22.1 (2.8) 23.0 (5.4) 27.5 (1.7) 28.4 (1.6) 28.3 (2.0) F=254.9 p < .001 ηp2 = .48
Attention/Executive Function
 TMT, Part A (sec) 38.7 (9.5) 71.3 (30.9) 40.3 (12.7) 32.5 (10.3) 34.0 (11.0) F=157.9 p < .001 ηp2 = .36
 TMT, Part B (sec) 107.2 (41.5) 258.5 (49.9) 106.2 (38.9) 82.9 (29.8) 81.6 (38.0) F=498.1 p < .001 ηp2 = .64
Memory
 AVLT Recall 2.6 (2.8) 2.4 (2.8) 2.0 (2.2) 7.1 (4.0) 7.9 (3.8) F=188.1 p < .001 ηp2 = .41
 AVLT Recognition 9.8 (3.4) 9.0 (3.8) 9.0 (3.1) 13.2 (1.7) 13.0 (2.3) F=142.1 p < .001 ηp2 = .34
Diagnostic Measures (raw)*
 LM II Recall 4.5 (3.2) 3.9 (3.1) 5.0 (3.3) 7.6 (2.8) 13.6 (3.2) F =97.4 p < .001 ηp2 = .59
 MMSE 27.1 (1.8) 26.7 (1.7) 27.4 (1.8) 28.4 (1.5) 29.1 (1.2) F =74.4 p < .001 ηp2 = .21
 CDR – Sum of Boxes 1.6 (0.9) 1.8 (0.9) 1.6 (0.9) 1.2 (0.7) 0.0 (0.1) F=224.7 p < .001 ηp2 = .45
CSF†/Genetic‡ Biomarkers
% high p-tau181p 67.6% 82.0% 59.7% 37.8% 31.4% X2=66.2 p < .001 φc = .34
% low Aβ1-42 66.2% 84.0% 67.5% 35.3% 31.4% X2=84.9 p < .001 φc = .38
% high p-tau181p/Aβ1-42 73.0% 84.0% 68.2% 40.4% 36.5% X2=72.2 p < .001 φc = .35
% APOE-ε4 carriers 53.6% 60.4% 58.6% 37.8% 27.7% X2=64.0 p < .001 φc = .25
Clinical Outcome§
% progression to dementia 40.6% 55.6% 34.7% 10.7% -- X2=100.6 p < .001 φc = .26
% reversion to normalcy 1.4% 1.0% 2.2% 9.2% --
Open in a new tab

Abbreviations: BNT, Boston Naming Test; TMT, Trail Making Test; AVLT, Rey Auditory Verbal Learning Test; LM, Logical Memory; p-tau181p, hyperphosphorylated tau; Aβ1-42, beta-amyloid; APOE, apolipoprotein E.

*

Data are summarized as mean (standard deviation), unless otherwise indicated.

†

Number of participants for CSF analysis: Dysnomic: n=74, Dysexecutive: n=50, Amnestic: n=154, Cluster-Derived Normal: n=156, Normal Control: n=156.

‡

Number of participants for APOE analysis: Dysnomic: n=151, Dysexecutive: n=101, Amnestic: n=285, Cluster-Derived Normal: n=278, Normal Control: n=282.

§

Number of participants progression/reversion analysis: Dysnomic: n=138, Dysexecutive: n=99, Amnestic: n=274, Cluster-Derived Normal: n=262.

Neuropsychological performance

As shown in Table 1, there were significant group differences on all six neuropsychological measures (ps < .001). Post-hoc t-tests with Bonferroni correction confirmed that the Dysnomic group performed worse than all other groups on measures of language, with the exception of equivalent performance between the Dysnomic and Dysexecutive groups on Animal Fluency. The Dysexecutive group performed worse than all other groups on measures of attention/executive functioning. The Amnestic group performed worse than the Cluster-Derived Normal and Normal Control groups on both measures of memory, and worse than the Dysnomic group on one measure of memory (AVLT recognition). There was no significant difference between the Cluster-Derived Normal and the Normal Controls on five of the six neuropsychological measures (p > .05); although there was a statistically significant difference in performance on AVLT Recall (p < .01), the Cluster-Derived Normal group’s performance was less than a one-word difference (7.1 vs. 7.9 words) and their mean score on this measure fell well within normal limits (z-score = −0.26).

Performances on ADNI’s diagnostic measures

On the WMS-R Logical Memory II subtest, which was used in ADNI’s MCI diagnosis and thus not included in the cluster analysis, the Dysnomic, Dysexecutive, and Amnestic groups performed similarly to each other (p > .05) but worse than the Cluster-Derived Normal group (ps < .001). A similar pattern was found on the MMSE, as the three impaired groups performed worse than the Cluster-Derived Normal group (ps < .001). Global CDR scores were 0.5 for all cluster groups, as this was a criterion for an MCI diagnosis; however, the Dysnomic, Dysexecutive, and Amnestic groups scored higher on the CDR Sum of Boxes compared to the Cluster-Derived Normal group (ps < .001).

3.3. Biomarker characteristics of the cluster and normal control groups

CSF biomarkers

CSF data were available for 53.2% of the sample (see footnote of Table 1). Based on established CSF cut-point concentrations for p-tau181p, Aβ1-42, and p-tau181p/Aβ1-42 [19], participants were classified into dichotomous groups (high/low) for each variable. Chi-square analysis showed significant differences between groups for all three CSF measures (p < .001; see Table 1). Specifically, all MCI groups demonstrated a greater percentage of individuals with positive CSF AD biomarkers (i.e., high p-tau181p, low Aβ1-42, high p-tau181p/Aβ1-42) compared to the Cluster-Derived Normal and the Normal Control groups, while percentages were comparable between the Cluster-Derived Normal and Normal Control groups. In addition, the Dysexecutive group had higher percentages of individuals with positive CSF AD biomarkers compared to the Dysnomic and Amnestic groups. When CSF measures were analyzed as continuous variables, the same pattern was found for all three measures (see Fig. 2): no differences between the Dysnomic, Dysexecutive, and Amnestic groups (ps > .05), but all had higher p-tau181p, lower Aβ1-42, and larger p-tau181p/Aβ1-42 compared to the Cluster-Derived Normal and Normal Control groups (ps < .001). No differences were observed between the Cluster-Derived Normal and Normal Control groups for any CSF measure (ps > .05).

Fig. 2.

Fig. 2

Open in a new tab

CSF (A) hyperphosphorylated tau (p-tau181p) concentrations (B) beta-amyloid (Aβ1-42) concentrations and (C) ratio of p-tau181p/Aβ1-42 for the cluster groups and normal control group. Error bars denote standard deviations.

A DFA was conducted with only the subgroup of MCI cases who had CSF data available (n=434). The model accurately classified 86.6% of participants, and cross-validation fell minimally to 85.3%. Thus, the classification rates with this subset were comparable to the rates in the entire MCI sample.

APOE

APOE genotypes were available for 98.9% of the sample (see footnote of Table 1). A 2 (APOE-ε4 vs. non-ε4) x 5 (group) chi-square analysis revealed significant group differences in APOE-ε4 frequencies (see Table 1). The Dysnomic, Dysexecutive, and Amnestic groups all had significantly more APOE-ε4 carriers (53.6–60.4%) than the Cluster-Derived Normal (37.8%) and Normal Control (27.7%) groups, although the Cluster-Derived Normal percentage was also significantly higher than the Normal Control group.

3.4. Longitudinal clinical outcomes

Progression/reversion rates

Longitudinal data (mean follow-up = 22.9 months; range 6–84 months), which was available for 93.7% of the MCI sample (see footnote of Table 1), showed that a subset of participants in ADNI’s MCI sample progressed to meet criteria for a diagnosis for probable AD, while a smaller subset reverted to normal (i.e., no longer met criteria for MCI) over time. The Dysexecutive group had slightly less follow-up (18.6 months) than the other three cluster groups (22–25 months; p < .05). A 3 (no change, progression from MCI to AD, reversion from MCI to NC) x 4 (cluster group) chi-square analysis revealed significant differences between the cluster groups (see Table 1), with the Cluster-Derived Normal group showing the lowest rate of progression to dementia (10.7%) and the highest rate of reversion to normalcy (9.2%). The Cluster-Derived Normal group also showed a different survival curve compared to the other cluster groups (see Fig. 3). The Dysexecutive group showed the highest rate of progression to dementia (55.6%). Cox regression including demographic, neuropsychological, and biomarker variables showed that reduced risk of progression to dementia was associated with better scores on the AVLT delayed recall (p < .001, hazard ratio = .504) and TMT, Part B (p < .01, hazard ratio = .848). The Normal Control group was not included in these analyses since they were selected on the basis of remaining normal (did not progress/revert) throughout the course of their participation in ADNI.

Fig. 3.

Fig. 3

Open in a new tab

Hazard function showing risk of progression to dementia across time for the cluster groups.

Post-hoc analysis showed that, within the Cluster-Derived Normal group, the 28 individuals who progressed to dementia were slightly older (p = .03), performed worse on memory testing (ps = .001), and had lower Aβ1-42 (p < .01) and slightly higher p-tau181p/Aβ1-42 (p = .04) in comparison to those who did not progress. Also, 15 of the 28 (53.6%) who progressed carried the APOE-ε4 allele. The mean time point at which a dementia diagnosis was made for these 28 individuals was 33.2 months post-screening.

4. Discussion

We empirically derived subgroups from the ADNI MCI cohort using cluster analysis based on performances on six neuropsychological measures. Four MCI subgroups emerged: Dysnomic, Dysexecutive, Amnestic, and a Cluster-Derived Normal group who performed within normal limits on all six neuropsychological measures (mean z-scores ranged from −0.26 to +0.87) despite their other performances on Logical Memory, MMSE, and Global CDR rating that lead to their ADNI MCI diagnosis. The Cluster-Derived Normal group comprised one-third (34%) of the ADNI MCI sample and was comparable to a robust normal control group in neuropsychological test performance and percentage of individuals with positive CSF biomarkers for AD. In addition, the Cluster-Derived Normal group had fewer APOE-ε4 carriers and fewer individuals with positive CSF biomarkers of AD than the Dysnomic, Dysexecutive, and Amnestic MCI groups. The Cluster-Derived Normal group was also less likely to progress to AD and more likely to revert to normal than the other three MCI groups.

These results are consistent with those of previous cluster-analytic studies showing heterogeneity in neuropsychological4 and biomarker profiles20 in MCI. Despite nearly all participants being classified as “amnestic MCI” by ADNI, results suggest that only one third of the ADNI MCI cohort was solely amnestic, with another third representing primarily dysnomic or dysexecutive subtypes. It is possible that combing subtypes of MCI may limit the generalizability of research findings. In addition to identifying subtypes of MCI, our results also suggest that a significant proportion of individuals in the ADNI MCI sample are cognitively normal once detailed testing is taken into account (i.e., a false-positive error in classification) and do not represent prodromal AD. It is plausible that at least a subset of the Cluster-Derived Normal group may represent a group of individuals who are nevertheless at risk for cognitive decline and AD, particularly given their lower performance on Logical Memory and their higher prevalence of the APOE-ε4 allele relative to the robust Normal Control group, although as a group their intact performances across the neuropsychological tests indicate that a diagnosis of MCI is not warranted.

This statistical method of classifying MCI based on neuropsychological test scores resulted in a significant improvement in the specificity of the diagnosis, as it identified 282 participants with potentially false positive diagnoses. However, it was at a cost of some modest corresponding decline in sensitivity, as a subset of 28 individuals (10.7%) in the Cluster-Derived Normal group did progress to dementia over time (Fig. 3 shows the increase in risk of dementia over time for the Cluster-Derived Normal group due to the inclusion of these 28 participants). Thus their original ADNI diagnosis of MCI could be considered accurate. However, it is important to note that nearly an equal number of individuals (24 participants; 9.2%) in the Cluster-Derived Normal group reverted a cognitively normal classification by ADNI at follow-up, suggesting roughly equal diagnostic errors in the opposite direction. All told, our findings suggest the very modest loss in sensitivity (i.e., 28 of the 282 participants) is far outweighed by the large gains in specificity (i.e., 254 of the 282 participants). In addition, the progression rate of the Cluster-Derived Normal group might be best considered in the context of base rates of cognitive decline for the overall ADNI cohort. Examination of the base rate of cognitive decline in ADNI’s entire normal control group of 404 normal control participants with neuropsychological and follow-up data in ADNI (not just the 284 participants retained for the robust normal group in the current study) was found to be 13%, with 2% of the normal control sample progressing to dementia and 11% progressing to MCI.

There are several possible shortcomings to the diagnostic criteria employed by ADNI that could account for low specificity and large numbers of false positive misclassifications. First, abnormal memory function was determined by a single memory score (delayed recall of Story A from WMS-R Logical Memory), despite evidence showing that isolated low memory test scores are quite common in older adult populations (e.g., 39% of healthy older adults in the WMS-III standardization sample scored in the impaired range on at least one memory measure) [21–23]. Such findings emphasize the importance of considering normal variability and base rates of low memory scores in healthy older adults when interpreting a single test score. Second, only half of the Logical Memory test was administered to ADNI participants (Story A), potentially diminishing its reliability. Third, there is evidence that measures of story memory may be less sensitive to incipient dementia relative to verbal list learning tasks [24,25], suggesting that a list learning test may be a better screening measure. Fourth, general cognitive function was assessed only with the MMSE, a crude measure with limited ability to differentiate healthy controls versus MCI, or MCI versus AD [26]. Finally, the MCI diagnosis required a global score of 0.5 on the CDR [27]. Given the wide range of cognitive and biomarker profiles seen within the four clusters that emerged from the ADNI MCI cohort, it is clear that the global CDR score of 0.5 does not capture variability in cognitive phenotype or level of severity of MCI. This conclusion is supported by previous research showing that global CDR scores of 0.5 in an MCI sample masked variability in cortical thinning and activities of daily living, and was not sensitive to level of MCI severity or in predicting progression to AD [28]. Other research also shows that reliance on global CDR scores in MCI diagnosis results in a high rate of false positive diagnostic errors [29]. The CDR may be susceptible to recall bias or influenced by psychiatric factors, and it is possible that “worried well” individuals could report enough difficulties to obtain a CDR score of 0.5 [29]. Subjective memory complaints can also be related to depressive or personality features [30], or knowledge that one carries a risk factor for AD [31]. In the current sample, the Cluster-Derived Normal group reported more depressive symptoms than Normal Controls on a self-report measure of depression (p < .001), but there were no differences between the four cluster groups (p > .05). This finding supports the possibility that reliance on subjective memory complaints in diagnosis may be another source of variability that contributes to false positive diagnostic errors.

The observed difficulties in conventional criteria for MCI diagnosis have implications for both practice and research. From a practice perspective, DSM-5 criteria for Mild Neurocognitive Disorder requires a “modest impairment in cognitive performance,” but does not state specifically how the determination of cognitive impairment should be made (“preferably documented by standardized neuropsychological testing or, in its absence, another quantified clinical assessment”). Our results suggest that false positive errors in diagnosis are more likely if such determination relies on a single cognitive measure, subjective complaints, or subjective rating scales, rather than based on more detailed neuropsychological evaluation. From a research perspective, findings from studies of the natural history or potential treatment of MCI could be diluted or obscured by the inclusion of individuals who are better classified as cognitively normal by a more thorough sampling of neuropsychological functions (i.e., false positive diagnostic errors). These implications will only assume greater importance as studies begin to examine “preclinical” AD [32] and assign such diagnoses based on fine-grained distinctions of “subtle cognitive declines.”

With regard to the three cognitively impaired MCI subgroups, the groups were similar with regard to APOE status, CSF AD biomarkers, and performance on measures used in ADNI’s diagnosis (e.g., MMSE, CDR, Logical Memory). However, the Dysexecutive MCI group was older, demonstrated impairment in multiple cognitive domains, had higher percentages of individuals with positive CSF AD biomarkers (when CSF was used as a dichotomous measure), and showed the highest rate of progression to dementia compared to the Dysnomic and Amnestic groups. It is not clear whether this group represents more “severe” form MCI or whether primary deficits in attention/executive functioning are impacting performance in other cognitive domains. Additional research is needed to explore whether these different cognitive phenotypes of MCI are associated with distinct clinical outcomes.

Strengths of the present study include a large, well-characterized sample, and use of robust norms [33] that were age- and education-adjusted and derived from a sample of normal control participants that excluded individuals with preclinical dementia (based on 1–7 years of follow-up). A limitation of the current study was the lack of visuospatial measures in the cluster analyses, particularly since a visuospatial MCI subgroup was identified by Clark et al. [4]. If this additional cognitive domain had been included, it is possible that some of the 28 individuals in the Cluster-Derived Normal group who ultimately progressed to AD might have been identified as belonging to a non-normal cluster. The possibility that including more or different neuropsychological measures could modify cluster solutions and potentially identify more individuals at risk for progression to AD will be explored in future studies, in addition to examining the effect of different normative reference methods on cluster solutions. Another future direction will be to compare the conventional MCI diagnostic criteria to actuarial neuropsychological MCI criteria put forth by Jak and Bondi [34] to determine whether this method reduces the number of false positive diagnostic errors. The overarching aim of these efforts is to improve diagnostic accuracy and better characterize distinct prodromal cognitive phenotypes, as the determination of biomarkers cannot substitute for accurate characterization of the clinical syndrome of MCI or prodromal AD. It is hoped that improving diagnostic accuracy will enhance biomarker study findings, opportunities for earlier interventions, and better clinical decision-making.

Panel: Research in Context.

Systematic review

The authors searched PubMed for studies related to misdiagnosis or misclassification of mild cognitive impairment. Review of the literature revealed that the conventional MCI diagnostic criteria are susceptible to errors. Specifically, isolated low scores on cognitive measures can result in false positive errors. The use of subjective memory complaints can elevate both false positive and false negative rates of MCI diagnoses.

Interpretation

Our study supports previous findings showing high rates of diagnostic errors based on conventional criteria, as one-third of our sample was misclassified as MCI. Results further show that this misdiagnosed subgroup had different CSF profiles, APOE allelic frequencies, and rates of progression to dementia in comparison to other MCI subtypes.

Future directions

Future research is needed to improve diagnostic accuracy and better characterize distinct prodromal cognitive phenotypes, including determining whether more comprehensive neuropsychological assessment reduces the number of false positive diagnostic errors.

Acknowledgments

This work was supported by NIH grants R01 AG012674 (MB), K24 AG026431 (MB), P50 AG05131 (DG), R01 AG16495 (RA), and an Alzheimer’s Association grant NIRG-13-281806 (CM). Data collection and sharing for this project was funded by the Alzheimer’s Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions from the following: Alzheimer’s Association; Alzheimer’s Drug Discovery Foundation; BioClinica, Inc.; Biogen Idec Inc.; Bristol-Myers Squibb Company; Eisai Inc.; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc.; GE Healthcare; Innogenetics, N.V.; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Medpace, Inc.; Merck & Co., Inc.; Meso Scale Diagnostics, LLC.; NeuroRx Research; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Synarc Inc.; and Takeda Pharmaceutical Company. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health (www.fnih.org). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer’s Disease Cooperative Study at the University of California, San Diego. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of California, Los Angeles. This research was also supported by NIH grants P30 AG010129 and K01 AG030514.

Footnotes

Conflict of Interest Disclosures: Dr. Bondi serves as Associate Editor for the Journal of the International Neuropsychological Society. Dr. Galasko serves as Editor for Alzheimer’s Research and Therapy, and as a paid consultant on Data Safety Monitoring Boards for Pfizer, Inc., Elan, Inc., and Balance Pharmaceuticals, Inc. Dr. Salmon serves as a consultant for CHDI Foundation, Novartis, and Bristol-Meyers Squibb. The other authors report no disclosures.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • 1.Petersen RC. Mild cognitive impairment as a diagnostic entity. J Intern Med. 2004;256(3):183–194. doi: 10.1111/j.1365-2796.2004.01388.x. [DOI] [PubMed] [Google Scholar]
  • 2.Petersen RC, Morris JC. Mild cognitive impairment as a clinical entity and treatment target. Arch Neurol. 2005;62(7):1160–1163. doi: 10.1001/archneur.62.7.1160. [DOI] [PubMed] [Google Scholar]
  • 3.Winblad B, Palmer K, Kivipelto M, Jelic V, Fratiglioni L, Wahlund LO, et al. Mild cognitive impairment – beyond controversies, towards a consensus: Report of the International Working Group on Mild Cognitive Impairment. J Intern Med. 2004;256(3):240–246. doi: 10.1111/j.1365-2796.2004.01380.x. [DOI] [PubMed] [Google Scholar]
  • 4.Clark LR, Delano-Wood L, Libon DJ, McDonald CR, Nation DA, Bangen KJ, et al. Are empirically derived subtypes of mild cognitive impairment consistent with conventional subtypes? J Int Neuropsychol Soc. 2013;19(6):1–11. doi: 10.1017/S1355617713000313. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Delano-Wood L, Bondi MW, Sacco J, Abeles N, Jak AJ, Libon DJ, et al. Heterogeneity in mild cognitive impairment: Differences in neuropsychological profile and associated white matter lesion pathology. J Int Neuropsychol Soc. 2009;15(6):906–914. doi: 10.1017/S1355617709990257. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Libon DJ, Xie SX, Eppig J, Wicas G, Lamar M, Lippa C, et al. The heterogeneity of mild cognitive impairment: A neuropsychological analysis. J Int Neuropsychol Soc. 2010;16(1):84–93. doi: 10.1017/S1355617709990993. [DOI] [PubMed] [Google Scholar]
  • 7.Ganguli M, Snitz BE, Saxton JA, Chang CC, Lee CW, Vander Bilt J, et al. Outcomes of mild cognitive impairment by definition: A population study. Arch Neurol. 2011;68(6):761–767. doi: 10.1001/archneurol.2011.101. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Koepsell TD, Monsell SE. Reversion from mild cognitive impairment to normal or near-normal cognition. Neurology. 2012;79(15):1591–1598. doi: 10.1212/WNL.0b013e31826e26b7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Nordlund A, Rolstad S, Klang O, Edman A, Hansen S, Wallin A. Two-year outcome of MCI subtypes and aetiologies in the Goteborg MCI study. J Neurol Neurosurg Psychiatry. 2010;81(5):541–546. doi: 10.1136/jnnp.2008.171066. [DOI] [PubMed] [Google Scholar]
  • 10.Summers MJ, Saunders NL. Neuropsychological measures predict decline to Alzheimer’s dementia from mild cognitive impairment. Neuropsychology. 2012;26(4):498–508. doi: 10.1037/a0028576. [DOI] [PubMed] [Google Scholar]
  • 11.Tyas SL, Salazar JC, Snowdon DA, Desrosiers MF, Riley KP, Mendiondo MS, et al. Transitions to mild cognitive impairments, dementia, and death: Findings from the nun study. Am J Epidemiol. 2007;165(11):1231–1238. doi: 10.1093/aje/kwm085. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Petersen RC, Aisen P, Boeve BF, Geda YE, Ivnik RJ, Knopman DS, et al. Criteria for mild cognitive impairment due to Alzheimer’s disease in the community. Ann Neurol. 2013 doi: 10.1002/ana.23931. [Epub ahead of print] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Petersen RC, Aisen PS, Beckett LA, Donohue MC, Gamst AC, Harvey DJ, et al. Alzheimer’s Disease Neuroimaging Initiative (ADNI): Clinical characterization. Neurology. 2010;74(3):201–209. doi: 10.1212/WNL.0b013e3181cb3e25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Wechsler D. Wechsler Memory Scale-Revised. New York: The Psychological Corporation; 1987. [Google Scholar]
  • 15.Landau SM, Harvey D, Madison CM, Reiman EM, Foster NL, Aisen PS, et al. Comparing predictors of conversion and decline in mild cognitive impairment. Neurology. 2010;75(3):230–238. doi: 10.1212/WNL.0b013e3181e8e8b8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Corder EH, Saunders AM, Strittmatter WJ, Schmechel DE, Gaskell PC, Rimmler JB, et al. Gene dose of apolipoprotein E type 4 allele and the risk of Alzheimer’s disease in late onset families. Science. 1993;261(5123):921–923. doi: 10.1126/science.8346443. [DOI] [PubMed] [Google Scholar]
  • 17.Mayeux R, Stern Y, Ottman R, Tatemichi TK, Tang MX, Maestre G, et al. The apolipoprotein epsilon 4 allele in patients with Alzheimer’s disease. Ann Neurol. 1993;34(5):752–754. doi: 10.1002/ana.410340527. [DOI] [PubMed] [Google Scholar]
  • 18.Saunders AM, Strittmatter WJ, Schmechel D, George-Hyslop PH, Pericak-Vance MA, Joo SH, et al. Association of apolipoprotein E allele epsilon 4 with late-onset familial and sporadic Alzheimer’s disease. Neurology. 1993;43(8):1467–1472. doi: 10.1212/wnl.43.8.1467. [DOI] [PubMed] [Google Scholar]
  • 19.Shaw LM, Vanderstichele H, Knapik-Czajka M, Clark CM, Aisen PS, Petersen RC, et al. Cerebrospinal fluid biomarker signature in Alzheimer’s disease neuroimaging initiative subjects. Ann Neurol. 2009;65(4):403–413. doi: 10.1002/ana.21610. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Nettiksimmons J, Decarli C, Landau S, Beckett L. Biological heterogeneity in ADNI amnestic mild cognitive impairment. Alzheimers Dement. 2014 doi: 10.1016/j.jalz.2013.09.003. [Epub ahead of print] [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Brooks BL, Iverson GL, White T. Substantial risk of “accidental MCI” in healthy older adults: Base rates of low memory scores in neuropsychological assessment. J Int Neuropsychol Soc. 2007;13(3):490–500. doi: 10.1017/S1355617707070531. [DOI] [PubMed] [Google Scholar]
  • 22.Brooks BL, Iverson GL, Holdnack JA, Feldman HH. Potential for misclassification of mild cognitive impairment: A study of memory scores on the Wechsler Memory Scale-III in healthy older adults. J Int Neuropsychol Soc. 2008;14(3):463–478. doi: 10.1017/S1355617708080521. [DOI] [PubMed] [Google Scholar]
  • 23.Palmer BW, Boone KB, Lesser IM, Wohl MA. Base rates of “impaired” neuropsychological test performance among healthy older adults. Arch Clin Neuropsychol. 1998;13(6):503–511. [PubMed] [Google Scholar]
  • 24.Rabin LA, Paré N, Saykin AJ, Brown MJ, Wishart HA, Flashman LA, et al. Differential memory test sensitivity for diagnosing amnestic mild cognitive impairment and predicting conversion to Alzheimer’s disease. Neuropsychol Dev Cogn B Aging Neuropsychol Cogn. 2009;16:357–376. doi: 10.1080/13825580902825220. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Tremont G, Miele A, Smith MM, Westervelt HJ. Comparison of verbal memory impairment rates in mild cognitive impairment. J Clin Exp Neuropsychol. 2010;32(6):630–636. doi: 10.1080/13803390903401328. [DOI] [PubMed] [Google Scholar]
  • 26.Mitchell AJ. A meta-analysis of the accuracy of the mini-mental state examination in the detection of dementia and mild cognitive impairment. J Psychiatr Res. 2009;43(4):411–431. doi: 10.1016/j.jpsychires.2008.04.014. [DOI] [PubMed] [Google Scholar]
  • 27.Morris JC. Clinical Dementia Rating (CDR): Current version and scoring rules. Neurology. 1993;43(11):2412–2414. doi: 10.1212/wnl.43.11.2412-a. [DOI] [PubMed] [Google Scholar]
  • 28.Chang YL, Bondi MW, McEvoy LK, Fennema-Notestine C, Salmon DP, Galasko D, et al. Global clinical dementia rating of 0.5 in MCI masks variability related to level of function. Neurology. 2011;76(7):652–659. doi: 10.1212/WNL.0b013e31820ce6a5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Saxton J, Snitz BE, Lopez OL, Ives DG, Dunn LO, Fitzpatrick A, et al. Functional and cognitive criteria produce different rates of MCI and conversion to dementia. J Neurol Neurosurg Psychiatry. 2009;80(7):737–743. doi: 10.1136/jnnp.2008.160705. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Reid LM, MacLullich AM. Subjective memory complaints and cognitive impairment in older people. Dement Geriatr Cogn Disord. 2006;22(5–6):471–485. doi: 10.1159/000096295. [DOI] [PubMed] [Google Scholar]
  • 31.Lineweaver TT, Bondi MW, Galasko D, Salmon DP. Effect of knowledge of APOE genotype on subjective and objective memory performance in healthy older adults. Am J Psychiatry. doi: 10.1176/appi.ajp.2013.12121590. In Press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Sperling RA, Aisen PS, Beckett LA, Bennett DA, Craft S, Fagan AM, et al. Toward defining the preclinical stages of Alzheimer’s disease: Recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011;7(3):280–292. doi: 10.1016/j.jalz.2011.03.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Sliwinski M, Lipton RB, Buschke H, Stewart W. The effects of preclinical dementia on estimates of normal cognitive functioning in aging. J Gerontol B Psychol Sci Soc Sci. 1996;51(4):P217–P225. doi: 10.1093/geronb/51b.4.p217. [DOI] [PubMed] [Google Scholar]
  • 34.Jak AJ, Bondi MW, Delano-Wood L, Wierenga C, Corey-Bloom J, Salmon DP, et al. Quantification of five neuropsychological approaches to defining mild cognitive impairment. Am J Geriatr Psychiatry. 2009;17(5):368–375. doi: 10.1097/JGP.0b013e31819431d5. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES