Low sensitivity in clinical diagnoses of dementia with Lewy bodies

Abstract

The success of future neurodegenerative disease (ND) therapies depends partly on accurate antemortem diagnoses. Relatively few prior studies have been performed on large, multicenter-derived datasets to test the accuracy of final clinical ND diagnoses in relation to definitive neuropathological findings. Data were analyzed from the University of Kentucky Alzheimer's Disease Center autopsy series and from the National Alzheimer's Coordinating Center (NACC) registry. NACC data are derived from 31 different academic medical centers, each with strong clinical expertise and infrastructure pertaining to NDs. The final clinical diagnoses were compared systematically with subsequent neuropathology diagnoses. Among subjects meeting final inclusion criteria (N = 2,861 for NACC Registry data), the strength of the associations between clinical diagnoses and subsequent ND diagnoses was only moderate. This was particularly true in the case of dementia with Lewy bodies (DLB): the sensitivity of clinical diagnoses was quite low (32.1% for pure DLB and 12.1% for Alzheimer's disease (AD + DLB) although specificity was over 95%. AD clinical diagnoses were more accurate (85.0% sensitivity and 51.1% specificity). The accuracy of clinical DLB diagnoses became somewhat lower over the past decade, due apparently to increased “over-calling” the diagnosis in patients with severe cognitive impairment. Furthermore, using visual hallucinations, extrapyramidal signs, and/or fluctuating cognition as part of the clinical criteria for DLB diagnosis was of minimal utility in a group (N = 237) with high prevalence of severe dementia. Our data suggest that further work is needed to refine our ability to identify specific aging-related brain disease mechanisms, especially in DLB.

Introduction

As the populations of Western countries age, the incidence and prevalence of neurodegenerative disease (ND) will rise significantly. There is thus a threat of increasing burdens from Alzheimer's disease (AD), dementia with Lewy bodies (DLB), and other, less prevalent NDs. The prospects of a potential healthcare crisis due to NDs may be offset by specifically tailored therapies targeting disease mechanisms. The therapeutic utility of such future treatments may depend on accurate antemortem diagnoses of the underlying pathological disease state.

ND diagnoses are made with the help of clinical guidelines produced through a consensus and/or consortium of experts. For example, the Consortium on DLB International Workshop guidelines specify fluctuating cognition, early well-formed hallucinations, delusions and paranoia, parkinsonism, and other supportive clinical features such as sensitivity to neuroleptics and REM sleep behavior disorder as diagnostic criteria for DLB [18]. Prior studies have demonstrated that these clinical findings—in particularly ideal diagnostic circumstances—may provide sensitive and specific prediction of confirmed neuropathological results [16]. Indeed, many clinical studies have employed these criteria for DLB as a gold standard in defining which patients have the disease.

Despite the optimism conveyed by the study of ideally chosen patients, the accuracy of clinical diagnoses of NDs can vary significantly depending on the clinical context [4, 7, 15, 21, 32]. Institutions with a relatively sophisticated approach tend to provide higher degrees of sensitivity and specificity in ND diagnoses. At the forefront of academic research on NDs in the United States are AD Centers (ADCs), of which 31 have been established across the country. ADCs house state-of-the art clinical and neuropathological infrastructure. Many of the researchers that helped establish and modify the consensus ND diagnostic criteria work at ADCs. Collectively, ADCs follow thousands of patients longitudinally with regular clinic visits that include cognitive assessments. Following death, a relatively high percentage of patients who are followed clinically (with percentages differing among individual ADCs) undergo autopsy evaluation. These clinical and pathological data are shared through the National Alzheimer's Coordinating Center (NACC) [2, 3]. We addressed the question of the accuracy of clinical (antemortem) diagnoses of NDs in ADCs using subjects from the NACC, including some separate analyses of data from the University of Kentucky ADC (UK ADC).

Methods

NACC Registry data

The NACC Registry represents detailed data obtained from 31 different ADCs across the United States [2, 3]. Details about the NACC Registry subjects that met inclusion criteria are provided in Supplemental file 1. Exclusion criteria and numbers excluded for NACC subjects are described in detail in Supplemental file 2. Briefly, the exclusion criteria were applied a priori to enable clinical–pathological correlations specific to the diagnoses of cortical LB and AD-type pathology. The following groups were excluded: subjects with no Mini-Mental State Examination (MMSE) score; no Braak neurofibrillary staging [5]; no Consortium to Establish a Registry for Alzheimer's Disease (CERAD) [20] neuritic plaques data; clinical history of stroke; clinical history of prion disease, triplet repeat diseases, brain cancer, frontotemporal dementia by pathology, or other disease that would by itself explain the clinical dementia syndrome. Because it was necessary for this analysis to compare clinical and pathological diagnoses, an additional 1,320 subjects were removed that did not have either adequate clinical or pathological diagnostic data (as described in Supplemental file 2). Subjects were included subsequent to 1995, to include an analysis of diagnostic accuracy both prior and subsequent to the National Institute of Aging-Reagan Institute criteria (NIA-RI) [1] and DLB Consortium [18] recommendations. Using these criteria, 2,861 individuals were included to study clinical-pathological correlation in NDs; these subjects clustered in 27 of the 31 ADCs. The average number of subjects per center was 106, the median was 85, and range was 28–296.

UK ADC data

Details about the UK ADC autopsy cohort (total cases = 527, with data directly germane to DLB diagnoses N = 237) are provided in Supplemental file 1. Briefly, aspects of UK ADC IRB protocols, patient recruitment, and longitudinal follow-up have been described previously [24]. These patients were followed in our ADC research clinic with a high prevalence of severe cognitive impairment proximal to death as described previously [22]; the mean final MMSE score was 11.4 ± 8.1. Clinical diagnoses were documented at the UK ADC Consensus Conference which includes neurologists, neuropsychologists, social workers, and other clinical staff members that had direct interaction with research subjects.

Clinical criteria

The clinical diagnoses of DLB, AD, AD + DLB, or none of these, were determined by data from the NACC Registry designed to document these antemortem diagnoses. The final clinical diagnoses and the data fields used to document them are described in Supplemental file 3. Unfortunately, not every patient in the NACC Registry has a clear documentation of why the diagnosis was made—NACC began documenting clinical findings such as visual hallucinations and fluctuating cognition in 2005 and too few autopsy-confirmed cases were available for meaningful analyses of these data. However, the UK ADC had documentation for 237 of its subjects for which there was documentation of visual hallucinations, extrapyramidal signs, and fluctuating cognition (documentation of which began in 1986) and these were included in the current study.

Pathological criteria

The presence or absence of neocortical LBs and AD-type pathology was inferred using the same method in the combined NACC dataset and then for additional analyses using only the UK ADC data. The presence of neocortical LBs was determined by indication of “diffuse neocortical type LB” disease in the data registry. “Pure DLB” required Braak neurofibrillary stages less than stage IV. For “pure AD”, the presence of AD-type pathology was determined using the NIA-RI criteria, where Braak Stages V/VI with CERAD “moderate” or “frequent” indicate the disease has pathological impact with >97% specificity [23]. If sufficient number of neocortical LBs were present to merit the designation of DLB by pathology, then AD was deemed also present (i.e., AD + DLB) with Braak neurofibrillary stages IV, V, or VI and CERAD “moderate” or “frequent”. Braak stage IV cases were included for AD + DLB because, in the presence of abundant neocortical LBs, the neurofibrillary pathology tends to be lower since the pathologies have additive clinical impact [23, 24].

Statistical methods

Agreement between clinical and neuropathological diagnoses was measured using kappa statistics. Using the neuropathological diagnosis as the gold standard, sensitivity, specificity and predictive values positive and negative were computed for various clinical diagnoses. Statistical significance for the kappa coefficient was determined at the 0.05 level.

Results

Information about the NACC cohort and criteria used for exclusion is shown on Table 1 and Supplemental file 1. Using these data and criteria, the clinical and pathological diagnoses from 2,861 patients could be cross-referenced to assess the accuracy in the clinical diagnoses. These data are shown in Table 2. Measures of agreement are shown on Table 3. The results indicate that the clinical diagnoses in persons involving neocortical LBs have poor sensitivity but high specificity. Sensitivity is much higher for clinical diagnoses of pure AD whereas specificity is far lower. The kappa coefficients indicate poor agreement.

Table 1.

Definition of term

Term	Definition
PATH PURE AD	CERAD moderate or frequent and Braak NF stage V–VI
PATH AD + DLB	CERAD moderate or frequent and Braak NF stage IV–VI with “diffuse DLB”
PATH PURE DLB	Braak NF stage <IV with “diffuse DLB”
Path NO AD or DLB	Braak NF stage <IV with NO “diffuse DLB”
CLIN NON-AD NON-DLB	Non-demented or non-AD/non-DLB dementia suspected
CLIN PURE AD	Pure AD suspected clinically
CLIN PURE DLB	Pure DLB suspected clinically
CLIN AD + DLB	AD + DLB suspected clinically
DLB UNDER-CALL	Pure AD or non-demented or non-DLB dementia clinically, but pathology shows pure DLB
DLB OVER-CALL	DLB or AD + DLB clinically, but pathology shows pure AD or no specific pathological changes or non-DLB dementia

Definition of terms used in the current study. Please see Supplemental file 2 for more detailed description of exclusion and inclusion criteria for the various clinical and pathological diagnostic groups

Table 2.

NACC Registry data used to compare clinical (antemortem) and pathological results

	PATH NO AD or DLB	PATH PURE AD	PATH PURE DLB	PATH AD + DLB	Total
CLIN NONDEM/NONADDEM	523	134	31	16	704
CLIN PURE AD	448	1,236	49	191	1,924
CLIN PURE DLB	10	26	52	11	99
CLIN AD + DLB	15	59	30	30	134
Total	996	1,455	162	248	2,861

A comparison of clinical (premortem) diagnoses and pathological diagnoses on autopsy using NACC Registry data (N = 2,861). Note that in this analysis all remaining cases without Alzheimer's disease (AD), dementia with Lewy bodies (DLB) or AD + DLB are included together, most of which are “Non-demented”. The overall prevalence of “pure DLB” by pathology in this population that has been enriched for various neurodegenerative diseases is 5.7%

Table 3.

Tests of agreement between clinical diagnoses and pathological diagnoses

Event	Sensitivity (%)	Specificity (%)	kappa* (95% CI)
AD versus others (control or AD + DLB or DLB)	85.0	51.1	0.36 (0.33, 0.39)
AD + DLB versus others (pure AD, pure DLB, control)	12.1	96.0	0.10 (0.05, 0.15)
DLB versus others (control, AD + DLB, pure AD)	32.1	98.3	0.37 (0.29, 0.45)

Statistical comparison of clinical and pathological diagnoses using data from Table 2. The results indicate poor sensitivity for diagnoses involving dementia with Lewy bodies (DLB), but high specificity in the clinical diagnoses. Sensitivity is much higher for clinical diagnoses of pure Alzheimer's disease (AD) whereas specificity is much lower. The kappa coefficients indicate relatively poor agreement levels

^*All kappa coefficients are significant at the p < 0.0001 level

We hypothesized that the degree of agreement between clinical and pathological diagnoses involving DLB cases would change based on patients' characteristics. Two different categories of potential error were specified in clinical diagnoses related to DLB, as shown in Table 1. An “over-call” of neocortical LBs denotes a clinical diagnosis that predicted the presence of neocortical LBs but without sufficient neocortical LBs found at autopsy. By contrast, an “under-call” of neocortical LBs occurs when the final clinical diagnosis did not suggest neocortical LBs although neuropathological results indicated “diffuse neocortical LBs”.

One specific hypothesis is that diagnostic accuracy depends partly on the degree of patients' global cognitive status as manifested by MMSE scores. A separate testable hypothesis is that individuals who were evaluated in different years (between 1995 and 2008) would have systematic differences in the agreements/accuracy between clinical and pathological diagnoses based on differences in diagnostic practices.

Figure 1 plots the rates of clinical-pathological disagreement by time periods. The proportion of total errors (antemortem under-calls or over-calls relative to subsequent pathology results) increases with time period, 4.8% (1995–1997); 5.9% (1998–2001); 6.9% (2002–2005); and 10.2% (2006–2008) (p < 0.0053; Chi-square test). The error rate increased dramatically in the most recent time period indicating that the accuracy of clinical diagnoses of neocortical LBs is declining over time; however it still remains around 90%. This may be partly the result of exclusion of patients with moderately severe cognitive impairments (MMSE scores ∼20) who tend to be clinically under-diagnosed for LBs while patients with severe cognitive impairments (MMSE scores <5) tend to be incorrectly clinically labeled as DLB (Fig. 2).

Fig. 1.

According to NACC Registry data, the tendency to “under-call” or “over-call” neocortical Lewy bodies (LBs) has increased in recent years (p < 0.0053 for any mis-call; Chi-square test). The definitions of clinically “under-calling” or “over-calling” neocortical LBs are presented on Table 1

Fig. 2.

Analyses of the NACC Registry data allow the correlation between particular patients' final mini-mental status examination (MMSE) scores and the clinicians' tendency to “under-call” or “over-call” neocortical Lewy bodies (LBs). There is an apparent tendency to over-call neocortical LBs in severe dementia patients. The definitions of clinically “under-calling” or “over-calling” neocortical LBs are presented on Table 1

Since patients with severe cognitive impairment showed higher ‘false-positive’ DLB diagnoses, we sought to more fully examine this patient population. The NACC Registry has not gathered enough data in this regard to date (such clinical indices were collected in NACC only since 2005), so subjects were evaluated from the UK ADC (N = 237) in which clinical fields had been indicated as “Yes” or “No” for DLB “core” diagnostic signs (visual hallucinations, extrapyramidal signs (more than two), and cognitive fluctuations) [17, 18]. This is a relatively severe-dementia group (average MMSE score 11.4 ± 8.1). Results are presented in Table 4. These particular clinical signs, in this clinical context, were unreliable predictors of the presence or absence of neocortical LBs on post-mortem examination. The number of core diagnostic signs does not significantly predict a diagnosis that includes DLB (pure DLB or AD + DLB) (p = 0.25) from a logistic regression model predicting diagnosis of DLB based on the number of signs present. Further, the proportion of cases with 0, 1, or 2 core signs is not significantly different for individuals whose pathological diagnoses included DLB (as above) versus subjects that did not based on the Chi-square test (p = 0.87).

Table 4.

Clinical–pathological correlation using UK ADC data on patients with severe cognitive impairments, N = 237 total

Number of clinical core features of DLB present	0	1	2	3
N (total 237)	194	32	10	1
N with pure DLB (%)	10 (5)	2 (6)	1 (10)	0
N with AD + DLB (%)	27 (14)	5 (16)	2 (20)	1 (100)
N with pure AD (%)	118 (61)	20 (63)	5 (50)	0

In patients with severe cognitive impairments (mean final MMSE score = 11.4 ± 8.1), the presence of the core symptoms of DLB (visual hallucinations, extrapyramidal signs, and fluctuating cognition) do not predict the presence of DLBs. The large majority of these patients were not diagnosed clinically with DLB despite the presence of the clinical features, because the symptoms started in late stages of the disease. The definitions of “pure DLB”, “pure AD”, or “AD + DLB” are based on neuropathological data as defined in Table 1

Discussion

Data from the NACC registry suggest that clinical ND diagnoses are only moderately successful at predicting the subtype of NDs found on autopsy when it includes the presence or absence of neocortical LB pathology. Rates of “over-calling” and “under-calling” the presence of LBs based on clinical indices also appear to vary according to the level of cognitive impairment, with more Type II (false negative) errors for clinical DLB diagnoses in moderately impaired individuals and more Type I (false-positive) errors in the DLB diagnoses of severely impaired individuals. This may, in part, be due to the presence of neuropsychiatric symptoms (delusions and hallucinations) in all three diagnostic groups, regardless of the presence of LB. This is not unusual, given that a large proportion of patients with AD exhibit delusions and hallucinations during the course of their disease progression [6, 8]. Perhaps as a result of symptom overlap in AD, DLB, and AD with DLB, it seems that the rate of misdiagnosis has increased in recent years despite the publication of refined clinical criteria for DLB [16, 17]. We also confirm that in late-stage cognitive impairment, specifically documented signs and symptoms associated with DLB, including visual hallucinations, extrapyramidal signs, and fluctuating cognition, are unhelpful for predicting the presence of neocortical LBs at autopsy. Therefore, while these clinical symptoms may be useful in milder cases of dementia, caution should be used when providing a diagnosis of LBD in patients with advanced dementia.

Use of NACC Registry data allowed us to evaluate the accuracy of ND diagnoses in a large cohort across 31 different academic medical centers with special expertise in ND clinical and pathological diagnoses. These results seem to be an accurate reflection of “real-life” challenges in ND clinical diagnoses. We also studied a population of cognitively impaired patients with specific neurological signs (visual hallucinations, parkinsonism, fluctuations, etc.). The goal of this study was not to determine “how accurate can we be?” in ND diagnoses, but rather, to gauge “how accurate have we been?”, and whether diagnostic accuracy (clinical–pathological agreement) is improving in academic medical centers. Prior studies have indicated that ND clinical diagnoses—in optimized circumstances—can predict neuropathology with a relatively high degree of sensitivity and specificity, although the results have varied appreciably from study to study (reviewed in Ref. [16]) and there clearly are both overlaps and differences in ND subtypes' clinical presentation [35].

The present study did not “pre-select” subjects based on any criterion other than eventual pathological diagnoses, and cases with AD, DLB, and AD + DLB pathology were all considered different diseases. This approach was chosen partly because DLB and AD + DLB tend to have distinct clinical manifestation, i.e., AD + DLB tends to have greater cognitive impairment according to several studies [13, 25, 26]. Furthermore, the fundamental biology of AD + DLB may be distinct—many “familial” (i.e., monoallelic) cases manifest AD and DLB pathologies concomitantly [33, 36]. We note also that even if we had pooled cases together with the pathologies of DLB and AD + DLB, the clinical diagnoses were still suboptimal when contrasted with autopsy results. The diagnostic challenges of AD + DLB cases have been described previously [14, 37]. Clearly, many clinical and biological issues related to the synergy of AD and DLB remain unresolved.

Another obstacle to clinical–pathological correlation in any ND is the prevalent, powerful, but unpredictable impact of cerebrovascular disease (CVD), both clinically and pathologically [9, 11, 23, 34]. In the present study we excluded all cases with a known stroke history because this variable dampens the ability to correlate clinical and pathological findings [23]. This history may have had different connotations in the different ADCs. Moreover, clinicobiological connections have been made between CVD and other progressive NDs, so, it is possible that removing cases with stroke histories introduces a bias for reasons that we are not now aware.

Although concomitant/mixed pathologies are perhaps the biggest difficulties in ND clinical–pathological correlation, the present study indicates that there are two other reasons specific to neocortical LBs that impair our ability to predict their presence at autopsy: first, the signs and symptoms of DLB as currently defined are not perfectly specific to the pathological state; and second, neocortical LBs may align with cognitive changes in a manner fundamentally different from AD-type plaques and tangles. To the first point, the data confirm that the presence or absence of extrapyramidal signs, visual hallucinations, and highly fluctuating cognition in late-stage disease are unlikely to yield a specific indication of the presence of neocortical LBs as they are nonspecific findings in the later stages of pathologically confirmed AD. As to the cognitive impact of neocortical LBs, there have been a number of recent papers suggesting that neocortical LBs are an additive pathology [9, 10, 19, 27–31], unlike AD-type pathology which can by itself lead to the most severe end-stage dementia.

Patients described in this study were followed longitudinally and only the final clinical diagnoses were used in the analyses. These data may indicate a tendency for diagnoses to change in patients studied over time; specifically, clinicians may switch diagnoses to DLB due to late-stage symptomatology. In longitudinal studies the diagnoses are frequently reviewed, so this tendency may give the appearance of “worsening” diagnostic accuracy in more progressed dementia cases. Compatible with this hypothesis, many of the subjects from the UK ADC with DLB-relevant symptoms, but lacking neocortical LBs at autopsy, showed severe cognitive impairment. This result among cognitively impaired persons is an important point, since we and others have seen that some DLB signs (e.g. parkinsonism) can be highly predictive of eventual manifestation of DLB pathology, provided that these signs appear early in the clinical course [12]. Accordingly, almost all of the patients described in Table 4 did not have the clinical diagnosis of DLB during life. It is quite possible that the temporal association of specific clinical features with the early stages of dementia rather than at the later or end-stage of disease could improve diagnostic accuracy for DLB. Indeed, the clinical criteria for DLB specifically state that such features should occur early in the course of the disease [17]. In the later or end-stages of dementia the diagnostic specificity of such associated features is lost. Future studies evaluating the temporal association of such features with onset of dementia may help improve diagnostic accuracy for the detection of pathological DLB in the antemortem setting.

The present study has some inherent limitations. To compare the clinical ND diagnoses with the pathological diagnoses referent to AD, DLB, and AD + DLB, we had to exclude thousands of cases with other strong pathologies including CVD, hippocampal sclerosis, frontotemporal diseases, Huntington's disease, brain cancer, and others. As noted above, the presence of CVD is a particularly challenging potential confound to these studies. In the future we may be able to include the “mixed pathology”, but this would require a (currently nonexistent) clinical–pathological rubric for CVD. Another limitation to the current study was that only cases with both clinical and pathological data were included, which resulted in hundreds of cases being excluded from the analyses. However, the remaining number of cases—almost 3,000—constitutes a large number of thoroughly evaluated cases.

There are also intrinsic difficulties in using data pooled from 31 different ADCs, each with different clinical and neuropathological protocols. However, even if some ADCs had much higher accuracy in ND diagnoses, that means only that others have lower accuracy and the goal of this study was to determine the overall rate of accuracy. With these caveats, these data, which are drawn from multiple, relatively sophisticated academic medical centers, indicate that the clinical criteria used in practice to diagnose DLB clinically are imperfect and should be used cautiously in predicting the pathological substrates for living patients' cognitive impairments.