Comparative Sensitivity of the MoCA and Mattis Dementia Rating Scale-2 in Parkinson’s disease

Abstract

Background

Clinicians and researchers commonly use global cognitive assessments to screen for impairment. Currently there are no published studies directly comparing the sensitivity and specificity of the MoCA and Mattis Dementia Rating Scale-2(DRS-2) in PD.

Objective

To identify the relative sensitivity and specificity of the MoCA and DRS-2 in PD.

Methods

The MoCA and DRS-2 were administered to training and validation cohorts. Cutoff scores were determined within the training cohort (n=85) to optimize sensitivity and specificity for cognitive impairment and were applied to an independent validation cohort(n=521).

Results

The MoCA was consistently sensitive across training and validation cohorts(90.0%, 80.3%, respectively), whereas the DRS-2 was not(87.5%, 60.3%, respectively). In individual domains, the MoCA remained sensitive to memory and visuospatial impairments(91.9% and 87.8%, respectively), whereas the DRS-2 was sensitive to executive impairments(86.2%).

Conclusion

The MoCA and DRS-2 demonstrated individual strengths. Future work should focus on developing domain-specific cognitive screening tools for PD.

Introduction

Parkinson’s disease (PD) increases the risk of developing cognitive impairment (CI), resulting in higher healthcare costs and decline in quality of life(1). Cognition is often gauged using global assessments targeting a wide range of abilities(2). The Montreal Cognitive Assessment (MoCA)(3) and the Mattis Dementia Rating Scale-2 (DRS-2)(4) are two of the most commonly used global assessments, both of which are among the measures recommended by the Movement Disorders Society Task Force(MDS-TF) on mild cognitive impairment in PD(5). A recent comprehensive review further recommends the MoCA and DRS-2 over other global assessments in PD(6), however these measures have not been directly compared in the same cohort of PD patients. This knowledge is essential for designing clinical trials targeting therapies for cognitive symptoms in PD.

The pattern, extent, and severity of CI in PD is heterogeneous(5), with multi-domain impairment observed in up to 35% of newly diagnosed patients(7). While both the MoCA and DRS-2 assess multiple domains, it is unclear which measure is more sensitive to domain-specific impairments. Although the MoCA and DRS-2 are not intended to replace comprehensive neuropsychological testing(5), there are circumstances where neuropsychological evaluation is unavailable or impractical. Due to their brevity and ease of administration, the MoCA and DRS-2 are often used in clinics where there are limited financial resources or time restrictions(8) and in studies that include thousands of participants, such as consortium biomarker and genetic studies(9–11). Consequently, guidelines for interpreting the MoCA and DRS-2 subsections for domain-specific impairments would be of great benefit(12).

In prior work, we have shown that the MoCA is sensitive to global CI as well as executive, visuospatial, and memory domain-specific impairments in PD(12). Here, we expand on these findings and directly compare the sensitivity of the MoCA and DRS-2 for detecting global and domain-specific CI in two independent PD cohorts.

Methods

Subjects

A training cohort of 85 participants with idiopathic PD, as previously described(12)(Table1), was used to determine cutoff points that optimized sensitivity and specificity for global and domain-specific CI. A validation cohort of participants with PD from the Pacific Udall Center(PUC)(Table1) was used to test these cutoff points. PUC collected detailed clinical(13) and neuropsychological data from February 2010 until October 2015, as previously described(14). Of the 721 participants with cross-sectional cognitive data, 521 who completed the MoCA, DRS-2, and cognitive battery were included(Table1). Of the 200 participants excluded, 196 were not administered one or more tests and 4 were unable to complete a test due to CI. All participants provided written informed consent to participate in the study following protocols approved by the institutional review boards at each participating institution.

Table 1.

Methodology

	Training Cohort Demographics			Validation Cohort Demographics
	PD no-CI mean (SD) [Range]	PD CI mean (SD) [Range]	t-test or χ2	PD no-CI mean (SD) [Range]	PD CI mean (SD) [Range]	t-test or χ2	Cohort differences t-test or χ2
# participants	45	40		126	395		<0.001*
Gender (M/F)	25/20	28/12	0.170	59/67	299/96	<0.001*	0.244
Age (yrs)	64.4 (8.0)	70.0 (8.8)	0.003*	63.4 (8.7)	67.8 (8.6)	<0.001*	0.797
	[42–80]	[42–86]		[43–83]	[35–84]
Education (yrs)	16.9 (2.3)	16.6 (2.3)	0.632	16.2 (2.5)	15.6 (2.6)	0.041*	<0.001*
	[12–20]	[12–20]		[12–20]	[8–20]
Durations (yrs)	4.0 (3.4)	6.3 (4.8)	0.014*	7.4 (5.41)	8.8 (6.5)	0.143	<0.001*
	[0–16]	[0–22]		[0–26]	[0–33]
MDS-UPDRS-III Off	18.1 (9.1)	25.1 (12.0)	0.011*	21.5 (10.4)	28.2 (12.9)	<0.001*	<0.001*
	[16–55]	[11–62]		[3–56]	[3–68]
Hoehn and Yahr Off	1.9 (0.4)	2.4 (0.8)	<0.001*	2.1 (0.5)	2.4 (0.7)	<0.001*	0.028*
	[1–3]	[1–4]		[1–4]	[1–5]
BDI-II	9.2 (7.6)	11.9 (8.3)	0.137	N/A	N/A		N/A
	[1–30]	[0–40]
GDS	N/A	N/A		5.5 (1.3)	6.0 (1.7)	0.002*	N/A
				[2–10]	[2–13]
MoCA	27.5 (2.1)	21.5 (5.1)	<0.001*	27.4 (2.1)	23.7 (3.2)	<0.001*	0.902
	[23–30]	[8–30]		[22–30]	[10–30]
DRS-2	139.8 (2.9)	129.8 (13.3)	<0.001*	139.8 (2.9)	135.0 (7.0)	<0.001*	0.183
	[132–144]	[89–142]		[131–144]	[103–144]
Domains	Training Cohort Neuropsychological Battery			Validation Cohort Neuropsychological Battery			MoCA	DRS-2
Attention/Working Memory	Digit Span (forward and backward)			Letter Number Sequencing			Target Detection, Serial Sevens, Digit Span (forward and backwards)	Attention
Executive	Symbol Digit Modalities Test (written and oral), Controlled Word Association Test (FAS), Trail Making Test B, Golden Stroop			Digit Symbol, Controlled Word Association Test (FAS), Trail Making Test B			Trail Making Test B, Phonemic Fluency, Verbal Abstraction	Conceptualization, Initiation/Perseveration
Memory	California Verbal Learning Test-II, Brief Visuospatial Memory Test-Revised			Hopkins Verbal Learning Test-Revised			Delayed Recall	Memory
Language	Boston Naming Test, Semantic Fluency (animals)			Semantic Fluency (animals)			Naming, Sentence Repetition	--
Visuospatial	Hooper Visual Organization Test, Judgement of Line Orientation			Judgement of Line Orientation			Clock Draw, 3D Figure Copy	Construction

CI, cognitive impairment; SD, standard deviation; MDS-UPDRS-III Off, Movement Disorders-sponsored revision of the Unified Parkinson Disease Rating Scale, part III (motor exam) performed off dopaminergic medications (0–132); Hoehn and Yahr Off, Hoehn and Yahr scale off dopaminergic medications (0–5); BDI-II, Beck Depression Inventory-II (0–63); GDS, Geriatric Depression Scale (0–15); MoCA, Montreal Cognitive Assessment (0–30); DRS-2, Mattis Dementia Rating Scale-2 (0–144)

^*indicates p-values < 0.05

^#Cohort differences were determined as the proportion of subjects in the CI and no-CI groups compared between cohorts using a chi squared tests or the average difference between the two cohorts as a whole (CI and no-CI) using a t-test.

Measures

Using MDS-TF recommendations, the training cohort defined CI as scores ≥1.5 standard deviations below age-and education-matched normative values on at least two separate neuropsychological measures, regardless of domain(5). In the validation cohort CI was assigned at a clinical consensus conference and required evidence of subjective and observed cognitive decline(14).

In both cohorts PD with CI were further classified as having dementia if the CI was severe enough to interfere with daily activities(training cohort n=12, validation cohort n=79)(15). Additionally, the cognitively impaired groups were categorized by domain according to the tests with a score ≥1.5 standard deviations below age-and education-matched normative values(5). We divided the MoCA and DRS-2 into subsections according to published criteria(Table1)(3, 4, 16, 17).

Statistical Analysis

χ-squared and t-tests[Microsoft Excel and IBM SPSS Statistics version 23.0 (IBM Corp., Armonk, NY)] were used to examine demographic group differences.

We used logistic regression with global and domain-specific cognitive classification(PD with or without CI) as the dependent variable and MoCA or DRS-2 total or subsection score as predictor variables, each in separate models. The DRS-2 and MoCA were not used by either cohort when determining cognitive classification. We then used a receiver operating characteristic (ROC) in the training cohort to determine the cutoff point that optimized cognitive classification sensitivity and specificity. These optimal cutoff points were then applied to the validation cohort to determine the final sensitivity and specificity. For sensitivity and specificity, ≥80% was defined as high, 79%−60% as moderate, and <60% as low.

Results

Table 1 presents demographic data. In the training cohort, participants with CI were older, had longer disease duration, more severe motor score(18), and lower total MoCA and DRS-2 score.

In the validation cohort, participants with CI were older, more likely to be male, had fewer years of education, more severe motor score(18), higher self-reported depression severity(19), and had lower total MoCA and DRS-2 scores.

Global Impairment

Both the MoCA(p=0.003) and DRS-2(p=0.01) were unique predictors of CI in the training cohort. For the MoCA, a cutoff score of ≤26 provided 90.0% sensitivity and 71.1% specificity, which is consistent with previously published data(20, 21). For the DRS-2, a cutoff score of ≤138 provided 87.5% sensitivity and 68.9% specificity.

Cutoff scores established in the training cohort were used to test the validation cohort(Table2). The MoCA provided 80.3% sensitivity and 68.3% specificity. The DRS-2 provided 60.3% sensitivity and 73.0% specificity. An exploratory ROC analysis for the validation cohort indicate that similar cutoff scores would have been selected if this cohort had been used as the training cohort (Supplemental Table3).

Table 2.

ROC analyses

		Sensitivity	Specificity	Diagnostic Accuracy	Positive Predictive Value	Negative Predictive Value	AUC	Lower Bound	Upper Bound	S.E	p	Cutoff Point	Total Points per Section
Training Cohort
Total
	MoCA	90.0%	71.1%	78.8%	73.5%	88.9%	0.883	0.810	0.955	0.037	<0.001	≤ 26	30
	DRS-2	87.5%	68.9%	77.7%	71.4%	86.1%	0.847	0.765	0.929	0.042	<0.001	≤ 138	144
Executive
	MoCA	78.8%	82.7%	81.2%	74.3%	86.0%	0.832	0.736	0.928	0.049	<0.001	≤ 3	4
	DRS-2	84.8%	50.0%	63.5%	51.9%	83.9%	0.773	0.662	0.883	0.056	<0.001	≤ 74	76
Visuospatial
	MoCA	93.3%	45.7%	54.1%	26.9%	97.0%	0.807	0.686	0.929	0.062	<0.001	≤ 3	4
	DRS-2	13.3%	94.3%	80.0%	33.3%	83.5%	0.538	0.370	0.706	0.086	0.645	≤ 5	6
Memory
	MoCA	84.6%	56.5%	62.4%	59.0%	65.2%	0.747	0.642	0.852	0.054	<0.001	≤ 3	5
	DRS-2	79.5%	50.0%	63.5%	57.4%	74.2%	0.725	0.617	0.834	0.055	<0.001	≤ 23	25
Attention
	MoCA	55.6%	76.1%	70.6%	37.9%	85.7%	0.681	0.527	0.834	0.078	0.019	≤ 5	6
	DRS-2	66.7%	68.7%	56.5%	32.7%	100.0%	0.818	0.715	0.922	0.053	<0.001	≤ 35	37
Language
	MoCA	68.8%	71.0%	70.6%	35.5%	90.7%	0.740	0.589	0.892	0.077	0.003	≤ 4	5
Validation Cohort
Total
	MoCA	80.3%	68.3%	77.4%	88.8%	52.4%
	DRS-2	60.3%	73.0%	63.3%	87.5%	37.0%
Executive
	MoCA	61.2%	67.0%	64.5%	58.3%	69.6%
	DRS-2	86.2%	33.0%	55.9%	49.2%	76.0%
Visuospatial
	MoCA	87.8%	51.5%	54.3%	13.4%	98.0%
	DRS-2	26.8%	94.8%	89.4%	30.6%	93.8%
Memory
	MoCA	91.9%	31.5%	50.3%	29.2%	64.4%
	DRS-2	61.6%	62.5%	62.2%	44.7%	76.8%
Attention
	MoCA	65.9%	62.3%	61.6%	13.2%	95.5%
	DRS-2	34.1%	75.2%	43.8%	9.1%	93.9%
Language
	MoCA	43.8%	73.4%	69.3%	21.2%	88.9%

Results from the ROC analysis. The cutoff values were determined in the Training Cohort (Top) to optimize Sensitivity and Specificity. These cutoff values were then applied to the Validation Cohort (Bottom) to determine the final diagnostic Accuracy, Sensitivity, and Specificity. ROC, receiver operating characteristic; MoCA, Montreal Cognitive Assessment; DRS, Mattis Dementia Rating Scale- 2; AUC, area under the curve; SE, standard error

MoCA Subsections

In the training cohort, all MoCA subsections significantly predicted domain specific impairment(12). The visuospatial and memory subsections provided high sensitivity(93.3% and 84.6%, respectively) and low specificity(45.7% and 56.5%, respectively); this remained true in the validation cohort(Table2).

DRS-2 Subsections

In the training cohort, only the executive and attention subsections significantly predicted domain-specific impairment. The executive subsection provided high sensitivity(84.8%), and low specificity(50.0%); this remained true in the validation cohort(Table2).

Discussion

Both the MoCA and DRS-2 were shown to be sensitive screening tools for CI in PD. Comparing across cohorts, the MoCA maintained high sensitivity in both the training(90.0%) and validation(80.3%) cohorts using a cutoff of ≤26/30, whereas the DRS-2 showed more variable sensitivity(87.5% in training cohort and 60.3% in validation cohort) using a cutoff of ≤138/144. Within the validation cohort, the visuospatial and memory subsections of the MoCA remained highly sensitive(87.8% and 91.9%, respectively), whereas the executive subsection of the DRS-2 remained highly sensitive(86.2%) to impairments.

Identifying CI in PD

Both the MoCA and DRS-2 were significant predictors of CI in PD; however, only the MoCA maintained reasonable sensitivity for detecting global CI across cohorts. The MoCA’s sensitivity and specificity in both cohorts was similar to previously published literature(20, 21), suggesting broad generalizability of our findings. In contrast, the sensitivity of the DRS-2 dropped from 87.5% to 60.3% when the determined cutoff(≤138/144) was applied to the validation cohort. Interestingly, this cutoff was lower than those previously published in PD(≤139/144 and ≤140/144)(22, 23). While sensitivity is improved somewhat at higher cutoffs, the specificity approaches 50%(Supplemental Table2). Clinically this inconsistency is important because the DRS-2 is often used as a cognitive screening tool, particularly for deep brain stimulation assessments(24), and as a cognitive outcome measure for several PD clinical trials(25, 26). The instability of the DRS-2 draws into question its ability to successfully categorize PD participants as with or without CI.

Markers of PD Domain-Specific CI

These results suggest that the MoCA and DRS present different strengths in identifying domain-specific impairment. Within the MoCA, the visuospatial and memory subsections display high sensitivity in both cohorts; the DRS-2 executive subsection displayed high sensitivity in both cohorts. This information would be relevant for investigators who are interested in examining PD participants with these specific cognitive deficits. For example, functional neuroimaging studies may want to recruit participants who are highly likely to have particular CI of interest for study. Using only the MoCA or DRS-2 subsections as a screening tool, they can recruit a concentrated cohort of PD patients with visuospatial, memory, or executive functioning impairment without the time and expense of performing complete neuropsychological testing on all potential participants. Given the heterogeneity of PD CI, such screening could help reduce sample size and false negative results. Further, recent genetic studies have identified potential genotype-cognitive phenotype associations within these domains. For instance, heterozygous mutations in the glucocerebrosidase gene are associated with more severe visuospatial impairments(27). By contrast, the APOE ε4 allele is associated with lower episodic memory, even in PD patients without dementia(28). While extensive neuropsychological testing is preferred for validation of these findings, the expense and logistics of testing thousands to tens-of-thousands of patients for genetic studies is an extraordinary challenge.

Methodological Considerations

Neuropsychological measures and the identification of CI differed between cohorts. The Stanford battery included more neuropsychological tests than the PUC battery. Additionally, CI in the training cohort was categorized based entirely on neuropsychological outcomes, while the validation cohort relied on neuropsychological and clinical data presented during consensus case conference. The validation cohort had a higher proportion of participants with CI(Table1). This is likely because the validation cohort participants tended to be older, have had a longer disease duration, and were at a more advanced Hohen-Yahr stage, and all of these factors increase the risk of CI in PD(1, 29).

However, when the ROC analyses were rerun using just neuropsychological data without consensus information, the results remained the same; the MoCA displayed consistent sensitivity(81.6%) and the DRS-2 displayed more variable sensitivity(68.2%).

Neuropsychological tests are not always process-pure. In this study we categorized Trails B and Stroop in the executive domain, whereas in other studies we and others have categorized these tests as measures of attention/working memory(5, 12). We did not find that re-categorization of these tests as attention/working memory substantially changed the primary results.

Several MoCA and DRS-2 subsections are derivatives of common neuropsychological tests, however, there are notable differences in test scoring and length which may result in global assessments failing to capture cognitive fatigue(30).

Conclusions and Future Directions

Together these data demonstrate that although both tests can sufficiently screen for PD CI, the MoCA provides more consistently high sensitivity than the DRS-2. Further, the MoCA and DRS-2 present different strength in identifying domain-specific impairment. However, our data also suggests that use of the MoCA and DRS-2 in this way will likely lead to some patients falsely being identified as impaired, since the specificity for domain-specific impairments was low. We hope that similar analyses will be applied to longitudinal PD cohorts, so we can better understand the limitations and advantages of the MoCA and DRS-2 in the identification of PD domain-specific impairments over time. Further, efforts should be made to develop new cognitive screening tools that are both sensitive and specific for identifying domain-specific CI in PD.