Domain-Specific Accuracy of the Montreal Cognitive Assessment Subsections in Parkinson’s Disease

Abstract

Objective

The Montreal Cognitive Assessment (MoCA) is among the most widely adopted screening tools for cognitive impairment because it includes tests in multiple domains and is available in 55 languages. The MoCA is often the only formal cognitive assessment available when comprehensive neuropsychological testing is not practical, such as rural clinical settings or large retrospective and multi-lingual research settings. However, the MoCA domain-specific subsections have never been formally assessed for sensitivity or specificity. Therefore, in Parkinson’s disease, we examined whether the subsections of the MoCA could identify cognitive impairment within specific cognitive domains.

Methods

We administered a comprehensive neuropsychological battery to 85 Parkinson’s disease participants, who were then categorized as with or without cognitive impairment, with respect to global cognition and in five cognitive domains. We then assessed the domain-specific categorization of the MoCA subsections compared to the full neuropsychology battery.

Results

All MoCA subsections predicted impairment in their respective cognitive domain. However, the executive subsection showed the highest sensitivity and specificity (89.3% and 82.5%, respectively), followed by visuospatial (93.3% and 45.7%, respectively) and memory (84.6% and 56.5%, respectively).

Conclusion

The MoCA is a useful screening tool for PD global cognitive and executive functions. The MoCA is also highly sensitive to visuospatial and memory impairment, but with limited specificity and accuracy these subsections should be interpreted with caution.

Introduction

Global cognitive assessments are often used as screening tools to detect cognitive impairment in Parkinson’s disease (PD) patients [1]. These tools are widely used in both the clinical and research settings and impairment on a scale of global cognitive abilities is the minimum requirement for level I criteria of mild cognitive impairment in PD [2]. The Montreal Cognitive Assessment (MoCA) is a 10-minute screening tool that has been validated in detecting cognitive impairment in PD and several studies have shown that a score less than 26 (out of 30) has 90% sensitivity and 53% to 75% specificity in detecting global PD cognitive impairment [3,4]. The high sensitivity to global impairment is in part because, unlike many other global cognitive assessments, the MoCA includes tests in five domains, including attention/working memory, executive function, episodic memory, language, and visuospatial [5]. Therefore, the MoCA is particularly useful in PD because these patients typically develop heterogeneous cognitive impairments [6]. For instance, studies that include a full neuropsychological battery have found multi-domain cognitive impairment, rather than single-domain cognitive impairment, is most common and is present in up to 35% of newly diagnosed PD patients [7,8].

Although the sensitivity and specificity of the MoCA has been established for identifying global cognitive impairment in PD, it is unclear how sensitive and specific the MoCA is at identifying impairment within individual cognitive domains. Currently, the MoCA is only used for level I diagnosis and not domain-specific diagnosis [2]. However, many clinicians and researchers do not have access to the recommended full neuropsychological battery, and therefore would benefit from guidelines on how to interpret the MoCA subsections when this is the only test available. For instance, the MoCA is likely to be the only cognitive data available for retrospective PD studies reliant on chart review. Further, one advantage of the MoCA over most other cognitive tests is that it is available in 55 different languages and could therefore be used in large multicultural research settings. However, without knowing the applicability of the MoCA subsections to domain-specific impairments these studies are all limited to simple general cognitive outcomes. With this in mind, we studied the accuracy, sensitivity, and specificity of the MoCA to predict domain-specific PD cognitive impairments.

Subjects and Methods

Subjects

We recruited 85 participants with idiopathic PD from the Stanford Movement Disorders clinic, the surrounding area, and Fox Trial Finder from March 2012 until September 2014 (Table 1). Participants were included if they had a diagnosis of PD based on UK Parkinson’s Disease Society Brain Bank criteria [9], were right handed, were fluent in English, and had no history of other significant neurological disease, bipolar disorder, schizophrenia, current substance abuse, or severe head trauma. All participants provided written informed consent to participate in the study following protocols approved by the Stanford Institutional Review Board.

Table 1.

Demographics of subjects (mean, standard deviation, range)

Group	Gender (M/F)	Age (yrs)	Education (yrs)	Duration (yrs)	MDS-UPDRS III OFF	Hoehn & Yahr	BDI-II	MoCA	LEDD
PD without cognitive impairment (n=45)	25/20	64.4 (8.0) 46 – 80	16.9 (2.3) 12 – 20	4.0 (3.4) 0 – 16	32.8 (10.8) 16 – 55	1.9 (0.4) 1 – 3	9.20 (7.64) 1 – 30	27.5 (2.1) 23 – 30	502.9 (339.1) 0 – 1580
PD with cognitive impairment (n=40)	28/12	70.0 (8.8) 42 – 86	16.6 (2.3) 12 – 20	6.3 (4.8) 0 – 22	38.2 (12.3) 11 – 62	2.4 (0.8) 1 – 4	11.89 (8.29) 0 – 40	21.5 (5.1) 8 – 30	479.3 (512.3) 0 – 1808
P value	0.170	0.003	0.632	0.014	0.035	<0.001	0.137	<0.001	0.806

MDS-UPDRS, Movement Disorders Society-sponsored revision of the Unified Parkinson’s Disease Rating Scale; BDI-II, Beck Depression Inventory-II; MoCA, Montreal Cognitive Assessment; LEDD, levodopa equivalence dose; Duration, duration since diagnosis; P, probability value

Measures

Each participant underwent a clinical evaluation that included a medical history, a physical examination, a Movement Disorders Society-sponsored revision of the Unified Parkinson’s Disease Rating Scale including Hoehn and Yahr stage [10], a depression screening (Beck Depression Inventory-II) [11], and comprehensive neuropsychological evaluation, including the MoCA. The neuropsychological battery consisted of at least 2 tests in five cognitive domains: (1) Attention/Working Memory: Trail Making Test A&B [12], Wechsler Memory Scale – Third Edition Digit Span (forward and backwards) [13], and the Golden Stroop [14]; (2) Executive Function: Symbol Digit Modalities Test (Written and Verbal) [15] and Controlled Word Association Test (FAS) [16]; (3) Episodic Memory: California Verbal Learning Test-II [17] and Brief Visuospatial Memory Test-Revised [18]; (4) Language: Boston Naming Test – Short Form A [19], Semantic Fluency (animals) [16], Delis-Kaplan Executive Function System Boys Names and Category Switching (Fruits and Furniture) [20]; (5) Visuospatial: Hooper Visual Organization Test [21] and Judgement of Line Orientation – Long Form V [22]. Using published criteria [2], participants were categorized as PD without cognitive impairment (n=45) or PD with cognitive impairment (n=40). Cognitive impairment was defined by scores ≥ 1.5 standard deviations below age and education matched normative values on at least two tests. The cognitively impaired group was further classified by subtype (attention/working memory, executive, episodic memory, language, visuospatial) according to the tests within each domain with a score ≥ 1.5 standard deviations below age and education matched normative values [2].

We examined the subsections of the MoCA according to published criteria [5]: (1) Attention/Working Memory included target detection task, serial sevens, and digit forward and backward; (2) Executive Function included Trails B task, phonemic fluency, and verbal abstraction; (3) Episodic Memory included recall task; (4) Language included a naming task and sentence repetition; and (5) Visuospatial included clock drawing and three-dimensional figure copy.

Statistical Analysis

All analyses were performed using IBM SPSS Statistics version 23.0 (IBM Corp., Armonk, NY). Using logistic regression, we first determined if MoCA subsection scores predicted whether a participant has impairment in the corresponding cognitive domain on the comprehensive neuropsychological evaluation. Using the receiver operating characteristic (ROC) curve, we then determined the optimal cutoff for diagnostic accuracy, sensitivity, and specificity of each subsection.

Results

Table 1 summarizes the demographic data from our cohort. PD with cognitive impairment were older with longer disease duration, more severe Movement Disorders Society-sponsored revisions of the Unified Parkinson’s Disease Rating Scale motor score, more severe Hoehn and Yahr stage, and more impaired MoCA total score. The groups did not differ on gender, years of education, severity of self-reported depression, or levodopa equivalence dose [23].

Using logistic regression with ROC analysis we tested the predictive ability of the MoCA for determining if participants have global cognitive impairment and impairment in the five cognitive domains (Table 2). For global cognitive impairment we found a cutoff of less than or equal to 26 provided 78.8% diagnostic accuracy with 90.0% sensitivity and 71.1% specificity, which is similar to previously published data [3,4], and with 88.9% positive predictive value (PPV), and 73.5% negative predictive value (NPV). Further, we found all MoCA subsections significantly predict cognitive impairment in the respective domain, however with varying sensitivity and specificity (Table 2). The executive subsection had the highest diagnostic accuracy (84.71%) with good sensitivity (89.3%), specificity (82.5%), and PPV (94.0%), but lower NPV (71.4%). The attention and language subsections had moderately lower accuracy (74.1% and 70.6%, respectively) with poor sensitivity (59.1% and 68.8%, respectively) and moderate specificity (79.4% and 71.0%, respectively). Both showed high PPV (84.2% and 90.7%, respectively), but poor NPV (53.6% and 64.5%, respectively). The memory and visuospatial subsections had the lowest accuracy (62.35% and 54.12%, respectively) but with good sensitivity (84.6% and 93.3%, respectively) and poor specificity (56.5% and 45.7%, respectively). Additionally, the memory subsection had poor PPV and NPV (62.9% and 48.0%, respectively), while the visuospatial subsection had relatively good PPV and NPV (96.9% and 73.6%, respectively).

Table 2.

ROC analysis of MoCA subsections

	AUC	Lower Bound	Upper Bound	S.E	P	Cutoff Point	Total Points per Section	Sensitivity	Specificity
Global	0.883	0.810	0.955	0.037	<.001	≤ 26	30	90.0%	71.1%
Executive	0.889	0.821	0.970	0.041	<.001	≤ 3	4	89.3%	82.5%
Visuospatial	0.807	0.686	0.929	0.062	<.001	≤ 3	4	93.3%	45.7%
Memory	0.747	0.642	0.852	0.054	<.001	≤ 4	5	84.6%	56.5%
Language	0.710	0.563	0.857	0.075	0.009	≤ 4	5	68.8%	71.0%
Attention	0.707	0.570	0.844	0.070	0.004	≤ 5	6	59.1%	79.4%

ROC, receiver operating characteristic; MoCA, Montreal Cognitive Assessment; AUC, area under the curve; SE, standard error; P, probability value

Education level is the only suggested correction for the MoCA. Our two groups did not differ in years of education and did not include any individuals with less than 12 years of education; therefore, we did not adjust our initial analysis for any clinical metrics. Because our PD participants with cognitive impairment were older than those without cognitive impairment we repeated the logistic regression controlling for age. In this age-adjusted model there was no significant change in accuracy, sensitivity, or specificity of the MoCA subsection scores compared to the original model.

Discussion

Examining the domain specific properties of the MoCA highlights the test’s strengths and limitations and is crucial in guiding interpretation. The MoCA is of particular interest in PD because it is a commonly used screening test for global cognitive impairment and includes tests within all five cognitive domains pertinent in PD. Our results for these domains reveal a wide range of sensitivity and specificity among the MoCA subsections, demonstrating that the MoCA is better at identifying certain domain-specific impairments than others. We found the executive function subsection of the MoCA was both sensitive and specific in detecting impairment when compared to traditional cognitive tests of executive functioning. The MoCA was sensitive in identifying visuospatial and memory impairments, but was not specific and therefore should be used with caution when diagnosing these subtypes. Our results also indicated that the MoCA is not particularly sensitive or specific in identifying attention or language impairments. Therefore, we do not recommend the attention or language subsections be used for domain-specific screening.

Due to its sensitivity and specificity, one practical use of the MoCA executive subsection is as a screening tool for research aimed at studying this specific domain in PD. A total MoCA score less than 26 is often used as an inclusion criterion when recruiting patients in studies targeting PD cognitive impairment; however, this can lead to a highly heterogeneous cohort with patients who have non-overlapping domain impairments. Instead, researchers can use a domain-specific cut-off, and the executive subsection cut-off in particular, to recruit more selectively and identify a complete cohort of executively impaired patients at inclusion.

To our knowledge, this is the first study to examine the diagnostic accuracy of the MoCA subsections with regard to specific cognitive domains in PD. This is critical since the MoCA is one of the recommended tests for level I diagnosis of PD with mild cognitive impairment [2], which is used when more comprehensive testing is not practical or available. Unfortunately, this is a common situation for many clinicians in more rural areas without access to a formally trained neuropsychologist, who can therefore only perform cognitive testing that requires minimal training. In addition, the MoCA is often the only cognitive assessment available in larger retrospective or chart review research studies. The MoCA can also be uniquely used in studies targeting under-served populations where participants might speak languages where traditional cognitive tests are not available. Indeed, while the MoCA subsections were never intended to be individually reviewed in the same way as the domains of a comprehensive neuropsychological battery, our results are the first to provide instructive guidelines for physicians and researchers in these important settings.

Our data also suggest a limitation of the MoCA subsections. The optimal cutoff for maximum sensitivity in each domain (including executive) was only loss of one point, suggesting a likely ceiling effect. Also, participant age did not change the results of our analysis, which could also be secondary to a ceiling effect. Although several of the MoCA subsection tests overlap with common tests used in neuropsychological batteries (for example, Trail Making Test B, digit span, and phonemic fluency), there are critical differences in test scoring and length that potentially influence the subdomain categorization. For instance, in the full length Trail Making Test we consider both the number sequencing task and the alternating number-letter sequencing task (1-A-2-B-3-C…) (i.e. versions A and B), as well as the time to complete the tasks. Thus, the full test includes variables specific to attention/working memory; by contrast, the Trail Making Test on the MoCA is shorter, untimed, and only includes the alternating number-letter sequencing task. Similarly, the MoCA’s digit span task is much shorter than the full digit span task used in the neuropsychological battery. The MoCA tests phonemic fluency as an aspect of executive function using the letter B, while the neuropsychological test includes multiple letters (F, A, and S). Finally, because the entire MoCA is substantially shorter than a full neuropsychological battery it is possible that the MoCA is not capturing cognitive fatigue seen in PD [24]. By contrast, the lengthiness of neuropsychological testing can be extremely stressful for some participants and in these settings our data suggest that the shorter iterations on the MoCA could be used in lieu of longer batteries.

Together, our data demonstrate that the MoCA subsections have high sensitivity and can therefore be used to screen for domain-specific executive, visuospatial, and memory impairments, particularly when more extensive testing is not available. However, our data also demonstrate that in the ideal setting the MoCA should be followed up with comprehensive neuropsychological testing to confirm the domain-specific impairments. It is critical to remember that the MoCA is a screening tool and cannot not be used to formally diagnose specific cognitive impairments and to effectively inform treatment. Further, these results from our modest cohort of PD participants warrant replication in a larger, independent cohort. Because the MoCA is available in 55 different languages, understanding the predictive metrics of the MoCA subsections will allow for cross-cultural meta-analyses and for direct comparisons among a wide variety of cohorts.

Highlights.

• The MoCA is sensitive and specific to executive function impairment in PD.

• The MoCA is sensitive, but not specific, for visuospatial and episodic memory impairment in PD.

• The MoCA is limited in its ability to detect domain-specific impairment.