Montreal Cognitive Assessment as Screening Measure for Mild and Major Neurocognitive Disorder in a Chilean Population

Abstract

Background

The Montreal Cognitive Assessment (MoCA) is a sensitive screening instrument for mild neurocognitive disorder (mild NCD). However, cut-off scores and accuracy indices should be established using representative samples of the population. In this context, the aim of this study was to update the normative values, and diagnostic efficiency statistics of the MoCA to detect mild NCD in the Chilean population.

Methods

This study included 226 participants from the north, center, and south of the country, classified into 3 groups: healthy elderly (HE; n = 113), mild NCD (n = 65), and major neurocognitive disorder (major NCD; n = 48).

Results

The optimal cut-off score to discriminate mild NCD from HE participants was 20 points with a sensitivity of 82.8% and a specificity of 84.1%. The observed balance between sensitivity and specificity shows a good test performance either to confirm or discard a diagnosis. The cut-off between mild NCD and major NCD from HE participants was 19 points with 85.6% of sensitivity and 90.3% of specificity.

Conclusion

Overall diagnostic accuracy can be considered as outstanding (AUC ≥0.904) when discriminating HE from both mild NCD and major NCD. These results showed that the MoCA is a suitable tool to identify mild NCD and major NCD.

Introduction

Several studies show that performance in different cognitive domains such as memory, language, or executive functions decline with age [1, 2, 3]. When changes in cognition become pathological, progressive decrements in performance are manifested in one or more cognitive domains. The fifth version of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) [4] defines specific criteria for both mild neurocognitive disorder (mild NCD) and major neurocognitive disorder (major NCD) diagnosis, which are in agreement with the diagnostic guidelines for mild cognitive impairment and Alzheimer's disease provided by the National Institute on Aging and Alzheimer's Association [5, 6].

Clinical evidence indicates that approximately 35% of amnestic mild NCD cases evolve to major NCD after 2 years of onset [7]. Furthermore, Petersen et al. [8] consider mild NCD as an intermediate stage of cognitive impairment that is often, but not always, a transitional phase from mild to major NCD. However, it has also been described that cognitive decline observed in major NCD is significantly higher than that described for mild NCD, and it is also accompanied by an important deterioration of daily life activities [7]. Thus, the clinical diagnosis of mild or major NCD is established mainly based on the person's performance on different cognitive domains, and its functionality in everyday activities [8]. Additionally, the existence of metabolic and physiological biomarkers, along with brain imaging techniques, enrich the diagnostic process, especially considering the multiple etiologies underlying neurodegenerative disorders (e.g., Alzheimer disease, Parkinson disease, or frontotemporal lobar degeneration, among others).

A global cognitive screening should evaluate multiple domains, such as executive functions, orientation, attention, memory, visuospatial abilities, and language. On the one hand, the Mini-Mental State Examination [9] is the most used screening measure to detect cognitive impairment and evaluates the previously mentioned cognitive domains, but executive functions [10]. On the other hand, the Montreal Cognitive Assessment (MoCA) [11] was designed to identify patients with mild NCD and, therefore, it incorporates a wider extent of cognitive domains, particularly on executive functions and memory [12].

Quiroga et al. [13] obtained normative values of the MMSE for the Chilean population, establishing a cut-off score of 21/22 to detect major NCD with a sensitivity of 93.6% and a specificity of 46.1%. However, a recent meta-analysis provided by Ciesielska et al. [12] suggest that the MMSE is not a suitable screening measure to detect mild NCD, mainly because of its low specificity. Following the SPin criterion (i.e., for a positive test result, a very high specificity rules in the diagnosis; [14]), the low specificity associated with the cut-off score of the Chilean version of the MMSE [13] may lead to high rates of false negative results. Regarding the MoCA, Delgado et al. [15] provided normative values, and diagnostic efficiency statistics reporting a cut-off score of 20 to discriminate amnestic mild NCD patients from healthy elderly (HE) participants, with a sensitivity of 75% and a specificity of 82%. Despite the good efficiency statistics reported by Delgado et al. [15], their study only considered a sample of individuals from Santiago de Chile (i.e., HE, mild and major NCD), which might affect the interpretation of results in other cities where prevalence rates are different from those observed in Santiago. Therefore, the present study broadens the sample to 3 country areas in Chile (i.e., North, Center, and South).

In 2015, the Alzheimer's Disease International reported that about 46.8 million people worldwide live with some kind of major NCD, with an expected increment of over 131 million by 2050 [16]. Therefore, efforts should be oriented at improving the existing screening tools that contribute to early mild NCD diagnosis, which has been defined as a priority public health issue by the World Health Association [17]. Following Patterson [18], both cognitive and pharmacological interventions are more effective at early stages of mild NCD; hence, the present study also aims at updating normative values and diagnostic efficiency statistics to detect mild NCD in the Chilean population, which may complement the report by Delgado et al. [15].

Material and Methods

Participants

The sample incorporated 226 participants without neurological conditions that can affect cognitive performance (e.g., Parkinson disease, cerebral palsy, brain tumors, stroke, TBI, etc.) The sample was stratified into 3 groups (N_HE = 113, M_age = 71.49, SD_age = 7.64; N_{mild NCD} = 65, M_age = 76.92, SD_age = 8.71; N_{major NCD} = 48, M_age = 82.17, SD_age = 7.89). Participants of the HE group were recruited at older community dwellings from SENAMA (i.e., National Service for Older Persons) from the cities of Iquique (north), Valparaiso (center) and Concepcion (south), while mild and major NCD groups were recruited in several neurology units of public and private hospitals from the same cities. All participants were assessed with the modified MMSE Chilean version and an interdisciplinary assessment, which included neuropsychologists, neurologists, and psychiatrists. The mild NCD group included only participants with self-reported subjective cognitive complaints, cognitive changes observed by his or her family, clinicians, and/or available evidence of objective deficits on one or more cognitive domains, in absence of functional impairment, and independence in daily life activities. The major NCD group included patients with a confirmed diagnosis, meeting the Clinical Dementia Rating Scale for Mild Dementia (score = 1). All participants were able to approve their participation in the study by signing an informed consent, previously evaluated by the Ethics Committee of the Universidad de Valparaíso. The present study was also carried out in compliance with the code of ethics of the World Medical Association Declaration of Helsinki [17, 19].

Measures

MoCA, Spanish Version

The MoCA [11] is a 10-min cognitive screening tool developed to assist clinicians in the detection of mild NCD. It consists of 7 scales targeting different cognitive domains: (a) Visuospatial-executive, (b) Naming, (c) Memory (i.e., delayed recall), (d) Attention, (e) Language, (f) Abstraction, and (g) Orientation. The MoCA has a maximum total score of 30 points. A meta-analysis conducted by Loureiro et al. [20] describes different validation studies of the MoCA in Latin American Spanish-speaking populations. Delgado et al. [15] validation study included a total of 172 participants (HE, mild NCD, and major NCD) from Santiago, Chile; providing good diagnostic efficiency indices (i.e., sensitivity, specificity). Overall diagnostic accuracy and cut-off points were obtained using ROC curves. The MoCA cut-off score for amnestic mild NCD was 20, with 75% sensitivity, and 82% specificity. The MoCA cut-off for subtle major NCD was 19, with 90% sensitivity and 86% specificity. Copyright and permissions were obtained from the original authors of the MoCA [11].

Modified MMSE, Chilean Version

The indicators of global cognitive performance were obtained using the modified MMSE Chilean version as gold standard [13]. The modification to the MMSE was made in 2 items: (a) replacement of the inverse spelling of the word “WORLD” (i.e., MUNDO) by the inverse repetition of a sequential 5-digit number, and (b) introduction of a second alternative in the item “draw 2 pentagons,” consisting of drawing 2 circles, counting the best of both answers on the score. The total score for the MMSE was 30 points, as in the original version. The validation study included a sample of 94 subjects over 65 years old (n = 76 HE; n = 18 patients with confirmed major NCD diagnosis). Overall diagnostic accuracy and cut-off points were estimated using ROC curves. The cut-off score for amnestic mild NCD was 21/22, with 94% sensitivity and 46% specificity.

Statistical Analyses

Descriptives

First analysis described the demographic characteristics of all participants in the sample, according to their belonging group (i.e., HE, mild NCD, major NCD). Mean and SDs were obtained for age, whereas frequencies and percentages were provided for sex, years of formal education, and country area. Comparisons between conditions were also conducted. A one-way ANOVA was performed for continuous variables (i.e., age), whereas χ² analyses were carried out for categorical variables (i.e., sex, education level, country area). Likewise, means and SDs of participants' performance in the MoCA total score and its cognitive domains were estimated for each group, separated by educational level (i.e., 1–12, and >12 years of education). Additionally, comparisons between average group performances were conducted. One-way ANOVA analyses were performed for all variables, reporting the respective statistics, p values, effect sizes, and post hoc comparisons.

Psychometric Properties

Criterion validity for the MoCA was estimated through the Pearson's correlation coefficient with the modified Chilean version of the MMSE [13] used as the gold standard. Reliability was estimated through the internal consistency analysis of the MoCA obtaining the Cronbach's α coefficient [21].

Diagnostic Efficiency Statistics

Indices of sensitivity and specificity were obtained for both the MoCA and the MMSE, contrasting 3 groups: (a) mild NCD versus HE, (b) mild NCD and major NCD versus HE, and (c) major NCD versus HE. Confidence intervals (95%) for both sensitivity and specificity were provided. Overall diagnostic efficiency statistics for each screening measure were estimated by calculating the area under the receiver operating characteristic curve (ROC curve). ROC plots were included to visualize and compare the overall diagnostic accuracy between both screening measures. Optimal cut-off scores were estimated for both the MoCA and MMSE, using the Youden index as a selection criterion [22]. Finally, predictive values for both the MoCA and MMSE are reported (i.e., positive and negative predictive value; PPV and NPV). However, as broadly known, both PPVs and NPVs are considerably affected by prevalence rates [23]. Therefore, we provided predictive values for the MoCA and the MMSE assuming different theoretical prevalence rates (i.e., 1, 5, 10, 25, and 50%) in order to provide a more comprehensive spectrum of the tests performance.

All descriptive, inferential, and psychometric analyses were conducted using JASP [24] and Jamovi [25]. ROC curves, AUC estimations, and optimal cut-off scores were obtained using easyROC, an online web tool for ROC curve analysis [26].

Results

Demographic characteristics of the sample are shown in Table 1. Likewise, Table 2 presents group's average performance for the MoCA total score, and its associated cognitive domains, grouped by participant's educational level. Additionally, both Tables 1 and 2 report comparisons between groups, including statistics and p values. In both MoCA total scores, and its cognitive domains, significant differences were observed between HE, mild NCD, and major NCD, for both educational levels (Table 2). Post hoc comparisons reveal that the HE group had a higher average performance than both mild and major NCD groups, in both MoCA total score and its cognitive domains. A similar pattern was observed for the comparison between the mild versus the major NCD group.

Table 1.

Sample demographics and comparisons between conditions

Parameter	Descriptives			Comparisons between conditions
	HE (n = 113)	mild NCD (n = 65)	major NCD (n = 48)	statistic	sig.
Age, years	71.49±7.64	76.92±8.71	82.17±7.89	F = 31.77 (2, 222)	<0.01*
Sex
Male  Female	27.4 (31) 72.6 (82)	27.7 (18) 72.3 (47)	35.4 (17) 64.6 (31)	χ2 = 1.14 (2)	0.57
Education
1–12 years	46.9 (53)	67.7 (44)	83.3 (40)	χ2 = 20.64 (2)	<0.01*
>12 years	53.1 (60)	32.3 (21)	16.7 (8)
Country zone
North  Center  South	28.3 (32) 27.4 (31) 44.2 (50)	34.4 (22) 14.1 (9) 51.6 (33)	72.90 (35) 6.3 (3) 20.8 (10)	χ2 = 33.4 (4)	<0.01*

Values are presented as mean ± SD or % (n). HE, healthy elderly.

^*α = 0.01 level.

Table 2.

MoCA descriptives for total score and cognitive domains by condition and years of education

Parameter	Descriptives			Comparisons between conditions
	HE	mild NCD	major NCD	F	sig.	ω2
Education 1–12 years	n = 53	n = 44	n = 39
MoCA total score	22.79±3.73	17.2±3.11	6.82±4.55	159.4 (2, 133)	<0.01*	0.74
Visuospatial executive	3.43±1.17	2.39±1.37	0.78±0.92	58.7 (2, 134)	<0.01*	0.46
Naming	2.78±0.59	2.48±0.70	1.40±1.17	32.4 (2, 134)	<0.01*	0.31
Attention	4.34±1.14	2.86±1.41	1.25±1.45	62.3 (2, 134)	<0.01*	0.47
Language	1.74±1.04	1.14±1.00	0.38±0.59	25.1 (2, 134)	<0.01*	0.26
Abstraction	1.47±0.72	0.80±0.82	0.33±0.66	28.4 (2, 134)	<0.01*	0.29
Memory (delayed recall)	2.21±1.55	1.16±1.36	0.08±0.35	33.3 (2, 134)	<0.01*	0.32
Orientation	5.85±0.41	5.36±0.99	1.73±1.91	151.8 (2, 134)	<0.01*	0.69
Education >12 years	n = 60	n = 20	n = 8
MoCA total score	24.68±2.68	17.2±5.11	10.13±6.11	68.2 (2, 86)	<0.01*	0.60
Visuospatial executive	3.97±1.03	2.86±1.32	1.75±1.17	18.8 (2, 86)	<0.01*	0.29
Naming	2.95±0.22	2.43±0.81	1.75±1.17	21.1 (2, 86)	<0.01*	0.31
Attention	5.15±0.90	4.10±1.48	2.63±1.99	19.5 (2, 86)	<0.01*	0.29
Language	2.17±0.89	1.67±1.02	0.88±0.84	8.2 (2, 86)	<0.01*	0.14
Abstraction	1.70±0.56	1.24±0.83	0.88±0.84	8.0 (2, 86)	<0.01*	0.14
Memory (delayed recall)	2.87±1.60	0.76±1.04	0.00±0.00	26.9 (2, 86)	<0.01*	0.37
Orientation	5.88±0.32	4.81±1.69	2.25±1.98	47.2 (2, 86)	<0.01*	0.51

HE, healthy elderly.

^*α = 0.01 level.

Criterion validity for the MoCA was estimated through its correlation with the MMSE. Since the univariate normality assumption was not met for both data distributions, we obtained its non-parametric approximation. Spearman's rho coefficient (rho = 0.798, p < 0.001) suggests a good association between both screening measures, and accounts for the concurrent validity of the MoCA using the MMSE as the gold standard. Regarding reliability, the MoCA showed a good internal consistency with Cronbach's coefficient α = 0.749.

The overall diagnostic accuracy of the MoCA for accurately discriminating between mild NCD patients and HE participants is considered outstanding (AUC value), according Hosmer and Lemeshow's [27] criteria, with an area under the ROC curve of 0.904. The MMSE AUC for discriminating mild NCD patients from HE participants was 0.839, which can be considered excellent following the same criteria [28]. Both MoCA and MMSE AUC values can be also defined as outstanding to differentiate mild NCD and major NCD patients from HE, and also between major NCD patients and HE participants (Table 3).

Table 3.

Diagnostic efficiency statistics for MoCA and MMSE screening tests

Screening	AUC	Cut-offa	Sensitivity	Specificity	Youden index	Sensitivity 95% CI	Specificity 95% CI
Mild NCD vs. HE
MoCA	0.904	20	0.828	0.841	0.669	0.713–0.911	0.760–0.903
MMSE	0.839	26	0.781	0.788	0.569	0.660–0.875	0.701–0.859
Mild and major NCD vs. HE
MoCA	0.942	19	0.856	0.903	0.759	0.776–0.915	0.832–0.950
MMSE	0.905	25	0.821	0.858	0.679	0.738–0.887	0.780–0.917
Major NCD vs. HE
MoCA	0.994	17	0.979	0.938	0.917	0.887–0.999	0.877–0.975
MMSE	0.994	23	0.979	0.973	0.952	0.889–0.999	0.924–0.994

HE, healthy elderly; AUC, area under the curve.

^aCut-off method = Youden index.

The optimal cut-off score to discriminate between mild NCD patients and HE participants was 20, with a sensitivity of 82.8% and a specificity of 84.1% (Table 3). Both sensitivity and specificity of the MoCA showed to be higher than those observed for the MMSE when discriminating patients with mild NCD from HE participants. The MoCA optimal cut-off score to differentiate both patients with mild NCD and major NCD from HE participants was 19, with 85.6 and 90.3% sensitivity and specificity, respectively. The mentioned indices were higher for the MoCA in comparison to the MMSE (Table 3). Finally, when differentiating major NCD patients from HE participants, both the MoCA and the MMSE showed equal sensitivity levels. However, the specificity of the MMSE was higher than that observed for the MoCA. The cut-off scores for both screening measures are presented in Table 3. Despite the latter, both screening measures showed the same overall diagnostic accuracy AUC = 0.994 (Table 3). Figure 1 shows the comparison between the ROC curves of the MoCA and the MMSE, for each patient versus HE group's comparison.

Fig. 1.

Comparison between the ROC curves of the MoCA and MMSE.

Finally, Table 4 provides PPV and NPV for both MoCA and MMSE assuming different theoretical prevalence rates in order to provide a more comprehensive spectrum of the test performance.

Table 4.

Predictive values for MoCA and MMSE assuming different theoretical prevalence rates

	Prevalence rates
	1%		5%		10%		20%		40%		50%
	PPV	NPV	PPV	NPV	PPV	NPV	PPV	NPV	PPV	NPV	PPV	NPV
Mild NCD vs. HE
MoCA  MMSE	0.050 0.036	0.998 0.997	0.215 0.162	0.989 0.986	0.367 0.290	0.978 0.970	0.566 0.479	0.951 0.935	0.776 0.711	0.880 0.844	0.839 0.787	0.830 0.783
Mild and major NCD vs. HE
MoCA  MMSE	0.082 0.055	0.998 0.998	0.317 0.233	0.992 0.989	0.495 0.391	0.983 0.977	0.688 0.591	0.962 0.950	0.855 0.794	0.904 0.878	0.898 0.853	0.863 0.827
Major NCD vs. HE
MoCA  MMSE	0.138 0.268	0.999 0.999	0.454 0.656	0.999 0.999	0.637 0.801	0.998 0.998	0.798 0.901	0.994 0.994	0.913 0.960	0.986 0.986	0.940 0.973	0.979 0.979

HE, healthy elderly; PPV, positive predictive value; NPV, negative predictive value.

Discussion

The present study examined the diagnostic efficiency indices of the MoCA in a Chilean population. Main results showed that the MoCA is a suitable tool to identify mild NCD and major NCD. On the one hand, indices of sensitivity and specificity for the MoCA were higher than those observed for the MMSE. On the other hand, our results are in agreement with those reported by Delgado et al. [15 when discriminating HE from patients with amnestic mild NCD. In particular, sensitivity and specificity indices obtained for the MoCA are higher than those reported by Delgado et al. [15], considering the same cut-off score. Our results reveal that the MoCA has a specificity level (84.1%) which is slightly higher than sensitivity (82.8%). According to the SPin criterion [14], this constitutes a desirable condition when the purpose of the screening application is to confirm a diagnosis [24]. The latter may have potential benefits for clinical practice. For instance, Patterson [18] highlighted the positive impact of early interventions on mild NCD and major NCD, which emphasizes the importance of early detection and diagnosis. Accordingly, the World Health Organization [17] emphasized the necessity of improving screening tools that contribute to early mild NCD diagnosis, which has been considered as a priority issue in public health.

The overall diagnostic accuracy of the MoCA (AUC value) can be considered outstanding [27], and it is also higher than the observed for the MMSE to differentiate both HE from mild NCD, and HE from mild and major NCD. However, to differentiate major NCD patients from HE, the specificity of the MMSE was slightly higher than that observed for the MoCA. However, the overall diagnostic accuracy of both screening measures was identical.

Our results are in line with studies conducted in different contexts, such as rehabilitation settings [28], cognitive decline in different neurodegenerative conditions [29, 30], neuropsychiatric disorders [31], longitudinal studies [32], and validation studies [12, 33, 34]. The mentioned studies highlight the diagnostic accuracy of the MoCA for early mild NCD detection. Validation studies in Spanish-speaking populations have been performed [15, 35, 36]; Pedraza et al. [37] reported sensitivity levels ranging from 72 to 80% to detect mild NCD, and sensitivity levels ranging from 64 to 82% to detect mild NCD. In all mentioned studies, specificity was lower than sensitivity, which may affect early detection, especially when population base rates are low [24]. Despite the latter, the same studies reported higher diagnostic efficiency indices for the MoCA, compared to those observed for the MMSE. The present study corroborated the findings of the mentioned studies.

Of particular interest is the balance between sensitivity and specificity observed in our study, which allows to obtain a good test performance either to confirm or discard a particular diagnosis. It is also important to remind that predictive values for positive results (PPV) tend to decrease when prevalence rates are particularly low, thus underestimating positive diagnosis. Therefore, results should be always interpreted cautiously, and it is highly recommended to consider local prevalence rates when diagnosing the presence or absence of a condition. In sum, the present study contributes to updating normative values for the described populations, for both total and domain-related MoCA scores for the Chilean population. Likewise, it also provides outstanding overall diagnostic accuracy indices to detect both mild NCD and major NCD, and optimal and reliable cut-off scores for early detection of mild NCD.

Statement of Ethics

All participants were able to approve their participation in the study by signing an informed consent form, previously evaluated by the Ethics Committee of the Universidad de Valparaíso. The present study was also carried out in compliance with the code of ethics of the World Medical Association Declaration of Helsinki [17, 19].

Conflicts of Interest Statement

The authors have no conflicts of interest to declare.

Funding Sources

This study was supported by CONICYT-FONDEF-FONIS + MINSAL grant, XIII Concurso Nacional de Proyectos de Investigación y Desarrollo en Salud, FONIS 2016 SA16I0162.

Author Contributions

S.B.-L., M.F.A.-S., A.O., M.G., M.V., J.L., and C.P.P.-S. made substantial contributions in the work design, data acquisition, analysis and interpretation, besides the drafting of the manuscript. All the authors approved the final version of the article.