Optimal Cutoff Scores for Dementia and Mild Cognitive Impairment in the Brazilian Version of the Montreal Cognitive Assessment among the Elderly

Abstract

Objective

To propose cutoff scores for the Brazilian version of the Montreal Cognitive Assessment (MoCA-BR) stratified by education in order to detect mild cognitive impairment (MCI) and mild Alzheimer's disease (AD) in the elderly.

Method

A transversal study in health centers was performed on 159 elderly people with 4–12 years of education and 70 of their peers with over 12 years of schooling. The MoCA-BR cutoff scores for screening cognitive impairment were determined based on an ROC curve analysis.

Results

The ROC curve analysis indicated that cutoff scores under 20 were good for screening elderly people with cognitive impairment with more than 12 years of education, and scores under 21 were good for screening those with 4–12 years of education.

Conclusions

MoCA-BR scores under 21 points (after adding 1 point to the elderly with ≤12 years of education) indicate a need to continue the diagnostic investigation with regular follow-ups.

Introduction

The distinction between cognitive alterations compatible with normal aging and pathological processes in the early stages, such as mild cognitive impairment (MCI) and mild AD, is sometimes difficult [1]. The normal aging process is associated with declines in processing speed and certain memory, language, visuospatial, and executive function abilities [2]. For this distinction, cognitive tests, including the Brazilian version of the Montreal Cognitive Assessment (MoCA-BR), were used. The MoCA is an instrument developed by Nasreddine et al. [3] as an MCI screening test.

This test has been proven to be sensitive to MCI and to predict future cognitive decline in several cognitively impaired states, including AD [4]. It represents a practical and effective method for differentiating between the cognitive performance of elderly people with normal cognitive aging and those with MCI. It is also useful in differentiating between MCI and AD in mild to moderate stages. The time for its application is 10–15 min [3, 5], with a maximum score of 30. The MoCA-BR was evaluated in a sample of 112 people over 65 years of age with at least 4 years of education, divided into groups according to their cognitive state [6].

Education is considered a criterion for the establishment of normative data for cognitive tests, and the impact the level of education has on cognitive performance has been well established in the literature [5, 7, 8, 9, 10, 11, 12, 13, 14, 15]. Furthermore, the author of the MoCA proposed adding 1 point to individuals with 12 years of education or less, aiming to correct the effect education has on MoCA performance [3].

Additionally, studies have determined different MoCA cutoff scores to discriminate individuals with MCI from cognitively healthy individuals, i.e., < 26 [3, 16, 17], < 23 [18], < 22 [19, 20], < 21 [13], and < 20 [21, 22]. Some studies [6, 12, 18, 23, 24, 25] have revealed that the originally recommended cutoff score of 26 leads to a higher rate of false positives than that found in the original study of Nasreddine et al.[3]. A recent meta-analysis revealed that an MoCA cutoff score of 23 lowers the false-positive rate and shows an overall better diagnostic accuracy [26]. Therefore, the present study aims to propose MoCA-BR cutoff scores stratified by education in order to detect MCI and mild AD in the elderly.

Materials and Methods

Study Design, Sample, Procedure, and Instruments

This is an observational, transversal, analytical study conducted at 4 respected health care centers for the elderly in Recife, Brazil. The population in the study was composed of people aged 65 years and older, of both genders, with at least 4 years of education. The participants were already being followed up in some of the centers where the study was conducted. They spontaneously sought medical care due to their memory complaints. Patients were also sent from other units for evaluation of suspected cognitive impairment. Other participants were healthy people from the community who were willing to participate in this study.

The participants were divided into the following 3 groups: control (cognitively healthy people), MCI (patients with MCI), and mild AD (people with an AD clinical dementia rating [CDR] of 1). All of the participants in this study were submitted to anamnesis ranging from medical history to habits, and the application of sociodemographic and clinical questionnaires containing general facts, i.e., age, gender, education, comorbidities, and medication. For evaluation of the functionality of the elderly, the Pfeffer Functional Activities Questionnaire [27] was applied by an experienced occupational therapist. For cognitive evaluation of the elderly in this study, a complete neuropsychological evaluation was performed by a neuropsychologist. To evaluate the stages of dementia, CDR, prepared by Hughes et al. [28] and updated by Morris [29], was used.

The inclusion and exclusion criteria for each group are described below.

• - The inclusion criteria for the control group were: age ≥65 years and education ≥4 years.

• - The exclusion criteria for the control group were: a diagnosis of dementia; a diagnosis of relevant neurological or psychiatric diseases or systematic uncontrolled chronic diseases that have an impact on cognition; a history of alcoholism or substance abuse; aphasia or a visual, hearing, or motor handicap; evidence of loss of autonomy or independence in daily activities; regular use of psychotropic drugs; and altered performance on neuropsychological tests.

• - The inclusion criteria of the MCI group were: age ≥65 years; education ≥4 years; a subjective cognitive complaint, preferably confirmed by an informant; an objective cognitive deficit confirmed by a low performance under the level expected on neuropsychological tests; normal general cognitive functions; and intact or minimally impaired functional activities. The exclusion criterion for the MCI group was a diagnosis of dementia.

• - The inclusion criteria for the AD group were: age ≥65 years; education ≥4 years; a clinical diagnosis of Alzheimer's disease (AD), supported by neuropsychological tests showing cognitive impairment and a functional evaluation showing a decrease in functionality; and mild stage dementia, evaluated through CDR, corresponding to a score equal to 1 using this tool. The exclusion criteria for the AD group were: a diagnosis of mixed dementia (AD associated with another type of dementia), and moderate or advanced stages of dementia evaluated by CDR.

Considering the inclusion and exclusion criteria in the medical interview, past and current medical histories of the elderly, and the performance of a complete neuropsychological evaluation performed by an experience professional, a team composed of a psychiatrist, a neuropsychologist, and a geriatric doctor classified, following a consensus meeting, the participants selected for the control group, the MCI group, and the AD CDR 1 group. The occupational therapist who applied the MoCA-BR did not participate in the consensus meeting for selection and classification of the elderly, in an effort to avoid the risk of bias by using the information on the MoCA-BR performance for the definition of a cognitive diagnosis. The MCI diagnosis was defined based on the criteria of Petersen [30]. In turn, the diagnosis of AD was based on the criteria of the National Institute of Neurological and Communicative Disorders and Stroke/Alzheimer's Disease and Related Disorders Association (NINCDS/ADRA) [31].

Statistical Analysis

For the statistical analysis, the Statistical Package for the Social Sciences (SPSS), version 21.0, was used. Application of the Kolmogorov-Smirnov and Shapiro-Wilk tests demonstrated an abnormal distribution of the MoCA-BR outcome score variable. Therefore, nonparametric tests were used to apply hypothesis tests. For comparison of performances on the MoCA-BR among the control, MCI, and AD groups, a nonparametric Kruskal-Wallis test, with use of the Dunn-Bonferroni test for post hoc analysis, was applied.

To determine the accuracy of the MoCA-BR in discriminating between the control and MCI, control and AD, and MCI and AD, ROC curves and determination of the area under the curve were used. Contingency tables (2 × 2) were used to calculate the number of individuals who were correctly classified regarding their cognitive state. The optimum cutoff scores stratified according to education range to detect cognitive impairment were established using the Youden index (calculated using the formula: sensitivity + specificity −1) and based on the percentage of those correctly classified. For rejection of the null hypothesis, a value of p < 0.05 and a CI of 95% were considered as significant levels.

Results

From a total of 159 elderly people with 4–12 years of education, 25 (15.8%) received an AD CDR 1 diagnosis, 67 (42.1%) were classified as having MCI, and 67 (42.1%) were classified as being cognitively healthy. One hundred twenty-six (79.2%) of the participants were female, and the average age was 74.62 ± 6.18 years. In turn, the average age of the 70 people with more than 12 years of education was 72.13 ± 5.76 years, and they were predominantly female (70.0%). There was no statistical difference between the diagnostic groups of both education ranges regarding gender or age (p < 0.05). There was a difference in cognitive performance between those with 4–12 years of education and those with over 12 years of education (p < 0.001, Mann-Whitney test).

When comparing the performances of the control, MCI, and AD CDR 1 groups on the MoCA-BR, a significant statistical difference was found between the scores of the 3 groups (p < 0.001) in both education ranges (Fig. 1). The results of the evaluation of the validity of the MoCA-BR to discriminate among ROC curves with stratification by education range is presented in Figure 2. The MoCA-BR areas under the curve stratified by range of education are presented in Table 1.

Fig. 1.

Comparison between the performance of the cognitively healthy group (control), those with MCI, and those with mild AD (AD CDR 1 group) on the MoCA-BR in participants with 4–12 (a) and more than 12 years of education (b). There was a significant statistical difference between the scores of the3 groups in both education ranges (p < 0.001; Kruskal-Wallis test). ** p ≤ 0.01; *** p < 0.001, obtained through post hoc analysis with the Dunn-Bonferroni test.

Fig. 2.

ROC curves for the MoCA-BR to differentiate individuals with MCI and AD CDR 1 from cognitively normal controls according to education range. a Control group vs. group with cognitive impairment (MCI and AD CDR 1) in elderly subjects with 4–12 years of education. b Control group vs. group with cognitive impairment (MCI and AD CDR 1) (more than 12 years of education). c Control group vs. MCI group (4–12 years of education). d Control group vs. MCI group (more than 12 years of education). e Control group vs. AD CDR 1 group (4–12 years of education). f Control group vs. AD CDR 1 group (more than 12 years of education).

Table 1.

Accuracy of the MoCA-BR in discriminating between cognitively healthy individuals and those with cognitive impairment, stratified by education

	AUC	SE	95% CI	p value (area = 0.5)a
Control vs. MCI and AD CDR 1
4–12 years of education	0.952	0.0146	0.907–0.980	<0.001
More than 12 years of education	0.953	0.0232	0.873–0.989	<0.001
Control vs. MCI
4–12 years of education	0.936	0.0193	0.881–0.971	<0.001
More than 12 years of education	0.950	0.0266	0.864–0.989	<0.001
Control vs. AD CDR 1
4–12 years of education	0.995	0.0050	0.952–1.000	<0.001
More than 12 years of education	0.963	0.0361	0.866–0.996	<0.001

AUC, area under the ROC curve.

^aObtained using the DeLong method [32].

The cutoff point designated by the MoCA-BR author is 25/26, indicating that points lower than 26 are probably MCI or dementia cases. When using this cutoff point in the present study, even though an elevated sensitivity (100.0%) was found, the specificity of the test was very low (21.8%), with only 62.4% of the participants correctly classified. Table 2 presents the optimum cutoff scores stratified according to each education range to detect cognitive impairment.

Table 2.

Cutoff scores of the MoCA-BR, according years of education

	Cutoff point	Sensitivity, %	Specificity, %	False negative, %	False positive, %	Youden index	Correctly classified cases, %
Control vs. MCI and AD CDR 1
4–12 years of education	19–20	89.1	83.6	10.9	16.4	0.727	86.8
More than 12 years of education	20–21	88.9	95.3	11.1	4.7	0.842	92.9
Control vs. MCI
4–12 years of education	19–20	85.1	83.6	14.9	16.4	0.687	84.3
More than 12 years of education	20–21	90.5	95.3	9.5	4.7	0.858	93.8
Control vs. AD CDR 1
4–12 years of education	17–18	96.0	100.0	4.0	0.0	0.960	98.9
More than 12 years of education	17–18	83.3	97.7	16.7	2.3	0.810	95.9

In the cutoff scores presented in the present study, the percentage of people correctly classified regarding their cognitive state was very high, varying from 84.3 to 98.9%. The sensitivity and specificity of the MoCA-BR were very expressive, varying from 83 to 96% and from 84 to 100%, respectively, in all of the ideal cutoff scores. To screen the cases of mild AD, the ideal cutoff point was 17/18, achieving over 95% hits regarding the cognitive state of the participants, showing excellent accuracy for the MoCA-BR in detecting cases of MCI as well as cases of mild AD.

Discussion

In the present study, cutoff scores for the detection of MCI and mild AD stratified by education range were proposed. As observed in others cognitive tests, MoCA-BR performance was affected by education [5, 7, 8, 9, 10]. Previous studies in other countries have reported similar effects in relation to education, with people with lower levels of education obtaining lower scores [5, 11, 13, 18, 19, 33, 34]. Due to the effect of education on the performance on the MoCA-BR, the cutoff scores were determined according to the education range.

To identify the cases of MCI, the MoCA-BR presented the cutoff of point less than 21 as ideal for people with over 12 years of education, and that of less than 20 for those with 4–12 years of education. Therefore, the cutoff point for those with more than 12 years of education was higher than for those with less education, according to the findings of other studies [35, 36, 37], highlighting the importance of considering the level of education when evaluating cognitive performance to achieve a greater accuracy in screening diagnosis. This study presented an elevated sensitivity and a low percentage of false negatives in the MoCA-BR in the screening of cognitive impairment. The cutoff scores proposed in the present study for the detection of MCI were higher than those for the detection of mild AD, since there is a lower cognitive decline in MCI. In other words, the performance of elderly with MCI is superior to that of those with AD.

Therefore, to increase the sensitivity of the test, based on an ROC curve analysis and comparison of the parameters of accuracy, a stricter cutoff point was found which was represented, in the case of the MoCA-BR battery, by values higher than those of AD (i.e., ≤19 and ≤20 vs. ≤17, respectively). Findings similar to these have been reported in other studies [6, 24, 23, 33, 38]. Because it is a more rigorous cognitive screening test, the MoCA-BR is a useful tool in groups where the ceiling effect is a problem.

The excellent accuracy of the MoCA-BR when using the cutoff scores proposed in the present study was in contrast with the low percentage of correct classifications when using the cutoff point proposed by the author of the MoCA (62.4%) [3], which is compatible with the results found in a recent meta-analysis [26]. This shows the importance of adapting the cutoff point according to the population under study. Other studies that used other versions of the MoCA also proposed cutoff scores inferior to those proposed by Nasreddine et al. [3], from values similar to those in the present study [22, 38] to a cutoff point of 14/15 in a study conducted in Spain with elderly people in the control and AD groups with an average education less than that of those in present study [39]. With the recommendation that 1 point be added to the MoCA-BR score obtained for participants with up to 12 years of education - similar to that recommended by the MoCA author [3] - the cutoff point suggested in the present study is 20/21 to discriminate between the cognitively healthy individuals and those with mild cognition impairment. People with fewer than 21 points must be referred for follow-up and a clear diagnosis.

The strengths of the present study are the use of NINCDS/ADRA criteria [31] for diagnosing AD, the MCI diagnosis based on the criteria of Petersen criteria [30], cutoff points stratified by education level, and complete characterization of the subject sample, employing not only standardized examinations and instruments but also neuropsychological testing. The designations of cognitively impaired and cognitively healthy were based on objective findings from neuropsychological testing. Additionally, this study is relevant due to the relative scarcity, even in the international literature and especially in the Brazilian literature, of studies proposing MoCA-BR cutoff scores stratified by education in order to detect MCI and mild AD. The use of a sample with a distribution close to that observed in the elderly population increases the equivalence with the target population.

The present study has limitations that should be addressed. Participants with less than 4 years of education were excluded. The MoCA presents items that are inadequate for individuals with little or no formal education (especially the items related to executive functions) [38, 40]. The use of the MoCA in this group would lead to a loss of discriminating power between normal and pathological, with an increase in false positives and false negatives, as occurred in studies which included illiterate or poorly educated older people [13, 34]. Moreover, the original MoCA study was developed for cognitive screening of people with a minimum formal education of 4 years [3]. Therefore, application of the results of this study to elderly people who have less than 4 years of education is not recommended. Another limitation is that the participants were recruited from one city in Brazil. Brazil is a country with continental dimensions, with a considerable socieodemographic and cultural regional variety, which could influence the performance of the elderly on the MoCA-BR. Thus, the cutoff points of this study should not be generalized or extrapolated to populations with characteristics different from those of the population in question, especially regarding education. However, the participants were randomly selected in proportions that were consistent within the elderly Brazilian population. The discrepancy was thus minimized. We suggest further studies recruiting participants from all over Brazil.

Concluding, the results of the new study indicate that cutoff scores of less than 21 and less than 20 for elderly people with 12 and 4–12 years of education, respectively, present excellent accuracy for diagnostic screening of cognitive impairment. Scores of under 21 points on the MoCA-BR - after adding 1 point to elderly individuals with ≤12 years of education - indicate a need for further diagnostic investigation and regular follow-up. The cutoff points presented can be used to inform future work using the MoCA-BR to screen for MCI and AD in older Brazilian people. Future studies could focus on early detection and treatment of cognitive dysfunctions in clinical practice.

Statement of Ethics

The subjects gave their informed consent and the study protocol was approved by the institute's committee on human research.

Disclosure Statement

The authors declare no potential conflict of interests with respect to the research, authorship, and/or publication of this article.