Use of the MoCA in Detecting Early Alzheimer's Disease in a Spanish-Speaking Population with Varied Levels of Education

Abstract

Background/Aims

Performance on the Montreal Cognitive Assessment (MoCA) has been demonstrated to be dependent on the educational level. The purpose of this study was to identify how to best adjust MoCA scores and to identify MoCA items most sensitive to cognitive decline in incipient Alzheimer's disease (AD) in a Spanish-speaking population with varied levels of education.

Methods

We analyzed data from 50 Spanish-speaking participants. We examined the pattern of diagnosis-adjusted MoCA residuals in relation to education and compared four alternative score adjustments using bootstrap sampling. Sensitivity and specificity analyses were performed for the raw and each adjusted score. The interval reliability of the MoCA as well as item discrimination and item validity were examined.

Results

We found that with progressive compensation added for those with lower education, unexplained residuals decreased and education-residual association moved to zero, suggesting that more compensation was necessary to better adjust MoCA scores in those with a lower educational level. Cube copying, sentence repetition, delayed recall, and orientation were most sensitive to cognitive impairment due to AD.

Conclusion

A compensation of 3-4 points was needed for <6 years of education. Overall, the Spanish version of the MoCA maintained adequate psychometric properties in this population.

Introduction

The Montreal Cognitive Assessment (MoCA) is a 10- to 20-min screening test designed to assist clinicians in detecting early or minor cognitive impairment [1]. Performance on the MoCA has been demonstrated to be dependent on the educational level. The initial 1-point correction for ≤12 years of education, suggested by Nasreddine et al. [1], was derived from a validation sample residing in Montreal with a mean educational level of approximately 12 years. More recently, in recognition of the necessity for further score correction in individuals with a lower education, the same group recommended a revised correction of +1 point for 10-12 years of education and +2 points for 4-9 years of education [2]. No score adjustment was suggested for those with <4 years of education. Score adjustments based on relatively homogeneous populations such as these can be sample specific and may not be generalizable to populations with diverse cultural, linguistic, and socioeconomic backgrounds. For example, studies carried out in the US [3,4], China [5], and Colombia [6] all reported educational effects on MoCA performance which may not be fully compensated by the suggested score adjustments.

The use of cognitive instruments in minorities or other populations outside the group in which it was originally developed can require linguistic translation and often also cultural adaptation. The MoCA has been translated into approximately 37 languages (available at http://www.mocatest.org/). Hispanics are the most rapidly growing group in the US. By 2020, 20% of the elderly population in California will be Hispanic [7]. Hispanics living in the US represent a heterogeneous group, consisting of people of diverse linguistic and cultural traditions from the Caribbean, South America, Central America, and Mexico. According to the Alzheimer's Association's 2013 ‘Alzheimer's disease facts and figures’ report [8], older Hispanics are about 1.5 times as likely to have Alzheimer's disease (AD) and other dementias compared to older Caucasians [9] but less likely than older Caucasians to have a diagnosis [10]. In order to improve our ability to detect the earliest cognitive changes in this ethnic group, a more thorough understanding of the testing behavior of screening instruments in these populations is required. In spite of an ongoing emphasis on optimizing the recruitment of Hispanic participants for dementia research, there is still a dearth of research on the utility of the MoCA in Spanish speakers. To date, only one study has reported using the Spanish version of the MoCA in a sample of Colombian elders (mean education 4.8 years) [6]. In that study, the MoCA total score showed a progressive increment across the following three educational levels in nondemented individuals: illiterates and those with incomplete primary schooling (<5 years of education) obtained a mean MoCA score of 17.0, those who completed primary school (5 years of education) obtained a mean MoCA score of 18.9, and those with an education beyond primary school (>5 years education) obtained a mean MoCA score of 21.6. Several MoCA items were influenced by levels of education: cube copying, clock drawing, serial subtraction, letter fluency, and abstraction. The results imply that to increase the accuracy of interpretation of the MoCA in participants of Hispanic background, score adjustments might be necessary for those with little formal education. To date, the generalizability of these findings to US Hispanics has not been demonstrated. Here, we investigated the need for and the required magnitude of score adjustment strategies for the Spanish version of the MoCA in Hispanics residing in Southern California.

The aims of the present study were to identify which MoCA items are most sensitive to early AD in a Spanish-speaking US Hispanic population and to determine the optimal adjustment strategy for low levels of education. We hypothesized that a compensation of >2 points would be needed for participants with low educational levels. Using data obtained at the Easton Center for Alzheimer's Disease Research at the University of California, Los Angles, we compared four alternative education adjustments for the raw MoCA score. We evaluated the effect of these adjustments on the cutoff score, sensitivity, and specificity, examined the psychometric properties of the Spanish version of the MoCA, and performed item-level analyses to detect MoCA items that were most subject to education bias.

Methods

Participants

We studied a sample of 50 predominantly Spanish-speaking Hispanics (36 females) enrolled in one of two research studies at the Easton Center for Alzheimer's Disease Research at the University of California, Los Angeles. Thirty-nine participants were enrolled in an ongoing longitudinal study of AD and related dementias in the elderly. They were between 51 and 90 years old at the time of testing [mean age 71.4 years, standard deviation (SD) 9.7]. The remaining 11 participants were enrolled in a study of familial AD and were members of families with dominantly inherited AD caused by a known mutation. They were aged between 23 and 51 years (mean age 39.9, SD 9.2).

In the combined group, the average number of years of schooling was 7.3 years (range 0-20, SD 5.3), and the median was 6 years. Sixteen participants (32%) had 6 years of education, 17 (34%) had <6 years, and 17 (34%) had >6 years.

All participants self-reported Hispanic ethnicity. Participants most frequently reported being of Mexican origin (52%), followed by Central American (30%), South American (14%), Cuban (2%), and Puerto Rican (2%).

Assessments

Participants underwent comprehensive clinical and neuropsychological assessments in Spanish including a Spanish version of the MoCA (version November 7, 2004) [11]. A MoCA total score (without education correction) as well as individual item scores were computed. All participants or their legal representatives provided written informed consent.

Clinical diagnosis was assigned by consensus by a research team comprised of neurologists and neuropsychologists after a comprehensive review of all available clinical and neuropsychological information, following the standardized criteria implemented in the National Alzheimer's Coordinating Center Uniform Data Set [12]. Participants received a diagnosis of normal cognition, mild cognitive impairment (MCI), ‘impaired but not MCI’, or dementia. All normal participants scored above −1.5 SD for their age- and/or education-adjusted norms on the neuropsychological measures. A diagnosis of MCI was given to an individual who exhibited cognitive impairment (i.e., at least 1.5 SD below the age- and/or education-adjusted norms) on at least one neuropsychological measure, in the context of generally intact activities of daily living based on informant reports [13]. The ‘impaired but not MCI’ group exhibited cognitive impairment in a single domain, with reported dependence in activities of daily living. To avoid circularity, the MoCA was not used as a criterion for the diagnoses.

A presumed etiology was established for any diagnosis other than normal, following specific criteria [14,15,16,17,18,19,20]. In particular, AD was determined using the National Institute of Neurological and Communicative Disorders and Stroke as well as the Alzheimer's Disease and Related Disorders Association (NINCDS-ADRDA) criteria [14]. For the 11 participants at risk of familial AD, the etiology was determined by considering whether or not a familial AD mutation was present in combination with clinical factors (e.g., their age relative to the typical age of AD symptom onset in their family and the presence of other conditions which might account for their performance).

Statistical Analyses

Analysis of variance (ANOVA) was used to test for differences in demographics and MoCA scores among diagnostic groups. For the nominal variable gender, the χ² test was used.

After examining the pattern of diagnosis-adjusted MoCA residuals in relation to education, we applied four alternative education-based adjustments for the raw score of the MoCA. To compare the effect of different adjustments, we used bootstrap sampling and examined two criteria: (1) the SD of the residuals, and (2) the association between education and the residuals. Sensitivity and specificity analyses were also carried out for the raw and each adjusted score using several cutoffs.

Finally, we evaluated the psychometric properties of the MoCA. We computed Cronbach's α coefficient and item-total correlations to examine the internal reliability and item discrimination power, respectively. To examine item validity, we used the clinical diagnosis as an external standard and performed ANOVA to test for item score differences among diagnostic groups. The effects of education on item scores was tested with a regression model for each item score adjusting for diagnosis. We performed two-tailed Student's t test to identify items most sensitive to cognitive impairment due to probable or possible AD, relative to the normal controls.

Sensitivity and specificity analyses were performed in R using library ‘ROCR’ [21]. All the other analyses were performed in SAS.

Results

Six participants were diagnosed as cognitively normal, 18 as MCI, 3 as ‘impaired but not MCI’, and 23 as dementia. The presumed etiologies for the cognitively impaired and the demented participants can been seen in table 1.

Table 1.

Diagnosis and presumed etiology in the study sample

Diagnosis	Subjects	Presumed etiology
Cognitively normal	6 (12%)	N/A
MCI	18 (36%)	Probable AD (n = 7)
		Alcohol-related dementia (n = 1)
		Undetermined etiology (n = 4)
		Depression (n = 3)
		Parkinson's disease (n = 1)
		Other: sleep deprivation (n = 1), anxiety (n = 1)
Impaired but not MCI	3 (6%)	Possible AD (n = 1)
		Depression (n = 1)
		Stroke (n = 1)
Dementia	23 (46%)	Probable AD (n = 13)
		Possible AD (n = 5)
		Dementia with Lewy bodies (n = 1)
		Vascular dementia, NINDS/AIREN probable (n = 1)
		Vascular dementia, NINDS/AIREN possible (n = 1)
		Parkinson's disease (n = 2)

N/A = Not assessed.

The MCI and ‘impaired but not MCI’ groups were combined to create the cognitively impaired not demented (CIND) group. Demographics and MoCA raw scores for the three diagnostic groups are shown in table 2. The groups did not differ in gender composition (χ² = 1.44, p = 0.49) or years of education (F = 1.10, p = 0.34). There was an age difference (F = 9.69, p < 0.001) due to the demented group being older than the other two groups (F = 16.0, p < 0.001). No age difference was found between the normal group and the CIND group (F = 0.00, p = 0.97). As expected, the MoCA score was significantly lower in the demented group (F = 63.1, p < 0.0001) relative to the other two groups. MoCA scores were similar between the normal and the CIND group (F = 2.91, p = 0.09).

Table 2.

Demographics and MoCA raw scores in three diagnostic groups

	Normal (n = 6)	CIND (n = 21)	Dementia (n = 23)
Female	4 (66.7%)	17 (81.0%)	15 (65.2%)
Age, years	56.6 ±16.8	56.4 ±17.3	73.9 ±8.9
Education, years	10.3 ±6.4	7.1 ±4.8	6.8 ±5.5
MoCA raw scores	24.8 ±3.0	20.8 ±5.3	10.2 ±5.3

Comparison of Alternative Score Adjustments

The residuals in the MoCA total score after removing diagnosis effects were plotted against education in figure 1. The regression model showed a positive association between scores and education level (education coefficient β = 0.27, p = 0.04). Those with <6 years of education showed the least balance around a reference line of zero. The majority of them produced sizeable negative residuals, indicating the need for compensatory score adjustment for this group. Based on the above observation, we tested the following four adjustment strategies.

Fig. 1.

Scatter plot of diagnosis-adjusted MoCA residuals and years of education. The solid line is the zero reference line, and the dashed line represents the best fit regression line.

Adjustment 1: we added 1 point to the participant's raw MoCA score if his/her education was ≤12 years. As this is the established adjustment method, we used it here for purposes of comparison – see Nasreddine et al. [1].

Adjustments 2 and 3 were based on the education correction proposed by Chertkow et al. [2]. We, however, proposed two modifications.

Adjustment 2: in addition to adding 1 point to the MoCA raw scores for those with 10-12 and 2 points for those with 4-9 years of education as proposed by Chertkow et al. [2], we also added 2 points for those with <4 years of education.

Adjustment 3: in addition to adding 1 point to the MoCA raw scores for those with 10-12 and 2 points for those with 4-9 years of education as proposed by Chertkow et al. [2], we added 3 points for those with <4 years of education.

Adjustment 4: we added 4 points if the education was ≤5 years and made no adjustment of the MoCA raw scores for an education ≥6 years. This adjustment was based on the observation of the residual plot in our own data.

To compare the four adjusted scores, for each of them, we computed the SD of the diagnosis-adjusted residuals and the regression coefficient of education. Using 1,000 bootstrap samples, we generated distribution statistics for these two variables. The mean and the SD of these variables are summarized in table 3. While the SD of the residuals was reduced by all four adjustments, the most improved statistics were seen with adjustment 4. SD residuals and β coefficients declined from 4.89 to 4.62 and from 0.27 to 0.07, respectively, between adjustment 1 and adjustment 4. These results showed that aggressive adjustment for those with the lowest education is the best strategy.

Table 3.

Comparisons of alternative MoCA score adjustments

	Raw score	Adjustment 1	Adjustment 2	Adjustment 3	Adjustment 4
Bootstrap sampling
SD (residuals)	4.89 ± 0.63	4.82 ± 0.63	4.76 ± 0.63	4.71 ± 0.60	4.62 ± 0.52
β (education)	0.27 ± 0.13	0.20 ± 0.13	0.14 ± 0.13	0.09 ± 0.13	0.07 ± 0.13
Sensitivity and specificity
AUC	0.926	0.928	0.931	0.937	0.934
Best cutoff	18	19	20	20	20
Sensitivity	0.91	0.91	0.96	0.96	0.96
Specificity	0.81	0.81	0.81	0.81	0.74

Sensitivity and Specificity of the MoCA

Because our sample had only 6 normal participants, we were unable to evaluate the sensitivity and specificity of the MoCA test for detecting MCI. Instead, we examined the MoCA's sensitivity and specificity for predicting the diagnosis of dementia. Using consensus diagnosis as the standard, the area under the ROC curve (AUC) was compared for the raw and adjusted MoCA scores (table 3). The best cutoff and the corresponding sensitivity and specificity for each adjusted score are listed in table 3. Similar to the results reported above, the AUC for predicting a diagnosis of dementia improved as compensation for lower education was introduced. The highest AUC was obtained when adjustment 3 was applied (AUC 0.937), followed by adjustment 4 (AUC 0.934). Using raw MoCA scores without adjustment, a cutoff of 18 (i.e., scores of ≤17 indicate dementia) yielded the best balance between sensitivity (0.91) and specificity (0.81). Depending on the adjustment method, the best cutoff lay between 18 and 20 (table 3). Sensitivity and specificity were not much affected by the score adjustments and remained at a similar level.

Psychometric Properties and Item Scores

Cronbach's α of the MoCA was 0.88, indicating good internal consistency. Average item scores for the whole sample and the three diagnostic groups are summarized in table 4. Item-total correlations were computed for the whole sample (table 4). The correlation coefficients ranged between 0.55 and 0.78 (table 4), showing good item discrimination.

Table 4.

MoCA item scores for three diagnostic groups

Itema	All	rb	Normal (n = 6)	CIND (n = 21)	Dementia (n = 23)	ANOVA, p valuec	AD etiology (n = 26)	Education effect, p valued
Trails (1)	0.40	0.55	0.50	0.67	0.13	0.0006*	0.31	0.37
Cube (1)	0.30	0.57	0.83	0.38	0.09	0.0005*	0.15*	0.10
Clock (3)	1.84	0.76	2.67	2.33	1.17	<0.0001*	1.58	0.07
Naming (3)	2.34	0.63	2.83	2.52	2.04	0.04	2.15	0.02
Attention
Digits (2)	1.28	0.70	1.83	1.48	0.96	0.01	1.04	0.14
Letters (1)	0.74	0.55	1.00	0.86	0.57	0.03	0.77	0.36
Serial subtraction (3)	1.56	0.67	2.33	1.86	1.09	0.01	1.27	0.02
Language
Repeat (2)	0.72	0.62	1.67	0.86	0.35	0.0005*	0.50*	0.25
Fluency (1)	0.42	0.55	0.83	0.48	0.26	0.03	0.27	0.10
Abstraction (2)	0.96	0.59	1.17	1.24	0.65	0.07	0.62	0.08
Delayed recall (5)	1.56	0.72	3.33	2.43	0.30	<0.0001*	0.50*	0.96
Orientation (6)	4.30	0.78	5.83	5.71	2.61	<0.0001*	3.27*	0.49

^*Significance level of 0.004.

^aFigures in parentheses are total possible points.

^br represents item-total correlation.

^cp value for ANOVA tests of item score difference among three diagnostic groups.

^dThe p value indicates the significance of the regression coefficient of education when adjusting for diagnosis.

We used consensus diagnosis as the external standard to evaluate item validity. Twelve ANOVA tests were performed to examine item differences among three diagnostic groups, and Bonferroni correction was used to adjust the significance level for multiple comparisons. The adjusted significance level was 0.05/12 = 0.004. Six items showed an overall significant difference among diagnostic groups: trail making (F = 8.64, p = 0.0006), cube copying (F = 8.92, p = 0.0005), clock drawing (F = 12.9, p < 0.0001), sentence repetition (F = 9.10, p = 0.0005), delayed recall (F = 15.3, p < 0.0001), and orientation (F = 29.1, p < 0.0001).

The average item scores for participants with cognitive impairment (CIND and dementia groups) with a primary etiology of probable AD or possible AD (n = 26) are also listed in table 4. Twelve t tests were performed to compare item scores between this group and the normal group, and, likewise, a Bonferroni-corrected significance level of 0.004 was used. Relative to the normal participants, participants with cognitive impairment due to AD performed worse on cube copying (t = 4.00, p = 0.0004), sentence repetition (t = 3.79, p = 0.0007), delayed recall (t = 4.95, p < 0.0001), and orientation (t = 5.43, p < 0.0001).

We used regression models to examine the effect of education in individual items while adjusting for diagnosis. Once again we used an adjusted significance level of 0.004. Though some items showed a trend of educational advantage (i.e., naming and serial subtraction), none of them reached the significance level of 0.004.

Our 3 subjects with ‘impaired but not MCI’ diagnosis were highly inhomogeneous in respect to their educational level and raw MoCA performance (years of education were 6, 2, and 2, with MoCA raw scores of 19, 22, and 8, respectively). In order to assure that their inclusion with the MCI group did not bias the results, all the above analyses were repeated after exclusion of these 3 participants. The results were very similar, except that the association between education and diagnosis-adjusted MoCA residuals became insignificant but still showed a positive trend (education coefficient β = 0.23, p = 0.08).

Discussion

In our analyses of MoCA data from a Spanish-speaking sample, we confirmed our hypothesis that more aggressive compensation than previously suggested is necessary to adjust for a low education level. Our data suggest that for subjects with lower levels of education a correction of 3-4 points is more appropriate. However, our results should be interpreted with caution. While the addition of 4 points to the raw MoCA scores for subjects with ≤5 years of education outperformed the other three approaches in terms of smaller residual variance of the score, reduced educational effect, and a relatively high AUC, this adjustment was derived entirely from our own data and would need to be replicated in an independent sample. While the bootstrap sampling procedure allowed us to compare the distribution of the estimates (SD of residuals and β coefficient), it relies on the key assumption that the observations are independent and identically distributed.

Compared to item average scores in the Montreal validation sample [1] (provided at http://www.mocatest.org/normative_data.asp), scores on serial 7 subtraction in the current sample were systematically lower in all three diagnostic groups. Given the educational level of the Montreal sample relative to that of our sample (11.9 vs. 7.5 years), the performance on this item is likely related to education. Our within-sample analyses showed that serial 7 subtraction had only a trend level association with education, most likely due to the small sample size. We similarly uncovered a trend for an education effect on naming, but it should be noted that the naming scores of our sample and the Montreal sample were similar. This discrepancy could be due to the fact that naming as a language test might not be directly comparable. Though the same test objects (i.e., lion, rhinoceros, and camel) were used in both versions, the difficulty of the naming task is related to linguistic and cultural factors such as word frequency and test object familiarity. There might be a Spanish language-specific educational advantage on naming performance. Other items where lower scores were observed in ours relative to the Montreal sample were sentence repetition and abstraction.

Compared to the Spanish sample in the Colombian study [6], nondemented participants in our sample seemed to perform better on trail making, cube copying, clock drawing, digits forward and backward, A-vigilance task, and delayed recall. When we restricted the comparison to Colombian participants with at least primary school education (≥5 years), our nondemented participants still performed better on cube copying (0.54 vs. 0.32), clock drawing (2.54 vs. 1.76), digits forward and backward (1.63 vs. 1.32), and delayed recall (2.67 vs. 1.68). There are several possible explanations for these differences. First, the Colombian study used a cross-sectional design and a community-based sample. In contrast, our participants were part of longitudinal studies, and some had already gained research experience and practice effects from earlier visits. For tasks such as trail making, digits forward and backward, and delayed recall, there might be a practice effect of having performed other similar neuropsychological tests in the previous year. Second, even when samples are matched by educational level, the nature and quality of education in different countries is likely to vary considerably. An important limitation of our dataset was that the country and language in which subjects received their education was unknown. Many study subjects were immigrants, having received their education in Latin American countries, though others were raised and educated in the US. Third, by virtue of living in Los Angeles, our Spanish speakers were inevitably exposed to an English-speaking environment, and, thus, most should be considered at least partially bilingual. The cognitive advantage of bilingualism has been extensively discussed [22,23,24] and may be greater for older people [25]. Though all subjects performed the MoCA in the language they felt to be most proficient in (Spanish), these factors likely influenced their performance.

Contrary to our expectations, we failed to see educational effects on any individual MoCA item in our sample. Educational effects have been reported by a number of studies on similar neuropsychological tests, for example, the trail making test [26,27] and serial subtraction [28,29]. Our failure to find such effects may be due to our small sample size or, in the case of trail making, the simplified shortened version in the MoCA battery.

This study has several other limitations. As this was a convenience sample, it did not have balanced diagnostic groups. The small number in the normal group (n = 6) limited our statistical power to detect a difference between normal participants and the CIND group, the main strength of the MoCA over other screening tools being to detect subtle cognitive impairment in nondemented persons [1]. Although raw and item-level MoCA scores were considerably higher in our normal controls relative to the CIND group, these differences did not reach statistical significance. Furthermore, our analyses on sensitivity and specificity were restricted, and we were unable to infer a cutoff score to detect MCI.

There was substantial heterogeneity in the presumed etiology of MCI and dementia in our sample. Etiologically distinct disorders can have different patterns of cognitive deficits [30]. Beside AD, the validity of the MoCA has been ascertained in Parkinson's disease [31,32,33], stroke and vascular dementia [34,35], and dementia with Lewy bodies [36]. However, the usefulness of MoCA for cognitive impairment induced by depression or other etiologies has not been determined. Our results could have been partly affected by the heterogeneity of the sample.

A unique feature of this study was the inclusion of 11 participants at risk of familial AD mutations. Though this population was younger (mean age 40 years) than that typically encountered when using the MoCA to screen for early signs of dementia, whether or not any observed impairment is due to AD can be reliably predicted by their mutation status. Therefore, their inclusion adds strength to our findings regarding which MoCA items are most sensitive to the early pathology of AD.

Results from our Hispanic sample may have limited generalizability. As discussed above, Spanish populations in different geographical locations could differ in various aspects including the degree of bilingualism and the quality of education received, so an application of the same normative data across these groups may not be possible. On the other hand, the heterogeneity of our sample reflects that of Hispanics in Los Angeles and, therefore, does inform the utility of the MoCA in this group. Although this population traces its origins to different Spanish dialect zones of Latin America [37], we assume that the dialectal effect on the MoCA assessment is trivial.

In summary, the Spanish version of the MoCA maintained adequate psychometric properties, and education correction was necessary to increase the accuracy of score interpretations. A compensation of 3-4 points was needed for those with <6 years of education. Items most likely to be affected by education were serial 7 subtraction and naming. The effect of the latter may only be restricted to the Spanish version. Cube copying, sentence repetition, delayed recall, and orientation were most challenging in subjects with cognitive impairment due to AD.