Detection of Mild Cognitive Impairment Among Community-Dwelling African Americans Using the Montreal Cognitive Assessment

Abstract

Objective

To establish a cut score for the Montreal Cognitive Assessment (MoCA) that distinguishes mild cognitive impairment (MCI) from normal cognition (NC) in a community-based African American (AA) sample.

Methods

A total of 135 AA participants, from a larger aging study, diagnosed MCI (n = 90) or NC (n = 45) via consensus diagnosis using clinical history, Clinical Dementia Rating score, and comprehensive neuropsychological testing. Logistic regression models utilized sex, education, age, and MoCA score to predict MCI versus NC. Receiver operating characteristic (ROC) curve analysis determined a cut score to distinguish MCI from NC based on optimal sensitivity, specificity, diagnostic accuracy, and greatest perpendicular distance above the identity line. ROC results were compared with previously published MoCA cut scores.

Results

The MCI group was slightly older (M_MCI = 64.76[5.87], M_NC = 62.33[6.76]; p = .033) and less educated (M_MCI = 13.07[2.37], M_NC = 14.36[2.51]; p = .004) and had lower MoCA scores (M_MCI=21.26[3.85], M_NC = 25.47[2.13]; p < .001) than the NC group. Demographics were non-significant in regression models. The area under the curve (AUC) was significant (MoCA = .83, p < .01) and an optimal cut score of <24 maximized sensitivity (72%), specificity (84%), and provided 76% diagnostic accuracy. In comparison, the traditional cut score of <26 had higher sensitivity (84%), similar accuracy (76%), but much lower specificity (58%).

Conclusions

This study provides a MoCA cut score to help differentiate persons with MCI from NC in a community-dwelling AA sample. A cut score of <24 reduces the likelihood of misclassifying normal AA individuals as impaired than the traditional cut score. This study underscores the importance of culturally appropriate norms to optimize the utility of commonly used cognitive screening measures.

Introduction

The detection of mild cognitive impairment (MCI) (Petersen et al., 1999) is of particular clinical and research interest because a high proportion of those diagnosed eventually develop Alzheimer’s disease (AD) (Manly et al., 2008). To enable early intervention, it is necessary to first identify those at increased risk for developing AD. Brief cognitive measures such as the Mini–Mental State Examination (MMSE) and the Montreal Cognitive Assessment (MoCA) are used in aging populations to screen for MCI and dementia. The MMSE (Folstein, Folstein, & McHugh, 1975) is perhaps the most widely used cognitive screening tool; however, it has reduced efficacy for detection of MCI in ethnic minorities (Bohnstedt, Fox, & Kohatsu, 1994; Escobar et al., 1986) and is less sensitive in discriminating between MCI and NC than the MoCA (Ciesielska et al., 2016; Tan et al., 2015). The MoCA (Nasreddine et al., 2005) is now commonly used to discriminate MCI from NC; however, several studies raise concern about the utility of the MoCA in minority populations (Luis, Keegan, & Mullan, 2009; Ng et al., 2015; Rossetti, Lacritz, Cullum, & Weiner, 2011; Sink et al., 2015) and in other parts of the world (Bosco et al., 2017).

The development of normative data for specific ethnic minorities for the MoCA has recently begun to be addressed (Goldstein et al., 2014; Rossetti et al., 2011; Sink et al., 2015). Most studies suggest an adjusted cut score specific to the population being examined, which is typically lower than the traditional score of 26 points (Del Brutto, Mera, Zambrano, Soriano, & Lama, 2015; Ojeda, Del Pino, Ibarretxe-Bilbao, Schretlen, & Pena, 2016; Rossetti et al., 2017; Zhou et al., 2015).

Initial studies suggest that clinically used standard cut scores may not be suitable for African Americans (AA), and may inappropriately classify NC as MCI, with numbers ranging from 80 to 94% of AA samples being classified as impaired using the traditional cut score of 26 points (Rossetti et al., 2017; Sink et al., 2015). There is a well-established need for a validated cut score in this population (Casaletto et al., 2015; Welsh et al., 1995), particularly given that AAs are disproportionately affected by AD (Barnes & Bennett, 2014). Goldstein and colleagues recently proposed a MoCA cut score of ≤24 for MCI in a sample presenting to a memory clinic (n = 38) in a predominantly inner-city, medically indigent urban area (Goldstein et al., 2014). The present study extends this work by providing a larger sample that is statistically powered to determine an optimal detection point. Our primary aim was to identify a cut score for the MoCA that best discriminates MCI from NC in a community-dwelling AA sample.

Methods

Participants

Participants (135 African Americans) were recruited from a larger longitudinal, population-based, multi-ethnic study of factors contributing to the development of cardiovascular disease in which African Americans were oversampled to ensure approximately 50% African American participation (Victor et al., 2004). Those over age 50 years were recruited for participation in longitudinal studies at the UT Southwestern Medical Center Alzheimer Disease Center (ADC). Thus, this cohort of AA individuals more closely approximates a community sample than a standard research center cohort or a memory clinic. All participants underwent neurocognitive assessment as part of their ADC visit and diagnoses of MCI (n = 90) or NC (n = 45) were made according to Peterson criteria (Petersen, 2004) based on consensus review of Clinical Dementia Rating score and comprehensive neuropsychological evaluations by clinicians blinded to MoCA results. In order to remove participants with possible cognitive impairment due to non-neurodegenerative conditions, exclusion criteria included neurological or other medical history that might substantially impact cognition (e.g., traumatic brain injury, brain tumor, epilepsy, and multiple sclerosis). All participants provided written informed consent, and the study protocol was approved by the Institutional Review Board of UT Southwestern Medical Center. Demographic characteristics are provided in Table 1.

Table 1.

Demographic characteristics

	MCI (n = 90), Mean (SD); Range or %	NC (n = 45), Mean (SD); Range or %	p Value
Age	64.76 (5.87); 50–80	62.33 (6.76); 51–81	.0332
Education	13.07 (2.37); 7–20	14.36 (2.51); 10–20	.0040
Female, n (%)	61 (68%)	34 (76%)	.3508
MoCA	21.26 (3.85); 13–29	25.47 (2.13); 20–29	<.0001

Note: Age and education are presented in years.

Measures

A standard ADC cognitive battery, including the MoCA, was administered in the same order by trained research personnel. The ADC battery consisted of the Uniform Data Set (Weintraub et al., 2009) (American National Adult Reading Test, Benson Figure, Category Fluency, Consortium to Establish a Registry for Alzheimer’s Disease battery, Craft Story, California Verbal Learning Test-II, Geriatric Depression Scale, Multilingual Naming Test, Mini–Mental State Examination, Number Span, Trail Making Test, Verbal Fluency, Wisconsin Card Sorting Test, Wechsler Memory Scale-III). The MoCA is a 30-point screening tool that requires approximately 10–15 min to administer, and evaluates aspects of attention, orientation, language, verbal memory, visuospatial abilities, and executive function. The individual MoCA items have been described in detail elsewhere (Nasreddine et al., 2005). The suggested 1-point correction for <12 years of education was not applied to the MoCA total score for the purpose of our analyses in light of prior work showing this approach is not sufficient to address education effects and negatively affects psychometric properties (Bernstein, Lacritz, Barlow, Weiner, & DeFina, 2011).

Statistical Analyses

Statistical analyses were conducted using IBM SPSS version 25. T-tests or Chi square tests, as appropriate, were conducted to determine demographic differences between MCI and NC groups. A stepwise logistic regression was performed to predict MCI using age, education, and sex as covariates. – (ROC) curve analysis was performed to determine a cut score to best distinguish MCI from NC groups. The cut score criteria included: (1) optimal sensitivity and specificity, (2) highest accuracy, and (3) the point of the curve with the furthest perpendicular distance from the diagonal line of equality (a marker of better classification results). Lastly, the cut score derived from this analysis was compared to the traditional, clinically used MoCA cut score of 26 to examine differences in diagnostic classification. Assumptions of all statistical tests were examined (normality, equal variance, etc.) and statistical significance was set to p < .05.

Results

The MCI group was slightly older (M_MCI = 64.76[5.87], M_NC = 62.33[6.76]; p = .033) and less educated (M_MCI = 13.07[2.37], M_NC = 14.36[2.51]; p = .004) and had lower cognitive scores on the MoCA (M_MCI = 21.26[3.85], M_NC = 25.47[2.13]; p < .001) than the NC group (see Table 1). MoCA scores were significantly correlated with age (r = −.21, p ≤ .017) and education (r = .43, p ≤ .001) but did not differ by gender (t[1,133] = −.174, p = .86).

An initial logistic regression model (predicting MCI vs. NC) included the MoCA total score with demographic covariates age, education, and sex. All three covariates were found to be non-significant predictors (p ≥ .34) and were not included in the final model. The MoCA significantly predicted MCI from NC with an odds ratio of 1.53 (95% CI: 1.30–1.80, p < .0001]). ROC analysis resulted in an AUC of 83% (95% CI: 75.5–89.5%, p < .0001), demonstrating that the MoCA accurately distinguished MCI from NC compared to chance. After evaluating several possibilities (Table 2), a cut score of <24 (less than 24), meaning that a score of 23 or below is impaired but a score of 24 or above is normal, resulted in both good sensitivity (72.2%) and specificity (84.4%), with 76.3% accuracy in distinguishing MCI from NC. Figure 1 provides graphic visualization of the ROC curve analysis with sensitivity on the Y-axis and 1 − specificity on the X-axis. The dot represents the point at which sensitivity and specificity are optimized. Compared with the traditional cut score of 26, this revised cut score demonstrated slightly lower sensitivity but much better specificity with similar accuracy (see Table 3).

Table 2.

Cut score determination

MoCA	Sensitivity (%)	Specificity (%)	PPV (%)	NNV (%)	Accuracy (%)
19.5	33.3	100.0	100.0	42.9	55.6
20.5	40.0	95.6	94.7	44.3	58.5
21.5	53.3	93.3	94.1	50.0	66.7
22.5	58.9	91.1	93.0	52.6	69.6
23.5	72.2	84.4	90.3	60.3	76.3
24.5	77.8	73.3	85.4	62.3	76.3
25.5	84.4	57.8	80.0	65.0	75.6
26.5	93.3	31.1	73.0	70.0	72.6
27.5	94.4	15.6	69.1	58.3	68.1

Fig. 1.

ROC curve analysis for MCI versus NC discrimination. Area under the curve = .826; 95% confidence interval: .755–.896; p < .0001.

Table 3.

Diagnostic accuracy of MoCA cut scores

	Traditional (<26), % (95% CI)	Revised (<24), % (95% CI)
Sensitivity	84.4 (77.0–91.9)	72.2 (63.0–81.5)
Specificity	57.8 (43.3–72.2)	84.4 (73.9–95.0)
Accuracy	75.6 (68.3–82.8)	76.3 (69.1–83.5)

Discussion

It is widely recognized that the standard MoCA cut score may not accurately distinguish MCI from NC in ethnic minority populations. This study expands on our prior normative work with the MoCA (Rossetti et al., 2017) and provides a cut score, based on optimal sensitivity, specificity, and diagnostic accuracy, to help differentiate persons with MCI from those with NC in an AA community-dwelling sample. Using the traditional cut score of <26, MoCA scores were classified as impaired in 84% of MCI and 42% of NC participants. Using the revised cut score of <24, 72% of MCI and only 16% of NC participants fell in the impaired range. This supports the notion that a large portion of non-impaired AAs may be inaccurately categorized through the use of “standard” cut scores derived from dissimilar demographic groups. However, as is the case with any cut score, sacrifices are made when determining how to optimize sensitivity and specificity. Clinicians are encouraged to utilize the provided table to select the cut score that works best for their setting.

Our findings are similar to other studies examining the diagnostic accuracy of the MoCA in AA cohorts. In a small, inner-city African American sample presenting to a memory clinic, a cut score of ≤24 greatly improved the MoCA’s accuracy for detecting MCI (63% specificity and 95% sensitivity vs. 31% specificity and 100% sensitivity using the 26 cut score) (Goldstein et al., 2014). Though the cut score presented by Goldstein and colleagues is one point higher than the present study, both proposed cut scores are similar and below the standard, clinically used cut score of 26. Both studies support the use of a lower cut score in order to reduce the likelihood of misclassifying cognitively normal AA individuals as impaired.

There are several limitations to our study worth noting that may limit generalizability. First, participants were a self-selected sub-set of a larger population-based study (Victor et al., 2004). Participants voluntarily chose to additionally enroll in this cognition-focused project, so there may be a self-selection bias. We do however believe that our sample is more representative of a community-based sample opposed to a typical research center cohort. Another limitation was the relatively small number of males in the sample, a common challenge in aging studies. However, sex was not found to be a significant covariate and there was no significant difference between the proportion of males in the MCI and NC groups. It should be emphasized that the MoCA is a screening tool rather than a diagnostic instrument, and the suggested cut score should be used as a general guideline to aid clinicians making referral decisions for patients with similar backgrounds.

This study presents a suggested cut score to help differentiate persons with MCI from those with normal cognition in a community-dwelling African American sample. Compared to previously published cut scores, using a MoCA cut score of <24 reduces the likelihood of misclassifying cognitively normal African American individuals as impaired, which has implications for both clinical treatment planning and participant characterization in research trials. These findings underscore the importance of population-specific norms to enhance confidence in interpretation, even for brief cognitive screening measures.

Funding

This work was supported in part by the Alzheimer’s Association (New Investigator Grant (NIRG)-14-322666), by the National Center for Advancing Translational Sciences of the National Institutes of Health under award Number UL1TR001105, and UT Southwestern Alzheimer’s Disease Center (NIH/NIA P30 AG12300-21). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

Conflict of interest

None declared.