Changes in Montreal Cognitive Assessment Scores Over Time

Abstract

This study explored the utility of the Montreal Cognitive Assessment (MoCA) in the detection of cognitive change over time in a community sample (age ranging from 58 to 77 years). The MoCA was administered twice approximately 3.5 years apart (n = 139). Participants were classified as mild cognitive impairment (MCI) or cognitively intact at follow-up based on multidisciplinary consensus. We excluded 33 participants who endorsed cognitive complaints at baseline. The MCI group (n = 53) showed a significant decrease in MoCA scores (M = −1.83, p < .001, d = 0.64). When accounting for age and education, the MCI group showed a decline of 1.7 points, while cognitively intact participants remained stable. Using Reliable Change Indices established by cognitively intact group, 42% of MCI participants demonstrated a decline in MoCA scores. Results suggest that the MoCA can detect cognitive change in MCI over a 3.5-year period and preliminarily supports the utility of the MoCA as a repeatable brief cognitive screening measure.

Introduction

Brief cognitive screening instruments are frequently used by clinicians and researchers to assess and monitor global cognitive function. The Mini-Mental State Examination (MMSE; Folstein, Folstein, & McHugh, 1975) and the Montreal Cognitive Assessment (MoCA; Nasreddine et al., 2005) are two widely used cognitive screening measures. The MMSE was originally designed to detect delirium in general medical and surgical patients and later was widely used as a screening instrument of cognitive decline (Damian et al., 2011; Folstein et al., 1975). Research has shown low sensitivity of MMSE to mild cognitive impairment (MCI) and has been increasingly supplanted by the MoCA (Wong et al., 2012), which was developed as a screening tool to detect MCI. The MoCA has been reported to be more sensitive than the MMSE to cognitive changes in both demented and healthy older adult populations (Damian et al., 2011; Gluhm et al., 2013; Lam et al., 2013).

Despite its widespread use, research on serial MoCA administrations is limited. A recent longitudinal evaluation of the MoCA in a healthy aging population showed variability in performance over time. Individuals with baseline MoCA scores below 26 demonstrated gains 12 months later and remained stable 48 months from initial administration. In contrast, individuals with MoCA scores 26 or above showed greatest increase 48 months after initial evaluation (Cooley et al., 2015). One study examining MoCA performance in a Parkinson disease sample found no significant change on the MoCA over a 3-year period of time (Lessig, Nie, Xu, & Corey-Bloom, 2012). Little is known about stability of MoCA in MCI and aging populations.

In this study, we examined MoCA scores over a period of 3.5 years in a sample of community-dwelling older adults. At the follow-up visit, participants were diagnosed with MCI based on published clinical criteria (Albert et al., 2011) or classified as cognitively intact. A secondary analysis in the study used this diagnostic classification to retrospectively investigate changes in MoCA scores over time.

Method

Participants were drawn from the Dallas Heart Study (DHS; Victor et al., 2004), a longitudinal investigation of cardiovascular disease risk factors in a population-based sample. The MoCA was added to the protocol during the second data collection phase of the DHS (n = 2,653) from 2007 to 2010. From this sample, participants who consented for enrollment in future studies were recruited by the Alzheimer’s Disease Center (ADC) at the University of Texas Southwestern Medical Center for yearly neurologic and cognitive assessment starting in December 2011.

Participants were recruited by the ADC between 2012 and 2014 with the following inclusion criteria (a) fluent in English, (b) aged 50 years or older, and (c) MoCA evaluations at two time points (DHS-II and ADC). This resulted in a sample of 139 participants. The study protocol was approved by the institutional review board of UT Southwestern Medical Center and all participants provided written informed consent.

The primary study aim explored change in MoCA in community-dwelling older adults. Therefore, to screen for possible cognitive impairment at baseline, we excluded participants who endorsed any memory complaints at the time of their first MoCA administration. Specifically, participants were asked three yes/no questions prior to completing the baseline MoCA: whether they believed they had any problems with their memory, if those problems interfered in daily functioning, and if they had any difficulty solving problems. Of the 139 participants with baseline and follow-up MoCA scores, we excluded 22 individuals who endorsed at least one of the three questions and 11 participants with missing data for one or more questions.

Supplementary analyses explored differences between the excluded cases and the study sample. There were no differences between the study sample (n = 106) and excluded participants (n = 33) with respect to age (t = 0.37, p = .71), education (t = −1.56, p = .12), baseline MoCA score (t = −1.16, p = .25), and MoCA score at follow-up (t = −1.97, p = .051). Sixty-seven percent of excluded participants were diagnosed with MCI at follow-up. Including these participants in the study sample may confound the results in a presumably healthy older adult sample, and therefore, they were excluded.

At follow-up, participants were assigned a diagnosis at a multidisciplinary consensus conference (including neuropsychologists, neurologists, and psychiatrists) based on history, clinical examination, Clinical Dementia Rating Scale (Morris, 1997), and a comprehensive neuropsychological battery. MCI was diagnosed based on published criteria (Albert et al., 2011), which included (a) reported concern for change in cognition by the participant, caregiver, or the clinician; (b) quantitative cognitive deficits in one or more cognitive domains which are greater than expected for the participant’s age and education; (c) the documented subtle but measureable cognitive deficit does not interfere with the participant’s ability to function independently but may require a greater effort or increased use of compensatory strategies to maintain the level of functioning; and (d) the cognitive deficits do not occur in the context of delirium and are not a sequelae of a mood disorder. In addition, a Clinical Dementia Rating Global Score of .5 was required to obtain a diagnosis of MCI.

Instrument

The MoCA is a 30-point screening tool that requires approximately 10 minutes to administer and briefly assesses several aspects of cognitive function including executive functioning, attention, language, abstraction, delayed recall, and orientation. The MoCA was scored without the suggested 1-point correction for ≤12 years of education as prior work has shown that this adjustment is inadequate and may adversely affect reliability of the MoCA in certain samples (Bernstein, Lacritz, Barlow, Weiner, & DeFina, 2011; Rossetti, Lacritz, Cullum, & Weiner, 2011).

Statistical Methods

Statistical analyses were conducted using IBM SPSS version 21.0 (IBM Corp, Armonk, NY). Chi-square test, Fisher’s test, or t tests were used to compare between group differences on demographic variables. An analysis of covariance was conducted using diagnosis as main effect (with an interaction term), and age and education as covariates. The statistical significance level was set at p < .05. A meaningful degree of cognitive change was established by calculating Reliable Change Index (RCI) confidence intervals (95%) from the test–retest results for the cognitively intact participants according to previously described methodology (Jacobson & Truax, 1991). Change in MoCA scores exceeding the RCI 95% confidence interval represents statistically reliable change that occurs only 5% of the time by chance. Annualized change in MoCA was also calculated for both groups by taking the difference between each participant’s follow-up score and baseline score and dividing by the number of total years between the assessments. Receiver operating characteristic curve analysis and corresponding sensitivity and specificity results were calculated for MoCA scores at follow-up.

Results

The time period between baseline and follow-up MoCA was approximately 3.5 years (range: 2–5 years, SD = 0.62). Of the 106 participants, the groups were evenly split, with 53 classified as cognitively intact and 53 diagnosed with MCI based on in-depth evaluation at follow-up. There were group differences in age and ethnicity. Education approached significance and since it is known to influence cognitive ability, it was used as a covariate in this study (Table 1). Additional analyses of age and ethnicity as covariates revealed significant interaction between age and MoCA (F = 5.23, p = .024, η² = 0.05). In contrast, ethnicity was not significantly related to other factors in the model (with time, diagnosis, age, or education) and was therefore not included as a covariate.

Table 1.

Demographic Characteristics by Diagnostic Classification at Follow-Up.

	Cognitively intact (n = 53)		MCI (n = 53)
	M (SD)	Range	M (SD)	Range	p
Age at baseline, years	60.2 (4.6)	55–70	63.4 (4.8)	55–72	.002*
Age at follow-up, years	65.5 (4.9)	58–76	67.9 (4.9)	60–77	.011*
Education, years	15.2 (2.7)	10–20	14.2 (2.9)	7–20	.074
Ethnicity, n (%)					<.001*a
Non-Hispanic Caucasian	42 (79)		16(31)
African American	9 (17)		31 (59)
Hispanic	2 (4)		5 (9)
Other	0		1 (1)
Gender, n (% female)	34 (64)		27 (51)		.17b
Baseline MoCA score	26.3 (2.2)		23.3 (2.9)		<.001*
Follow-up MoCA score	25.6 (2.2)		21.5 (2.9)		<.001*

Note. MCI = mild cognitive impairment; MoCA = Montreal Cognitive Assessment.

^aFisher’s exact test.

^bChi-square test.

^*p < 0.05.

A significant decrease in MoCA scores was seen in the MCI group (mean change = −1.83, SD = 2.6, t = −5.15, p < .001, d = 0.64), while the cognitively intact group did not demonstrate significant change over 3.5 years (mean change = −0.62 points, SD = 2.29, t = −1.93, p = .054, d = 0.29; Figure 1).

Figure 1.

MoCA performance over time.

Note. T1 = baseline; T2 = follow-up; MCI = mild cognitive impairment; MoCA = Montreal Cognitive Assessment. Significant decrease in scores for MCI participants at follow-up compared with baseline score, p < .001.

After adjusting for the covariates of education (F = 13.47, p < .001, η² = 0.12) and age at follow-up (F = 0.63, p = .43, η² = 0.006), MoCA scores (at two time points) by group (MCI vs. cognitively intact) interaction approached significance (F = 3.80, p = .054, η² = 0.04) along with main effects for group (F = 52.73, p < .001, η² = 0.34). When adjusting for education and age, the change in MoCA scores in the cognitively intact group was −0.75 (26.13 vs. 25.38) and in the MCI group was −1.70 (23.45 vs. 21.75).

Results for the RCI calculation are provided in Table 2. Based on the RCI cutoff (±1.73 points) established by the cognitively intact group, 42% of MCI participants exhibited a significant decline in MoCA scores over the 3.5 years, 49% remained stable, and 9% demonstrated increased scores. Analysis of annualized change by diagnosis showed an annual 0.52 point decrease in individuals with MCI, and a 0.17 point decrease in MoCA scores in the cognitively intact group.

Table 2.

Test–Retest Reliability Coefficients and Reliable Change Indices Based on Standard Error of Measurement (SEM) for Cognitively Intact Group.

	Test-retest reliability	SEM	SE diff.	RC
Baseline to follow-up	0.92	0.63	0.88	± 1.73

Note. SEM = standard error of measurement; SE diff. = standard error of the difference; RC = reliable change at 95% confidence interval. Test–retest reliability coefficients based on the correlation between the mean MoCA score at baseline and follow-up visit.

Receiver operating characteristic analysis (Figure 2) comparing MoCA scores for the two groups (MCI vs. cognitively intact) at follow-up demonstrated an area under the curve of 0.88 (95% confidence interval [0.82, 0.95]). At the recommended MoCA cutoff score of at or above 26 (Nasreddine et al., 2005), sensitivity was 51%, while specificity was 96%, likelihood ratio = 13.5. A MoCA score at or above 24 represented 89% sensitivity and 75% specificity, likelihood ratio = 3.62.

Figure 2.

ROC curve analysis for MoCA scores at follow-up.

Note. ROC = receiver operating characteristic; MoCA = Montreal Cognitive Assessment; MCI = mild cognitive impairment. Dashed line: MoCA score at follow-up for participants with MCI. Area under the curve 0.88 (95% confidence interval [0.82, 0.95]). At the recommended cutoff score of at or above 26 (Nasreddine et al., 2005), sensitivity was 51%, while specificity was 96%, likelihood ratio = 13.5. A MoCA score at or above 24 represented 89% sensitivity and 75% specificity, likelihood ratio = 3.62.

Discussion

This study provides preliminary information about MoCA performance over a 3.5-year interval in a community-based sample aged 58 to 77 years. Participants were categorized as cognitively intact or diagnosed with MCI based on an in-depth evaluation at follow-up. Our main findings are (a) Individuals classified as cognitively intact at follow-up demonstrated stable MoCA scores over the 3.5-year period, (b) Participants diagnosed with MCI at follow-up showed significant decline in scores during the same 42-month time period, (c) A reliable change of ±1.73 points in this time period represented a clinically meaningful difference.

The MoCA scores of cognitively normal participants remained stable at 42 months, which is consistent with established findings that healthy controls generally demonstrate steady scores over longer intervals between administrations of cognitive measures (Kramer et al., 2007; Unger, van Belle, & Heyman, 1999). This finding differs from a recent longitudinal study of the MoCA where healthy older adults demonstrated significant improvements in MoCA scores from baseline to 12 months and baseline to 48 months (Cooley et al., 2015). The difference may be due in part to the time gap between administrations of the MoCA in that sample (12 and 24 months) compared with our study sample where the MoCA was repeated approximately 42 months apart. The increased scores after yearly evaluations of the MoCA in healthy older adults may be attributed to practice effects, which has also been noted in the MMSE literature (Cooley et al., 2015; Hensel, Angermeyer, & Riedel-Heller, 2007; Jacqmin-Gadda, Fabrigoule, Commenges, & Dartigues, 1997). In this study, no significant practice effects were observed on the MoCA over 42 months. In fact, cognitively intact participants demonstrated a 0.17 annual decrease in the MoCA score. This may be attributed to other factors such as aging, baseline general cognitive ability, length of the interval between test administrations, undetected cognitive decline, and lower frequency of test exposure in this study, all of which have been previously shown to influence longitudinal change in cognitive assessments (Salthouse, 2013, 2014).

To date, there is limited literature on change in MoCA score over time in MCI populations. In this study, presumably healthy older adults were diagnosed with MCI at follow-up. These participants demonstrated a small but significant decline on the MoCA 3.5 years after the initial administration. Participants in this sample did not endorse cognitive complaints at baseline; however, presence of significant differences in MoCA scores at baseline between those later diagnosed with MCI or normal aging (through retrospective review), raises the possibility of subtle cognitive changes at the time of the initial MoCA administration in the MCI group.

The mean MoCA score in the MCI group at baseline was 23, which would suggest impairment using standard criteria (Nasreddine et al., 2005). However, prior examination of the MoCA in a similar cohort suggested the need for caution when applying recommended cut scores to diverse populations based on factors such as age, education, and ethnicity (Gluhm et al., 2013; Luis, Keegan, & Mullan, 2009; Rossetti et al., 2011; Waldron-Perrine & Axelrod, 2012). The MCI group was significantly older than the cognitively intact group, which may in part explain the significant decline in scores over time in the former group (Gluhm et al., 2013). The sample was also relatively highly educated (mean education = 15 years) and it is possible that individuals with limited education may manifest a different rate or pattern of change in MoCA over time than reported here. Although the MCI group had a lower representation of non-Hispanic Caucasians, our analysis did not reveal an interaction between ethnicity and MoCA score. However, it is possible that the MoCA’s detection of change over time differs in certain sociodemographic groups. Even when accounting for age and education, participants with MCI demonstrated a significant change in MoCA score over the 42-month period in this study.

A RCI indicated that a MoCA score should exceed ±1.73 points to represent clinically meaningful difference. This threshold increases the likelihood that an individual’s change in performance reflects actual change in cognitive ability rather than related to extraneous factors. Using this metric, approximately a fourth of MCI participants (~13/53) exhibited reliable decline over the 3.5-year period. This is in keeping with prior findings suggesting that MCI is a heterogeneous syndrome showing diverse trajectories, with some patients declining while others remain stable or even improving over time (Ganguli, Dodge, Shen, & DeKosky, 2004; Lopez et al., 2007). However, these findings may also be related to study sample bias and lack of diagnostic clarification at baseline, which is elaborated below. Additionally, the MoCA was only administered at two time points 42 months apart, which limits data on closely assessing the trajectory of MCI.

There are several limitations in this study. There is an inherent bias in recruiting from a community-based sample for longitudinal neurocognitive evaluations. Although we restricted the cohort to presumably cognitively healthy individuals at baseline by excluding individuals who endorsed subjective cognitive complaints, the diagnosis was only formally established at follow-up which restricts our understanding of the patient’s objective cognitive status at baseline. This is especially a concern as half the original sample was diagnosed with MCI at follow-up, which is greater than expected from the general population. The retest interval was longer than most studies evaluating practice effects. It would be useful to track MoCA performance in MCI, as well as in groups with dementia, over a longer period of time in addition to yearly follow-ups to help determine the utility of the MoCA as a measure of disease progression.

In conclusion, we found a significant and reliable decline in MoCA scores over a 3.5-year period in individuals retrospectively diagnosed with MCI, while healthy controls did not show appreciable change over time. Age and education played a role in the pattern of MoCA change over time. The absence of practice effects on the MoCA in the cognitively intact sample warrants further evaluation given the prior research showing contrary results on the MoCA (Cooley et al., 2015) based on the length of time between administrations of the MoCA and the impact of the initial exposure to the MoCA, as has been investigated with other cognitive measures (Salthouse, 2014).

To our knowledge, this is the first study to investigate MoCA performance over time in a community-dwelling population that included MCI and cognitively intact participants. The utility of MoCA as a screening tool that can detect change over time is useful from a clinical perspective, as the MoCA is typically administered in clinical care settings such as primary care, stroke clinics, and community settings as a means to screen for cognitive function and assess for change over time (Gluhm et al., 2013; Wong et al., 2012).