The MoCA Memory Index Score: An Efficient Alternative to Paragraph Recall for the Detection Of Amnestic Mild Cognitive Impairment

Abstract

Objective

To compare ability of two measures of delayed memory (word list, story paragraph) to discriminate normal cognition from amnestic Mild Cognitive Impairment (aMCI).

Methods

Demographic, neuropsychological, and diagnostic data contributed by 34 Alzheimer’s Disease Centers (ADCs) to the National Alzheimer’s Coordinating Center (NACC) characterized 2,717 individuals with a diagnosis of either Normal Control (NC) (n=2,205) or aMCI (n=512). The Montreal Cognitive Assessment Memory Index Score (MoCA-MIS) assessed delayed word recall, and the Craft Story 21, delayed story recall. Logistic regression and receiver operator characteristic (ROC) curves controlling for age, sex and education assessed the ability of each test to differentiate NC and aMCI.

Results

The MoCA-MIS had significantly better sensitivity and specificity (area under the ROC curve 0.83 versus 0.80, p = 0.004). At sensitivity 80%, the specificity of the MoCA-MIS was 69.1%, compared to 62.8% for the Craft story.

Conclusions

These data suggest that the MoCA-MIS, a recall score from items within the MoCA, is better at discriminating normal cognition from aMCI than the Craft Story. Word recall may be an efficient alternative to paragraph recall for diagnostic screening within clinical practice and research settings.

INTRODUCTION

Mild Cognitive Impairment (MCI) describes individuals for whom functional capacity is relatively intact, but who, on objective testing, show cognitive decline in at least one area of neuropsychological functioning.¹ These individuals are at high risk for progression to clinical dementia.² Professionals in a general practice setting and those investigating Alzheimer's disease (AD) in clinical trials and cohort studies often require identification of individuals in early stages of disease. A deficit in episodic memory assessed by delayed recall of new information is the most common domain of cognitive function which triggers a diagnosis of MCI.³ Therefore, a sensitive measure of delayed recall is an especially important component of neuropsychological assessment. Delayed paragraph recall tasks are commonly used for this purpose, but word list recall tasks are also used. A study comparing story and word list recall in subjects with aMCI found that a word-list recall task performed better than story recall in discriminating subjects with high versus low levels of the amyloid-β PET biomarker indication of a pre-clinical Alzheimer’s neurodegenerative process.⁴ The study also found that word-list recall was significantly associated with hippocampal volume while story memory was not.

The word list recall task embedded in the Memory section of the Montreal Cognitive Assessment (MoCA)⁵ allows calculation of a Memory Index Score (MoCA-MIS) which includes free and cued recall and is typically affected early in neuropathological changes associated with AD.⁶ The MoCA-MIS may be a useful tool for differentiating amnestic MCI (aMCI) from normal cognition in older adults. In this study, we investigate whether the MoCA-MIS performs comparably to delayed paragraph recall for differentiating aMCI from normal cognition using a large sample of subjects carefully characterized by Alzheimer’s Disease Centers (ADCs) that contribute to the National Alzheimer’s Coordinating Center (NACC) database.

METHODS

Participants

The NACC includes data collected from 34 ADCs funded by the National Institute on Aging (NIA). The current version (V3) of the NACC Uniform Data Set (UDS), initiated in March 2015, includes the MoCA and Craft Story 21. We report data from March 2015 through February 2016, including subjects with a diagnosis of normal cognition (NC) and either single domain aMCI or multiple domain aMCI at the first assessment in which the MoCA and Craft Story were administered. Participants were classified as NC or aMCI by the ADC from which the data originated. Diagnoses were based on clinical assessments according to criteria prescribed by NACC; ADC personnel who performed the assessments made diagnostic judgements independent of biomarker and imaging studies.⁷ In this study, we included 2,717 participants who were given a diagnosis of NC (n=2,205) or aMCI (n=512, 199 single-domain and 313 multi-domain) and completed the MoCA, Craft Story 21, and Clinical Dementia Rating (CDR) scale.⁸

Procedures

Diagnoses of MCI were based on the presence of concern about a change in cognition obtained from the subject or informant, or indication of a change in performance on longitudinal neuropsychological testing. In addition, a diagnosis of MCI requires relatively preserved basic activities of daily living and no more than mild functional impairment on complex tasks.¹ Finally, the diagnosis requires objective identification of at least one area of cognitive impairment. Amnestic MCI (aMCI) is defined as MCI with impairment in memory alone (single domain) or impairment in memory and at least one other area of cognition (multi-domain). Each ADC had individual Institutional Review Board (IRB) approval and written informed consent was obtained for each participant. Research use of the NACC database was approved by the University of Washington IRB. The San Diego State University IRB waived the requirement for a formal proposal due to non-human interaction and use of de-identified data.

Measures

The Montreal Cognitive Assessment⁵ is a screening instrument composed of eight sections including visuospatial/executive, naming, memory, attention, language, abstraction, delayed recall, and orientation. The memory section provides two trials to learn a word list of five nouns followed by a delay in which subjects are asked additional questions from other sections of the MoCA (i.e., attention, sentence repetition, letter fluency, similarities). The delay is variable, but estimated at five minutes followed by free recall of the 5-word list. This is followed by a category-cued semantic recall condition, and, finally, a multiple choice-cued recall from presentation of the correct item paired with two items within the same category but not on the list. Only the points earned in the delayed free recall condition of the memory section (1 point per correct word) are added to the MoCA total score. The MoCA-MIS⁶ includes points for the free recall condition and the cued conditions (3 points for each word on free recall, 2 for each on category-cued recall, 1 for each on multiple-choice recall). The MoCA-MIS score is part of MoCA version 8.1 and maybe downloaded from www.mocatest.org. We also considered two alternative scoring methods, one based on free recall data alone (3 points per correct word), and a second based on the sum of the free (3 points per word) and the category-cued, semantic (2 points per word) recall performance.

The Craft Story 21 is a paragraph story recall test that replaced the Logical Memory subtest of the Wechsler Memory Scale-Revised⁹ used in prior versions of the UDS. As with Logical Memory, the examiner reads the story aloud once, then asks the participant to repeat the details of the story in the same words read by the examiner or in his/her own words. Points for verbatim (exact content words) and paraphrase recall (similar contextual story units) are summed individually. After approximately 20 minutes, the participant is asked to recall the story again. The delay interval is filled primarily with measures involving visual figures and numbers, some of which have verbal components.¹⁰ These include the Benson Complex Figure recall, number span forward and backward, and Trail Making A and B. One verbal test, category fluency (animals and vegetables), also is administered during the delay. If the subject recalls no items from the Craft Story after the delay, the examiner provides a cue (‘It was a story about a boy’). For this study, only the delayed paraphrase recall score (range: 1–25) was used in the analyses.

The Clinical Dementia Rating scale⁸ is based on a numerical scale (range: 0.0 to 3.0) used to quantify the severity of symptoms of dementia, primarily for dementia associated with AD. According to the NACC UDS Researcher’s Data Dictionary (Version 3.0, March 2015), zero denotes no impairment, 0.5, questionable impairment, 1.0, mild impairment, 2.0, moderate impairment, and 3.0, severe impairment.

Statistical Methods

Analyses were performed in R studio version 0.99.491.¹¹ Clinical diagnosis was the main outcome variable, dichotomized into whether the subject had an aMCI or NC diagnosis. We used the non-parametric Wilcoxon rank-sum test to compare groups on continuous variables and the non-parametric Fisher’s exact test to compare categorical variables. Separate logistic regression models of clinical diagnosis were fit using the MoCA-MIS, the Craft Story paraphrase recall, and the two alternative scorings of the MoCA word recall data. The covariates age, sex and education were controlled for in all models. The discriminate utility of the resulting models was summarized using receiver operating (ROC) curves. Areas under the ROC curves were compared using the Delong’s non-parametric z test.¹²

RESULTS

Demographic and clinical characteristics of the diagnostic groups (NC, aMCI) are provided in Table 1. The CDR global scale classified 95.0% of the NCs as CDR=0 (normal) and 93.6% of the aMCIs as CDR > 0 (cognitively impaired). The NC group was slightly younger than the aMCI group. There were more females than males in the NC group and more males than females in the aMCI group. No significant differences were observed for education and race; the average education level was a bachelor’s degree, and the primary reported race was white.

Table 1.

Demographic /clinical characteristics for subjects with a diagnosis of Normal Cognition or amnestic Mild Cognitive Impairment (aMCI).

	Normal		aMCI
	n = 2,205		n = 512
	Mean	SD	Mean	SD
* Age	72.7	10.0	76.1	9.2
Education	16.3	2.8	16.0	3.1
	Sample size	%	Sample size	%
* Sex
Male	758	34.4	278	54.3
Female	1447	65.6	234	45.7
+ Race
White	1852	84.5	437	85.5
Non-white	341	15.6	4	14.5
CDR global
Normal	2,094	95.0	33	6.4
Cognitively Impaired	110	5.0	479	93.6

^*p < .001

⁺Race unknown for 12 Normal and 1 aMCI subject;

CDR = Clinical Dementia Rating

The mean MoCA-MIS total score was significantly higher in the NC group than the aMCI group (M = 12.2, SD = 2.8 versus M = 7.8, SD = 3.8, p < 0.001), as was the Craft Story paragraph recall score (M = 14.2, SD = 5.2 versus M = 8.1, SD = 5.5, p < 0.001). Figure 1 shows a boxplot summary of the distribution of scores for the MoCA–MIS and the Craft Story paraphrase recall. Interestingly, Craft Story inter-quartile ranges of the aMCI and NC groups overlap, while for the MoCA-MIS, the lower quartile of scores for the NC group is equivalent to the upper quartile of the aMCI group, suggesting greater potential discriminant utility of the MoCA–MIS.

Figure 1.

Distribution of scores on the Montreal Cognitive Assessment-Memory Index Score (MoCA-MIS) and Craft Story 21 by clinical diagnosis of normal cognition and amnestic Mild Cognitive Impairment (MCI).

ROC curves controlling for age, sex and education are presented in Figure 2. The area under the ROC curve for the MoCA-MIS was modestly but statistically significantly greater than the area under the ROC curve for the Craft Story paraphrase recall (0.83 versus 0.80, z = −2.84, p = .004). The curves are most divergent within the critical range of optimal sensitivity and specificity of the two instruments (Figure). For example, at 80% sensitivity, the specificity of the MoCA-MIS was 69.1%, while the specificity of the Craft Story was 62.8%.

Figure 2.

ROC curves for the Montreal Cognitive Assessment-Memory Index Score (MoCA-MIS) and Craft Story 21 discrimination of normal cognition from amnestic Mild Cognitive Impairment (MCI).

We also investigated the performance of two alternative memory performance scores calculated from the MoCA word recall items, one based on only the free recall information (3 points per correct word), and one based on the free recall (3 points per correct word) plus semantic cued recall (2 points per correct word). There was no difference in area under the curve for the MoCA word recall scorings, (range 0.82 to 0.83), and all scorings had greater area under the curve than the Craft Story (all p-values < 0.022).

DISCUSSION

The MoCA–MIS provided a greater level of specificity for a given level of sensitivity in detecting individuals with aMCI than the Craft Story paraphrase delayed recall. These results suggest that for screening to distinguish aMCI from NC, a score derived from word recall items may be preferable to story recall given that word recall performs demonstrably better and is substantially less burdensome for the patient, and for the psychometrist to administer and score.

Both the Craft Story and the MoCA word list are introduced to the subject as tests of memory for new information that will be revisited to elicit recall after a delay. Likely due to hippocampal neuropathology associated with AD,¹³ subjects with aMCI are expected to have more difficulty with retrieval of new information after a delay than NC subjects, and possibly, to rely on cueing to a greater extent. MoCA memory testing differs from the Craft Story delay recall in several ways. First, there are differences in the timing and nature of cueing available to aid recall. Opportunities for recall via semantic and multiple choice cueing are offered on the MoCA word list after the delayed free recall condition. Multiple relationships between content elements of the Craft Story (e.g., mother/son; soccer/field/game; dogs/barking) provide a framework and, therefore, some degree of inherent cueing prior to the delay, as well as one cue after the delay if the subject has no memory of the story. However, results indicate that when comparing the two diagnostic groups, cueing does not improve the sensitivity of the MoCA word list over the free recall. That is, performance on the MoCA free recall score provided a level of specificity no better than, but similar, to the MoCA-MIS, and better than the specificity of the Craft Story delayed recall.

Other differences between the MoCA-MIS and the Craft Story are the length of the delay interval and the nature of the questions posed during the delay (i.e., potential interference). The delay for the story is approximately 20 minutes, and for the MoCA, approximately 5 minutes. Crocco et al.¹⁴ found that controlling for global memory deficits, aMCI subjects had significantly greater proactive and retroactive interference effects than NC subjects on a unique recall paradigm. The longer story delay is filled with tests of visuospatial memory, number /letter sequencing, and category fluency. The MoCA word list delay consists of 3 tests of attention (number span, letter vigilance, serial number subtractions) and 3 measures of language (sentence repetition, letter fluency, verbal abstractions). Both working memory and other types of executive skills have been identified as early changes in prodromal AD.^15–17 The focus of the MoCA intervening tasks on measures often associated with executive functions (i.e. attention /vigilance; rapid shifting from one task to another) may increase difficulty retrieving the word list, and, therefore, favor the NCs. Since these problems with executive functions may be subtle and go undetected in single domain aMCI and are likely a second area of dysfunction in at least some of the multiple domain aMCI in this study, they may have contributed to the signal differentiating NC from aMCI.

The study has several potential limitations

Both MoCA and paragraph recall data were available at the time of diagnosis using NACC UDS criteria and may have influenced diagnostic decisions in favor of one or the other of the instruments in the ROC curve analyses presented here. However, we expect that this potential bias would be stronger for the paragraph recall measure for two reasons. First, while the MoCA total score is available at the time of diagnosis, scores from the MoCA-MIS are not. Second, paragraph recall has long been considered an important component of the NACC battery. Paragraph recall is commonly used along with level of functional impairment and other measures of memory to determine diagnosis, but its predominance would tend to bias in favor of story recall in determining a diagnosis of NC or aMCI. As a final limitation, we note that the generalizability of the study is limited by a NACC sample that is more highly educated and composed of a higher frequency of white subjects than the population at large.

The ability to differentiate NC subjects from those with aMCI is important given the need to identify the earliest cognitive changes associated with AD neuropathology. Improved ability to differentiate normal cognition and prodromal AD, as well as test brevity, burden on subjects, and ease of administration and scoring, constitute important features contributing to methodological decision making for clinical research, trials, and practice. The MoCA is an easily administered 10-minute global cognitive assessment instrument. Remarkably, both the MoCA-MIS and a scale calculated using only the free recall information performed as well as or better than the relatively more cumbersome paragraph recall test. It may be that considering several factors (e.g., encoding and cueing options, interference) in addition to the type of task (story versus word list) used to assess memory, can provide clues in identifying the most efficient and accurate ways to screen for subtle shifts from normal to impaired memory.

Table 2.

Scores on the Montreal Cognitive Assessment (MoCA) and Craft Story for subjects with a diagnosis of Normal Cognition or amnestic Mild Cognitive Impairment (aMCI).

	Clinical Diagnosis
	Normal		aMCI
	n = 2,205		n = 512
	Mean	SD	Mean	SD
MoCA total	26.2	2.9	22.3*	3.5
MoCA-Memory Index Score (MoCA-MIS)	12.2	2.8	7.8*	3.8
Craft Story Recall (Paraphrase Scoring)	14.2	5.2	8.4*	5.5

^*p ≤ .001