It's all about cognitive trajectory: Accuracy of the cognitive charts–MoCA in normal aging, MCI, and dementia

Abstract

Background

The Montreal Cognitive Assessment (MoCA) is an established cognitive screening tool in older adults. It remains unclear, however, how to interpret its scores over time and distinguish age‐associated cognitive decline (AACD) from early neurodegeneration. We aimed to create cognitive charts using the MoCA for longitudinal evaluation of AACD in clinical practice.

Methods

We analyzed data from the National Alzheimer's Coordinating Center (9684 participants aged 60 years or older) who completed the MoCA at baseline. We developed a linear regression model for the MoCA score as a function of age and education. Based on this model, we generated the Cognitive Charts‐MoCA designed to optimize accuracy for distinguishing participants with MCI and dementia from healthy controls. We validated our model using two separate data sets.

Results

For longitudinal evaluation of the Cognitive Charts‐MoCA, sensitivity (SE) was 89%, 95% confidence interval (CI): [86%, 92%] and specificity (SP) 79%, 95% CI: [77%, 81%], hence showing better performance than fixed cutoffs of MoCA (SE 82%, 95% CI: [79%, 85%], SP 68%, 95% CI: [67%, 70%]). For current cognitive status or baseline measurement, the Cognitive Charts‐MoCA had a SE of 81%, 95% CI: [79%, 82%], SP of 84%, 95% CI: [83%, 85%] in distinguishing healthy controls from mild cognitive impairment or dementia. Results in two additional validation samples were comparable.

Conclusions

The Cognitive Charts‐MoCA showed high validity and diagnostic accuracy for determining whether older individuals show abnormal performance on serial MoCAs. This innovative model allows longitudinal cognitive evaluation and enables prompt initiation of investigation and treatment when appropriate.

Key points

• Our innovative model applied to the MoCA showed high validity and diagnostic accuracy for determining whether older individuals show abnormal performance on serial cognitive testing. The Cognitive Charts‐MoCA allow longitudinal evaluation of a patient's cognitive performance over time.

• This is a simple and easy to use tool able to discriminate between healthy cognition, mild cognitive impairment, and dementia using the MoCA.

• The Cognitive Charts‐MoCA should allow earlier detection of cognitive changes than current methods, hence possibly accelerating initiation of anti‐dementia treatments.

Why does this paper matter?

Because early detection and treatment of major neurocognitive disorders such as Alzheimer's disease begin with a screening tool that quickly determines whether the cognitive trajectory is compatible with normal aging, mild cognitive impairment or dementia. The Cognitive Charts‐MoCA meet these requirements with high sensitivity (SE)‐specificity (SP) ratios to help frontline clinicians make evidence‐based decisions.

BACKGROUND

Dementia is of pandemic proportions. ¹ Task forces suggest cognitive screening be part of an annual visit, ² performed early and timely with longitudinal data points. ³ , ⁴ , ⁵ Numerous studies show cognitive impairment is unrecognized in 27%–81% of cases in primary care. ⁵ Recent guidelines on mild cognitive impairment (MCI) have suggested that subjective cognitive complaints alone are insufficient to screen for MCI but authors have recommended validated cognitive assessment tools for early detection of cognitive impairment. ² , ⁶

We previously validated ready‐to‐use cognitive charts for follow‐up of age‐associated cognitive decline (AACD), analogous to pediatric growth charts, based on the Mini‐Mental State Examination (MMSE). ⁷ The Cognitive Charts‐MMSE were designed to help clinicians position their patient's cognition using normative data and follow their trajectory over time. ⁷ The MMSE however, has ceiling effects and poor SE for identification of MCI or mild neurocognitive disorder. ⁸ , ⁹ The Montreal Cognitive Assessment (MoCA) has demonstrated better diagnostic accuracy in MCI, has less ceiling effect, and a higher test–retest reliability than MMSE. ¹⁰ , ¹¹ , ¹² The MoCA is considered the best cognitive screening instrument for amnestic MCI and Alzheimer's and is available in 100 languages.

We applied the Cognitive Charts' method to three databases to generate charts based on the MoCA (Cognitive Charts‐MoCA) for analysis of cognitive status and follow‐up of trajectory on repeated tests. We aimed to discriminate healthy controls (HC) from those with MCI and dementia.

METHODS

Study population

The National Alzheimer Coordinating Center (NACC) ¹³ database contained 10,411 subjects, 9684 of which were aged 60 years or more and measured yearly for up to 4 years. Of these, 5168 remained normal (i.e., no cognitive deficits) during follow‐up and were used to construct the charts. Our classification algorithm pertaining to initial evaluation was then tested on 9684 participants and prospective algorithm was tested on 4002 participants who were not prevalent cases of dementia. The Alzheimer's Disease Neuroimaging Initiative (ADNI) ¹⁴ database contained 1258 initial evaluations of participants. It was used to validate the classification algorithm pertaining to initial evaluation as was the Hudon database (425 participants). Additionally, the algorithm was validated using 129 participants who were not prevalent cases of dementia and who were evaluated annually up to 5 years. All databases used standard criterion for identification of MCI and/or dementia (Table 1).

TABLE 1.

Characteristics of the study populations from three databases

Characteristics a	Healthy controls	Mild cognitive impairment	Incident dementia
Participants from NACC (n = 9684)	n = 5168	n = 2015	n = 194
Age at start of the study, year	73.66 ± 7.67	75.59 ± 7.96	76.82 ± 7.70
Education, year	16.24 ± 2.77	15.94 ± 2.97	16.35 ± 3.09
MoCA score at start of the study c	26.11 ± 2.88	22.55 ± 3.56	20.95 ± 3.54
MoCA score at end of the study	26.13 ± 2.94	22.27 ± 3.68	18.06 ± 4.39
Participants from ADNI (n = 1258)	n = 477	n = 580	n = 2
Age at start of the study, year	75.06 ± 6.75	73.47 ± 7.19	71 ± 9.9
Education, year	16.90 ± 5.92	16.32 ± 5.58	17 ± 1.4
MoCA score at start of the study	25.12 ± 3.72	23.02 ± 3.59	20.5 ± 2.1
MoCA score at end of the study	24.88 ± 4.20	21.65 ± 5.19	17.5 ± 2.1
Participants from Hudon b (n = 425)	n = 141	n = 214	n = 31
Age at start of the study, year	71.04 ± 6.80	72.70 ± 6.71	73.94 ± 5.60
Education, year	14.77 ± 3.34	13.37 ± 3.95	12.03 ± 3.74
MoCA score at start of the study	26.47 ± 2.16	23.12 ± 3.38	22.10 ± 2.86
MoCA score at end of the study	26.45 ± 2.11	23.21 ± 3.32	19.26 ± 3.42

Abbreviations: ADNI, Alzheimer's disease neuroimaging initiative; MoCA, Montreal cognitive assessment; NACC, national Alzheimer's coordinating center.

^a Healthy controls remained healthy for the duration of their follow‐up. Mild Cognitive Impairment (MCI) were diagnosed MCI at least once during the study but never progressed to dementia. Incident Dementia were diagnosed at least once with dementia during follow‐up but not at first evaluation. Prevalent Dementia were diagnosed with dementia at their first evaluation.

^b Hudon is a local French‐Canadian database.

^c MoCA is scored out of a maximum of 30.

Statistical analysis

Repeated measures regression was performed on HC's MoCA along with age and education to linearize HC's trajectory and determine factors in the relationship between Cognitive Quotient $\left(\mathit{\text{QuoCo}}=\frac{\mathit{\text{MoCA score}}}{\mathit{age}}\times 1000\right)$ and standardized age (S _A  = Age‐ years of education) (Supplementary Appendix 1a and similar methodology in Bernier et al. ⁷ ). A mathematical model was derived with minimal loss of accuracy for use in clinical practice: $\mathit{\text{QuoCo}}=696-5.81S_A$. Data suggested no further relation between MoCA and education above 18 years. This model was used to develop percentile curves and initial discriminant zones. Because the MoCA allows for discrimination between MCI and dementia, we adapted our methodology to provide zones for both conditions, maximizing the Youden index. We then compared the mean HC's cognitive decline versus those with MCI and dementia, and set the optimal interval between percentile lines to discriminate HC‐MCI and MCI‐dementia transitions, maximizing the index. From these results, we selected a set of percentile curves such that half intervals (distance between a full and dashed line) could be used to detect changes in cognition and full intervals (distance between two full lines) to detect dementia.

Our classification algorithm was internally validated using the NACC database. Because such estimates tend to be optimistic, external validation was performed on independent datasets. We applied the initial value classification on the ADNI dataset and the full initial and longitudinal algorithm on the French‐Canadian Hudon dataset. In all cases, we compared the results of the charts to the MoCA cutoffs (26/30 for MCI and 18/30 for dementia). ²⁵ SE, SP, and predictive values were calculated with appropriate confidence intervals (CI). Performance of the charts was further compared to the Receiver Operating Characteristic (ROC) curve of the MoCA score itself. Finally, sub‐group analyses were performed for age, sex, education, and initial MoCA score, comparing the performance of the charts in each subgroup to cutoffs.

RESULTS

AACD (QuoCo scores in relation to S_A) are plotted as percentiles (Figure 1). Intervals are intended to discriminate between AACD, MCI, and dementia. Therefore, the transition which is a decline greater than one half (but less than one full) interval from baseline suggests a change in status (HC to MCI, or MCI to dementia). A decline greater than one full interval represents two changes in status, thus dementia. The gray area indicates current cognitive status. If the first MoCA falls in the light gray zone, further investigations for MCI are warranted, and if it falls in the dark gray, dementia investigation performed. Once baseline is established, repeated MoCAs can be used such that further decline by a half interval suggests a change in cognition while a decline of a full interval suggests dementia. This tool is not recommended in patients aged less than 60 years old. Clinical cases are illustrated as Supplementary material (Supplementary Appendix 1b).

FIGURE 1.

Cognitive charts based on the MoCA (Cognitive Charts‐MoCA). Standardized age (S_A) is plotted in relation to Cognitive Quotient (QuoCo). Each percentile line (indicated far right) represents expected age‐related decline on the Cognitive Charts‐MoCA scores and are spaced to detect abnormal cognitive changes. A decline of more than any half intervals (equivalent to the distance between a full and dashed line) but less than any full could be used to detect changes in one step of cognition (Normal‐MCI or MCI‐Dementia) and a decline of more than anyone full intervals (equivalent to the distance between two full lines) to detect dementia. Gray zones discriminate status: normal (white), MCI (light gray) and dementia (dark gray). These charts cannot be used for patients younger than 60 years.

Accuracy of the charts is outlined in Supplementary Appendix 1c. In the NACC where 4002 non‐prevalent cases of dementia were evaluated up to 4 times for a total of 5786 transitions, 89% of dementia evaluations were correctly identified by the transitions (while 11% were identified as MCI) and 61% of MCI evaluations were correctly identified by the transitions (while 27% were identified as dementia). All Youden indexes for transitions were superior to fixed cutoffs. Transitions showed up to 96% SE and 79% SP for the detection of dementia. External validation confirmed most of the results obtained in the NACC.

The ROC curve presented in Figure 2A shows that our transition algorithm is superior to cutoffs in confirming a deficit. Additionally, it better confirmed the absence of dementia than cutoffs. Figure 2B shows charts' performance in different strata of participants. Use of transitions attenuates variations among age and education as illustrated by a more stable polygon than cutoffs. Cutoff data varies significantly across subgroups, particularly when comparing individuals along education or baseline cognition (e.g., very low Youden of 0.084 for individuals with less than 12 years of education versus 0.51 if equal or greater than 12 years). By contrast, the Youden factor varies from 0.52 to 0.68 in both groups using transitions. For all categories, the Youden factor of the charts yields better results than cutoffs and much less variations between categories.

FIGURE 2.

(A) The ROC curve (black line) and confidence interval (dashed lines) for the MoCA. The ROC curve shows the performance of the MoCA score at various cutoffs. Performance using the 26‐points' cutoff is illustrated as an X along the curve (Point B). The diagnostic performance of the Cognitive Charts is presented as the CI (Cross at Point A). The positive and negative likelihood ratios are represented by lines C and D, respectively. The region where point A is situated with regards to both lines C and D indicates that the Cognitive Charts performed better at confirming the presence of a cognitive impairment than the single MoCA cutoff. ³⁰ (B) This graph illustrates performance of the CC–MoCA versus cutoffs in ‘any change to an abnormal state’ so for example from ‘normal to MCI’ or ‘MCI to dementia’. It indicates that the CC–MoCA are far more stable, and in various participants' subgroups, in comparison to cutoffs.

DISCUSSION

Similar to ‘growth charts’ used in pediatrics, we applied the charts' method to MoCA data to offer a validated and accurate technique to position any patient based on age, education, and MoCA scores, and track its longitudinal profile of cognitive decline over time. In turn, this can prompt earlier intervention and treatment for an adult who ‘fell off’ the curve. This is the first time MoCA test scores are integrated into a ready to use set of charts, a method, which appears superior to standard cutoffs as shown by Youden indexes. By converting MoCA into cognitive quotient (QuoCo) plotted against standardized age (S_A), the interpretation is modulated for age and education. Abnormal decline on the charts should prompt further detailed investigation according to current guidelines while the absence of decline reliably identifies those individuals who do not need further cognitive work up. ² , ³ For more information and use of charts QuoCo©, see quoco.org.

The charts may help better interpretation of cognitive performances. In addition to variable cutoffs integrated as zones on the charts, plotting a patient's trajectory may add new perspectives. First, the patient visualizes his performance and is used as his own comparison on repeated measures. Second, clinicians may be prone to obtain a baseline MoCA while their older patients still have normal cognition in order to compare with future changes. ¹⁵ Third, the stability of scores' interpretation using the charts may help detect abnormal patients who are younger and/or highly educated but still have scores over 25 and ‘normalize’ older and/or less educated patients that show scores lower than 26. By tracking cognitive decline over repeated measures, physicians will rapidly see if something's wrong with their patient's cognitive trajectory. In turn, this may favor early investigation, better differential diagnosis, rapid counseling, control of risk factors, and initiation of therapies.

MoCA variations over time have not been studied. Among healthy adults MoCA scores were previously shown as stable over 3–4 years, ¹⁶ decreasing ¹⁷ or increasing. ¹⁸ Practice effects, difference in mean number of years of education between studies, variable criteria and diagnosis of MCI as well as variation in mean age between normal and MCI groups can explain such discrepancies. Incorporation of demographic variables into MoCA corrections provides better diagnostic classification. ¹⁹ By integrating age and education into the charts, we were able to smooth out most of these limitations.

Our results are in accordance with previous analyses of the impact of age and education on the MoCA. Mean score in large samples of older adults decrease with age ²⁰ , ²¹ and many experts suggest that a single cutoff regardless of age may cause over‐estimation of cognitive impairment, especially in the older adults. ²² Scores are lower as age increases and higher with higher education. ²³ , ²⁴ Based on these studies, as much as 26.1%–49% of scores' variability is attributed to age and education. A Cochrane review ²⁵ examined diagnostic accuracy of the MoCA for detecting dementia when using a cutoff of 26. In these studies, the MoCA showed high SE (94%) and above, but low SP (60% and below). As expected, SE increased and SP decreased when the MoCA cut‐point was higher. SE/SP at baseline and for any clinical state using charts presented here were 84%/81% (Youden 0.65). This represented a better performance than single cutoff 65%/91% (Youden 0.56). Trajectory intervals used for follow‐up assessment of change reached a SE of 89% with an SP of 79%, while the use of the single cutoff performed respectively at 82% and 68%. The Youden index is 0.68 for the charts' trajectory compared to 0.50 for the single cutoff.

Although dementia screening itself is not recommended in most guidelines (USPSTF, ⁵ 5th CCCDTD ³ ) there is a trend toward testing of older individuals who present for reasons other than memory complaints. ²⁶ The USA's Annual Wellness Visit includes a mandatory assessment of cognition. ²⁷ A recent study showed that screening based on patients' complaints is prone to underdiagnosis. ²⁸ There are more preventive interventions possible than expected as 12 modifiable risks factors are now known. ²⁹ In addition to symptomatic treatments that may slow functional decline, a diagnosis of dementia can also reduce uncertainty for individuals and their families, facilitate access to services, and future planning. The charts are simple tools to optimize and assist interpretation of cognitive screening tests but they do not replace dementia evaluation which requires history, examination, laboratory, and imaging.

Our work is a practical application of the charts' method, building on our 2017 publication. ⁷ Nonetheless, the study has limitations. Charts can be generated based on other tests in order to improve its ability to predict an individual's status. But the method does not change the original psychometric properties of the test (i.e., internal consistency, reliability, and construct validity); they remain embedded into the creation of the charts. Clinicians should not diagnose MCI or dementia only based on this tool. Although the NACC is a large database, it only provides up to 4 years of yearly follow‐ups. This is partly due to the recent adoption of the MoCA as a screening tool. As such, there is very limited long‐term data on this test. Nonetheless, participants ranged from 62 to 101 years old, hence providing a representative age range where dementia is prevalent. Because of the frequent evaluations, nearly all dementia cases were previously screened as MCI. It is not clear how the charts perform for patients who transitioned from normal to dementia in less than a year. Different and more complex modeling choices may have provided better performance on the charts but at the expense of decreased external validation. Moreover, the external datasets (ADNI and Hudon) remain imperfect in that they did not follow the same design as the construction (NACC) dataset. Consequently, only the initial algorithm could be validated in ADNI while the full algorithm was validated in a smaller (Hudon) study; this may generate uncertainty in the validation estimates of the charts' accuracy. Finally, samples are large but not diverse in education, race, ethnicity, or language and this may limit widespread use. Further validation is necessary before we can conclude on the generalizability of the charts.

AUTHOR CONTRIBUTIONS

Patrick J. Bernier: Original idea, draft of the paper. Christian Gourdeau: Original idea, draft of the paper, statistical analyses and modeling. Jean‐Pierre Beauchemin, Philippe Voyer, Carol Hudon, Robert Laforce Jr: Draft of the paper. Pierre‐Hugues Carmichael: Statistical analyses and modeling.

CONFLICT OF INTEREST

The method to generate cognitive charts based on a cognitive test is under U.S. Patent No. 11.395.621 protection retained by Patrick J. Bernier and Christian Gourdeau. The other authors have no conflicts to report.

SPONSOR'S ROLE

This work was supported by La Chaire de recherche sur les aphasies primaires progressives–Fondation de la Famille Lemaire (app-ffl.ulaval.ca) and by an unrestricted grant from Boehringer‐Ingelheim.

Supporting information

Appendix S1. (a) Derivation of the model. (b) Clinical cases. (c) Accuracy of the cognitive charts based on three databases.

Bernier PJ, Gourdeau C, Carmichael P‐H, et al. It's all about cognitive trajectory: Accuracy of the cognitive charts–MoCA in normal aging, MCI, and dementia. J Am Geriatr Soc. 2023;71(1):214‐220. doi: 10.1111/jgs.18029