Montreal Cognitive Assessment — Single Cutoff Achieves Screening Purpose

Abstract

Background and Purpose

The study evaluated the performance between norm-derived age and education adjusted vs single cutoff scores of the Montreal Cognitive Assessment, Hong Kong version (HK-MoCA) in classifying cognitive impairment in Chinese older adults.

Methods

Total scores of HK-MoCA were collected from 315 subjects (128 with dementia, 122 with mild cognitive impairment (MCI) and 65 normal) attending a public district hospital-based cognition clinic from 2012 to 2017. The HK-MoCA total scores were evaluated using different cutoffs. Norm-derived age and education adjusted cutoff scores were at 16th, 7th, and 2nd percentiles. Comparison was made with the single cutoff scores validated in a local study with 21/22 for MCI and 18/19 for dementia.

Results

Single cutoff score of HK-MoCA differentiated MCI from normal with sensitivity of 0.861 and specificity of 0.723. To detect dementia, its sensitivity was 0.922, and specificity was 0.923. In identifying cognitive impairment, the sensitivity and specificity were 0.932 and 0.723, respectively. However, age and education adjusted cutoff scores achieved high specificities at all levels of cognitive impairment with trade-off of sensitivities. The accuracy of correctly classifying tested subjects into appropriate groups was 85.3% if single cutoff was used though the consistency between norm-derived cutoffs and expert diagnoses were only 59.0%, 54.2%, and 53.9% at 16th, 7th, and 2nd percentiles, respectively. The consistency decreased with older age and lower education level, and majority of misclassifications were false negatives.

Conclusion

HK-MoCA is a convenient screening tool to detect cognitive impairment. Administration time is relatively short, and it has incorporated essential cognitive domains. Single cutoff scores with inherent simple education adjustment achieved screening purpose of mild cognitive impairment and dementia in Chinese older adults.

Background

Dementia is a major neurodegenerative disorder and often begins with focal cognitive or behavioural disturbances. Poor memory is one of the commonest presenting complaints and other features include disturbances of behaviour, language, mood, personality or perception. Individuals concerned about their memory loss are mostly seen by primary care physicians. A cognitive screening measure with high sensitivity and specificity is essential for them to decide who needs a more detailed evaluation or make a referral for comprehensive geriatric assessments, including neuropsychological evaluations.

The Montreal Cognitive Assessment (MoCA) is a brief, useful and validated cognitive screening instrument with a cutoff score of 26 to differentiate mild cognitive impairment (MCI) or dementia from normal.¹ There is a one-point adjustment for individuals with formal education of 12 years or fewer. Systematic review highlighted the necessity for cross-cultural considerations when using the MoCA as a screening tool.² Some studies have revealed that the originally recommended cutoff score of 26 leads to a higher false positive misclassification especially on those with increasing age and/or low education. Moreover, the one-point correction for education has been debated as insufficient to compensate for educational differences.³ Demographically adjusted norms may help improve the diagnostic accuracy.

In Hong Kong, a local version of MoCA, the Hong Kong version (HK-MoCA), was validated in Chinese older adults.^4,5 A score of less than 22 is considered positive for screening and calls for further diagnostic assessment. However, another study raised the issue that the single cutoff score of HK-MoCA was associated with a substantially high misclassification rate especially in older and less-educated patients with stroke.⁶ The study reported that using norm-derived cutoff as reference, a single cutoff at 21/22 yielded a classification discrepancy of 55.8%, with the majority of the misclassifications being false positives and the highest rate among patients with the lowest education duration of zero to three years. These results advised against the use of single cutoff scores of HK-MoCA on patients with cognitive impairment due to vascular cause. However, the applicability of these findings to other patients with non-vascular causes is not known. Therefore, this study aimed at examining whether such findings also apply to a more heterogenous group of subjects with cognitive impairment due to various causes among Chinese older adults.

Methods

This was a cross-sectional study conducted in a public hospital-based cognition clinic from August 2012 to June 2017. It recruited a total of 315 community-dwelling and Cantonese-speaking Chinese adults aged 60 years or above. They were seen for suspected cognitive impairment and all of them had given informed consent. They were divided into three groups: subjects with dementia, subjects who met the criteria for MCI, and cognitively normal controls (NC). Patients were excluded if they had a history, as documented in medical records, of significant head trauma, subdural hematoma, neurodegenerative disorders, central nervous system infection, epilepsy, brain tumor, significant psychiatric disorders (such as major depression or schizophrenia), alcoholism, or substance abuse. Besides, persons with inability to use a pen or with communication barriers such as deafness or significant language or speech problem were also excluded. Lastly, patients diagnosed with an advanced stage of dementia, accordingly with Global Deterioration Scale (GDS) of six or above, were not recruited. This study was conducted in accordance with the Declaration of Helsinki and ethical approval was obtained from the Kowloon Central/Kowloon East Cluster Clinical Research Ethics Committee of the Hong Kong Hospital Authority.

All subjects were assessed using HK-MoCA and Cantonese version of Mini-Mental State Examination (CMMSE) at the first visit. Their basic demographic information such as age, gender, and education level were collected. In addition, cardiovascular risk factors and established vascular diseases including diabetes mellitus, hypertension, hyperlipidemia, coronary heart disease, and stroke, as well as clinical information about drinking and smoking habits were recorded. Diagnosis of MCI and subtypes of dementia were made by experienced geriatricians and psycho-geriatricians according to the Diagnostic and Statistical Manual of Mental Disorders 4th edition (DSM-IV) criteria for dementia and Petersen’s criteria for mild cognitive impairment.⁷ Patients with cognitive impairment were used to describe patients with MCI or dementia.

The norm-derived age and education adjusted cutoff scores of the HK-MoCA were based on a previously validated study of cognitively normal people aged 65 years or above.⁶ As designed in Wong’s study,⁶

the subjects were stratified with age as the first level, followed by education as second level. Age was divided into 3 strata: 65 to 69, 70 to 79, and 80 years or above. Nested within each age stratum were five education strata of 0 to 3, 4 to 6, 7 to 9, 10 to 12, and more than 12 years, except that for the age stratum of age 80 or above. In the latter stratum, there were only two education strata of 0 to 6 and more than 6 years because of the relatively small sample size in this group.

From these, cutoff scores at 2nd, 7th, and 16th percentiles were described for the psychometric classification for major (2nd percentile) and mild (16th percentile) neurocognitive disorders in the Diagnostic and Statistical Manual of Mental Disorders 5th edition as well as Petersen's revised diagnostic criteria of MCI (7th percentile).⁸ Table 1 showed the norm-derived corrected cutoff scores of HK-MoCA⁶ (with permission).

Table 1.

Age and Education Corrected Normative Data of Total Score of MoCA.⁶

	Education	Cutoff at Percentile
		16th	7th	2nd
Age 65–69 years	0–3	17	14	9
	4–6	19	18	13
	7–9	21	19	16
	10–12	22	20	17
	>12	25	23	21
		16th	7th	2nd
Age 70–79 years	0–3	15	14	11
	4–6	18	15	10
	7–9	20	18	15
	10–12	22	19	18
	>12	22	20	16
		16th	7th	2nd
Age ≥ 80 years	0–6	13	13	10
	>6	17	15	13

Notes: Impairment is determined when score ≤age and education corrected percentile cutoff. Reproduced with permission from Wong A, Law SN, Liu WY, et al. Montreal cognitive assessment one cutoff never fits all. Stroke. 2015;46(12):3547–355. Available from: https://www.ahajournals.org/doi/10.1161/STROKEAHA.115.011226?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub%20%200pubmed .6

Statistical Analyses

All statistical analyses were performed using the Statistical Package for the Social Sciences (SPSS) version 23.0 and a p-value <0.05 was considered statistically significant. Continuous data were presented as mean and standard deviation while categorical data were shown as frequencies and percentages. Demographic and the HK-MoCA total scores among the three groups were analyzed by one-way analysis of variance (ANOVA), Fisher’s exact test or chi-squared test, as appropriate.

The HK-MoCA total scores of the 315 subjects were evaluated using different cutoffs. Norm-derived age and education adjusted cutoff scores were at 16th, 7th, and 2nd percentiles. Comparison was made with the single cutoff of 21/22 for MCI and cognitive impairment and 18/19 for dementia. These single cutoff values were validated in a local study⁴ using inherent simple education adjustment of adding one point to subjects with six or fewer years of formal education. Using expert diagnoses as reference, classifications of participants’ performance by single or age and education adjusted cutoff scores were described as consistent when it was agreed with expert diagnoses. A false negative was defined when a participant who was diagnosed as impaired by an expert but classified as unimpaired by the tested cutoff scores. A false positive was defined when a participant was diagnosed as unimpaired by expert but impaired by the tested cutoff scores. Sensitivities and specificities were shown to describe the diagnostic ability of various methods of cutoff in identifying MCI, dementia and cognitive impairment. Area under curves (AUC) were compared to distinguish which cutoff method had higher diagnostic power. For those inconsistent cases, portions of false negative and false positive were explored.

Results

Demographic characteristics and HK-MoCA scores for the three cognition groups were summarized in Table 2. Based on expert diagnoses, the study sample included 65 cognitively normal controls, 122 MCI, and 128 subjects with dementia. Of all the subjects with dementia, 50.0% had Alzheimer’s disease, 16.4% had vascular dementia, and the remaining 33.6% had dementia of mixed etiology. Fifty-seven percent of recruited subjects were females. The mean (±SD) age of all subjects was 77.32±7.96 years and among which 136 subjects (43.2%) were ≥80 years old. Another 122 subjects (38.7%) were between 70 and 79 years of age and the remaining 57 (18.1%) were younger than 70. Regarding their education status, 156 (49.5%) subjects were illiterate or received three years or less, 69 (21.9%) received four to six years, 42 (13.3%) had seven to nine years, 28 (8.9%) achieved 10 to 12 years, and only 20 (6.4%) had more than 12 years of education. The mean years of education achieved were 4.79±4.70 and 71.4% of the subjects received only primary or below formal education.

Table 2.

Demographic Characteristics by Cognition Groups

Demographic Characteristics	NC (n=65)	MCI (n=122)	Dementia (n=128)	Total (n=315)	F	p-value
Gender					–	χ2=5.170; p=0.075
Female	36 (55.4%)	62 (50.8%)	74 (57.8%)	172 (54.6%)
Male	29 (44.6%)	60 (49.2%)	54 (42.2%)	143 (45.4%)
Age, years						<0.001*
<70	21 (32.3%)	25 (20.5%)	11 (8.6%)	57 (18.1%)
70–79	28 (43.1%)	47 (38.5%)	47 (36.7%)	122 (38.7%)
≥80	16 (24.6%)	50 (41.0%)	70 (54.7%)	136 (43.2%)
Average age	73.20±8.43	77.22±7.97	79.50±6.84	77.32±7.96	14.704
Years of education						<0.001*
0–3	18 (27.7%)	56 (45.9%)	82 (64.0%)	156 (49.5%)
4–6	18 (27.7%)	25 (20.5%)	26 (20.3%)	69 (21.9%)
7–9	11 (16.9%)	18 (14.8%)	13 (10.2%)	42 (13.3%)
10–12	9 (13.8%)	16 (13.1%)	3 (2.4%)	28 (8.9%)
≥12	9 (13.8%)	7 (5.7%)	4 (3.1%)	20 (6.4%)
Average years	6.91±4.98	5.28±4.68	3.24±4.02	4.79±4.70	15.549
HK-MoCA total score	23.86±5.40	17.32±4.31	12.15±3.93	16.57±6.21	155.032	<0.001*
CMMSE total score	27.06±2.67	21.84±3.99	17.00±3.92	20.94±5.32	163.348	<0.001*

Notes: Data are shown as frequency (percentage) while average values are expressed as mean ±SD. *Significant at 0.05.

Abbreviations: NC, normal controls; MCI, mild cognitive impairment; HK-MoCA, Montreal Cognitive Assessment, Hong Kong version; CMMSE, Mini-Mental State Examination, Cantonese version.

Statistically significant differences among the three cognition groups (NC, MCI, and dementia) were found in the variables of age (F_{[2, 312]}=14.704, p-value <0.001), years of education (F_{[2, 312]}=15.549, p-value <0.001) and HK-MoCA total score (F_{[2, 312]}=155.032, p-value <0.001). There was no significant difference among the three groups by gender (chi-squared test=5.170, p-value=0.075). In particular, demented subjects were significantly older (79.50±6.84 years; p-value <0.001) and had lower education level (3.24±4.02 years; p-value=0.002) than those of the other two groups.

To differentiate MCI from normal controls with a single cutoff, it had sensitivity of 0.861 with 95%CI (95%CI: 0.786, 0.917), specificity of 0.723 (95%CI: 0.598, 0.827) and the area under curve (AUC) of receiver operating curve (ROC) was 0.792 (95%CI: 0.719, 0.865) (Table 3). Using norm-derived age and education adjusted cutoffs, sensitivities, specificities and AUC were 0.287 (95%CI: 0.209, 0.376), 0.969 (95%CI: 0.893, 0.996) and 0.628 (95%CI: 0.549, 0.707) and 0.410 (95%CI: 0.322, 0.503), 0.908 (95%CI: 0.810, 0.965) and 0.659 (95%CI: 0.581, 0.737) when the 7th and 16th percentiles were selected respectively. To detect dementia from normal, single cutoff had sensitivity of 0.922 (95%CI: 0.861, 0.962), specificity of 0.923 (95%CI: 0.830, 0.975) and AUC of 0.922 (95%CI: 0.876, 0.969) while norm-derived cutoff at 2nd percentile had sensitivity of 0.359 (95%CI: 0.276, 0.442), specificity of 0.985 (95%CI: 0.917, 1.000) and AUC of 0.672 (95%CI: 0.598, 0.746). For identifying cognitive impairment from normal, the sensitivity, specificity and AUC were 0.932 (95%CI: 0.893, 0.960), 0.723 (95%CI: 0.598, 0.827) and 0.828 (95%CI: 0.760, 0.895) in a single cutoff, respectively in comparison with 0.572 (95%CI: 0.508, 0.634), 0.908 (95%CI: 0.810, 0.965) and 0.740 (95%CI: 0.680, 0.800) in age and education adjusted cutoff at 16th percentile.

Table 3.

Diagnostic Power Comparisons Between Different Cutoffs in MCI and Dementia Subjects

	Sensitivity (95%CI)	Specificity (95%CI)	AUC (95%CI)	p-value
MCI vs normal				0.000*,a
Single cutoff (21/22)	0.861 (0.786, 0.917)	0.723 (0.598, 0.827)	0.792 (0.719, 0.865)
7th percentile	0.287 (0.209, 0.376)	0.969 (0.893, 0.996)	0.628 (0.549, 0.707)
16th percentile	0.410 (0.322, 0.503)	0.908 (0.810, 0.965)	0.659 (0.581, 0.737)
Dementia vs normal				<0.000*
Single cutoff (18/19)	0.922 (0.861, 0.962)	0.923 (0.830, 0.975)	0.922 (0.876, 0.969)
2nd percentile	0.359 (0.276, 0.442)	0.985 (0.917, 1.000)	0.672 (0.598, 0.746)
Cognitive Impairmentb vs normal				0.000*,a
Single cutoff (21/22)	0.932 (0.893, 0.960)	0.723 (0.598, 0.827)	0.828 (0.760, 0.895)
7th percentile	0.496 (0.432, 0.560)	0.969 (0.893, 0.996)	0.733 (0.675, 0.790)
16th percentile	0.572 (0.508, 0.634)	0.908 (0.810, 0.965)	0.740 (0.680, 0.800)

Notes: *Significant at 0.05. ^aSingle cutoff is significantly higher than both 7th and 16th percentiles. ^bCognitive impairment (MCI + dementia).

Abbreviations: AUC, area under curve; MCI, mild cognitive impairment.

The accuracy of correctly classifying tested subjects into appropriate groups was 87.3% if single cutoff was used, though the consistency between norm-derived cutoffs and expert diagnoses were only 58.3%, 52.4%, and 54.9% at 16th, 7th, and 2nd percentiles respectively. Analyses also showed that consistency decreased with older age and lower education level. Almost all misclassifications were false negatives, with the highest rate ≤50.7% among older subjects and achieved lower education (zero to six years). Norm-derived cutoff scores tended to miss cases identified as impaired by experts in all age and education levels. On the contrary, based on the 16th percentile age and education adjusted cutoff scores, only six false positive cases were found in MCI patients. They were younger than 80 and received various years of education. Figure 1 showed the “discrepancy in classification” by expert diagnoses versus norm-derived cutoff scores across age and education groups. Hundred percent stacked bar charts were presented for easier appreciation of the overall trend of false negatives by using age and education adjusted cutoff scores. Considering the effect of age and education level separately, only education adjusted cutoff scores were statistically significant (p-value <0.05) on the consistency with expert diagnoses. Hence, education has greater impact on the agreement with expert diagnoses.

Figure 1.

Classification of agreement of norm-derived age and education adjusted cutoff scores of HK-MoCA with expert diagnoses at different percentiles.

Discussion

HK-MoCA is a handy screening instrument to detect cognitive impairment for its relatively short administration time and incorporation of most essential cognitive domains. Single cutoffs are commonly used to determine impaired performance on HK-MoCA. This study compared the performance of single cutoff and age and education adjusted cutoff scores of HK-MoCA for the purpose of cognitive screening.

Single cutoff scores of HK-MoCA have been validated and proven comparative to CMMSE in Chinese older adults in Hong Kong.^4,5 It has adopted a simple education adjustment by adding one point for an individual who has six years or fewer formal education. Compared to norm-derived age and education adjusted cutoff scores, single cutoff scores have less classification discrepancy to expert diagnoses and acceptable sensitivity. Moreover, this study included 128 demented subjects of which 50% had Alzheimer’s disease, 16% were of vascular cause and 34% of mixed etiology. This helps to support the usefulness of single cutoff scores in the screening of cognitive impairment due to various common causes in Chinese older adults. Conversely, this study showed that fine adjustment of age and education has caused a significant compromise of sensitivity and tightened the threshold of detection of cognitive impairment. For instance, among the cutoff scores at 16th percentile (Table 1), several were lowered to 13, 15, and 17 marks. This has resulted in higher number of false negative cases in most of the age and education levels.

The primary purpose of screening is to detect early disease or risk factors for disease in large numbers of apparently healthy individuals. As general practitioners and family physicians are the usual assessors and users of the screening instrument, it is important to keep HK-MoCA simple and easy for interpretation in order to facilitate their correct usage. A single cutoff would serve this purpose well. Besides, it also raises the difficulty for general practitioners and family physicians to explain to patients the meaning of 16th, 7th, and 2nd percentiles in screening when the norm-derived age and education adjusted cutoff scores are adopted.

Furthermore, using age and education adjusted cutoff scores has problem of comparability across age and time. For instance, a 79 year old man with 10 years of education scored 18 in HK-MoCA. According to the age and education adjusted cutoff, he was below the 2nd percentile and screened positive for dementia. However, in one-year's time when he became 80, if he stayed the same in repeat testing, his HK-MoCA total score would be above the 16th percentile cutoff (Table 1). This would raise confusion to the appropriate approach of further action be it a specialist referral or “wait and see approach”.

The present study showed that the single cutoff score had lower specificity and higher false positive rate when compared with age and education adjusted cutoffs. This should bear less consequence as a slightly high false positive rate meant more screened positive older adults would receive confirmatory assessment by specialists subsequently. However, a high false negative rate can have significant detrimental effect because of delayed detection. Early detection of cognitive impairment can facilitate early diagnosis and intervention. Patients, their families and even the society will benefit from early recognition of the condition, as there will be significant reduction of health-care burden and expenditures because of early implementation of care plan and intervention. Based on the above reasons, the use of single cutoff scores of HK-MoCA for screening purposes is encouraged.

Conclusion

HK-MoCA is a convenient tool in aiding the detection of cognitive impairment for its relatively short administration time and its composition with essential cognitive domains. In this study, a single cutoff for each level of cognitive impairment with inherent simple education adjustment has a better sensitivity when compared with norm-derived age and education adjusted cutoffs with a trade-off of slightly lowered specificity. Hence, single cutoff scores achieve the screening purpose of mild cognitive impairment and dementia in Chinese older adults.

Disclosure

The authors report no conflicts of interest in this work.