Reliability, Validity, and Optimal Cutoff Score of the Montreal Cognitive Assessment (Changsha Version) in Ischemic Cerebrovascular Disease Patients of Hunan Province, China

Abstract

Background/Aims

The goal of this study was to examine the reliability and validity of the Changsha version of the Montreal Cognitive Assessment (MoCA-CS) in ischemic cerebrovascular disease patients of Hunan Province, China, and to explore the optimal cutoff score for detecting vascular cognitive impairment-no dementia (VCI-ND) and vascular dementia (VD).

Methods

Three hundred and thirty-eight ischemic cerebrovascular disease patients (131 with normal cognition, 111 with VCI-ND, and 96 with VD) and 132 healthy controls were recruited. All participants accepted examination by the MoCA-CS, Mini-Mental State Examination (MMSE), and other related scales. A detailed neuropsychological battery was used for making a final cognitive diagnosis. SPSS 16.0 statistical software was used for reliability, validity examination, and optimal cutoff score detection.

Results

Cronbach's α of the MoCA-CS was 0.884, and test-retest and interrater reliability of the MoCA-CS were 0.966 and 0.926, respectively. MoCA-CS scores were highly correlated with MMSE scores (r = 0.867) and simplified intelligence quotients (r = 0.822). The results indicate that 1 point should be added for subjects with less than 6 years of education, and that the optimal cutoff score for detecting VCI-ND is 26/27 (sensitivity 96.1%, specificity 75.6%), whereas the optimal cutoff score for detecting VD is 16/17 (sensitivity 92.7%, specificity 96.3%).

Conclusion

The MoCA-CS has good reliability and validity, and is a useful cognitive screening instrument for detecting VCI in the Chinese population.

Introduction

The Montreal Cognitive Assessment (MoCA; http://www.mocatest.org) is a cognitive-screening test which was recently introduced to detect patients with mild cognitive impairment (MCI). These patients usually score within the normal range on the most widely used cognitive screening tool, the Mini-Mental State Examination (MMSE) [1,2]. In a period of 6 years, the original MoCA has expanded from two versions (English and French) to more than 30 different versions, has become utilized in many more countries, and is now the most popular cognitive screening tool for detecting MCI.

Cerebrovascular disease is the second most common cause of acquired cognitive decline and dementia. Vascular cognitive impairment (VCI) refers to all forms of mild-to-severe cognitive impairment associated with and presumed to be caused by cerebrovascular disease [3]. VCI can be divided into two stages: VCI-no dementia (VCI-ND) and vascular dementia (VD) [4]. Studies have shown that cognitive deficits associated with VCI can cover all cognitive domains, with executive function and attention being two of the most sensitive domains for detecting VCI [5,6]. Increasingly, studies have found that the MMSE lacks testing items related to executive function, which makes it unsuitable for detecting VCI (especially VCI-ND) [7,8]. In contrast, the MoCA includes executive function and attention (among many different cognitive domains), making it an ideal test for screening VCI. Indeed, the National Institute of Neurological Disorders and Stroke-Canadian Stroke Network (NINDS-CSN) VCI harmonization standard recommended the MoCA as a 5-min protocol for detecting VCI [9].

The Changsha version of the MoCA (MoCA-CS) is a Chinese version of MoCA that was modified from the original MoCA (English version) by our research team based on mainland cultural, linguistic, and population characteristics. This study assessed the reliability and validity of the MoCA-CS in a sample of ischemic cerebrovascular disease patients in Hunan Province, China, and explored the optimal cutoff scores for detecting VCI-ND and VD.

Methods

MoCA-CS Development

We were granted permission to translate and revise the English version of the MoCA from its original author in March 2010. This original version was translated into Chinese and then retranslated into English by a third party unfamiliar with the test. A questionnaire was sent to an 8-member panel of experts (8 professors in related fields) to assess the consistency of the two English versions. Based on the shortcomings of the Beijing version of the Chinese MoCA that we discovered in the course of our previous research, we designed the questionnaire to collect advice from the expert panel to be used in adapting our translated MoCA. After discussion with MoCA's original author, three items remained unresolved (type B trail making, copy cube, and fluency), and were subsequently tested in a small sample trial (20 healthy subjects over 40 years of age) to determine the final outcomes. After repeated discussions with MoCA's original author and additional amendments, the MoCA-CS was finalized in July 2010 [10], and currently can be downloaded from the official website (http://www.mocatest.org).

Participants

Participants fell into two groups: ischemic cerebrovascular disease patients (ischemic patients) and controls with no cerebral related disease. Ischemic patients were recruited from the neurology and geriatrics departments of our hospital and from inhabitants in surrounding communities if they satisfied all the following criteria: at least 40 years of age, ethnically Chinese with the ability to understand Mandarin (speaking in Chinese dialects was acceptable), a history of ischemic cerebrovascular disease (lesions evident on MRI/CT or in the absence of lesions, a clinical diagnosis of a transient ischemic attack [11]), and the absence of any acute strokes in the last 3 months. Additionally, patients with a history of other brain disorders that might affect cognition (e.g. hemorrhagic cerebral vessel disease, hydrocephalus, intracranial infection, idiopathic epilepsy, or craniocerebral trauma), coexisting medical conditions that might affect cognition (e.g. psychiatric disorders, vitamin B₁₂ deficiency, hypothyroidism, chronic alcoholism, or chemical poisoning), or other conditions that might affect the cognitive assessment process (e.g. severe audiovisual disorders, aphasia, apraxia, or dyskinesia) were excluded. Control subjects comprised volunteers from inpatient caregivers, inpatients from outside the departments of neurology and geriatrics, and inhabitants from surrounding communities. All the volunteers met the following criteria: no history of any cerebral related disease and no complaints of cognitive impairment. All participants were enrolled from August 2010 to November 2011. This study was approved by the Ethics Committee of the 3rd Xiangya Hospital, Central South University, Hunan, China. Written informed consent was obtained from all participants or their legally responsible representatives.

Assessment Process

Assessment was done by 5 individuals who underwent unified training. Demographic materials, past medical history, personal history, and family history of each participant were collected. The ADL (Activity of Daily Living) scale [12], FAQ (Functional Activities Questionnaire) [13], CES-D (Center for Epidemiological Survey, Depression) scale [14], NPI-Q (Neuropsychiatric Inventory, Questionnaire Version) [15], and NIHSS (National Institutes of Health Stroke Scale) [16] were used to assess participants’ daily living condition, social function, mental symptoms, and neurological symptoms. The HIS (Hachinski Ischemic Score) was used to help distinguish the cause of dementia (excluding Alzheimer's disease for a score ≤4). Each participant received assessments with the MoCA-CS, MMSE [17], and CDR (Clinical Dementia Rating) [18] for cognitive screening. According to VCI harmonization standards recommended by the NINDS-CSN [9], the time interval between administering the MoCA-CS and the MMSE was at least 1 h to minimize interference. Those who met the criteria of the negative screen (MMSE scores ≥29 and MoCA-CS scores ≥28) and had no complaint of cognitive impairment were assigned to the cognitively normal (CN) group, while those whose scores did not meet the criteria for a negative screen (positive screen) or who complained of cognitive impairment (whether positive or negative in screening) received a detailed neuropsychological assessment to help to judge their cognitive level. The negative screen criteria were based on our previous small-sample studies that indicated an MMSE score greater than 29 or an MoCA-CS score greater than 27 was an indication of normal cognitive abilities. In order to make sure those with negative screens were really cognitively normal, we increased the required score for the MoCA-CS by 1. The detailed neuropsychological assessment contained four subtests of the Wechsler Adult Intelligence Scale-Chinese revised (WAIS-RC; block design, digit span, similarities, and arithmetic) [19], three subtests of the Wechsler Memory Scale-Chinese revised (WMS-RC, test A; logical memory and 5-min delayed logical memory tests, experience and orientation, and visual recognition) [20], and the Stroop test (including word reading, color naming, interference-word, and interference-color) [21]. Subjects with negative screens who cooperated well also completed the logical memory, 5-min delayed logical memory, and Stroop tests in order to collect more information regarding memory ability and executive function of the participants.

The simplified intelligence quotient (S-IQ) from three subtests (similarities, digit span, and block design) was computed using methods described in the manual of the simplified WAIS-RC. In order to examine test-retest and interrater reliability of the MoCA-CS, we retested two different randomly selected subsamples 3–5 weeks after the first cognitive assessment. One subsample was retested by the same assessor, while the other was retested by a different assessor.

Cognitive Diagnosis

We used MCI diagnostic criteria from the International Working Group as a standard reference for cognitive defects found in VCI-ND: the patient is neither normal nor demented; there is evidence of cognitive deterioration shown by either an objectively measured decline over time or a subjective report of decline by either the subject or an informant in conjunction with objective cognitive deficits, and daily living activities are preserved with intact or minimally impaired complex instrumental functions [22]. We adopted one of the definitions of objective cognitive deficits proposed by Busse et al. [23]: testing results >1.0 SD below the mean level of age- and education-matched norms in any test of our neuropsychological battery. Additionally, we determined if the level of cognitive deficits observed was caused by cerebrovascular disease. The Beijing Tiantan Hospital of China has recently made a feasible diagnostic algorithm for VCI [24], which was used in this study to help make the final diagnosis. The National Institute of Neurological Disorders and Stroke and Association Internationale pour la Recherché et l'Enseignement en Neurosciences (NINDS-AIREN) criteria [25] were used to diagnose VD. Two experienced clinical experts (both deputy directors of neurology) took part in the cognitive diagnosis and made independent diagnoses while being blinded to results from the MoCA-CS and MMSE. If their diagnoses matched, the final diagnosis was easily determined. If their diagnoses differed, a discussion in regular research group meetings was required to determine the final diagnosis. All medical, neuropsychological, and neuroimaging materials were considered when making the diagnoses. Based on these diagnoses, ischemic patients were divided into three groups: CN, VCI-ND, and VD. Controls with abnormal assessment results were excluded from the study, while those with normal cognitive function were deemed healthy controls (HC) and included in subsequent analyses.

Statistical Analysis

SPSS software version 16.0 (SPSS, Chicago, Ill., USA) was used for statistical analysis. We examined the effects of age, education, and gender on MoCA-CS test results by regressing the raw scores of the MoCA-CS (without the +1 point educational adjustment) on these factors using a multivariate linear model. The reliability and validity of the MoCA-CS were tested in ischemic patients (excluding HC), including the internal consistency coefficient (estimated by Cronbach's α), test-retest and interrater reliability (assessed by calculating Pearson correlation coefficients between the baseline and follow-up MoCA-CS scores), and concurrent validity (tested by correlating MoCA-CS scores with MMSE scores and S-IQ). The optimal cutoff scores for VCI-ND and VD were explored based on ROC analysis. The Kappa consistency test was used to evaluate the consistency between clinical-expert diagnoses (treated as a gold standard) and the MoCA-CS. Significance was tested at α = 0.05.

Results

Demographics

Figure 1 is a flow diagram detailing the participants. A total of 560 participants enrolled in the study, with 470 (ischemic patients: 338; HC: 132) satisfying criteria for analysis. Table 1 shows a comparison of several factors across the HC, NC, VCI-ND, and VD participant groups. Aside from gender composition, significant differences were found in age composition, years of education, MoCA-CS scores, MMSE scores, and S-IQ between the four groups. Subjects in the VD group were significantly older than those in the other three groups. There were no significant differences in MoCA-CS scores, MMSE scores, or S-IQ between the two cognitively normal groups (HC and CN). Performance on these three cognitive measurement tests was significantly higher in the cognitively normal groups compared to the cognitively impaired groups (VCI-ND and VD), and in the VCI-ND group compared to the VD group.

Fig 1.

Flow diagram of participants. BD = Block design; D = digit span; S = similarities; A = arithmetic. * Score of CES-D ≥30.

Table 1.

Demographic and cognitive comparison of the 4 groups

	HC (n = 132)	CN (n = 131)	VCI-ND (n = 111)	VD (n = 96)	χ2/ANOVA/Kruskal-Wallis	Adjacent pairwise comparisonsa
M/F	68/64	72/59	61/50	53/43	χ23 = 0.474, p = 0.925	–
Age	66.56 ± 11.93	66.63 ± 12.20	69.41 ± 11.22	73.15 ± 8.12	H(3) = 21.485, p = 0.000	HC=CN=VCI-ND<VD
Education	11.07 ± 3.64	11.48 ± 3.66	8.14 ± 4.24	6.27 ± 4.37	F3,466 = 43.668, p = 0.000	HC=CN>VCI-ND>VD
MoCA-CS	27.38 ± 1.67	27.24 ± 1.69	22.30 ± 3.58	12.10 ± 3.33	H(3) = 327.218, p = 0.000	HC=CN>VCI-ND>VD
MMSEb	28.89 ± 1.17	28.68 ± 1.50	26.04 ± 2.51	18.80 ± 4.97	H(3) = 275.741, p = 0.000	HC=CN>VCI-ND>VD
S-IQc	114.34 ± 10.18	114.44 ± 10.36	98.53 ± 10.09	83.48 ± 7.44	H(3) = 156.894, p = 0.000	HC=CN>VCI-ND>VD

Values are means ± SD (age and education are given in years).

^ap < 0.05 for post hoc test.

^bIn a subsample of HC (n = 128), CN (n = 130), VCI-ND (n = 103), VD (n = 92).

^cIn a subsample of HC (n = 53), CN (n = 45), VCI-ND (n = 81), VD (n = 64).

The Effects of Age, Education, and Gender upon Raw Scores of the MoCA-CS

Table 2 shows the regression results in five subsamples of participants: HC, all ischemic patients (CN+VCI-ND+VD), CN, VCI-ND, and VD. Gender did not enter regression equations in any of the groups, indicating that it had no significant effect on raw MoCA-CS scores. Educational level had a significant effect on raw scores for all groups, with higher education associated with better performance on the MoCA-CS. Age was a significant factor in three groups (HC, all ischemic patients, and VD), indicating that in healthy cognitively normal people, ischemic patients, and VD patients, older subjects performed worse than younger subjects on the MoCA-CS; however, this phenomenon was not significant in ischemic patients with normal cognition or those with VCI-ND. In order to compensate for the effects of education on the raw MoCA-CS scores, the original MoCA recommended adding 1 point to the raw scores for subjects with ≤12 years of education [1]. As the average level of education in China is likely different from that of Canada, the original 12-year education cutoff was probably unsuitable for the Chinese population. In this study, only 56 of 338 ischemic patients (16.6%) and 37 of 132 HC (28.0%) had greater than 12 years of education, indicating that the 12-year education cutoff is likely too high for the Chinese population, and that if test results are to be accurate in China, setting a suitable cutoff is necessary.

Table 2.

Multiple linear regression analysis

Groups	Ba	SEb	Betac	t	P	R2	p
HC
Constant	27.861	1.094		25.464	0.000	0.093	0.006
Gender	–0.097	0.299	–0.028	–0.323	0.748
Age	–0.029	0.012	–0.197	–2.299	0.023
Education	0.122	0.041	0.259	2.973	0.004
CN+VCI-ND+VD
Constant	22.343	2.534		8.818	0.000	0.343	0.000
Gender	0.881	0.649	0.062	1.356	0.176
Age	–0.141	0.029	–0.227	–5.026	0.000
Education	0.815	0.069	0.532	11.763	0.000
CN
Constant	25.104	1.158		21.680	0.000	0.116	0.001
Gender	0.267	0.315	0.078	0.849	0.397
Age	–0.006	0.012	–0.040	–0.455	0.650
Education	0.171	0.042	0.364	4.045	0.000
VCI-ND
Constant	21.706	2.606		8.331	0.000	0.202	0.000
Gender	0.096	0.681	0.013	0.141	0.888
Age	–0.046	0.030	–0.137	–1.547	0.125
Education	0.390	0.078	0.442	4.974	0.000
VD
Constant	18.218	3.171		5.744	0.000	0.183	0.000
Gender	1.065	0.652	0.155	1.633	0.106
Age	–0.129	0.041	–0.305	–3.171	0.002
Education	0.179	0.075	0.227	2.368	0.020

Dependent variable: raw scores of MoCA-CS; independent variables: gender (male = 1; female = 2), age (years), education (years).

^aUnstandardized coefficients.

^bStandard error of unstandardized coefficients.

^cStandardized coefficients.

We divided all the ischemic patients into five education groups based on their Chinese educational background: ≤2, 3–6, 7–9, 10–12, and >12 years. After comparing every possible pair of groups, we found that the raw MoCA-CS scores differed significantly between each group (table 3). We then combined the two groups with the smallest difference in raw scores (7–9 and 10–12 years), and performed comparisons again with four groups. The new comparison revealed that the difference in raw scores was larger between the 3- to 6- and 7- to 12-year groups than between the 7- to 12- and >12-year groups. We therefore made the combination shown in step 3 of table 3, creating two groups with an education cutoff at 6 years (38.76% ≤6 years and 61.24% >6 years). This result was suitable for the Chinese population, and identical to two other Asian studies [26,27] which involved subjects with similar educational backgrounds. Therefore, we recommend adding 1 point to raw MoCA-CS scores for subjects with ≤6 years of education.

Table 3.

Formation process of the 6-year education cutoff in ischemic cerebrovascular disease patients (n = 338)

Educational level groupsa				Kruskal-Wallis/t test	Adjacent pairwise comparisonsb
Step 1
≤2 (n = 37) 13.16 ± 5.36	3–6 (n = 94) 18.37 ± 6.70	7–9 (n = 76) 21.53 ± 5.92	10–12 (n= 75) 23.32 ± 6.59	>12 (n = 56) 26.25 ± 3.83	H(4) = 105.965, p = 0.000	(≤2)<(3–6)<(7–9)<(10–12)<(>12)
Step 2
≤2 (n = 37) 13.16 ± 5.36	3–6 (n = 94) 18.37 ± 6.70	7–12 (n = 151) 22.42 ± 6.30		>12 (n = 56) 26.25 ± 3.83	H(3) = 101.126, p = 0.000	(≤2)<(3–6)<(7–12)<(>12)
Step 3
≤6 (n = 131, 38.76%) 16.90 ± 6.75		>6 (n = 207, 61.24%) 23.45 ± 5.98			t’251.861 = −9.081, p = 0.000	–

^aValues are means ± SD of raw scores of the MoCA-CS; groups are divided by number of years of education.

^bp < 0.05 for post hoc test.

MoCA-CS Reliability

Cronbach's α of the MoCA-CS was 0.884, indicating a high level of internal consistency within the MoCA-CS. The Pearson correlation coefficient between the baseline MoCA-CS scores (with the educational adjustment) and a subset (n = 58) of retest scores associated with the same assessor was 0.966 (p < 0.01), again indicating a high level of test-retest reliability. The Pearson correlation coefficient between the baseline MoCA-CS scores and those from a subset (n = 53) with a different assessor was 0.926 (p < 0.01), which showed that the interrater reliability of the MoCA-CS was also high.

MoCA-CS Validity

Two validation measures (MMSE scores and simplified IQs) were used to test the concurrent validity of the MoCA-CS. MoCA-CS scores were found to be highly and positively correlated with both a subset (n = 325) of MMSE scores (r = 0.867, p < 0.01) and a subset (n = 190) of S-IQs (r = 0.822, p < 0.01), indicating high concurrent validity.

Establishment of the Optimal Cutoff Score

The area under the ROC curve (AUC) was 0.949 (95% CI: 0.929–0.970) for CN versus VCI (including VCI-ND and VD), and 0.990 (95% CI: 0.983–0.997) for no dementia (including CN and VCI-ND) versus VD. Table 4 shows that the maximally accurate cutoff score (corresponding to the largest Youden index) for detecting VCI-ND was 23/24 (23.5). Its specificity was excellent (99.2%), but the sensitivity was low (74.9%). The original study [1] and several other MoCA-related studies [28,29,30] have shown that the MoCA has sensitivities above 90% for detecting MCI. In this study, 26/27 (26.5) was the lowest cutoff score with sensitivity above 90%; furthermore, it corresponded to a peak Youden index compared to nearby cutoff scores. Therefore, 26/27 is recommended as the optimal cutoff for detecting VCI-ND. The corresponding Kappa coefficient was 0.742 (p < 0.01) when compared with the diagnostic results provided by clinical experts. For detecting VD, 16/17 (16.5) corresponded to the largest Youden index, and had excellent sensitivity (92.7%), specificity (96.3%), positive predictive value (90.8%), and negative predictive value (97.1%). As this was the only cutoff score that surpassed 90% on each of these indices, it is recommended as the optimal cutoff score for detecting VD. The corresponding Kappa coefficient was 0.884 (p < 0.01) when compared with diagnostic results provided by clinical experts.

Table 4.

Indexes corresponding to different cutoff scores

CN vs. VCI (VCI-ND + VD)						No dementia (CN + VCI-ND) vs. VD
Cutoff	Sen	Spe	PPV	NPV	YI	Cutoff	Sen	Spe	PPV	NPV	YI
20.50	0.623	1.000	1.000	0.627	0.623	13.50	0.646	1.000	1.000	0.877	0.646
21.50	0.681	1.000	1.000	0.665	0.681	14.50	0.792	1.000	1.000	0.924	0.792
22.50	0.700	1.000	1.000	0.679	0.700	15.50	0.875	0.992	0.977	0.952	0.867
23.50	0.749	0.992	0.994	0.714	0.741	16.50	0.927	0.963	0.908	0.971	0.890
24.50	0.807	0.893	0.923	0.745	0.700	17.50	0.927	0.938	0.856	0.970	0.865
25.50	0.884	0.817	0.884	0.817	0.701	18.50	0.948	0.897	0.784	0.977	0.845
26.50	0.961	0.756	0.861	0.925	0.717	19.50	0.969	0.888	0.732	0.986	0.857
27.50	0.981	0.481	0.749	0.940	0.462	20.50	0.990	0.860	0.736	0.995	0.850
28.50	1.000	0.252	0.679	1.000	0.252	21.50	1.000	0.814	0.681	1.000	0.814
29.50	1.000	0.053	0.625	1.000	0.053	22.50	1.000	0.798	0.662	1.000	0.798

Numbers in bold represent the potential optimal cutoff point. Sen = Sensitivity; Spe = specificity; PPV = positive predictive value; NPV = negative predictive value; YI = Youden index.

Discussion

Use of the MoCA has been increasing worldwide, and this tendency has recently begun in China as well. However, there are two main problems in the current MoCA-related studies in mainland China. Firstly, there is no unified Chinese version of the MoCA, making it difficult to accurately compare results from different studies. Secondly, there are insufficiencies in each version that make it unsuitable for the population of mainland China. More specifically, several subitems are unsuitable for the culture and linguistic background of mainland China, and one subitem cannot be applied to the large number of illiterates among the Chinese elderly. We addressed these issues with the MoCA-CS, a new Chinese version of the MoCA that was translated and adapted independently by our research team, developed especially for use in the population of mainland China, and which can be administered to illiterates.

In this study, we reexamined the effects of age, education, and gender on raw MoCA-CS scores. Unlike previous studies [1,26,28] that reported MoCA performance is affected only by education, and not by age or gender, we found both age and education had significant effects on raw MoCA-CS scores. Education significantly affected MoCA-CS raw scores in all groups, while only age did in the HC, all ischemic patients, and VD groups, but not in the CN and VCI-ND groups. Age might have an effect for the following reasons: in healthy aging people, although cognition remains normal, performance on the MoCA-CS shows a tendency to decline. Another consideration is that results from the group that included all ischemic patients may have been affected by the fact that older subjects had more severe brain damage than younger subjects. In the CN and VCI-ND groups, results were similar to previous studies [1,26,28], showing that only education significantly affects raw MoCA-CS scores. The reason why the all-ischemic group appears to be dominated by the VD portion might be because the VD group contained a high number of older patients with low MoCA-CS scores (table 1 shows that the age of the VD group was significantly higher than in the other three groups).

Here, we also examined the reliability and validity of the MoCA-CS in a sample of Chinese ischemic patients. Cronbach's α of the MoCA-CS was 0.884, similar to previous studies [1,26,27,29,31,32], which showed Cronbach's α in different versions of the MoCA, ranging from 0.72 [26] to 0.90 [32]. Our result demonstrated that after translation and adaption from the original English version, good internal consistency was maintained in the MoCA-CS. The test-retest and interrater reliability statistics of the MoCA-CS were very good, and previous studies [1,26,27,29,31,33] have shown similar results with test-retest reliability ranging from 0.75 [27] to 0.96 [26], and interrater reliability from 0.81 [33] to 0.92 [29]. For validity estimation, we estimated the concurrent validity of the MoCA-CS primarily by correlating its final scores with MMSE scores and S-IQ – both found high correlation, indicating optimal concurrent validity of the MoCA-CS. In previous studies [1,27,31,32,34], MMSE scores were the most commonly used validation measure of the MoCA. The correlation between MoCA and MMSE scores has been shown to be from 0.65 [27] to 0.87 [1]. In addition to the MMSE, many other cognitive estimation tools have been chosen as validation measures, such as the Alzheimer's Disease Assessment Scale-Cognitive [32], Cambridge Cognitive Examination [29], Hasegawa's Dementia Scale, and CDR [31]. No previous studies have used S-IQ as a validation measure. Our reliability and validity results indicate that the MoCA-CS reaches the standards of neuropsychological assessment, which is an effective VCI screening tool in mainland China.

Using ROC analysis, we found high AUCs for CN versus VCI and for no-dementia versus VD, demonstrating that the MoCA-CS effectively detects VCI-ND and VD. It was a more powerful diagnostic tool for VD detection. However, we must emphasize that all VD samples in this study were mild or moderate dementia patients. Severe dementia patients (CDR = 3) who could not cooperate to complete the MoCA-CS were excluded. The AUCs obtained in previous studies were similar to ours. In an American community study, the AUC for MCI versus controls was 0.97 and the AUC for combined MCI/AD versus controls was 0.96 [28]. In a Korean study, the AUC by MoCA (Korean version) for MCI versus normal controls was 0.94, and 0.98 for Alzheimer's disease versus normal controls [27]. In another Asian study, the AUCs for predicting MCI and Alzheimer's disease groups by the MoCA (Japanese version) were 0.95 and 0.99, respectively [31]. Here, we adjusted the original 12-year education cutoff [1] to 6, which was consistent with two other Asian studies [26,27]. We recommend 26/27 (sensitivity 96.1%, specificity 75.6%) as the optimal cutoff for detecting VCI-ND, and it should be used for screening populations at high-risk for VCI-ND who need further cognitive testing. The diagnosis of VCI-ND is based on a thorough clinical evaluation, including cognitive, functional, behavioral, and image assessments. Because of population and cultural differences, cutoff adjustment is very common for different MoCA versions. An American study found that using the recommended cutoff score of 26, the MoCA detected 97% of those with cognitive impairment, but specificity was only fair (35%). Using a lower cutoff score of 22/23, the MoCA exhibited excellent sensitivity (96%) and specificity (95%) [28]. Two other Asian studies adjusted the cutoff score to 21/22 (Hong Kong version; sensitivity 73%, specificity 75%) [26] and 22/23 (Korean version; sensitivity 89%, specificity 84%) [27] for screening patients for cerebral small-vessel disease and MCI. In the studies in mainland China, the cutoff score had been adjusted to 20/21 [Xie He Hospital version (a Chinese version which cannot be downloaded from MoCA's official website); sensitivity 84.6%, specificity 76.0%] [35] and 23/24 (did not specify which Chinese version was used, sensitivity 92.3% and specificity 90.6%) [36]. Compared with other studies using other Chinese versions of the MoCA in mainland China, our recommended 26/27 cutoff score was closer to the original 25/26 cutoff score [1], and our corresponding sensitivity and specificity were also more similar to the original study of the MoCA (sensitivity 90%, specificity 87%) [1]. In addition, the MoCA-CS was the first Chinese version that adjusted the 12-year educational cutoff by adding 1 point to the score, which will have the benefit of more accurate VCI screening. Therefore, it is probably the most thoroughly revised Chinese version of the MoCA at present, and it accounts for the cultural background of mainland China better.

There were some limitations to our present study. First, all subjects in this study came from one area of China (Hunan Province) and had one kind of disease (ischemic cerebrovascular disease), which may limit the scope to which the present cutoff scores may apply. Before the MoCA-CS can be widely used throughout mainland China, it will be necessary to establish national norms which can be applied to different types of diseases. Second, the sample size of this study was relatively small, and future studies should include more subjects.

In summary, the MoCA-CS was developed especially for administration to the population of mainland China. It has good reliability and validity, and is suitable for screening VCI in populations with ischemic cerebrovascular disease in Hunan Province, China. It has the potential to be applied throughout mainland China, although establishing national norms in the future will be necessary.

Disclosure Statement

No conflicts of interest to disclose.