The Diagnostic Utility of the NINDS-CSN Neuropsychological Battery in Memory Clinics

Abstract

Aims

To examine the diagnostic utility of the National Institute of Neurological Disorders and Stroke and the Canadian Stroke Network (NINDS-CSN) neuropsychological battery in memory clinics comparing controls with patients with no cognitive impairment (NCI), patients with cognitive impairment-no dementia (CIND) at varying severity levels (mild/moderate), and patients with dementia.

Methods

A total of 405 participants with NCI, CIND or dementia were assessed with the NINDS-CSN battery. The discriminatory properties of all three protocols (5, 30 and 60 min) before and after education stratification (none/primary vs. secondary/above) were examined by receiver operating characteristic curves.

Results

Overall, the shorter protocols are equivalent to the longer protocol in diagnosing dementia, regardless of education. To discriminate between nondementia groups, before education stratification, the 5-min protocol showed varied discriminatory properties between different diagnostic/severity groups. After stratification, the 5-min protocol was broadly equivalent to the longer protocols in lower-education groups [area under the curve (AUC) range: 0.77-0.87] but was less accurate in the higher-education groups (AUC range: 0.68-0.78). The 30- and 60-min protocol constantly showed moderate-to-excellent differentiating capacities regardless of education (AUC range: 0.80-0.90).

Conclusion

The NINDS-CSN neuropsychological battery can be applied in memory clinics and effectively discriminate between cognitively intact individuals and those with cognitive impairments of varying severity. Furthermore, level of education should be taken into consideration when choosing protocols with different lengths for cognitive assessment.

Introduction

The National Institute of Neurological Disorders and Stroke and the Canadian Stroke Network (NINDS-CSN) convened researchers and clinicians across disciplines to recommend minimum standards for the description and study of vascular cognitive impairment (VCI). As part of this endeavor, the NINDS-CSN neuropsychological battery was developed for the identification and characterization of individuals with VCI, particularly in the early stages [1].

The NINDS-CSN battery consists of 3 protocols (5, 30 and 60 min) and bridges the gap between comprehensive neurocognitive assessment and brief screening tests. It is quick and easy to administer and may reduce the likelihood of patient fatigue. To date, the battery has been validated in VCI patients in non-English-speaking groups in Hong Kong and Korea [2,3]. Whilst demographic factors such as age, gender, and particularly, education are common confounders in the assessment of cognitive function, especially in elderly Asian populations [4,5], and the diagnosis of dementia more generally [6], the NINDS-CSN battery has been shown to be feasible for lower-educated participants [2].

Whilst the NINDS-CSN battery has been successful in detecting VCI, including those with more severe cognitive impairment [3,7], most dementia patients also have significant cerebrovascular disease [8,9]. Presently, no study has investigated its applicability in discriminating between patients with dementia, patients with cognitive impairment-no dementia (CIND) and controls with no cognitive impairment (NCI). In memory clinic settings, CIND may constitute over one fifth of the total patient population, and over one third of CIND patients demonstrate underlying vascular pathology [10,11]. Furthermore, CIND patients compared to NCI demonstrate a five times greater risk of developing dementia within 5 years [12]. The degree of CIND is significantly important in determining the risk of conversion. Narasimhalu et al. [13] showed in an ischemic stroke cohort that patients with CIND-moderate were at higher risk of conversion compared with CIND-mild patients.

In view of the potential applicability of the NINDS-CSN battery in general memory clinic settings, we examined its diagnostic utility in differentiating between NCI controls, CIND patients at varying severity levels (mild/moderate) and dementia patients. A further aim was to examine the discriminatory properties of the battery in relation to education.

Methods

Participants

Four hundred and five participants were recruited from 2 hospitals in Singapore. Cases were consecutive eligible patients with impairment in at least 1 cognitive domain on a locally validated standard neuropsychological battery [the Vascular Dementia Battery (VDB)] [14,15]. Controls with no cognitive impairment on the VDB were recruited from the community. The inclusion criteria were: 50 years or older, a clinical diagnosis of NCI, CIND or dementia, sufficient language skills for neuropsychological assessment, and informed consent. Participants with major psychiatric illness or substance abuse disorder were excluded. The presence of major psychiatric illness or substance abuse was based on self-report and then subsequently verified in patients' medical records.

Consensus Diagnosis

A team of neurologists, psychologists and research personnel determined diagnoses for all participants by consensus. All available clinical and investigational data, including details from the clinical assessment, blood investigations, neuropsychological testing, and MRI scans, were reviewed as a part of the diagnostic process. Neuropsychological domain impairments and diagnoses were based on the VDB. On the VDB, failure to pass more than half of the tests in a cognitive domain was considered indicative of impairment. Participants with impairment in at least 1 domain, but who did not meet the criteria for dementia, were classified as CIND. CIND was further divided into CIND-mild (impairment in 1-2 domains) and CIND-moderate (impairment in 3 or more domains). Participants with no objective cognitive impairment were classified as NCI. Dementia diagnosis was made according to DSM-IV criteria.

Neuropsychological and Clinical Assessment

All participants completed the NINDS-CSN battery independent of their test results on the VDB. The battery was administered by qualified psychologists and research personnel.

The 60-min protocol [2] addresses 6 cognitive domains: (1) working memory: digit span forward and backward [16]; (2) executive function: verbal fluency [17], Color Trail Test A&B [18]; (3) language: 15-item modified Boston Naming Test [19]; (4) visuomotor speed: Symbol Digit Modalities Test [20]; (5) visuospatial function: Rey Complex Figure Test-copy [21], and (6) memory: Rey Complex Figure Test-immediate/delayed recall and recognition [21], Hopkins Verbal Learning Test-immediate/delayed recall and recognition [22].

The 30-min protocol addresses 2 cognitive domains: (1) executive function: verbal fluency, and Symbol Digit Modalities Test, and (2) memory: Hopkins Verbal Learning Test-immediate/delayed recall and recognition.

The 5-min protocol addresses 3 cognitive domains derived from the Singapore version of the Montreal Cognitive Assessment (www.mocatest.org): (1) memory: 5-word registration, delayed recall and recognition (category cue and multiple-choice cue); (2) orientation, and (3) executive function: verbal fluency.

Participants also underwent history taking, clinical examination, blood tests, and CT and/or MRI scanning of the brain.

Statistical Analysis

All individual test raw scores on the NINDS-CSN battery were transformed to standardized z-scores using the means and standard deviations (SDs) of the NCI group. The score for each domain was created by averaging the z-scores of individual tests and standardized using the mean and SD of the NCI group. To obtain the global cognition domain z-score for each patient, the domain z-scores were averaged and standardized using the mean and SD of the NCI group.

A normality test was performed prior to the computation of z-scores. The performance of the NCI group was normally distributed, hence no additional transformation was required (skewness = 1.59, kurtosis = 1.10).

Differences in neuropsychological cognitive scores between the groups were tested using ANOVAs with age, gender, years of education, and total scores on the Geriatric Depression Scale and the Neuropsychiatric Inventory as covariates. Subsequently, receiver operating characteristic (ROC) curve analysis was performed to explore the discriminant ability of the NINDS-CSN neuropsychological battery in distinguishing between different diagnostic categories. The ROC analysis was repeated after stratification by educational level (none/primary vs. secondary/above). Further stratification by other demographic factors such as age (≥75 vs. <75 years) and gender was conducted on top of education stratification. However, no significant improvement was observed. Hence this analysis was omitted from the report. All analyses were performed using SPSS 22.0. Significance was determined with a p value of <0.05. Areas under the curve (AUCs) of the ROCs were statistically compared [23] and significance was determined at p < 0.05.

Results

Of the 405 participants, 94 (23.2%) were classified as NCI, 102 (25.2%) were CIND-mild, 58 (14.3%) were CIND-moderate, and 151 (37.3%) were dementia cases. Of the whole sample, 53.3% were female and 81.0% were Chinese. Participants' age ranged from 50 to 95 years and years of education from 0 to 23 (table 1).

Table 1.

Demographics and clinical assessment results

	NCI (n = 94)	CIND-mild (n = 102)	CIND-mod (n = 58)	Dementia (n = 151)	p (trend)
Age, years	67.7± 5.8	69.6±8.1	73.5± 8.5	76.5± 7.6	<0.001
Female	50 (53.2)	42 (41.2)	33 (56.9)	91 (60.3)	0.03
Chinese ethnicity	85 (90.4)	81 (79.4)	45 (77.6)	117 (77.5)	0.07
Education, years	9.9± 5.1	8.0 ±4.6	5.9± 5.3	4.6± 4.5	<0.001
Hypertension	54 (57.4)	68 (66.7)	41 (70.7)	125 (82.8)	<0.001
Hyperlipidemia	65 (69.1)	75 (73.5)	45 (77.6)	109 (72.2)	0.72
Diabetes	19 (20.2)	35 (34.3)	23 (40.0)	65 (43.0)	0.003
Smoking history	11 (11.7)	9 (8.8)	6 (10.3)	8 (5.3)	0.32
Previous stroke	20 (21.3)	39 (38.2)	23 (39.7)	40 (26.5)	0.08
Previous ischemic heart disease	4 (4.3)	5 (4.9)	6 (10.3)	11 (7.3)	0.42
GDS total score	1.4± 2.0	2.3 ±3.0	2.0± 2.3	8± 5.3	<0.001
NPI total score	0.8± 2.6	1.6 ±4.1	1.8± 3.8	3.1± 3.1	<0.001
MMSE score	27.4± 2.0	25.8 ±3.0	22.4± 3.4	10.0± 14.5	<0.001

Bold type indicates significance. Figures are means ± SD or numbers with percentages in parentheses. GDS = Geriatric Depression Scale; NPI = Neuropsychiatric Inventory; MMSE = Mini Mental State Examination.

Neuropsychological performance on the NINDS-CSN battery is presented in figure 1. Significant differences in global scores were seen across NCI, CIND-mild, CIND-moderate and dementia groups (p < 0.001).

Fig. 1.

Global scores on the 5-, 30- and 60-min NINDS-CSN protocols in NCI, CIND-mild, CIND-moderate and dementia groups.

Overall, the 5-min protocol was equivalent to longer protocols in diagnosing dementia regardless of education. To differentiate between various nondementia groups, before stratification, the 5-min protocol showed varying discriminatory properties between different diagnostic groups (AUC range: 0.73-0.80), whilst the longer protocols indicated overall good to moderate differentiating capacities (AUC range: 0.77-0.86). After stratification, the 5-min protocol was significantly better in differentiating CIND from NCI participants in the lower-educated group than in the higher-educated group (AUC = 0.87 and 0.73, respectively, p < 0.001), and in differentiating CIND-mild from NCI participants (AUC = 0.81 and 0.68, respectively, p < 0.001) (online suppl. table 1; see www.karger.com/doi/10.1159/000445050 for all online suppl. material). On the other hand, the 30- and 60-min protocols consistently revealed moderate/excellent ability in discriminating between various diagnostic groups (AUC range: 0.80-0.90) (table 2).

Table 2.

AUCs for global cognitive scores on the NINDS-CSN battery in NCI, CIND-mild and CIND-moderate groups

Diagnostic categories	5-min global performance (95% CI)	30-min global performance (95% CI)	60-min global performance (95% CI)
Whole sample (n = 405)
Nondementia vs. dementia	0.95 (0.93–0.97)	0.96 (0.94–0.97)	0.94 (0.92–0.96)
CIND vs. dementia	0.92 (0.89–0.95)	0.93 (0.90–0.96)	0.91 (0.87–0.94)
NCI vs. CIND	0.80 (0.74–0.85)	0.85 (0.80–0.90)	0.84 (0.79–0.89)
NCI vs. CIND-mild	0.73 (0.66–0.80)	0.79 (0.73–0.85)	0.77 (0.71–0.84)
CIND-mild vs. CIND-moderate	0.79 (0.72–0.87)	0.81 (0.74–0.88)	0.86 (0.80–0.92)
None or primary level of education (n = 213)
Nondementia vs. dementia	0.93 (0.90–0.97)	0.94 (0.91–0.97)	0.93 (0.89–0.96)
CIND vs. dementia	0.91 (0.86–0.95)	0.92 (0.87–0.96)	0.90 (0.85–0.94)
NCI vs. CIND	0.87 (0.81–0.94)	0.90 (0.83–0.96)	0.90 (0.84–0.95)
NCI vs. CIND-mild	0.81 (0.71–0.91)	0.84 (0.75–0.94)	0.81 (0.70–0.91)
CIND-mild vs. CIND-moderate	0.77 (0.66–0.88)	0.80 (0.70–0.90)	0.86 (0.78–0.94)
Secondary and above level of education (n = 192)
Nondementia vs. dementia	0.96 (0.93–1.00)	0.97 (0.94–1.00)	0.95 (0.92–0.99)
CIND vs. dementia	0.94 (0.90–0.99)	0.95 (0.90–0.99)	0.92 (0.87–0.97)
NCI vs. CIND	0.73 (0.65–0.81)	0.85 (0.79–0.91)	0.87 (0.81–0.92)
NCI vs. CIND-mild	0.68 (0.59–0.77)	0.82 (0.74–0.89)	0.83 (0.76–0.90)
CIND-mild vs. CIND-moderate	0.78 (0.65–0.90)	0.80 (0.69–0.91)	0.86 (0.79–0.95)

Discussion

Our study shows that the NINDS-CSN neuropsychological battery can discriminate between patients with different diagnoses and severities of cognitive impairment in memory clinics. To discriminate between various diagnostic groups, whilst the 5-min protocol shows varied results, longer protocols are more consistent in these discriminations. Furthermore, whilst the 5-min battery is more influenced by education, it is more robust in detecting differences in baseline cognitive ability in the lower-education group.

We found that short protocols are equivalent to longer protocols in diagnosing dementia. This is consistent with a previous US study [24], indicating that short tests suffice for the purpose of general dementia diagnosis. Longer protocols may be of greater help in assessing specific neuropsychological characteristics of different dementia subtypes.

The NINDS-CSN battery has previously been validated in post-stroke patients with a high likelihood of developing VCI [25]. After classifying the cognitive status by a brief screening test, all three protocols were equivalent in differentiating stroke patients from controls (AUC range: 0.86-0.88). However, in this study, cognitive status was initially classified via a more comprehensive independent neuropsychological examination for greater specificity. This study shows that protocols were not equivalent and the more extensive 30- and 60-min protocols provided greater discriminatory power. This highlights the importance of neuropsychological assessment in disease diagnosing and severity staging.

An important aspect in the choice of protocol is the influence of education on performance. For higher-educated participants, the 30- and 60-min protocols' differentiating capacity was comparative, but for low-education patients, the longer protocols provided no additional discriminatory power. As such, whilst longer protocols are better for higher-educated participants, the 5-min protocol suffices as a diagnostic measure in lower-educated participants.

This study has strengths and weaknesses. It is the first to examine the discriminatory properties of the NINDS-CSN battery in a general memory clinic population classified by consensus and comprehensive neuropsychological testing. A further strength of the study was the use of a comprehensive neuropsychological battery for the diagnosis and staging of cognitive impairment independent of the NINDS-CSN battery. The main limitation is that, as a cross-sectional study, it is not possible to determine the validity of the battery as a prognostic tool for dementia. Furthermore, this study had a relatively small sample size (n = 94) of controls not matched with cases in age and education. This limits the accuracy of z-score calculation. In addition, with a low MMSE mean score of 10, the present study had a severe dementia population which may potentially limit the representativeness of the results. Finally, due to limitations of brief batteries, such as the NINDS-CSN battery, there are inherent limitations on profiling the nature of memory impairments derived from different pathologies. While the NINDS-CSN battery can discriminate between patients with no cognitive impairment and those with cognitive impairment or dementia, it cannot differentiate between the different types of cognitive impairment or dementia. In conclusion, whilst neuropsychological testing is necessary for diagnosis and staging in memory clinic settings, shorter forms of objective assessment may be more suitable for this purpose in places (such as many parts of Asia) where comprehensive neuropsychological services may not be available. The present findings show that the NINDS-CSN battery can be applied in memory clinics and effectively discriminate between those with NCI and those with cognitive impairments of varying severity. In view of the high conversion rate in CIND populations and the importance of early diagnosis, the effective use of neuropsychological tools such as the NINDS-CSN battery may ultimately lead to more timely and accurate diagnosis and better management of patients with cognitive impairment.

Disclosure Statement

The authors declare no conflicts of interest.