Brief Memory and Executive Test: evaluation of a new screening test for cognitive impairment due to small vessel disease

Abstract

Background: cerebral small vessel disease (SVD) is the most common cause of vascular cognitive impairment (VCI). Despite this, there is a paucity of rapid simple screening tools to identify cognitive impairment in SVD and differentiate it from other common dementia types.

Objective: to validate a new screening test for cognitive impairment in SVD, the Brief Memory and Executive Test (BMET) battery, and examine its ability to detect SVD and differentiate it from Alzheimer's disease (AD).

Subjects: 45 patients with SVD, 27 patients with AD and 80 normal controls.

Methods: the BMET includes brief tests of executive functioning and processing speed, with comparative tests of memory and orientation. Group discrimination was calculated using discriminant function analysis.

Results: the BMET took an average of 10 min to administer. It showed high sensitivity (91%) and specificity (85%) in differentiating SVD patients with cognitive impairment from AD patients. As a comparison the mini-mental state examination had lower sensitivity (63%) and specificity (62%).

Conclusions: the BMET is a simple and quick to administer clinical tool for the detection of VCI in SVD and its differentiation from AD impairment. Further multicentre studies are required to evaluate and compare it with other existing screening tests.

Introduction

Cerebral small vessel disease (SVD) is the most common cause of vascular cognitive impairment (VCI) and becomes increasingly common with increasing age [1, 2]. In SVD, disease of small perforating end arteries supplying the white matter and deep grey matter leads to both focal lacunar infarcts, and more diffuse regions of ischaemia (leukoaraiosis). The pattern of cognitive impairment found in SVD differs from cortical dementias and large vessel stroke populations. Executive dysfunction and impaired information processing speed are prominent while deficits of episodic memory, which are an early feature of Alzheimer's disease (AD), are usually mild or absent [3, 4].

This distinct cognitive profile means that neuropsychological procedures need to be tailored specifically to aid in the assessment of SVD and to be used for clinical management and research into this condition. For example, it has been shown recently that whilst patients with SVD due to the single gene disorder CADASIL do not appear to show measurable cognitive improvement following cholinergic-based treatment when simple screening devices such as the ADAS cog [5] are used, they display changes on executive tasks such as the Trailing Making test [6].

A step forward in this regard has been the development of a neuropsychological protocol for VCI by the National Institute for Neurological Disorders and Stroke (NINDS) [7]. Their neuropsychological working group designed a 60 min battery that incorporates executive function and processing speed tasks, including the Trail Making test and Digit Symbol test, as well as tests of a range of other neuropsychological functions, including memory, visuospatial and language function. Because this test might be too long for an assessment procedure, they also designed a 30 min battery which included measures of executive functioning, fluency, the Digit Symbol test and the Trailing Making test. An additional approach, similar in nature to the NINDS short battery, is that developed by O'Sullivan et al. [4] taking ∼20 min to administer. This battery achieved high sensitivity and specificity when validated on patients with SVD, able to correctly categorise 88% of patients; 91% when restricted to those with mini-mental state examination (MMSE) [8] scores 28 or above.

Whilst these batteries achieve sufficient sensitivity and specificity they may be too long to act as brief screening instruments in a clinical setting. Despite this, there are few brief procedures to select from that are specifically tailored to VCI and hence appropriate for SVD. Currently still frequently used in this context is the MMSE. Whilst this has utility in rapid identification of moderate to severe deficits in AD, it has less sensitivity to mild impairment [9, 10], and it has poor sensitivity and specificity for SVD, for example, only assigning 67% of patients correctly in the study by O'Sullivan et al. [4]. An alternative is the Montreal Cognitive Assessment (MoCA) designed to detect mild cognitive impairment [11]. The MoCA has been validated in stroke patients (non-differentiated by subtype) [12, 13] and shown greater sensitivity than the MMSE for this population. Nevertheless, the executive component is relatively brief with the trail making procedure short and unspeeded (scoring 0/1). This may therefore lose some of the variance in patients where impairment is mild and restricted to the executive function and processing speed components and it may also not be sensitive enough to compare cognitive disorder groups where the profile of impairments is more important than the overall score. Finally, another screening procedure which has relevant tests of executive functioning is the CAMCOG-R, part of the CAMDEX-R [14], with short subtests that cover orientation, language, memory, praxis, attention, abstract-thinking and perception, as well as tests of executive functioning.

The procedures described here have either involved the adoption of batteries developed for more generic use or combined tests from different existing batteries. We took the view that it would be helpful to develop a novel paper-based brief screening instrument (designed to take ∼10 min) specifically tailored for SVD, called the Brief Memory and Executive Test (BMET). This battery is designed to detect cognitive impairment in patients with SVD and to allow differentiation from the cognitive deficit seen in AD. The BMET incorporates features of existing neuropsychological tests, tailored to those features most sensitive to the deficit seen in VCI. The procedures are designed such that each test is made short, whilst preserving sufficient sensitivity to detect early cognitive changes. We present the initial validation of the BMET in a group of carefully phenotyped SVD cases, all with magnetic resonance imaging (MRI) confirmation of disease, as well as in AD and normal controls.

Methods

Subjects

Forty-five subjects with SVD (mean age 69.7 years, SD = 8.2; 25 male) defined as a clinical lacunar stroke syndrome [15] with radiologically confirmed lacunar infarction were recruited from specialised cerebrovascular services in hospitals in London, UK. MRI was performed in all SVD patients and patients were included whether or not there was evidence of leukoaraiosis. Exclusion criteria were any stroke mechanism other than SVD, including a cardioembolic source, extracranial or intracranial large cerebral artery stenosis (>50%), or cortical infarction; unable to perform the test, or unable or unwilling to consent. Patients were all studied at least 3 months after any stroke to avoid the influence of effects due to acute ischaemia on cognition.

For comparison, 27 subjects diagnosed with AD (mean age 75.3 years, SD = 6.8; 13 male) were recruited from specialist dementia clinics from the same region. AD was diagnosed according to DSM IV criteria [16]. All patients had neuropsychological testing and showed deficits in two or more areas of cognition: progressive worsening of memory and other cognitive functions; no disturbance of consciousness; onset between ages 40 and 90; and absence of systemic disorders or other brain diseases that could account for the progressive deficits in memory and cognition, including brain infarction on imaging. All AD patients had brain imaging with CT or MRI as part of routine clinical care to exclude other pathologies. Where original brain scans were available (23/27 cases) these were reviewed to screen out infarcts and confluent leukoaraiosis.

In addition, a normal control group of 80 healthy individuals (mean age: 68.1 years, SD = 7.9; 36 male) was recruited from other studies, patient contacts and a family doctor practice in the same area (see Table 1 for participant demographics). All control subjects had no past history of stroke or other central neurological conditions.

Table 1.

Group demographics and test scores

	SVD			AD	Controls
	All	Without cognitive impairment	With cognitive impairment
Number	45	24	21	27	80
Male sex (%)	56	63	48	48	45
Mean (SD) age, year	69.7 (8.2)	68.4 (7.4)	71.1 (9.1)	75.3 (6.8)	68.1 (7.9)
Ethnicity
White	34 (75.6%)	24 (100%)	10 (47.6%)	25 (92.6%)	74 (92.5%)
Black (Caribbean)	5 (11.1%)	0 (0%)	5 (23.8%)	0 (0%)	1 (1.2%)
Black (African)	4 (8.9%)	0 (0%)	4 (19%)	0 (0%)	0 (0%)
Other	2 (4.4%)	0 (0%)	2 (9.5%)	2 (7.4%)	5 (6.2%)
Treated hypertension	38 (84.4%)	21 (87.5%)	17 (81%)	4 (14.8%)	26 (32.5%)
Treated hypercholesterolaemia	38 (84.4%)	22 (91.7%)	16 (76.2%)	7 (25.9%)	17 (21.2%)
Treated diabetes	17 (21.3%)	14 (58.3%)	14 (66.7%)	0 (0%)	5 (11.1%)
Leukoaraiosis grade on MRI
0	6 (13.3%)	5 (20.8%)	1 (4.8%)	12 (44.4%)
1	12 (26.7%)	8 (33.3%)	4 (19%)	6 (22.2%)
2	12 (26.7%)	10 (41.7%)	2 (9.5%)	4 (14.8%)
3	12 (26.7%)	0 (0%)	12 (57.1%)	0 (0%)
UA	3 (6.7%)	1 (4.2%)	2 (9.5%)	5 (18.5%)
Cognitive tests
BMET
Orientation	9.4 (1.0), Z = −2.5	9.9 (0.3), Z = 0	8.9 (1.1), Z = −5	6.9 (2.5), Z = −15	9.9 (0.2)
5-item repeat	12.8 (2.5), Z = −1.3	13.6 (2.3), Z = −0.5	11.9 (2.5), Z = −2.1	12.4 (2.3), Z = 1.6	14.2 (1.1)
Motor sequencing	21.4 (28.8), Z = −1.8	11.3 (9.5), Z = −0.3	32.9 (38.2), Z = −3.5	26.3 (26.7), Z = −2.5	9.6 (6.6)
L-N matching	18.1 (7.8), Z = −2.2	21.8 (5.9), Z = −1.4	13.9 (7.6), Z = −3.1	15.6 (8.5), Z = −2.7	28.1 (4.6)
Letter sequencing	83.6 (53.5), Z = −3.3	52.9 (33.6), Z = −1.2	118.8 (50.6), Z = −5.7	87.6 (48.9), Z = −3.6	35.3 (14.6)
N-L sequencing	129.8 (102.4), Z = −3.5	59.3 (29.3), Z = −0.5	210.3 (96.5), Z = −7.0	195.5 (107.8), Z = −6.4	48.2 (23.0)
5-item recall	1.3 (2.0), Z = −0.6	2.2 (1.8), Z = −0.1	0.3 (1.7), Z = −1.1	0.7 (1.7), Z = −0.9	2.4 (2.0)
5-item recognition	2.4 (2.4), Z = −0.8	3.5 (1.6), Z = −0.1	1.2 (2.7), Z = −1.6	−0.4 (3.9), Z = −2.7	3.6 (1.5)
Awareness	2.8 (0.8), Z = −0.7	3.2 (0.8), Z = −0.1	2.4 (0.6), Z = −1.3	2.4 (0.8), Z = −1.3	3.3 (0.7)
MMSE	26.4 (3.4), range = 17–30	28.4 (1.2), range = 26–30	24.1 (3.7), range = 17–30	22.0 (4.4), range = 15–30	28.9 (1.4), range = 24–30
Education
None	51.1%	58.3%	42.9%	25%	15%
Secondary	11.1%	8.3%	14.3%	12.5%	18.8%
Further ed.	26.7%	25%	28.6%	12.5%	31.2%
Degree	8.9%	8.3%	9.5%	16.7%	23.8%
Higher degree	0%	0%	0%	4.2%	11.2%
U/A	2.2%	0%	4.8%	0%	0%
NART FSIQ	108.0 (10.0)	112.3 (9.3)	102.4 (8.1)	109.7 (21.2)	114.8 (18.7)
CDR	0.4 (0.3)	0.3 (0.3)	0.5 (0.3)	1.0 (0.4)	0 (0 )

Data for the SVD group are presented for the group as a whole and then divided into those with and without cognitive impairment.

Original MRI (or for AD, MRI or CT scans) were used to grade the degree of leukoaraiosis using the Fazekas scale into absent (0), mild (1), early confluent (2) and severe confluent (3). UA, unavailable; Education = highest formal qualification. Z-scores were calculated using Cohen's ▵, negative = a score lower than controls.

Ethics

All research was conducted as a part of studies approved by a UK NHS ethics committee. Full written consent was obtained for all participants in this study.

Cognitive testing and the BMET

The BMET includes tests of executive functioning (sequencing: motor, letter, number-letter switching) and processing speed (letter-number matching) designed to be sensitive to the effects of SVD. Comparative tests of verbal memory (repetition, recall and recognition) and orientation are included because the differential sensitivity of these types of tests improves discrimination with other patient groups. Each sub-test was designed to be brief and easily administered relying only on paper forms. Furthermore this screening battery is developed such that all tests are standardised on the same normative population, facilitating comparison of individual test performance. Details of the individual tests are outlined in Supplementary data available in Age and Ageing online, Appendix S1.

Administration of the test was trialled on a sample of control subjects by nine participants to ascertain ease and speed of administration. The participants included seven neurologists, one research nurse and two study coordinators. Their mean time taken to administer the test was 10 min 32 s.

In addition to the BMET, the MMSE [8] and the Clinical Dementia Rating Scale [17] were administered to participants on the same day. Retesting was carried out with a sample of 31 control cases (mean age at retest 65.3; SD = 12.9) after a 1 year delay (mean = 12.0 months; SD = 0.9).

Analysis

Raw scores for each BMET subtest were entered into a discriminant function analysis using SPSS v.16 (SPSS, Inc., 2008, Chicago, IL, USA, www.spss.com). The discriminant score (D) is calculated from the individual test scores. The analysis weights the individual scores such that the maximum group discrimination is achieved. One AD patient had data missing for sequencing tests and was therefore excluded from this analysis.

Categorisation was carried out across all three groups and then for patients with cognitive impairment only. Due to their inclusion criteria, all of the AD group patients had cognitive impairment. By contrast, the SVD group were selected according to the presence of clinical and radiological SVD, and had a range of cognitive abilities. To examine the ability of the test to discriminate between the cognitive profiles of AD and SVD in cognitively impaired subjects, we first identified those SVD subjects with cognitive deficits, defined as being 1.5 SD below the control mean on any four of the eight diagnostic tests (i.e. not including awareness and without specific profiling). We then performed an analysis comparing SVD patients with cognitive impairment with AD patients. Using these criteria, 21 (47%) of SVD patients showed cognitive deficits.

To determine test–retest reliability, overall test scores at baseline and after 1 year were correlated for the subset of controls completing both time points. Overall scores were calculated by averaging the Z-scores (based on the control mean and SD) for each subtest.

Internal consistency was calculated for all participants using Cronbach's alpha.

Results

Discrimination of SVD cases from AD and controls

The BMET correctly classified 67% of SVD patients, 89% of controls and 58% of AD patients (see Table 2; for figure, see Supplementary data available in Age and Ageing online, Appendix S2). The MMSE performed less well for SVD patients, correctly classifying 24%; and similarly well for the controls and AD groups, correctly classifying 86% and 67%, respectively.

Table 2.

Categorisation of controls, SVD (all) and AD patients, based on the discriminant function analysis

	SVD	AD	Controls	Sensitivity (%)	Specificity (%)
BMET
Clinical group
SVD (n = 45)	30	4	11	67	85
AD (n = 26)	7	15	4	58	97
Controls (n = 80)	9	0	71	89	79
MMSE
Clinical group
SVD (n = 45)	11	12	22	24	86
AD (n = 27)	6	18	3	67	89
Controls (n = 80)	9	2	69	86	65

The BMET subtests were categorised into two discriminant functions. The first accounted for 75.5% of the variance and maximally discriminated AD patients (centroid = −2.27) from the other two groups (control centroid = 0.80; SVD centroid = −0.10). The first function was related mostly to memory tasks with the largest absolute correlations for orientation (r = 0.90), five item recognition (r = 0.58), five item recall (r = 0.31), number-letter sequencing (r = −0.67), motor sequencing (r = −0.28) and awareness (r = .40). The second function accounted for 24.6% of the variance and maximally discriminated SVD patients (group centroid = 0.97) from AD patients (centroid = −0.49) and controls (centroid = −0.39) and showed a greater weighting of executive function tasks with the largest absolute correlations for letter-number matching (r = −0.71), letter sequence time (r = 0.64) and five item repetition (r = −0.35).

Discrimination of cognitively impaired SVD cases from AD

Twenty-one (47%) of SVD patients showed cognitive deficits defined as being 1.5 SD below the control mean on any four of the eight diagnostic tests. The demographics for the SVD groups with and without cognitive impairment are given in Table 1. We compared the profile of impairment in the two groups with cognitive deficits. The discriminant analysis indicated that the BMET correctly classified 91% of SVD patients with cognitive impairment and 85% of AD patients. Specificities were also high (85% for SVD and 91% for AD). The MMSE performed less well with lower sensitivities (SVD 62%, AD 63%) and specificities (SVD 63%, AD 62%) (Table 3).

Table 3.

Categorisation of SVD subjects with cognitive impairment from AD patients, based on the discriminant function analysis

Clinical Group	SVD (cog deficits)	AD	Sensitivity (%)	Specificity (%)	Positive predictive values (%)
BMET screening test
Clinical group
SVD (n = 21)	19	2	91	85	83
AD (n = 26)	4	22	85	91	92
MMSE
Clinical group
SVD (n= 21)	13	8	62	63	57
AD (n = 27)	10	17	63	62	68

Age and educational level

Given that there is some group variation in age and education we performed further analysis with these measures included in the discriminant analysis. We found that for the BMET there was little change in the results (SVD = 95%; AD = 85%); for the MMSE there was a slight increase in group discrimination when these two factors were included (SVD = 70%; AD = 65%).

MMSE < 27

It might be argued that the AD group show more severe impairment and hence this drives the discriminability rather than differential pattern of results. Because the pattern is different in the two groups it is not possible to match them on a single measure of severity. However, as an approximate comparison method the discriminant analysis was repeated, and this time the AD and SVD groups were selected on the basis of MMSE score <27 [18]; 16 of the SVD patients and 21 of the AD patients fitted this criterion. The results were similar to the original analysis indicating that the BMET correctly classified 94% of SVD patients and 85% of AD patients; and the MMSE correctly classified 63% of SVD patients and 67% of AD patients.

Reliability

In the retest subgroup the BMET showed good test–retest reliability in normal controls after 1 year (r = 0.80) and high internal consistency for the executive (α = 0.85) and memory components (α = 0.71).

Discussion

This paper describes a new rapid screening test designed to identify VCI due to SVD and discriminate it both from normal cognition and from other causes of cognitive impairment such as AD. Through an initial validation on SVD and AD the purpose of the study was to assess the potential of the BMET instrument, exploring its discriminative sensitivity and specificity. The results suggest improvements in comparison to the MMSE for the total sample of SVD patients.

The SVD patients were then split into those with cognitive impairment and those without to facilitate a more direct comparison with AD, who were by definition cognitively impaired. This analysis revealed that half of the SVD patients had significant cognitive impairment, a figure consistent with that of previous reports [4]. The sensitivity and specificity of the BMET in discriminating between the two groups was found to be around 90%, substantially higher than the MMSE (around 60%). The analysis was repeated, selecting patients based on MMSE scores <27, the results were similar, indicating robust discrimination of the two groups based on their impairment profiles.

It is well known that the MMSE, and other short cognitive tests designed for AD, are insensitive to the subcortical pattern of cognitive impairment seen in patients with VCI due to SVD. The need for better screening tools has been recognised for VCI, as has the need to increase the range of validated tools that can be used in clinical practice. The BMET procedure has been specifically tailored to this population of patients, based on previous studies by our group. This study shows that the BMET has both the brevity and sensitivity to be useful in this setting. The BMET has been designed to include a sufficiently extensive executive component with brief comparison tasks, including orientation, episodic and working memory tasks.

The current study needs to be followed up by a more extensive validation, including further comparisons with existing screening instruments. In this regard an alternative is the expanded version of the MMSE, although this procedure has only one test more specifically sensitive to SVD, a Symbol-Digit Coding procedure, and this version takes approximately twice as long. However, it has been validated successfully on a subcortical dementia population [19]. The MoCA is an appropriate alternative but might prove to be less sensitive because of the highly abbreviated tests of executive functioning. The CAMCOG, whilst incorporating executive tests is comparatively lengthy and contains tests beyond those needed specifically for screening in this group. Nevertheless, further validation and comparison with such instruments should provide a further indication of relative efficacy.

In this study, we focused on a group of patients with ‘pure’ SVD, in whom any cognitive deficits are likely to be caused by SVD rather than any concomitant pathology, as a deliberate preliminary strategy to validate the procedure. We recruited patients who presented with SVD (i.e. clinical lacunar stroke and radiological SVD). A subsidiary issue, but nevertheless an important consideration is that evaluating cognitive tests in dementia is complex due to the overlap between AD and vascular dementia. Post-mortem studies have shown the two frequently coexist, and the presence of vascular pathologies is a major determinant as to whether AD pathology results in dementia [20]. In light of this, further studies using this screening instrument could be conducted in patients with varying degrees of pathology types.

The current findings are in keeping with previous studies that have shown that cognitive impairment is common in the SVD patients [4], and the pattern of impairment is fairly homogeneous with prominent impairment of executive function and information processing speed and relative preservation of orientation [21]. In this regard, screening instruments are not only useful by means of detecting a condition relevant profile, but also in evaluating treatments which specifically target relevant cognitive deficits. For example, in CADASIL, no effect of cholinesterase inhibition on V-ADASS-Cog scores was found while a significant effect on measures of executive function was present [6]. Brief assessments such as the BMET may be useful for monitoring patients, with parallel versions designed to overcome the consequences of any practice effects.

In summary, our study suggests that the BMET shows promise in identifying VCI due to SVD and differentiating it from AD. Further multicentre studies are now required with this instrument for further validation in different settings, and to determine sensitivity and specificity in larger patient groups.

Key points.

• There is a need for short cognitive screening procedures tailored to SVD.

• The BMET was designed to be sensitive to the profile of SVD.

• The initial validation showed good discrimination of AD and SVD with sensitivity and specificity around 90%.

• Further studies are required to determine the sensitivity and specificity in a larger population.

Supplementary data

Supplementary data mentioned in the text is available to subscribers in Age and Ageing online.

Conflicts of interest

None declared.

Funding

Recruitment to the study was supported by the English National Institute of Health Research Clinical Stroke Research Network. This research was supported by The Stroke Association (TSA 2006/12).