Methodological factors in determining risk of dementia after TIA and stroke: (III) Applicability of cognitive tests

Abstract

Background and Purpose

Cognitive assessment is recommended after stroke but there are few data on the applicability of short cognitive tests to the full spectrum of patients. We therefore determined the rates, causes and associates of untestability in a population-based study of all TIA and stroke.

Methods

Patients with TIA or stroke prospectively recruited (2002-2007) into the Oxford Vascular Study had ≥1 short cognitive test (mini-mental-state examination (MMSE), telephone interview of cognitive status (TICSM), Montreal cognitive assessment (MOCA), and abbreviated mental test score (AMTS)) at baseline and on follow-up to 5 years.

Results

Among 1097 consecutive assessed survivors (mean age/sd 74.8/12.1 years, 378 TIA), numbers testable with a short cognitive test at baseline, 1, 6, 12 and 60 months were 835/1097 (76%), 778/947 (82%), 756/857 (88%), 692/792 (87%) and 472/567 (83%) 88% (331/378) of assessed TIA patients were testable at baseline compared to only 46% (133/290) of major stroke (p<0.001). Untestability was also associated with older age, premorbid dependency, death on follow-up and with both pre- and post-event dementia (all p<0.01). Untestability (and problems with testing) were commonly caused by acute stroke effects at baseline (153/262 (58%): dysphasia/anarthria/hemiparesis=84 (32%), drowsiness=58 (22%) and acute confusion=11 (4%)) whereas sensory deficits caused relatively more problems with testing at later time points (24/63 (38%) at 5 years).

Conclusions

Substantial numbers of patients with TIA and stroke are untestable with short cognitive tests. Future studies should report data on untestable patients and those with problems with testing in whom the likelihood of dementia is high.

Introduction

Cognitive screening is recommended after stroke to guide clinical management and to measure outcome in stroke trials since stroke increases the risk of dementia, and dementia predisposes to stroke.^1–3 Short cognitive tests are required since long batteries of tests are not practicable outside small-scale research studies.⁴ Existing studies on short cognitive tests such as the Mini-Mental-State-Examination (MMSE)⁵ and Montreal Cognitive Assessment (MoCA),⁶ have demonstrated their validity against neuropsychological batteries in selected subgroups^4,7,8 but the applicability of these tests to the total population (ie the proportion testable) with TIA and stroke is unclear.^4,9

This paper is the last in a series of three examining the methodological issues in measuring rates of TIA and stroke-associated dementia. The two previous papers have shown that widely used baseline selection criteria and selective attrition from face-to-face follow-up result in TIA and stroke cohorts that are unrepresentative of the whole with those at highest risk of dementia being excluded.^10,11 This third paper examines the third and final major potential source of bias in assessing cognitive outcomes after TIA and stroke: the applicability of cognitive tests. Previous studies in non-cerebrovascular populations have shown that sensory impairments and more severe cognitive impairment impact on the applicability of such tests^12–14 and that cognitive impairment associates with frailty.¹⁵ We therefore hypothesised that risk factors for dementia including non-stroke characteristics such as older age and sensory deficits as well as more severe cerebrovascular events, would be associated with untestability in TIA and stroke but there are few data from inclusive cohorts with long term follow-up.⁹

We therefore undertook a longitudinal population-based study of all TIA and stroke to determine the rates, reasons and associates of untestability using a short cognitive test (MMSE⁵ MoCA,⁶ Telephone Interview for Cognitive Status¹⁶ and abbreviated mental test score (AMTS¹⁷)) with follow-up to 5-years.

Methods

Patients with TIA or stroke were prospectively recruited from 1^st April 2002-31^st March 2007 into the Oxford Vascular Study (OXVASC), a prospective population-based cohort study of all acute vascular events occurring within a defined population of 92 728 covered by 100 general practitioners (GPs-primary care) in nine GP practices in Oxfordshire, UK.^18,19 The study was approved by the local research ethics committee. Informed written consent (or assent from relatives) was obtained for study interview and follow-up either in person or where not possible, by telephone, and also consent/assent for indirect follow-up using primary care physician records, hospital records and death certificate data. In cases where patients died before first assessment or where a family member could not be contacted for assent in patients lacking capacity (eg owing to dysphasia or dementia), the ethics committee approved review of the patient’s medical records.

The methods of OXAVSC have been described in detail elsewhere.^18,19 Patients were ascertained as soon as possible after the initial TIA or stroke by study clinicians through a combination of hot and cold pursuit.²⁰ TIA and stroke were defined clinically by WHO criteria.²¹ Stroke was dichotomized as minor or major using a cut-off of >3 on the NIHSS (National Institutes of Health Stroke Scale) as this had been previously found to discriminate best between those seen in the emergency clinic versus those presenting directly to the emergency department.²² Baseline brain and vascular imaging was performed and all cases were reviewed by a senior vascular neurologist (PMR). Leukoaraiosis was defined as absent, mild, moderate or severe as described previously.²³ Patient data were collected by interview using a standardised form and general practitioner records.^18,19 Risk factors were recorded at study entry. Functional status, assessed using modified Rankin²⁴ and Barthel²⁵ scores, was done at baseline and at all follow-ups.

Follow-up interviews were done by trained research nurses at 1 and 6 months and 1, 5 and 10 years either in the out-patient clinic or by home visit where hospital clinic visit was not possible. Telephone or email follow-up was performed where face-to-face follow-up was not possible.

Cognitive testing was done at all follow-ups as described previously^10,11 using at least one of the MMSE,⁵ MoCA⁶ and TICSm,¹⁶ all of which have been validated against the National Institute of Neurological Disorders and Stroke-Canadian Stroke Network (NINDS-CSN) Vascular Cognitive Impairment Harmonisation Standards Neuropsychological Battery.^7,8,26 The MMSE was done at all time-points until 1^st April 2005 when the baseline MMSE was replaced by the 10 point Abbreviated Mental Test Score (AMTS).¹⁷ From April 2007, the MoCA was introduced for all follow-ups from 6 months onwards. The TICSm (2002-2009) or T-MoCA (2009-)²⁷ was done by telephone when face-to-face follow-up was not feasible. Reasons for cognitive untestability and problems with cognitive testing in otherwise testable patients including visual impairment, hemiparesis, and dysphasia were recorded.²⁸ The methods of dementia ascertainment have been described elsewhere.^10,11

Statistical Analysis

Surviving patients were classified as “assessed” if they had any type of study assessment vs “not-assessed” if they did not. Within the assessed group, patients were classified as “testable” if a cognitive test could be undertaken vs untestable if it could not. Within the testable group, patients were classified as “fully testable” if there was no problem interfering with testing vs “testable but with a problem” if there was a problem such as poor vision that interfered with testing. Reasons for lack of study assessment were examined in the two previous papers on baseline selection bias, and attrition on follow-up of patients included in the study at baseline.^10,11 For this paper, we examined the testability (with a short cognitive test) of the group of patients who had received a study assessment at each given time-point. The characteristics of the testable and untestable patients were compared within the group of assessed patients and non-assessed survivors were not included in the untestable group since testing was never attempted in these patients. Demographic and clinical differences between dead and surviving, tested and untestable patients and those completing a test versus those with problems with testing were compared using analysis of variance (ANOVA) or χ² test as appropriate. Hazard ratios for death before next follow-up were calculated for untested versus testable patients adjusted for age and event severity.

Results

One thousand two hundred thirty-six patients were ascertained (mean age/sd 75.2/12.1 years, 582 (47%) male and 403 (33%) TIA, 370 (30%) major stroke, 65 (5%) primary intracerebral haemorrhage, 992 (80.1%) first ever events). Rates of study assessment, cognitive testability and problems with testing in otherwise testable patients at baseline and for each follow-up point to 5 years are shown in figure 1. The median (IQR) time from event to baseline assessment was 4 (2-8) days. Among assessed survivors, numbers testable with a short cognitive test were 835/1097 (76%) at ascertainment, 778/947 (82%) at 1 month, 756/857 (88%) at 6 months, 692/792 (87%) at 1 year and 472/567 (83%) at 5 years (figures 1-3, supplemental data tables at http://stroke.ahajournals.org). 88% (331/378) of assessed TIA patients were testable at baseline compared to only 46% (133/290) of assessed major stroke survivors (figure 3). The proportion of untestable major stroke survivors fell with time after event whereas the proportion of untestable TIA survivors rose (figure 3). Testable patients were younger, and had less pre-morbid disability and less pre- and post-event dementia, even after adjustment for age, than untestable patients (all p<0.001, figure 3, supplemental data tables I-V at http://stroke.ahajournals.org).

Figure 1.

Numbers and characteristics of patients dead, assessed and not-assessed, untestable vs testable, fully testable vs testable but with a problem, at baseline and at each follow-up.

Figure 3.

Proportion of patients in the cohort with TIA (top), minor stroke (middle) and major stroke (bottom) who were dead, not assessed, untestable, tested but with a problem and fully tested, by time since event.

Untestability at baseline was associated with greater risk of death on follow-up even after adjustment for age and severity of index event (adjusted hazard ratio (HR) for death before 5 years was 1.97, 95% CI 1.59-2.44, p<0.001 with a higher risk for early death before one month (HR= 5.54, 1.71-10.14, p<0.001)). Similarly, untestability at all time-points after baseline was associated with increased likelihood of death before the next follow-up although this became less marked with increasing time since event: HR=4.37 (2.55-7.51, p<0.001) at one month, HR=3.17 (1.56-6.28, p=0.11) at six months and HR=1.60 (1.12-2.26, p=0.004) at 1-year.

Reasons for being untestable are shown in the table. The relative frequencies of the reasons for untestability changed over time: non-testability at baseline (n=262) was frequently caused by stroke-related issues such as dysphasia (75 (29%)), reduced conscious level (58 (22%)) or acute confusion (11 (4%)) whereas non-availability for testing, either face-to-face or telephone (33/95 (33%) was more important at 5-year follow-up. At 5-year follow-up, when both the MMSE and MoCA were administered, 7 tested patients did only one of the two tests. N=4 had MMSE only (n=1 very deaf and did not attempt the MoCA, n=1 declined MoCA as he did not want to answer any more questions, n=1 the wife did not wish the researcher to continue, n=1 patient had dementia and became very upset by his inability to do the MMSE). N=4 had MoCA only (n=1 the researcher missed out the MMSE, n=1 care home patient was too restless and agitated to do the MMSE, n=1 frail patient was too fatigued to do the MMSE as well as the MoCA).

Within the group of testable patients, the proportion with problems interfering with testing were similar at ascertainment (71/835 (9%)) and thereafter: 73/778 (9%) at 1 month, 63/756 (8%) at 6 months, 58/692 (8%) at 1 year and 63/472 13%) at 5 years (figures 1-3, table) Testable patients with problems interfering with testing were older, had more severe events, more pre-morbid dependency and more dementia than those without testing problems (supplementary tables I-V at http://stroke.ahajournals.org). Over a half of problems with testing at baseline were caused by acute stroke effects (39/71 (56%): acute confusion n=15 (21%), hemiparesis n=14 (20%), dysphasia n=7 (10%) and being unwell n=3 (4%)) whereas visual impairment/deafness was the predominant problem at later time points (n=20 (28%) at baseline versus n=24 (33%), n=23 (37%), n=23 (40%) and n=24 (38%) at 1 month, six months, one and five years, table).

Table.

Reasons for untestability or for problems with testing in testable patients by follow-up time-point.

	Follow-up time
	Baseline	1 month	6 months	1 year	5 years
Tested with no problem*	764 (69)	705 (74)	693 (81)	637 (80)	409 (72)
Untestable, reason
Dysphasia	75 (29)	28 (17)	14 (14)	13 (13)	7 (7)
Unwell/low GCS	58 (22)	16 (9)	7 (7)	7 (7)	2 (2)
Moved away	N/A	8 (5)	27 (27)	30 (30)	33 (33)
Visual impairment/deafness	8 (3)	2 (1)	0 (0)	1 (1)	1 (1)
Hemiparesis	1 (<1)	2 (1)	1 (1)	0 (0)	1 (1)
Dementia	11 (4)	10 (6)	10 (10)	6 (6)	6 (6)
Acute confusion/delirium	11 (4)	1 (<1)	0 (0)	0 (0)	0 (0)
Dysarthria	8 (3)	0 (0)	0 (0)	0 (0)	0 (0)
No English	5 (2)	6 (4)	4 (4)	6 (6)	5 (5)
Declined cognitive test	3 (1)	5 (3)	2 (2)	2 (2)	3 (3)
Late ascertainment/referral	40 (15)	54 (32)	13 (13)	1 (1)	0 (0)
Recurrent event	1 (<1)	3 (2)	14 (14)	19 (19)	28 (28)
Other/no reason recorded	41 (16)	32 (19)	9 (9)	4 (4)	9 (9)
Total	262 (100)	169 (100)	101 (100)	97 (100)	95 (100)
Testable but problem, reason
Visual impairment/deafness	20 (28)	24 (33)	23 (37)	23 (40)	24 (38)
Hemiparesis	14 (20)	16 (22)	9 (14)	4 (7)	5 (8)
Dysphasia	7 (10)	13 (18)	15 (24)	17 (29)	7 (11)
Acute confusion/delirium	15 (21)	0 (0)	0 (0)	0 (0)
Other neurological disorder	2 (3)	4 (5)	6 (10)	4 (7)	2 (3)
Unwell	3 (4)	3 (4)	1 (2)	1 (2)	0 (0)
English second language	4 (6)	2 (3)	4 (6)	3 (5)	2 (3)
Mechanical problem eg arthritis, arm fracture	1 (1)	2 (3)	1 (2)	0 (0)	5 (8)
Illiteracy	1 (1)	2 (3)	2 (3)	1 (2)	0 (0)
Dementia	3 (4)	4 (5)	1 (2)	3 (5)	8 (13)
Incomplete test	1 (1)	0 (0)	0 (0)	1 (2)	1 (2)
Other/no reason recorded	0 (0)	3 (4)	1 (2)	1 (2)	9 (14)
Total	71 (100)	73 (100)	63 (100)	58 (100)	63 (100)

Numbers are n (%). *Percentage expressed in relation to the total number assessed at that timepoint.

Discussion

In our longitudinal study of short cognitive tests in over 1000 patients with TIA and stroke from a defined population, a quarter of all assessed patients were untestable at baseline including nearly a half of those with major stroke. Around a sixth of those tested at baseline had a problem interfering with testing usually from stroke-related impairments such as dysphasia. On follow-up, rates of untestability and of problems interfering with testing were generally lower. As well as severity of event, untestability was associated with older age, premorbid dependency and both pre- and post-event dementia.

We designed our study using short cognitive tests rather than a neuropsychiatric battery to facilitate obtaining cognitive data on as many patients as possible in a large pragmatic study which included multiple non-cognitive assessments. Our findings demonstrate that requiring completion of even a short cognitive test as a condition of entry into a study or to measure cognitive outcome will result in a selected unrepresentative subgroup of patients. This effect will likely be greater in studies using extensive neuropsychological batteries which are poorly tolerated by frail elderly patients although there are few reported data. Untestability and problems with testing in testable patients were associated with risk factors for post-stroke dementia and it was thus unsurprising that rates of both pre- and post-event dementia were significantly higher in these groups. Requiring completion of a cognitive test as a condition of study entry or to assess cognitive outcomes will therefore result in under-estimation of the true cognitive impairment rate.²⁹

The problem of missing cognitive data resulting from untestability will be greatest in studies of severe (hospitalised) stroke and least in studies of TIA. The vast majority of TIA patients and most minor stroke patients were testable at baseline and rates of untestability only rose at 5 years when patients moved out of study area and telephone assessment was not feasible. The proportion of testable survivors increased with time since event probably because of a combination of high death rates in untestable patients and some recovery in survivors, particularly in those with major stroke.^11,30 Two recent studies of hospitalised stroke patients have shown slightly higher baseline testability rates using the MoCA although the cohorts were overall younger than in the current study.^31,32

Concerning the specific reasons for untestability, acute stroke effects including, in order of prevalence, dysphasia, reduced conscious level, hemiparesis and delirium were common in the acute phase with findings similar to those of the Heidelberg study.³² Residual effects of stroke prevented testing less commonly at later time points when sensory difficulties (eg deafness) and moving away became more important. Rates of dementia severe enough to preclude testing remained relatively constant. Similarly, problems interfering with testing were related to acute stroke effects soon after the event whereas unrelated problems such as poor vision were more important at later follow-ups. Sensory deficits or other non-stroke related problems such as dominant arm disability, fatigue and frailty will be an issue for any study assessing cognition (and indeed will have meant that some patients would have been untestable even prior to their cerebrovascular event) but such data are rarely included in published studies^4,9 despite the fact that such testing difficulties may lead to inappropriate classification of cognitive impairment or exclusion of those at-risk.^12–15

Strengths of our study include the ascertainment and follow-up to 5-years and beyond of all TIA and stroke from a defined population and the careful prospective documentation of reasons for untestability and problems with testing. Our data further the understanding of the applicability of short cognitive tests to the TIA and stroke population, many of whom are elderly with significant co-morbidity. There are however, some limitations to our study. First, we did not formally compare the feasibility of the different short tests but very few patients completed the MMSE and not the MoCA at 5 years when both tests were administered (ie patients completed both or neither) and where only one test was done, this was usually because of fatigue or patient/carer distress. Second, reasons for untestability included lack of patient availability to do a cognitive test in those who were assessed ie because the patient had moved away and was unable to do a telephone test or because information was received via an informant but this is likely to be an issue for all longitudinal studies with long term follow-up. Third, researchers did not always record the reason for lack of cognitive test and in some cases this may have been for logistical reasons such as lack of time rather than patient factors.

In conclusion, untestability with a short cognitive test after TIA and stroke was associated with risk factors for post-stroke dementia including older age and more severe stroke and nearly a half of those assessed with major stroke were untestable at baseline. Large studies or trials relying on cognitive test results alone to inform cognitive impairment should report data on untestable patients, correct for problems interfering with testing and include risk-factor adjusted estimation of probability of impairment in non-tested patients.

Figure 2.

Proportion of patients in the cohort overall who were dead, not assessed, untestable, tested but with a problem and fully tested, by time since event.