External validation of a six simple variable model of stroke outcome and verification in hyper‐acute stroke

Abstract

We aimed to validate a previously described six simple variable (SSV) model that was developed from acute and sub‐acute stroke patients in our population that included hyper‐acute stroke patients. A Stroke Outcome Study enrolled patients from 2001 to 2002. Functional status was assessed at 6 months using the modified Rankin Scale (mRS). SSV model performance was tested in our cohort. 538 acute ischaemic (87%) and haemorrhagic stroke patients were enrolled, 51% of whom presented to hospital within 6 h of symptom recognition. At 6 months post‐stroke, 42% of patients had a good outcome (mRS ⩽2). Stroke patients presenting within 6 h of symptom recognition were significantly older with higher stroke severity. In our Stroke Outcome Study dataset, the SSV model had an area under the curve of 0.792 for 6 month outcomes and performed well for hyper‐acute or post-acute stroke, age < or ⩾75 years, haemorrhagic or ischaemic stroke, men or women, moderate and severe stroke, but poorly for mild stroke. This study confirms the external validity of the SSV model in our hospital stroke population. This model can therefore be utilised for stratification in acute and hyper-acute stroke trials.

The six simple variable (SSV) model comprises easily collected reliable variables: age, pre‐stroke functional status, living alone pre‐stroke, being able to walk unaided, lift both arms off the bed and have a normal verbal Glasgow Coma Score. It predicts independent survival at 6 months following stroke as well as more complex models, and has been externally validated with area under the receiver operating characteristic curve (AUC) of 0.84–0.88.¹,² It has been used to adjust for case mix when comparing the quality of hospital based stroke services³ and for stratification in randomised trials.⁴ The SSV model was developed in patients presenting up to 30 days post‐stroke and has been validated in those presenting within 2 days of stroke.¹ It has not been previously validated in hyper‐acute stroke patients.

Methods

We enrolled patients with stroke admitted consecutively between 2001 and 2002 to the stroke service at the Halifax Infirmary in the Stroke Outcomes Study. A neurologist collected clinical variables, including a stroke severity score (severity scored as mild (1–4), moderate (5–7) and severe (8–10), depending on symptoms, signs and functional impairment)⁴ and the six simple variables¹ during the first assessment. All patients had cranial CT or MRI performed acutely. The main outcome measure was independent survival (modified Rankin score (mRS) ⩽2) assessed at 6 months post‐stroke by telephone interview by a single assessor trained in administering the mRS. At follow‐up, patient information from the discharge summary was available.

Data are presented as mean (SD) unless otherwise stated. Comparisons between groups were made using χ² and Mann–Whitney tests where appropriate, with significance at p<0.05. SSV model prediction was calculated for each patient with a stroke using the published coefficients.¹ Model discrimination was assessed using the AUC, computed by a non‐parametric method.⁵ An AUC of 1 implies perfect discrimination whereas an AUC of 0.5 implies the model performs no better than chance. The AUC for alive and independent at 6 months was determined according to the following subgroups which we felt were clinically important: age <75 or ⩾75 years old, the presence or absence of intracerebral haemorrhage, stroke severity and time to presentation (<6 h or ⩾6 h). Calibration was assessed using calibration curves (observed versus predicted probability of a good outcome). To estimate the standard errors of AUCs and the confidence intervals of the observed probability of good outcome, bootstrapping was used by re‐sampling 500 times. Mean AUC values were compared using analyses of variance. All analyses were conducted using SAS software, version 9.1 (SAS Institute Inc., Cary, North Carolina, USA).

Results

Of 598 patients admitted to the stroke service between 2001 and 2002, 38 refused consent, 13 had repeat admissions (second admission excluded) and nine were lost to follow‐up, leaving a final study group of 538 patients (70 haemorrhagic and 468 ischaemic strokes) (table 1). Forty‐seven per cent of patients were women. Compared with patients presenting ⩾6 h after stroke symptom recognition, hyper‐acute patients (ie, presenting <6 h) were significantly older, with higher stroke severity (table 1). At 6 months post‐stroke, 42% had a good outcome (mRS ⩽2) and 24% were dead.

Table 1 Patient characteristics and comparison of hyper‐acute versus post‐acute stroke patients.

	Enrolled patients	Hyper‐acute (<6 h)	Post‐acute (⩾6 h)	OR of a good outcome
n	538	273	265
Stroke severity score*	6 (5–8)	7 (6–9)	6 (5–7)††	0.49 (0.43–0.56)††
Haemorrhagic stroke	70 (13%)	32 (12%)	38 (14%)	0.58 (0.26–1.00) †
Received tissue plasminogen activator	29 (5%)	29 (11%)	0††	0.60 (0.23–1.40)
Six simple variables
(1) Age*	74 (61–80)	75 (65–81)	71 (59–79)††	0.95 (0.91–0.98)††
(2) Living alone pre‐stroke	135 (25%)	64 (23%)	71 (27%)	1.1 (0.7–1.6)
(3) Independent pre‐stroke	437 (81%)	214 (78%)	223 (84%)	53 (14–447)††
(4) Verbal GCS = 5	347 (65%)	145 (53%)	202 (76%)††	7.1 (4.5–11.5)††
(5) Able to lift both arms off bed	353 (66%)	147 (54%)	206 (78%)††	9.2 (5.6–15.5)††
(6) Able to walk without assistance	150 (28%)	54 (20%)	96 (36%)††	7.6 (4.8–12.1)††

GCS, Glasgow Coma Score.

*Data are expressed as median (interquartile range) and odds ratios with 95% CI. Odds ratios for age and stroke severity are per unit.

†p<0.05, ††p<0.001.

Testing the SSV model (variables listed in table 1) for a good outcome at 6 months produced an AUC of 0.792 (SE 0.024) with good calibration curves (available from authors). The SSV model performed well for different subgroups: age < or ⩾75 (AUC 0.800 (0.017) vs 0.846 (0.020); NS), haemorrhage versus infarct (0.846 (0.038) vs 0.779 (0.031); p<0.05) and hyper‐acute versus post‐acute (0.802 (0.031) vs 0.761 (0.034); NS). It performed reasonably for moderate (0.675 (0.033)) and severe stroke (0.782 (0.026); p<0.001 compared with moderate and mild stroke) but no better than chance for mild stroke (0.457 (0.029)). Haemorrhagic stroke had a higher median stroke severity compared with ischaemic stroke (8 vs 6; p = 0.0002).

Discussion

We confirm that the SSV model for predicting independent survival at 6 months has external validity in our stroke population and we demonstrate for the first time that the model performs well in a large population of hyper‐acute stroke patients, 11% of whom received thrombolysis. As shown previously,¹ the SSV model performed less well in minor stroke. The reason for this needs further analysis, however, patients initially seen with mild strokes at first assessment may develop stroke progression or recurrence, or a new illness (such as a myocardial infarction) which is not predicted by the SSV model. Also, an outcome of mRS ⩽2 may be a less discriminating outcome in mild stroke as it was achieved in 84% of mild stroke patients. The model performs significantly better for patients with higher stroke severity and for haemorrhagic stroke, probably because the latter were more severe than ischaemic strokes. The good model performance in haemorrhagic stroke is important as the original study from which the SSV model was developed may have underestimated the proportion of patients with haemorrhagic stroke.¹

The SSV model uses variables that can be easily collected compared with some models that use scales that require training (eg, the National Institutes of Health Stroke Scale). This is noteworthy given that non‐neurologists routinely assess the majority of stroke patients in hospitals worldwide. It would be of interest to directly compare SSV model performance with models that use other stroke scales in validation cohorts.

Our population reflects inpatients from a tertiary stroke referral centre and teaching hospital, and the SSV model would benefit from further validation in less academic units. Our study benefits from the low rate lost to follow‐up and the high consent rate.

In conclusion, this study confirms the external validity of the SSV model in hospitalised stroke patients, providing the first evidence of validity in hyper‐acute strokes. The SSV model can therefore be utilised for stratification in acute stroke trials. However, its use in clinical management (particularly in selecting which patients are suitable for specific treatments—eg, thrombolysis) cannot be recommended until it has been evaluated in randomised controlled trials.

Abbreviations

AUC - area under the receiver operating characteristic curve

mRS - modified Rankin score

SSV - six simple variable