Outcome Prediction in Acute Stroke Patients Considered for Endovascular Treatment: a Novel Tool

Summary

Functional outcome following emergent intra-arterial thrombectomy is variable and likely reflects the heterogeneous characteristics of acute stroke patients.

The aims of our study were (1) to study which pre-treatment variables correlate with functional outcome and (2) to devise a tool which would reliably predict outcome.

Prospective data of patients treated with intra-arterial mechanical thrombectomy in our institution between 2010 and 2012 were collected. A preliminary univariate analysis of baseline variables was performed and data outliers were identified by constructing scatter and box plots.

Systematic bivariate analysis was then carried out using a linear regression model and the individual contributing weights of the variables to outcome calculated. The B and constant values from the regression were used to construct a predictive formula.

Fifty-seven patients, 35 males (61.4%) and 22 females (38.6%) with a mean age of 62.3 years (range 26-87) were included in the cohort. Statistical correlations of baseline variables and functional outcome showed that age, National Institutes of Health Stroke Scale at presentation and CT leptomeningeal collaterals were strongly correlated (p<0.01), and were later included in the linear regression model. A tool was devised from the regression formula combining weighted inputs of the three variables. Regression statistics and residual analysis were then performed to assess the accuracy and reliability of the proposed tool.

The proposed tool is easy to use and reliably predicts functional outcome prior to endovascular therapy. It may help clinical decision-making in the acute setting and offers ‘tailor-made' outcome expectations.

Introduction

Stroke remains a major global cause of mortality and disability. Stroke prevention and treatment have evolved rapidly in recent years. Clinicians and researchers alike have explored the role of acute stroke treatment which would potentially relieve the burden of this devastating condition. Promising approaches are likely to include a combination of intravenous and intra-arterial revascularization techniques. Though the PROACT II trial ¹ stands as a proof that vessel recanalization should be beneficial, recent randomized control trials ^2-4 have failed to demonstrate additional benefits compared to standard stroke treatment. It is clear that a way to better identify those patients who might benefit most from endovascular therapy needs to be developed.

Decisions to proceed with endovascular treatment are often complex and guidelines to aid such process are in general limited.

With this in mind, we used inference statistics to build a tool to predict long-term functional outcome in patients undergoing endovascular stroke treatment. The novel tool may aid patient selection and identify those instances where the benefits of acute intra-arterial stroke therapy outweigh potential risks.

Methods

Patients and Techniques

We recruited consecutive patients who presented with an anterior circulation ischaemic stroke (total anterior (TACI) or partial anterior (PACI) subtypes as defined by the OCSP (Oxfordshire Community Stroke Project) Classification) and were referred for intra-arterial mechanical thrombectomy at our institution between 1^st January 2010 and 31^st December 2012.

Baseline variables including the patient’s demographical data, background medical history and details of the acute presentation were recorded at the time of presentation and entered into a purpose-built departmental stoke database. All the patients were assessed by a specialist stroke consultant at the time of presentation, and underwent a non-enhanced CT, followed by CT angiography. Presenting stroke severity was measured using the National Institutes of Health Stroke Scale (NIHSS). A decision to proceed with intra-arterial thrombectomy was taken in the light of clinical and imaging findings, including confirmation of large vessel occlusion on CT angiography (CTA). A treatment window of eight hours was used (in anterior circulation stroke) for mechanical thrombectomy.

Procedural details including time to intra-arterial therapy, the length of the procedure, the type of device used and the number of device passes were recorded.

Functional outcome was measured by the caring physician at the end of the procedure and at 24 hours using the NIHSS. Long-term outcome was assessed at 30 and 90 days post-procedure using the modified Rankin score (mRS).

Our institution is a tertiary referral centre for acute stroke, and has been offering acute endovascular treatment for acute stroke since 2009. We used inference statistics to construct a predictive outcome tool.

Statistical Analysis

A preliminary univariate analysis was performed and data outliers were identified by constructing scatter and box plots. Two cases were excluded from the cohort as the time of presentation was well outside the treatment window in the first case, and the quality of CT angiography was sub-optimal in the second case.

Systematic bivariate analyses were then performed to assess for correlation of baseline variables with good outcome (mRS 0-2) at 90 days. Statistical correlations were performed using Pearson correlation and chi-square tests for continuous and categorical data respectively. Correlations with a p value of <0.05 were considered statistically significant and were later included in a linear regression model. Assumptions of predictor variables used in the linear regression were met. This involved the performance of the lack of fitness test.

All statistical analyses were carried out using IBM SPSS Statistics for Windows v20.0 (IBM Corp, Armonk, NY, USA).

The constant and beta values of the three baseline variables from the regression were used to construct a predictive formula.

The formula:

$${\displaystyle y=B_1{x}_1+B_2{x}_2+B_3{x}_3+c}$$

is a modification of the basic linear regression formula where y is the outcome (mRS at 90 days), x represents the three variables (age, NIHSS at presentation and CT collaterals), B are the weighted inputs of the three variables and c is the constant produced by the linear regression.

The patient database was exported to and the predictive formula included in a Microsoft Excel™ worksheet (Microsoft, 2007, Redmond, WA, USA) and comparisons of the functional and predicted outcome using the formula were performed. The results were included in a 2×2 contingency table and the sensitivity, specificity, positive (PPV) and negative (NPV) predictive values and accuracy of the tool were calculated. Regression statistics and residual analysis were also performed in SPSS™ to assess the accuracy and reliability of the tool.

Finally, correlations of outcome scores (mRS) and outcome groups (poor vs good) between the predicted and clinical outcomes were performed.

The main advantages of the proposed tool are that it uses absolute values for age and NIHSS as opposed to ‘cut points’ to categorise these variables and the weights of the individual variables in the formula are statistically derived.

The proposed tool was further validated on 15 consecutive acute stroke cases that presented to our institution following development of the tool.

Results

Fifty-five patients, 34 males (61.8%) and 21 females (38.18%) with a median age of 62.8 years (range 26-87) were included in our cohort. The mean NIHSS at the time of presentation was 15.8 (range 6-27) and the average time to intra-arterial therapy was 300.1 minutes (range 135-480). Demographic and procedural details are summarised in Table 1. Twenty-nine patients had a good functional outcome (mRS 0-2) at 90 days while 26 patients had a poor outcome (mRS 3-6). The stratified 90 day functional outcome is shown in Figure 1.

Table 1.

Clinical and procedural details of the primary cohort.

	N (%)	Mean	Range	SD
DEMOGRAPHICS
Age	55	62.75	26 - 87	13.71
Gender Male Female	34 (61.81%) 21 (38.18%)	−	−	−
Transfer from external institution Yes No	30 (54.55) 25 (45.45)	−	−	−
CLINICAL INFORMATION
NIHSS at presentation	55	15.81	6 - 27	5.97
Side of occlusion Right Left	29 (52.72%) 26 (47.27%)	−	−	−
Intravenous thrombolysis Yes No	41 (74.55%) 14 (25.45%)	−	−	−
CT collaterals 2 Poor (grade 0-1) Fair/Good (grade 2-4)	23 (41.81%) 32 (58.18%)	−	−	−
PROCEDURAL INFORMATION
Time to IA therapy (min)	55	294.49	135 - 480	108.45
Length of procedure (min)	55	68.78	19 - 248	42.06
Number of device passes	55	2.41	1 - 8	1.69
CLINICAL OUTCOME
mRS at 90 days (continuous)	55	2.65	0 - 6	2.26
mRS at 90 days (dichotomized) Poor (mRS3-6) Good (mRS 0-2)	26 (47.3%) 29 (52.7%)	−	−	−

Figure 1.

We first performed correlation analysis of baseline variables with functional outcome and the results are summarized in Table 2. Age, presentation NIHSS and CT leptomeningeal collateral score ⁵ were shown to have statistically significant associations (p<0.05) with functional outcome at 90 days.

Table 2.

Correlation analysis of baseline variables with 90 day functional outcome.

Baseline Variables	N	Mean	Range	Std. Dev.	CORRELATIONS with 90 day mRs (continuous)
					p	Test
1. Age (continuous)	55	62.8	26-87	13.7	0.001280**	PC
2. Gender Male Female	34 21	−	−	−	0.291679	CS
3. NIHSS at presentation (continuous)	55	15.8	6-27	6.0	0.003266**	PC
4. Time to i.a. therapy	55	300.7	135-586	115.8	0.133302	PC
5. i.v. thrombolysis Yes No	41 14	−	−	−	0.237911	CS
6. Side affected Right Left	29 26	−	−	−	0.469910	CS
7. Leptomeningeal collaterals on CT (dichotomized) Poor Fair/Good	23 32	−	−	−	0.008747**	CS
8. Tandem ICA stenosis Yes No	18 37	−	−	−	0.211304	CS
(CS = chi-square test, PC = Pearson correlation test) - ** - Correlation is significant at the 0.01 level (2 tailed) - * - Correlation is significant at the 0.05 level (2 tailed)

The three variables were then included in a linear regression model (one-way ANOVA). The R and R² values of the model were 0.683 and 0.467 respectively. The value of the F statistic of the ANOVA analysis was 8.531. The analysis produced a constant value of −0.729 and individual beta values for the three variables as follows: Age = 0.049, NIHSS = 0.071 and CT collaterals =−1.456. The individual variables retained statistical significance (p < 0.05) within the model.

Replacing the values in the formula:

$${\displaystyle y=B_1{x}_1+B_2{x}_2+B_3{x}_3+c}$$

the derived values from the linear regression yield our predictive equation:

$${\displaystyle \begin{array}{c}\mathrm{mRS}@\mathrm{90days}=(0.049\ast \mathrm{age})+(0.071\ast \mathrm{NIHSS})+\\ +(-1.456\ast \mathrm{CT}\mathrm{collaterals})-0.729\end{array}}$$

The clinical and predicted mRS scores were used to produce a 2×2 contingency table shown in Figures 2 and 3. The tool had a sensitivity and specificity of 93.10% and 57.69% respectively for predicting a poor outcome (mRS = 3-6). The positive predictive value and negative predictive value were 71.05% and 88.24% respectively and the overall tool accuracy was 76.36%. False positives in the Table represent a type 1 error whereas the false negatives correspond to a type 2 error.

Figure 2.

Figure 3.

A scatter plot was constructed by plotting the functional outcome against the regression adjusted predicted value and is shown in Figure 4. A line of best fit was drawn to assess the correlation between the dependent and predicted values. The trend line had a slope which approached unity and therefore suggests reliable predictions. A plot of regression standardised predicted values against regression standardised residuals (Figures 5 and 6) showed symmetric distribution of the scatter points around the 0 mark with a line of best fit which intersects the 0 point on the y-axis. This again confirmed a lack of systematic bias in the predictive tool.

Figure 4.

Figure 5.

Figure 6.

Residual statistics showed a mean standard predicted value of 0.0 with a standard deviation of 1.0. The mean standard error of the predicted value was equal to 0.59 with a standard deviation of 0.11. The Pearson coefficient for comparisons of predicted and clinical outcome mRS scores was 0.503373 with a 2-tailed p value of 0.000194. Similarly the correlation between the predicted and clinical outcome groups (poor vs good outcome) was highly significant with a p value of 0.001863 (chi-square test).

The tool (Figure 7) performed equally well in the validation phase where it was tested on 15 cases not belonging to the cohort from which the tool was devised. The predicted outcomes were highly correlated with the 90 day functional outcome with a p value of 0.003096 in the validation phase (chi-square test).

Figure 7.

Discussion

The US Food and Drug Administration (FDA) approved intravenous recombinant tissue plasminogen activator (rt-PA) for the treatment of ischaemic stroke following the results of the National Institute of Neurological Disorders and Stroke (NINDS, 1995) trial ⁶ which showed an improved 90 day functional outcome in patients receiving intravenous rt-PA compared to placebo. The initial three hour window was revised and extended to 4.5 hours following the ECASS 3 trial ⁷. The latter also prompted the European Stroke Organization to include intravenous thrombolysis in its 2009 Stroke Guidelines and to recommend a 4.5 hour window.

Despite a wider time window, intravenous thrombolysis still presents numerous challenges. A significant number of patients fail to present within the necessary time window ⁸ and the efficacy of treatment decreases with time post ictus. The early recanalization rate of intravenous rt-PA is just 30−50%1,9 and re-occlusions are frequent ^10,11.

Intra-arterial options may offer alternative treatments for acute ischaemic stroke in patients with large vessel occlusion. The main goal of intra-arterial stroke therapy is to restore flow to the ischaemic territory by recanalizing the primary arterial occlusive lesion allowing reperfusion of the distal arterial bed ¹². Successful recanalization was shown to correlate with improved functional outcomes ^13,14.

A clear benefit of administering prourokinase with a catheter-based approach directly into the clot was demonstrated in the PROACT-II trial ¹.

Intra-arterial mechanical thrombectomy is an option in patients who do not respond to intravenous thrombolysis ¹⁵. Two single-arm studies of the Merci device (the MERCI trial ¹⁶ and the Multi MERCI trial ¹⁷) showed a significantly higher successful recanalization rate of large occluded arteries in acute ischaemic stroke compared to the control arm of PROACT-II ¹. PROACT II ¹ proved that vessel recanalization in acute ischaemic stroke was beneficial. An independent association between vessel recanalization and improved functional outcomes was proved in the MERCI ¹⁶ and Multi MERCI trials ¹⁷.

The second generation of endovascular devices, namely the Trevo retriever and the Solitaire device, are both types of retrievable stents and have recently gained regulatory approval in the US.

The Solitaire device was shown to achieve better recanalization rates and improved clinical outcomes compared to the Merci retriever (a first-generation device) in the SWIFT trial ¹⁸, a small randomized controlled trial published in The Lancet. Similarly TREVO 2 ¹⁹, a multicentre randomized control trial, proved that the Trevo retriever was safer and more efficient when compared to its predecessor, the Merci device.

Three recently published randomized controlled trails, namely the Mechanical Retrieval and Recanalization of Stroke Clots Using Embolectomy ² (MR RESCUE), SYNTHESIS ³ and IMS III ⁴ published in the New England Journal of Medicine, apparently failed to prove that endovascular therapy was superior to standard medical care in acute ischaemic stroke patients. These trials demonstrated that first-generation thrombectomy devices or intra-arterial thrombolysis do not significantly improve long-term functional outcomes compared with intravenous therapy. The National Institute of Neurological Disorders and Stroke (NINDS) indefinitely put on hold its randomized controlled trial IMS III ⁴, which sought to enrol up to 900 patients at more than 50 centres in the US to assess whether a combined intravenous and intra-arterial approach to recanalization was superior to standard IV rt-PA, citing ‘futility' as the cause for ending the trial. These trails, however, do not necessarily reflect the current practice in most stroke centres offering intra-arterial thrombectomy treatment for acute ischaemic stroke. A second generation stent retriever device was only used in four patients in the IMS III trial ⁴. In SYNTHESIS ³ the patients receiving intra-arterial therapy did not receive intravenous rt-PA, which is not the case in our institution where emergent intravenous thrombolysis is administered uniformly unless contraindicated. The generalizability of MR RESCUE ² is hampered by the limited use of IV therapy, and the heterogeneity of imaging and site expertise throughout the study.

The Solitaire FR Thrombectomy for Acute Revascularization (STAR) study ²⁰ was a prospective, single arm clinical study which recruited 202 patients from 14 centres across Europe, Australia and Canada. The preliminary neurological outcomes achieved would support the use of the Solitaire FR device (a second generation thrombectomy device) in acute ischaemic stroke care. A good 90 day functional outcome (mRS = 0-2) was achieved in 57.9 % and reported a mortality rate of 6.9%. However, this trial was not randomized.

There is therefore hunger in the stroke community for truly randomized endovascular device clinical trials. Two randomized trials, namely THERAPY ²¹ and Solitaire FR with the Intention for Thrombectomy as Primary Endovascular Treatment for Acute Ischemic Stroke (SWIFT PRIME ²²), are now in their embryonic phases.

Pragmatic Ischaemic Stroke Thrombectomy Evaluation (PISTE ²³) is a randomized trial based in the UK where enrolment is about to commence. The trial aims to be ‘pragmatic' and relevant to current practice.

Decisions regarding patient eligibility for endovascular treatment are complex and driven by thorough clinical assessment in the light of imaging findings. The benefit of mechanical thrombectomy is likely limited to a particular patient group. There is a need to better define which patients would benefit most from endovascular stroke treatment.

With this in mind we aimed to devise a tool to aid the selection process and therefore treat patients who would benefit most, without exposing patients unlikely to benefit from the procedure to unjustified risk.

Modifiable (including blood pressure and vessel recanalization) and non-modifiable predictors (like age, imaging findings and medical co-morbidities) have been identified and shown to influence the clinical outcome following acute stroke ²⁴.

Several approaches to tentatively predict the functional outcome following endovascular stroke treatment have been described. Both imaging and clinical parameters have been used in these approaches.

Early ischaemic changes on CT, measured by the Alberta Stroke Programme Early CT score (ASPECTS ²⁵), were found to have a significant correlation with functional outcome among patients in the PROACT-II ¹ study of intra-arterial prourokinase. ASPECTS ²⁵ was found to reliably predict the functional outcome in a graded fashion but struggled to discriminate between individual outcomes.

A scoring system encompassing three baseline variables, namely age, NIHSS, and hyperglycaemia on admission, was recently validated as a predictor for poor functional outcome ²⁶.

The Totalled Health Risks in Vascular Events (THRIVE) Score ²⁷, developed using data from the MERCI and Multi MERCI trials, was found to strongly predict 90-day functional outcomes. The possibility that the predictive power of THRIVE could be improved by introducing imaging findings in the scoring system was raised in the same paper.

Because of a lack of experience with the technique, CT perfusion was not routinely performed for acute large vessel ischaemic stroke at our institution. However, Hopyan et al. demonstrated a statistically significant incremental benefit with CT perfusion when used in combination with non-contrast CT brain and CT angiography in the certainty of stroke diagnosis ²⁸. In this study, the reviewers of CT perfusion data were either in or less than one year post fellowship. This simulates the real time experience where non-expert reviewers may have to review CT perfusion studies. Diffusion-weighted MRI, performed within 30 days of the stroke, was used to determine final infarct volume. The use of all three CT modalities increased the sensitivity of stroke diagnosis by 18.2% and CT perfusion increased the specificity of stroke diagnosis by 20.3%.

There remain limitations and concerns about the use of CT perfusion in this setting, including the time taken to perform and post process the study, expertise required to interpret the same, the reliability of data obtained and additional intravenous iodinated contrast load and radiation dose for the patient ^29,30. Many of these concerns have been addressed in the medical literature ^28,31. CT scanners with increased numbers of detectors can allow for whole head coverage ³². The increased performance capability of CT perfusion enables visualisation of the penumbra or vascular territory at risk of infarction, which cannot possibly be seen with non-contrast CT brain. The ECASS-II trial demonstrated that failure to recognize a large infarct core in the territory of the middle cerebral artery contributed to symptomatic intracranial haemorrhage with intravenous thrombolysis ³³. CT perfusion can serve as a technique to predict which patients are at risk of large infarct core and symptomatic intracranial haemorrhage ³⁴. CT perfusion is rapidly performed (approximately one minute) and post processed (five to seven minutes), offering an enormous and practical time advantage over MRI. Further validation studies will be performed for CT perfusion in acute large vessel ischaemic stroke. As a means of quantifying penumbra and infarct core, it may serve as a reliable component of future outcome prediction scores in patients with acute large vessel ischaemic stroke.

Table 3.

Clinical and procedural details of the validation group.

	N (%)	Mean	Range	SD
DEMOGRAPHICS
Age	15	64.80	43 - 81	10.46
Gender Male Female	8 7	−	−	−
Transfer from external institution Yes No	10 5	−	−	−
CLINICAL INFORMATION
NIHSS at presentation	15	17.33	6 - 29	5.81
Side of occlusion Right Left	6 9	−	−	−
Intravenous thrombolysis Yes No	11 4	−	−	−
CT collaterals 2 Poor (grade 0-1) Fair/Good (grade 2-4)	4 11	−	−	−
PROCEDURAL INFORMATION
Time to arterial puncture (min)	15	199.38	86 - 305	57.03
Length of procedure (min)	15	69.08	20 - 138	34.01
Number of device passes	15	2.50	0 - 6	1.55
CLINICAL OUTCOME
mRS at 90 days (continuous)	15	2.80	0 - 6	2.88
mRS at 90 days (dichotomized) Poor (mRS3-6) Good (mRS 0-2)	7 8	−	−	−

Analysis of the Tool Performance

Our preliminary analysis showed that only three out of the eight baseline variables were significantly associated with long-term functional outcome following stroke. Age, NIHSS and CT collateral score ⁵ maintained their individual correlations with outcome after being combined in a linear regression model.

The proposed tool is easy to use and allows the clinician to insert directly the actual values for age and NIHSS into the formula without the need to categorize these values. Collateral grading ⁵ was also simplified into poor (grade 0-1) and fair/good (grade 2-4), which facilitates the use of the proposed tool.

The tool is reliable and has an accuracy of 76.4%. Acute stroke patients form a heterogeneous group, and it is often impossible to predict the outcome on an individual basis. Though chance plays a substantial role in patient outcome, the R value of the regression model was 0.69, indicating that the baseline variables included in the formula, explain a significant degree of the long-term functional outcome. The tool was shown to have a sensitivity of 93.1% for predicting poor outcome, which shows how it may be valuable in excluding the patients who are not likely to benefit from the treatment. The plot of the dependent and predicted values had a slope approaching unity and therefore offered reliable predictions throughout the outcome scale. Residual analyses also showed a lack of systematic bias of the predictive tool. Finally, the correlations between the predicted and clinical outcome groups (poor vs good outcome) were found to be highly significant.

Strengths of the Proposed Tool

The proposed tool is easy to use and was shown to be both accurate and reliable. It allows the use of absolute baseline values (i.e. age and NIHSS values), without the need to categorise such variables using pre-set cut-offs. The weighting of the different variables in the formula was statistically derived from the linear regression model rather than being assigned an arbitrary value. This reflects the individual predictive power of each of the variables within our dataset. The weighting can easily be derived and replaced to fit different cohorts. The tool produces a predicted 90 day mRS score as opposed to simply suggesting a poor versus a good outcome. The regression statistics also produced values for the standard deviations around this predicted score. This has a strong clinical implication as the predicted long-term outcome is graded and gives a degree of confidence around the prediction.

The use of multiple intra-arterial (including first and second generation) thrombectomy devices increases the generalizability of the tool as it does not reflect the performance of a single device but rather that of everyday clinical practice.

The tool performed equally well in 15 consecutive patients who presented after the development of the tool. This offers a degree of validation (though limited in sample number) in a separate cohort from the one in which it was developed.

Finally, the residual statistics proved the reliability of the predictive tool and failed to show any systematic biases when the predicted and actual outcomes were compared.

Limitations of the Study

The tool was devised in a small cohort (N = 55) of patients presenting to a single centre which reduces the statistical power of the analysis, and makes further subgroup analysis unreliable. Having devised the tool and defined the method of calculation, the same could however be done for a larger cohort of patients, perhaps selecting subgroups for example according to the site of occlusion or combining data from different centres. It also means that the patients had similar characteristics, and the pathway from diagnosis to treatment reflected our own practice. This may have introduced centripetal bias in our study. Validation of the tool was mainly performed using the same cohort from which it was devised. It is known that this type of validation is suboptimal and often results in spurious over positive results. Though the tool was further validated in 15 patients who presented after the development of the tool, a more extensive validation possibly using data from different centres is necessary.

Conclusions

If validated in future analyses, our proposed tool promises to aid patient selection for mechanical thrombectomy in clinical practice. The novel tool is easy to use, and offers reliable ‘tailor-made’ prediction outcomes. Having established a statistical approach to develop this tool, the same approach could be used with larger / different cohorts of patients, perhaps with other assessment techniques including perfusion studies, or looking at cohorts who receive different treatments. It is also important to acknowledge that perfusion studies may have an important role in treatment decision-making in acute ischaemic stroke. Both CT and MR perfusion studies provide crucial information on cerebral perfusion parameters and aim to differentiate infarct core from a salvageable ischaemic penumbra. Perfusion maps may be acquired at the same time as the CT or CTA and will likely aid the decision as to whether to proceed with intra-arterial intervention.

It seems essential that in a setting where the additional value of acute intra-arterial throm-bectomy is being questioned, such predictive tools be developed. Our proposed formula may aid patient selection and determine the instances where the benefits of acute intra-arterial stroke therapy outweigh potential risks. Such a tool would not necessarily provide a ‘cut-off’ point for treatment versus non-treatment but would give a realistic level of expectation of outcome to be used as part of the decision-making process. It also aids the consent process where the risks and benefits reflect a particular situation on the basis of significant baseline variables.

Note Added in Proof

Presentation in part or whole a meetings: European Stroke Conference 2013 and CIRSE 2013.