A Review of Pre-Intervention Prognostic Scores for Early Prognostication and Patient Selection in Endovascular Management of Large Vessel Occlusion Stroke

Abstract

Background

Endovascular therapy (ET) has emerged as a highly effective treatment for acute large vessel occlusion stroke (LVOS). Tools that facilitate optimal patient selection of patients for ET are needed in order to maximize therapeutic benefit in a cost-effective manner. Several pre-intervention prognostic scores for prediction of outcomes in LVOS patients and patient selection for ET have been developed and validated, but their clinical use has been limited. Here, we review existing pre-intervention prognostic scores, compare their prognostic accuracies and levels of validation and identify gaps in current knowledge.

Summary

We have reviewed published literature pertinent to development, validation, and implementation of pre-intervention prognostic scores for LVOS. Using receiver operating characteristic curve analysis, the prognostic accuracies of validated pre-interventional scores (Pittsburgh Response to Endovascular therapy [PRE], Totaled Health Risks in Vascular Events [THRIVE], Houston Intra-Arterial Therapy-2 (HIAT-2), Stroke Prognostication using Age and NIHSS [SPAN-100]) were compared in published work. Pre-intervention scores predicted functional out comes at 3 months with moderate prognostic accuracies (area under the receiver operator characteristic curve range 0.68–0.73). Using successful reperfusion (mTICI 2B/3) as the therapeutic objective of ET and 3-month modified Rankin Score 0–2 as good clinical outcome, patients most likely to clinically benefit from endovascular reperfusion can be identified using the PRE and HIAT-2 scores. Scores that incorporate collateral imaging or perfusion-based estimation of core and penumbra have not been published. Existing scores are predominantly limited to anterior circulation LVOS, and implementation studies of pre-interventional scores are lacking.

Key Messages

Pre-intervention prognostic scores can serve as useful adjuncts for patient selection in ET for acute LVOS. Pre-intervention scores including HIAT-2, THRIVE, SPAN-100, and PRE have comparable moderate prognostic accuracies for good 3-month outcomes and can identify patients who derive maximal benefit from successful reperfusion. Improvements in prognostic accuracy may be achieved by incorporating variables such as collateral status and perfusion imaging data. Implementation and impact studies using pre-intervention scores are needed to guide clinical application.

Introduction

Several recent randomized clinical trials have established the therapeutic efficacy of endovascular therapy (ET) in reperfusion of large vessel occlusion stroke (LVOS) translating to improved long-term functional outcomes [1, 2, 3, 4, 5]. Based on these trial results, revised American Heart Association guidelines support the use of ET in patients presenting within 6 h of symptom onset, Alberta Stroke Program Early CT Score (CT ASPECTS) ≥6 and initial NIHSS ≥6, regardless of intravenous thrombolysis [6]. Recently, ET performed between 6 and 24 h from presumed symptom onset in LVOS patients with clinical-radiographic mismatch was found to be immensely beneficial [7]. These trials have also highlighted (1) the high recanalization rates with newer generation stent-retriever devices, (2) an urgent need for developing strategies that minimize treatment delays, and (3) need for universally applicable and validated tools for optimizing patient selection for ET that combine commonly available clinical and radiographic data [8, 9]. Optimized patient selection criteria for ET are imperative to ensure that no patients who can benefit from ET are missed, while unnecessary procedures in patients with no benefit from ET are minimized [10]. Such selection criteria, if based on universally available clinical and radiographic tools, could also be used to rapidly triage patients and identify LVOS patients who should be expeditiously transferred to comprehensive stroke centers for ET.

Clinical decision-making for LVOS patient selection for ET is a complex and multifactorial process which is performed by vascular neurologists and interventional neuroradiologists at hyperacute and acute stages of stroke care. Clinical variables commonly considered include measures of patient frailty (age, pre-stroke baseline functional status, and comorbidities), clinical stroke severity at presentation (measured by the NIHSS or other scales) and time from last seen normal. Radiographic variables include measures of established core infarct on initial brain imaging (CT ASPECTS or core volume measured by CT/MR), brain tissue at risk of ischemic damage (clinical-radiological mismatch or perfusion-based mismatch), and collateral status.

While the treating physician's clinical judgement based on clinical and radiographic variables remains the primary checkpoint for decision making for ET, the cognitive heuristics used by physicians (Fig. 1) to arrive at a final decision can be highly variable and lead to inter-physician variability and heterogeneity in decision-making [11]. While the degree of inter-physician variability for ET patient selection has not been studied, the existence of such variability is supported by Saposnik et al.[11, 12] who found superior predictive accuracy and precision of the iSCORE over clinical judgement by stroke physicians. Prognostic scores that combine clinical and/or radiographic variables that are available to the clinician prior to ET (pre-intervention scores) have been validated across a wide range of LVOS populations and appear to have moderate prognostic accuracy for good clinical outcome defined as modified Rankin Scale (mRS) score of 0–2 at 3 months. Some of the existing prognostic scores specifically developed for LVOS patients are able to identify the patients most likely to derive benefit from successful reperfusion of an LVO [13, 14]. In the fields of cancer, hemorrhagic stroke, and transient ischemic attack, prognostic scores are widely used as adjuncts in decision making [15, 16, 17], but despite the development and validation of numerous pre-intervention stroke scores for LVOS patients, their application to clinical care has been very limited. The purpose of this review is to summarize existing pre-intervention prognostic scores that can be used for patient selection for ET, assess their level of validation, contrast their prognostic accuracies, and identify knowledge gaps and barriers to implementation that require further research.

Fig. 1.

Cognitive heuristics involved in decision-making by physicians while selecting LVOS patients for ET.

Development and Validation of Prognostic Scores

Prognostic scores are derived based on independent variables identified during statistical model building (typically univariate followed by multivariable regression) to predict an outcome of interest, using large clinical datasets [18]. In the LVOS field, 3-month good outcome is typically defined as mRS 0–2, while mRS 4–6 typically represents a poor functional outcome that is often not associated with a good quality of life [19, 20]. Pre-intervention variables that are typically relevant to decision making for ET, highlighted in Figure 1, include clinical, demographic and radiographic factors. Once independent predictors are identified, the overall model fit (e.g., Hosmer-Lemeshow statistic) and significance of the model (such as Wald statistic or likelihood ratio tests) are assessed. If developing a prognostic scoring system is the goal, the significant variables can be assigned points or can be weighted based on their regression coefficients. Next, the prognostic score is validated in internal and/or external cohorts. External validation in large patient cohorts derived from either institutional registries or randomized clinical trial data is the validation method of choice. Prognostic accuracy is assessed by the C-statistic or area under the receiver operator characteristic curve (AUC) and is compared in derivation and validation cohorts. AUC closer to 1 indicates higher prognostic accuracy while an AUC of 0.5 suggest that the prognostic accuracy of the model is as good as chance. Arbitrary thresholds (> 0.9: excellent, 0.8–0.9: good, 0.7–0.8: fair, 0.6–0.7: poor) are typically used. The degree of agreement between derivation and validation cohorts is also assessed by comparing observed rates of the outcome across strata of the score (quartiles or tertiles). Once the score is validated across several patient cohorts, the ability of the tool to alter clinical decision making or alter patient outcomes needs to be assessed in prospective impact and implementation studies. Impact testing determines whether a validated prognostic score is used by physicians, changes therapeutic decisions, improves clinically relevant process parameters, improves patient outcome, or reduces costs. Subsequently, the score may undergo implementation testing to determine whether actual dissemination of the prognostic score in daily practice is able to guide physicians with their patient management and clinical decisions. Readers are referred to a comprehensive review of prognostic score development, validation, and implementation by Toll et al.[18] for further reading.

Pre-Intervention Prognostic Scores for LVOS and Their Prognostic Accuracies

LVOS is associated with larger infarct volumes as well as higher morbidity and mortality as compared to patients without LVO [21, 22]. The differences in etiologies and outcomes between LVOS and non-LVOS patients strongly suggest that the determinants of long-term clinical outcome are also likely to be different. Several prognostic scores have been developed for acute ischemic stroke patients, and their ability to predict overall long-term morbidity and mortality and hemorrhagic complications with intravenous thrombolysis have been developed (Table 1) [12, 14, 23, 24, 25, 26, 27, 28, 29]. A comparison of AUCs of various published pre-intervention scores also reveals a ceiling AUC of approximately 0.80 regardless of score complexity or cohort being tested. Of these, only six have been validated in LVOS populations. These include the Pittsburgh Response to Endovascular therapy (PRE) score, Houston Intra-Arterial Therapy (HIAT) score, HIAT-2 score, Stroke Prognostication using Age and NIHSS (SPAN) index, the Totaled Health Risks in Vascular Events (THRIVE) score and the Stanford Age and Diffusion-weighted imaging (SAD) score [14, 23, 24, 25, 30, 31]. The scoring schema and risk strata rele vant from a patient-selection standpoint using these six scores are shown in Table 2.

Table 1.

Comparison of prognostic scores in acute ischemic stroke

Clinical score	Variables, n	All ischemic strokes	IV tPA	Hemorrhage after tPA	Validation in LVOS cohorts	ET patient selection	AUC for good outcome
ASTRAL	6	+					0.7–0.9
iSCORE	11	+	+	+			0.7–0.8
DRAGON	5	+	+				0.8
HIAT	3	+	+	+	+		0.7
SPAN 100	2		+	+	+		0.68–0.73
SAD	2				+		0.69–0.82
SITS	9			+			0.60–0.70
THRIVE	5	+	+	+	+		0.60–075
HIAT-2	4				+	+	0.75
PRE	3				+	+	0.79

Table 2.

Comparison of components and relative weights of factors in pre-intervention prognostic scores

PRE score	Age (years) + 2 × NIHSS - 10 × ASPECTS
PRE −25 to +49	Highly likely to benefit from successful reperfusion
PRE ≥50	Not likely to benefit despite successful reperfusion
SPAN	Age (years) + NIHSS
SPAN≥100	High-risk group, poor outcomes in general
THRIVE	Points (range 0–9)
NIHSS
≤10	0
11–20	2
≥21	4
Age
≤59 years	0
60–79 years	1
≥80 years	2
Comorbidities (HTN, DM, or AFIB)	1 each
THRIVE 6–9	Poor outcome
HIAT	Points (range 0–3)
Age >75 years	1
NIHSS >18	1
Glucose >150 mg/dL	1
HIAT ≥2	Poor outcome after ET
HIAT-2	Points (range 0–8)
Age
≤59 years	0
60–79 years	2
≥80 years	4
Glucose
<150 mg/dL	0
≥150 mg/dL	1
NIHSS
≤10	0
≥10	1
HIAT-2 ≥5	Poor outcome, limited benefit from ET
iSCORE (an online calculator is available at www.sorcan.ca/iscore/)
iSCORE ≥200	Overall very poor prognosis, not validated for ET
SAD score	Points (range 0–4)
DWI volume
≤15 mL	0
>15 mL	1
Age
≤55 years	0
56–69 years	1
70–79 years	2
≥80 years	3

The THRIVE score incorporates age, initial NIHSS, and medical comorbidities [32], and the iSCORE relies on multiple pre-treatment clinical variables [12, 33]. SPAN-100 is an index that summates patient age and NIHSS and is able to predict clinical outcomes following in travenous thrombolysis and ET [25, 34]. The scores discussed thus far do not include pre-treatment imaging parameters such as established core infarct volume or collateral imaging, all of which have been found to be independent predictors of clinical outcome in LVOS patients [21, 35]. Early non-contrast head CT imaging is universally obtained in acute stroke care, and an estimation of established core infarct (hypodensity) can be quantified using the CT ASPECTS [36]. CT ASPECTS was incorporated as a categorical variable in the HIAT-2 score, along with age, NIHSS, and glucose and was found to improve overall prognostic accuracy [14]. Subsequently, the PRE score was developed by incorporating patient age, initial NIHSS, and CT ASPECTS as continuous variables and was found to be superior to THRIVE and iSCORE, and comparable to the HIAT-2 score [23, 31]. Incorporation of pre-intervention core volume imaging measured by diffusion-weighted MRI imaging into a prognostic score has also been performed in LVOS patients (Stanford Age and DWI or SAD score), and further external validation is anticipated [24]. A close inspection reveals a ceiling effect in prognostic accuracies of approximately 0.80 for pre-intervention scores. Online calculators are available for the iSCORE (http://www.sorcan.ca/iscore/), THRIVE (https://www.mdcalc.com/thrive-score-stroke-outcome), and PRE scores (https://www.mdcalc.com/pittsburgh-response-endovascular-therapy-pre-score).

Complexity versus Prognostic Accuracy of Stroke Prognostic Scores

An ideal prognostic pre-intervention score for decision making for ET in LVOS patients should be simple and should include the least number of prognostically important variables while remaining highly predictive of the outcome and should be valid across a diverse range of cohorts. Table 1 shows validated pre-intervention scores for ischemic stroke with their individual characteristics, degree of complexity, and prognostic accuracy [29]. The number of variables in pre-intervention prognostic scores ranges from 2 to 11. As seen in Table 1, the complexity of the prognostic model does not necessarily indicate its predictive accuracy. To exemplify, the highly complex 11-variable iSCORE and a relatively simple 5-variable DRAGON score have very comparable discriminative power (AUC). Similarly, PRE and HIAT-2 scores have similar AUC despite the prior containing 3 variables as compared to 4 in the latter. In such cases where two scores have equal predictive accuracy, one may be tempted to use the simpler one. However, the AUC values must be interpreted in the context of the population being studied. If another variable not included in the score has a strong therapeutic effect (such as ET), the AUC of the score in the treated group is likely to be very different than in the untreated group. In our recent validation study of the PRE score in the TREVO2 trial, we found that the PRE score had a much higher AUC in the Trevo-treated arm as compared to the MERCI-treated arm [37]. The significantly higher rate of mTICI 2B/3 reperfusion in the Trevo arm, as compared to the MERCI arm, may have most likely explained this difference.

Can Pre-Intervention Scores Distinguish between Those Likely to Benefit and Those Who Are Unlikely to Benefit from ET?

While all prognostic scores presented in Tables 1 and 2 can predict long-term clinical outcome in stroke patients, the relevant question regarding pre-treatment scores in LVOS is whether these tools can help the clinician confidently distinguish between patients who are likely to derive clinical benefit from those patients where ET is not likely to improve clinical outcome. Studies involving the PRE and HIAT-2 score have been performed to address this question using institutional registry data [14, 23]. In these analyses, the rates of good clinical outcome in successfully reperfused (mTICI 2B/3) and non-reperfused patients (mTICI 0–2A) were compared within each risk score stratum. PRE scores in the range −24 to 49 with successful reperfusion show improved chances of achieving good outcome at 3 months as compared to those with a PRE score ≥50 [23]. This finding was subsequently replicated in a clinical trial cohort [37]. In the original PRE score development paper, we also identified a group of patients with very low PRE scores (≤–24) who appeared to not benefit from success ful reperfusion. Although this potentially “too good to treat” group is intriguing, the numbers of patients in these groups were very small, necessitating further studies to validate this finding [23]. Even within the PRE score ≥50 subgroup where the benefit of ET seems to be small, there is a possibility that these retrospective studies were underpowered to detect small differences in good outcome. Furthermore, the fact that none of the pre-intervention score thresholds have a 100% specificity for poor outcome, it is emphasized that these scores should only act as adjunctive tools and not replace clinical judgement while excluding patients from potentially beneficial ET. To overcome the inherent limitations of institutional registry data and retrospective analyses, future research must validate established pre-intervention scores and their risk thresholds in more recent randomized stroke clinical trial datasets. This may answer the question whether pre-intervention scores can be used to confidently distinguish patients who can benefit from ET from those who are not likely to benefit from ET for LVOS.

Can Current Pre-Intervention Prognostic Scores Be Improved?

Currently validated pre-intervention scores, despite their complexity, have approached a ceiling in prognostic accuracy. This indicates that any further improvements in prognostic accuracy will require addition of pre-treatment radiographic and clinical variables that have not been considered or have been hard to reliably measure previously. While CT ASPECTS and MRI DWI lesion volume have been evaluated [23, 24], the improvement in prognostic accuracy after addition of perfusion-based core and/or penumbral volume measures or collateral status to existing prognostic models has not been evaluated. Non-invasive vascular imaging with CT or MR angiography and perfusion imaging are increasingly being performed in an efficient manner to confirm the presence of LVOS prior to ET and estimate clinical and radiographic mismatch and collateral status, especially in patients with low NIHSS scores or those with predominantly subcortical symptoms or absence of hyperdense vessel sign on non-contrast CT. CT ASPECTS is a powerful tool to quantify established hypodensity on the initial non-contrast CT scan; however, its interrater reliability is modest and, unlike core volume that is estimated by perfusion imaging, CT-ASPECTS is not volumetric and its correlation with core volume is poor [38]. However, CT-ASPECTS also captures information on lesion topology by assigning equal weights to areas in the middle cerebral artery territory despite different region volumes [39]. Since pre-ET core volume is a critical determinant of final outcome, patients with large core infarcts are frequently excluded from ET. While thresholds such as CT ASPECTS ≤6 or core volume ≥70 mL are suggested as exclusion criteria for ET, many younger patients with ASPECTS 5–7 or core volumes 70–100 mL may still benefit from ET especially if non-eloquent brain tissue is involved [40, 41]. Therefore, a volumetric assessment of core infarct must be considered along with infarct location during the decision making process especially in those patients falling outside existing guidelines. Existing prognostic scores such as the PRE and HIAT-2 scores incorporate CT ASPECTS information, and the SAD score incorporates MRI DWI core volume [24]. Prognostic accuracy of pre-intervention scores could also be potentially improved by incorporation of quantitative perfusion imaging data, collateral status, ASPECTS decay and time elapsed from symptom onset to patient arrival and groin puncture [35, 42].

Pre-stroke functional status is another very important variable that is universally considered in patient selection for ET but is not part of prognostic scores for LVOS patients. Patients with baseline mRS > 2 are frequently excluded from ET, and almost all recent ET trials excluded patients above a baseline mRS of 2 or Barthel Index > 90. Whether ET benefits patients with higher levels of pre-stroke disability, remains unclear. Measures such as pre-stroke mRS or Barthel index or other measures of medical comorbidities (renal function, cancer, cognitive impairment), frailty as well as social/family support, are also likely to impact outcomes following ET but have not been incorporated into pre-intervention LVOS scores. A combination of all these variables along with stroke-specific clinical and radiographic data may improve the prognostic accuracy of pre-intervention scores but may increase the complexity, thereby limiting applicability. With the common use of smartphones and internet resources during acute patient care, smartphone applications can be used to succinctly present comprehensive prognostic information based on all existing pre-intervention scores to the physician. Despite addition of the variables discussed above, it is predicted that only modest improvements in prognostic accuracies will be achieved. This is possibly due to strong prognostic impacts of post-intervention variables such as time to recanalization, follow-up neurological examination (NIHSS at 24 h), final infarct volume, procedural complications, post-stroke complications, and rehabilitation on long-term outcomes [13, 43, 44].

Another current knowledge gap in the field is that current pre-intervention scores for LVOS are applicable only to the anterior circulation. Although the THRIVE and SPAN-100 score variables are not specific to the anterior circulation, the LVOS cohorts used for their validation were limited to the anterior circulation. The PRE and HIAT-2 scores are also specific to the anterior circulation. For posterior circulation stroke, a posterior circulation ASPECTS (pc-ASPECTS) has been developed, although its utility may be mitigated by limitations of CT imaging of the posterior fossa [45, 46]. Incorporation of the pc-ASPECTS performed on CT or pre-ET MRI imaging into pre-intervention prognostic model has not been performed.

Barriers to Implementation of Pre-Intervention Prognostic Scores to ET and Future Directions

Despite the development and validation of numerous pre-intervention stroke scores, none of these have been used in the practice of ET or as enrollment criteria for clinical research studies. For ischemic stroke in general, the iSCORE has been compared to clinical judgement of stroke physicians and showed higher prognostic accuracy and precision over clinical judgement [47]. However, in the hemorrhagic strokes, clinical judgement was found to better correlate with actual 3-month outcomes, as compared to validated prognostic scores for hemorrhagic stroke. Such data in the LVO ischemic stroke population are lacking [48, 49]. With the recent publication of positive ET trials, existing pre-intervention scores should be validated in large datasets to determine appropriate thresholds for optimal patient selection for ET. In order to be clinically useful in decision making, pre-intervention score thresholds must be identified that have close to 100% specificity for poor outcome despite successful endovascular reperfusion. Beyond validation in prospective studies and existing randomized trial datasets, these scores also need to be evaluated for face validity and need to undergo rigorous impact and implementation testing before clinical applicability. Face validity refers to whether the score is intuitive to the user and is a measure of how usable the score is likely to be in clinical practice. If a score lacks face validity despite prognostic accuracy, it is not likely to be ever used. Impact testing determines whether a validated prognostic score is used by physicians, changes therapeutic decisions, improves clinically relevant process parameters, improves patient outcome or reduces costs. Subsequently, the score may undergo implementation testing to determine whether actual dissemination of the prognostic score in daily practice is able to guide physicians with their patient management and clinical decisions. To our current knowledge, none of the pre-intervention ET scores have undergone implementation or impact testing. Since a ceiling prognostic accuracy of 0.8 has been consistently observed across all existing pre-intervention scores, it is likely that large improvements in prognostic accuracy of pre-intervention scores are unlikely even with inclusion of other pre-intervention variables. It is therefore suggested that the stroke community identify potential pre-intervention scoring systems that have the highest face validity, and then direct future implementation and impact studies to determine clinical applicability of pre-intervention LVO scores.

Conclusions

We have reviewed and contrasted existing pre-treatment prognostic scores for LVOS and have highlighted several limitations and knowledge gaps in the field of prognostic scores as decision making tools for patient selection for ET in anterior circulation LVOS. Overall, prognostic accuracies of existing pre-intervention scores that incorporate clinical and radiographic variables (THRIVE, HIAT-2, PRE scores) are moderate and comparable. There is scope for further improvement of prognostic accuracy by incorporating more advanced imaging or core volume and collateral status in addition to other clinical factors. Face validity, impact, and implementation studies of existing scores are lacking. The next steps in the field should focus on improving prognostic accuracy while retaining general applicability and simplicity, and then be followed by prospective impact and implementation testing. Fur thermore, prognostic scores should be used as adjuncts and not replacements of clinical judgement.

Disclosure Statement

The authors have no financial disclosures or conflicts of interest.