Automated Prediction of final infarct volume in patients with large vessel occlusion acute ischemic stroke

Abstract

Introduction:

Determinations of infarct size in patients with large vessel occlusion (LVO) acute ischemic stroke (AIS) play a key role in the identification of candidates for endovascular stroke therapy (EVT). An accurate, automated method to quantify infarct at the time of presentation using widely available imaging modalities would improve screening for EVT. Here, we compare the performance of three measures of infarct core at presentation including an automated method using machine learning (ML).

Methods:

We identified patients with LVO AIS who underwent successful EVT at four comprehensive stroke centers. Patients were included if they underwent concurrent non-contrast head CT (NCHCT), CT angiogram (CTA) and CT Perfusion (CTP) with RAPID (IschemaView, Stanford CA) imaging at the time of presentation and MRI 24–48 hours after reperfusion. NCHCTs were analyzed using ASPECTS graded by Neuroradiology or Neurology expert readers. CTA source images were analyzed using a previously described ML model named DeepSymNet. Final infarct volume (FIV) was determined from diffusion weighted MRI sequences using manual segmentation. The primary outcome was the performance of the three infarct core measurements (NCHCT-ASPECTS, CTA with DeepSymNet, and CTP-RAPID) to predict FIV, which was measured using area under the curve (AUC) ROC analysis.

Results:

Among 76 patients with LVO AIS that underwent EVT and met inclusion criteria, median age was 67 [54–76], 44% were female, and 35% were white. Median NIHSS was 16 [12–22] and median NCHCT ASPECTS on presentation was 8 [IQR 7–8]. Median time between last known well and arrival was 156 minutes [73–303], and between NCHCT/CTA/CTP to groin puncture was 73 minutes [54–81]. The AUC of the ROC was obtained at three different cutoff points: 10, 30 and 50 mL FIV. At 50 ml FIV cutoff, the AUC of the ROC of ASPECTS was 0.74 and of CTP Core Volume was 0.72 and of DSN was 0.82. Differences in AUCs for the three predictors at were not significant for the three FIV cutoffs.

Conclusion:

In a cohort of patients with LVO AIS that achieved reperfusion, determinations of infarct core at presentation by NCHCT-ASPECTS and a ML model analyzing CTA source images was equivalent to CTP in predicting FIV. These findings suggest that the information to accurately predict infarct core in patients with LVO AIS is present in conventional imaging modalities (NCHCT and CTA) and accessible by ML methods.

Introduction

Endovascular stroke therapy (EVT) is a highly effective intervention in patients with large vessel occlusion (LVO) acute ischemic stroke (AIS), and its effectiveness is largely dependent upon the extent of irreversibly injured, or infarcted, tissue at the time of patient presentation.¹ At present, the optimal method to determine infarct core at the time of presentation remains unknown. The Alberta Stroke Program Early CT Score (ASPECTS) has been developed to quantify the extent of ischemia using non-contrast head CT (NCHCT), which is nearly universally available. On the other hand, ASPECTS grading can be subject to high inter-rater variability and can require a higher level of expertise for accurate reading than is available in many lower-volume centers that evaluate AIS.² CT perfusion (CTP), on the other hand, benefits from automated determinations of infarcted tissue and played a large role in recent clinical trials. CTP is less widely available, however, and in addition has been shown to under- and over-estimate infarct core.^3–5 A methodology that can accurately detect infarcted tissue, in an automated fashion, using imaging techniques that are currently in widespread use at all hospitals that evaluate AIS including lower-volume centers, would improve screening for EVT.

To this end, we previously developed a machine learning (ML) model called DeepSymNet that successfully predicted infarct core, as compared to concurrently acquired CTP, using a much more widely available modality, CT angiogram (CTA), in an automated fashion.^6,7 In this study, we hypothesize that NCHCT-ASPECTS and CTA with DeepSymNet perform adequately well in identifying infarct core relative to CTP with RAPID (IschemaView, Stanford CA) in patients with LVO AIS. We compare the performance of these modalities at predicting the final infarct volume (FIV) in patients who successfully underwent EVT.

Methods

Study Population

The data that support the findings of this study are available from the corresponding author on reasonable request. Our study population consisted of a subset of the PRactical IMplementation of MEchanical thrombectomy (PRIME) study. PRIME is a prospective observational cohort study examining all patients diagnosed with Acute Ischemic Stroke (AIS) or transient ischemic attack at 11 Joint Commission certified stroke centers within the same health system across the Greater Houston area. For this study, we identified a subset of this population that presented to four of these hospitals that are Comprehensive Stroke Centers (CSCs), with on-site CTP and EVT capabilities, between March 2016 and April 2019. Patients were included if they were diagnosed with anterior circulation LVO AIS, underwent an imaging workup that included simultaneous NCHCT, CTA, and CTP with RAPID post-processing at the time of presentation. In all patients, CTA was used to assess the presence of LVO. LVO was defined as an occlusion of the intracranial internal carotid artery (ICA), A1 or A2 segments of the anterior cerebral artery, M1 or M2 segments of the middle cerebral artery. Subjects also were required to have undergone EVT with successful reperfusion (TICI 2b/3), and to have had FIV imaging consisting of MRI at 24–48 hours. In all patients, NCHCT, CTA and CTP were acquired concurrently, which was at the time of presentation to the hospital.

The study was reviewed and approved by the Committee for the Protection of Human Subjects at the University of Texas Health Science Center at Houston and waiver of consent and HIPAA authorization was granted (IRB ID: HSC-MS-18-0796).

Measurements

All study data were collected and managed using REDCap electronic data capture tools.^8,9 Quantification of ischemic core at the time of presentation was made by three methods. First, ASPECTS was determined from NHCHT using neuroradiology report, or if not available, by expert reviewer with experience in AIS imaging determinations (SAS). Second, CTP’s were post-processed using RAPID. Finally, CTA source images were analyzed using DeepSymNet-v2 (see below). We defined FIV on diffusion weighted imaging (DWI) MRI performed within 72 hours after successful EVT, consistent with prior studies.^3,10–12 FIV was measured utilizing manual region of interest measurements (Horos, Annapolis, MD).

DeepSymNet-v2

DeepSymNet-v2 is a ML algorithm based on a 3-dimensional convolutional neural network that utilizes brain hemisphere symmetry information in order to learn imaging patterns that are predictive of outcomes related to stroke. utilizes. In our previous works, we described the first iteration of the algorithm, DeepSymNet, which was used to detect LVO, estimate infarct core and detect hemorrhagic stroke.^6,7,13 In this study, we utilized a new version of the ML model which has been improved by including CTA-specific template, multi-resolution pipeline and skull stripping. The model was further improved by using symmetric and non-symmetric paths, reducing the number of parameters by using more efficient building blocks, skip connections and batch normalization.^14,15

The algorithm for this ML model was trained and internally validated on an external CTA dataset acquired from patients outside this analysis cohort, allowing for a meaningful estimation of the algorithm generalizability and to fine tune its architecture. At training time, the algorithm used as target a binary variable indicating a large (>50ml) or small (<50ml) infarct core, which was computed from the CTP image and the RAPID analysis software. At inference time, i.e. when the algorithm is evaluated in this analysis, the algorithm generates a real number ranging from 0 to 1 in indicating the non-calibrated likelihood of having a small infarct core, with values closer to 1 more indicative of a smaller core.

Primary Outcome

The primary outcome of this study was the performance of the three infarct core measures – DeepSymNet-v2 model (using CTA), NCHCT-ASPECTS, CTP-RAPID – at predicting FIV in this cohort of patients with successful EVT. This outcome was measured using area under the ROC curve statistic at three FIV cutoffs (10 mL, 30 mL and 50 mL). These cutoff values were chosen to show the performance of the algorithm at different core sizes. They were based on clinically relevant values, while also ensuring we had sufficient data with our distribution of core sizes to perform the analysis. The 10 mL cutoff was chosen as a lower value. The 30mL and 50mL cutoffs were chosen to be consistent with the core thresholds used in the DAWN trial.¹⁶

Statistical Analysis

For the description of the characteristics of the study population, percentages are reported for categorical variables, and medians [IQR] for continuous variables. All statistical tests were 2-sided and conventional levels of significance (α = 0.05) were used for interpretation. Model performance was measured using receiver-operating characteristic area under the curve (ROC AUC) statistics. Chi-square tests were performed to test for significance between AUCs for the predictors. All data analyses were performed using Stata 16 (StataCorp LLC, College Station, TX) and Prism 7 (GraphPad, La Jolla, CA) software. Data are presented at median [IQR].

Results

Among 76 patients with LVO AIS that underwent EVT and met inclusion criteria, median age was 67 [54–76], 44% were female, and 35% were white. Fifty percent had a history of hypertension, 21% had hyperlipidemia and 11% had prior stroke. Median NIHSS was 16 [12–22] and 65% had a pre-stroke modified Rankin score of 0. Median ASPECTS on presentation was 8 [IQR 7–8]. Median time between last known well and arrival was 156 minutes [73–303], and between NCHCT/CTA/CTP to groin puncture was 73 minutes [54–81]. 51% of the patients received IV-tPA, and 82% achieved TICI2b. More detail on demographics and clinical characteristics can be found in Table 1.

Table 1:

Demographic and Clinical Characteristics

	All (n=76)
Age, median(IQR)
Female, %	44.7
Race, %
White	36.8
African American	18.42
Asian	5.26
Other	39.47
Ethnicity, %
Hispanic	14.4
PMHx, %
Prior stroke	11.8
Hyperlipidemia	21
Hypertension	50
Diabetes	22.3
Smoking	10.5
Pre-stroke modified Rankin Scale, categorical, %
0–2	82.9
3–5	17.1
NIHSS, median(IQR)	16(12–22)
Baseline ASPECTS, median (IQR)	8(7–8)
RAPID ischemic core volume, median(IQR)	4.5(0–21.5)
FIV, median (IQR)	11.39(2–37.5)
Time Intervals, median (IQR)
LKW to Arrival	156(73–303)
CT to GP	73(54–81)
CT to Recanalization	111(100–126.5)
Target Occlusion Site, %
ICA	13.3
M1	66.6
M2	16
Other	4
Occlusion laterality left hemisphere, %	55.26
IV tPA, %	51
Final TICI score, %
2b	25.3
3	74.6

The distribution of infarct core predictions based on NCHCT-ASPECTS, CTP-RAPID and DeepSymNet-v2 relative to FIV can be seen in Figure 1. The performance of these three measures at predicting FIV can be seen in Figure 2. The AUC of the ROC was obtained at three different cutoff points: 10, 30 and 50 mL FIV. At 10 ml FIV cutoff, the AUC of the ROC of ASPECTS was 0.67 and of CTP Core Volume was 0.63 and of DeepSymNet-v2 (DSN) was 0.75. Differences in AUCs for the three predictors at were not significant (p=0.11). At 30 ml FIV cutoff, the AUC of the ROC of ASPECTS was 0.76 and of CTP Core Volume was 0.73 and of DSN was 0.82. Differences in AUCs for the three predictors at were not significant (p=0.36). At 50 ml FIV cutoff, the AUC of the ROC of ASPECTS was 0.74 and of CTP Core Volume was 0.72 and of DSN was 0.82. Differences in AUCs for the three predictors at were not significant (p=0.44).

Figure 1. Distribution of FIV against three measurements of infarct core at presentation.

Scatter plots showing the distribution of FIV vs. (a) ASPECTS (b) CTP-RAPID Infarct core and (c) DeepSymNet-v2 probability on imaging obtained at the time of patient presentation.

Figure 2. Performance of three infarct core measurement techniques at presentation to predict FIV.

Receiver-operating curves at three FIV cutoffs of (a) 10 mL, (b) 30 mL and (c) 50 mL demonstrating the performance of three measures of infarct core at predicting FIV using imaging obtained at the time of presentation prior to EVT.

Discussion

In this study of different methods to measure infarct core at the time of presentation in patients with LVO AIS, all three measurements showed good performance at predicting FIV in patients with recanalization. Our automated ML model analyzing CTA source images showed superior performance relative to NCHCT-ASPECTS and CTP-RAPID across the range of FIV sizes, though this difference was not statistically significant. The performances of all three models improved with identification of larger FIVs.

We chose to use FIV in patients with successful recanalization as the gold standard for infarct at presentation because the known inaccuracies of other methods particularly in the earlier time windows.⁴ There are however some limitations to this approach as infarct volumes may grow from the time of imaging to reperfusion, and possibly even afterwards. On the other hand, this technique is consistent with prior studies evaluating the performance of ML methods for infarct core detection at the time of presentation.^10,17,18 Further, testing imaging methods using FIV in reperfused patients may have another advantage, as it measures the ability of the imaging methods to answer a clinically relevant question: if this patient is able to achieve rapid reperfusion, what will be the expected outcome?

Here, we found that NCCT-ASPECTS and DeepSymNet-v2 using CTA images performed equivalently to CTP-RAPID, across a range of infarct sizes. These two imaging modalities, NCHCT and CTA, are far more widely available than CTP. Of these two methods, DeepSymNet-v2 boasts an advantage of being automated, requiring no expert input. In addition, the algorithm can be run on CTA source images in less than 1 minute. Given the known wide inter-rater reliability of NCHCT-ASPECTS, an automated approach to identify infarct is preferable, although recent advances in automated ASPECTS are improving in performance.¹⁰ In this cohort, the addition of CTP to NCHCT and CTA did not add prognostic value to the imaging outcome.

Our study has several limitations. As mentioned above, it is possible that the infarct volume could grow even after the < 72 hour MRI time point used here to define FIV. On the other hand, this time point has been used in many prior studies, and shown to correlate closely with 90-day clinical outcome.¹⁹ Further, because we limited our cohort to patients that underwent EVT, the proportion of patients with very large infarcts was relatively small. While our findings would certainly benefit from validation in another dataset, we did however observe improved performance of ASPECTS and DeepSymNet-v2 with larger FIVs. Finally, our cohort contained primarily patients evaluated within the first 6 hours after last known well, and it is possible that these findings would not be replicated in later time window patients. Prior studies on the other hand have shown improved performance of NCHCT-ASPECTS with increasing time after last known well.²⁰

In a cohort of patients with LVO AIS that achieved reperfusion, determinations of infarct core at presentation by NCHCT-ASPECTS and a ML model analyzing CTA source images was equivalent to CTP in predicting FIV. The ML model demonstrated superior performance, but this difference was not statistically significant. These findings suggest that the information to accurately predict infarct core in patients with LVO AIS is present in conventional imaging modalities (NCHCT and CTA) and accessible by ML methods.