Prediction of delayed reperfusion in patients with incomplete reperfusion following thrombectomy

Abstract

Background:

The clinical course of patients with incomplete reperfusion after thrombectomy, defined as an expanded Thrombolysis in Cerebral Infarction (eTICI) score of 2a–2c, is heterogeneous. Patients showing delayed reperfusion (DR) have good clinical outcomes, almost comparable to patients with ad-hoc TICI3 reperfusion. We aimed to develop and internally validate a model that predicts DR occurrence in order to inform physicians about the likelihood of a benign natural disease progression.

Patients and methods:

Single-center registry analysis including all consecutive, study-eligible patients admitted between 02/2015 and 12/2021. Preliminary variable selection for the prediction of DR was performed using bootstrapped stepwise backward logistic regression. Interval validation was performed with bootstrapping and the final model was developed using a random forests classification algorithm. Model performance metrics are reported with discrimination, calibration, and clinical decision curves. Primary outcome was concordance statistics as a measure of goodness of fit for the occurrence of DR.

Results:

A total of 477 patients (48.8% female, mean age 74 years) were included, of whom 279 (58.5%) showed DR on 24 follow-up. The model’s discriminative ability for predicting DR was adequate (C-statistics 0.79 [95% CI: 0.72–0.85]). Variables with strongest association with DR were: atrial fibrillation (aOR 2.06 [95% CI: 1.23–3.49]), Intervention-To-Follow-Up time (aOR 1.06 [95% CI: 1.03–1.10]), eTICI score (aOR 3.49 [95% CI: 2.64–4.73]), and collateral status (aOR 1.33 [95% CI: 1.06–1.68]). At a risk threshold of R = 30%, use of the prediction model could potentially reduce the number of additional attempts in one out of four patients who will have spontaneous DR, without missing any patients who do not show spontaneous DR on follow-up.

Conclusions:

The model presented here shows fair predictive accuracy for estimating chances of DR after incomplete thrombectomy. This may inform treating physicians on the chances of a favorable natural disease progression if no further reperfusion attempts are made.

Introduction

The European Stoke Society (ESO), European Society of Minimally Invasive Neurological Therapy (ESMINT), and American Heart Association (AHA) guidelines recommend that interventionalists should attempt a Thrombolysis in Cerebral Infarction (TICI) grade 3 angiographic reperfusion, if achievable with reasonable safety.^1,2 Although in many cases complete angiographic reperfusion can be achieved after the first pass, rescue maneuvers may be performed to achieve complete reperfusion when distal emboli remain.^3–5

However, it is expected that not all patients with incomplete angiographic reperfusion, defined as expanded TICI (eTICI) score 2a–2c, may benefit from adjunctive reperfusion efforts. ⁵ Outcomes of patients with incomplete reperfusion are heterogeneous and dependent on different clinical and baseline characteristics.^6,7 Specifically, more than 50% of patients with incomplete reperfusion show spontaneous delayed reperfusion (DR) at 24 h, which is associated with a favorable clinical course almost identical to TICI3 patients and minimal infarct growth. ⁸ While even in these patients there may be a benefit of additional reperfusion attempts, the benefit is certainly smaller than in patients with a persistent perfusion deficit (PPD) on follow-up imaging. For the complex decisions regarding proceeding with additional reperfusion attempts or stopping the procedure, it may be helpful to know if the patient is more likely to have a favorable (i.e. high chance of DR) or unfavorable (high chance of PPD) natural disease progression. ⁵

The primary aim of this study was to develop and internally validate a model that predicts DR after incomplete mechanical thrombectomy (MT). The outcome of the model might be taken into consideration when deciding on whether to pursue additional reperfusion attempts for small remaining vessel occlusions or stopping the procedure.

Patients and methods

Study design

We performed a retrospective observational analysis of a single-center registry for all consecutive acute ischemic stroke (AIS) patients admitted between 02/2015 and 12/2021 who had undergone MT. Written informed consent was obtained from the patients. This study received ethics committee approval (Kantonale Ethikkommission Bern, reference ID 2019-00547) and was performed in accordance with the Declaration of Helsinki and its later amendments. Reporting has been performed according to the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement. Study data are available from the corresponding author upon reasonable request following receipt of a research plan and clearance by the ethics committee.

Perfusion imaging

All patients underwent MRI or CT follow-up imaging 24 ± 12 h after the intervention as part of standard care in our institution. All perfusion imaging was performed as part of clinical routine. Olea sphere software (Olea Sphere v2.3; Olea Medical, La Ciotat, France) was used to generate perfusion images from MRI and syngo.via (Siemens) from CT scans. Perfusion imaging outcome was dichotomized into DR and PPD. DR was defined as resolution of a focal angiographic perfusion deficit from the final thrombectomy angiography with normalization of tissue perfusion at the corresponding neuroanatomical territory on the 24 ±12 h follow-up perfusion imaging maps (Figure 1(a)). While PPD was observed in case of a perfusion imaging abnormality, which corresponded to the antegrade capillary phase deficit from the final thrombectomy angiography imaging (Figure 1(b)). Patients who presented with reocclusions on the follow-up imaging (n = 17) were classified as having a PPD because it cannot be stated with certainty whether these patients will develop a reocclusion after the procedure or not. Details on grading and evaluation of perfusion imaging outcome have been described previously. ⁸ Only large-vessel occlusion anterior circulation AIS patients who had final eTICI grade 2a–2c and available 24 ± 12-h follow-up perfusion imaging were included in the final cohort (Supplemental Figure I).

Figure 1.

Delayed reperfusion and persistent perfusion deficit.

Details on grading and evaluation of perfusion imaging outcome have been described previously. ⁸ In short, TTP and Tmax perfusion maps were evaluated as they had the highest sensitivity. Final angiography runs are displayed with high contrast to emphasize the capillary phase deficits. (a) Patient with a right-sided M1 occlusion as noted on the first angiography run (top left) and admission perfusion imaging (top right). After the intervention, the patient was graded with incomplete reperfusion score of eTICI 2b67 due to the deficit on the final angiography run (bottom left). On the 24-h follow-up perfusion imaging it is evident that this patient has experienced delayed reperfusion (bottom right). (b) Patient with a right-sided M1 occlusion as noted on the first angiography run (top left) and admission perfusion imaging (top right). After the intervention, the patient was graded with an incomplete reperfusion score of eTICI 2b50 due to the deficit on the final angiography run (bottom left). On the 24-h follow-up perfusion imaging, it is evident that this patient still has a persistent perfusion deficit (bottom right).

Model development and specifications

We have developed a prediction model “PROCEED” (PeRfusion OutComE prEDiction), which can be used to estimate the likelihood of DR occurring after incomplete thrombectomy has been established on the final angiography run. The aim of PROCEED is to assist in identification and selection of patients with a high likelihood of DR, and who therefore may be less likely to benefit from any adjunctive reperfusion efforts when incomplete angiographic reperfusion is established on the final angiography run (Supplemental Figure II).

The model was corrected for under- and overfitting, avoiding systematic under- or overestimation with steps described below in this paragraph. Missing values were replaced with median and mode values for continuous and categorical variables, respectively. Considering low number of missing values (<5%) no multiple imputations were used. Patients’ baseline and interventional characteristics that could be obtained during routine stroke admission and work-up were included in the initial analyses (Supplemental Table I). In order to reduce the number of variables and find the ones which will aid in model’s ability to discriminate, we have performed variable pre-selection via bootstrapped (n = 1000) backward stepwise logistic regression based on the Akaike information criteria (AIC) score. AIC-based selection was chosen because it measures the quality of the entire model, unlike the p-value, which only provides information on specific variables within the model. Following this statistical pre-selection we have executed final variable selection with interval validation via bootstrapping method and have used the output of interval validation to choose the most optimal classification algorithm. Lastly, random forest algorithm was selected as the one with the highest accuracy for final model development. Bootstrapped resampling of the final model was performed to obtain the 95% confidence interval (CI) for all model performance metrics (Supplemental Figure I). The final version of PROCEED has been uploaded in a digital repository as an interactive open access tool: https://proceed.shinyapps.io/model/. Time points shown in an online tool have been calculated with the Cox regression that analyzed time from the end of the intervention to follow-up perfusion imaging within the model.

Model variables

Variable pre-selection was based on evidence in current literature and AIC score of the logistic regression output. Variables included in the final model are: age, sex, atrial fibrillation, anticoagulants and antiplatelets prestroke, National Institutes of Health Stroke Scale (NIHSS) score on admission, Onset-to-Door time, Intervention-To-Follow-Up time, intravenous thrombolysis, number of maneuver counts, eTICI, and collateral score. Stroke severity was evaluated with the NIHSS score upon admission. Sites of arterial occlusion were evaluated on initial imaging: intracranial carotid artery (ICA), proximal segment of the middle cerebral artery (M1), Sylvian segment of the middle cerebral artery (M2), cortical segment of the middle cerebral artery (M3), pre-communicating and post-communicating segment of the anterior cerebral artery (A1–2). The eTICI scale was used to grade reperfusion success on the final angiography series as follows: 1%–49% reperfusion of the affected territory was graded as eTICI 2a, 50%–66% as eTICI 2b50; 67%–89% as eTICI 2b67; and 90%–99% as eTICI 2c. The American Society of Intervention and Therapeutic Neuroradiology and Society of Interventional Radiology (ASITN/SIR) Collateral Flow Grading System was used for collateral grading on pretreatment angiography. Grades range from 0 (no visible collateral) to 4 (complete and rapid collateral blood flow in the entire ischemic territory), as described previously. ⁹ Onset-to-Door time refers to the time from symptom onset until the admission of the patient to the emergency department of the treating hospital. Intervention-to-Follow-Up time is the captured time between the last angiography series run and the time of the first follow-up imaging.

Statistical analysis

Fischer’s exact and Chi-squared tests were used for categorical variables, and Mann-Whitney-U and Welsch’s T-test were used for continuous variables. Results are reported as “median [interquartile range (IQR)]” or “n (%).” Statistical handling of variables is reported in Supplemental Table I. The model’s discrimination is reported with C-statistics (Harrell’s concordance), which varies from 0.5 to 1, where 0.5–0.7 indicates good, 0.7–0.8 strong, and 1 perfect discrimination. It estimates the likelihood that a randomly selected patient, who has a higher predicted probability of achieving DR, will actually have DR. For binary outcomes, C-statistics is similar to the area under the receiver operating characteristics curve (AUC). Discrimination and calibration were quantified together with a Brier score, which evaluates the goodness-of-fit for a predicted probability. Brier scores ranges from 0 for total accuracy to 1 for complete inaccuracy. This is presented graphically with a calibration plot for achieving DR, together with calibration intercept and slope. The intercept serves as a measurement of predicted probabilities indicating wherever they are too low or too high, while the slope suggests the predictor’s strength in the cohort. ¹⁰ Ideally, the intercept should be equal to 0, and slope to 1. ¹⁰ Importance of variables included in the model is presented with the Mean Decrease Accuracy (MDA) index, and MDA index points were plotted in descending order of importance. MDA expresses how much accuracy the model losses by excluding a certain variable. Variables with higher MDA index are more important for successful performance of the model. Summary of model’s performance is presented with a confusion matrix in a contingency table. Most often used metrics for presenting the results of the confusion matrix are precision and an F1 score. Precision of the matrix accounts for all the positive cases and provides a rate on how many of them were predicted correctly, while the F1 score harmonizes precision and sensitivity, making them comparable. All reported model metrics are presented with a mean and 95% CI from bootstrapped model replication. All statistical analyses were performed in R v4.0.0 with the packages outlined in the Supplemental Table II.

Clinical decision curve

Decision curve analyses are used to add information on the clinical utility of a certain prediction model. ¹¹ Two important metrics in clinical decision curves are net benefit and threshold probability. ¹¹ Net benefit is a weighted difference of true and false positives: it increases with true positive and decreases with false positive cases. Threshold probabilities decide how important doing an additional attempt or maneuver is in a patient that would have DR (false positives) compared to not doing an additional attempt or maneuver in patients that would develop DR in any case (true positives). Higher threshold probabilities give more weight to false rather than true positives. Conversely, higher net benefits give more weights to true rather than false positives. Depending on the weights, context and scenario in which the model is being used, model might favor high or low threshold probabilities. ¹¹ In the present framework, model should have low threshold probabilities and with high net benefit. We predefined a threshold probability range between 20% and 40%, which corresponds to the odds range of 1:4–2:3. This means that pursuing additional attempts in a patient who would be likely to have DR would be 4–1.5 times worse than not pursuing additional attempts. The lower and upper ends of the threshold probability range are based on typical patients who would unquestionably be considered for adjuvant attempts after incomplete reperfusion, considering potential risks and benefits. When the golden reference standard does not exist, threshold probability range is based on the “Treat All” scenario. In our case, lower end of the threshold probability range was set at 20%, as the net benefit between “Treat All” option and the prediction model is almost identical below this point. Upper end was set at 40%, as the “Treat All” option shows zero benefit after this cutoff, meaning no further consideration would be reasonable. Decision curves also provide information on net reduction in additional attempts or maneuvers, where differences between true and false negatives are weighted: net reduction becomes higher the more true negatives there are. ¹¹ Net reduction is usually reported as a rate per 100 patients. Reporting net reduction is especially valuable if the current gold standard for a certain condition is to treat all patients. ¹¹

Results

A total of 477 patients were included in the analysis. Median age of the final cohort was 74 years (IQR 63 - 81) and 48.8% were female, out of which 279 (58.5%) had DR, with a median Intervention-to-Follow-Up time of 20 h and 37 min (IQR 16 h 42 min–23 h 51 min). Patients with DR were more likely to have atrial fibrillation (DR vs PPD: 36.2% vs 25.8%; p = 0.02), longer Intervention-to-Follow-Up time (DR vs PPD: 21 h 40 min vs 19 h 16 min; p < 0.001), lower number of maneuver counts (DR vs PPD: 1 [1, 2] vs 2 [1, 3]; p < 0.001), higher likelihood of functional outcome (3-month mRS 0–2 DR vs PPD: 65.2% vs 40.6%; p < 0.001), better final reperfusion eTICI score (eTICI 2c DR vs PPD: 54.1% vs 17.2%; p < 0.001), and better collateral status (DR vs PPD: 2 [1, 3] vs 2 [1, 2]; p < 0.001), as seen in Table 1. There was a strong correlation between achieved degree of reperfusion and number of attempts (p < 0.001), that is, patients in the higher spectrum of the eTICI scale tended to have lower number of maneuver counts (Supplemental Figure III).

Table 1.

Baseline characteristics of included patients.

	Overall	Delayed reperfusion	Persistent perfusion deficit	p
Number of patients	477	279	198
Age (years) (median [IQR])	74 [63, 81]	74 [64, 80]	75 [61, 82]	0.389
Sex = Female (%)	233 (48.8)	133 (47.7)	100 (50.5)	0.605
Atrial fibrillation = Yes (%)	152 (31.9)	101 (36.2)	51 (25.8)	0.021
Coronary heart disease = Yes (%)	81 (17.0)	56 (20.1)	25 (12.6)	0.044
Diabetes = Yes (%)	81 (17.0)	43 (15.4)	38 (19.2)	0.337
Hyperlipidemia = Yes (%)	299 (62.7)	179 (64.2)	120 (60.6)	0.488
Hypertension = Yes (%)	330 (69.2)	198 (71.0)	132 (66.7)	0.367
Smoking status = Yes (%)	101 (21.2)	54 (19.4)	47 (23.7)	0.298
Previous stroke = Yes (%)	59 (12.4)	30 (10.8)	29 (14.6)	0.258
Previous TIA = Yes (%)	26 ( 5.5)	15 ( 5.4)	11 (5.6)	1
Systolic blood pressure on admission (mmHg) (median [IQR])	151 [133, 171]	150 [132, 169]	152 [135, 172]	0.325
Diastolic blood pressure on admission (mmHg) (median [IQR])	80 [70, 93]	80 [69, 93]	80 [71, 93]	0.882
Creatinine on admission (μmol/L) (median [IQR])	78 [65, 92]	78 [64, 92]	78 [66, 93]	0.571
Glucose on admission (mmol/L) (median [IQR])	6.5 [5.8, 7.9]	6.4 [5.8, 7.8]	6.7 [5.8, 7.9]	0.399
Anticoagulants pre-stroke = Yes (%)	61 (12.8)	30 (10.8)	31 (15.7)	0.15
Antiplatelets pre-stroke = Yes (%)	130 (27.3)	80 (28.7)	50 (25.3)	0.47
NIHSS on admission (median [IQR])	13 [7, 18]	13 [7, 18]	12 [6.25, 18]	0.684
Known time of symptom onset (%)				0.916
No	100 (21.0)	58 (20.8)	42 (21.2)
Wake up	91 (19.1)	55 (19.7)	36 (18.2)
Yes	286 (60.0)	166 (59.5)	120 (60.6)
Onset-To-Door (h) (median [IQR])*	3.08 [1.70, 7.10]	3.20 [1.68, 7.18]	2.98 [1.72, 6.63]	0.971
Intervention-to-Follow-up (h) (median [IQR])	20.62 [16.70, 23.85]	21.67 [18.09, 24.99]	19.28 [14.51, 22.89]	<0.001
Intravenous thrombolysis = Yes (%)	177 (37.1)	104 (37.3)	73 (36.9)	1
Number of device passes (median [IQR])**	2 [1, 3]	1 [1, 2]	2 [1, 3]	<0.001
Fazekas score (%)***				0.097
0	142 (30.1)	84 (30.1)	58 (30.1)
1	200 (42.4)	123 (44.1)	77 (39.9)
2	90 (19.1)	44 (15.8)	46 (23.8)
3	40 ( 8.5)	28 (10.0)	12 ( 6.2)
Occlusion sites (%)				0.115
ICA	96 (20.1)	58 (20.8)	38 (19.2)
M1	244 (51.2)	151 (54.1)	93 (47.0)
M2	125 (26.2)	63 (22.6)	62 (31.3)
M3	8 ( 1.7)	6 ( 2.2)	2 ( 1.0)
A1–2	4 ( 0.8)	1 ( 0.4)	3 ( 1.5)
mRS score post-stroke (median [IQR])	2 [1, 4]	2 [1, 4]	3 [2, 5]	<0.001
mRS score 0–2 post-stroke (%)****	273 (55.6)	182 (65.2)	91 (40.6)	<0.001
eTICI (%)				<0.001
2a	35 ( 7.3)	4 ( 1.4)	31 (15.7)
2b50	70 (14.7)	15 ( 5.4)	55 (27.8)
2b67	187 (39.2)	109 (39.1)	78 (39.4)
2c	185 (38.8)	151 (54.1)	34 (17.2)
ASITN/SIR collateral score (median [IQR])*****	2 [1, 3]	2 [1, 3]	2 [1, 2]	<0.001

TIA: transient ischemic attack; mRS: modified Rankin Score; eTICI: expanded Treatment in Cerebral Infarction; ASITN/SIR: American Society of Intervention and Therapeutic Neuroradiology and Society of Interventional Radiology.

^*Data missing for 19 patients.

^**Data missing for 10 patients.

^***Data missing for 5 patients with persistent perfusion deficit.

^****Data missing for 18 patients with persistent perfusion deficit.

^*****Data missing for 16 patients.

The following variables were included in the final prediction model: age, sex, atrial fibrillation, Intervention-to-Follow-Up time, maneuver count, eTICI, and collateral status. In the final set, the following predictor effects were found to be strongly associated with DR: atrial fibrillation (aOR 2.06 [95% CI: 1.23–3.49]), Intervention-To-Follow-Up time (aOR 1.06 [95% CI: 1.03–1.10] per hour increase), final eTICI score (aOR 3.49 [95% CI: 2.64–4.73]), and collateral status (aOR 1.33 [95% CI: 1.06–1.68], Supplemental Table III and Figure IV). Of the variables included in the final model, the following had the highest MDA index: eTICI (43.1%, 95% CI: 39.7–45.8), collateral status (15.3%, 95% CI: 13.1–16.7), atrial fibrillation (10.7%, 95% CI: 8.9–12.7), and maneuver count (9.7%, 95% CI: 8.2–10.9), as shown in Figure 2.

Figure 2.

Mean decrease in accuracy of all model variables.

Importance of variables included in the model is presented with the Mean Decrease Accuracy (MDA) index. Variables with higher MDA index are more important for successful performance of the model. MDA index was highest for the expanded treatment in cerebral infarction (eTICI) score, collateral status graded with the American Society of Interventional and Therapeutic Neuroradiology/Society of Interventional Radiology (ASITN/SIR) score, atrial fibrillation, and number of maneuver counts. Variables which provided highest impact of model’s accuracy were mostly all related to the interventional, rather than baseline patient characteristics.

Discriminative ability of the model was good, with internally validated C-statistics of 0.79 (95% CI: 0.71–0.84). Calibration was within a corresponding range of 0.18 (95% CI: 0.17–0.19), with intercept and slope having values of −0.19 and 0.97, respectively (Figure 3). In general, the model tended to overestimate rates of PPD (true vs predicted rates of PPD: 42% vs 67%) with a precision of 0.73 (95% CI: 0.69–0.76%) and an F1-score of 0.79 (95% CI: 0.75–0.82). A complete overview of the model’s performance is shown in Supplemental Figure V.

Figure 3.

Discrimination and calibration of the model’s prediction for perfusion imaging outcome.

Calibration plot for predicting perfusion imaging outcome. The intercept serves as a measurement of predicted probabilities indicating wherever they are too low or too high, whereas the slope represents the predictor’s strength in the cohort. Ideally, the intercept should be equal to 0, and the slope to 1. Discrimination between low and high likelihood of achieving delayed reperfusion was good with intercept −0.19 (95% CI: −0.30 to −0.06) and slope 0.97 (95% CI: 0.81–1.11).

The prediction model outperformed both decision scenarios (i.e., “Treat All” and “Treat None”) by a wide margin. The model has the highest net benefit across a wide range of threshold probabilities, except if the risk threshold of having a PPD is around 10%, in which case use of the model should be avoided. Using the prediction model in 0%–10% threshold probability range would have no added value, as all patients within this threshold range should be treated. If we interpret this through odds, risk threshold of 10% would represent odds of 1:9. Meaning, at the risk threshold of 10% missing a high likelihood PPD is nine times worse than doing an additional attempt. For the risk threshold R = 20% the prediction model has a 70% likelihood of correctly identifying a patient who will develop DR at 24 h (Figure 4(a)). Thus, with that same threshold, pursuing additional attempts in a patient with high-likelihood DR is four times worse than not doing anything. At a risk threshold of R = 30%, use of the prediction model could potentially reduce the number of additional attempts in one out of four patients who will have spontaneous DR, without missing any patients who do not show spontaneous DR on follow-up (Figure 4(b)). Similar results were generated even when excluding eTICI 2c patients (Supplemental Figure VI).

Figure 4.

Clinical decision curves.

Standardized net benefit is a weighted difference of true and false positives: it increases with more true positives and decreases with more false positives. Threshold probabilities decide how important doing an additional attempt or maneuver in a patient that would have DR is (false positives) compared to not doing an additional attempt or maneuver in patients that would develop DR in any case (true positives). The net reduction in unnecessary interventions weighs the differences between true and false negatives: the net reduction becomes higher the more true negatives there are. It is usually reported as a rate per 100 patients. The prediction model outperforms both the “Treat All” and “Treat None” scenarios by a wide margin. (a) Net benefit of the prediction model and “Treat All” option overlap in the threshold probability range from 0% to 10%. Using the prediction model in this threshold probability range would have no added value, as all patients within this threshold range should be treated. With a risk threshold of R = 20%, pursuing additional attempts or maneuvers in a patient with high-likelihood DR (false positive) is 4x worse (Cost:Benefit Ratio 1:4) than not doing anything, as the patient would be likely to develop DR on follow-up anyway (true positive). (b) At a risk threshold of R = 30%, use of a prediction model would reduce the number of unnecessary interventions in one out of four patients (standardized net reduction = 0.25), without missing an intervention for any patient who would eventually have DR.

For demonstration purposes, we calculated predicted probabilities of achieving DR for two hypothetical patients. The first is a 65-year-old male who does not have atrial fibrillation. The patient has an ASITN/SIR collateral score of 3, and after two maneuvers he achieved a final reperfusion score of eTICI 2b67. On 24-h follow-up imaging, this patient has an 82.4% (95% CI: 79.0–84.8) chance of achieving DR. The second hypothetical patient is a 65-year-old female, again with no atrial fibrillation. She also has an ASITN/SIR collateral score of 3, and after two maneuvers she achieved a final reperfusion score of eTICI 2b50. On the 24-h follow-up imaging, this patient has a 39% (95% CI: 35.6–41.4) chance of achieving DR. We can see that, even though these two hypothetical patients have fairly similar clinical presentation and interventional characteristics, Patient 1 is more than twice as likely to achieve DR than Patient 2. Our model is available as an online tool at: https://proceed.shinyapps.io/model/.

Discussion

The main findings of this study are as follows: (1) The internally validated model has fair predictive accuracy for determining perfusion imaging outcome and may inform treating physicians on the chances of favorable natural disease progression if no further reperfusion attempts are made. (2) The variables with highest predictive value for achieving DR were: age, sex, atrial fibrillation, Intervention-to-Follow-Up time, maneuver count, eTICI score, and collateral status. (3) Intervention-related variables contributed more toward accurate prediction of DR.

Potential model implications

Patients with incomplete reperfusion (e.g. eTICI 2b50) are expected to have small remaining perfusion deficits caused by the occlusion of medium and small distal vessels. The decision on proceeding or stopping with an intervention in these patients is surrounded by uncertainties and is dependent on many factors.^3–5 Potential adjunctive reperfusion efforts include secondary distal MT, administration of intra-arterial lytic, or additional antithrombotics.^12–17

Data on distal stent retrievers or aspiration thrombectomy are heterogeneous with limited generalizability due to selection bias.^3–5,18 Some observational studies report better reperfusion rates after performing additional maneuvers.^3,4 However, complications associated with mechanical maneuvers seem to increase the more distal the occlusion site is.^12,13,18 Other adjunctive reperfusion efforts would include the administration of intra-arterial lytics or antithrombotics.^14–17 Although intra-arterial lytics are thought to be less invasive than mechanical therapies, they may increase the risk of bleeding.^19,20

In summary, several propositions have been made for different adjunctive reperfusion strategies, but none of them are devoid of risk.^12–17 Taking these risks may be unwarranted in patients who are likely to develop DR, which is associated with lack of infract growth and a comparable outcome to patients with primary eTICI3 reperfusion. ⁸ Therefore, a better prediction of what patients show a naturally benign disease progression may be of value. While the present model offers fair prediction of perfusion imaging outcomes, it is not clear if there may still be a benefit of immediate complete reperfusion even in patients which show DR.

Predictor values

PROCEED is based on seven baseline and interventional characteristics: age, sex, atrial fibrillation, Intervention-to-Follow-Up time, maneuver count, eTICI, and collateral status.

Association between age, sex, and perfusion outcome has already been established. Younger patients and males tend to have significantly better perfusion outcome with lower rates of incomplete reperfusion. ²¹ Our findings are also supported by previous descriptions of reperfusion status serving as a function of time, showing gradual increase in complete reperfusion rates as time passes.^8,22 Atrial fibrillation (AF) has been previously described as associated with lower recanalization rates in patients receiving intravenous thrombolysis.^23,24 However, these analyses were conducted before the widespread use of MT. A more recent multicenter registry analysis investigated the effects of MT in AF patients and found comparable recanalization rates between AF and non-AF patients. ²⁵ Another recent single-center analysis has reported substantially higher reperfusion rates achieved with MT when comparing AF to non-AF patients. ²⁶ A subtle unexpected finding from the present analysis is that AF seems to favor DR. As there is a lack of literature on the relationship between AF and DR, we presently do not have a tangible pathophysiological explanation for this finding. AF could serve as a proxy for another variable of interest, as we have only focused on baseline and interventional variables which are easily obtainable in the setting of acute patient management (see Methods).

Number of ⩽2 maneuver counts has shown a strong association for achieving successful reperfusion.^9,27 It is known that with an increase of maneuver counts the likelihood of complete reperfusion strongly reduces: for example, if reperfusion is not achieved within five maneuvers, the likelihood of successful reperfusion decreases by half. ²⁷ Potential explanation for this would be that thrombi imperviable to mechanical manipulation are also more likely to be resistant to the effects of lytics or to succumb to the effects of autolysis on follow-up, leaving a PPD in the distal tissue. eTICI score appeared to be the strongest predictor of delayed reperfusion with also the highest MDA index. This has also promoted international societies to recommend achieving near-complete or complete reperfusion whenever possible in acute stroke patient care.^1,2 As eTICI grade increases so does the percentage of reperfused territory where for example eTICI 2c patients will have substantially smaller distal deficits when compared to eTICI 2b67 patients. ²⁸ Far distal perfusion deficit entails possible presence of a smaller thrombi which is more likely to spontaneously dissolve and enable complete DR. ²⁹ Pursuit in achieving the highest possible eTICI score is offset by an increased risk of interventional complications from adjuvant reperfusion efforts (e.g. vessel puncture or perforation). True equipoise of this cost-benefit ratio is presently unknown and should be regarded when deciding to pursue additional reperfusion attempts. Good collateral circulation has been continually associated with successful reperfusion. ³⁰ Developed collaterals may allow continued perfusion to the area distal of the occlusion, so even if complete reperfusion is not achieved at the end of MT, patient might still experience complete DR due to increased collateral flow and vascular remodeling. ³⁰

Despite having similar baseline and interventional characteristics (as shown in our example in the Results: 50%–66% vs 67%–89% reperfusion of target downstream territory), predictions of perfusion imaging outcome can widely differ. When considering individual patient outcome independent value of these characteristics is reduced, because in reality outcome is not influenced by just one single factor, but rather by a combination of them. ³¹ The combination of these interventional metrics together with baseline values increases the accuracy by which heterogeneous perfusion imaging outcomes, that are clinically relevant, can be identified.

Model validation

ESO, ESMINT, and AHA guidelines all recommend an individualized decision-making approach for patients with incomplete reperfusion after thrombectomy.^1,2 For these purposes, validated prediction models tend to be preferable to physicians’ estimates, which are inherently limited compared to information contained in large datasets used for building prediction models. ³¹ However, a recently conducted systematic review highlighted several methodological concerns with many published clinical prediction models. ³² Several studies had not internally validated their models nor corrected for over- and under-fitting, whereas other models excluded patients with missing values, making the use and interpretation of these models worrisome. ³² We did not exclude any patients with missing values and implemented thorough methodological safeguards during validation and overall fitting of the model (see Methods). Great care was taken when initially screening for potential confounding variables, as all variables ought to be easily obtainable in all comprehensive stroke centers, so that all required information are available when making the “proceed-or-stop” decision in the angiography suite. Despite having good discrimination and calibration, the present model tended to overestimate true rates of PPD. Predictive value of the model should generally be as high as possible, ³² to ensure that necessary further treatment options are not deferred for patients who would not have DR and consequently would not achieve a good functional outcome without adjunctive reperfusion efforts. Further updates on the current model are planned. External validation will be performed on patients enrolled in the Endovascular Therapy for Ischemic Stroke with Perfusion-Imaging Selection (EXTEND-IA), Tenecteplase Versus Alteplase Before Endovascular Therapy for Ischemic Stroke (EXTEND-IA TNK) part 1 and part 2 trials, who have available follow-up perfusion imaging data obtained 24 ± 12 h after the intervention. (clinicaltrials.gov, unique identifier: NCT01492725, NCT02388061, and NCT03340493, respectively).

Limitations

The present model has several limitations. Its single-center retrospective study design might limit the generalizability of reported findings. More than half of patients did not receive intravenous alteplase before MT, which is a lower frequency than reported in national registries.^33,34 However, the impact of intravenous thrombolysis on the occurrence of DR remains unclear. ³⁵ Patients without perfusion follow-up imaging were excluded from the present analysis. This selection bias might overestimate absolute rates of DR, as these might differ in the patients not undergoing perfusion imaging on follow-up. Performance of the model was compared to reference scenarios “Treat All” and “Treat None”; however, in the real world the reference scenario is the physician making the individual decision. Future options should explore this as a reference for benefit. Validations using external datasets are required to fully explore the model’s performance.

Conclusion

The model presented here shows fair predictive accuracy for estimating chances of delayed reperfusion after incomplete thrombectomy. This may inform treating physicians on the chances of a favorable natural disease progression if no further reperfusion attempts are made.