Early experience utilizing artificial intelligence shows significant reduction in transfer times and length of stay in a hub and spoke model

Abstract

Background

Recently approved artificial intelligence (AI) software utilizes AI powered large vessel occlusion (LVO) detection technology which automatically identifies suspected LVO through CT angiogram (CTA) imaging and alerts on-call stroke teams. We performed this analysis to determine if utilization of AI software and workflow platform can reduce the transfer time (time interval between CTA at a primary stroke center (PSC) to door-in at a comprehensive stroke center (CSC)).

Methods

We compared the transfer time for all LVO transfer patients from a single spoke PSC to our CSC prior to and after incorporating AI Software (Viz.ai LVO). Using a prospectively collected stroke database at a CSC, demographics, mRS at discharge, mortality rate at discharge, length of stay (LOS) in hospital and neurological-ICU were examined.

Results

There were a total of 43 patients during the study period (median age 72.0 ± 12.54 yrs., 51.16% women). Analysis of 28 patients from the pre-AI software (median age 73.5 ± 12.28 yrs., 46.4% women), and 15 patients from the post-AI software (median age 70.0 ± 13.29 yrs., 60.00% women). Following implementation of AI software, median CTA time at PSC to door-in at CSC was significantly reduced by an average of 22.5 min. (132.5 min versus 110 min; p = 0.0470).

Conclusions

The incorporation of AI software was associated with an improvement in transfer times for LVO patients as well as a reduction in the overall hospital LOS and LOS in the neurological-ICU. More extensive studies are warranted to expand on the ability of AI technology to improve transfer times and outcomes for LVO patients.

Introduction

Endovascular treatments (EVT) such as intra-arterial thrombolysis or mechanical thrombectomy (MT) are widely proven methods to treat patients suffering a large vessel occlusion (LVO). However, treatment efficacy has been proven to be highly time dependent based on several randomized published clinical trials.^1–4 The vast majority of delays are developed and elongated through the transfer of patients from various stroke centers due to the additional travel time and the multiple personnel involved.

Through the utilization of artificial intelligence (AI) programs such as Viz.ai, which utilizes AI powered LVO detection technology that can now automatically identify suspected LVO through CT angiogram (CTA) imaging and can alert on call stroke teams, the stroke workflow can smoothly transition from its traditional serial processes into parallel processes which allows for a reduction in transfer times and the possibility of improved outcomes. The primary focus of our study was to compare the time interval between CTA and door-in for all LVO transfer patients from a single spoke primary stroke center (PSC) to our comprehensive stroke center (CSC) prior to (February 2017 and November 2018) and after (November 2018 to May 2019) incorporating AI software.

Methods

A retrospective study was conducted including LVO patients who originally presented to a primary stroke center and were transferred to a nearby comprehensive stroke center. All patients who presented at this PSC were transferred to our CSC if endovascular treatment was initially deemed necessary. Our selection criteria for patients in this study included imaging, clinical, and time-frame criteria. For both populations, selected patients must have presented with an LVO on CT angiogram at the primary stroke center. All patients were transferred to the CSC with the intent of having endovascular treatment performed and were divided into the two groups based on whether they were transferred between February 2017 and November 2018 (pre-AI software implementation) or between November 2018 to May 2019 (post-AI software implementation); workflow within these time frames is outlined in Figure 1. The AI Software utilized in this study was Viz.ai LVO (Viz.ai, Inc. San Francisco CA). Viz.ai LVO is a parallel workflow tool approved by the U.S. Food and Drug Administration to analyze CT angiogram images of the brain acquired in the acute setting, send notifications that a suspected large vessel occlusion has been identified, and put together a review of those images. The AI software, on average, alerts the on-call physicians within six minutes after CT angiogram is completed and alerts the physician through a built-in ringtone specific for cases that have a confirmed LVO. Images can then be accessed through the mobile application; they are compressed for informational purposes only and not intended for diagnostic use beyond notification. This computer-aided triage system utilizes an algorithm to analyze images for indicators associated with stroke. These AI algorithms are a type of clinical decision support software that can assist providers in identifying the most appropriate treatment plan for a patient’s disease or condition. The Viz.ai LVO software was reviewed through the De Novo premarket review pathway, a regulatory pathway for new types of medical devices that are low to moderate risk. Imaging software for the development of automated perfusions maps as well as other radiological findings are available from Viz.ai also in the form of Viz.ai CTP. This is comparable to the Rapid software that has been present in the market for some years which is used to quantify core infarct (irreversibly damaged) vs penumbra (potentially salvageable).

Figure 1.

Depiction of the Hub and Spoke Network Before and After the Implementation of AI Software using mean time of CT Angiogram at primary stroke center to door-in at comprehensive stroke center.

Patients who had a modified Rankin Scale (mRS) score of <3 had the risks and benefits of EVT discussed with their family. Endovascular treatment was withheld from four patients in the post-AI software population due to the fact that they experienced thrombolytic recanalization from IV tPA or did not meet criteria for endovascular treatment once at CSC due to extensive infarction. A prospectively maintained database was compiled at the CSC where the process for data collection was reviewed and approved at the institution; all patient details regarding procedures were recorded and stored. The study was approved by the local Institutional Review Board.

Data collected

Baseline variables and radiographic, clinical, and safety outcome rates were also included. Baseline variables include age, gender, ethnicity, admission NIHSS score, time from last seen well to presentation, and patients’ past medical history of hypertension, diabetes mellitus, atrial fibrillation, and cigarette smoking. Safety outcomes included rates of symptomatic ICH where symptomatic ICH was defined as a patient having ICH on a 24-hour CT scan with a worsening NIHSS of 4 or greater. MRI and non-contrast CT were utilized for all follow-up imaging which was done 24 ± 4 hours after intervention. Clinical outcomes included mRS scores upon discharge, length of stay in the neurological ICU, length of overall hospital stay, and mortality rates at discharge. The radiographic outcome was the rate of successful recanalization (modified Thrombolysis in Cerebral Infraction (TICI ≥2 b)).

“PSC to CSC Transfer,” which included all patients, was defined as time of CTA at PSC to door-in at CSC. “CTA to Puncture” was defined as CTA time at PSC to groin puncture at CSC. “CSC Door-In to Puncture” was defined as the time from door-in at CSC emergency department to time of groin puncture. The door-in to door-out (DIDO) time interval in the PSC was also analyzed. Given the design of this analysis and that it involves a historical control, the longer the historical control, the more valid the data. Therefore, a larger time frame was utilized in the pre-AI population versus that of the post-AI population. These results are part of an ongoing analysis with scheduled interim analyses every 6 months.

Statistical analysis

For this data set, we performed an univariate analysis of the baseline variables and outcomes which included t test for continuous variables (age, NIHSS upon admission, etc.), z test for co-morbid conditions and outcomes, and chi-squared test for categorical data in order to identify differences in baseline characteristics (gender, race/ethnicity). Outcomes in comparison included mRS 0–2 score at discharge, post-TICI 2B-3, mortality rates, successful recanalization, symptomatic hemorrhage rates, overall LOS, and LOS in the neurological ICU.

To adjust for imbalances pre-AI and post-AI patients, logistic regression analyses were performed to determine the correlation between the implementation of AI software and (1) mortality rate, (2) good outcome at discharge (mRS score 0–2), (3) good TICI score (2B-3), and (4) symptomatic ICH rates. All variables that were determined to be statistically significant in the in the univariate analysis were added to the logistic regression model. In the model analysis, the variables included were atrial fibrillation (categorical) and NIHSS score upon admission (continuous). P-values for the medians associated with length of stay and the four analyzed time intervals were calculated through the utilization of the Mann-Whitney U Test in order to compare outcomes between the independent groups. Due to the small sample sizes in this exploratory study, a significance level was determined at p < 0.20. Statistical analysis was performed using MedCalc statistical software.

Results

There was a total of 43 patients during the study period (median age 72.0 ± 12.54 yrs., 51.16% women) transferred from a PSC to a CSC. Analysis of 28 patients from the pre-AI software group (median age 73.5 ± 12.28 yrs., 46.4% women), and 15 patients from the post-AI software was performed (median age 70.0 ± 13.29 yrs., 60.00% women). Out of the 43 patients, 25.6% (11/43) were transferred and treated with endovascular treatment (EVT) after the implementation of AI software. 75.7% of all vessels analyzed pre- and post-AI were middle cerebral artery (MCA) M1 and M2 occlusions, and the remaining 24.3% of vessels analyzed were internal cerebral artery terminus (ICA-T) occlusions. All baseline and procedural data were available for all patients treated with EVT, and all patient data was included in the analysis. Results of the univariate analysis for baseline characteristics and clinical outcomes are summarized in Table 1.

Table 1.

Results from univariate analysis of baseline characteristics and outcomes between patients treated before and after the implementation of the AI software.

Characteristics	Outcomes		ρ value
	Pre-AI software(N = 28)	Post-AI software(N = 15)
Age (mean ± SD)	71.64 ± 12.28	69.13 ± 13.29	0.549
Gender			0.396
Men	15 (53.6%)	6 (40.0%)
Women	13 (46.4%)	9 (60.0%)
Race/Ethnicity			0.252
White	5 (17.9%)	5 (30.0%)
Hispanic	23 (82.1%)	10 (70.0%)
African American	0 (0.0%)	0 (0.0%)
Asian	0 (0.0%)	0 (0.0%)
NIHSS upon admission	18.25 ± 7.43	14.07 ± 6.75	0.071
IV tPA Use at PSC	9 (32.1%)	5 (33.3%)	0.936
Co-Morbid Conditions
Diabetes Mellitus	12 (42.9%)	7 (46.7%)	0.811
Hypertension	25 (89.3%)	13 (86.7%)	0.798
Atrial Fibrillation	10 (35.7%)	1 (6.7%)	0.0375
Cigarette Smoking	2 (7.1%)	2 (13.3%)	0.505
Thrombolysis in Cerebral Infarctiona
Good (post TICI 2B-3)	24 (85.7%)	9 (81.8%)	0.762
Poor (post TICI 0-2A)	4 (14.3%)	2 (18.2%)	0.762
Outcome (Discharge)
Good (mRS dc score 0–2)	8 (28.6%)	6 (40.0%)	0.223
Poor (mRS dc score 3–6)	20 (71.4%)	9 (60.0%)	0.223
Mean Transfer Time (mins.)	171.29 ± 110.58	105.27 ± 62.09	0.0163
Median Transfer Time (range)	132.5 (56–539)	110 (29–237)	0.0470
Median DIDO at PSC (range)	122 (46–244)	105 (59–174)	0.106
Median Length of Stay (Days)
Admission to Discharge	9.7 ± 4.9	7.2 ± 2.5	0.0324
Neuro-ICU to General	6.4 ± 3.8	2.9 ± 1.6	0.0039
In-hospital Complication
Symptomatic intracerebral hemorrhage	2 (7.1%)	1 (6.7%)	0.953
Asymptomatic intracerebral hemorrhage	1 (3.6%)	0 (0.0%)	0.459
Mortality (Discharge)	6 (21.4%)	4 (26.7%)	0.698

^aN = 11 for Thrombolysis in Cerebral Infarction Scores Statistical Comparison.

Table 1 expresses that the DIDO time interval for the PSC was reduced in the post-AI population (122 vs 105 min [p = 0.106]), and furthermore, Figure 2 expresses the median times following the implementation of AI software and shows the PSC to CSC median transfer to be considerably shortened in the post-AI population (132.5 vs 110 minutes [p = 0.047]) as well as the CTA at PSC to groin puncture at CSC (216 vs 127 minutes [p = 0.026]), pre- and post-AI, respectively.

Figure 2.

Time to Treatment Metrics Before and After the Implementation of AI Software. Outlier(s) at 944 mins for “CTA to Puncture” Pre-Viz.ai and 971 for “CTA to Puncture” Post-AI.

Figure 3 summarizes the median length of stay from admission to discharge and length of stay in the Neuro-ICU, and shows that the overall stay was considerably shortened in the Post-AI software population (9 vs 7 days [p = 0.124]) as well as in the Neuro-ICU (5.5 vs 3 days [p = 0.00086]), pre- and post-AI software, respectively.

Figure 3.

Overall and Neuro-ICU Length of Stay Before and After the Implementation of AI Software.

The mortality rate at discharge did not show a statistical difference between the two groups (p = 0.698). Reperfusion was successful (TICI 2B-3) in 85.7% (24/28) patients in the pre-AI software group compared to 81.8% (9/11) patients in the post-AI software group. Good clinical outcomes at discharge (mRS ≤ 2) was achieved in 28.6% (8/28) of patients in the pre-AI software group compared to 40.0% (6/15) of patients in the post-AI software group. The mean NIH Stroke Scale upon admission for pre-AI software patients was 18.25 ± 7.43 compared to 14.07 ± 6.75 for post-AI Software patients (p = 0.071). IV tPA use at the PSC was similar between the two groups, with 32.1% (9/25) of patients receiving pre-AI software and 33.3% (5/15) of patients receiving post- AI software.

Atrial fibrillation and NIHSS score upon admission demonstrated a statistically significant difference between the pre-AI and post-AI populations. All other variables, disregarding LOS and transfer time, did not show a significant difference (p > 0.20), and therefore were not included into the multivariate analysis. Multivariate analysis results are summarized in Table 2. The odds of mortality rate (OR 0.290, 95% CI 0.254–2.222), good clinical outcome at discharge (OR 0.571, 95% CI 0.167–1.102), good TICI score (OR 5.414, 95% CI 1.721–17.032), and rate of S-ICH (OR 1.744, 95% CI 0.132–23.124) were similar among those who underwent EVT prior to the implementation of AI and EVT post implementation of AI after adjusting for potential confounding variables.

Table 2.

Multivariate analysis evaluating effect of AI software on outcomes of large vessel occlusion patients who underwent endovascular therapy.

Outcomes	Unadjusted		Adjusted for atrial fibrillation and NIHSS upon admission
	OR (95% CI)	ρ value	OR (95% CI)	ρ value
Mortality rate	1.333 (0.310–5.727)	0.698	0.290 (0.254–2.222)	0.331
Good outcome at discharge (mRS score 0–2)	0.600 (0.161–2.244)	0.223	0.571 (0.167–1.102)	0.661
Good TICI score (2B–3)	0.923 (0.149–5.735)	0.932	5.414 (1.721–17.032)	0.854
Symptomatic hemorrhage	0.929 (0.077–11.16)	0.953	1.744 (0.132–23.124)	0.417

CI = confidence interval; OR = odds ratio; mRS = modified Rankin Scale.

Discussion

The current study represents a small, multicenter retrospective series of LVO patients treated prior to and after the implementation of AI software in a hub and spoke network. Our study showed that the implementation of AI software does show a significant reduction in overall LOS, LOS in the neuro-ICU, transfer time between PSC and CSC, and CTA to Puncture times. Our study emphasized the importance of speedy endovascular therapy in LVO patients who presented to a PSC needing CTA and EVT. The median transfer time from the PSC to the CSC was reduced by 22.5 minutes, the median DIDO time at PSC was reduced by 17 minutes and the median time from CTA at PSC to Puncture at CSC was reduced by 89 minutes. Interestingly, the CSC door-in to puncture time was slightly longer than the post-AI population. It is generally expected that time intervals within the CSC generally decrease over time due to improvement of the stroke team. However, it is possible that patients in the post-AI population may have arrived during off hours, resulting in delays of the call team’s response time. With a larger sample size, it is true that we would likely see an improvement within the CSC time of door-in to groin puncture.

Furthermore, as shown in Figure 2, the PSC to CSC median transfer time was considerably shortened in the Post-AI population as well as the CTA at PSC to groin puncture at CSC. Overall, the faster rates of treatment which became possible due to the expedited workflow and transfer process created by AI Software can lead to improved levels of improved functional independence in LVO patients. Goyal et al. published a similar study in order to investigate variables that affect time spent during discrete steps in AIS care.⁵ The study found that a symptom onset to reperfusion time of 150 minutes resulted in 91% probability of functional independence; this probability decreased 10% over the next hour, and 20% with every subsequent hour of delay.⁵ In our hub and spoke network, AI software eliminated seven steps which were present in the historical workflow, and therefore resulted in a superior, expedited workflow. McTaggart et al. published a similar study in which 14 regional PSCs were instructed on the use of a three-step protocol that aimed at reducing time spent at PSC. When this protocol was fully implemented at the various PSCs, a significant reduction in the median time for PSC arrival to CSC groin puncture was noted, and resulted in patients being twice as likely to have favorable outcomes (50% vs 25%, p < 0.04).⁶ Additionally, Ng et al. posted a large, retrospective study which focused on the characterization of transfer workflow from three high-volume PSCs to a single CSC.⁷ Median transfer time was measured to be 128 minutes (IQR 107–164), of which 82.8% was spent at PSCs. The lengthiest component of this was computed-tomography-to-retrieval-request.⁷ Similarly, this step was considerably shortened in the post-AI software track of our study (Figure 1).

Our study found that clinical outcomes—namely, functional independence at discharge (mRS 0–2) and mortality rate—were not significantly different between the two groups. However, it is reasonable to speculate that in time and with larger populations, a difference in functional independence will become apparent due to the implementation of AI Software. For instance, Froehler et al. published a large, real-world study in which all transfer patient (n = 445) outcomes were compared to those who arrived directly (n = 539) to the endovascular-capable centers.³ This study revealed a significantly increased median time from stroke onset to revascularization for direct versus transfer patients receiving imaging prior to MT at an endovascular-capable center (192 mins. vs 311.5 mins., respectively). In turn, this revealed a much higher outcome of mRS 0–2 at 90 days for patients within the direct treatment population (p = 0.035).³ Khatri et al. published a large, retrospective analysis (n = 240) which further highlights the importance in avoiding delays from stroke onset to angiographic reperfusion.² For patients diagnosed with complete proximal arterial occlusions who received endovascular treatment within seven hours of symptom onset, the adjusted relative risk for every 30 minute delay displays the association between increased time and decreased amount of good modified TICI outcomes (2B-3) and good outcomes at 90 days (mRS 0–2).² As shown through Figures 1 and 2, the transfer times are considerably reduced in our hub and spoke network after the implementation of AI Software, and therefore, a larger, multicenter study may yield results which show a considerable improvement in mRS, reperfusion rates, and mortality rates at discharge.

There are several reasons for the prominent difference in treatment and transfer times between the pre-AI and post-AI populations. In the historical track prior to the implementation of AI software, the Radiology Technologist at the PSC was required to reconstruct and interpret the images, inform the ED Physician of findings, confirm LVO, and communicate with the ED Physician at the PSC and House Supervisor at CSC in order to ultimately transfer the patient. However, AI improves workflow by eliminating the majority of protracted interpersonal communication which occurs at PSC. This software then neatly packages the imaging and information and allows for the physician at the CSC to interpret the imaging, confirm LVO, and clear the patient for transfer to CSC.

From the perspective of cost, a study published in 2018 elaborates on the cost consequences of delayed endovascular treatments whilst further accentuating the negative effect on outcomes.¹ Kunz et al. utilized the Markov model which estimated lifetime quality-adjusted life years (QALYs) of EVT-treated patients and associated costs based on puncture times. Within the first six hours, every hour of delay resulted in average losses of 0.65 QALYs and increased healthcare costs by $6,418/QALY and societal costs by $9,443/QALY.¹ Furthermore, based on the analysis of the data from the HERMES collaboration of pooled patient-level data from seven trials (MR CLEAN, ESCAPE, REVASCAT, SWIFT, PRIME, EXTEND IA, THRACE, and PISTE), published in 2015–2016, analyzed patient groups were assigned to receive EVT. These studies found that on average, every hour of delay in starting the EVT was associated with a loss of 0.64 quality-adjusted life years, which in turn results in a loss of 7.7 months of disability-free living for the patient. Additionally, every hour of delay reduced the economic value of care of the EVT by $63,558 due to the increased LOS in the Neuro-ICU and general hospital stay.⁸

Our study suggests that the utilization of AI in the workflow and transfer of patients from a PSC to a CSC has the potential to improve outcomes among LVO patients. A considerable reduction in transfer times, CTA to puncture times, and overall LOS and LOS in the Neuro-ICU were noted after the incorporation of AI Software. Transfer times may represent the single biggest factor which can be modified in order to increase the likelihood of a positive outcomes in regard to reperfusion and mRS scores of 0–2 in LVO patients. These findings and improvements in transfer and treatment times represent a major opportunity to expedite EVT and improve patient outcomes within hub and spoke networks.

Limitations

The most important limitations in this study are its retrospective nature and small sample sizes. The small sample sizes decrease the power to detect differences among subgroups of patients for endpoints such as mortality and good outcomes. However, through the utilization of a Nomogram derived from Whitley et al. and calculations of sample size and power,⁹ it is valid to assume that our current sample size was poised only to identify large differences in endpoints, and therefore are primarily hypothesis generating in our study design. The fact that the pre-AI group had larger admission NIHSS scores which in turn contributed largely to safety outcome rates was a finding that required rigorous controlling; it must be noted that the multivariate adjustment does not necessarily fully address this imbalance. Additionally, it is important to recognize that AI software can sometimes have false-positive and false-negative alerts. Lastly, four patients in the post-AI population did not have any form of EVT performed resulting in a limited sample size for our CSC to door-in to groin puncture and recanalization rate analysis. Larger multicenter prospective studies would be necessary to corroborate the results of our study.

Conclusion

In conclusion, the implementation of AI in this hub and spoke network ultimately reduced transfer time from the PSC to CSC, overall LOS, and LOS in the neurological-ICU. This data furthers the idea that AI software along with an improved workflow is a very effective tool that may allow for reduced costs and improved patient outcomes. More extensive studies are warranted to expand on the ability of AI technology such as Viz.ai LVO to improve transfer times and outcomes in LVO patients.

Contributors

AEH provided research question, analyzed the data, and revised the paper. VMR developed the statistical analyses, drafted the paper, and revised the paper. RRR revised the paper. WGT revised the paper. AIQ revised the paper.

Declaration of conflicting interests

The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: AEH: Consultant for Medtronic, Microvention, Penumbra, Stryker, Genentech, Balt, Viz.ai, and GE Healthcare.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Ameer E Hassan https://orcid.org/0000-0002-7148-7616