The implementation of artificial intelligence significantly reduces door-in-door-out times in a primary care center prior to transfer

Abstract

Introduction

Viz LVO artificial intelligence (AI) software utilizes AI-powered large vessel occlusion (LVO) detection technology which automatically identifies suspected LVO through CT angiogram (CTA) imaging and alerts on-call stroke teams. This analysis was performed to determine whether AI software can reduce the door-in-door-out (DIDO) time interval within the primary care center (PSC) prior to transfer to the comprehensive care center (CSC).

Methods

We compared the DIDO time interval for all LVO transfer patients from a single-spoke PSC to our CSC prior to (February 2017 to November 2018) and after (November 2018 to June 2020) incorporating AI. Using a stroke database at a CSC, demographics, DIDO time at PSC, modified Rankin Scale (mRS) at 90-days, mortality rate at discharge, length of stay (LOS), and intracranial hemorrhage rates were examined.

Results

There were a total of 63 patients during the study period (average age 69.99 ± 13.72, 55.56% female). We analyzed 28 patients pre-AI (average age 71.64 ± 12.28, 46.4% female), and 35 patients post-AI (average age 68.67 ± 14.88, 62.9% female). After implementing the AI software, the mean DIDO time interval within the PSC significantly improved by 102.3 min (226.7 versus 124.4 min; p = 0.0374).

Conclusion

The incorporation of the AI software was associated with a significant improvement in DIDO times within the PSC as well as CTA to door-out time in the PSC. More extensive studies are warranted to expand on the ability of AI technology to improve transfer times and outcomes for LVO patients.

Introduction

Mechanical thrombectomy (MT) is a proven safe and efficacious means of treating large vessel occlusion (LVO) and ameliorates the clinical outcomes of patients based on the results of six published randomized clinical trials.^1–6 There is an imperative need for the speedy verification of suspected LVOs as it has been previously determined that morbidity and mortality outcomes are highly dependent on treatment time.^7–10 The presence of LVO in particular is associated with higher rates of severe cerebral infarction and resultingly worsened outcomes when endovascular therapy is delayed. ¹¹

The hub-and-spoke model presented in this manuscript has been developed such that potential stroke patients are sent to the nearest emergency department available. This model oftentimes results in patients being sent to the primary stroke center (PSC), which is located approximately twenty-five miles (forty kilometers) from the comprehensive stroke center (CSC). At the PSC, patients are given emergency care and, if the eligibility criteria are met, IV tPA is administered. After CT angiogram (CTA), patients are transferred from the PSC to the CSC where thrombectomy and further intervention is performed. To reduce the time from symptom onset to vessel reperfusion, it is imperative that there be a speedy door-in-door-out (DIDO) time interval at a PSC, which lacks on-site thrombectomy intervention. Artificial intelligence (AI) programs may be used as a tool to improve communication between physicians and staff within the PSC and between the PSC and CSC to realize a faster DIDO time interval.

The AI software analyzed in this study is Viz LVO, which utilizes AI-powered LVO detection technology that can automatically identify suspected LVO through CTA imaging and alert on-call stroke teams. This unified alert system has the capability to transition the previously serial workflow into a parallel process that may allow for a faster triage of patients arriving at the PSC.^12,13 The primary focus of this study was to compare the DIDO time interval for all LVO transfer patients from a single spoke PSC to a CSC prior to (February 2017 to November 2018) and after (November 2018 and June 2020) implementing Viz LVO software.

Methods

This study involved a retrospective analysis of patients who originally presented to the primary stroke center (PSC), all with signs of LVO. These patients were classified into groups based on the date of treatment. Patients between February 2017 and November 2018 were transferred and treated prior to the implementation of the AI software and patients between November 2018 and June 2020 were transferred and treated after the implementation of the AI software. The AI software utilized in this study was Viz LVO (Viz.ai, Inc. San Francisco CA), which is a parallel workflow tool approved by the U.S. Food and Drug Administration (Viz ContaCT) to analyze CT angiogram (CTA) images of the brain acquired in the acute setting. The images of suspected LVOs are compiled and sent as a notification to the stroke team. Viz LVO was implemented as a commercial device at the PSC, and this study is a chart review of standard of care records. The risks and benefits of endovascular therapy treatment were discussed with patient families. This study received local Institutional Review Board (IRB) approval of waiver of consent and XXXX.

Inclusion criteria included patients who presented consecutively with signs of LVO on CTA at the PSC. All patients, except five in the pre-AI population and eight in the post-AI population, were treated with mechanical thrombectomy with or without angioplasty and stenting in the cath lab. Patients had follow-up imaging 24 ± 4 h after intervention which included MRI and non-contrast CT. Additional inclusion criteria for patients in the post-AI group was to be flagged by the AI software as having signs of LVO upon initial CTA at the PSC. Patients who initially presented and had a CTA at the comprehensive stroke center (CSC) were excluded from the study.

Prior to the implementation of the AI software, patients were triaged in a linear process. They were initially treated by the emergency department (ED) physician and referred for a CTA that was conducted by the technologist After this scan was read by the radiologist, the information regarding vessel occlusion was sent to the ED physician and the neurologist who recommended care. This information was then referred to the interventionalist, the patient was transferred to the CSC, and the thrombectomy was performed.

Following the implementation of the AI software, patient triage occurred in a more unified fashion and parallel process. Once signs of LVO were detected by the software, the triage alerts of the CTA scan were simultaneously sent to the mobile devices of the ED physician, radiologist, neurologist, and interventionalist The patient was then transferred and treated based on the unified communication of all the involved parties. The historic and revised triage of patients are presented in Figure 1.

Figure 1.

Depiction of the door-in-door-out patient triage in the primary care center prior to and after the implementation of the artificial intelligence software.

Data collected

Patients who met the inclusion and exclusion criteria listed in the methods section were consecutively selected for this study from the database. Baseline variables and clinical, radiographic, and safety outcomes were included. Baseline variables included age, sex, ethnicity, admission National Institutes of Health Stroke Scale (NIHSS) score, door-in-door-out time interval at the PSC, and past medical history of hypertension, atrial fibrillation, coronary artery disease, smoking status, diabetes mellitus, and a history of stroke/transient ischemic attack. Safety outcomes included rates of hemorrhagic transformation, mass effect, symptomatic intracerebral hemorrhage (ICH) and asymptomatic ICH. Symptomatic ICH was classified in this study by the European Cooperative Acute Stroke Study definition as any type of intracerebral hemorrhage on any posttreatment imaging after the start of thrombolysis and increase of ≥ 4 NIHSS points from baseline, from the lowest value within 7 days, or leading to death. Measures of mass effect were evaluated in cerebral edema, including changes in hemisphere volume, cerebral spinal fluid (CSF) volumetric analysis, and midline shift. MRI and non-contrast CT were utilized for all follow-up imaging which was done 24 ± 4 h after intervention. Clinical outcomes included 90-day mRS scores, length of hospital stay (LOS), and mortality rates. The radiographic outcome was the rate of reperfusion (modified Thrombolysis in Cerebral Infraction (mTICI ≥ 2b)) which refers to post-procedural digital subtraction angiography. Good clinical outcomes were defined as mRS scores of 0-2 at 90 days.

Statistical analysis

A univariate analysis of the dataset was performed using MedCalc statistical software for baseline variables and outcomes with a chosen significance level of 0.05. The analysis included a t-test for continuous variables (age, NIHSS upon admission, etc.), z-test for co-morbid conditions and outcomes, and chi-squared test for categorical data in order to identify differences in baseline characteristics (sex and race/ethnicity). The outcomes that were compared included mRS 0-2 score at 90 days, post-mTICI 2b-3, mortality rates at discharge, symptomatic hemorrhage rates, and overall LOS. P-values associated with the median length of stay, median DIDO, and median CTA to door out were calculated through the utilization of the Mann-Whitney U Test in order to compare outcomes between the independent groups. Mortality post-discharge from the CSC was not tracked.

In order to adjust for imbalances between patients treated prior to and after the implementation of the AI software, a logistic regression analysis was performed to determine correlation between the AI software and (1) mortality rate, (2) good outcome at 90 days (mRS score 0-2), (3) good mTICI score (2b-3), and (4) symptomatic intracerebral hemorrhage (ICH). The model analysis included the categorical variables age, IV tPA use, and atrial fibrillation. This analysis was also performed using MedCalc statistical software.

Results

There were a total of 63 patients during the study period (mean age 69.99 ± 13.72 years., 55.6% female) who initially presented to the primary stroke center (PSC). An analysis of 28 patients from the pre-AI software group (mean age 71.64 ± 12.28 years., 46.4% female) and 35 patients from the post-AI software group (mean age 68.67 ± 14.88 years., 62.9% female) was performed. The mean NIH Stroke Scale upon admission for the pre-AI software patients was 18.25 ± 7.43 compared to 16.01 ± 6.01 for the patients in the post-AI software group (p = 0.202). IV tPA use at the PSC was similar between the two groups, with 32.1% (9/28) of patients receiving IV tPA in the pre-AI group versus 17.1% (6/35) of patients receiving tPA in the post-AI group (p = 0.165). All baseline data was available for patients presenting to the PSC, and all data was included in the analysis. Results of the univariate analysis for baseline demographics and clinical characteristics are summarized in Table 1.

Table 1.

Baseline demographics and clinical characteristics of patients presenting to the primary stroke center prior to and after the implementation of the ai software.

Characteristics	Outcomes		P value
	Pre-AI Software (N = 28)	Post-AI Software (N = 35)
Age (mean ± SD)	71.64 ± 12.28	68.67 ± 14.88	0.389
Sex			0.192
Male	15 (53.6%)	13 (37.1%)
Female	13 (46.4%)	22 (62.9%)
Race/Ethnicity			0.670
White	5 (17.9%)	5 (14.3%)
Hispanic	23 (82.1%)	30 (85.7%)
African American	0 (0.0%)	0 (0.0%)
Asian	0 (0.0%)	0 (0.0%)
NIHSS upon admission	18.25 ± 7.43	16.01 ± 6.01	0.202
IV tPA Use at PSC	9 (32.1%)	6 (17.1%)	0.165
Co-Morbid Conditions
Diabetes mellitus	12 (42.9%)	13 (37.1%)	0.645
Hypertension	25 (89.3%)	28 (80.0%)	0.316
Atrial fibrillation	10 (35.7%)	8 (22.9%)	0.262
History of Stroke/TIA	6 (21.4%)	7 (20.0%)	0.889
Coronary Artery Disease	7 (25.0%)	9 (25.7%)	0.948
Cigarette smoking	2 (7.1%)	4 (11.4%)	0.565

Abbreviations: SD, standard deviation; NIHSS, NIH Stroke Scale; PSC, primary stroke center; mTICI, thrombolysis in cerebral infarction; 90-day mRS, modified Rankin Scale; TIA, transient ischemic attack; Significance Level: 0.05

A significant difference was reported in the mean door-in-door-out time interval at the PSC with 226.7 ± 242.9 min for the pre-AI software group versus 124.4 ± 57.6 min for the post-AI software group (p = 0.037); there was also a significant difference reported in the median for this metric (p = 0.042). A significant difference was also reported in the mean CTA to door-out time interval at the primary stroke center with 179.2 ± 126.2 min for the pre-AI software group versus 79.7 ± 43.5 min for the post-AI software group (p = 0.021); there was also a significant difference reported in the median for this metric (p = 0.038). The mortality rates at discharge did not show a significant difference between the two groups (p = 0.458). Procedural data regarding mTICI scores were only available for 23 of the 28 patients in the pre-AI software group and 27 of the 35 patients in the post-AI software group as some patients achieved adequate reperfusion from IV tPA, adequate reperfusion from natural clot dissolution, or were no longer considered to be eligible for thrombectomy after being transferred to the comprehensive stroke center based on the interpretation of the interventionalist Reperfusion was successful (mTICI 2b-3) in 82.6% (19/23) of patients in the pre-AI software group compared to 85.2% (23/27) of patients in the post-AI software group (p = 0.956). Good clinical outcome at 90 days (mRS 0-2) was achieved in 28.6% (8/28) of patients in the pre-AI software group compared to 40.0% (14/35) of patients in the post-AI software group (p = 0.344). Length of stay (LOS) from admission to discharge was similar between the two groups, with a median LOS of 9 [6-12] days for pre-AI patients and a median LOS of 6.5 [3.5-11] days for post-AI patients (p = 0.119). LOS in the neuro-ICU was also similar between the two groups with a median LOS of 5.5 [3-8] days for pre-AI patients and a median LOS of 4 [2-7.5] days for post-AI patients (p = 0.221). Results of the univariate analysis for treatment outcomes and average door-in-door-out (DIDO) times are summarized in Table 2.

Table 2.

Outcomes of patients presenting to the primary stroke center prior to and after the implementation of the ai software.

Characteristics	Outcomes		P value
	Pre-AI Software (N = 28)	Post-AI Software (N = 35)
Time Intervals, minutes
Mean DIDO, ± SD	226.7 ± 242.9	124.4 ± 57.6	0.037
Median DIDO, [IQR]	202 [110 − 269]	114 [58 − 147]	0.042
Mean CTA to Door-out	179.2 ± 126.2	79.7 ± 43.5	0.021
Median CTA to Door-out	138 [102 − 226]	59 [41.5 − 115]	0.038
Modified Thrombolysis in Cerebral Infarction
Good (post mTICI 2b-3)	19 (82.6%)	23 (85.2%)	0.804
Poor (post mTICI 0-2a)	4 (14.3%)	4 (14.8%)	0.804
In-hospital complication
Symptomatic intracerebral hemorrhage	2 (7.1%)	2 (5.7%)	0.817
Asymptomatic intracerebral hemorrhage	1 (3.6%)	0 (0.0%)	0.260
Hemorrhagic Transformation	3 (10.7%)	1 (2.8%)	0.203
Mass Effect	3 (10.7%)	2 (5.7%)	0.466
Outcome
Good (mRS 90-day score 0-2)	8 (28.6%)	14 (40.0%)	0.344
Poor (mRS 90-day score 3-6)	20 (71.4%)	21 (60.0%)	0.344
Length of Stay, Median [IQR]
Admission to Discharge (Days)	9 (6 − 12)	6.5 (3.5−11)	0.119
Neuro-ICU to General (Days)	5.5 (3−8)	4 (2−7.5)	0.221
Mortality at Discharge	6 (21.4%)	5 (14.3%)	0.458

*N = 23 for mTICI analysis of pre-AI, *N = 27 for mTICI analysis of post-AI.

Abbreviations: SD, standard deviation; IQR, interquartile range; mTICI, modified thrombolysis in cerebral infarction; mRS, modified Rankin Scale; TIA, transient ischemic attack; DIDO, door-in-door-out; Significance Level: 0.05

The effect of the AI software on patient outcomes was analyzed by multivariate logistic regression with adjustments for age, IV tPA use (prior to the CTA), and atrial fibrillation (Table 3). The odds of mortality (OR 0.937, 95% CI 0.846–1.037), good clinical outcome (OR 0.743, 95% CI 0.191–1.397), good mTICI score (OR 0.434, 95% CI 0.078–2.433), and rate of symptomatic ICH (OR 1.526, 95% CI 0.852–3.625) were similar amongst those who originally presented to the PSC post-implementation of the AI software after adjusting for potentially confounding variables.

Table 3.

Multivariate analysis evaluating the effect of the artificial intelligence software on outcomes of large vessel occlusion patients.

Outcomes	Unadjusted		Adjusted for age, IV tPA use, and atrial fibrillation
	OR (95% CI)	P Value	OR (95% CI)	P Value
Mortality Rate	0.611 (0.165–2.261)	0.458	0.937 (0.846–1.037)	0.370
Good outcome (mRS score 0-2)	0.600 (0.207–1.174)	0.344	0.743 (0.191–1.397)	0.288
Good mTICI score (2b—3)	0.826 (0.182–3.75)	0.804	0.434 (0.078–2.433)	0.735
Symptomatic Hemorrhage	0.788 (0.104–5.98)	0.817	1.526 (0.852–3.625)	0.195

*Abbreviations: CI, confidence interval; OR, odds ratio; mRS, modified Rankin Scale; mTICI, modified thrombolysis in cerebral infarction

Discussion

This study represents a small, single-center, retrospective series of LVO patients who presented consecutively to a primary stroke center (PSC) prior to and after the implementation of an AI technology. After implementation of the AI software, the DIDO time interval for patients in the PSC was significantly improved by an average of 102 min. This reduction of time at the PSC is important for patients to receive speedy endovascular therapy at the CSC. A driving factor for the time reduction after implementation of Viz LVO is the AI software's capability to identify large vessel occlusions in a reliable and accurate manner that surpasses the ability of well-trained professionals. In a recent study which analyzed 610 CTAs using the Viz software, LVO was perceived by the technology with a sensitivity of 87.6%, specificity of 88.5%, and accuracy of 87.9%. ¹⁴

Historically, there have been an array of optimization efforts to reduce time in treatment for patients with acute stroke, including improved workflow strategies as discussed by Fonarow et al. ¹⁵ and total quality improvement methods for reducing delays as discussed by Tilley et al. ¹⁶ However, even in experienced systems, a recurrent time delay of approximately 100 min has been recorded in patients who are transferred from a facility that is not capable of endovascular treatment (EVT) to one where the patient is treated with EVT. ¹⁶ This same study showed that a symptom onset to reperfusion time of 150 min resulted in a 91% chance of functional independence, with this metric decreasing by 10% in the next hour and approximately 20% with each hour of delay that followed. ¹⁶ As represented in Figure 1, the diagnostic and transfer process was reduced after Viz LVO implementation by a number of steps that may have accounted for the significant improvement in the DIDO times within the PSC.

A study from Ng et al. presented consecutive patients who were transferred from three high-volume PSCs to a single comprehensive stroke center. This research showed that 82.8% of transfer workflow at the PSC was composed of DIDO, which was calculated as patient time of arrival at the PSC to time of arterial puncture at the CSC. ¹⁷ Based on a key performance index, the authors set a 75-min target time for the PSC workflow which validates the importance of optimizing this time metric. As seen in our study through both the DIDO time metric as well as the CTA to door-out metric, this target time has not yet been met, but it has been markedly improved since the implementation of the AI software. There is an argument to be made that as time goes on, stroke centers improve their triage methods, ¹⁸ and that time alone may be the reason for the improved DIDO and CTA to door-out time intervals seen in this study. To account for this, similar time periods (22 months prior to AI implementation versus 20 months following AI implementation) of consecutive patients were utilized in this analysis.

This study suggests that the implementation of software such as Viz LVO is able to streamline stroke workflow within the primary care center of a hub-and-spoke network. A significant reduction in the DIDO time interval as well as the CTA to door-out time interval were noted in the post-AI population of this study. Improving treatment time remains paramount in the treatment of LVO due to the harsh physiological effects of cerebral ischemia. The findings in this study may represent a considerable opportunity to accelerate endovascular treatment and potentially improve the safety and clinical outcomes of LVO patients.

Limitations

The most important limitations in this study are its small sample sizes and retrospective nature. These small sample sizes reduce the power to detect differences among subgroups of patients for endpoints such as mortality and good outcomes. However, it is important to mention that our current sample size was only utilized in order to identify large discrepancies in endpoints and be primarily hypothesis generating in this design. Lastly, five patients in the pre-AI population and eight patients in the post-AI population did not have any form of EVT performed, resulting in a limited sample size for our recanalization rate analysis. Larger multicenter, prospective studies would be necessary to corroborate the results of our study and expand on the impact that AI software may have on patient outcomes.

Conclusion

After the implementation of the AI software in this hub and spoke model, the door-in-door-out time interval and the CTA to door-out time interval was significantly reduced at the primary stroke center prior to patient transfer. Other primary clinical outcomes remained similar amongst pre-AI and post-AI populations. This data may serve to promote the existing literature which has begun to show that AI software such as Viz AI may allow for the improvement of patient outcomes. More extensive studies are warranted to expand on the ability of AI technology such as Viz.ai LVO to improve transfer times and outcomes in LVO patients.