Abstract
This quality improvement study evaluates the use of artificial intelligence to accelerate triage of patients presenting to the emergency department with chest pain.
Eight million US emergency department (ED) visits for chest pain occur annually.1 Approximately 6% are caused by life-threatening conditions.2 Interventions that address ED overcrowding, including improved triage, may speed time to identification and treatment of these conditions.
Methods
In this multisite quality improvement study, we compared treatment intervals for adult patients (aged ≥18 years) with chest pain before and after implementation of an artificial intelligence (AI)–informed, outcomes-driven decision support system for ED triage (TriageGO; Beckman Coulter).3,4 The Western-Copernicus Group institutional review board approved this study, with a waiver of informed consent after minimal risk determination. The study follows the SQUIRE reporting guideline.
At the time of ED arrival, TriageGO uses machine learning algorithms to estimate probabilities for critical care, emergency procedures, and hospital admission using demographics, arrival mode, vital signs, chief complaints, and active medical problems as predictors; it translates outcome probabilities to recommended acuity levels (1-5, with lower values indicating higher acuity).3 TriageGO implementation at 3 EDs within a university health system was staggered between May 19, 2021, and April 4, 2023. At all sites, TriageGO replaced the emergency severity index.5
Patients who visited a study site with a chief complaint of chest pain 180 days before or after implementation of TriageGO were included in this analysis. Downstream protocols for diagnosis and treatment of chest pain remained consistent before and after intervention. Patient characteristics, triage acuity distributions, ED length of stay (LOS) (arrival to departure), time to procedure (arrival to procedure start time), and rates of hospitalization, intensive care unit (ICU) admission, emergency surgery, and 30-day mortality were compared.
To limit confounding, we performed adjusted analyses using median regression models that controlled for patient (age, sex, hospitalization status, emergency surgery status, in-hospital mortality, ICU admission) and system (daily volume, time of day, ED boarding census) factors. The eMethods in Supplement 1 provide additional details.
Python, version 3.8.11 (Python Software Foundation) was used for all analyses. A 2-sided P < .05 by Wilcoxon rank sum or χ2 test was considered significant.
Results
A total of 12 147 ED visits were analyzed, with 6188 visits before implementation (median [IQR] age, 52.5 [36.6-65.5] years; 3283 females [53.1%] and 2905 males [46.9%]) and 5959 after (median [IQR] age, 51.9 [36.7-64.2] years; 3075 females [51.6%] and 2884 males [48.4%]). All patient factors were similar between study periods (Table 1). After implementation of TriageGO, vs before implementation, fewer patients were assigned to high acuity levels 1 or 2 (1317 [22.1%] vs 1708 [27.6%]) or mid acuity level 3 (3263 [54.8%] vs 4086 [66.0%]) and more to low acuity levels 4 or 5 (1379 [23.1%] vs 394 [6.4%]) (χ24 = 771.6; P < .001).
Table 1. Cohort Characteristics and Clinical Outcomes.
| Patients, No. (%) | P valuea | |||
|---|---|---|---|---|
| Total (N = 12 147) | Before implementation (n = 6188) | After implementation (n = 5959) | ||
| Patient characteristics | ||||
| Age, median (IQR), y | 52.2 (36.6-64.9) | 52.5 (36.6-65.5) | 51.9 (36.7-64.2) | .25 |
| Sex | ||||
| Female | 6358 (52.3) | 3283 (53.1) | 3075 (51.6) | .11 |
| Male | 5789 (47.7) | 2905 (46.9) | 2884 (48.4) | |
| Comorbidities | ||||
| Hypertension | 4136 (34.0) | 2112 (34.1) | 2024 (34.0) | .86 |
| Diabetes | 1993 (16.4) | 989 (16.0) | 1004 (16.8) | .21 |
| Hyperlipidemia | 3385 (27.9) | 1722 (27.8) | 1663 (27.9) | .94 |
| Smoker | 1372 (11.3) | 641 (10.4) | 731 (12.3) | .001 |
| Coronary artery disease | 1679 (13.8) | 842 (13.6) | 837 (14.0) | .50 |
| Prior MI | 101 (0.8) | 51 (0.8) | 50 (0.8) | >.99 |
| Prior CABG | 13 (0.1) | 4 (0.1) | 9 (0.1) | .24 |
| Triage vital signs, median (IQR) | ||||
| Heart rate, beats/min | 82.0 (71.0-95.0) | 83.0 (71.0-95.0) | 82.0 (71.0-94.0) | .11 |
| Systolic blood pressure, mm Hg | 138.0 (124.0-154.0) | 139.0 (124.0-156.0) | 137.0 (123.0-153.0) | <.001 |
| Respiratory rate, breaths/min | 18.0 (16.0-18.0) | 18.0 (16.0-18.0) | 18.0 (16.0-18.0) | .24 |
| Oxygen saturation, % | 98.0 (97.0-100) | 98.0 (97.0-100) | 98.0 (97.0-100) | .20 |
| Temperature, °F | 98.0 (97.5-98.3) | 98.0 (97.6-98.4) | 97.9 (97.4-98.3) | <.001 |
| Triage acuityb | ||||
| 1 (Highest) | 114 (0.9) | 17 (0.3) | 97 (1.6) | <.001 |
| 2 | 2911 (24.0) | 1691 (27.3) | 1220 (20.5) | |
| 3 | 7349 (60.5) | 4086 (66.0) | 3263 (54.8) | |
| 4 | 1712 (14.1) | 388 (6.3) | 1324 (22.2) | |
| 5 (Lowest) | 61 (0.5) | 6 (0.1) | 55 (0.9) | |
| ED disposition | ||||
| Hospitalized | 2245 (18.5) | 1097 (17.7) | 1148 (19.3) | .05 |
| Discharged | 9504 (78.2) | 4893 (79.1) | 4611 (77.4) | |
| Left | 331 (2.7) | 168 (2.7) | 163 (2.7) | |
| Clinical outcomes | ||||
| Patients hospitalized from the ED | ||||
| Any emergency procedurec | 396 (3.3) | 197 (3.2) | 199 (3.3) | .67 |
| Cardiovascular interventiond | 322 (2.7) | 162 (2.6) | 160 (2.7) | .86 |
| Cardiac catheterization | 294 (2.4) | 142 (2.3) | 152 (2.6) | .39 |
| ICU admission | 126 (1.0) | 66 (1.1) | 60 (1.0) | .81 |
| 30-d Mortality | 62 (0.5) | 27 (0.4) | 35 (0.6) | .30 |
| Patients discharged from the ED | ||||
| 72-h Return with admission | 466 (3.8) | 221 (3.6) | 245 (4.1) | .13 |
| 72-h Return with emergency procedure | 6 (<.01) | 3 (<.01) | 3 (0.1) | >.99 |
| 30-d Mortality | 19 (0.2) | 12 (0.2) | 7 (0.1) | .40 |
Abbreviations: CABG, coronary artery bypass graft; ED, emergency department; ICU, intensive care unit; MI, myocardial infarction.
Compared using χ2 test for categorical variables and Wilcoxon rank sum test for continuous variables.
Triage acuities represent final assignments by triage nurses.
Emergency procedures were defined as any procedure or surgery performed in a dedicated suite within 12 hours of ED departure.
Cardiovascular interventions include any procedure or surgery performed on the heart or great vessels, with cardiac catheterization as a subset.
Median LOS was unchanged for discharged patients but was reduced from 657.5 (95% CI, 382.0-1413.5) to 502.0 (95% CI, 331.5-986.0) minutes for hospitalized patients. After adjustment for potential confounders, LOS was reduced by 76.4 minutes (95% CI, 28.6-124.1 minutes) for hospitalized patients and was most pronounced for emergency procedures (128.0 [95% CI, 3.9-286.0] minutes). Adjusted median time to emergency cardiovascular procedures was reduced by 205.4 minutes (95% CI, 23.0-387.8 minutes), including cardiac catheterization (by 243.2 minutes; 95% CI, 43.7-442.7 minutes) (Table 2). There were no changes in 30-day mortality or 72-hour ED returns requiring hospitalization or emergency procedures.
Table 2. Emergency Department Length of Stay and Time to Procedure Before and After Implementation of Artificial Intelligence–Informed Triage .
| Variable | Time, median (95% CI), min | ||||
|---|---|---|---|---|---|
| Total | Implementation period | Difference | |||
| Before | After | Unadjusted | Adjusteda | ||
| Length of stay b | |||||
| Discharged | 288.0 (200.0 to 415.0) | 288.0 (201.0 to 417.0) | 288.0 (200.0 to 415.0) | 0 (−8.0 to 8.0) | 3.7 (−4.2 to 11.5) |
| Hospitalized | 560.0 (353.0 to 1232.5) | 657.5 (382.0 to 1413.5) | 502.0 (331.5 to 986.0) | 155.5 (98.5 to 214.0) | 76.4 (28.6 to 124.1) |
| Any emergency procedurec | 346.0 (79.5 to 692.5) | 419.0 (120.0 to 889.0) | 291.0 (57.0 to 546.5) | 128.0 (3.9 to 286.0) | 121.5 (10.4 to 232.7) |
| Cardiovascular interventiond | 256.5 (59.2 to 526.8) | 318.5 (87.0 to 626.0) | 166.0 (49.8 to 434.0) | 152.5 (34.0 to 276.0) | 159.5 (53.0 to 265.9) |
| Cardiac catheterization | 245.5 (56.0 to 537.0) | 331.0 (77.2 to 690.2) | 148.0 (47.8 to 434.0) | 183.0 (36.5 to 325.5) | 164.9 (47.7 to 282.1) |
| Time to procedure e | |||||
| Any emergency procedurec | 480.0 (60.0 to 1020.0) | 600.0 (120.0 to 1140.0) | 420.0 (60.0 to 915.0) | 180.0 (0.0 to 480.0) | 102.3 (−57.8 to 262.5) |
| Cardiovascular interventiond | 300.0 (60.0 to 900.0) | 480.0 (60.0 to 960.0) | 180.0 (30.0 to 840.0) | 300.0 (119.3 to 480.0) | 205.4 (23.0 to 387.8) |
| Cardiac catheterization | 360.0 (60.0 to 900.0) | 480.0 (60.0 to 975.0) | 120.0 (0.0 to 840.0) | 360.0 (120.0 to 540.0) | 243.2 (42.0 to 444.5) |
Adjusted analyses were performed using median regression models that controlled for patient factors (age, sex, hospitalization status, emergency surgery status, in-hospital mortality, intensive care unit admission) and system factors (daily volume, time of day, emergency department [ED] boarding census).
Emergency department length of stay represents the interval from ED arrival to ED departure.
Emergency procedures were defined as any procedure or surgery performed in a dedicated suite within 12 hours of ED departure.
Cardiovascular interventions include any procedure or surgery performed on the heart or great vessels, with cardiac catheterization as a subset.
Time to procedure represents the interval from ED arrival to procedure start time.
Discussion
The findings show that implementation of TriageGO led to a wider distribution of patients with chest pain across ED triage levels. Alignment of triage decision-making with clinical risk allowed for safe diversion of more patients to lower acuity levels, driving timely attention to those at highest risk for adverse outcomes. This timely attention facilitated decreased ED LOS and faster transitions to inpatient care and cardiovascular interventions. Time savings associated with TriageGO were additive to any conferred by specific diagnostic approaches, which were unchanged between study periods. This study was limited by its performance within a single health system and nonrandomized study group allocation. Decision support informed by AI may improve patient stratification at triage and expedite chest pain management.
eMethods
eReferences.
Data Sharing Statement
References
- 1.Emergency department visits. National Center for Health Statistics. Published November 4, 2023. Accessed December 29, 2023. https://www.cdc.gov/nchs/fastats/emergency-department.htm
- 2.Hsia RY, Hale Z, Tabas JA. A national study of the prevalence of life-threatening diagnoses in patients with chest pain. JAMA Intern Med. 2016;176(7):1029-1032. doi: 10.1001/jamainternmed.2016.2498 [DOI] [PubMed] [Google Scholar]
- 3.Levin S, Toerper M, Hamrock E, et al. Machine-learning-based electronic triage more accurately differentiates patients with respect to clinical outcomes compared with the emergency severity index. Ann Emerg Med. 2018;71(5):565-574.e2. doi: 10.1016/j.annemergmed.2017.08.005 [DOI] [PubMed] [Google Scholar]
- 4.Levin S, Toerper M, Hinson J, et al. Machine-learning-based electronic triage: a prospective evaluation. Ann Emerg Med. 2018;72(4):S116. doi: 10.1016/j.annemergmed.2018.08.299 [DOI] [PubMed] [Google Scholar]
- 5.Gilboy N, Tanabe P, Travers D, Rosenau AM, eds. Emergency Severity Index (ESI): A Triage Tool for Emergency Department Care. 4th ed. Agency for Healthcare Research and Quality; 2012. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eMethods
eReferences.
Data Sharing Statement