Abstract
Aims
An artificial intelligence-augmented electrocardiogram (AI-ECG) algorithm can identify left ventricular systolic dysfunction (LVSD). We sought to determine whether this AI-ECG algorithm could stratify mortality risk in cardiac intensive care unit (CICU) patients, independent of the presence of LVSD by transthoracic echocardiography (TTE).
Methods and results
We included 11 266 unique Mayo Clinic CICU patients admitted from 2007 to 2018 who underwent AI-ECG after CICU admission. Left ventricular ejection fraction (LVEF) data were extracted for patients with a TTE during hospitalization. Hospital mortality was analysed using multivariable logistic regression. Mean age was 68 ± 15 years, including 37% females. Higher AI-ECG probability of LVSD remained associated with higher hospital mortality [adjusted odds ratio (OR) 1.05 per 0.1 higher, 95% confidence interval (CI) 1.02–1.08, P = 0.003] after adjustment for LVEF, which itself was inversely related with the risk of hospital mortality (adjusted OR 0.96 per 5% higher, 95% CI 0.93–0.99, P = 0.02). Patients with available LVEF data (n = 8242) were divided based on the presence of predicted (by AI-ECG) vs. observed (by TTE) LVSD (defined as LVEF ≤ 35%), using TTE as the gold standard. A stepwise increase in hospital mortality was observed for patients with a true negative, false positive, false negative, and true positive AI-ECG.
Conclusion
The AI-ECG prediction of LVSD is associated with hospital mortality in CICU patients, affording risk stratification in addition to that provided by echocardiographic LVEF. Our results emphasize the prognostic value of electrocardiographic patterns reflecting underlying myocardial disease that are recognized by the AI-ECG.
Keywords: Artificial intelligence, Cardiac intensive care unit, Echocardiography, Electrocardiogram, Left ventricular dysfunction, Mortality
Introduction
The degree of medical complexity and prevalence of critical care diagnoses are increasing in the cardiac intensive care unit (CICU) population over time.1,2 Intensive care unit (ICU) severity of illness scores have very good discrimination for hospital mortality in CICU cohorts but lack optimal calibration.3–7 Clinical measurements intended for other uses have been repurposed to predict mortality in CICU patients. The Braden Skin Score, which was developed to predict pressure injuries in hospitalized patients, is a potent predictor of mortality in CICU patients and is included in a novel CICU-specific mortality risk prediction score.8,9
Currently, available risk stratification algorithms do not integrate markers of cardiac function, which could further refine risk stratification in CICU patients.3–6,9,10 Left ventricular systolic dysfunction (LVSD), defined as a reduced left ventricular ejection fraction (LVEF) on transthoracic echocardiography (TTE), is a major determinant of outcomes in patients with cardiovascular disease.11 Unexpectedly, LVSD has not been consistently associated with outcomes in all CICU patient subgroups.12–15 This highlights the limitations of LVEF as a definitive assessment of myocardial function.15,16
Artificial intelligence-augmented electrocardiogram (AI-ECG) algorithms can recognize patterns characteristic of underlying myocardial disease using a standard, 10-s, 12-lead electrocardiogram (ECG).17–20 One novel AI-ECG algorithm provided excellent discrimination for LVSD with an overall accuracy of >85% in more than 100 000 total patients.17–19 Insofar as the AI-ECG identifies ECG correlates of underlying myocardial disease, AI-ECG algorithms can identify underlying subtle patterns associated with mortality risk.21 Although the AI-ECG algorithm retained very good discrimination for LVSD in a CICU population with a high prevalence of LVSD, the association between AI-ECG parameters and outcomes has not been described for CICU patients. The aim of this study was to evaluate the ability of the AI-ECG algorithm to predict mortality in CICU patients, and to determine if this mortality prediction was affected by the presence of LVSD on TTE.
Methods
Study population
This retrospective database study was approved by the Institutional Review Board of Mayo Clinic Rochester under a waiver of informed consent as minimal risk to patients. Consecutive unique adults admitted to the CICU at Mayo Clinic Rochester from 1 January 2007 to 30 April 2018 were included in the database if they had not previously declined consent for their medical records to be used for research.1,9 The study population included patients with an ECG performed after CICU admission and during hospitalization.
Data sources
Demographic, clinical, vital sign, laboratory, outcome, and diagnosis data were extracted from the electronic medical record using the Multidisciplinary Epidemiology and Translational Research in Intensive Care (METRIC) Data Mart, along with data on critical care procedures and therapies. Admission diagnoses were defined as all ICD-9/-10 diagnosis codes recorded within 1 day before or after CICU admission.7,9 The Charlson comorbidity index, individual comorbidities, and severity of illness scores, including the Sequential Organ Failure Assessment and APACHE-III and IV scores, were extracted from the electronic medical record using previously validated algorithms.3,4,6,7
The Mayo Clinic Cardiovascular Data Mart was queried electronically for the TTE closest to CICU admission, and available LVEF data from this TTE were included if it was performed during or within 1 day before or after hospitalization. The LVEF value was determined hierarchically: calculated Simpson’s biplane method was used preferentially; if this was not available, then other calculated LVEF methods were used; and finally, if LVEF could not be calculated, then visual LVEF estimation was used.10,12 The severity of LVSD was defined according to American Society of Echocardiography (ASE) guidelines: mild LVSD (LVEF 41–51% for males and 41–53% for females), moderate LVSD (LVEF 30–40%), and severe LVSD (LVEF < 30%).16
AI-ECG algorithm
The novel proprietary AI-ECG algorithm for detection of LVSD used in this analysis was derived and validated in nearly 100 000 patients from the Mayo Clinic with a paired ECG and echocardiogram.17–19 A deep convolutional neural network was trained to identify LVEF ≤35% by echocardiography using digitized raw data from a standard 10-s, 12-lead ECGs sampled at 500 Hz from ∼36 000 patients using the GE-Marquette (Marquette, WI, USA) platform.17,18 The AI-ECG algorithm used a neural network to transform and integrate raw ECG data using 2-s segments with 1-s overlap from each individual ECG lead to produce a single output variable. The output of this AI-ECG algorithm provides a probability (between 0 and 1) that LVSD (i.e. LVEF ≤35% by echocardiography) is present, without providing data about which ECG features contributed to this probability.17,18 The AI-ECG data were obtained for the first ECG performed after CICU admission electronically, without manual ECG review.
AI-ECG and TTE LVSD groups
Among patients with available LVEF data from TTE, the optimal AI-ECG cut-off for LVSD was used to classify patients based on the presence of observed LVEF ≤35% by TTE as the ‘gold standard’. If the AI-ECG probability of LVSD was below the cut-off, a true negative (TN) AI-ECG was defined as LVEF >35% by TTE and false negative (FN) was defined as LVEF ≤35% by TTE. If the AI-ECG probability of LVSD was above the cut-off, a true positive (TP) AI-ECG was defined as LVEF ≤35% by TTE and false positive (FP) was defined as LVEF >35% by TTE (Supplementary material online, Figure S1).
Statistical analysis
All-cause CICU, hospital, 1-year mortality were determined using electronic review of medical records for notification of patient death and last follow-up date. Mortality data were extracted from Mayo Clinic electronic databases, the state of Minnesota electronic death certificates, and the Rochester Epidemiology Project database. Data are reported as number (percent) for categorical variables and mean ± standard deviation for continuous variables. Patients were divided based on quintiles of the AI-ECG predicted probability of LVSD. Groups were compared using the Pearson χ2 test for categorical variables and the Wilcoxon rank-sum test for continuous variables. To assess discrimination, receiver-operator characteristic (ROC) curves were generated using logistic regression, and the area under the ROC curve (AUC) value was determined with 95% confidence intervals (CIs) via 1000-sample bootstrapping. The optimal cut-off was defined as the highest value of Youden’s J index = (sensitivity + specificity) – 1 on ROC analysis. Odds ratio (OR) and 95% CI values for hospital mortality were determined using logistic regression before and after adjustment for demographics, comorbidities, severity of illness scores, and CICU procedures and therapies (Supplementary material online, Table S1). One-year mortality was assessed using Kaplan–Meier survival analysis, and groups were compared by the log-rank test. Hazard ratio (HR) values for 1-year mortality were determined using Cox proportional-hazards analysis before and after adjustment for these same variables. Separate logistic regression and Cox proportional-hazards models were constructed including LVEF in the subgroup of patients with available TTE data. Two-tailed P-values <0.05 were considered significant. Statistical analysis was performed using JMP version 14.0 Pro (SAS Institute, Cary, NC, USA).
Results
Study population
Using a pre-existing CICU database of 12 428 unique patients (Supplementary material online, Figure S1), we excluded 1162 patients: 424 without an ECG after CICU admission and 738 whose first ECG after CICU admission was not during hospitalization. The final study population of 11 266 patients had a mean age of 67.6 ± 15.0 years and included 37.3% females (Table 1). Admission diagnoses included: acute coronary syndrome (ACS), 45.2%; heart failure (HF), 48.3%; cardiogenic shock, 12.5%; and cardiac arrest, 12.2% (Table 1).
Table 1.
Baseline characteristics of the final study population, hospital survivors, and inpatient deaths
| Variables | Final study population (n = 11 266) | Inpatient deaths (n = 979) | Hospital survivors (n = 10 287) | P-value |
|---|---|---|---|---|
| Demographics | ||||
| Age (years) | 67.6 ± 15.0 | 71.6 ± 13.7 | 67.2 ± 15.1 | <0.001 |
| Female, n (%) | 4202 (37.3%) | 393 (40.1%) | 3809 (37.0%) | 0.05 |
| Caucasian, n (%) | 10 410 (92.6%) | 881 (90.0%) | 9529 (92.6%) | 0.003 |
| CICU length of stay (days) | 2.6 ± 4.4 | 3.4 ± 4.3 | 2.5 ± 4.5 | <0.001 |
| Hospital length of stay (days) | 8.2 ± 13.8 | 9.6 ± 19.7 | 8.1 ± 13.1 | <0.001 |
| CICU mortality | 589 (5.2%) | 589 (60.2%) | 0 (0%) | NA |
| Hospital mortality | 979 (8.7%) | 979 (100%) | 0 (0%) | NA |
| One-year mortality | 2459 (21.8%) | 979 (100%) | 1480 (14.4%) | NA |
| Comorbidities | ||||
| Charlson comorbidity index | 2.3 ± 2.6 | 3.1 ± 2.9 | 2.3 ± 2.6 | <0.001 |
| Prior myocardial infarction | 2085 (18.5%) | 194 (19.8%) | 1891 (18.4%) | 0.28 |
| Prior heart failure | 2146 (19.1%) | 259 (26.5%) | 1887 (18.4%) | <0.001 |
| Prior stroke | 1324 (11.8%) | 167 (17.1%) | 1157 (11.3%) | <0.001 |
| Prior diabetes mellitus | 3199 (28.4%) | 328 (33.5%) | 2871 (28.0%) | <0.001 |
| Prior lung disease | 2129 (18.9%) | 244 (24.9%) | 1885 (18.4%) | <0.001 |
| Prior chronic kidney disease | 2248 (20.0%) | 279 (28.5%) | 1969 (19.2%) | <0.001 |
| Prior dialysis | 564 (5.0%) | 103 (10.5%) | 461 (4.5%) | <0.001 |
| Admission diagnosesa | ||||
| Acute coronary syndrome | 5056 (45.2%) | 420 (42.9%) | 4636 (45.5%) | 0.13 |
| Heart failure | 5398 (48.3%) | 643 (65.8%) | 4755 (46.6%) | <0.001 |
| Shock | 1724 (15.4%) | 525 (53.7%) | 1199 (11.8%) | <0.001 |
| Cardiogenic shock | 1395 (12.5%) | 424 (43.4%) | 971 (9.5%) | <0.001 |
| Cardiac arrest | 1357 (12.2%) | 432 (44.2%) | 925 (9.1%) | <0.001 |
| Ventricular fibrillation arrest | 710 (6.4%) | 173 (17.7%) | 537 (5.3%) | <0.001 |
| Respiratory failure | 2723 (24.4%) | 631 (64.5%) | 2092 (20.5%) | <0.001 |
| Sepsis | 730 (6.5%) | 218 (22.3%) | 512 (5.0%) | <0.001 |
| Severity of illness scores | ||||
| Day 1 SOFA score | 3.5 ± 3.2 | 7.7 ± 4.2 | 3.1 ± 2.8 | <0.001 |
| APACHE-III score | 61.0 ± 25.1 | 93.0 ± 33.4 | 57.9 ± 21.8 | <0.001 |
| APACHE-IV predicted death (%) | 17.0 ± 19.9 | 44.1 ± 28.9 | 14.4 ± 16.6 | <0.001 |
| Admission Braden Skin Score | 17.7 ± 3.4 | 14.1 ± 3.6 | 18.0 ± 3.2 | <0.001 |
| Procedures and therapies | ||||
| Inpatient coronary angiogram | 6807 (60.4%) | 443 (45.2%) | 6364 (61.9%) | <0.001 |
| Inpatient PCI | 4148 (36.8%) | 228 (23.3%) | 3920 (38.1%) | <0.001 |
| IABP in CICU | 1024 (9.1%) | 163 (16.6%) | 861 (8.4%) | <0.001 |
| Pulmonary artery catheter | 1047 (9.3%) | 162 (16.6%) | 885 (8.6%) | <0.001 |
| Red blood cell transfusion | 1306 (11.6%) | 228 (23.3%) | 1078 (10.5%) | <0.001 |
| Dialysis in CICU | 545 (4.8%) | 175 (17.9%) | 370 (3.6%) | <0.001 |
| CRRT | 229 (2.0%) | 109 (11.1%) | 120 (1.2%) | <0.001 |
| Non-invasive ventilator | 1747 (15.5%) | 232 (23.7%) | 1515 (14.7%) | <0.001 |
| Invasive ventilator | 1913 (17.0%) | 538 (55.0%) | 1375 (13.4%) | <0.001 |
|
Vasoactive drugs # vasoactive drugs |
2830 (25.1%) 0.5 ± 1.0 |
640 (65.4%) 1.5 ± 1.5 |
2190 (21.3%) 0.4 ± 0.8 |
<0.001 <0.001 |
| In-hospital cardiac arrest | 284 (2.5%) | 127 (13.0%) | 157 (1.5%) | <0.001 |
| Echocardiographic and AI-ECG findings | ||||
| Inpatient TTE | 9582 (87.2%) | 779 (85.6%) | 8803 (87.3%) | 0.14 |
| TTE within 1 day of CICU admission | 6963 (61.8%) | 574 (58.6%) | 6389 (62.1%) | 0.03 |
| LVEF (%)a | 47.3 ± 16.5 | 40.5 ± 18.5 | 47.9 ± 16.1 | <0.001 |
| Normal LVEF for sexa | 3843 (46.6%) | 219 (31.3%) | 3624 (48.0%) | <0.001 |
| Mild LVSDa | 1514 (18.4%) | 101 (14.4%) | 1413 (18.7%) | |
| Moderate LVSDa | 1470 (17.8%) | 137 (19.6%) | 1333 (17.7%) | |
| Severe LVSDa | 1415 (17.2%) | 242 (24.6%) | 1173 (15.6%) | |
| LVEF ≤35% by TTEa | 2277 (27.6%) | 326 (46.6%) | 1951 (25.9%) | <0.001 |
| AI-ECG probability of LVSD | 0.352 ± 0.373 | 0.490 ± 0.378 | 0.339 ± 0.370 | <0.001 |
| Predicted LVEF ≤35% | 4257 (37.8%) | 529 (54.0%) | 3728 (36.2%) | <0.001 |
| True negativea | 4540 (55.1%) | 240 (34.3%) | 4300 (57.0%) | <0.001 |
| False positivea | 1425 (17.3%) | 133 (19.0%) | 1292 (17.1%) | |
| False negativea | 567 (6.9%) | 67 (9.6%) | 500 (6.6%) | |
| True positivea | 1710 (20.8%) | 259 (37.0%) | 1451 (19.2%) |
Data displayed as n (%) for categorical variables or mean ± standard deviation for continuous variables. Reported P-value is for between-groups comparison using Pearson χ2 test (categorical variables) or Wilcoxon rank-sum test (continuous variables) comparing hospital survivors and inpatient deaths.
ACS, acute coronary syndrome; AI-ECG, artificial intelligence-enhanced electrocardiogram; APACHE, Acute Physiology and Chronic Health Evaluation; CICU, cardiac intensive care unit; CRRT, continuous renal replacement therapy; ECG, electrocardiogram; IABP, intra-aortic balloon pump; LVEF, left ventricular ejection fraction; LVSD, left ventricular systolic dysfunction; PCI, percutaneous coronary intervention; SOFA, Sequential Organ Failure Assessment; TTE, transthoracic echocardiogram.
Admission diagnoses were not mutually-exclusive and may sum to greater than 100%.
AI-ECG and LVSD by TTE
The AI-ECG was performed during the CICU stay in 92.3% of patients, including the day of CICU admission in 77.9% of patients. The mean probability of LVSD by AI-ECG was 0.352 ± 0.373. LVEF data from TTE were available in 8242 patients (73.2% of the study population), and the mean LVEF was 47.3 ± 16.5%; LVEF was measured by the biplane method in 2095 (25.4%) patients. The AI-ECG and TTE were separated by a mean of 0.8 ± 5.2 days, and 54.2% of patients had the TTE and AI-ECG on the same day. LVSD by ASE criteria was present in 53.3% of patients: mild LVSD, 18.3%; moderate LVSD, 17.8%; severe LVSD, 17.2%. The AI-ECG had an AUC of 0.83 (95% CI 0.82–0.84) for predicting LVEF ≤35% by TTE. At the optimal cut-off of 0.389, the AI-ECG had a sensitivity and specificity of 75.1% and 76.1% for LVEF ≤35% by TTE, respectively; overall accuracy was 75.8% with a positive and negative predictive value of 54.5% and 88.9%, respectively. The AI-ECG predicted LVEF ≤35% in 37.8% of patients, and 27.6% of patients had observed LVEF ≤35% by TTE (Table 1). Based on predicted vs. observed LVEF ≤35% by TTE, patients were classified as TP, 20.7%; TN, 55.1%; FP, 17.3%; and FN, 6.9% (Supplementary material online, Figure S1 and Table 2). There were substantial differences in baseline characteristics across these predicted versus observed LVEF ≤35% groups (Table 2).
Table 2.
Baseline characteristics of the 8242 patients with LVEF data from TTE based on the concordance or discordance between AI-ECG and TTE for LVEF ≤35%
| Variables | True negative (n = 4540) | False positive (n = 1425) | False negative (n = 567) | True positive (n = 1710) | P-value |
|---|---|---|---|---|---|
| Demographics | |||||
| Age (years) | 66.9 ± 15.0 | 71.1 ± 13.6 | 68.0 ± 15.2 | 68.0 ± 13.9 | <0.001 |
| Female, n (%) | 1780 (39.2%) | 546 (38.3%) | 250 (44.1%) | 441 (25.8%) | <0.001 |
| Caucasian, n (%) | 4211 (92.8%) | 1333 (93.5%) | 521 (91.9%) | 1552 (90.8%) | 0.02 |
| CICU length of stay (days) | 2.3 ± 4.0 | 3.0 ± 7.0 | 3.2 ± 3.1 | 3.7 ± 5.5 | <0.001 |
| Hospital length of stay (days) | 6.9 ± 10.9 | 9.8 ± 20.2 | 9.7 ± 10.7 | 12.4 ± 16.5 | <0.001 |
| CICU mortality | 131 (2.9%) | 78 (5.5%) | 39 (6.9%) | 152 (8.9%) | <0.001 |
| Hospital mortality | 240 (5.3%) | 133 (9.3%) | 67 (11.8%) | 259 (15.2%) | <0.001 |
| One-year mortality | 693 (15.3%) | 371 (26.0%) | 152 (26.8%) | 597 (34.9%) | <0.001 |
| Comorbidities | |||||
| Charlson comorbidity index | 1.9 ± 2.4 | 2.7 ± 2.7 | 2.2 ± 2.5 | 2.9 ± 2.7 | <0.001 |
| Prior myocardial infarction | 615 (13.6%) | 308 (21.7%) | 98 (17.3%) | 489 (28.6%) | <0.001 |
| Prior heart failure | 440 (9.7%) | 349 (24.6%) | 86 (15.2%) | 597 (35.0%) | <0.001 |
| Prior stroke | 423 (9.3%) | 191 (13.5%) | 69 (12.2%) | 246 (14.4%) | <0.001 |
| Prior diabetes mellitus | 1127 (24.9%) | 482 (34.0%) | 155 (27.4%) | 597 (35.0%) | <0.001 |
| Prior lung disease | 774 (17.1%) | 26 (18.8%) | 122 (21.6%) | 333 (19.5%) | <0.001 |
| Prior chronic kidney disease | 645 (14.2%) | 355 (25.0%) | 108 (19.1%) | 459 (26.9%) | <0.001 |
| Prior dialysis | 159 (3.5%) | 101 (7.1%) | 31 (5.5%) | 119 (7.0%) | <0.001 |
| Admission diagnosesa | |||||
| Acute coronary syndrome | 2534 (56.4%) | 716 (50.5%) | 347 (61.5%) | 745 (43.8%) | <0.001 |
| Heart failure | 1379 (30.7%) | 819 (57.7%) | 425 (75.6%) | 1504 (88.3%) | <0.001 |
| Shock | 466 (10.4%) | 243 (17.1%) | 186 (33.1%) | 512 (30.1%) | <0.001 |
| Cardiogenic shock | 339 (7.5%) | 191 (13.5%) | 163 (29.0%) | 464 (27.2%) | <0.001 |
| Cardiac arrest | 426 (9.5%) | 182 (12.8%) | 117 (20.8%) | 300 (17.6%) | <0.001 |
| VF arrest | 211 (4.7%) | 87 (6.1%) | 78 (13.9%) | 174 (10.2%) | <0.001 |
| Respiratory failure | 902 (20.1%) | 453 (31.9%) | 214 (38.1%) | 628 (36.9%) | <0.001 |
| Sepsis | 252 (5.6%) | 115 (8.1%) | 62 (11.0%) | 164 (9.6%) | <0.001 |
| Severity of illness scores | |||||
| Day 1 SOFA score | 2.9 ± 2.9 | 4.0 ± 3.4 | 4.6 ± 3.7 | 4.7 ± 3.5 | <0.001 |
| APACHE-III score | 56.2 ± 22.9 | 66.1 ± 25.4 | 69.1 ± 28.1 | 69.1 ± 25.8 | <0.001 |
| APACHE-IV predicted death (%) | 13.8 ± 17.3 | 20.8 ± 21.7 | 22.9 ± 23.2 | 23.2 ± 22.7 | <0.001 |
| Admission Braden Score | 18.1 ± 3.2 | 17.1 ± 3.4 | 16.4 ± 3.6 | 16.9 ± 3.5 | <0.001 |
| Procedures and therapies | |||||
| Inpatient coronary angiogram | 2947 (64.9%) | 852 (59.8%) | 405 (71.4%) | 1097 (64.2%) | <0.001 |
| Inpatient PCI | 2072 (45.6%) | 532 (37.3%) | 234 (41.3%) | 503 (29.4%) | <0.001 |
| IABP in CICU | 283 (6.2%) | 133 (9.3%) | 105 (18.5%) | 338 (19.8%) | <0.001 |
| Pulmonary artery catheter | 239 (5.3%) | 118 (8.3%) | 82 (14.5%) | 345 (20.2%) | <0.001 |
| Red blood cell transfusion | 461 (10.2%) | 216 (15.2%) | 102 (18.0%) | 228 (13.3%) | <0.001 |
| Dialysis in CICU | 136 (3.0%) | 96 (6.7%) | 35 (6.2%) | 156 (9.1%) | <0.001 |
| CRRT | 64 (1.4%) | 40 (2.8%) | 17 (3.0%) | 73 (4.3%) | <0.001 |
| Non-invasive ventilator | 588 (13.0%) | 276 (19.4%) | 108 (19.0%) | 376 (22.0%) | <0.001 |
| Invasive ventilator | 565 (12.4%) | 294 (20.6%) | 175 (30.9%) | 455 (26.6%) | <0.001 |
| Vasoactive drugs | 761 (16.8%) | 372 (26.1%) | 209 (36.9%) | 768 (44.9%) | <0.001 |
| # vasoactive drugs | 0.3 ± 0.8 | 0.5 ± 1.0 | 0.7 ± 1.2 | 0.9 ± 1.2 | <0.001 |
| In-hospital cardiac arrest | 72 (1.6%) | 41 (2.9%) | 26 (4.6%) | 64 (3.8%) | <0.001 |
| Echocardiographic and AI-ECG findings | |||||
| TTE within 1 day of CICU admission | 3549 (78.2%) | 1069 (75.0%) | 482 (85.0%) | 1277 (74.7%) | <0.001 |
| AI-ECG on day of CICU admission | 3688 (81.2%) | 1131 (79.4%) | 454 (80.1%) | 1310 (76.6%) | <0.001 |
| TTE and AI-ECG same day | 1861 (41.0%) | 572 (40.1%) | 319 (56.3%) | 727 (42.5%) | <0.001 |
| LVEF (%) | 57.3 ± 9.3 | 50.5 ± 9.8 | 28.4 ± 6.4 | 24.1 ± 7.4 | <0.001 |
| AI-ECG probability LVSD | 0.076 ± 0.093 | 0.726 ± 0.189 | 0.135 ± 0.110 | 0.850 ± 0.168 | <0.001 |
Data displayed as n (%) for categorical variables or mean ± standard deviation for continuous variables. Reported P-value is for between-groups comparison using Pearson χ2 test (categorical variables) or Wilcoxon rank-sum test (continuous variables) across groups.
ACS, acute coronary syndrome; AI-ECG, artificial intelligence-enhanced electrocardiogram; APACHE, Acute Physiology and Chronic Health Evaluation; CICU, cardiac intensive care unit; CRRT, continuous renal replacement therapy; ECG, electrocardiogram; IABP, intra-aortic balloon pump; LVEF, left ventricular ejection fraction; LVSD, left ventricular systolic dysfunction; PCI, percutaneous coronary intervention; SOFA, Sequential Organ Failure Assessment; TTE, transthoracic echocardiogram; VF, ventricular fibrillation. aAdmission diagnoses were not mutually-exclusive and may sum to greater than 100%.
Hospital mortality
A total of 979 (8.7%) patients died in the hospital, including the 589 (5.2%) that died during the CICU stay. Hospital survivors differed substantially from inpatient deaths (Table 1). Inpatient deaths had a higher AI-ECG probability of LVSD (0.490 ± 0.378 vs. 0.339 ± 0.370, P < 0.001) and a lower LVEF (40.4 ± 18.5% vs. 47.9 ± 16.1%, P < 0.001). LVEF was inversely associated with hospital mortality (unadjusted OR 0.88 per 5% higher, 95% CI 0.86–0.90, P < 0.001; AUC 0.62, 95% CI 0.59–0.64; optimal cut-off 40%). The AI-ECG probability of LVSD was directly associated with hospital mortality (unadjusted OR 1.11 per 0.1 higher, 95% CI 1.09–1.13, P < 0.001; AUC 0.63, 95% CI 0.61–0.65; optimal cut-off 0.075). Addition of the AI-ECG probability of LVSD to the LVEF increased the AUC for discrimination of hospital mortality (AUC 0.64 vs. 0.60, P < 0.001 by De Long test). Addition of the AI-ECG probability of LVSD to the APACHE-III score modestly increased the AUC value for discrimination of hospital mortality (AUC 0.83 vs. 0.82, P < 0.001 by De Long test). CICU and hospital mortality increased with the severity of LVSD (Figure 1A) and with increasing AI-ECG probability of LVSD (Figure 1B) or higher AI-ECG probability of LVSD quintile (Supplementary material online, Figure S2). Among patients with ACS, both AI-ECG probability of LVSD (unadjusted OR 1.07 per 0.1 higher, 95% CI 1.04–1.11, P < 0.001) and LVEF by TTE (unadjusted OR 0.82 per 5% higher, 95% CI 0.78–0.86, P < 0.001) were both associated with hospital mortality; associations were not significant among patients with HF (P = 0.05 for LVEF by TTE and P = 0.09 for AI-ECG probability of LVSD).
Figure 1.
CICU and hospital mortality as a function of LVSD based on current ASE guidelines16 (A) and the AI-ECG probability of LVSD (B). P < 0.001 for trends across categories. AI-ECG, artificial intelligence-augmented electrocardiogram; ASE, American Society of Echocardiography; CICU, cardiac intensive care unit; ECG, electrocardiogram; LVSD, left ventricular systolic dysfunction; TTE, transthoracic echocardiogram.
Hospital mortality was higher in patients with either LVEF ≤35% predicted by AI-ECG (12.4% vs. 6.4%, P < 0.001) or LVEF ≤35% observed by TTE (14.3% vs. 6.2%, P < 0.001). CICU and hospital mortality varied based on the presence of predicted (by AI-ECG) or observed (by TTE) LVEF ≤35% (Figure 2). Patients with a TN AI-ECG had the lowest hospital mortality (all P < 0.001). Patients with a TP AI-ECG had higher hospital mortality than patients with an FP AI-ECG (P < 0.001) and similar hospital mortality to patients with an FN AI-ECG (P = 0.05); patients with an FP or FN AI-ECG had similar hospital mortality (P = 0.10). The AI-ECG probability of LVSD was incrementally associated with hospital mortality (Figure 3) among patients with normal LVEF (unadjusted OR 1.13 per 10% higher, 95% CI 1.08–1.18, P < 0.001) or mild LVSD (unadjusted OR 1.08 per 10% higher, 95% CI 1.02–1.15, P = 0.005), but not in patients with moderate or severe LVSD (P > 0.1).
Figure 2.
CICU and hospital mortality based on predicted (by AI-ECG) and observed (by TTE) LVEF ≤35%. *P < 0.001 compared with all other groups. #P < 0.001 compared with false positive AI-ECG and P = 0.05 compared with false-negative ECG. P = 0.10 for comparison of false-positive and false-negative AI-ECG. AI-ECG, artificial intelligence-augmented electrocardiogram; CICU, cardiac intensive care unit; ECG, electrocardiogram; LVEF, left ventricular ejection fraction; TTE, transthoracic echocardiogram.
Figure 3.
Heat map demonstrating hospital survival (A) and 1-year survival (B) as a function of LVSD on TTE based on current ASE guidelines16 (Y axis) and AI-ECG probability of LVSD quintile (X axis). Darker colours represent a higher risk of hospital death. AI-ECG, artificial intelligence-augmented electrocardiogram; LVSD, left ventricular systolic dysfunction; TTE, transthoracic echocardiogram.
After multivariable adjustment, the AI-ECG probability of LVSD remained directly associated with hospital mortality (adjusted OR 1.05 per 0.1 higher, 95% CI 1.03–1.08, P < 0.001). This persisted after adjustment for LVEF (adjusted OR 1.05 per 0.1 higher, 95% CI 1.02–1.08, P = 0.003); LVEF remained inversely associated with hospital mortality (adjusted OR 0.96 per 5% higher, 95% CI 0.93–0.99, P = 0.02) (Supplementary material online, Table S1). After multivariable adjustment, patients with a TP AI-ECG had higher hospital mortality than patients with either an FN AI-ECG (adjusted OR 1.79, 95% CI 1.24–2.59, P = 0.002) or FP AI-ECG (adjusted OR 1.56, 95% CI 1.18–2.06, P = 0.002), whereas patients with an FP AI-ECG had similar adjusted hospital mortality to patients with an FN AI-ECG (P = 0.49) or TN AI-ECG (P = 0.25).
One-year mortality
A total of 2459 (21.8%) patients died within 1 year after CICU admission (including hospital deaths), and 1244 (11.0%) had a follow-up duration of less than 1 year but were alive at last follow-up. One-year survival was progressively lower as a function of increasing AI-ECG probability of LVSD quintile (Supplementary material online, Figure S3A; P < 0.001 by log-rank). One-year survival was lower in patients with moderate or severe LVSD by TTE (Supplementary material online, Figure S3B; P < 0.001 by log-rank), although patients with normal LVEF and mild LVSD had similar 1-year mortality (P = 0.26). Patients with either observed LVEF ≤35% by TTE or predicted ≤35% by AI-ECG had higher 1-year mortality (P < 0.001 by log-rank). The association between AI-ECG predicted probability of LVSD and 1-year mortality was greater among patients with mild or no LVSD by TTE (Supplementary material online, Figure S4). One-year mortality varied based on the presence of predicted (by AI-ECG) or observed (by TTE) LVEF ≤35% (Figure 4). Patients with TN AI-ECG had the lowest 1-year mortality, and patients with TP AI-ECG had the highest 1-year mortality, while patients with either FN or FP AI-ECG had similar 1-year mortality (P = 0.48); the results did not change when the analysis was limited to hospital survivors.
Figure 4.
Kaplan–Meier 1-year survival curves based on predicted (by AI-ECG) and observed (by TTE) LVEF ≤35% group. P < 0.001 between either true negative or true positive and all other groups and P = 0.48 for false positive vs. false negative by log-rank. AI-ECG, artificial intelligence-augmented electrocardiogram; ECG, electrocardiogram; LVEF, left ventricular ejection fraction; TTE, transthoracic echocardiogram.
After multivariable adjustment, AI-ECG probability of LVSD remained associated with higher 1-year mortality (adjusted HR 1.04 per 0.1 higher, 95% CI 1.03–1.05, P < 0.001), even after adjusting for LVEF in patients with available data (adjusted HR 1.03 per 0.1 higher, 95% CI 1.01–1.04, P = 0.003); LVEF remained inversely associated with 1-year mortality (adjusted HR 0.97 per 5% higher, 95% CI 0.95–0.99, P < 0.001). After multivariable adjustment, patients with a TP AI-ECG had higher 1-year mortality than patients with either an FN AI-ECG (adjusted HR 1.29, 95% CI 1.07–1.55, P = 0.008) or FP AI-ECG (adjusted HR 1.35, 95% CI 1.18–1.55, P < 0.001), whereas patients with an FP AI-ECG had similar mortality to patients with an FN AI-ECG (P = 0.62) or TN AI-ECG (P = 0.13).
Discussion
In this analysis of more than 11 000 CICU patients, we demonstrate that an AI-ECG algorithm designed to identify LVSD can also identify patients with an increased risk of dying during and after hospitalization. While the AI-ECG algorithm only identified echocardiographic LVSD with moderate accuracy, the mortality association of the AI-ECG probability of LVSD extended beyond what could be explained by reduced LVEF alone. Indeed, the AI-ECG prediction of LVSD had a stronger association with mortality in patients without significant LVSD, highlighting the prognostic importance of subclinical myocardial disease. Even after adjustment for relevant covariates and LVEF, a higher AI-ECG probability of LVSD remained associated with an increased risk of hospital and 1-year mortality. Our findings in patients with concordant versus discordant AI-ECG and TTE suggest that the ECG patterns recognized by the AI-ECG algorithm that can predict LVSD are reflective of underlying myocardial disease with prognostic relevance even in a critically-ill CICU cohort. This suggests that a myopathic process detected by the AI-ECG may be impacting cardiac electrical activity and outcomes prior to the development of overt mechanical dysfunction identified by imaging.17–19 The AI-ECG may complement TTE for mortality risk stratification by evaluating components of myocardial electrical functioning that are not readily assessed (particularly for patients with mild or no LVSD).
A recent study by Raghunath et al.21 showed that an AI-ECG algorithm could be trained to predict death during follow-up in nearly 1.8 million unselected patients. As a model designed to predict mortality, their AI-ECG algorithm had substantially higher discrimination for 1-year mortality (AUC 0.85) than we observed using our AI-ECG model, even among patients whose ECG was interpreted as ‘normal’ by a cardiologist.21 The AI-ECG can identify prognostically relevant ECG findings that may not be discernable to the human eye. Deriving and validating an AI-ECG model specifically for prediction of mortality might improve the mortality prediction performance in the CICU. A substantial limitation of most AI-ECG algorithms (including the one evaluated in this study) is the focus only on the ECG itself, without integrating other clinically relevant patient-level data that could improve prediction and risk-stratification; future iterations of AI-ECG algorithms ideally would include clinical information. While our study was built to further validate the utility and reproducibility of this ECG AI algorithm amongst CICU patients, as algorithms such as these reach routine clinical implementation, following standards being developed by consensus bodies will be important to ensure consistency and scientific reliability.22,23
Discrimination of hospital mortality by the AI-ECG probability of LVSD remained modest (AUC 0.64) albeit slightly superior to LVEF by TTE; neither of these measures alone is ideal for prediction of hospital mortality in CICU patients. More sophisticated TTE modalities including Doppler and strain imaging can improve mortality risk-stratification in critically-ill patients beyond standard 2D TTE measures such as LVEF, and it will be necessary for future studies to demonstrate additive prognostic value of the AI-ECG beyond of these advanced imaging techniques.10,13,14,24 The AI-ECG is expected to be less sensitive to image quality, which can preclude use of advanced TTE imaging modalities in some critically-ill patients.
AI algorithms can predict death among hospitalized patients, including ICU patients and patients with HF by identifying patterns of vital sign and laboratory abnormalities.25,26 There is precedent for the use of prolonged ECG monitoring data to predict mortality in critically-ill patients.27 Our AI-ECG algorithm utilizes a standard 12-lead ECG without the need for prolonged monitoring, leveraging a ubiquitous clinical test for mortality risk stratification without the need for additional cost or personnel time. The AI-ECG should be thought of as a complementary modality to TTE, rather than a replacement.
Our AI-ECG algorithm was originally designed to identify patients with significant LVSD (defined as LVEF ≤35%) based on subtle and prognostically relevant ECG changes caused by underlying myocardial disease.17–20 We observed an incremental association between the AI-ECG predicted probability of LVSD and mortality beyond that conferred by LVEF, which is one of the best-established markers of mortality risk among patients with cardiovascular disease.11 We believe that the independent and additive associations of LVEF by TTE and AI-ECG probability of LVSD with mortality reflect their complementary abilities to characterize distinct aspects of myocardial disease that are associated with mortality.12,13,15,16 The overall accuracy of the AI-ECG for identifying LVSD was modest, with only 76% diagnostic accuracy for LVEF ≤35% and a substantial number of FP and FN results. Patients with discordant AI-ECG and TTE for LVSD (FP or FN) had similar outcomes, providing further evidence of the clinical relevance of the ECG features identified by the AI-ECG algorithm; the prevalence of discrepant TTE and AI-ECG almost certainly would have differed if biplane LVEF measurements were uniformly available.
Limitations
This retrospective cohort analysis has important limitations, including potential bias resulting from missing data and unmeasured confounding variables, and our results should be considered hypothesis-generating rather than definitive. Our CICU population differs from other CICU cohorts, most notably due to the lower number of racial and ethnic minorities represented; external validation in a distinct CICU cohort would strengthen our findings.28 Notably, this CICU population differs from the mixed inpatient/output populations used to derive and validate the AI-ECG for identification of LVSD, with higher patient acuity and a greater prevalence of LVSD.17–19 The automated AI-ECG algorithm provides a probability of LVSD without providing details regarding which ECG characteristics contributed to prediction, and our analysis was performed without manually over-reading of the ECG or TTE; likewise, we did not have available data on the heart rate, rhythm or clinical interpretation of the ECG itself. The LVEF cut-off ≤35% used in the original derivation and validation studies for the AI-ECG algorithm was specifically chosen to identify patients with asymptomatic LVSD that might warrant further evaluation and initiation of evidence-based therapies, yet fails to capture a substantial number of patients with clinically significant LVSD of lesser severity.17–19 Importantly, the AI-ECG and TTE were not simultaneous and performed on different days in almost half of patients, and it is conceivable that changes in either LVEF or ECG findings between the ECG and TTE could have led to misclassification of LVSD by the AI-ECG; serial AI-ECG data were not available to assess whether the association between the AI-ECG findings and mortality changes over time. Furthermore, various methods of LVEF assessment were used and only one in four patients had LVEF quantified using the biplane method—while this does reflect clinical practice in CICU patients who often have poor image quality precluding quantitative methods of LVEF measurement, this variability in LVEF measurement could have impacted our results. Given the limited use of biplane LVEF measurement, we cannot be sure that cases of discrepant TTE LVSD and AI-ECG LVSD are not due to misclassification of patients by TTE, or that the additive prognostic value of AI-ECG over TTE does not simply reflect identification of patients with inaccurate LVEF measurements. Finally, we did not adjudicate post-discharge deaths using national vital statistics, so our 1-year survival analysis should be considered exploratory due to potential bias from patients lost to follow-up.
Conclusions
A novel AI-ECG algorithm developed for prediction of LVSD provided robust mortality risk stratification in a CICU population beyond that conferred by the echocardiographic LVEF. Automated integration of AI-ECG data into the electronic health record could leverage this technology to facilitate LVSD identification and mortality risk-stratification in CICU patients. Future research is needed to understand how best to prevent adverse events in patients with an abnormal AI-ECG in the absence of LVSD. Prospective validation studies are needed to confirm the association between AI-ECG and mortality.
Supplementary material
Supplementary material is available at European Heart Journal: Acute Cardiovascular Care online.
Conflict of interest: Mayo Clinic has licensed the underlying technology to EKO, a maker of digital stethoscopes with embedded ECG electrodes. Mayo Clinic may receive financial benefit from the use of this technology, but at no point will Mayo Clinic benefit financially from its use for the care of patients at Mayo Clinic. P.A.F., F.L.-J., S.K., and Z.I.A. may also receive financial benefit from this agreement. The other authors have no relevant disclosures. Data collection and statistical analysis was performed independently by J.C.J., who was not involved in the development or validation of the proprietary technology and does not have any financial stake in the technology.
Supplementary Material
References
- 1. Jentzer JC, van Diepen S, Barsness GW, Katz JN, Wiley BM, Bennett CE, Mankad SV, Sinak LJ, Best PJ, Herrmann J, Jaffe AS, Murphy JG, Morrow DA, Wright RS, Bell MR, Anavekar NS.. Changes in comorbidities, diagnoses, therapies and outcomes in a contemporary cardiac intensive care unit population. Am Heart J 2019;215:12–19. [DOI] [PubMed] [Google Scholar]
- 2. Sinha SS, Sjoding MW, Sukul D, Prescott HC, Iwashyna TJ, Gurm HS, Cooke CR, Nallamothu BK.. Changes in primary noncardiac diagnoses over time among elderly cardiac intensive care unit patients in the United States. Circ Cardiovasc Qual Outcomes 2017;10:e003616. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Jentzer JC, Bennett C, Wiley BM, Murphree DH, Keegan MT, Gajic O, Wright RS, Barsness GW.. Predictive value of the sequential organ failure assessment score for mortality in a contemporary cardiac intensive care unit population. J Am Heart Assoc 2018;7:e008169. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Jentzer JC, Murphree DH, Wiley B, Bennett C, Goldfarb M, Keegan MT, Murphy JG, Wright RS, Barsness GW.. Comparison of mortality risk prediction among patients ≥70 versus <70 years of age in a cardiac intensive care unit. Am J Cardiol 2018;122:1773–1778. [DOI] [PubMed] [Google Scholar]
- 5. Jentzer JC, Bennett C, Wiley BM, Murphree DH, Keegan MT, Barsness GW.. Predictive value of individual sequential organ failure assessment sub-scores for mortality in the cardiac intensive care unit. PLoS One 2019;14:e0216177. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Bennett CE, Wright RS, Jentzer J, Gajic O, Murphree DH, Murphy JG, Mankad SV, Wiley BM, Bell MR, Barsness GW.. Severity of illness assessment with application of the APACHE IV predicted mortality and outcome trends analysis in an academic cardiac intensive care unit. J Crit Care 2019;50:242–246. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Jentzer JC, van Diepen S, Murphree DH, Ismail AS, Keegan MT, Morrow DA, Barsness GW, Anavekar NS.. Admission diagnosis and mortality risk prediction in a contemporary cardiac intensive care unit population. Am Heart J 2020;224:57–64. [DOI] [PubMed] [Google Scholar]
- 8. Jentzer JC, Anavekar NS, Brenes-Salazar JA, Wiley B, Murphree DH, Bennett C, Murphy JG, Keegan MT, Barsness GW.. Admission Braden Skin Score independently predicts mortality in cardiac intensive care patients. Mayo Clin Proc 2019;94:1994–2003. [DOI] [PubMed] [Google Scholar]
- 9. Jentzer JC, Anavekar NS, Bennett C, Murphree DH, Keegan MT, Wiley B, Morrow DA, Murphy JG, Bell MR, Barsness GW.. Derivation and validation of a novel cardiac intensive care unit admission risk score for mortality. J Am Heart Assoc 2019;8:e013675. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Jentzer JC, Wiley BM, Anavekar NS, Pislaru SV, Mankad SV, Bennett CE, Barsness GW, Hollenberg SM, Holmes DR, Oh JK.. Noninvasive hemodynamic assessment of shock severity and mortality risk in the cardiac intensive care unit. JACC Cardiovasc Imaging 2020;3510, DOI: 10.1016/j.jcmg.2020.05.038. [DOI] [PubMed] [Google Scholar]
- 11. Wehner GJ, Jing L, Haggerty CM, Suever JD, Leader JB, Hartzel DN, Kirchner HL, Manus JNA, James N, Ayar Z, Gladding P, Good CW, Cleland JGF, Fornwalt BK.. Routinely reported ejection fraction and mortality in clinical practice: where does the nadir of risk lie? Eur Heart J 2020;41:1249–1257. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Jentzer JC, Anavekar NS, Mankad SV, Khasawneh M, White RD, Barsness GW, Rabinstein AA, Kashani KB, Pislaru SV.. Echocardiographic left ventricular diastolic dysfunction predicts hospital mortality after out-of-hospital cardiac arrest. J Crit Care 2018;47:114–120. [DOI] [PubMed] [Google Scholar]
- 13. Jentzer JC, Chonde MD, Shafton A, Abu-Daya H, Chalhoub D, Althouse AD, Rittenberger JC.. Echocardiographic left ventricular systolic dysfunction early after resuscitation from cardiac arrest does not predict mortality or vasopressor requirements. Resuscitation 2016;106:58–64. [DOI] [PubMed] [Google Scholar]
- 14. Landesberg G, Gilon D, Meroz Y, Georgieva M, Levin PD, Goodman S, Avidan A, Beeri R, Weissman C, Jaffe AS, Sprung CL.. Diastolic dysfunction and mortality in severe sepsis and septic shock. Eur Heart J 2012;33:895–903. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Vallabhajosyula S, Pruthi S, Shah S, Wiley BM, Mankad SV, Jentzer JC.. Basic and advanced echocardiographic evaluation of myocardial dysfunction in sepsis and septic shock. Anaesth Intensive Care 2018;46:13–24. [DOI] [PubMed] [Google Scholar]
- 16. Lang RM, Badano LP, Mor-Avi V, Afilalo J, Armstrong A, Ernande L, Flachskampf FA, Foster E, Goldstein SA, Kuznetsova T, Lancellotti P, Muraru D, Picard MH, Rietzschel ER, Rudski L, Spencer KT, Tsang W, Voigt JU.. Recommendations for cardiac chamber quantification by echocardiography in adults: an update from the American Society of Echocardiography and the European Association of Cardiovascular Imaging. J Am Soc Echocardiogr 2015;28:1–39.e14. [DOI] [PubMed] [Google Scholar]
- 17. Attia ZI, Kapa S, Lopez-Jimenez F, McKie PM, Ladewig DJ, Satam G, Pellikka PA, Enriquez-Sarano M, Noseworthy PA, Munger TM, Asirvatham SJ, Scott CG, Carter RE, Friedman PA.. Screening for cardiac contractile dysfunction using an artificial intelligence-enabled electrocardiogram. Nat Med 2019;25:70–74. [DOI] [PubMed] [Google Scholar]
- 18. Attia ZI, Kapa S, Yao X, Lopez-Jimenez F, Mohan TL, Pellikka PA, Carter RE, Shah ND, Friedman PA, Noseworthy PA.. Prospective validation of a deep learning electrocardiogram algorithm for the detection of left ventricular systolic dysfunction. J Cardiovasc Electrophysiol 2019;30:668–674. [DOI] [PubMed] [Google Scholar]
- 19. Noseworthy PA, Attia ZI, Brewer LC, Hayes SN, Yao X, Kapa S, Friedman PA, Lopez-Jimenez F.. Assessing and mitigating bias in medical artificial intelligence: the effects of race and ethnicity on a deep learning model for ECG analysis. Circ Arrhythm Electrophysiol 2020;13:e007988. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Tison GH, Zhang J, Delling FN, Deo RC.. Automated and interpretable patient ECG profiles for disease detection, tracking, and discovery. Circ Cardiovasc Qual Outcomes 2019;12:e005289. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Raghunath SM, Ulloa Cerna AE, Jing L, vanMaanen DP, Stough J, Hartzel DN, Leader JB, Kirchner HL, Good CW, Patel AA, Delisle BP, Alsaid A, Beer D, Haggerty CM, Fornwalt BK. Prediction of mortality from 12-lead electrocardiogram voltage data using a deep neural network. Nat Med. 2020;26:886-891. [DOI] [PubMed]
- 22. Vollmer S, Mateen BA, Bohner G, Kiraly FJ, Ghani R, Jonsson P, Cumbers S, Jonas A, McAllister KSL, Myles P, Granger D, Birse M, Branson R, Moons KGM, Collins GS, Ioannidis JPA, Holmes C, Hemingway H.. Machine learning and artificial intelligence research for patient benefit: 20 critical questions on transparency, replicability, ethics, and effectiveness. BMJ 2020;368:l6927. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Collins GS, Moons KGM.. Reporting of artificial intelligence prediction models. Lancet 2019;393:1577–1579. [DOI] [PubMed] [Google Scholar]
- 24. Vallabhajosyula S, Rayes HA, Sakhuja A, Murad MH, Geske JB, Jentzer JC.. Global longitudinal strain using speckle-tracking echocardiography as a mortality predictor in sepsis: a systematic review. J Intensive Care Med 2019;34:87–93. [DOI] [PubMed] [Google Scholar]
- 25. Kwon JM, Kim KH, Jeon KH, Lee SE, Lee HY, Cho HJ, Choi JO, Jeon ES, Kim MS, Kim JJ, Hwang KK, Chae SC, Baek SH, Kang SM, Choi DJ, Yoo BS, Kim KH, Park HY, Cho MC, Oh BH.. Artificial intelligence algorithm for predicting mortality of patients with acute heart failure. PLoS One 2019;14:e0219302. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Holmgren G, Andersson P, Jakobsson A, Frigyesi A.. Artificial neural networks improve and simplify intensive care mortality prognostication: a national cohort study of 217,289 first-time intensive care unit admissions. J Intensive Care 2019;7:44. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Porta A, Colombo R, Marchi A, Bari V, De Maria B, Ranuzzi G, Guzzetti S, Fossali T, Raimondi F.. Association between autonomic control indexes and mortality in subjects admitted to intensive care unit. Sci Rep 2018;8:3486. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Goldfarb M, van Diepen S, Liszkowski M, Jentzer JC, Pedraza I, Cercek B.. Noncardiovascular disease and critical care delivery in a contemporary cardiac and medical intensive care unit. J Intensive Care Med 2019;34:537–543. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.