This cross-sectional study investigates if deep neural networks can predict left ventricular ejection fraction from angiograms compared with ejection fraction measured by transthoracic echocardiography.
Key Points
Question
Can deep neural networks (DNNs) predict left ventricular ejection fraction (LVEF) from angiograms compared with transthoracic echocardiogram (TTE)–measured LVEF?
Findings
In this cross-sectional study of 4042 adult angiograms matched with corresponding TTEs from 3679 UCSF patients, a DNN was trained to estimate reduced LVEF (≤40%) and continuous LVEF. The DNN showed good discrimination (binary) in internal and external validation data sets.
Meaning
Findings suggest that LVEF can be estimated from commonly obtained coronary angiograms using video-based DNNs, but caution is warranted when interpreting DNN estimates at LVEF extremes.
Abstract
Importance
Understanding left ventricular ejection fraction (LVEF) during coronary angiography can assist in disease management.
Objective
To develop an automated approach to predict LVEF from left coronary angiograms.
Design, Setting, and Participants
This was a cross-sectional study with external validation using patient data from December 12, 2012, to December 31, 2019, from the University of California, San Francisco (UCSF). Data were randomly split into training, development, and test data sets. External validation data were obtained from the University of Ottawa Heart Institute. Included in the analysis were all patients 18 years or older who received a coronary angiogram and transthoracic echocardiogram (TTE) within 3 months before or 1 month after the angiogram.
Exposure
A video-based deep neural network (DNN) called CathEF was used to discriminate (binary) reduced LVEF (≤40%) and to predict (continuous) LVEF percentage from standard angiogram videos of the left coronary artery. Guided class-discriminative gradient class activation mapping (GradCAM) was applied to visualize pixels in angiograms that contributed most to DNN LVEF prediction.
Results
A total of 4042 adult angiograms with corresponding TTE LVEF from 3679 UCSF patients were included in the analysis. Mean (SD) patient age was 64.3 (13.3) years, and 2212 patients were male (65%). In the UCSF test data set (n = 813), the video-based DNN discriminated (binary) reduced LVEF (≤40%) with an area under the receiver operating characteristic curve (AUROC) of 0.911 (95% CI, 0.887-0.934); diagnostic odds ratio for reduced LVEF was 22.7 (95% CI, 14.0-37.0). DNN-predicted continuous LVEF had a mean absolute error (MAE) of 8.5% (95% CI, 8.1%-9.0%) compared with TTE LVEF. Although DNN-predicted continuous LVEF differed 5% or less compared with TTE LVEF in 38.0% (309 of 813) of test data set studies, differences greater than 15% were observed in 15.2% (124 of 813). In external validation (n = 776), video-based DNN discriminated (binary) reduced LVEF (≤40%) with an AUROC of 0.906 (95% CI, 0.881-0.931), and DNN-predicted continuous LVEF had an MAE of 7.0% (95% CI, 6.6%-7.4%). Video-based DNN tended to overestimate low LVEFs and underestimate high LVEFs. Video-based DNN performance was consistent across sex, body mass index, low estimated glomerular filtration rate (≤45), presence of acute coronary syndromes, obstructive coronary artery disease, and left ventricular hypertrophy.
Conclusion and relevance
This cross-sectional study represents an early demonstration of estimating LVEF from standard angiogram videos of the left coronary artery using video-based DNNs. Further research can improve accuracy and reduce the variability of DNNs to maximize their clinical utility.
Introduction
Coronary heart disease is the leading cause of adult death worldwide,1 and coronary angiography provides the criterion-standard assessment for nearly all clinical decision-making, from medications to bypass surgery.2 Left ventricular ejection fraction (LVEF) is important to quantify near the time of coronary angiography because it can affect intraprocedural decisions and patient treatment.3 Reduced LVEF has multiorgan consequences that can affect the angiographic procedure and optimal interpretation of its findings. This is especially important in the setting of acute coronary syndromes (ACSs) where systolic function can change dynamically and where up-to-date LVEF assessment is often missing.4,5
LVEF can be ascertained before coronary angiography with transthoracic echocardiography, although this is not always available, particularly in patients with ACS who require urgent coronary angiography. LVEF can be assessed intraprocedurally through left ventriculography, which is an additional procedure requiring insertion of a pigtail catheter into the LV and injection of additional iodinated contrast,6 which can increase the risk of postprocedural acute kidney injury (AKI)7 and radiation exposure.8,9 The increased contrast required for ventriculography doubles the risk of AKI,7 limiting its use in patients at higher risk of AKI.10 Paradoxically, some of these patients can benefit the most from intraprocedural determination of systolic function.7 Although the use of ventriculography has decreased over time, it is still performed in up to 50% to 80% of angiograms with variation by institution,11,12 and remains a cornerstone approach to determine systolic function during coronary angiography. Unanticipated shortages in iodinated contrast media13 have also recently restricted the use of ventriculography. Novel methods to assess LVEF at the point of care during coronary angiography would expand the available options to perform this important physiologic determination.
Video-based deep neural networks (DNNs) can learn subtle patterns from medical data to accomplish certain tasks beyond what physicians can achieve with that data,14,15 providing an opportunity to assess cardiac systolic function in real time from standard angiographic images without additional cost or procedures.14,15 Standard coronary angiogram fluoroscopic videos likely capture subtle differences in coronary artery blood flow patterns and epicardial vessel motion that are altered in patients with LV systolic dysfunction.16,17,18 We hypothesized that LVEF could be estimated from standard coronary angiograms using video-based DNNs. We trained and tested a video-based DNN on a large real-world data set of clinical angiograms, then externally validated it in a separate data set from an institution in a different country.
Methods
Study Participants and Study Data Sets
This cross-sectional study was reviewed and approved by the University of California, San Francisco (UCSF) institutional review board; informed consent was exempted because of impracticality of obtaining consent from the large retrospective cohort. External validation was reviewed and approved by the University of Ottawa Heart Institute human research ethics board. We included coronary angiographic studies from all patients 18 years or older from UCSF between December 12, 2012, and December 31, 2019, who also had a transthoracic echocardiogram (TTE) performed either 3 months before or up to 1 month after the coronary angiogram (eFigure 1A in Supplement 1). The TTE closest to the date of the angiogram was used to determine the TTE LVEF. Patient race and ethnicity data were not directly available in the angiogram video metadata in our data set, nor was this information readily available in the other data sources pertaining to this study; therefore, these data were not included in the study analyses. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guidelines.
All coronary angiography study videos were extracted in the Digital Imaging and Communication in Medicine (DICOM) format and converted to a 512 × 512-pixel video file removing all identifying information. We identified the procedure indication by analyzing the procedure textual report using a large natural language processing model.19 The procedure indication was stratified into ACS (non–ST-segment elevation myocardial infarction/unstable angina or ST-segment elevation myocardial infarction) or non-ACS. All ACS text-report indications and 400 non-ACS indications were manually inspected to confirm language model performance. Videos from all angiographic studies were first analyzed using CathAI, a previously described DNN pipeline,20 to automatically identify videos containing the left coronary artery (LCA) as their primary anatomic structure. Patients and their respective coronary angiograms/LVEF pairs were randomly assigned to training (70%), development (10%), and test (20%) data sets (eFigure 1 in Supplement 1).
Algorithm Development
CathEF, a video-based DNN, was based on X3D architecture21 (eTable 2 in Supplement 1) and was trained to accept raw LCA angiogram videos to predict a continuous LVEF percentage value (Figure 1). For some analyses, we then dichotomized this continuous prediction into an LVEF value of less than or equal to 40% or greater than 40% because LVEF less than or equal to 40% defines significant LV dysfunction and has been a common cutoff for heart failure with reduced EF.4,5,22 In a sensitivity analysis, we also dichotomized using an LVEF cutoff value of less than or equal to 50%, which defines mildly reduced LVEF.5 More details are available in eMethods in Supplement 1.
Figure 1. Video-Based Deep Neural Network (DNN) Artificial Intelligence (AI) Algorithm Applied to a Coronary Angiography Input Video.
Angiographic videos of the left coronary artery (LCA) are first selected from a complete angiographic study using a DNN pipeline (eMethods in Supplement 1) and input into the video-based DNN algorithm to predict the left ventricular ejection fraction (LVEF). Predictions of LVEF from all available LCA videos from the same coronary angiography study, often obtained from different angiographic projections, were then averaged to obtain the final DNN-predicted LVEF for a patient.
The video-based DNN predictions from all LCA videos derived from the same angiographic study were averaged to obtain study-level performance. We also examined the DNN’s performance when averaging video-based DNN predictions using the first occurrence of videos from 3 prespecified projection angles (left anterior oblique [LAO] cranial, catheter and spine on the right side of the image; right anterior oblique [RAO] cranial, catheter and spine on the left side of the image; and RAO caudal) (eTable 1 in Supplement 1). These projections were chosen a priori as they mirror the anatomic projections of the heart often visualized by TTE: 2-chamber, 4-chamber, and 3-chamber long-axis echocardiographic views, which are commonly used to determine LVEF.23 Use of 3 common projections may simplify clinical application of the DNN. To better understand what aspects of angiogram videos were important to predict low LVEF, we applied guided class-discriminative gradient class activation mapping (GradCAM).24
External Validation
We randomly selected coronary angiogram studies from patients performed at the University of Ottawa Heart Institute (UOHI) between July 1, 2020, and March 31, 2021, and matched them with TTEs performed within 3 months before and/or up to 1 month after the angiogram. All UOHI TTEs were performed using a standardized view acquisition protocol, and LVEF was measured using the biplane Simpson method. In a subset of patients, left ventriculograms were available providing an alternate estimate of cardiac systolic function. Angiograms lacking a TTE LVEF were excluded. Three board-certified cardiologists (C.Y.G., Z.A., K.C.) performed a medical record review to extract the LVEF and patient-specific data. The TTE LVEF closest in date to the coronary angiogram was used. The video-based DNN was then applied to angiograms to predict DNN LVEF.
Statistical Analysis
Differences in means of continuous variables between reduced LVEF and normal LVEF groups were compared using the 2-sample t test assuming equal variance. Tests of significance were 2-sided. A P value of .05 or less was considered significant. DNN performance was described in the test and the external validation data sets using the mean absolute error (MAE), which was the absolute difference between DNN LVEF and TTE LVEF, the Pearson correlation,25 and the intraclass correlation (ICC; with ICC type 2,2)26 to determine correlation between DNN LVEF and TTE LVEF. We also present scatterplots and Bland-Altman27 plots to describe the association between the DNN-predicted LVEF and the LVEF derived from TTEs. We present performance of a video-based DNN according to various strata (eMethods in Supplement 1).
Results
A total of 4042 UCSF coronary angiogram studies with paired TTEs from 3679 patients were identified within the study period. After excluding short videos or DICOMs with invalid metadata or absence of LCA, the UCSF study cohort consisted of 3960 coronary angiograms from 3404 patients (providing 26 087 videos of the LCA). Mean (SD) patient age was 64.3 (13.3) years, 2212 patients were male (65%), and 1192 patients were female (35%). Angiograms and TTEs were obtained on average 57 days apart, but most angiograms were done either 1 month before (1255 of 3960 [31.7%]) or 1 week after (1107 of 3960 [28.0%]) the angiogram. A total of 604 patients (17.7%) had LVEF of 40% or less, whereas 2800 patients (82.2%) had mildly reduced or normal LVEF greater than 40%. The mean (SD) LVEF value in the low-LVEF group was 28.9% (7.7%) vs 60.8% (9.3%) in the group with midrange or normal LVEF (P < .001) (Table 1).28 Patients with LVEF of 40% or less were less likely to be female (163 [27.0%] vs 1027 [36.7%]; P < .001) and had similar body mass index (BMI; calculated as weight in kilograms divided by height in meters squared) with mean (SD) BMI of 27.2 (5.9) vs 27.7 (5.7) in those with LVEF greater than 40% (P = .04). Procedural indications were similar in both groups, and 7% of UCSF angiograms (274 of 3960) were done for ACS.
Table 1. Baseline Characteristics of the Main Study Cohort by Left Ventricular Ejection Fractiona.
| Characteristic | Patients with LVEF ≤40% (n = 604) | Patients with LVEF >40% (n = 2800) | P value |
|---|---|---|---|
| Age, mean (SD), y | 63.2 (14.6) | 64.5 (13.0) | .04 |
| Sex, No. (%) | |||
| Female | 161 (26.7) | 1031(36.8) | <.001 |
| Male | 443 (73.3) | 1769 (63.2) | |
| No. of patients with BMI data available, No. (%) | 604 (100) | 2800 (100) | |
| BMI, mean (SD)b | 27.2 (5.9) | 27.7 (5.7) | .04 |
| BMI <25, No. (%) | 266 (44.0) | 1066 (38.0) | .05 |
| BMI ≤25-<30, No. (%) | 189 (31.3) | 962 (34.4) | .05 |
| BMI ≥30, No. (%) | 149 (24.7) | 772 (27.5) | .05 |
| Total coronary angiography studies, No. | 724 | 3236 | |
| LCA videos per study, mean (SD), No. | 4.50 (1.96) | 4.33 (1.96) | .01 |
| Coronary angiogram indication | |||
| STEMI, No. (% of angiograms) | 10 (1.4) | 44 (1.4) | .86 |
| NSTEMI or unstable angina, No. (% of angiograms) | 44 (6.1) | 176 (5.4) | |
| Non-ACS, No. (% of angiograms) | 670 (92.5) | 3016 (93.2) | |
| TTE details | |||
| Absolute time between TTE and coronary angiogram, mean (SD), d | 42.4 (87.1) | 60.8 (97.9) | <.001 |
| TTEs done before the angiogram, No. (%) | 468 (64.3) | 2239 (69.2) | .64 |
| Mean LVEF, mean (SD), % | 28.9 (7.7) | 60.8 (9.3) | <.001 |
| TTE timing relative to angiogram, No. (%) | |||
| >60 d Before (% of angiograms) | 151 (20.9) | 961 (29.7) | <.001 |
| 30-60 d Before (% of angiograms) | 54 (7.5) | 286 (8.8) | .29 |
| 1-30 d Before (% of angiograms) | 263 (36.2) | 992 (30.7) | <.001 |
| 0-14 d After (% of angiograms) | 232 (32.0) | 875 (27.0) | <.001 |
| >14 d After (% of angiograms) | 24 (3.3) | 122 (3.8) | .35 |
| LVH grade on TTE, No./total No. (%)c | |||
| None | 165/486 (34.0) | 1467/2334 (62.8) | <.001 |
| Mild | 146/486 (30.0) | 456/2334 (19.5) | |
| Moderate | 111/486 (22.8) | 207/2334 (8.9) | |
| Severe | 64/486 (13.2) | 204/2334 (8.7) | |
| Angiographic studies with lab values, No. | 209 | 775 | NA |
| eGFR, mL/min/1.73 m2, mean (SD)d | 61.2 (23.8) | 64.9 (26.1) | .17 |
| eGFR >60, No. (%)d | 67 (32.1) | 320 (41.3) | .02 |
| eGFR 45-60, No. (%)d | 100 (47.8) | 321 (41.3) | .02 |
| eGFR 30-45, No. (%)d | 25 (12.0) | 56 (7.2) | .01 |
| eGFR <30, No. (%)d | 17 (8.1) | 78 (10.6) | .89 |
Abbreviations: BMI, body mass index; eGFR, estimated glomerular filtration rate; LCA, left coronary angiography; LVEF, left ventricular ejection fraction; LVH, left ventricular hypertrophy; NA, not applicable; NSTEMI, non–ST-segment elevation myocardial infarction; STEMI, ST-segment elevation myocardial infarction; TTE, transthoracic echocardiogram.
Where data were only available for subgroups of the full cohort, subgroup sample size is denoted by N. Differences in means of continuous variables between 2 groups were compared using the 2-sample t test assuming equal variance. Differences in proportions of categorical variables between 2 groups were compared using the χ2 test. Tests of significance were 2-sided.
Calculated as weight in kilograms divided by height in meters squared.
This value was abstracted from the TTE reports. Some TTE reports did not have any information containing LVH and therefore this represents a subset of our full cohort. The severity of LVH was described according to the American Society of Echocardiography guidelines.28
Using the serum creatinine level, we calculated the eGFR using the Modification of Diet in Renal Disease equation and classified patients as having normal baseline kidney function (eGFR, >60) and mild (eGFR, 45-60), moderate (eGFR, 30-45), or severe (eGFR, <30) chronic kidney disease.
In the UCSF test data set (n = 813), the video-based DNN had a strong ability to discriminate LVEF of 40% or less from LVEF greater than 40% with an AUROC of 0.911 (95% CI, 0.887-0.934) (Table 2) and to discriminate LVEF less than 50% or 50% or greater with an AUROC of 0.879 (95% CI, 0.852-0.907) (eTable 5 in Supplement 1). There were 22.7 greater odds (95% CI, 14.0-37.0) of reduced LVEF in those DNN predicted as LVEF 40% or less; specificity was 81.3% (95% CI, 78.8%-84.1%), and sensitivity was 83.9% (95% CI, 78.2%-89.1%) (Table 2 and eFigure 2A in Supplement 1). DNN-predicted continuous LVEF correlated with TTE LVEF (ICC, 0.77; 95% CI, 0.73-0.82; Pearson r = 0.71; 95% CI, 0.67-0.74) (Table 2 and Figure 2A) and had an MAE of 8.5% (95% CI, 8.1%-9.0%) and an MAE of 3.7% (95% CI, 2.7%-9.3%) when averaging predictions across an angiographic study. Although the DNN-predicted continuous LVEF differed 5% or less compared with TTE LVEF in 38.0% of test data set studies (309 of 813), differences greater than 15% were observed in 15.2% (124 of 813) (eFigure 4 in Supplement 1). When the DNN used only the 3 prespecified and routinely obtained angiographic projection angles to determine LVEF of 40% or less, AUROC was 0.908 (95% CI, 0.880-0.935) in 761 studies (Table 2), with a specificity of 82.8% (95% CI, 79.8%-85.6%) and a sensitivity of 84.1% (95% CI, 78.2%-90.1%). When using a reduced LVEF cutoff of 50% as a sensitivity analysis, DNN performance was similar to that at the 40% cutoff (eTable 5 in Supplement 1).
Table 2. Performance of the Video-Based Deep Neural Network (DNN) to Identify Left Ventricular Ejection Fraction (LVEF) of 40% or Less by Transthoracic Echocardiography.
| Variable | No. | AUROC (95% CI) | Diagnostic OR (95% CI) | Sensitivity (95% CI)a | Specificity (95% CI)a | PPV (95% CI)a | NPV (95% CI)a | MAE (95% CI) | Pearson r (95% CI) | ICC (95% CI) | P valueb |
|---|---|---|---|---|---|---|---|---|---|---|---|
| UCSF test data set | |||||||||||
| All available LCA videos per study | 813 | 0.911 (0.887-0.934) | 22.7 (14.0-37.0) | 83.9 (78.2-89.1) | 81.3 (78.8-84.1) | 49.0 (43.0-54.7) | 96.0 (94.4-97.4) | 8.5 (8.1-9.0) | 0.71 (0.67-0.74) | 0.77 (0.73-0.82) | NA |
| Female | 258 | 0.945 (0.915-0.974) | 56.9 (16.3-198.1) | 90.3 (80.0-100.0) | 85.9 (81.5-89.9) | 46.7 (34.7-58.3) | 98.5 (96.8-100.0) | 8.3 (7.6-9.1) | 0.7 (0.62-0.77) | 0.85 (0.75-0.91) | [Ref] |
| Male | 555 | 0.894 (0.863-0.926) | 17.3 (10.1-29.6) | 82.1 (74.6-88.3) | 79.0 (75.4-82.5) | 49.7 (43.2-56.1) | 94.6 (92.3-96.6) | 8.6 (8.1-9.1) | 0.7 (0.66-0.74) | 0.8 (0.75-0.86) | .23 |
| LCA videos from 3 projectionsc | 761 | 0.908 (0.880-0.935) | 25.5 (15.3-42.5) | 84.1 (78.2-90.1) | 82.8 (79.8-85.6) | 50.7 (44.8-57.3) | 96.1 (94.5-97.7) | 8.6 (8.1-9.0) | 0.7 (0.66-0.73) | 0.78 (0.73-0.83) | NA |
| Female | 246 | 0.952 (0.924-0.979) | 60.4 (17.2-212.4) | 90.0 (79.2-100.0) | 87.0 (82.5-91.0) | 49.1 (36.8-62.0) | 98.4 (96.7-100.0) | 8.2 (7.5-9.0) | 0.71 (0.62-0.77) | 0.85 (0.76-0.92) | [Ref] |
| Male | 515 | 0.886 (0.848-0.923) | 19.4 (11.0-34.2) | 82.4 (75.0-88.8) | 80.6 (77.1-84.2) | 51.2 (43.9-58.5) | 94.9 (92.5-96.8) | 8.7 (8.2-9.3) | 0.68 (0.62-0.73) | 0.81 (0.76-0.87) | .11 |
| UOHI external validation data set | |||||||||||
| All available LCA videos per study | 776 | 0.906 (0.881-0.931) | 27.3 (17.6-42.4) | 77.9 (72.1-83.6) | 88.6 (86.2-90.9) | 66.0 (59.7-71.9) | 93.4 (91.3-95.3) | 7.0 (6.6-7.4) | 0.72 (0.68-0.76) | 0.62 (0.56-0.67) | NA |
| Female | 273 | 0.9 (0.856-0.944) | 15.2 (7.3-31.6) | 65.2 (51.6-77.4) | 89.0 (85.3-92.8) | 54.5 (42.2-67.6) | 92.7 (89.2-95.6) | 6.9 (6.2-7.6) | 0.67 (0.57-0.75) | 0.65 (0.54-0.74) | [Ref] |
| Male | 503 | 0.906 (0.875-0.937) | 35.8 (20.5-62.5) | 82.5 (76.3-88.7) | 88.3 (85.1-91.3) | 70.3 (63.4-77.2) | 93.8 (91.4-96.1) | 7.0 (6.5-7.5) | 0.74 (0.69-0.78) | 0.65 (0.58-0.71) | .50 |
| LCA videos from 3 projectionsc | 653 | 0.905 (0.878-0.932) | 22.8 (14.3-36.4) | 70.2 (69.6-77.4) | 90.6 (89.5-91.8) | 67.3 (63.5-72.7) | 91.7 (91.5-93.2) | 7.1 (6.8-7.5) | 0.73 (0.72-0.75) | 0.66 (0.62-0.69) | NA |
| Female | 230 | 0.909 (0.867-0.952) | 15.2 (6.7-34.6) | 59.5 (45.1-75.1) | 91.2 (91.1-93.8) | 56.4 (46.5-66.5) | 92.1 (90.7-95.5) | 7.4 (6.9-7.4) | 0.67 (0.55-0.76) | 0.69 (0.54-0.73) | [Ref] |
| Male | 423 | 0.9 (0.864-0.935) | 26.5 (14.9-47.0) | 74.0 (63.1-78.4) | 90.3 (89.6-92.6) | 71.3 (69.4-75.7) | 91.4 (89.1-93.4) | 7.2 (6.7-7.7) | 0.74 (0.7-0.76) | 0.67 (0.61-0.7) | .81 |
Abbreviations: AUROC, area under the receiver operating characteristic curve; ICC, intraclass correlation; MAE, mean absolute error; NA, not applicable; NPV, negative predictive value; PPV, positive predictive value; Ref, reference; UCSF, University of California, San Francisco; UOHI, University of Ottawa Heart Institute.
The cutoff for determining sensitivity, specificity, PPV, and NPV for DNN-LVEF category was 0.1.
The P values for interaction were calculated using 2-sided Wald tests between the video-based DNN probability and the respective covariates for LVEF.
The 3 prespecified, commonly obtained angiographic projection angles are: left anterior oblique cranial, right anterior oblique cranial, and right anterior oblique caudal projections.
Figure 2. Scatterplot of the Video-Based Deep Neural Network (DNN)–Predicted Left Ventricular Ejection Fraction (LVEF) Compared With Transthoracic Echocardiogram (TTE) LVEF.
A, Scatterplot of DNN-predicted LVEF compared with TTE LVEF in the University of California, San Francisco, test data set (n = 813 studies). B, Scatterplot using the University of Ottawa Heart Institute external validation data set (n = 776 studies). Each dot represents the DNN-predicted LVEF (using all available angiographic projections) compared with the TTE LVEF. The diagonal lines represent the regression best-fit line of DNN LVEF to TTE LVEF. The translucent bands around the regression line show 95% CIs and are estimated using bootstrapping. The black orthogonal lines represent the line of identity for an LVEF of 40% on TTE. Pearson correlation and associated P value are shown.
In general, the algorithm remained consistent across strata of sex, BMI, presence of obstructive coronary stenosis, presence and severity of left ventricular hypertrophy (LVH), and low kidney function (eGFR ≤45), in whom the additional contrast from ventriculography can be harmful (eTables 3 and 5 in Supplement 1). In patients undergoing coronary angiography for ACS, where up-to-date LVEF assessment may often not be available, video-based DNN performance was similar to that for other non-ACS indications, with similar MAE (7.6%; 95% CI, 5.9%-9.4% vs 8.7%; 95% CI, 8.2%-9.1%; P = .20). The right anterior oblique (RAO) caudal and anteroposterior caudal angiographic projections achieved the highest AUC for discriminating low LVEF (eTable 3 in Supplement 1); straight anteroposterior view performed worse. We also examined variability of DNN predictions for distinct angiogram videos with the same projection obtained during the same study. There was a mean (SD) 0.1% (4.3%) difference in DNN-LVEF predictions between successive videos within the same study taken from the same angiographic projection (1927 distinct videos; 35.5% of total), and the mean (SD) variance between projections was 5.1% (4.5%). The performance of various training approaches for the DNN can be found in eTable 7 in Supplement 1. Generally, the best performance was achieved when using 5-second video clips (consisting of 72 frames) for training the DNN with the X3D architecture.
External Validation of Video-Based DNN in a Different Hospital
A total of 1034 coronary angiogram studies from 995 patients performed at UOHI were randomly selected. After matching the angiogram studies with TTEs performed within 3 months before and/or up to 1 month after the angiogram, in the UOHI external validation cohort, 776 angiograms had paired TTEs (from 744 patients; 159 patients [16.8%] with LVEF ≤40% and 585 patients [76.3%] with LVEF >40%) (eFigure 1B and eTable 6 in Supplement 1). Mean (SD) patient age was 68.6 (12.1) years, 482 patients were male (65%), and 262 patients were female (35%) (eTable 6 in Supplement 1). The UOHI coronary angiograms and TTEs were performed within a median of 5.5 days of each other. In the external validation data set, the video-based DNN had an AUROC of 0.906 (95% CI, 0.881-0.931) for binary LVEF of 40% or less or greater than 40%, diagnostic odds ratio was 27.3 (95% CI, 17.6-42.4), specificity was 88.6% (95% CI, 86.2%-90.9%), and sensitivity was 77.9% (95% CI, 72.1%-83.6%) (Table 2 and eFigure 2B in Supplement 1). DNN-predicted continuous LVEF remained valid, with an MAE of 7.0% (95% CI, 6.6%-7.4%) and strong correlations with TTE LVEF (ICC, 0.62; 95% CI, 0.56-0.67; Pearson r = 0.72; 95% CI, 0.68-0.76) (Table 2 and Figure 2B) when averaging predictions across an angiographic study. When the DNN used only the common 3 angiographic projections, AUROC was 0.905 (95% CI, 0.878-0.932) (Table 2). Performance was consistent across sex and BMI and in patients undergoing angiography for ACS and across ranges of kidney insufficiency (eTable 4 in Supplement 1). DNN-predicted continuous LVEF differed 5% or less compared with TTE LVEF in 47.7% of UOHI studies (370). In a sensitivity analysis, when using the LVEF cutoff of 50% or less, performance of the video-based DNN remained similar (AUROC, 0.883; 95% CI, 0.858-0.909) using all projections and AUROC of 0.875 (95% CI, 0.846-0.904) using only the 3 projections.
Understanding How the Video-Based DNN Works
To better understand how the video-based DNN can predict reduced LVEF from angiograms, we applied the GradCAM AI explainability technique to identify regions in the angiogram video important for the DNN to classify LVEF of 40% or less. GradCAM consistently highlighted epicardial LCA vessel segments and septal perforators predominantly during systole (vs diastole) as the strongest predictors of low LVEF (eFigure 5 in Supplement 1 and Video). This suggests that the DNN identifies coronary arterial and myocardial patterns during systole to predict reduced LVEF, likely focusing on the appearance and/or the systolic motion of the LCA.
Video. Gradient Class Activation Mapping (GradCAM) Artificial Intelligence (AI) Explainability Technique Applied to a Video-Based Deep Neural Network (DNN) Trained to Predict Left Ventricular Ejection Fraction.
Angiogram video from a patient with normal left ventricular ejection fraction (left) and the corresponding video of guided GradCAM saliency maps (right). The guided GradCAM saliency maps highlight pixels in each frame that contribute the most to the DNN’s prediction of low left ventricular ejection fraction in this video. Pixels predominantly around the coronary artery tree are highlighted, mainly during the systolic phase of the cardiac cycle.
Discussion
Findings of this cross-sectional study suggest for the first time, to our knowledge, that standard coronary angiogram videos may be used to provide an automated estimation of LVEF (AUROC, 0.906-0.911 for LVEF ≤40%; MAE, 7.0%-8.5% vs TTE LVEF), thereby providing information from coronary angiography alone that is not usually accessible to clinicians and typically requires additional testing. The video-based DNN performed equally well in patients presenting for ACS in whom up-to-date LV functional data are often missing before urgent angiography. The DNN achieved this by simultaneously examining all frames of an angiogram video to determine both static and moving (interimage) features, as opposed to image-based neural networks that consider only static features. DNN performance was similar when only provided with the 3 most frequently obtained angiographic projection angles (AUC, 0.905-0.908) and was consistent in patients with obstructive CAD and those with preexisting kidney disease—all populations who stand to benefit most from limiting contrast administration.8,10 Importantly, the DNN generalized very well, without any additional training, to unselected real-world angiograms from an external quaternary health care institution in another country. The video-based DNN represents a technological advancement to coronary angiography offering a novel approach to evaluate LV systolic function that can be applied routinely during any standard coronary angiogram without additional equipment or procedures. It enables real-time, dynamic assessment of cardiac function during coronary angiography, thereby providing a noninvasive alternative to left ventriculography in patients for whom additional risk or contrast is suboptimal.
Automated assessment of LV systolic function during coronary angiography is clinically beneficial for multiple reasons, including establishing prognosis and initiating appropriate treatment in patients with acute myocardial infarction associated with LV dysfunction or when recent LVEF assessment is unavailable before angiography. The DNN allows for automated intraprocedural (near real-time) estimation of LVEF from coronary angiogram videos using AI algorithms alone, making it possible to estimate LVEF routinely after any standard LCA fluoroscopy. This represents a significant technological advancement to coronary angiography, adding important capability of real-time cardiac function assessment without requiring additional image acquisitions or procedures. If properly validated, this technology could potentially change the standard of care. For example, in patients presenting with ACS, if the DNN estimates a normal LVEF, it may decrease the urgency to obtain a predischarge TTE.
In patients presenting emergently with ACS where prior LVEF assessment may be unavailable, the present standard of care relies on left ventriculography to assess LVEF intraprocedurally (eFigure 6 in Supplement 1).11,12 However, ventriculography requires an additional procedure and is contraindicated in certain patients due to associated complications.6,7,10,29,30 Furthermore, ventriculography requires additional contrast (30-40 mL) to opacify the LV cavity, doubling the odds of contrast-induced nephropathy in at-risk patients.7 In comparison, clinical deployment of the DNN would use software algorithms alone to automatically predict LVEF from routinely obtained LCA angiograms. Several prior studies16,17,18 performed heuristic calculations from coronary angiogram–derived measurements to identify systolic dysfunction, but they were not automated, would be cumbersome to perform in real time, and none are in common clinical use.
DNN performance was consistent across sex, a wide range of BMI values and LVH severity, and also functioned well in patients with ACS, obstructive coronary stenosis, and kidney insufficiency in both the test and external validation data sets. Notably, patients with these latter conditions are also at increased risk of LV systolic dysfunction, are at higher risk for contrast-induced nephropathy, and in the case of ACS, often do not have recent prior LVEF assessment available; therefore, these patients would benefit greatly from ad hoc LV function evaluation. The DNN exhibited lower MAE (7%-8.5%) than the reported variability in ventriculography (14%), suggesting that DNN values could be closer to TTE LVEF than ventriculography.31 The MAE of 7.0% to 8.5% that was observed between DNN-predicted continuous LVEF and TTE LVEF is also comparable with prior reports of TTE interreader variability, both for 2-dimensional (2D)–contrast TTE (8.0%) and 3D-contrast TTE (7.4%).31 Further, the median DNN error that we observed was substantially lower than the mean error at 3.7%, suggesting that outliers (at LVEF extremes) may have been driving much of the overall DNN error, whereas video-based DNN has less error at moderate LVEF ranges. Indeed, we observed overestimation of LVEF at severely reduced values (<30%) and underestimation of LVEF at higher values (>65%) (eFigure 3 in Supplement 1). It is also worth noting that the administration of contrast during angiography can temporarily depress the LVEF.32 However, we did not observe a consistent decrease in the DNN-predicted LVEF compared with TTE LVEF (eFigures 2 and 3 in Supplement 1).
The video-based DNN also functioned well when using only 3 standard angiographic projections that were available in 85% to 90% of the angiograms in both our internal and external data sets. The 3 angiographic projections—RAO cranial, LAO cranial, and RAO caudal—were chosen because they represent angiographic projections of the heart analogous to the apical 2-chamber, 4-chamber, and 3-chamber projections on the TTE,23 respectively, which are used to determine the LVEF.28 By extracting novel information from routinely collected data, this video-based DNN expands the utility of the standard angiogram by providing clinically actionable insights into LVEF at the point of care. Its efficient X3D architecture is fast and processes an angiogram in less than 1 second per video (3 seconds for the 3 projections) on commonly available hardware (GeForce GTX 1080 Ti [Nvidia]), enabling near real-time prediction of LVEF during angiogram acquisition.
We applied GradCAM24 to highlight important regions in the video that aid the DNN to predict cardiac dysfunction. GradCAM consistently highlighted the algorithm’s focus on several cardiac areas during systole. During systole, the intramyocardial coronary arteries move with the heart and are compressed by the contracting heart muscle that surrounds them,33 likely providing a surrogate for LV function. In patients with decreased cardiac systolic function, the video-based DNN appeared to recognize differences in these myocardial and artery movements and possibly other anatomic characteristics between low LVEF and borderline/normal LVEF angiogram videos.
Limitations
Our work is best understood in the context of its limitations. The video-based DNN tended to underestimate high LVEF values (>65%) and overestimate severely reduced LVEF values (≤30%), warranting caution when interpreting DNN predictions in patients with high pretest probability of either very low or high EF. It is important to note that this study represents an early demonstration of using video-based DNNs to estimate LVEF from standard LCA angiogram videos. Further research is necessary to improve accuracy and reduce the variability of the DNN to maximize its clinical utility. Alternative loss functions, such as the Huber loss function,34 and resampling techniques may help to reduce the bias at LVEF extremes but may also have tradeoffs in terms of generalizability. In addition, the DNN’s very high negative predictive value supports its use to rule out the presence of reduced LVEF in the majority of patients. It should be noted that the video-based DNN does not provide information on regional wall motion abnormalities, which may be present despite normal LVEF. The DNN was trained using angiogram projections of the LCA because it both supplies and anatomically lies above the left ventricle. Although our sensitivity analysis showed that cardiac dysfunction could also be identified using angiogram projections of the right coronary artery, this did not improve performance, and thus, additional algorithm refinement may be needed to optimize performance. The ability of the video-based DNN to function in the setting of abnormal flow in 1 of the major epicardial vessels (ie, ST-elevation myocardial infarction) remains to be evaluated; however, the algorithm performed equally well in patients presenting with ACS indications as in those presenting with non-ACS indications. Furthermore, our model was developed using TTEs obtained 3 months before or up to 1 month after the angiogram, during which LVEF may have changed, especially in patients with ACS where myocardial stunning could be present in the acute phase. However, DNN performance remained valid in the UHOI data set where the median difference between TTE and angiogram was much lower, at 5.5 days. In addition, there was no significant heterogeneity in the performance of the DNN with TTEs performed farther from the coronary angiography (eTable 3 in Supplement 1). Future validation of the video-based DNN using TTEs performed as close to the time of angiography as possible will be needed. To this end, the study team is currently conducting a multicenter prospective validation study in patients with ACS to compare the performance of the DNN with more contemporaneous TTEs performed within 48 hours of ACS.35
Conclusions
In conclusion, findings of this cross-sectional study suggest that a video-based DNN enabled estimation of LV systolic function using routinely obtained LCA angiogram videos, but caution is warranted at extremes of LVEF. The algorithm remained valid in a hospital in a different country and had similar performance regardless of sex, BMI, ACS, presence of obstructive coronary artery disease, and presence and severity of LVH.
eMethods
eReferences
eFigure 1. Creation of Study Data Sets
eFigure 2. Confusion Matrices for CathEF Prediction of Binary Reduced LVEF (≤/>40%)
eFigure 3. Bland-Altmann Plots Examining Systematic Bias Between CathEF and TTE
eFigure 4. Box Plots of the Difference Between CathEF Prediction of LVEF and Human Calculated LVEF on TTE, Stratified According to the Underlying LVEF Strata
eFigure 5. Guided Grad-CAM Highlighting Areas of the Image Used to Predict LVEF <40% or LVEF ≥40
eFigure 6. Standard Fluoroscopy Image From a Left Ventriculogram Procedure
eTable 1. Definition of the Angiographic Projection Angle Classes
eTable 2. Hyperparameters Searched for CathEF
eTable 3. Performance of CathEF to Identify LVEF ≤40/>40% by Transthoracic Echo in Strata of the Test Data Set
eTable 4. Performance of CathEF to Identify LVEF≤40/>40% by Transthoracic Echo in Strata of the UOHI External Validation Data Set
eTable 5. Performance of CathEF to Identify LVEF<50/≥50% by Transthoracic Echo in Strata of the Test Data Set
eTable 6. Baseline Characteristics of the External Validation Cohort
eTable 7. Development Set Performance of Alternative Training Schemes and Hyperparameters
Data Sharing Statement
References
- 1.Virani SS, Alonso A, Aparicio HJ, et al. ; American Heart Association Council on Epidemiology and Prevention Statistics Committee and Stroke Statistics Subcommittee . Heart disease and stroke statistics 2021 update: a report from the American Heart Association. Circulation. 2021;143(8):e254-e743. doi: 10.1161/CIR.0000000000000950 [DOI] [PubMed] [Google Scholar]
- 2.Patel MR, Calhoon JH, Dehmer GJ, et al. ACC/AATS/AHA/ASE/ASNC/SCAI/SCCT/STS 2017 appropriate use criteria for coronary revascularization in patients with stable ischemic heart disease: a report of the American College of Cardiology Appropriate Use Criteria Task Force, American Association for Thoracic Surgery, American Heart Association, American Society of Echocardiography, American Society of Nuclear Cardiology, Society for Cardiovascular Angiography and Interventions, Society of Cardiovascular Computed Tomography, and Society of Thoracic Surgeons. J Am Coll Cardiol. 2017;69(17):2212-2241. doi: 10.1016/j.jacc.2017.02.001 [DOI] [PubMed] [Google Scholar]
- 3.Yancy CW, Jessup M, Bozkurt B, et al. ; American College of Cardiology Foundation; American Heart Association Task Force on Practice Guidelines . 2013 ACCF/AHA guideline for the management of heart failure: a report of the American College of Cardiology Foundation/American Heart Association task force on practice guidelines. J Am Coll Cardiol. 2013;62(16):e147-e239. doi: 10.1016/j.jacc.2013.05.019 [DOI] [PubMed] [Google Scholar]
- 4.Maddox TM, Januzzi JL Jr, Allen LA, et al. ; Writing Committee . 2021 update to the 2017 ACC expert consensus decision pathway for optimization of heart failure treatment: answers to 10 pivotal issues about heart failure with reduced ejection fraction: a report of the American College of Cardiology solution set oversight committee. J Am Coll Cardiol. 2021;77(6):772-810. doi: 10.1016/j.jacc.2020.11.022 [DOI] [PubMed] [Google Scholar]
- 5.McDonagh TA, Metra M, Adamo M, et al. ; ESC Scientific Document Group . 2021 ESC guidelines for the diagnosis and treatment of acute and chronic heart failure. Eur Heart J. 2021;42(36):3599-3726. doi: 10.1093/eurheartj/ehab368 [DOI] [PubMed] [Google Scholar]
- 6.Moscucci M. Grossman & Baim’s Cardiac Catheterization, Angiography, and Intervention. 8th ed. Wolters Kluwer; 2013. [Google Scholar]
- 7.Schönenberger E, Martus P, Bosserdt M, et al. Kidney injury after intravenous vs intra-arterial contrast agent in patients suspected of having coronary artery disease: a randomized trial. Radiology. 2019;292(3):664-672. doi: 10.1148/radiol.2019182220 [DOI] [PubMed] [Google Scholar]
- 8.Marenzi G, Lauri G, Assanelli E, et al. Contrast-induced nephropathy in patients undergoing primary angioplasty for acute myocardial infarction. J Am Coll Cardiol. 2004;44(9):1780-1785. doi: 10.1016/j.jacc.2004.07.043 [DOI] [PubMed] [Google Scholar]
- 9.Khatri P, Kasner SE. Ischemic strokes after cardiac catheterization: opportune thrombolysis candidates? Arch Neurol. 2006;63(6):817-821. doi: 10.1001/archneur.63.6.817 [DOI] [PubMed] [Google Scholar]
- 10.Azzalini L, Spagnoli V, Ly HQ. Contrast-induced nephropathy: from pathophysiology to preventive strategies. Can J Cardiol. 2016;32(2):247-255. doi: 10.1016/j.cjca.2015.05.013 [DOI] [PubMed] [Google Scholar]
- 11.Heidenreich PA, Lin S, Knowles JW, et al. Variation in use of left ventriculography in the Veterans Affairs Health Care System. Circ Cardiovasc Qual Outcomes. 2013;6(6):687-693. doi: 10.1161/CIRCOUTCOMES.113.000199 [DOI] [PubMed] [Google Scholar]
- 12.Witteles RM, Knowles JW, Perez M, et al. Use and overuse of left ventriculography. Am Heart J. 2012;163(4):617-23.e1. doi: 10.1016/j.ahj.2011.12.018 [DOI] [PubMed] [Google Scholar]
- 13.Greater New York Hospital Association . GE contrast media shortage and conservation strategies. Accessed September 20, 2022. https://www.gnyha.org/news/ge-contrast-media-shortage-and-conservation-strategies
- 14.Attia ZI, Kapa S, Lopez-Jimenez F, et al. Screening for cardiac contractile dysfunction using an artificial intelligence–enabled electrocardiogram. Nat Med. 2019;25(1):70-74. doi: 10.1038/s41591-018-0240-2 [DOI] [PubMed] [Google Scholar]
- 15.Avram R, Olgin JE, Kuhar P, et al. A digital biomarker of diabetes from smartphone-based vascular signals. Nat Med. 2020;26(10):1576-1582. doi: 10.1038/s41591-020-1010-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Wandt B, Kähäri A, Zizala J, Bojö L, Wranne B. Usefulness of coronary angiography for assessing left ventricular function. Am J Cardiol. 1998;82(3):384-386. doi: 10.1016/S0002-9149(98)00326-9 [DOI] [PubMed] [Google Scholar]
- 17.Kähäri A, Emilsson K, Danielewicz M, Andersson T, Wandt B. Circumflex artery motion; a new angiographic method for assessment of left ventricular function. Scand Cardiovasc J. 2003;37(2):80-86. doi: 10.1080/14017430310001195 [DOI] [PubMed] [Google Scholar]
- 18.Özdemir K, Altunkeser BB, Sökmen G, et al. Usefulness of coronary angiography for assessing left ventricular systolic function. Acta Cardiol. 2001;56(4):239-242. doi: 10.2143/AC.56.4.2005650 [DOI] [PubMed] [Google Scholar]
- 19.Impira . DocQuery: document query engine. Accessed September 22, 2022. https://huggingface.co/spaces/impira/docquery
- 20.Avram R, Olgin JE, Wan A, et al. CathAI: fully automated interpretation of coronary angiograms using neural networks. Arxiv. Published online June 14, 2021. https://arxiv.org/abs/2106.07708
- 21.Feichtenhofer C. X3D: expanding architectures for efficient video recognition. Arxiv. Preprint posted online April 9, 2020. doi: 10.1109/CVPR42600.2020.00028 [DOI]
- 22.Nicod P, Gilpin E, Dittrich H, et al. Influence on prognosis and morbidity of left ventricular ejection fraction with and without signs of left ventricular failure after acute myocardial infarction. Am J Cardiol. 1988;61(15):1165-1171. doi: 10.1016/0002-9149(88)91148-4 [DOI] [PubMed] [Google Scholar]
- 23.Piazza N, Mylotte D, Theriault Lauzier P. Fluoroscopic “heart chamber” anatomy - the case for imaging modality-independent terminology. EuroIntervention. 2016;12(Y):Y9-Y15. doi: 10.4244/EIJV12SYA3 [DOI] [PubMed] [Google Scholar]
- 24.Selvaraju RR, Cogswell M, Das A, Verdantam R, Parikh D, Dhruv B. Grad-CAM: visual explanations from deep networks via gradient-based localization. Int J Comput Vis. 2019;128(2):336-359. doi: 10.1007/s11263-019-01228-7 [DOI] [Google Scholar]
- 25.Karl P, Alice L. On the laws of inheritance in man: I. inheritance of physical characters. Biometrika. 1903;2(4):357-462. doi: 10.2307/2331507 [DOI] [Google Scholar]
- 26.Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155-163. doi: 10.1016/j.jcm.2016.02.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Bland JM, Altman DG. Statistical methods for assessing agreement between 2 methods of clinical measurement. Lancet. 1986;1(8476):307-310. doi: 10.1016/S0140-6736(86)90837-8 [DOI] [PubMed] [Google Scholar]
- 28.Lang RM, Badano LP, Mor-Avi V, et al. Recommendations for cardiac chamber quantification by echocardiography in adults: an update from the American Society of Echocardiography and the European Association of Cardiovascular Imaging. J Am Soc Echocardiogr. 2015;28(1):1-39.e14. doi: 10.1016/j.echo.2014.10.003 [DOI] [PubMed] [Google Scholar]
- 29.Marine JE, Smith TW, Monahan KM. High-grade atrioventricular block caused by his-purkinje injury during contrast left ventriculography. Circulation. 2001;104(14):E77-E78. doi: 10.1161/hc3901.096398 [DOI] [PubMed] [Google Scholar]
- 30.Deligonul U, Jones S, Shurmur S, Oskarsson H. Contrast cine left ventriculography: comparison of 2 pigtail catheter shapes and analysis of factors determining the final quality. Cathet Cardiovasc Diagn. 1996;37(4):428-433. doi: [DOI] [PubMed] [Google Scholar]
- 31.Hoffmann R, Barletta G, von Bardeleben S, et al. Analysis of left ventricular volumes and function: a multicenter comparison of cardiac magnetic resonance imaging, cine ventriculography, and unenhanced and contrast-enhanced 2-dimensional and 3-dimensional echocardiography. J Am Soc Echocardiogr. 2014;27(3):292-301. doi: 10.1016/j.echo.2013.12.005 [DOI] [PubMed] [Google Scholar]
- 32.Cohn PF, Horn HR, Teichholz LE, Kreulen TH, Herman MV, Gorlin R. Effects of angiographic contrast medium on left ventricular function in coronary artery disease—comparison with static and dynamic exercise. Am J Cardiol. 1973;32(1):21-26. doi: 10.1016/S0002-9149(73)80082-7 [DOI] [PubMed] [Google Scholar]
- 33.Telkova IL, Tepliakov AT. [Relationships between changes of coronary blood flow, energy metabolism of the myocardium, and hyperinsulinemia in patients with ischemic heart disease]. Kardiologiia. 2005;45(8):61-68. doi: 10.1093/bjaceaccp/mki012 [DOI] [PubMed] [Google Scholar]
- 34.Huber PJ. Robust estimation of a location parameter. Ann Math Statist. 1964;35(1):73-101. doi: 10.1214/aoms/1177703732 [DOI] [Google Scholar]
- 35.LVEF prediction during ACS using AI algorithm applied on coronary angiogram videos (CathEF). ClinicalTrials.gov identifier: NCT05317286. Updated March 7, 2023. Accessed April 13, 2023. https://www.clinicaltrials.gov/ct2/show/NCT05317286
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eMethods
eReferences
eFigure 1. Creation of Study Data Sets
eFigure 2. Confusion Matrices for CathEF Prediction of Binary Reduced LVEF (≤/>40%)
eFigure 3. Bland-Altmann Plots Examining Systematic Bias Between CathEF and TTE
eFigure 4. Box Plots of the Difference Between CathEF Prediction of LVEF and Human Calculated LVEF on TTE, Stratified According to the Underlying LVEF Strata
eFigure 5. Guided Grad-CAM Highlighting Areas of the Image Used to Predict LVEF <40% or LVEF ≥40
eFigure 6. Standard Fluoroscopy Image From a Left Ventriculogram Procedure
eTable 1. Definition of the Angiographic Projection Angle Classes
eTable 2. Hyperparameters Searched for CathEF
eTable 3. Performance of CathEF to Identify LVEF ≤40/>40% by Transthoracic Echo in Strata of the Test Data Set
eTable 4. Performance of CathEF to Identify LVEF≤40/>40% by Transthoracic Echo in Strata of the UOHI External Validation Data Set
eTable 5. Performance of CathEF to Identify LVEF<50/≥50% by Transthoracic Echo in Strata of the Test Data Set
eTable 6. Baseline Characteristics of the External Validation Cohort
eTable 7. Development Set Performance of Alternative Training Schemes and Hyperparameters
Data Sharing Statement