Preoperative risk prediction of major cardiovascular events in noncardiac surgery using the 12-lead electrocardiogram: an explainable deep learning approach

Abstract

Background

The Revised Cardiac Risk Index (RCRI) only modestly predicts major adverse cardiovascular events after noncardiac surgery. We investigated whether preoperative 12-lead ECGs analysed with deep learning could improve risk prediction.

Methods

In a retrospective cohort of 37 081 adults undergoing major noncardiac surgery (2008–2019, MIMIC-IV database), convolutional neural networks were trained to predict in-hospital myocardial infarction, in-hospital mortality, and a composite of in-hospital myocardial infarction, in-hospital stroke, and 30-day mortality. Models using ECG waveforms alone were compared with fusion models that combined ECGs with 34 routinely collected clinical variables. The primary outcome was discrimination, assessed by the area under the receiver-operating characteristic curve (AUROC) with 10-fold cross-validation and permutation tests vs the RCRI. A generative counterfactual framework provided waveform-level explanations.

Results

The fusion model yielded an AUROC=0.858 (95% confidence interval [95% CI], 0.845–0.872) for myocardial infarction, AUROC=0.899 (95% CI, 0.889–0.908) for in-hospital mortality, and AUROC=0.835 (95% CI, 0.827–0.843) for the composite outcome. Fusion model AUROC values exceeded those of ECG-only models (P≤0.002) and the RCRI (myocardial infarction: P=0.001; composite: P<0.001). Counterfactual analysis highlighted prolonged QRS duration, low-voltage complexes, and ST-segment depression as electrophysiologic patterns that consistently correlated with higher predicted risk.

Conclusions

A multimodal deep-learning model that integrates preoperative ECG waveforms with routinely collected clinical data improves prediction of major adverse cardiovascular events, compared with the Revised Cardiac Risk Index. This fully automated approach provides explainable, patient-specific insights, and may improve perioperative risk stratification.

Editor’s key points.

• •Preassessment tools fail to predict many major adverse cardiovascular events after noncardiac surgery.
• •The authors investigated whether the preoperative 12-lead electrocardiograms (ECGs) analysed with deep learning could improve risk prediction for major adverse cardiovascular events.
• •In a retrospective cohort of 37 081 adults undergoing major noncardiac surgery, convolutional neural networks were trained to predict major adverse cardiovascular events.
• •The authors compared models using ECG waveforms alone with fusion (multimodal) models that combined preoperative ECGs with routinely collected clinical variables.
• •Fusion multimodal modelling was superior to ECG analysis alone, although ECG abnormalities were consistently associated with higher predicted risk.
• •Fusion multimodal modelling needs to be prospectively validated in other perioperative settings.

Despite advances in perioperative safety, adult patients undergoing noncardiac surgery still sustain major adverse cardiovascular and cerebrovascular events (MACCEs), including myocardial infarction (MI) and ischaemic stroke.1, 2, 3 These complications prolong hospitalisation,⁴ increase medical costs,⁵ and reduce quality of life.⁶ Accurate preoperative risk stratification⁷ may reduce MACCE,⁸ through medical optimisation,⁹ and modifying surgical and anaesthetic approaches,¹⁰ including perioperative monitoring.¹¹ Moreover, appropriately risk stratified patients can make informed decisions before surgery.¹² However, existing preoperative risk tools, such as the Revised Cardiac Risk Index (RCRI), offer only modest discrimination.¹³ Therefore, there is an unmet need for more accurate tools for risk stratification.

The 12-lead ECG is widely used to diagnose a range of cardiac conditions,^14,15 and may also be valuable in predicting future events such as paroxysmal atrial fibrillation16, 17, 18 or sudden cardiac death.^19,20 This research has leveraged very large datasets of 12-lead ECGs to train supervised machine learning algorithms. The central conjecture in these studies is that ECG waveforms contain previously unidentified predictive features which might be characterised as latent because they are not recognisable even by trained clinical practitioners. Convolutional neural networks applied to 12-lead ECG can match expert performance for arrhythmia classification.^21,22 However, these classification tasks do not address the problem of predicting perioperative risk, for which limited endpoints, lack of multimodal fusion, and lack of explainability have limited their clinical generalisability.²³

Model explainability in this space is dominated by modelling methods that assign an importance score to each time point, telling us where the model looked, but—because they leave the original waveform unchanged—not revealing how the signal would need to change to alter the prediction. For noisy, high-frequency ECG traces, the resulting data can limit clinical interpretability.²⁴ Counterfactual approaches synthesise an alternative, minimally modified ECG that provides a concrete, patient-specific trajectory from the current physiological state to one associated with a different risk class. Counterfactual ECGs—typically via generative adversarial or diffusion frameworks—have focused thus far on arrhythmia or MI, rather than perioperative cardiovascular risk.^25,26

Here, we explored whether counterfactual explanations to perioperative endpoints could overcome previous limitations by fusing a waveform convolutional neural network (CNN) with 34 routine clinical variables in a large cohort, where we trained a deep learning (DL) model to predict the risk of major adverse cardiovascular events in major noncardiac surgery.

Methods

Study design

In this retrospective observational study, data were extracted from the open access MIMIC-IV database (version 2.2),^25,26 which includes 299 712 patients across 431 231 admissions to Beth Israel Deaconess Medical Center (BIDMC; Boston, MA, USA) between 2008 and 2019 (Fig. 1). The study qualified for institutional review board exemption from the BIDMC Institutional Review Board.

Fig 1.

Task of interest, selection criteria, and model architecture. (a) Schematic of the task of interest, which is to predict adverse postoperative outcomes using preoperative ECGs. (b) Flowchart used for patient selection. (c) Model architecture. A convolutional neural network backbone is used to make predictions based on preoperative ECGs alone (WF model) or in combination with routinely collected clinical variables and patient characteristics available before surgery (fusion model). RCRI, Revised Cardiac Risk Index.

Inclusion criteria

We included adult patients undergoing major noncardiac surgery identified via procedure International Classification of Diseases (ICD) codes recorded during their stay, using Procedure Classes groupings provided by the Healthcare Cost and Utilization Project (HCUP).²⁷ We used the Clinical Classification Software (CCS), provided by HCUP (Rockville, MD, USA), to determine whether patients first underwent a major cardiac or noncardiac surgery (Fig. 1b). We only included ECGs obtained at least 1 day before surgery, because MIMIC does not conclusively determine if the ECG was administered before the procedure or after it.

Primary endpoints

The primary objective was to predict major adverse postoperative events using preoperative 10-s 12-lead ECG. We identified three postoperative endpoints of interest: MI, in-hospital mortality (IHM), and a composite of stroke, MI, and mortality (composite). MI and stroke were determined using ICD-9 and ICD-10 codes (Supplementary Table S1). IHM was determined from a combination of in-hospital records and state death records.

Evaluation

We adopted two evaluation approaches. In the first, we used a standard k-fold validation with 10 folds. We randomly divided the dataset into 10 partitions, split by subject (so that no subject could be in both the train and test partitions). Then, for each partition, we trained on the nine other partitions and evaluated on the held-out test partition. Statistical analyses were then calculated across the 10 test folds. In the second evaluation method, we trained and tested our method on chronologically distinct cohorts. Specifically, we trained our model on patients admitted between 2008 and 2016 and tested our model on a cohort admitted between 2017 and 2019. The goal of the second evaluation was to simulate a single-site sequential validation.²⁸ That is, we trained the model on earlier admissions from the same institution and evaluated it on a chronologically later, independent cohort from the same site, mirroring how a deployed model would perform when prospectively used on subsequent patients.

Models to predict outcomes

We developed two models to predict outcomes (Fig. 1c). Our first model examined whether there are latent factors within the ECG that are predictive of the outcomes of interest. The second model determined if the ECG contains information complementary to structured electronic healthcare record (EHR) data that increases the accuracy of prediction of postoperative outcomes as compared with structured data alone.

First, we developed a DL model to predict adverse postoperative outcomes in noncardiac surgical patients using ECG waveforms alone (WF model). The WF model used only preoperative ECGs to predict postoperative outcomes. We selected a 1D CNN as our model backbone, based in part on the architecture proposed by Attia and colleagues.²² This architecture takes in a raw, 10-s 12-lead ECG sampled at 500 Hz (i.e. a $\left[5000,12\right]$-dimensional array of voltages) and is trained to output an estimated probability of a given condition (for full details on the architecture of the network model and an explanation of the hyperparameter optimisation process, see Supplementary material, including Supplementary Table S3).

The second model was a fusion model that combines ECG waveform data with routinely collected clinical variables to predict these outcomes. We combined the CNN backbone with basic patient characteristics (age, sex), admission type (emergency, elective), each of the six components of the RCRI, and 26 indicators from the Elixhauser Comorbidity Score (ECS) identified by ICD-9 and ICD-10 codes (Supplementary material).²⁹

Model training

We implemented the waveform and fusion models in TensorFlow (version 2.15.0; Google, Mountainview, CA) and the counterfactual model in PyTorch (version 2.1.1; Meta, Menlo Park, CA). We trained models for up to 100 epochs using an Adam optimiser with an initial learning rate of 1×10⁻³. We randomly selected 10% of patients (and their corresponding ECGs) for validation and calculated the validation loss at each epoch. If the validation loss did not decrease for three consecutive epochs, we reduced the learning rate by a factor of 0.5. If the validation loss did not decrease for six epochs, training was terminated, and the weights corresponding to the minimum validation loss were restored. We applied minimal preprocessing to our data beyond the ‘housekeeping’ filters used before the data were stored. These initial filters include a 0.005–150 Hz analogue band-pass filter, a narrow 60-Hz notch filter, and the vendor’s built-in baseline-wander removal filter. ECGs were dropped if any entries within the ECG waveform were invalid (i.e. if any of the voltages were recorded as null values because of lead disconnection or data corruption, comprising less than 2% of all ECGs). In the fusion model, we applied standard scaling to age but left the rest of the binary variables unscaled.

Model performance

The primary metric of interest was area under the receiver-operating curve (AUROC). In addition, we also calculated the threshold-independent area under the precision-recall curve (AUPRC; Supplementary Fig. S1). For threshold-dependent measures (sensitivity, specificity, positive predictive value [PPV], negative predictive value [NPV], odds ratio [OR]), we set a risk score cutoff for high and low risk as $\hat{y}=0.05$; for RCRI, high-risk patients were defined by a composite RCRI score ≥2.^30,31 For each model, we calculated the risk score for all preoperative ECGs (e.g. if a patient had five preoperative ECGs, each was fed into the model at inference time to produce a risk score $\hat{y}$). When only including the ECG taken temporally closest to surgery, we observed changes in the area under the curve (AUC) of 0.01–0.02 across all outcomes. We then selected the maximum risk score across all preoperative ECGs for a given patient as that patient’s risk of an adverse postoperative event.

Model comparisons

Our primary comparison benchmark was the RCRI (Supplementary material). Each RCRI component was coded as a binary variable, with the sum equal to the RCRI score (0–6). In addition, we compared our WF and fusion models with one trained using Elixhauser Comorbidity Index (ECI) variables, and three CNN-based models from Hannun and colleagues,²¹ Ribeiro and colleagues,³² and Ouyang and colleagues,²³ trained using the same training and evaluation schemes and data as our model. In addition, we implemented a non-DL-based ECG classifier based on the machine-generated features from the ECG hardware (see Supplementary material).

Explainability

To identify the characteristics relevant to classification decisions, we used a generative approach which creates counterfactual ECGs. Briefly, after the training paradigm previously described, we used a classifier that, given an input ECG, produced a risk score (e.g. the probability of postoperative MI). Our counterfactual model intervened on this input waveform by introducing subtle, physiologically plausible modifications to the underlying morphology such that it elicited a different, user-defined risk score $\delta \in \left[0,1\right]$ (Supplementary material). In this way, we created counterfactual ECGs at varying risk levels, allowing for the identification of features driving classifier decisions. By comparing high- and low-risk counterfactuals, we could capture, visually and empirically, morphological characteristics which the algorithm considers in accomplishing the classification task.

Statistical analysis

Our primary performance metric of interest was AUROC; unless otherwise noted, we follow the format AUROC (95% confidence interval [95% CI] lower–upper), where CIs are determined via a bootstrapping approach. To generate CIs, we rely on a bootstrapping approach with 10 000 iterations. In comparing our approach with the RCRI, we also compute the Net Reclassification Index³³ (NRI) between our models and RCRI. Briefly, the NRI measures how well a new model (e.g. WF and fusion models) reclassifies subjects relative to a baseline (RCRI). The NRI quantifies the correct upward or downward movement in risk categories for individuals, and its value can range from −2 to 2. Positive values indicate an improvement in classification with the new model, whereas negative values suggest a deterioration (Supplementary material).

For statistical tests, we apply a permutation test with 10 000 iterations, followed by a Bonferroni correction³⁴ to establish statistical significance given multiple comparisons.

Results

Subject characteristics

The analysis was conducted on 37 081 adult patients undergoing noncardiac surgery, with a mean age of 64 yr (18–91 yr; 49.4% male). Comorbid conditions were common, including ischaemic heart disease, diabetes mellitus requiring insulin, and elevated creatinine (Table 1). Approximately one-third of procedures were high-risk intraperitoneal, intrathoracic, suprainguinal vascular surgery, or a combination of these.³⁰

Table 1.

Population sample characteristics. Shown are patient characteristics and outcome and RCRI components for the overall sample, and the temporally stratified one (see Methods). Values for the outcomes and RCRI indicate the percentage of patients who experienced the outcome or presented with the risk factor, respectively. CHF, congestive heart failure; CVD, cardiovascular disease; IHD, ischaemic heart disease; IHM, in-hospital mortality; MI, myocardial infarction; NHPI, Native Hawaiian and Pacific Islander; RCRI, Revised Cardiac Risk Index. ∗Significant differences P<10⁻³ between the development cohort and test cohort, based on a normal test for proportion differences in unpaired samples.²⁹

	Total	Development cohort (pre-2017)	Test cohort (2017–2019)
Patients (n)	37 081	33 837	3244
Age, yr (mean)	64.4	64.5	62.9
Male sex (%)	49.4	49.4	48.7
Race and ethnicity (%)
White	72.6	72.7	71.2
Black	12.2	12.5	8.5
Hispanic/Latino	4.5	4.6	4.0
Asian	3.0	2.9	3.8
Am. Ind./Al. Nat.	0.2	0.2	0.1
NHPI	0.1	0.1	0.1
Unknown	7.5	7.1	12.2
Outcomes (%)
MI	1.6	1.5	2.6∗
IHM	2.2	2.1	3.1∗
Composite	6.6	6.4	8.7∗
RCRI (%)
Previous IHD	16.0	16.9	6.7∗
Previous CHF	10.8	11.4	4.3∗
Previous CVD	6.6	6.9	3.5∗
High-risk procedure	36.7	35.3	52.2∗
Diabetes mellitus requiring insulin	26.0	26.6	19.9∗
Creatinine >2	13.3	13.0	9.3∗

Model performance and cross validation

The fusion model demonstrated superior performance across all three outcomes, achieving an AUROC of 0.858 (0.845–0.872), 0.899 (0.889–0.908), and 0.835 (0.827–0.843) for MI, IHM, and the composite outcome, respectively (Table 2 and Fig. 2a–d). This performance was superior to the WF model and RCRI across all three outcomes. We found that the WF model performed well for predicting MI (AUROC: 0.791 [0.771–0.809]) and IHM (0.761 [0.744–0.777]), but only moderately for the composite outcome (0.717 [0.707–0.728]). The WF model was only superior for the composite outcome, compared with RCRI (P=1×10ˆ[−4]) and demonstrated a close calibration with the ground truth (Supplementary Fig. S2).

Table 2.

Cross-validation model comparison. This table shows the performance of different models across the three outcomes of interest. The performance of our model (both waveform-only and fusion) with the SOC baseline (RCRI), a model trained using only Elixhauser components, three other convolutional neural network architectures, and a traditional ML model based on ECG features and an XGBoost classifier.³⁵ Threshold-dependent measures (‘sensitivity’…‘OR’) are calculated using a threshold of 0.05 for the ML algorithms, and a threshold of 2 for RCRI. Results at a threshold of 0.01 and 0.1 are shown in Supplementary Table S2. Bolded values indicate the highest performance for each metric; underlined values indicate the second-highest performance. AUPRC, area under the precision-recall curve; AUROC, area under the receiver operator curve; IHM, in-hospital mortality; MI, myocardial infarction; NPV, negative predictive value; PPV, positive predictive value; OR, odds ratio; RCRI, Revised Cardiac Risk Index; WF, ECG waveforms alone.

Model	AUROC	AUPRC	Sensitivity	Specificity	PPV	NPV	OR
MI
WF model	0.791 (0.771–0.809)	0.067 (0.056–0.082)	0.558 (0.518–0.599)	0.857 (0.854–0.861)	0.059 (0.053–0.065)	0.992 (0.991–0.993)	7.21 (6.14–8.51)
Fusion	0.858 (0.845–0.872)	0.122 (0.103–0.146)	0.580 (0.540–0.620)	0.900 (0.897–0.903)	0.085 (0.076–0.093)	0.993 (0.992–0.993)	11.45 (9.78–13.44)
RCRI	0.766 (0.747–0.784)	0.051 (0.044–0.060)	0.705 (0.668–0.742)	0.726 (0.722–0.731)	0.040 (0.036–0.043)	0.994 (0.993–0.994)	6.14 (5.15–7.37)
Elixhauser components	0.790 (0.771–0.808)	0.072 (0.061–0.087)	0.481 (0.440–0.522)	0.891 (0.888–0.894)	0.066 (0.059–0.074)	0.991 (0.990–0.992)	7.15 (6.08–8.38)
Hannun and colleagues21	0.771 (0.751–0.791)	0.060 (0.051–0.073)	0.502 (0.461–0.542)	0.869 (0.866–0.873)	0.058 (0.051–0.064)	0.991 (0.990–0.992)	6.36 (5.41–7.46)
Ribeiro and colleagues32	0.750 (0.730–0.770)	0.062 (0.052–0.078)	0.420 (0.380–0.459)	0.880 (0.876–0.883)	0.053 (0.047–0.059)	0.990 (0.988–0.991)	5.06 (4.31–5.93)
PreOpNet23	0.715 (0.694–0.736)	0.037 (0.033–0.044)	0.913 (0.889–0.935)	0.264 (0.260–0.269)	0.019 (0.018–0.021)	0.995 (0.993–0.996)	3.70 (2.86–5.08)
Machine features	0.687 (0.665–0.710)	0.038 (0.033–0.046)	0.327 (0.289–0.366)	0.887 (0.884–0.890)	0.044 (0.038–0.050)	0.988 (0.987–0.989)	3.69 (3.09–4.36)
IHM
WF model	0.761 (0.744–0.777)	0.071 (0.063–0.082)	0.582 (0.549–0.615)	0.788 (0.784–0.792)	0.059 (0.054–0.064)	0.988 (0.987–0.989)	4.92 (4.31–5.64)
Fusion model	0.899 (0.889–0.908)	0.192 (0.171–0.218)	0.727 (0.695–0.756)	0.884 (0.881–0.888)	0.125 (0.116–0.134)	0.993 (0.992–0.994)	17.92 (15.46–20.91)
RCRI	0.736 (0.721–0.752)	0.050 (0.045–0.057)	0.657 (0.625–0.689)	0.728 (0.723–0.733)	0.052 (0.048–0.056)	0.989 (0.988–0.991)	4.91 (4.28–5.68)
Elixhauser components	0.834 (0.819–0.850)	0.136 (0.120–0.156)	0.611 (0.577–0.644)	0.885 (0.881–0.888)	0.107 (0.099–0.116)	0.990 (0.989–0.991)	10.87 (9.49–12.46)
Hannun and colleagues21	0.746 (0.729–0.762)	0.063 (0.056–0.072)	0.464 (0.430–0.499)	0.855 (0.851–0.858)	0.068 (0.061–0.074)	0.986 (0.985–0.987)	4.81 (4.21–5.51)
Ribeiro and colleagues36	0.723 (0.706–0.740)	0.058 (0.051–0.067)	0.428 (0.394–0.462)	0.839 (0.836–0.843)	0.057 (0.051–0.063)	0.985 (0.983–0.986)	3.73 (3.25–4.27)
PreOpNet23	0.659 (0.640–0.678)	0.045 (0.039–0.052)	0.974 (0.963–0.984)	0.080 (0.077–0.082)	0.023 (0.022–0.025)	0.993 (0.990–0.996)	3.24 (2.22–5.36)
Machine features	0.682 (0.664–0.700)	0.046 (0.041–0.052)	0.340 (0.309–0.373)	0.859 (0.856–0.863)	0.052 (0.046–0.058)	0.983 (0.981–0.984)	3.04 (2.64–3.50)
Composite
WF model	0.717 (0.707–0.728)	0.162 (0.152–0.175)	0.845 (0.830–0.859)	0.420 (0.415–0.426)	0.093 (0.089–0.097)	0.975 (0.972–0.977)	3.68 (3.317–4.11)
Fusion model	0.835 (0.827–0.843)	0.304 (0.286–0.323)	0.855 (0.841–0.868)	0.645 (0.640–0.650)	0.145 (0.140–0.151)	0.984 (0.983–0.986)	9.28 (8.32–10.40)
RCRI	0.668 (0.657–0.678)	0.114 (0.107–0.121)	0.516 (0.496–0.536)	0.736 (0.731–0.741)	0.121 (0.115–0.127)	0.956 (0.953–0.958)	2.73 (2.52–2.94)
Elixhauser components	0.773 (0.763–0.783)	0.256 (0.239–0.274)	0.831 (0.816–0.845)	0.520 (0.515–0.526)	0.109 (0.105–0.113)	0.978 (0.975–0.980)	4.85 (4.38–5.37)
Hannun and colleagues21	0.712 (0.702–0.723)	0.150 (0.140–0.161)	0.809 (0.793–0.825)	0.474 (0.469–0.479)	0.098 (0.094–0.102)	0.972 (0.970–0.975)	3.54 (3.21–3.92)
Ribeiro and colleagues36	0.685 (0.674–0.696)	0.145 (0.135–0.156)	0.752 (0.735–0.769)	0.499 (0.493–0.504)	0.096 (0.092–0.100)	0.966 (0.963–0.969)	2.82 (2.59–3.09)
PreOpNet23	0.674 (0.663–0.684)	0.121 (0.114–0.129)	0.999 (0.998–1.000)	0.003 (0.002–0.003)	0.066 (0.064–0.069)	0.979 (0.948–1.000)	3.204 (1.25–5.16)
Machine features	0.651 (0.640–0.663)	0.116 (0.109–0.125)	0.744 (0.726–0.761)	0.461 (0.455–0.466)	0.089 (0.085–0.093)	0.962 (0.959–0.965)	2.34 (2.15–2.56)

Fig 2.

Model performance. (a) Schematic of the k-fold validation scheme for the main results, where data are split into 10 folds (by patient ID). Train folds are indicated in blue, and test folds in purple. Test folds are combined to evaluate the final performance of the model, as shown by the ROC curves (with AUROC values inset) in predicting postoperative (b) MI, (c) IHM, and (d) composite outcome. Shaded areas denote 95% confidence intervals from 10 000 bootstrapped samples. (e) Number of samples in the temporal stratification analysis. Blue bars are used to train the model (corresponding to patients admitted between 2008 and 2016), and samples corresponding to the orange bar (2017–2019) are used to evaluate it. The corresponding performance is shown in panels (f–h). Corresponding AUPRC plots are shown in Supplementary Figure S1. AUPRC, area under the precision-recall curve; AUROC, area under the receiver operator curve; IHM, in-hospital mortality; MI, myocardial infarction; OR, odds ratio; RCRI, Revised Cardiac Risk Index; WF, ECG waveforms alone.

Model comparisons

The WF model performed similarly, or outperformed, previous models,^21,32 for MI and the composite outcome, but not IHM (Table 2). It also outperformed PreOpNet²³ across all outcomes. In addition, the WF model outperformed a non-CNN-based one, using nine ECG features extracted via the Marquette 12SL algorithm³⁷ and input into an XGBoost³⁸ classifier.

Additional analyses

We computed the net reclassification improvement (NRI) between our models and the RCRI. For MI, NRI was 0.412 (0.234–0.587) and 0.706 (0.537–0.867) for the WF and fusion models, respectively. For IHM, NRI was lower for both the WF model (0.174 [0.033–0.316]) and the fusion model (0.614 [0.481–0.746]). The NRI for the composite outcome was similar between the WF and fusion models, with values of 0.267 (0.210–0.324) and 0.292 (0.237–0.347), respectively (Supplementary Table S4). Sex and race did not impact model performance, but models performed better for younger patients (Supplementary material).

Additional modelling in 15 578 individuals showed that our model was also valuable in both non-emergency and elective surgeries (Supplementary material). We also found that our model performed well for ECGs coded as ‘normal’ by commercial software generating automated reports (Supplementary material). When we simulated a single-site sequential validation, using a separate test cohort of 3244 individuals for whom data were collected after the development cohort (Table 1), the fusion model again outperformed the WF model across all three outcomes (Supplementary Table S5).

Explainability

Our counterfactual model (Fig. 3a and b) reproduced morphological characteristics of six common cardiovascular conditions (Fig. 3c). In addition to visual examples from single patients, our method replicated known morphological changes representative of four common conditions (Fig. 3d). In the exploratory analysis using the prediction task, we found that counterfactual ECGs can visually illustrate the morphological differences associated with varying levels of risk for postoperative MI. For example, a low-risk ECG exhibits normal QRS duration and standard timing intervals, whereas a high-risk ECG exhibits a slightly prolonged QRS duration along with alterations in other morphological features, such as an absence of P-waves and moderate depression of the ST-complex (Fig. 4a). In the feature analysis, we found a positive correlation between risk score and QRS duration, and a negative correlation between risk score and the other six timing features (Fig. 4b).

Fig 3.

Diagnostic explainability. (a) Schematic for the counterfactual generation model. (b) Schematic of our counterfactual validation approach. (c) Plotted are example ECG strips from the first 2.5 s of the waveform from lead II for the conditions labelled above each column of strips. Red arrows indicate morphological changes in the high-risk CF that are representative of the condition of interest. (d) Morphological differences relevant to four of the seven conditions of interest across the entire diagnostic test set to verify the model reproduces morphological changes characteristic of conditions. ‘True -’ indicates the distribution for patients without the condition, and ‘True +’ indicates the distribution of patients with the condition. Then, for all patients (both true-positives and true-negatives), we simulate low- and high-risk CF and extract the corresponding measurements. CF, counterfactuals. ∗∗P<0.0001.

Fig 4.

Prognostic explainability. (a) Low (top) and high (bottom) counterfactual ECG strips for a single patient. Within each strip, the first three subplots show the first 2.5 s of electrical activity (with lead numbers inset), and the bottom row shows the full 10-s ECG for lead II. (b) Heatmap of the correlation between counterfactual risk score (δ) and each of seven standard ECG features. See the Supplementary material for a description of each feature.

Discussion

Using a large, publicly available perioperative dataset, we created a deep-learning algorithm that predicted cardiovascular risk from routine 12-lead ECGs. The waveform-only convolutional model—trained solely on preoperative ECG signals—outperformed the RCRI for all three prespecified outcomes: postoperative MI, IHM, and the composite of stroke, MI, and 30-day mortality. Within the same dataset and training paradigm, the WF model also surpassed the best previously published CNN-based ECG approach. Incorporating 34 routinely collected clinical variables into a fusion architecture yielded still higher discrimination across every endpoint. We also found, by developing counterfactual analysis, that patients with prolonged QRS complexes are at a higher risk of adverse postoperative outcomes, consistent with prolonged QRS in conditions that indicate cardiac pathology.^39,40 The negative correlations with the other six timing features may similarly reflect abnormal cardiac electrophysiology, potentially increasing the likelihood of complications.^35,41,42

The ECG is generally regarded as a diagnostic tool for diagnosing pre-existing or acutely developing cardiac pathology. Here, we challenged this view by demonstrating the ECG has significant predictive ability in the perioperative setting. We evaluated our approach on a large cohort of 37 081 hospitalised patients, and on a temporally stratified cohort of 3227 patients from a chronologically later period. Our model performed well using both paradigms, even allowing for significant population differences between the temporally stratified groups. It is worth noting that the outcome classes were highly imbalanced, with events occurring in fewer than 6% of patients across all endpoints. The model’s consistent discrimination despite this imbalance suggests that it captures clinically meaningful signal rather than simply reflecting class prevalence; however, the model may be sensitive to changes if the previous probability of the outcome changes (e.g. if deployed in a clinical environment with substantially higher or lower postsurgical complication rates).

We also introduce a novel explanatory approach for waveform prediction models based on the generation of counterfactuals. Most DL models are effectively ‘black boxes’, with complex internal mechanisms that are not readily intelligible. The current standard approach to explainability DL ECG models mostly relies on saliency-based gradient methods⁴³ or model-agnostic perturbation techniques, such as locally interpretable model-agnostic explanations.⁴⁴ Although these techniques show, roughly, ‘where’ the model is looking, they often fall short of explaining ‘what’ it is looking at—for example, the specific morphological features influencing the model’s predictions—particularly when these features are subtle or not previously well established. Building on recent generative methods for explainability,45, 46, 47 our research introduces a novel approach using counterfactual explanations to enhance the interpretability of ECG classification models. This method, inspired by foundational work in counterfactual reasoning,^48,49 involves generating ‘counterfactual’ ECGs—modified versions of the original ECG that are minimally altered to change the predicted risk level of an outcome, such as MI. By adjusting a control parameter, $\mathrm{\delta }$, which represents the desired risk level, our model can produce ECGs that not only differ in their risk prediction but also highlight the specific changes in waveform morphology responsible for the different predictions.⁴³ This approach allows a visual and empirical exploration of how various morphological features contribute to the model’s decisions, offering a more nuanced understanding than that provided by existing methods. Beyond demonstrating the feasibility of counterfactual ECG generation, we observed that high-risk counterfactuals consistently displayed (a) prolonged QRS duration, (b) attenuated QRS amplitude (low-voltage complexes), (c) ST-segment depression or T-wave inversion, and (d) loss or blunting of P-waves. Of these, QRS prolongation had the strongest positive correlation with predicted risk, whereas shorter RR, PR, and QTc intervals were negatively correlated. When such features are present on a preoperative ECG, even in patients deemed low risk by traditional indices, our findings have several potential clinical implications, including: (a) additional evaluation for ventricular conduction abnormalities, occult ischaemia, or structural heart disease; (b) optimisation of modifiable factors (e.g. electrolyte imbalances, rate control, blood pressure control); and (c) the level of intraoperative haemodynamic monitoring.

Our work has several limitations. The lack of external validation raises questions of generalisability across different patient characteristics and clinical settings. However, we note that our model continues to perform well in a temporally distinct sample, despite statistically significant differences in outcome prevalence and comorbidities between development and test cohorts. The reliance on ICD codes for outcome identification could introduce biases owing to inaccuracies in coding practices. As highlighted in various studies,^50,51 the veracity of ICD coding can be compromised by multiple factors (e.g. experience of medical record coders, incomplete physician documentation), potentially affecting the fidelity of our training labels. However, we note that our model performs similarly for MI and IHM; whereas the former may be subject to label noise, the latter is not, given the unambiguity of mortality and importance of accurate reporting. This suggests that our models capture latent features within the ECG waveform that are predictive of patient outcomes.

In practical terms, the fusion model’s inputs—all drawn from routinely collected 12-lead ECGs and variables routinely stored in the electronic health record—mean that it can run unobtrusively in the background, deliver a risk score within seconds, and flag high-risk patients without adding workflow burden or hardware costs; its computation time on a standard computer (less than 0.1 s per patient) is well below the cadence at which new ECGs arrive, making real-time deployment feasible. Nevertheless, several caveats temper these advantages. First, our counterfactual engine requires the user (e.g. an anaesthetist) to specify a ‘desired’ lower-risk score, which introduces subjectivity and may limit scalability until automated target-setting heuristics are developed. Second, the counterfactuals manipulate a static preoperative snapshot; because patients are not yet manifesting acute complications, the edits illustrate hypothetical waveform changes that could shift risk rather than real-world pathophysiology, so they should be interpreted as educational guides rather than prescriptive treatment cues. Additional work will be needed before counterfactual explanations can fully support bedside decision-making.

In summary, our findings suggest that for patients undergoing noncardiac surgery, the 12-lead ECG is a powerfully discriminative prognostic tool. Waveform-based predictive features are not immediately recognisable and can be revealed using DL algorithms. Future research should focus on external validation of this model and prospective implementation in real-world clinical settings. Multi-institutional studies are required to test model transportability and recalibration across diverse patient populations.

Authors’ contributions

Conception and design: RDS, JLG, COT, CH

Core model design and implementation: CH, AP, AA, JY, XC

Statistical analysis: CH, SS, SR

Manuscript drafting: CH, AP, AA, SS, SR

Reading and revision of the manuscript: all authors

Data availability statement

We will publicly release the code relevant to replicating the core analyses upon acceptance of the manuscript. The MIMIC-IV data are publicly available. In addition, this study adheres to the TRIPOD-AI checklist,⁵² which can be found in the Supplementary material.

Funding

National Science Foundation Graduate Research Fellowship (grant no. DGE2139757 to CH and SS).

Declaration of interest

The authors declare that they have no conflicts of interest.

Handling Editor: Gareth Ackland

Appendix A. Supplementary data

The following is the Supplementary data to this article: