Electrocardiogram-based deep learning to predict mortality in paediatric and adult congenital heart disease

Abstract

Background and Aims

Robust and convenient risk stratification of patients with paediatric and adult congenital heart disease (CHD) is lacking. This study aims to address this gap with an artificial intelligence-enhanced electrocardiogram (ECG) tool across the lifespan of a large, diverse cohort with CHD.

Methods

A convolutional neural network was trained (50%) and tested (50%) on ECGs obtained in cardiology clinic at the Boston Children’s Hospital to detect 5-year mortality. Temporal validation on a contemporary cohort was performed. Model performance was evaluated using the area under the receiver operating characteristic and precision-recall curves.

Results

The training and test cohorts composed of 112 804 ECGs (39 784 patients; ECG age range 0–85 years; 4.9% 5-year mortality) and 112 575 ECGs (39 784 patients; ECG age range 0–92 years; 4.6% 5-year mortality from ECG), respectively. Model performance (area under the receiver operating characteristic curve 0.79, 95% confidence interval 0.77–0.81; area under the precision-recall curve 0.17, 95% confidence interval 0.15–0.19) outperformed age at ECG, QRS duration, and left ventricular ejection fraction and was similar during temporal validation. In subgroup analysis, artificial intelligence-enhanced ECG outperformed left ventricular ejection fraction across a wide range of CHD lesions. Kaplan–Meier analysis demonstrates predictive value for longer-term mortality in the overall cohort and for lesion subgroups. In the overall cohort, precordial lead QRS complexes were most salient with high-risk features including wide and low-amplitude QRS complexes. Lesion-specific high-risk features such as QRS fragmentation in tetralogy of Fallot were identified.

Conclusions

This temporally validated model shows promise to inexpensively risk-stratify individuals with CHD across the lifespan, which may inform the timing of imaging/interventions and facilitate improved access to care.

Structured Graphical Abstract

Structured Graphical Abstract.

A large and diverse paediatric and adult congenital heart disease cohort was used to train and test an artificial intelligence-enhanced electrocardiogram algorithm to accurately predict 5-year mortality across a range of congenital heart disease lesions. In an effort to interpret model behaviour, model explainability analysis was performed. AI-ECG, artificial intelligence-enhanced electrocardiogram; ASD, atrial septal defect; CAVC, complete atrioventricular canal defect; CNN, convoluted neural network; CoA, coarctation of the aorta; DORV, double outlet right ventricle; ECG, electrocardiogram; HLHS, hypoplastic left heart syndrome; LV, left ventricle; LVEF, left ventricular ejection fraction; PA, pulmonary atresia; RV, right ventricle; TAPVR, totally anomalous pulmonary venous return; TGA, transposition of the great arteries; ToF, tetralogy of Fallot; VSD, ventricular septal defect.

See the editorial comment for this article ‘Expanding artificial intelligence to understudied populations: congenital heart disease as the next frontier', by E.K. Oikonomou and R. Khera, https://doi.org/10.1093/eurheartj/ehae737.

Introduction

With recent medical and surgical advancements, most children born with congenital heart diseases (CHDs) now survive into adulthood.¹ In the USA alone, it is estimated that approximately 2.4 million individuals are living with CHD, including approximately 1 million children and 1.4 million adults.² These conditions require lifelong follow-up involving frequent clinic visits and cardiac testing with significantly higher associated costs throughout the lifespan.^3–5 The necessity to facilitate the management of this growing complex population with a wide range of structural anomalies and distinct long-term consequences has motivated efforts to risk stratify CHD patients across the lifespan.⁶ Doing so may reduce a significant portion of unnecessary and expensive testing, while simultaneously identifying individuals that might benefit from closer monitoring or intervention thereby improving clinical outcomes.

However, the development of robust risk prediction models—particularly in CHD—faces multiple challenges including a paucity of big data, limited data sources, reliance on expensive imaging modalities (with an associated need for paediatric cardiologist expertise), and suboptimal model performance.⁶ This premise motivated multiple recent calls for artificial intelligence applications to improve risk stratification in CHD.^6,7

Major adverse outcomes in this unique population often relate to ventricular dysfunction or arrhythmias, and extraction of conventional electrocardiogram (ECG) features such as QRS duration in CHD [e.g. tetralogy of Fallot (ToF)] has shown value in risk stratification.⁸ In addition, deep learning-based artificial intelligence-enhanced ECG (AI-ECG) algorithms show promise for diagnostic and prognostic applications in adults,^9–15 making it similarly conceivable that AI-ECG may aid risk stratification also in the CHD population. However, there remains a paucity of available AI-ECG applications to congenital cardiology¹⁶ given (i) the absence of extensive data sets, impeding similar research; (ii) AI-ECG algorithms derived from adults with structurally normal hearts would be expected to have poor generalizability to CHD cohorts. To this end, there remain (to our knowledge) no AI-ECG algorithms to predict mortality in CHD.

In this work, we leverage an exhaustive electronic database at a large congenital heart centre to address this gap. To do so, a convolutional neural network was trained to predict 5-year mortality using >100 000 ECGs on nearly 40 000 patients and tested on an equally sized cohort as well as on a contemporary cohort. Subgroup analysis defined model performance within a range of specific CHDs. Survival analysis investigated longer-term survival for patients. Finally, saliency mapping and median waveform analysis provided lesion-specific model explainability.

Methods

Our study adheres to the TRIPOD + AI guidelines.¹⁷

Study population and patient assignment

Patient data were utilized from the Boston Children’s Hospital between 1990 (the earliest available digitized ECG in the clinic) until June 2018 (to minimize right censoring in predicting 5-year mortality). Inclusion criteria consisted of any patient presenting to the cardiology clinic at the Boston Children’s Hospital with at least one ECG performed. Temporal validation for predicting 1-year mortality was performed on any patient presenting to the cardiology clinic at the Boston Children’s Hospital with at least one ECG performed from July 2018 to July 2022 (to minimize right censoring in predicting 1-year mortality). All data were retrieved from an internal database in November 2023.

Similar to prior work,^13,16 a group stratified design was used to partition the main cohort at the patient level such that all ECGs for a given patient were restricted to either the training or internal testing cohort. Patients were randomly partitioned 50:50 into training and internal testing cohorts. Patients from the training data set were excluded from the temporal validation cohort.

Data retrieval

Raw ECG signals were exported from the MUSE ECG data management system (GE Healthcare, Chicago, IL, USA), which contains 1D vectors of data (sampling rate of 250 Hz for 10-s duration) for each lead (I, II, and V1–6). From these vectors, Einthoven’s law¹⁸ and the Goldberger equation¹⁹ were implemented to obtain leads III, aVF, aVL, and aVR. Other data retrieved from an internal database at the Boston Children’s Hospital include age, sex, and physician-reviewed ECGs measurements (e.g. QRS interval, QRS axis, T axis, P axis, PR interval, QT interval, QTc, and heart rate).

Patients with known CHD, cardiomyopathy, pre-excitation syndromes, and channelopathies were identified based on the institutional Fyler coding system,²⁰ which has been mapped into the International Paediatric and Congenital Cardiac Code International Classification of Diseases-11 nomenclatures.²¹

Quality control and data pre-processing

An ECG was discarded if any lead was not 2500 samples long or if any lead recording had missing lead information. After quality control, a high pass filter was utilized²² to account for recording errors (e.g. baseline wander and electrical interference) with cut-off frequency 0.8 Hz, rejection band 0.2 Hz, ripple in passband 0.5 dB, and attenuation in rejection band 40 dB. Finally, the ECG was then trimmed to 2048 samples (∼8 s) to facilitate conveniently working with convolutional neural networks.

Definition of outcomes

The primary outcome was 5-year mortality after an ECG. All-cause mortality was used as the primary outcome given (i) heart failure is the most common cause of death in adult CHD²³ and (ii) it aligns with previous AI-ECG work^14,15 and recent CHD work.²⁴ To obtain all-cause mortality, date of death was retrieved from an internal institutional database. Secondary outcomes include 1-, 2-, 3-, and 4-year mortality after an ECG. For temporal validation, 1-year mortality was used to minimize right censoring.

To perform a secondary Kaplan–Meier survival analysis for time to mortality, we obtained the date last known alive within the internal database (i.e. last recorded event in the institutional database for a given patient).

Model selection, architecture, and training

Similar to our prior work,¹⁶ the AI-ECG model was developed solely on the training set, which was further partitioned 95% for training and 5% for validation to perform hyperparameter tuning. 12 × 2048 ECG samples were used as inputs to a convolutional neural network that is similar to the residual network previously described²⁵ that is adapted for unidimensional signals. A diagram and details of the architecture used in this study are shown in Mayourian et al.¹⁶ The output of the last block is fed into a fully connected layer with a sigmoid activation function as the outcomes (1-, 2-, 3-, 4-, and 5-year mortality) within this single model are not mutually exclusive.

The final hyperparameters were obtained via a grid search on the training set among the following options: kernel size [3, 9, 17], batch size [8, 32, 64], and initial learning rate [0.01, 0.001, 0.0001, 0.00001]. The average cross-entropy was minimized using the Adam optimizer. Maximum 150 epochs were used with early stopping based on validation loss. The model with the lowest validation loss during hyperparameter tuning was selected as the final model. The final hyperparameters were kernel size 17, batch size 32, and learning rate 0.001.

Performance evaluation and statistical analyses

Given the potential for loss to follow-up, model performance for all binary outcomes (e.g. 1- and 5-year mortality) was assessed only on ECGs with documented follow-up after the outcome timeframe or mortality events within the outcome timeframe. Consistent with prior works,^16,26–28 multiple ECGs per patient were allowed in the training cohort. In contrast, model performance was evaluated on one ECG per patient. The ECG selected for testing was either the first available, the last available, or a randomly selected ECG per patient.

Given the imbalanced data set (i.e. low prevalence of mortality), the area under the receiver operating characteristic curve (AUROC) and area under the precision-recall [i.e. positive predictive value (PPV) sensitivity] curve (AUPRC) were computed. To benchmark the model, age at ECG (previously used to benchmark AI-ECG predictions of mortality¹⁴), QRS duration (a conventional ECG predictor of mortality in ToF²⁹), and QTc duration (an independent risk factor for sudden cardiac death³⁰) were used. In addition, left ventricular ejection fraction (LVEF)—an established determinant of cardiovascular morbidity and mortality^31,32—from an echo within 2 days of the paired ECG was used as a benchmark when available. Other performance metrics evaluated included PPV, negative predictive value (NPV), sensitivity, and specificity. These metrics were calculated at the classification threshold maximizing the sensitivity and specificity (i.e. the Youden index) in the training set. For all metrics, a higher value is indicative of better performance. Resampling with 1000 bootstraps was implemented to obtain performance metric median and 95% confidence intervals (CIs). Area under the receiver operating characteristic curves were compared when applicable via the DeLong test.³³

Subgroup analyses

Subgroup analyses were performed on the test set for each disease of interest. Given that conduction disturbances are also a marker of disease progression in CHD,⁸ we also assessed model performance when stratifying by QRS duration and presence of complete right bundle branch block. Area under the receiver operating characteristic curves and AUPRCs were calculated for each subgroup.

Survival analysis

Cox proportional hazards regression was used to evaluate factors associated with time from ECG until all-cause mortality. Patients who did not experience death were censored at the time of last known follow-up. Patients with unknown follow-up time after ECG were excluded from this secondary analysis. The Kaplan–Meier methodology was used to estimate primary outcome event rates.

Model explainability

In an effort to interpret model behaviour, analyses of median waveforms and saliency mapping were performed.

Median waveform analysis is a technique to visually represent single beats of the highest and lowest risk ECGs.^16,27 Herein, a subset of test set ECGs (100 for the overall cohort, 25 for specific lesions) with the highest predicted probability for a given outcome were used to create high-risk median waveforms, and a corresponding set with the lowest predicted probabilities was used to create low-risk median waveforms. Median waveforms were generated in each lead using the NeuroKit Python toolbox³⁴ by (i) detecting QRS complexes, (ii) interpolating all ECGs to the same heart rate, (iii) computing the median voltage across beats for each patient, and (iv) computing the median voltage across patients for each time bin in the cardiac cycle.^16,27

Saliency mapping aims to identify which features of the ECG input contribute to model prediction by highlighting components of the ECG where a change in input (i.e. ECG voltage) leads to a relatively large change in prediction.²⁷ Saliency maps were created using a Shapley Additive Explanations (SHAP) framework³⁵ for the high-risk ECGs. The above median waveform steps were similarly implemented on SHAP values over time. Darker regions in saliency maps correspond to greater contribution to the prediction of 5-year mortality.

Software

Programming codes used to perform the analyses are available upon reasonable request. The convolutional neural network used the Keras framework with a TensorFlow (Google) backend using Python 3.9.³⁶ Deep learning was executed on institutional graphics processing units. All other pre- and post-processing codes were written in Python 3.9³⁶ and R 4.0,³⁷ which was executed locally.

Results

Internal patient population characteristics

The training cohort comprised of 112 804 ECGs from 39 784 patients [median age at ECG 7.7 [interquartile range (IQR), 1.6–14.8; range 0–85] years; 52% male]. The first digitized ECG included in this cohort was from 1990. As shown in Table 1, a wide range of CHD lesions were included, including 11% with ventricular septal defects, 4.5% with cardiomyopathy, 3.8% with coarctation of the aorta, 3.5% with ToF, 2.2% with D-loop transposition of the great arteries, 1.2% with hypoplastic left heart syndrome, 1.0% with L-loop transposition of the great arteries, 0.8% with dextrocardia, and 0.7% with tricuspid atresia. The internal testing cohort included 112 575 ECGs from 39 784 distinct patients [median age at ECG 7.9 (IQR, 1.5–14.8; range 0–92) years; 52% male] and composed of a similar breakdown of congenital heart lesions. The contemporary cohort included 42 927 ECGs from 25 537 patients. Numerous differences in baseline characteristics were noted in the contemporary cohort including higher prevalence of all disease groups (except ventricular septal defects) and older age at ECG [median age at ECG 10.5 (IQR, 3.1–16.7) years].

Table 1.

Internal cohort baseline characteristics

Characteristic	Training	Testing	Temporal validation
Demographics
Patients	39 784	39 784	25 537
Sex
Female	19 061 (48%)	19 009 (48%)	12 496 (49%)
Male	20 718 (52%)	20 769 (52%)	13 036 (51%)
Unknown	5 (<0.1%)	6 (<0.1%)	5 (<0.1%)
Diagnosisa
ToF	1374 (3.5%)	1352 (3.4%)	1029 (4.0%)
Cardiomyopathy	1810 (4.5%)	1773 (4.5%)	1273 (5.0%)
ASD	1498 (3.8%)	1551 (3.9%)	2638 (10%)
CAVC	248 (0.6%)	205 (0.5%)	339 (1.3%)
CoA	1500 (3.8%)	1470 (3.7%)	1298 (5.1%)
DORV	526 (1.3%)	530 (1.3%)	515 (2.0%)
D-loop TGA	884 (2.2%)	914 (2.3%)	729 (2.9%)
HLHS	487 (1.2%)	465 (1.2%)	553 (2.2%)
L-loop TGA	380 (1.0%)	389 (1.0%)	343 (1.3%)
PA	621 (1.6%)	627 (1.6%)	509 (2.0%)
TAPVR	240 (0.6%)	257 (0.6%)	262 (1.0%)
Tricuspid Atresia	271 (0.7%)	247 (0.6%)	195 (0.8%)
VSD	4388 (11%)	4196 (11%)	2674 (10%)
Dextrocardia	304 (0.8%)	326 (0.8%)	290 (1.1%)
WPW syndrome	1070 (2.7%)	1003 (2.5%)	678 (2.7%)
Channelopathy	223 (0.6%)	231 (0.6%)	283 (1.1%)
ECG characteristics
ECGs	112 804	112 575	42 927
Age at ECG (years)	7.7 (1.6, 14.8)	7.9 (1.5, 14.8)	10.5 (3.1, 16.7)
CRBBB	5749 (5.1%)	5873 (5.2%)	2572 (6.0%)
Outcomes
Mortalities	806 (2.0%)	870 (2.2%)	225 (0.9%)
Age of death (years)	18.9 (8.0, 32.4)	19.4 (7.5, 32.2)	13.1 (4.7, 22.6)
1-year mortality after ECGb	909 (1.2%)	781 (1.0%)	283 (1.0%)
5-year mortality after ECGb	2964 (4.9%)	2779 (4.6%)

Age intervals denote interquartile range.

ASD, atrial septal defect; CoA, coarctation of the aorta; CAVC, complete atrioventricular canal defect; CRBBB, complete right bundle branch block; DORV, double outlet right ventricle; HLHS, hypoplastic left heart syndrome; LVEF, left ventricular ejection fraction; PA, pulmonary atresia; ToF, tetralogy of Fallot; TAPVR, total anomalous pulmonary venous return; TGA, transposition of the great arteries; VSD, ventricular septal defect; WPW, Wolff–Parkinson–White.

^aNote select lesions included, some of which overlap.

^bDenominator is equal to the total number of ECGs with either mortality events within the outcome timeframe or with documented follow-up after the outcome timeframe.

In the training cohort, there were 806 (2.0%) mortality events at median age 18.9 (IQR, 8.0–32.4) years. In the testing cohort, there were 870 (2.2%) mortality events at median age 19.4 (IQR, 7.5–32.2) years. Five-year mortality after ECGs in the training and test cohorts was 4.9% and 4.6%, respectively. Similar 1-year mortality events (1.0%) were noted across internal test and temporal validation cohorts, with a younger age of mortality in the temporal validation cohort [median age 13.1 (IQR, 4.7–22.6) years]. Electrocardiograms with mortality within 5 years had higher heart rates, longer QRS and QTc intervals, and lower paired echo ejection fraction (see Supplementary data online, Table S1).

Survival in the training and testing cohorts was similar [hazard ratio 1.0 (95% CI 0.9–1.0), P = .3; log-rank P = .2] and spanned across the lifespan (Figure 1A). As shown in Figure 1B and C, a large portion of patients survive to adulthood, with survival rates varying by lesion. The lowest long-term survival was lesions that typically involve single ventricle palliation (tricuspid atresia for RV pathology, hypoplastic left heart syndrome for left ventricular pathology).

Figure 1.

Kaplan–Meier survival analysis of training and testing cohorts. (A) Kaplan–Meier curve survival analysis of training (green) and testing (red) cohorts demonstrates the large, diverse cohorts across the congenital heart disease lifespan. Lesion-specific survival curves for (B) left ventricular and (C) right ventricular pathology are shown when pooling training and testing cohorts. Number at risk within each group inset below. CoA, coarctation of the aorta; DORV, double outlet right ventricle; HLHS, hypoplastic left heart syndrome; LV, left ventricular; PA, pulmonary atresia; RV, right ventricular; TAPVR, total anomalous pulmonary venous return; TGA, transposition of the great arteries; ToF, tetralogy of Fallot

Model performance

During testing (Figure 2A), the model achieved the following performance in 5-year mortality when using the random, last, and first ECGs: AUROCs of 0.79 (95% CI 0.77–0.81), 0.82 (95% CI 0.80–0.83), and 0.75 (95% CI 0.72–0.77), respectively, and AUPRCs of 0.17 (95% CI 0.15–0.19), 0.25 (95% CI 0.23–0.28), and 0.11 (95% CI 0.09–0.12), respectively. The randomly selected ECG performance outperformed age at ECG [AUROC 0.58 (95% CI 0.55–0.61); AUPRC 0.11 (95% CI 0.08–0.13); P < .001], QRS duration [AUROC 0.57 (95% CI 0.54–0.60); AUPRC 0.07 (95% CI 0.06–0.08); P < .001], QTc duration [AUROC 0.48 (95% CI 0.45–0.50); AUPRC 0.05 (95% CI 0.04–0.06); P < .001], and paired echo LVEF [AUROC 0.62 (95% CI 0.57–0.68) [AUPRC 0.10 (95% CI 0.07–0.15); P < .001] when predicting 5-year mortality (Figure 2B), as well as 1-year mortality (see Supplementary data online, Figure S1). Artificial intelligence-enhanced ECG similarly outperformed these metrics when using the last available ECG (see Supplementary data online, Figure S2).

Figure 2.

Electrocardiogram-based deep learning model performance. (A) Artificial intelligence-enhanced electrocardiogram model performance to predict 5-year mortality evaluated using a random (blue), last (orange), and first (green) electrocardiogram per patient. (B) Performance benchmarking of the random electrocardiogram (blue) to age at electrocardiogram (orange), QRS duration (green), QTc duration (red), and left ventricular ejection fraction (purple). (C) Comparison of internal testing (blue) and temporal validation (orange) performance to predict 1-year mortality. Area under the receiver operating characteristic curve and area under the precision-recall curve metric values for each model and outcome are inset. Dotted line represents chance. 95% confidence intervals are shown using bootstrapping. AUROC, area under the receiver operating characteristic curve; AUPRC, area under the precision-recall curve; ECG, electrocardiogram; LVEF, left ventricular ejection fraction; PPV, positive predictive value

When using a random ECG, the sensitivity, specificity, and PPV were 0.66 (95% CI 0.62–0.70), 0.78 (95% CI 0.78–0.79), and 12.4% (95% CI 11.6%–13.1%), respectively, with 76.4% (95% CI 75.6%–77.1%) predicted negative (see Supplementary data online, Table S2). A NPV of ∼98% was achieved independent of ECG used (see Supplementary data online, Table S2). Model calibration is shown in Supplementary data online, Figure S3.

Contemporary model performance

During temporal validation, AUROC of 0.79 (95% CI 0.74–0.83) and AUPRC of 0.04 (95% CI 0.03–0.06) was achieved to predict 1-year mortality (Figure 2C).

Subgroup analysis

In a subgroup analysis (Figure 3), model performance appeared lesion and age dependent. When using a random ECG per patient, model performance in extra-cardiac pathophysiology (i.e. coarctation of the aorta and total anomalous pulmonary venous return) was as follows: AUROCs of 0.86 (95% CI 0.81–0.91) and 0.80 (95% CI 0.72–0.88), respectively, and AUPRCs of 0.30 (95% CI 0.19–0.41) and 0.38 (95% CI 0.20–0.56), respectively. For a wide range of congenital heart lesions with accumulating pathophysiologic myocardial burden over time (e.g. cardiomyopathy, pulmonary atresia, double outlet right ventricle, hypoplastic left heart syndrome, and tricuspid atresia), lower AUROC with higher AUPRC patterns were noted. These trends were consistent when using the last available ECG, and less so when using the first available ECG. Similarly, the dextrocardiac subgroup had a lower AUROC [0.71 (95% CI 0.61–0.81)] with higher AUPRC [0.21 (95% CI 0.10–0.32)]. Model performance was lower in L-loop transposition of the great arteries (Figure 3). Across all lesions with sufficient data available for comparison, AUROC and AUPRC trended higher for AI-ECG compared with LVEF (see Supplementary data online, Figure S4).

Figure 3.

Model performance in congenital heart disease subgroups. Forest plot showing artificial intelligence-enhanced electrocardiogram area under the area under the receiver operating characteristic curve (red) and area under the precision-recall curve (black) performance when stratifying by lesion when using a random (left), last (middle), and first (right) electrocardiogram. Area under the receiver operating characteristic curve and area under the precision-recall curve metric values for each model and outcome are inset. 95% confidence intervals are shown using bootstrapping. ASD, atrial septal defect; CoA, coarctation of the aorta; CAVC, complete atrioventricular canal defect; DORV, double outlet right ventricle; ECG, electrocardiogram; HLHS, hypoplastic left heart syndrome; PA, pulmonary atresia; ToF, tetralogy of Fallot; TAPVC, total anomalous pulmonary venous connection; TGA, transposition of the great arteries; VSD, ventricular septal defect

Given that conduction disturbances are also a marker of disease progression in CHD,⁸ we also assessed model performance when stratifying by QRS duration and presence of complete right bundle branch block. In patients with QRS duration ≤ 120 ms, AUROC of 0.77 (95% CI 0.75–0.80) and AUPRC of 0.14 (95% CI 0.12–0.16) were achieved. For QRS duration > 120 ms, AUROC of 0.76 (95% CI 0.72–0.80) and AUPRC of 0.20 (95% CI 0.15–0.24) were obtained. In patients with complete right bundle branch block, AUROC of 0.78 (95% CI 0.73–0.83) and AUPRC of 0.21 (95% CI 0.13–0.29) were achieved. In patients without complete right bundle branch block, AUROC of 0.79 (95% CI 0.77–0.81) and AUPRC of 0.18 (95% CI 0.15–0.21) were obtained.

Survival analysis

Longer-term survival was assessed when stratifying patients into low- (<threshold) or high-risk (≥threshold) groups based on AI-ECG predictions (Figure 4). When using a random ECG, there was 15-year survival of 96% and 80% for low- vs. high-risk ECGs, respectively. When using a random ECG, high-risk patients were 4.9 times (95% CI 4.3–5.6) more likely to experience mortality (P < .001). This phenomenon was more pronounced when using the last ECG, with 15-year survival of 95% and 72% in low- and high-risk groups, respectively, and a hazard ratio of 7.2 (95% CI 6.3–8.3) (P < .001).

Figure 4.

Kaplan–Meier survival analysis based on artificial intelligence-enhanced electrocardiogram risk stratification. Kaplan–Meier curve survival analysis when stratifying patients as low- (blue) or high-risk (orange) based on artificial intelligence-enhanced electrocardiogram predictions using random (left), last (middle), or first (right) electrocardiogram per patient. Number at risk within each group inset below. Hazard ratio inset below (high- vs. low-risk group based on artificial intelligence-enhanced electrocardiogram predictions) with 95% confidence interval using Cox regression analysis. P-value statistic below based on log-rank testing. ECG, electrocardiogram; HR, hazard ratio

In the internal test cohort, the AI-ECG model predictions [c-index 0.74 (95% CI 0.72–0.76)] outperformed QRS duration [c-index 0.52 (95% CI 0.50–0.54)] and LVEF [c-index 0.64 (95% CI 0.60–0.68)] in Cox model survival discrimination (Table 2). The addition of age and/or LVEF as predictors to AI-ECG provided no added value in the c-index (Table 2).

Table 2.

Cox models on the internal test cohort incorporating artificial intelligence-enhanced electrocardiogram predictions

Cox model	Variable z-score	P-value	c-index (95% CI)
Age only	8.4	<2e−16	0.58 (0.56–0.60)
QRS duration only	5.8	5e−9	0.52 (0.50–0.54)
LVEF only	−11.5	<2e−16	0.64 (0.60–0.68)
AI-ECG only	15.2	<2e−16	0.74 (0.72–0.76)
Age ± LVEF			0.65 (0.61–0.69)
Age	7.0	3e−12
LVEF	−10.3	<2e−16
Age ± AI-ECG			0.71 (0.69–0.73)
Age	6.7	2e−11
AI-ECG	14.0	<2e−16
LVEF ± AI-ECG			0.72 (0.70–0.74)
LVEF	−6.4	2e−10
AI-ECG	10.9	<2e−16
Age ± LVEF ± AI-ECG			0.72 (0.70–0.74)
Age	6.0	2e−9
LVEF	−5.7	1e−8
AI-ECG	10.2	<2e−16

AI-ECG, artificial intelligence-enhanced electrocardiogram; CI, confidence interval; LVEF, left ventricular ejection fraction.

Lesion-specific survival analyses demonstrated effective risk stratification in nearly all lesions (Figure 5), including dextrocardia [hazard ratio 2.4 (95% CI 1.1–5.3); P = .02; Supplementary data online, Figure S5), but not complete atrioventricular canal defects [hazard ratio 3.5 (95% CI 0.8–15.8); P = .10; Supplementary data online, Figure S6].

Figure 5.

Lesion-specific Kaplan–Meier survival analysis based on artificial intelligence-enhanced electrocardiogram risk stratification. Lesion-specific Kaplan–Meier curve survival analysis when stratifying patients as low- (blue) or high-risk (orange) based on artificial intelligence-enhanced electrocardiogram predictions. Hazard ratio inset below (high- vs. low-risk group based on artificial intelligence-enhanced electrocardiogram predictions) with 95% confidence interval using Cox regression analysis. P-value statistic below based on log-rank testing. Initial sample size in each cohort inset. ASD, atrial septal defect; CoA, coarctation of the aorta; DORV, double outlet right ventricle; HR, hazard ratio; HLHS, hypoplastic left heart syndrome; PA, pulmonary atresia; ToF, tetralogy of Fallot; TAPVC, total anomalous pulmonary venous connection; TGA, transposition of the great arteries; VSD, ventricular septal defect

Model explainability

The most salient features (Figure 6) of an ECG to predict 5-year mortality in the overall cohort include S waves (limb lead III and precordial leads V2, V3, and V6) and T waves (limb leads II and V6). High-risk features to predict 5-year mortality include wide QRS complexes with deep S waves and low-amplitude waveforms (Figure 6).

Figure 6.

Explainability of artificial intelligence-enhanced electrocardiogram predictions. Visualization of median waveforms generated in each lead using electrocardiograms from the highest (red) and lowest (green) artificial intelligence-enhanced electrocardiogram predictions of the overall cohort, as well as cardiomyopathy, tetralogy of Fallot, and hypoplastic left heart syndrome subgroups. Saliency mapping demarcates regions of the electrocardiogram waveform having greatest (dark blue) and least (light blue) influence on each outcome. Saliency was averaged over the highest predicted electrocardiograms for each outcome. HLHS, hypoplastic left heart syndrome; ToF, tetralogy of Fallot

When performing saliency mapping and median waveform analysis within subgroups, lesion-specific high-risk signatures were identified (Figure 6). In cardiomyopathy, V2 and V3 were less salient than the overall cohort; however, high-risk features appeared similar to the overall cohort. In right-sided myopathy such as ToF, V6 was less salient than the overall cohort; in addition, while low-risk features included a right bundle branch block, high-risk features included QRS fragmentation. In left-sided pathology such as hypoplastic left heart syndrome, saliency maps were more focused on the QRS complex compared with the overall cohort. High-risk features included tall and wide R waves in V1–3 with a deep S wave in II–III and lateral precordial leads.

Discussion

For several decades, risk stratification has been of great interest in the CHD field using conventional and artificial intelligence approaches.^6,7,24 The ongoing challenge in developing robust risk prediction models in CHD has led to multiple recent calls for AI applications to improve risk stratification in CHD.^6,7 In this work, we address this gap by developing and validating the first (to our knowledge) ECG-based deep learning algorithm to predict mortality in children and adults with CHD. We demonstrate that AI-ECG can successfully predict 5-year mortality, outperforming conventional markers such as age, LVEF, and QRS duration. The encouraging performance when using the first available, the last available, or a random ECG per patient demonstrates the promise of our model to predict mortality during initial and follow-up assessments to aid in lifelong risk stratification. Model explainability analysis provides transparency and interpretability for clinicians and may help generate hypotheses for underlying myopathy ECG signatures predictive of mortality. Altogether, our findings demonstrate the prognostic value of AI-ECG in CHD across the lifespan, which may (i) enhance current risk stratification strategies, (ii) prioritize patients for diagnostic studies and/or interventions, and (iii) facilitate improved access to care (Structured Graphical Abstract).

Conventional electrocardiogram predictors of mortality in congenital heart disease

There is a paucity of conventional ECG analyses to predict morbidity and mortality in patients with CHD. The majority of applications are in ToF: several studies have recognized severe QRS prolongation as a risk factor for mortality in this patient population.³⁸ However, the sensitivity of QRS duration > 180 ms to predict mortality was <50% in recent studies.³⁹ Subsequent work by Bokma et al.²⁹ demonstrated QRS fragmentation—related to myocardial fibrosis and dysfunction—is superior to QRS duration in predicting mortality. QRS fragmentation has also been associated with mortality in cardiomyopathy.⁴⁰ In addition, a longer QRS has been previously predictive of mortality in hypoplastic left heart syndrome,⁴¹ D-loop transposition of the great arteries,⁴² and Fontan circulation,⁴³ all with limited performance.

This has motivated more recent efforts to incorporate imaging modality data into risk prediction algorithms.⁴⁴ However, even such algorithms continue to have suboptimal performance and are reliant on expensive modalities (e.g. cardiac magnetic resonance imaging) that require subspecialized expertise. As shown herein, AI-ECG provides an inexpensive, ubiquitous alternative that is predictive across a wide range of lesions.

Clinical significance and implications

Imaging modalities (e.g. cardiac magnetic resonance and echocardiography) conventionally used to aid in risk stratification in CHD have practice limitations (time-intensive, resource-consuming, and need for subspecialist expertise) that hinder its widespread use. In contrast, ECGs is rapid and cost-effective and can be conveniently acquired at every cardiology visit, which facilitates easier and more frequent use that may guide clinical decision-making. We envision this AI-ECG algorithm may serve as a screening or surveillance tool and potentially improve access to care.

Given the objective to risk stratify patients from a preventative lens, we opted to train and test using ECGs from cardiology clinic. In this setting, AI-ECGs could be of significant screening and surveillance value. As shown in Supplementary data online, Table S2, a NPV of 98% was achieved to predict 5-year mortality, with 15-year survival of 96% for low-risk ECGs. To this end, as a screening tool, low-risk AI-ECG predictions may help reduce follow-up frequency, the need for non-invasive imaging, diagnostic catheterizations, or implantable cardioverter-defibrillators. On the other hand, given the 5-year mortality PPV of 12%, it may help identify high-risk patients requiring closer monitoring.

As a surveillance tool, a congenital cardiologist could conceivably monitor AI-ECG predictions at each cardiology visit. Monitoring AI-ECG predictions over time may provide insight into responsiveness to interventions (e.g. pulmonary valve replacement in ToF) or inter-stage monitoring of single ventricle patients.

Finally, for low-resource settings with limited access to advanced modalities, this algorithm may help improve access to care. Notably, despite the majority of CHD patients being adults, approximately half remain without regional CHD services.⁴⁵ This technology may therefore contribute to the democratization of specialty expertise and circumvent the requirement for specialized cardiac magnetic resonance knowledge to risk stratify certain lesions (e.g. ToF).

Significance of temporal validation

From 1990 (the first ECG available in this cohort) through today, CHD has rapidly evolved. For example, surgical and medical advances have contributed to the nearly 40% decrease in CHD mortality from 1999 to 2017,⁴⁶ with >97% of children with CHD expected to reach adulthood.⁴⁷ From an institutional standpoint, the Boston Children’s Hospital volume and complexity have significantly increased (Table 1). Given the range of confounding factors that could affect mortality, we opted to perform temporal validation with reassuringly similar performance.

Artificial intelligence-enhanced electrocardiogram model insights gained

The overall CHD and cardiomyopathy cohorts had similar high-risk features, suggesting common final signatures on an ECG indicative of high-risk mortality. Indeed, both CHD and cardiomyopathy have non-trivial overlap, and both largely contribute to paediatric heart failure.⁴⁸ The common features noted herein include wide QRS complexes with deep S waves and low-amplitude waveforms, which may correspond to heterogeneous slow activation of the myocardium due to scar and/or myocardial stress. In addition, select high-risk patterns were noted in each disease process. For example, QRS fragmentation was noted in high-risk ECGs for ToF (Figure 6). Future work is required to investigate the relation of these features with progressive cardiomyopathy and paediatric heart failure.

Finally, we note variation in model performance by disease subtype. Interestingly, effective risk stratification was achieved in a majority of diseases (including dextrocardia), but not complete atrioventricular canal defects. The reasons for poor performance in this group are not readily apparent but may involve the high heterogeneity of this group given its association with heterotaxy syndrome and situs abnormalities, trisomy 21, and inherently abnormal QRS axis (superior axis deviation) compared with the rest of the cohort.

Limitations and future directions

There are several limitations of this work. First, although heart failure is the most common cause of death in adult CHD,²³ all-cause mortality rather than cardiac mortality was used for the primary outcome in this study. In addition, while all-cause mortality is routinely coded at our institution, it is conceivable that positive outcomes are undocumented. We attempted to mitigate this limitation in our binary outcome analysis by including only ECGs with mortality events within the outcome timeframe or documented follow-up after the outcome timeframe. Similarly, in our survival analysis, we censored at time of the last known follow-up. Nevertheless, future use of state or national death index database should be considered. Second, while several recent AI-ECG works also used all-cause mortality as the primary endpoint,^14,15 similar clinically meaningful outcomes such as heart transplant are also of interest. Third, while temporal validation was achieved, it is of great interest to obtain external validation for each CHD lesion. Fourth, only one example of thresholding was used to evaluate model performance, as further consideration is required to weigh the impact of resultant false negatives (which may lead to clinical consequences of missed pathology) and false positives (which may lead to unnecessary extraneous testing), as well as optimally set thresholds across institutions. To this end, multicentre external validation to further refine thresholds for clinical implementation is warranted. Similarly, multicentre collaboration via federated learning⁴⁹ may help improve training/testing sample sizes, which may further improve performance. Given our objective to develop an inexpensive and convenient risk stratification tool, only ECG inputs were utilized; nevertheless, multimodal inputs may lead to improved model performance (especially for complex lesions) requiring further investigation.⁵⁰ The limitations of saliency mapping must be noted.⁵¹ Lastly, diagnostic categories in this study are quite heterogeneous. For example, the cardiomyopathy category includes dilated, hypertrophic, and restrictive cardiomyopathy, among others. Similarly, multiple CHD diagnoses can occur simultaneously, such that patients can be assigned into multiple categories. In addition, the cohort includes patients with and without repairs for CHD lesions.

Future work therefore includes model refinement and external validation for each lesion of interest, multicentre collaboration, consideration of multimodal inputs, and prospective trials (to determine how to properly implement such tools to support clinical decision-making). Finally, we note that a recent randomized clinical trial implementing AI-ECG alerts led to decreased all-cause mortality in the general adult population.⁵² It is similarly of great interest to implement a similar study design in the distinct CHD population.

Conclusions

In conclusion, these findings demonstrate the promise of AI-ECG to inexpensively and conveniently risk stratify individuals with CHD across the lifespan. This tool may facilitate the prioritization of patients for future interventions/studies, provide meaningful insight into novel ECG waveforms suggestive of mortality, and potentially reduce disparities by improving access to care. Future multicentre collaboration and prospective trials are warranted.

Supplementary data

Supplementary data are available at European Heart Journal online.

Declarations

Disclosure of Interest

All authors declare no disclosure of interest for this contribution.

Data Availability

Requests for the Boston Children’s Hospital data and related materials will be internally reviewed to clarify if the request is subject to intellectual property or confidentiality constraints. Shareable data and materials will be released under a material transfer agreement for non-commercial research purposes. The use of the Boston Children’s Hospital data was approved by its Institutional Review Boards.

Funding

Funding support received from the Thrasher Research Fund Early Career Award (J.M.), Boston Children’s Hospital Electrophysiology Research Education Fund (J.M. and J.K.T.) and National Institutes of Health grant R00-LM012926 from the National Library of Medicine (W.G.L.).

Ethical Approval

Institutional Review Board or Ethics Committee approval was obtained.

Pre-registered Clinical Trial Number

None supplied.