Table 2.
Model performance for incident AF in test sets
|
Massachusetts General Hospital
(N=4,166) |
Brigham and Women’s Hospital
(N=37,963) |
UK Biobank
(N=41,033) |
|||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Model | Hazard ratio (per 1-SD) | 5-year AUROC | 5-year average precision | Calibration slope | ICI‡ | Hazard ratio (per 1-SD) | 5-year AUROC | 5-year average precision | Calibration slope | ICI | Hazard ratio (per 1-SD) | 2-year AUROC | 2-year average precision | Calibration slope | ICI |
| Deep learning architectures | |||||||||||||||
| ECG-AI | - | 0.823* (0.790–0.856) |
0.27 (0.21–0.34) |
- | 0.0231 | - | 0.747* (0.736–0.759) |
0.19*† (0.17–0.20) |
- | 0.0124 | - | 0.705 (0.659–0.724) |
0.060* (0.043–0.087) |
- | 0.0768 |
| Cox proportional hazards models | |||||||||||||||
| Age and sex | 2.91 (2.44–3.47) |
0.768 (0.732–0.805) |
0.16 (0.13–0.20) |
1.05 (0.88–1.23) |
0.0074 | 2.48 (2.35–2.62) |
0.730 (0.717–0.743) |
0.14 (0.13–0.15) |
0.94 (0.88–1.00) |
0.0072 | 2.21 (1.96–2.50) |
0.728 (0.702–0.755) |
0.018 (0.015–0.024) |
1.48 (1.25–1.71) |
0.0019§ |
| CHARGE-AF | 3.36 (2.98–4.30) |
0.802* (0.767–0.836) |
0.21* (0.17–0.26) |
0.68 (0.58–0.77) |
0.0320 | 2.78 (2.63–2.94) |
0.752* (0.741–0.763) |
0.17* (0.15–0.18) |
0.57 (0.53–0.60) |
0.0344 | 2.26 (2.00–2.55) |
0.732 (0.704–0.759) |
0.020 (0.016–0.026) |
0.87 (0.75–1.00) |
0.0011§ |
| ECG-AI | 2.45 (2.23–2.69) |
0.823* (0.790–0.856) |
0.27* (0.21–0.34) |
1.06 (0.95–1.17) |
0.0212 | 2.05 (1.98–2.11) |
0.747* (0.736–0.759) |
0.19*† (0.17–0.20) |
0.81 (0.77–0.84) |
0.0129 | 2.01 (1.88–2.14) |
0.705 (0.673–0.737) |
0.060*† (0.044–0.090) |
0.75 (0.68–0.82) |
0.0035§ |
| CH-AI | 3.74 (3.24–4.33) |
0.838*† (0.807–0.869) |
0.30*† (0.24–0.38) |
1.13 (1.01–1.25) |
0.0120 | 2.76 (2.64–2.88) |
0.777*† (0.766–0.788) |
0.21*† (0.19–0.23) |
0.77 (0.74–0.81) |
0.0108 | 2.27 (2.11–2.44) |
0.746 (0.716–0.776) |
0.059*† (0.042–0.083) |
1.01 (0.92–1.10) |
0.0001§ |
*
p<0.05 for comparison against age and sex
†
p<0.05 for comparison against CHARGE-AF
‡
Integrated calibration index (ICI), a quantitative measure of the average difference between predicted event risk and observed event incidence, weighted by the empirical distribution of event risk.30 Smaller values indicate better calibration.
§
Values reflect ICI after recalibration to the baseline 2-year AF risk in the UK Biobank
Difference in c-index for CH-AI vs ECG-AI: AUROC MGH p=NS, BWH p<0.05, UK Biobank p<0.05; average precision MGH p<0.05, BWH p<0.05, p=NS
AUROC = area under the receiver operating characteristic curve; ICI = integrated calibration index; SD = standard deviation