Table 2.
Performance characteristics of models in the validation set for predicting clinically relevant AHREs in patients with pacemaker.
| Logistic regression | RF | SVM | XGB | |
|---|---|---|---|---|
| Model performance | ||||
| AUROC (95% CI) | 0.669 (0.536–0.803) | 0.742 (0.637–0.835) | 0.675 (0.561–0.789) | 0.745 (0.631–0.847) |
| AUPRC (95% CI) | 0.182 (0.104–0.274) | 0.224 (0.119–0.397) | 0.182 (0.102–0.337) | 0.240 (0.125–0.424) |
| F1 score (95% CI) | 0.853 (0.783–0.881) | 0.888 (0.845–0.925) | 0.865 (0.821–0.905) | 0.896 (0.857–0.931) |
| Accuracy (95% CI) | 0.753 (0.677–0.819) | 0.805 (0.734–0.865) | 0.773 (0.698–0.836) | 0.818 (0.748–0.876) |
| Sensitivity (95% CI) | 0.815 (0.739–0.876) | 0.881 (0.815–0.931) | 0.830 (0.755–0.889) | 0.889 (0.823–0.936) |
| Specificity (95% CI) | 0.316 (0.126–0.566) | 0.263 (0.091–0.512) | 0.368 (0.163–0.616) | 0.316 (0.126–0.566) |
| PPV (95% CI) | 0.194 (0.074–0.375) | 0.238 (0.082–0.472) | 0.233 (0.099–0.423) | 0.286 (0.113–0.522) |
| NPV (95% CI) | 0.894 (0.824–0.943) | 0.895 (0.830–0.941) | 0.903 (0.837–0.949) | 0.902 (0.839–0.947) |
| Brier score | ||||
| Overall | 0.181 | 0.138 | 0.158 | 0.141 |
| Reliability | 0.086 | 0.038 | 0.058 | 0.054 |
| Resolution | 0.013 | 0.008 | 0.008 | 0.021 |
| Uncertainty | 0.108 | 0.108 | 0.108 | 0.108 |
AHREs atrial high-rate episodes, AUPRC area under the precision-recall curve, AUROC area under receiver operating characteristic, CI confidence interval, NPV negative predictive value, PPV positive predictive value, RF random forest, SVM support vector machine, XGB extreme gradient boosting.