Table 2.
Summary of model performances for predicting oral anticoagulation prescription in training (5-fold cross-validation) and test sets
Regular model: CHA2DS2-VASc components |
Enhanced ML model: CHA2DS2-VASc components + new features |
||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Accuracy | AUROC | PRAUC | Precision | Recall | Accuracy | AUROC | PRAUC | Precision | Recall | ||
XGBOOST | Test set | 0.69 | 0.62 | 0.76 | 0.70 | 0.96 | 0.77 | 0.81 | 0.89 | 0.79 | 0.89 |
Cross-validation | 0.70 | 0.64 | 0.77 | 0.70 | 0.95 | 0.78 | 0.83 | 0.90 | 0.80 | 0.89 | |
Logistic regression | Test set | 0.69 | 0.60 | 0.73 | 0.70 | 0.96 | 0.73 | 0.75 | 0.86 | 0.77 | 0.87 |
Cross-validation | 0.69 | 0.60 | 0.73 | 0.70 | 0.96 | 0.74 | 0.76 | 0.86 | 0.76 | 0.88 | |
Random forest | Test set | 0.68 | 0.59 | 0.73 | 0.68 | 0.99 | 0.76 | 0.79 | 0.85 | 0.79 | 0.88 |
Cross-validation | 0.75 | 0.78 | 0.88 | 0.76 | 0.93 | 0.99 | 0.99 | 0.85 | 0.99 | 0.99 | |
LASSO-penalized logistic regression | Test set | 0.69 | 0.60 | 0.73 | 0.70 | 0.96 | 0.74 | 0.76 | 0.88 | 0.76 | 0.89 |
Cross-validation | 0.69 | 0.60 | 0.73 | 0.70 | 0.96 | 0.74 | 0.76 | 0.99 | 0.76 | 0.89 |
AUROC = area under the receiver-operating characteristic curve; CHA2DS2-VASc = congestive heart failure, hypertension, age ≥75 years, diabetes mellitus, prior stroke or transient ischemic attack or thromboembolism, vascular disease, age 65–74 years, sex category; ML = machine learning; PRAUC = area under the precision-recall curve.