Table 3.
Performance comparison among different machine-learning models.
| Model | AUC (95% CI) | Sensitivity (%) (95% CI) | Specificity (%) (95% CI) | PPV (%) (95% CI) | NPV (%) (95% CI) | F1 score (%) (95% CI) | Accuracy (%) (95% CI) | |
|---|---|---|---|---|---|---|---|---|
| Logistic regression | original | 0.730 (0.729–0.730) | 14.7 (14.3–15.0) | 96.6 (96.5–96.6) | 63.7 (63.2–64.2) | 73.4 (73.3–73.5) | 83.4 (83.4–83.5) | 72.8 (72.7–72.9) |
| SMOTE | 0.728 (0.727–0.728) | 64.1 (63.7–64.4) | 68.5 (68.1–68.9) | 45.5 (45.2–45.7) | 82.3 (82.2–82.4) | 74.8 (74.5–75.0) | 67.2 (67.0–67.4) | |
| Random forest | original | 0.976 (0.975–0.976) | 87.5 (87.0–87.9) | 96.7 (96.5–96.9) | 91.6 (91.1–92.1) | 95.0 (94.8–95.1) | 95.8 (95.7–96.0) | 94.0 (93.8–94.3) |
| SMOTE | 0.979 (0.978–0.980) | 92.5 (92.1–92.9) | 93.0 (92.7–93.3) | 84.4 (83.9–85.0) | 96.8 (96.6–97.0) | 94.9 (94.7–95.0) | 92.9 (92.7–93.1) | |
| Support vector machine | original | 0.860 (0.858–0.861) | 51.3 (51.0–51.6) | 97.3 (97.2–97.4) | 88.5 (88.2–88.8) | 83.0 (82.9–83.1) | 89.6 (89.5–89.6) | 83.9 (83.8–84.0) |
| SMOTE | 0.871 (0.870–0.872) | 63.3 (62.9–63.7) | 91.4 (91.3–91.6) | 75.2 (74.9–75.5) | 85.9 (85.8–86.0) | 88.6 (88.5–88.7) | 83.3 (83.1–83.4) |
AUC, area under receiver operating characteristic; CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value.