Table 4.
Machine learning algorithm performance for the top 5 models identified by traditional programming versus MILO.
| Method | Accuracy (95% CI) | AUROC (95% CI)* | Sensitivity (95% CI) | Specificity (95% CI) | Features |
|---|---|---|---|---|---|
| A. Traditional programming | |||||
| Logistic regression | 86 (80–90) | 0.96 (0.88–1.00) | 98 (89–100) | 82 (75–88) | 16a |
| Deep neural network | 81 (75–86) | 0.96 (0.85–1.00) | 94 (83–99) | 77 (70–83) | 10b |
| k-nearest neighbor | 81 (75–86) | 0.92 (0.84–1.00) | 98 (89–100) | 76 (68–82) | 10b |
| Support vector machine | 85 (79–89) | 0.97 (0.86–1.00) | 98 (89–100) | 81 (74–87) | 14c |
| Random forest | 79 (73–85) | 0.92 (0.84–1.00) | 94 (83–99) | 75 (67–82) | 10b |
| B. MILO | |||||
| k-nearest neighbor | 90 (85–94) | 0.96 (0.85–1.00) | 96 (86–99) | 88 (82–93) | 5e |
| Logistic regression | 87 (81–91) | 0.95 (0.83–1.00) | 98 (89–100) | 83 (77–89) | 23f |
| Naïve bayes | 89 (84–93) | 0.95 (0.84–1.00) | 94 (83–99) | 87 (81–92) | 11d |
| Random forest | 84 (79–89) | 0.94 (0.84–1.00) | 96 (86–99) | 81 (74–87) | 23f |
| Deep neural network | 84 (79–89) | 0.95 (0.85–1.00) | 100 (93–100) | 80 (72–86) | 17 g |
| Support vector machine | 86 (80–90) | 0.97 (0.87–1.00) | 98 (89–100) | 82 (75–88) | 11d |
| Gradient boosting machine | 81 (75–86) | 0.94 (0.88–1.00) | 96 (86–99) | 76 (69–83) | 5e |
BUN blood urea nitrogen, CI confidence interval, CVP central venous pressure, DBP diastolic blood pressure, GCS Glascow Coma Score, HCT hematocrit, HGB hemoglobin, HR heart rate, MAP mean arterial pressure, MODS multiple organ dysfunction score, PLT platelet count, RR respiratory rate, SBP systolic blood pressure, SO2 oxygen saturation, TCO2 total CO2, and WBC white blood cell count.
*Area under the ROC curves are reported in fractions.
aMAP, RR, body temperature, GCS, WBC, HGB, HCT, PLT, Na+ , K+ , BUN, creatinine, BUN/creatinine, glucose, TCO2, and MODS.
bBody temperature, WBC, HGB, HCT, Na+ , K+ , BUN, creatinine, BUN/creatinine, and TCO2.
cRR, body temperature, GCS, WBC, HGB, HCT, PLT, Na+ , K+ , BUN, creatinine, BUN/creatinine, TCO2, and MODS.
dSBP, MAP, HR, TEMP, HCT, Na+ , K+ , BUN, BUN/creatinine, anion gap, and TCO2.
eHR, body temperature, HGB, BUN, and TCO2.
fSBP, DBP, MAP, CVP, RR, HR, body temperature, GCS, SO2, WBC, HGB, HCT, PLT, Na+ , K+ , Cl−, anion gap, BUN, creatinine, BUN/creatinine, glucose, TCO2, and MODS.
gMAP, HR, RR, TEMP, WBC, HGB, HCT, PLT, Na+ , K+ , BUN, creatinine, BUN/creatinine, glucose, anion gap, TCO2, and MODS.