Table 3.
Model prediction comparisons with the test data set.
| Model | Sensitivity | Specificity | Accuracy | Balanced accuracy | AUROCa | |
| 921 features (including ICD-10b) | ||||||
|
|
AdaBoostc | 0.9739 | 0.9888 | 0.9887 | 0.9813 | 0.9974 |
|
|
XGBoostd | 0.9689 | 0.9888 | 0.9888 | 0.9789 | 0.9972 |
|
|
LightGBMe | 0.9701 | 0.9887 | 0.9887 | 0.9794 | 0.9973 |
|
|
GBMf | 0.7954 | 0.9506 | 0.9504 | 0.8730 | 0.9350 |
|
|
ERTg | 0.9010 | 0.9270 | 0.9270 | 0.9140 | 0.9580 |
|
|
LRh | 0.9584 | 0.9922 | 0.9922 | 0.9753 | 0.9952 |
|
|
RFi | 0.9433 | 0.9537 | 0.9538 | 0.9476 | 0.9848 |
|
|
DNNj | 0.9694 | 0.9845 | 0.9845 | 0.9770 | 0.9931 |
|
|
AdaBoost+XGBoost | 0.9700 | 0.9889 | 0.9889 | 0.9794 | 0.9973 |
|
|
AdaBoost+LightGBM | 0.9702 | 0.9888 | 0.9888 | 0.9795 | 0.9974 |
|
|
XGBoost+LigtGBM | 0.9694 | 0.9888 | 0.9888 | 0.9791 | 0.9973 |
|
|
AdaBoost+XGBoost+LightGBM | 0.9698 | 0.9888 | 0.9888 | 0.9793 | 0.9973 |
|
|
In-hospital mortality AIk [5] | 0.9468 | 0.9761 | 0.9760 | 0.9614 | 0.9929 |
| 878 features (ICD-10 only) | ||||||
|
|
AdaBoost | 0.8313 | 0.9476 | 0.9475 | 0.8895 | 0.9485 |
|
|
XGBoost | 0.6933 | 0.9751 | 0.9748 | 0.8342 | 0.8943 |
|
|
LightGBM | 0.6995 | 0.9745 | 0.9742 | 0.8370 | 0.8953 |
|
|
GBM | 0.9730 | 0.9774 | 0.9774 | 0.9752 | 0.9958 |
|
|
ERT | 0.9739 | 0.9810 | 0.9810 | 0.9775 | 0.9938 |
|
|
LR | 0.7623 | 0.9558 | 0.9557 | 0.8591 | 0.9437 |
|
|
RF | 0.6583 | 0.9740 | 0.9737 | 0.8161 | 0.9313 |
|
|
DNN | 0.8809 | 0.8959 | 0.8959 | 0.8884 | 0.9438 |
|
|
AdaBoost+XGBoost | 0.7046 | 0.9747 | 0.9744 | 0.8397 | 0.9452 |
|
|
AdaBoost+LightGBM | 0.7057 | 0.9744 | 0.9741 | 0.8400 | 0.9452 |
|
|
XGBoost+LigtGBM | 0.6950 | 0.9748 | 0.9745 | 0.8349 | 0.8954 |
|
|
AdaBoost+XGBoost+LightGBM | 0.7024 | 0.9746 | 0.9743 | 0.8385 | 0.9450 |
|
|
In-hospital mortality AI [5] | 0.2838 | 0.9751 | 0.9751 | 0.6298 | 0.7675 |
| 43 features (excluding ICD-10) | ||||||
|
|
AdaBoost | 0.9743 | 0.9862 | 0.9862 | 0.9802 | 0.9965 |
|
|
XGBoost | 0.9684 | 0.9886 | 0.9886 | 0.9785 | 0.9965 |
|
|
LightGBM | 0.9691 | 0.9884 | 0.9884 | 0.9788 | 0.9966 |
|
|
GBM | 0.7954 | 0.9506 | 0.9504 | 0.8730 | 0.9350 |
|
|
ERT | 0.9010 | 0.9270 | 0.9270 | 0.9140 | 0.9580 |
|
|
LR | 0.9519 | 0.9906 | 0.9906 | 0.9713 | 0.9935 |
|
|
RF | 0.9040 | 0.9492 | 0.9492 | 0.9266 | 0.9806 |
|
|
DNN | 0.9345 | 0.9784 | 0.9783 | 0.9565 | 0.9860 |
|
|
AdaBoost+XGBoost | 0.9687 | 0.9887 | 0.9887 | 0.9787 | 0.9966 |
|
|
AdaBoost+LightGBM | 0.9693 | 0.9886 | 0.9885 | 0.9789 | 0.9966 |
|
|
XGBoost+LigtGBM | 0.9690 | 0.9885 | 0.9885 | 0.9788 | 0.9965 |
|
|
AdaBoost+XGBoost+LightGBM | 0.96914 | 0.9886 | 0.9885 | 0.9788 | 0.9966 |
|
|
In-hospital mortality AI [5] | 0.9441 | 0.9855 | 0.9855 | 0.9648 | 0.9923 |
| Traditional methods | ||||||
|
|
Inclusive SRRl | 0.8716 | 0.9389 | 0.9388 | 0.9052 | 0.9328 |
|
|
Exclusive SRR | 0.9070 | 0.9227 | 0.9227 | 0.9149 | 0.9567 |
|
|
KTASm | 0.9465 | 0.9777 | 0.9777 | 0.9621 | 0.9405 |
aAUROC: area under the receiver operating characteristic curve.
bICD-10: International Classification of Disease 10th revision.
cAdaBoost: adaptive boosting.
dXGBoost: extreme gradient boosting.
eLightGBM: light gradient boosting machine.
fGBM: gradient boosting machine.
gERT: extremely random trees.
hLR: logistic regression.
iRF: random forest.
jDNN: deep neural network.
kAI: artificial intelligence.
lSRR: survival risk ratio.
mKTAS: Korean Triage and Acuity Scale.