Skip to main content
. 2022 May 31;10(5):e35293. doi: 10.2196/35293

Table 3.

Information on the MLa prediction model development, validation, and performance, and on the severity of illness score performance.

Author ML model type (AUROCb test) Data training/test (split %) Features K-fold/validation External validation data set ML AUROC external Severity of illness score model type (AUROC)
Pirracchio et al, 2015 [1]

  • Ensemble SICULAc (0.85)

 24,508 17 5-fold cross-validation 200 0.94

  • SAPSd-II (0.78)

  • APACHEe-II (0.83)

  • SOFf (0.71)
Nielsen et al, 2019 [24]

  • NNg (0.792)

 10,368
(80/20)		 44 5-fold cross-validation 1528 0.773

  • SAPS-II (0.74)

  • APACHE-II (0.72)
Nimgaonkar et al, 2004 [25]

  • NN (0.88)

 2962
(70/30)		 15 N/Ah N/A N/A

  • APACHE-II (0.77)
Xia et al, 2019 [26]

  • Ensemble-LSTMi (0.85)

  • LSTM (0.83)

  • DTj (0.82)

 18,415
(90/10)		 50 Bootstrap and RSMk N/A N/A

  • SAPS-II (0.77)

  • SOFA (0.73)

  • APACHE-II (0.74)
Purushotham et al, 2018 [27]

  • NN (0.87)

  • Ensemble (0.84)

 35,627 17/22/
136		 5-fold cross-validation External benchmark N/A

  • SAPS-II (0.80)

  • SOFA (0.73)
Nanayakkara et al, 2018 [28]

  • DT (0.86)

  • SVMl (0.86)

  • NN (0.85)

  • Ensemble (0.87)

  • GBMm (0.87)

 39,560
(90/10)		 29 5-fold cross-validation N/A N/A

  • APACHE-III (0.8)
Meyer et al, 2018 [29]

  • NN (0.95)

 5898
(90/10)		 52 10-fold cross-validation 5989 0.81

  • SAPS-II (0.71)
Meiring et al, 2018 [7]

  • DT (0.85)

  • NN (0.86)

  • SVM (0.86)

 80/20 25 21,911
LOOn 		N/A N/A

  • APACHE-II (0.83)
Lin et al, 2019
[30]

  • DT (0.86)

  • NN (0.83)

  • SVM (0.86)

 19,044 15 5-fold cross-validation N/A N/A

  • SAPS-II (0.79)
Krishnan et al, 2018 [31]

  • NN-ELMo (0.99)

 10,155
(75/25)		 1 10-fold cross-validation N/A N/A

  • SAPS (0.80)

  • SOFA (0.73)

  • APSp-III (0.79)
Kang et al, 2020 [32]

  • SVM (0.77)

  • DT (0.78)

  • NN (0.776)

  • k-NNq (0.76)

 1571
(70/30)		 33 10-fold cross-validation N/A N/A

  • SOFA (0.66)

  • APACHE-II (0.59)
Johnson et al, 2013 [33]

  • LRr univariate (0.902)

  • LR multivariate (0.876)

 39,070
(80/20)		 10 10-fold cross-validation 23,618 0.837 (univariate); 0.868 (multivariate)

  • APS-III (0.86)
Holmgren et al, 2019 [34]

  • NN (0.89)

 217,289
(80/20)		 8 5-fold cross-validation N/A N/A

  • SAPS-III (0.85)
Garcia-Gallo et al, 2020 [35]

  • SGB-LASSOs (0.803)

 5650
(70/30)		 18
140
37		 10-fold cross-validation N/A N/A

  • SOFA (0.58)

  • SAPS (0.70)
El-Rashidy et al, 2020 [36]

  • Ensemble (0.93)

 10,664
(75/25)		 80 10-fold cross-validation External benchmark N/A

  • APACHE-II (0.73)

  • SAPS-II (0.81)

  • SOFA-II (0.78)
Silva et al, 2006 [37]

  • NN (0.85)

 13,164
(66/33)		 12 Hold out N/A N/A

  • SAPS- II (0.8)
Caicedo-Torres et al, 2019 [38]

  • NN (0.87)

 22,413 22 5-fold cross-validation N/A N/A

  • SAPS-II (0.73)
Deshmukh et al, 2020 [39]

  • XGBt (0.85)

 5691
(80/20)		 34 5-fold cross-validation N/A N/A

  • APACHE-IV (0.8)
Ryan et al, 2020 [40]

  • DT (0.86)

 35,061
(80/20)		 12 5-fold cross-validation 114 0.91

  • qSOFAu (0.76)
Mayaud et al, 2013 [41]

  • GAv+LR (0.82)

 2113
(70/30)		 25 BBCCVw N/A N/A

  • APACHE-III (0.68)

aML: machine learning.

bAUROC: area under the receiver operating curve.

cSICULA: Super ICU Learner Algorithm.

dSAPS: Simplified Acute Physiology Score.

eAPACHE: Acute Physiology and Chronic Health Evaluation.

fSOFA: Sequential Organ Failure Assessment.

gNN: neural network.

hN/A: not applicable.

iLSTM: long short-term memory.

jDT: decision tree.

kRSM: random subspace method.

lSVM: support vector machine.

mGBM: gradient boosting machine.

nLOO: leave one out.

oELM: extreme learning machine.

pAPS: Acute Physiology Score.

qk-NN: k-nearest neighbor.

rLR: logistic regression.

sSGB-LASSO: stochastic gradient boosting least absolute shrinkage and selection operator.

tXGB: extreme gradient boosting.

uqSOFA: Quick Sequential Organ Failure Assessment.

vGA: genetic algorithm.

wBBCV: bootstrap bias–corrected cross-validation.