Table 2.
Binary outcome prediction performance on different models using restricted sample data (positive label percentage: 7% for mortality, 5% for intubation, 7% for ICU transfer)
| Time window | Event | Model type | AUROC | AUPRC |
|---|---|---|---|---|
| 24 h | mortality | LG | 0.78 (0.02) | 0.51 (0.02) |
| RF | 0.61 (0.02) | 0.24 (0.03) | ||
| SVM | 0.60 (0.02) | 0.22 (0.02) | ||
| XGB | 0.65 (0.02) | 0.25 (0.01) | ||
| RNN + CEL | 0.83 (0.03) | 0.53 (0.05) | ||
| RETAIN + CEL | 0.86 (0.02) | 0.50 (0.05) | ||
| RNN + CL | 0.91∗ (0.03) | 0.62∗ (0.04) | ||
| RETAIN + CL | 0.91 (0.01) | 0.59 (0.05) | ||
| intubation | LG | 0.76 (0.02) | 0.33 (0.02) | |
| RF | 0.75 (0.01) | 0.34 (0.01) | ||
| SVM | 0.75 (0.02) | 0.32 (0.02) | ||
| XGB | 0.77 (0.02) | 0.39 (0.02) | ||
| RNN + CEL | 0.79 (0.03) | 0.35 (0.03) | ||
| RETAIN + CEL | 0.80 (0.02) | 0.35 (0.03) | ||
| RNN + CL | 0.88∗ (0.02) | 0.48∗ (0.03) | ||
| RETAIN + CL | 0.88 (0.02) | 0.45 (0.02) | ||
| ICU transfer | LG | 0.77 (0.01) | 0.39 (0.02) | |
| RF | 0.74 (0.01) | 0.36 (0.02) | ||
| SVM | 0.76 (0.02) | 0.38 (0.01) | ||
| XGB | 0.78 (0.02) | 0.36 (0.01) | ||
| RNN + CEL | 0.78 (0.01) | 0.41 (0.02) | ||
| RETAIN + CEL | 0.76 (0.03) | 0.43 (0.04) | ||
| RNN + CL | 0.86 (0.02) | 0.53∗ (0.03) | ||
| RETAIN + CL | 0.85∗ (0.02) | 0.51 (0.03) | ||
| 48 h | mortality | LG | 0.77 (0.02) | 0.33 (0.02) |
| RF | 0.57 (0.02) | 0.12 (0.02) | ||
| SVM | 0.62 (0.02) | 0.19 (0.03) | ||
| XGB | 0.69 (0.02) | 0.29 (0.02) | ||
| RNN + CEL | 0.85 (0.03) | 0.55 (0.04) | ||
| RETAIN + CEL | 0.90 (0.03) | 0.53 (0.03) | ||
| RNN + CL | 0.92∗ (0.03) | 0.63 (0.04) | ||
| RETAIN + CL | 0.91 (0.02) | 0.64∗ (0.04) | ||
| intubation | LG | 0.79 (0.01) | 0.36 (0.02) | |
| RF | 0.78 (0.02) | 0.35 (0.01) | ||
| SVM | 0.79 (0.01) | 0.34 (0.01) | ||
| XGB | 0.82 (0.02) | 0.40 (0.01) | ||
| RNN + CEL | 0.70 (0.03) | 0.34 (0.04) | ||
| RETAIN + CEL | 0.74 (0.02) | 0.31 (0.03) | ||
| RNN + CL | 0.83 (0.02) | 0.44 (0.02) | ||
| RETAIN + CL | 0.85∗ (0.02) | 0.44∗ (0.03) | ||
| ICU transfer | LG | 0.79 (0.02) | 0.41 (0.01) | |
| RF | 0.75 (0.01) | 0.35 (0.01) | ||
| SVM | 0.77 (0.01) | 0.37 (0.02) | ||
| XGB | 0.79 (0.01) | 0.41 (0.02) | ||
| RNN + CEL | 0.72 (0.04) | 0.38 (0.03) | ||
| RETAIN + CEL | 0.75 (0.04) | 0.43 (0.04) | ||
| RNN + CL | 0.82∗ (0.01) | 0.51∗ (0.02) | ||
| RETAIN + CL | 0.82 (0.02) | 0.50 (0.03) |
All predictions are calculated from 10-fold cross-validation, for which we record the mean value and standard deviation as confident intervals across folds. Asterisks (∗) indicate best model performance per event.