Table 2.
Evaluation Metric | |||||||||
---|---|---|---|---|---|---|---|---|---|
Dataset | Model | Threshold | AUROC | Accuracy | Precision | F1 Score | AUPRC | TPR | FPR |
MIMIC | Short | 1st | 0.78 (SD 0.02) | 0.76 (SD 0.01) | 0.39 (SD 0.02) | 0.53 (SD 0.03) | 0.35 (SD 0.03) | 0.80 (SD 0.05) | 0.25 (SD 0.02) |
2nd | 0.69 (SD 0.03) | 0.83 (SD 0.01) | 0.49 (SD 0.03) | 0.48 (SD 0.04) | 0.32 (SD 0.03) | 0.48 (SD 0.06) | 0.10 (SD 0.02) | ||
Long | 1st | 0.80 (SD 0.01) | 0.77 (SD 0.03) | 0.41 (SD 0.04) | 0.55 (SD 0.03) | 0.37 (SD 0.03) | 0.85 (SD 0.04) | 0.25 (SD 0.04) | |
2nd | 0.75 (SD 0.02) | 0.85 (SD 0.02) | 0.55 (SD 0.05) | 0.57 (SD 0.03) | 0.40 (SD 0.03) | 0.61 (SD 0.06) | 0.10 (SD 0.03) | ||
MIMIC Incomplete absorption | Short | 1st | 0.73 (SD 0.03) | 0.72 (SD 0.03) | 0.64 (SD 0.04) | 0.71 (SD 0.03) | 0.59 (SD 0.03) | 0.79 (SD 0.06) | 0.33 (SD 0.06) |
2nd | 0.67 (SD 0.05) | 0.71 (SD 0.04) | 0.74 (SD 0.06) | 0.56 (SD 0.11) | 0.57 (SD 0.05) | 0.47 (SD 0.12) | 0.12 (SD 0.04) | ||
Long | 1st | 0.77 (SD 0.02) | 0.76 (SD 0.03) | 0.68 (SD 0.05) | 0.74 (SD 0.03) | 0.63 (SD 0.04) | 0.82 (SD 0.06) | 0.28 (SD 0.07) | |
2nd | 0.73 (SD 0.03) | 0.75 (SD 0.03) | 0.79 (SD 0.09) | 0.66 (SD 0.06) | 0.63 (SD 0.05) | 0.57 (SD 0.09) | 0.12 (SD 0.07) | ||
MIMIC Mal- absorption | Short | 1st | 0.76 (SD 0.10) | 0.64 (SD 0.17) | 0.44 (SD 0.25) | 0.57 (SD 0.23) | 0.44 (SD 0.25) | 1.00 (SD 0.00) | 0.48 (SD 0.21) |
2nd | 0.75 (SD 0.11) | 0.73 (SD 0.06) | 0.41 (SD 0.20) | 0.50 (SD 0.21) | 0.39 (SD 0.15) | 0.78 (SD 0.33) | 0.28 (SD 0.12) | ||
Long | 1st | 0.75 (SD 0.07) | 0.65 (SD 0.10) | 0.39 (SD 0.16) | 0.53 (SD 0.17) | 0.38 (SD 0.15) | 0.93 (SD 0.13) | 0.43 (SD 0.14) | |
2nd | 0.71 (SD 0.16) | 0.79 (SD 0.10) | 0.57 (SD 0.35) | 0.48 (SD 0.26) | 0.45 (SD 0.24) | 0.53 (SD 0.34) | 0.12 (SD 0.12) | ||
MIMIC UTI | Short | 1st | 0.77 (SD 0.03) | 0.71 (SD 0.03) | 0.38 (SD 0.07) | 0.52 (SD 0.07) | 0.35 (SD 0.07) | 0.87 (SD 0.05) | 0.33 (SD 0.03) |
2nd | 0.74 (SD 0.04) | 0.81 (SD 0.02) | 0.49 (SD 0.08) | 0.54 (SD 0.08) | 0.38 (SD 0.08) | 0.63 (SD 0.10) | 0.15 (SD 0.03) | ||
Long | 1st | 0.78 (SD 0.02) | 0.72 (SD 0.03) | 0.39 (SD 0.06) | 0.54 (SD 0.06) | 0.36 (SD 0.06) | 0.87 (SD 0.04) | 0.31 (SD 0.04) | |
2nd | 0.77 (SD 0.04) | 0.83 (SD 0.02) | 0.54 (SD 0.09) | 0.58 (SD 0.07) | 0.42 (SD 0.07) | 0.66 (SD 0.12) | 0.13 (SD 0.05) | ||
MIMIC Pneumonia | Short | 1st | 0.76 (SD 0.03) | 0.68 (SD 0.03) | 0.28 (SD 0.05) | 0.42 (SD 0.07) | 0.26 (SD 0.05) | 0.86 (SD 0.07) | 0.35 (SD 0.03) |
2nd | 0.76 (SD 0.03) | 0.82 (SD 0.02) | 0.41 (SD 0.05) | 0.51 (SD 0.04) | 0.32 (SD 0.05) | 0.68 (SD 0.07) | 0.16 (SD 0.03) | ||
Long | 1st | 0.77 (SD 0.02) | 0.70 (SD 0.04) | 0.30 (SD 0.06) | 0.44 (SD 0.07) | 0.28 (SD 0.05) | 0.87 (SD 0.03) | 0.32 (SD 0.04) | |
2nd | 0.74 (SD 0.04) | 0.82 (SD 0.03) | 0.41 (SD 0.09) | 0.48 (SD 0.08) | 0.31 (SD 0.07) | 0.62 (SD 0.10) | 0.14 (SD 0.04) | ||
MIMIC Sepsis | Short | 1st | 0.82 (SD 0.05) | 0.71 (SD 0.05) | 0.38 (SD 0.15) | 0.53 (SD 0.17) | 0.38 (SD 0.15) | 1.00 (SD 0.00) | 0.36 (SD 0.10) |
2nd | 0.79 (SD 0.12) | 0.82 (SD 0.08) | 0.53 (SD 0.25) | 0.56 (SD 0.20) | 0.43 (SD 0.18) | 0.76 (SD 0.27) | 0.17 (SD 0.09) | ||
Long | 1st | 0.77 (SD 0.07) | 0.68 (SD 0.08) | 0.35 (SD 0.14) | 0.48 (SD 0.14) | 0.33 (SD 0.13) | 0.90 (SD 0.13) | 0.35 (SD 0.08) | |
2nd | 0.76 (SD 0.18) | 0.80 (SD 0.10) | 0.46 (SD 0.21) | 0.52 (SD 0.22) | 0.42 (SD 0.16) | 0.68 (SD 0.33) | 0.16 (SD 0.07) | ||
eICU | Short | 1st | 0.72 (SD 0.02) | 0.75 (SD 0.03) | 0.36 (SD 0.05) | 0.47 (SD 0.05) | 0.29 (SD 0.04) | 0.68 (SD 0.06) | 0.24 (SD 0.04) |
2nd | 0.65 (SD 0.05) | 0.85 (SD 0.02) | 0.60 (SD 0.12) | 0.42 (SD 0.10) | 0.31 (SD 0.06) | 0.34 (SD 0.10) | 0.05 (SD 0.02) | ||
Long | 1st | 0.72 (SD 0.02) | 0.75 (SD 0.02) | 0.36 (SD 0.06) | 0.46 (SD 0.04) | 0.29 (SD 0.04) | 0.67 (SD 0.07) | 0.24 (SD 0.04) | |
2nd | 0.64 (SD 0.06) | 0.84 (SD 0.02) | 0.48 (SD 0.17) | 0.38 (SD 0.15) | 0.28 (SD 0.07) | 0.33 (SD 0.15) | 0.06 (SD 0.03) |
MIMIC;Medical Information Mart for Intensive Care, ICU;Intensive care unit, UTI;Urinary tract infection, SD;Standard deviation, AUROC;Area under the receiver operating characteristic, AUPRC;Area under precision-recall, TPR;True positive rate, FPR;False positive rate