Table 3.
Predictive performance of models for MAKE prediction in the derivation (UKY) and validation (UTSW) cohorts.
| Performance metrics (95% CI) | Maximum AKI KDIGO Severity Score | Clinical model * | ||
|---|---|---|---|---|
| Derivation Cohort (UKY) | Validation Cohort (UTSW) | Derivation Cohort (UKY) | Validation Cohort (UTSW) | |
| AUC | 0.66(0.66–0.66) | 0.67(0.67–0.67) | 0.78 (0.78–0.78) | 0.73 (0.72–0.74) |
| Difference in AUC (vs. KDIGO score) | - | - | 0.12 | 0.06 |
| - P value | - | - | <0.001 | <0.001 |
| Accuracy | 0.62(0.61–0.63) | 0.68(0.64–0.72) | 0.72 (0.71–0.72) | 0.67 (0.66–0.69) |
| Precision | 0.45(0.44–0.46) | 0.43(0.38–0.47) | 0.55 (0.54–0.56) | 0.42 (0.40–0.43) |
| Sensitivity | 0.61(0.58–0.63) | 0.54(0.47–0.62) | 0.69 (0.69–0.70) | 0.67 (0.64–0.69) |
| Specificity | 0.62(0.60–0.65) | 0.73(0.65–0.81) | 0.73 (0.72–0.73) | 0.68 (0.64–0.71) |
| F1 | 0.51(0.50–0.51) | 0.47(0.46–0.47) | 0.61 (0.61–0.62) | 0.51 (0.50–0.52) |
| PPV | 0.45(0.44–0.46) | 0.43(0.38–0.47) | 0.55 (0.54–0.56) | 0.42 (0.40–0.43) |
| NPV | 0.77(0.77–0.78) | 0.82(0.82–0.83) | 0.83 (0.83–0.83) | 0.85 (0.85–0.86) |
| Calibration Intercept | −0.74(−0.75 to −0.73) | −0.97(−0.98 to −0.96) | −0.72 (−0.73 to −0.70) | −1.10 (−1.17 to −1.02) |
| Calibration Slope | 1.02(0.99–1.05) | 1.25(1.20–1.29) | 1.07 (1.05–1.09) | 1.34 (1.27–1.41) |
| NRI % (vs. KDIGO score) | ||||
| - Categorical | - | - | 0.20[0.17 to 0.22] | 0.11[0.06 to 0.16] |
| - P value | - | - | <0.001 | <0.001 |
The proposed clinical model included 14 features, refer to the Methods section for details. The machine learning method for the reported performance evaluation of maximum AKI KDIGO severity score is Logistic Regression and for the proposed clinical model is Random Forest.
Abbreviations: APACHE (acute physiologic assessment and chronic health evaluation), AUC (area under the curve), CI (confidence interval), F1 (F1 score), KDIGO (Kidney Disease: Improving Global Outcomes), MAKE (major adverse kidney events), NPV (negative predictive value), NRI (net reclassification index), PPV (positive predictive value), SOFA (sequential organ failure assessment), UKY (University of Kentucky), UTSW (University of Texas Southwestern)