Table 3.
Model evaluation | Prediction models | |||
---|---|---|---|---|
Developed model | Inouye et al. 1993 | Mehta, et al. 2011b | Sager et al. 1996 | |
OR (95% CI) | 1.12 (1.08–1.17) | 2.44 (1.56–3.81) | 1.26 (1.12–1.41) | 1.62 (1.24–2.12) |
Discrimination a | 0.75 (0.68–0.83) | 0.67 (0.59–0.74) | 0.68 (0.61–0.76) | 0.65 (0.57–0.73) |
Calibration | Chi2 = 6.35, p = 0.499 c | Chi2 = 0.11, p = 0.948 d | Chi2 = 11.80, p = 0.0667 e | Chi2 = 11.80, p = 0.0667 f |
Clinical usefulness |
Cutoff value = 13 Sensitivity = 71% Specificity = 70% PPV = 54% NPV = 83% Correctly classified = 70% |
Cutoff value = 1 Sensitivity = 46.0% Specificity = 80.2% PPV = 54% NPV = 75% Correctly classified = 69% |
Cutoff value = 4 Sensitivity = 65% Specificity = 60% PPV = 45% NPV = 78% Correctly classified = 62% |
Cutoff value = 4 Sensitivity = 60% Specificity = 61% PPV = 44% NPV = 76% Correctly classified = 61% |
Abbreviations: PPV Positive Predictive value, NPV Negative Predictive Value, CI Confidence Interval;
a Discrimination was assessed using the C-index, and models were compared using the chi2 test for equality for two or more Receiver Operating Characteristic areas: chi2 = 12.8, p = 0.005;
b Sensitivity analysis for Mehta et al. 2011 using a complete case analysis instead of multiple imputation: OR = 1.18 (1.03–1.36); Discrimination: 0.63, 95% CI (0.54–0.73); Calibration: chi2 = 4.27, p = 0.640;
Calibration was assessed using the Hosmer – Lemeshow goodness of fit test. The goodness of fit could only be assessed in quantiles of c 9 groups, d 4 groups, e 8 groups and f 5 groups because of ties in the data;