Table 2.
Internal Validation Test Performance Metrics of the Mayo Risk Score and the Combined Biomarker Model
| Pooled Cohorts (N = 456) |
||
|---|---|---|
| Mayo (P.A.) | CBM | |
| ROC AUC | 0.74 (0.67–0.80) | 0.86 (0.81–0.90) |
| Difference | 0.124 (95% CI: 0.091–0.156) | |
| Precision-recall AUC | 0.77 (0.70–0.83) | 0.88 (0.85–0.91) |
| Likelihood ratio test | ||
| Residual degree of freedom | 453 | 449 |
| Residual deviance | 543.55 | 409.16 |
| Degree of freedom | — | 4 |
| Deviance | — | 134.39 |
| P value | — | <2×10−16 |
| Youden cutoff | 0.35 (0.21–0.48) | 0.36 (0.27–0.56) |
| Optimal criterion | 0.44 (0.35–0.53) | 0.59 (0.50–0.68) |
| Sensitivity | 0.75 (0.65–0.89) | 0.80 (0.69–0.90) |
| Specificity | 0.70 (0.55–0.82) | 0.82 (0.67–0.92) |
| DLR− | 0.35 (0.21–0.57) | 0.25 (0.13–0.35) |
| DLR+ | 2.49 (1.90–3.58) | 4.32 (2.67–8.52) |
| Rule-out threshold: 0.1 | ||
| Sensitivity | 0.98 (0.93–1.00) | 0.98 (0.92–1.00) |
| Specificity | 0.09 (0.00–0.22) | 0.43 (0.32–0.53) |
| DLR− | 0.13 (0.00–0.44) | 0.06 (0.00–0.18) |
| Rule-in threshold: 0.7 | ||
| Sensitivity | 0.03 (0.00–0.19) | 0.45 (0.35–0.55) |
| Specificity | 0.99 (0.94–1.00) | 0.96 (0.91–1.00) |
| DLR+ | 3.72 (0.43–inf) | 13.1 (5.57–inf) |
| Reclassification | ||
| cNRI case | — | 0.15 (0.10–0.20) |
| cNRI control | — | 0.12 (0.07–0.18) |
Definition of abbreviations: AUC = area under the ROC curve; CBM = combined biomarker model; CI = confidence interval; cNRI = bias-corrected clinical net reclassification index; DLR = diagnostic likelihood ratio; inf = infinity; Mayo = Mayo model; P.A. = prevalence adjusted; ROC = receiver operating characteristics.
The baseline Mayo clinic model calibrated to the training set, then adjusted to a prevalence of cancer of 0.33. All values reported as mean (95% CI) of the test set in the 200 repeated threefold splitting procedures.