Table 4.
Discrimination, calibration and clinical utility measures of pathologic and pre-operative prediction models
| Pathologic model | Pre-operative model | ||||
|---|---|---|---|---|---|
| Probability of LN positivity | |||||
| Criteria | Parametre | Initial set | Validation set | Initial set | Validation set |
| AUC | Est | 0.754 | 0.767 | 0.681 | 0.687 |
| 95%CI | [0.742–0.765] | [0.75–0.783] | [0.668–0.693] | [0.669–0.705] | |
| AUC (Bootstrap, | Est | 0.755 | 0.766 | 0.682 | 0.686 |
| B = 2000) | 95%CI | [0.744–0.767] | [0.762–0.769] | [0.669–0.694] | [0.682–0.69] |
| Clinical utility | < 10% | 19 (0%) | 7 (0%) | 1 (0%) | 0 (0%) |
| 10–20% | 2255 (32%) | 1096 (31%) | 579 (7%) | 279 (7%) | |
| 20–30% | 1258 (18%) | 586 (17%) | 3137 (40%) | 1487 (38%) | |
| 30–40% | 1108 (16%) | 559 (16%) | 2478 (32%) | 1315 (33%) | |
| 40–50 | 391 (5%) | 188 (5%) | 228 (3%) | 125 (3%) | |
| > = 50% | 2102 (29%) | 1076 (31%) | 1429 (18%) | 746 (19%) | |
| Calibration | p-value | 0.158 | 0.332 | 0.815 | 0.200 |
| Probability of macrometastases | |||||
| AUC (Delong) | Est | 0.780 | 0.798 | 0.718 | 0.727 |
| 95%CI | [0.767–0.792] | [0.78–0.815] | [0.703–0.732] | [0.707–0.746] | |
| AUC (Bootstrap, | Est | 0.780 | 0.796 | 0.717 | 0.725 |
| B = 2000) | 95%CI | [0.767–0.793] | [0.793–0.799] | [0.703–0.732] | [0.721–0.728] |
| Clinical utility | < 10% | 2029 (28%) | 1004 (29%) | 358 (5%) | 184 (5%) |
| 10–20% | 2289 (32%) | 1075 (31%) | 5049 (64%) | 2450 (62%) | |
| 20–30% | 512 (7%) | 262 (7%) | 829 (11%) | 465 (12%) | |
| 30–40% | 726 (10%) | 358 (10%) | 307 (4%) | 162 (4%) | |
| 40–50 | 644 (9%) | 378 (11%) | 573 (7%) | 306 (8%) | |
| > = 50% | 933 (13%) | 435 (12%) | 736 (9%) | 385 (10%) | |
| Calibration | p-value | 0.427 | 0.024 | 0.568 | 0.174 |