Table 3.
Performance of the CAD models based on the determined sample size for various fixed and adaptive sample size methods. Results are shown as medians (with interquartile ranges) across 500 repetitions
| Sample size | Bootstrap-corrected performance | |||
|---|---|---|---|---|
| Sample size method | N | EPP | AUC | Slope |
| Basic strategy | ||||
| Fixeda: 10 EPP | 300 (250–300) | 11 (10–11) | 0.706 (0.684–0.726) | 0.774 (0.751–0.795) |
| Fixeda: Riley’s method | 700 (700–700) | 28 (27–28) | 0.713 (0.701–0.726) | 0.894 (0.886–0.901) |
| Adaptive: stopping rule 1b | 850 (750–900) | 33 (30–35) | 0.717 (0.705–0.727) | 0.909 (0.905–0.914) |
| Adaptive: stopping rule 2b | 1500 (1400–1550) | 59 (56–62) | 0.719 (0.712–0.725) | 0.949 (0.946–0.952) |
| Restricted cubic splines | ||||
| Fixeda: 10 EPP | 350 (350–400) | 11 (10–11) | 0.703 (0.684–0.721) | 0.766 (0.745–0.783) |
| Fixeda: Riley’s method | 900 (900–900) | 26 (26–27) | 0.712 (0701–0.724) | 0.887 (0.877–0.894) |
| Adaptive: stopping rule 1b | 1100 (1050–1200) | 31 (28–33) | 0.715 (0.705–0.724) | 0.907 (0.904–0.911) |
| Adaptive: stopping rule 2b | 1850 (1800–1900) | 58 (55–60) | 0.718 (0.712–0.724) | 0.947 (0.945–0.949) |
| Firth’s correction | ||||
| Fixeda: 10 EPP | 300 (250–300) | 11 (10–11) | 0.706 (0.682–0.726) | 0.822 (0.797–0.846) |
| Fixeda: Riley’s method | 700 (700–700) | 28 (27–28) | 0.712 (0.700–0.726) | 0.913 (0.904–0.923) |
| Adaptive: stopping rule 1b | 750 (700–800) | 31 (28–32) | 0.713 (0.703–0.727) | 0.922 (0.917–0.928) |
| Adaptive: stopping rule 2b | 1500 (1450–1600) | 59 (56–63) | 0.718 (0.710–0.726) | 0.958 (0.956–0.961) |
| Including backward selection | ||||
| Fixeda: 10 EPP | 300 (250–300) | 11 (10–11) | 0.690 (0.666–0.715) | 0.798 (0.772–0.819) |
| Fixeda: Riley’s method | 700 (700–700) | 28 (27–28) | 0.708 (0.694–0.722) | 0.894 (0.883–0.905) |
| Adaptive: stopping rule 1b | 800 (750–900) | 33 (29–36) | 0.712 (0.701–0.723) | 0.909 (0.905–0.915) |
| Adaptive: stopping rule 2b | 1550 (1400–1650) | 61 (56–65) | 0.716 (0.709–0.724) | 0.948 (0.946–0.951) |
AUC area under the receiver operating characteristic curve (or c-statistic), slope calibration slope, EPP events per parameter
aThe analysis went in batches of 50 patients, therefore fixed sample sizes were rounded upwards to the next multiple of 50
bStopping rule 1: calibration slope ≥ 0.9 and AUC optimism < = 0.02 at two consecutive assessments. Stopping rule 2: calibration slope ≥ 0.9 and AUC optimism < = 0.01 at two consecutive assessments