Table 4.
Performance of the predictive model evaluated by the area under the receiver operating characteristic curve (AUC), negative predictive value (NPV), positive predictive value (PPV), sensitivity, specificity, and accuracy using the 90th, 95th, and 99th percentiles as thresholds in the 2 validation cohorts.
| Characteristic | Prediction score threshold | |||
| >90th percentile | >95th percentile | >99th percentile | ||
| Validation cohort 1 (n=289,426)a | ||||
|
|
AUC (95% CI) | 0.779 (0.774-0.785) | 0.779 (0.774-0.785) | 0.779 (0.774-0.785) |
|
|
NPV (95% CI) | 98.6% (98.6%-98.7%) | 98.4% (98.4%-98.4%) | 98.0% (98.0%-98.0%) |
|
|
PPV (95% CI) | 9.6% (9.3%-9.9%) | 13.5% (13.1%-14.0%) | 25.5% (24.1%-27.2%) |
|
|
Sensitivity (95% CI) | 43.8% (42.5%-45.0%) | 30.5% (29.3%-31.7%) | 11.5% (10.7%-12.3%) |
|
|
Specificity (95% CI) | 90.7% (90.6%-90.8%) | 95.6% (95.5%-95.7%) | 99.2% (99.2%-99.3%) |
|
|
Accuracy (95% CI) | 89.6% (89.5%-89.7%) | 94.1% (94.1%-94.2%) | 97.3% (97.3%-97.3%) |
| Validation cohort 2 (n=1815)b | ||||
|
|
AUC (95% CI) | 0.766 (0.686-0.845) | 0.766 (0.686-0.845) | 0.766 (0.686-0.845) |
|
|
NPV (95% CI) | 98.8% (98.4%-99.1%) | 98.3% (98.0%-98.7%) | 98.0% (97.8%-98.3%) |
|
|
PPV (95% CI) | 8.3% (5.9%-10.7%) | 9.2% (5.1%-13.9%) | 16.7% (4.3%-29.6%) |
|
|
Sensitivity (95% CI) | 52.5% (37.5%-67.5%) | 30.0% (15.0%-45.0%) | 12.5% (2.5%-22.5%) |
|
|
Specificity (95% CI) | 86.9% (85.3%-88.4%) | 93.4% (92.2%-94.5%) | 98.6% (98.0%-99.1%) |
|
|
Accuracy (95% CI) | 86.2% (84.4%-87.7%) | 92.0% (90.7%-93.2%) | 96.7% (96.1%-97.3%) |
aHosmer-Lemeshow test: χ298=13.1; P=.99.
bHosmer-Lemeshow test: χ28=4.8; P=.78.