Table 3.
Model performance evaluation and comparison
| Model | AUC (95% CI) | Accuracy | Sensitivity | Specificity | P value | |
|---|---|---|---|---|---|---|
|
Training Cohort (Centre I) |
Clinical model | 0.771 (0.741–0.802) | 0.715 | 0.595 | 0.828 | 0.000 |
| Rad ANN model | 0.856 (0.830–0.880) | 0.791 | 0.964 | 0.629 | 0.000 | |
| Combined model | 0.899 (0.878–0.920) | 0.831 | 0.929 | 0.739 | - | |
|
Validation Cohort I (Centre II) |
Clinical model | 0.689 (0.566–0.805) | 0.752 | 0.435 | 0.833 | 0.005 |
| Rad ANN model | 0.781 (0.669–0.870) | 0.681 | 0.826 | 0.644 | 0.173 | |
| Combined model | 0.826 (0.732–0.910) | 0.717 | 0.739 | 0.711 | - | |
|
Validation Cohort II (Centre III) |
Clinical model | 0.620 (0.514–0.718) | 0.789 | 0.250 | 0.919 | 0.000 |
| Rad ANN model | 0.809 (0.733–0.875) | 0.686 | 0.806 | 0.658 | 0.850 | |
| Combined model | 0.812 (0.735–0.881) | 0.762 | 0.694 | 0.779 | - | |
|
Validation Cohort III (Centre IV) |
Clinical model | 0.643 (0.572–0.716) | 0.756 | 0.444 | 0.828 | 0.000 |
| Rad ANN model | 0.783 (0.722–0.835) | 0.706 | 0.764 | 0.693 | 0.112 | |
| Combined model | 0.803 (0.748–0.852) | 0.738 | 0.667 | 0.754 | - |
Note CI, confidence interval; AUC, Receiver Operating Characteristic curves and Area Under the Curve. BPNN, the back propagation neural network algorithm. The P value was calculated by DeLong test