Table 2.
Accuracy, sensitivity, specificity, and AUC of the diagnostic model (BlcaMIL) and human pathologists.
| a. Accuracy, Sensitivity, and Specificity in the Diagnostic Model (BlcaMIL) | |||||
| Accuracy (95% CI) | Sensitivity (95% CI) | Specificity (95% CI) | AUC (95% CI) |
||
| Training set | 0.998 (0.996, 0.999) |
0.999 (0.998, 1.000) |
0.998 (0.996, 0.999) |
1.000 (1.000, 1.000) |
|
| Internal validation set | 0.998 (0.996, 1.000) |
1.000 (1.000, 1.000) |
0.996 (0.992, 1.000) |
1.000 (1.000, 1.000) |
|
| External validation set | 0.987 (0.981, 0.994) |
0.984 (0.971, 0.998) |
0.986 (0.979, 0.993) |
0.993 (0.990, 0.997) |
|
| b. Comparison of the BlcaMIL model and human pathologists in the external validation set | |||||
| Accuracy (95% CI) | Sensitivity (95% CI) | Specificity (95% CI) | p-Value * | Kappa # | |
| BlcaMIL Model | 0.987 (0.981, 0.994) |
0.984 (0.971, 0.998) | 0.986 (0.979, 0.993) | - | - |
| Expert Uropathologist A | 0.991 (0.987, 0.995) | 0.988 (0.981, 0.995) | 0.996 (0.989, 1.000) | 1.000 | 0.909 |
| Expert Uropathologist B | 0.993 (0.991, 0.995) | 0.991 (0.987, 0.995) | 0.996 (0.989, 1.000) | 1.000 | 0.925 |
| Junior Pathologist C | 0.876 (0.852, 0.900) | 0.834 (0.811, 0.858) | 0.940 (0.904, 0.976) | <0.0001 | 0.711 |
* A paired Chi-squared test (McNemar’s test) was used to examine differences in accuracy between the BlcaMIL model and each uropathologist. # Inter-observer agreement between the BlcaMIL model and each uropathologist assessed by the Cohen kappa coefficient. CI, Confidence Interval.