Table 2.
Summary of the system’s performance and statistics for PD-L1 and PD-1 in the BCCA and MA31 cohorts
| Biomarker | Group (+model) | AUC (95% CI) | NPV | Sensitivity | Specificity | Accuracy | # Patients | # Patients predicted as: (%) | % Positive patients | % Positive patients in: | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Low-PS | High-PS | Low-PS | High-PS | |||||||||
| PD-L1 | BCCA-CV | 0.911 (0.891 – 0.925) | 0.976 | 0.916 | 0.672 | 0.907 | 2516 | 1446 (57.5%) | 1070 (42.5%) | 16.60% | 2.40% | 35.70% |
| BCCA-test | 0.915 (0.883 – 0.937) | 0.983 | 0.943 | 0.646 | 0.900 | 860 | 473 (55.0%) | 387 (45.0%) | 16.30% | 1.70% | 34.10% | |
| BCCA-test-con | 0.928 (0.902 – 0.948) | 0.991 | 0.971 | 0.646 | 0.904 | 856 | 469 (54.8%) | 387 (45.2%) | 15.90% | 0.90% | 34.10%s | |
| MA31 | 0.854 (0.771 – 0.908) | 0.993 | 0.957 | 0.563 | 0.869 | 275 | 143 (52.0%) | 132 (48.0%) | 8.40% | 0.70% | 16.70% | |
| MA31-cal | 0.886 (0.805 – 0.934) | 0.989 | 0.913 | 0.747 | 0.862 | 268 | 185 (69.0%) | 83 (31.0%) | 8.60% | 1.10% | 25.30% | |
| MA31-cal-con | 0.919 (0.864 – 0.952) | 1.000 | 1.000 | 0.757 | 0.876 | 258 | 181 (70.2%) | 77 (29.8%) | 7.40% | 0.00% | 24.70% | |
| PD-1 | BCCA-CV | 0.848 (0.807 – 0.883) | 0.998 | 0.974 | 0.422 | 0.966 | 2618 | 1074 (41.0%) | 1074 (41.0%) | 3.00% | 0.20% | 4.90% |
| BCCA-test | 0.825 (0.697 – 0.890) | 0.994 | 0.913 | 0.377 | 0.966 | 877 | 322 (36.7%) | 555 (63.3%) | 3.30% | 0.60% | 4.90% | |
| BCCA-test-con | 0.875 (0.808 – 0.920) | 1.000 | 1.000 | 0.377 | 0.968 | 875 | 320 (36.6%) | 555 (63.4%) | 3.10% | 0.00% | 4.90% | |
MA31-cal stands for the calibrated MA31 model. BCCA-con and MA31-con stand for the analysis after removing cases with PD-L1/PD-1 annotations that were discordant between the pathologists. The specificity states the percent of negative cases that were classified by the system as low-PS (certainly negative), while the sensitivity can be seen as the percent of positive cases that passed the quality assurance. I.e., cases that were not recommended for re-staining or re-interpretation. The trade-off between the sensitivity and specificity is visualized by the corresponding ROC curves (Fig. 2). Note that the specificity can also be interpreted as the probability that the system will detect any random false positive misclassification. This is because it is the probability of a true negative sample, which was erroneously classified as positive, to be in the low-PS group. NPV - Negative predictive value.