Table 1.
Annotation level | Performance metric | 1st quartile | Median | Mean | 3rd quartile |
---|---|---|---|---|---|
Class | Sensitivity/recall | 99.01 | 99.50 | 99.31 | 99.96 |
Specificity | 99.95 | 99.98 | 99.96 | 100.00 | |
Precision | 77.73 | 88.33 | 87.56 | 100.00 | |
ROC-AUC | 99.94 | 99.98 | 99.97 | 100.00 | |
PR-AUC | 80.32 | 92.86 | 88.95 | 100.00 | |
Group | Sensitivity/recall | 99.76 | 100.00 | 98.91 | 100.00 |
Specificity | 99.93 | 100.00 | 99.92 | 100.00 | |
Precision | 74.74 | 100.00 | 85.65 | 100.00 | |
ROC-AUC | 99.98 | 100.00 | 99.92 | 100.00 | |
PR-AUC | 78.30 | 100.00 | 88.90 | 100.00 | |
Mechanism | Sensitivity/recall | 99.72 | 100.00 | 97.50 | 100.00 |
Specificity | 99.94 | 99.98 | 99.95 | 100.00 | |
Precision | 73.79 | 88.17 | 85.58 | 100.00 | |
ROC-AUC | 99.98 | 100.00 | 99.81 | 100.00 | |
PR-AUC | 76.39 | 94.57 | 87.21 | 100.00 | |
Model | Sensitivity/recall | 99.72 | 100.00 | 97.50 | 100.00 |
Specificity | 97.43 | 99.93 | 100.00 | 100.00 | |
Precision | 49.98 | 100.00 | 75.73 | 100.00 | |
ROC-AUC | 99.97 | 100.00 | 99.91 | 100.00 | |
PR-AUC | 52.78 | 100.00 | 79.80 | 100.00 |
All metrics excluding the AUC were calculated at the E-value threshold of 1e-25, which optimized the PR curve. We note that other thresholds might be useful depending on the false-positive tolerance of the use-case