TABLE 1.
Statistics for the 15-antimicrobial phenotypic susceptibility/resistance prediction task for all antimicrobial resistance (AMR) determinant pipeline/database combinationsa.
| AMR Pipeline | AMR Database | % Accuracy (95% Confidence Interval) | Cohen’s Kappa (%) | Corrected Accuracy P-Valueb | Corrected McNemar’s Test P-Valueb | Sensitivity (%) | Specificity (%) |
| ABRicate | ARG-ANNOT | 97.2 (96.3–97.9) | 87.3 | 1.72E-59 | 2.67E-05 | 82.8 | 99.5 |
| ABRicate | CARD | 97.2 (96.3–97.9) | 87.3 | 1.72E-59 | 2.67E-05 | 82.8 | 99.5 |
| ABRicate | NCBI | 97.2 (96.3–97.9) | 87.4 | 1.72E-59 | 9.94E-05 | 83.2 | 99.4 |
| ABRicate | ResFinder | 97.2 (96.3–97.9) | 87.3 | 1.72E-59 | 2.67E-05 | 82.8 | 99.5 |
| AMRFinderPlus | NCBI | 96.5 (95.6–97.3) | 83.9 | 1.41E-50 | 1.41E-08 | 77.5 | 99.5 |
| ARIBA | ARG-ANNOT | 96.8 (95.9–97.5) | 85.3 | 7.37E-54 | 6.63E-07 | 79.8 | 99.5 |
| ARIBA | CARD | 95.8 (94.8–96.6) | 79.9 | 3.08E-42 | 3.14E-12 | 71.8 | 99.6 |
| ARIBA | MEGARes | 96.5 (95.6–97.3) | 84.3 | 1.41E-50 | 1.53E-04 | 80.2 | 99.1 |
| ARIBA | NCBI | 96.8 (95.9–97.6) | 85.5 | 1.55E-54 | 1.06E-06 | 80.2 | 99.5 |
| ARIBA | ResFinder | 96.7 (95.8–97.5) | 85.0 | 3.45E-53 | 4.15E-07 | 79.4 | 99.5 |
| BTyper | ARG-ANNOT | 97.2 (96.3–97.9) | 87.3 | 1.72E-59 | 2.67E-05 | 82.8 | 99.5 |
| BTyper | MEGARes | 97.2 (96.3–97.9) | 87.3 | 1.72E-59 | 2.67E-05 | 82.8 | 99.5 |
| SRST2 | ARG-ANNOT | 97.4 (96.6–98.1) | 88.4 | 1.69E-62 | 1.63E-04 | 84.4 | 99.5 |
| SRST2 | ResFinder | 97.3 (96.5–98.0) | 88.1 | 9.85E-62 | 1.04E-04 | 84.0 | 99.5 |
aStatistics were calculated using the confusionMatrix function in the caret package in R, with resistant (“R”) phenotypes/genotypes treated as the “positive” result and susceptible (“S”) phenotypes/genotypes treated as the “negative” result; See Supplementary Table 3 for an extended version of this table.
bAdjusted using a Bonferroni correction.