Table 1. Prediction results for majority prediction rule.
TPR | TNR | ACC | F1 | MCC | |
---|---|---|---|---|---|
PaPrBaG | 0.91 | 0.70 | 0.88 | 0.93 | 0.54 |
Bowtie2 | 0.95 | 0.66 | 0.91 | 0.95 | 0.61 |
Pathoscope2 | 0.94 | 0.72 | 0.91 | 0.95 | 0.62 |
Kraken | 0.97 | 0.64 | 0.93 | 0.96 | 0.66 |
Kraken-16 | 0.99 | 0.19 | 0.89 | 0.94 | 0.37 |
BLAST | 0.96 | 0.60 | 0.92 | 0.95 | 0.61 |
Bowtie2All Strains | 0.96 | 0.60 | 0.91 | 0.95 | 0.58 |
Pathoscope2All Strains | 0.96 | 0.66 | 0.92 | 0.95 | 0.63 |
NBC | 0.99 | 0.23 | 0.90 | 0.94 | 0.41 |
Bowtie2 + PaPrBaG | 0.97 | 0.77 | 0.95 | 0.97 | 0.71 |
Pathoscope2 + PaPrBaG | 0.97 | 0.81 | 0.95 | 0.97 | 0.74 |
BLAST + PaPrBaG | 0.97 | 0.74 | 0.95 | 0.97 | 0.70 |
Kraken + PaPrBaG | 0.97 | 0.76 | 0.95 | 0.98 | 0.73 |
Pathoscope2 + Kraken | 0.97 | 0.69 | 0.94 | 0.97 | 0.70 |
Pathoscope2 + NBC | 0.99 | 0.44 | 0.95 | 0.98 | 0.60 |
Bowtie2 + Kraken + PaPrBaG | 0.98 | 0.78 | 0.96 | 0.98 | 0.75 |
The first set of entries shows the performance of the individual methods. Bowtie All Strains and Pathoscope All Strains represent a variation where the reference data set contains all strains of a species in the training set. Below the horizontal line, we show results for the combination of methods with the consensus filter. In these cases, the performance is given for those genomes that have predictions agreeing between two or more individual classifiers. Overall, combining PaPrBaG with Bowtie2 and Kraken yields the best performance. (TPR = True positive rate, TNR = True negative rate, ACC = Accuracy, F1 = F1-score, MCC = Matthews Correlation Coefficient).