Table 2.

Evaluated performance metrics and their corresponding definitions and formulas

Metric	Definition	Formula	Assay-specific definitions	Bioinformatics assay
				Genotypic AMR characterization	Virulence gene detection		Serotype determination	Plasmid replicon detection	Sequence typing
					stx	other
Repeatability	Agreement of the assay based on intra-assay replicates	Repeatability=100 %×(# intra-assay replicates in agreement)/(total # intra-assay replicates)	Intra-assay replicate	Running the bioinformatics workflow twice on the same dataset in the same computational environment
Reproducibility	Agreement of the assay based on inter-assay replicates	Reproducibility=100 %×(# inter-assay replicates in agreement)/(total # inter-assay replicates)	Inter-assay replicate	Running the bioinformatics workflow twice on the same dataset in a separate computational environment
Accuracy	The likelihood that results of the assay are correct	Accuracy=100 %×(TP+TN)/(TN +FN+TP+FP)	True positive result	Resistance predicted for antibiotic group for which there is phenotypic resistance	Gene detected by workflow and PCR		Same serotype detected as by PCR-based methods	Plasmid replicon detected by workflow and PlasmidFinder tool	Detection of the same allele as the PubMLST.org platform
Precision	The likelihood that detected results of the assay are truly present	Precision=100 %×TP/(TP+FP)	False negative result	Sensitivity predicted for antibiotic group for which there is phenotypic resistance	Gene detected by PCR but not the workflow		Different serotype detected as by PCR-based methods	Plasmid replicon detected by PlasmidFinder tool but not the workflow	Detection of a different allele as the PubMLST.org platform
Sensitivity	The likelihood that a result will be correctly picked up by the assay when present	Sensitivity=100 %×TP/(TP+FN)	True negative result	Sensitivity predicted for antibiotic group for which there is phenotypic sensitivity	Gene not detected by workflow nor by PCR		No serotype detected in negative control sample	Plasmid replicon not detected by workflow nor by PlasmidFinder tool	No detection of an allele in negative control sample
Specificity	The likelihood that a result will not be falsely picked up by the assay when not present	Specificity=100 %×TN/(TN+FP)	False positive result	Resistance predicted for antibiotic group for which there is phenotypic sensitivity	Gene detected by workflow but not by PCR		Serotype detected in a negative control sample	Plasmid replicon detected by workflow but not by PlasmidFinder tool	Detection of an allele in negative control sample

TP, True positive; FP, false positive; FN, false negative; TN, true negative.