Multicenter Evaluation of the Acuitas AMR Gene Panel for Detection of an Extended Panel of Antimicrobial Resistance Genes among Bacterial Isolates

ABSTRACT

The Acuitas antimicrobial resistance (AMR) gene panel is a qualitative, multiplex, nucleic acid‐based in vitro diagnostic test for the detection and differentiation of 28 antimicrobial resistance markers associated with not susceptible results (NS; i.e., intermediate or resistant) to one or more antimicrobial agents among cultured isolates of select Enterobacterales, Pseudomonas aeruginosa, and Enterococcus faecalis. This study was conducted at four sites and included testing of 1,224 deidentified stocks created from 584 retrospectively collected isolates and 83 prospectively collected clinical isolates. The Acuitas results were compared with a combined reference standard including whole-genome sequencing, organism identification, and phenotypic antimicrobial susceptibility testing. The positive percent agreement (PPA) for FDA-cleared AMR targets ranged from 94.4% for MCR-1 to 100% for armA, CTX-M-2, DHA, IMP, OXA-9, SHV, vanA, and VEB. The negative percent agreement (NPA) for the majority of targets was ≥99%, except for AAC, AAD, CMY-41, P. aeruginosa gyrA mutant, Sul1, Sul2, and TEM targets (range, 96.5% to 98.5%). Three AMR markers did not meet FDA inclusion criteria (GES, SPM, and MCR-2). For each organism, 1 to 22 AMR targets met the minimum reportable PPA/NPA and correlated with ≥80% positive predictive value with associated NS results for at least one agent (i.e., the probability of an organism carrying an AMR marker testing NS to the associated agent). We demonstrate that the Acuitas AMR gene panel is an accurate method to detect a broad array of AMR markers among cultured isolates. The AMR markers were further associated with expected NS results for specific agent-organism combinations.

INTRODUCTION

Antimicrobial resistance (AMR) is one of the greatest global public health threats that has been recognized by many international bodies, including the World Health Organization and the U.S. Centers for Disease Control and Prevention (CDC) (1–3). Worldwide, over 700,000 people die from AMR infections annually, and this number is projected to grow to 10 million deaths per year by 2050 (4). A recent study reestimated the previously calculated data for annual deaths in the United States due to multidrug-resistant (MDR) organism infections to be as high as 162,000 (5). Furthermore, AMR infections lead to substantial health care costs, accounting for $4.6 billion annually in the United States (6). One initiative to address the threat of AMR has focused on rapid diagnostic tools to support patient management, infection prevention and control, and antimicrobial stewardship programs.

Detection of antimicrobial resistant organisms in clinical laboratories has traditionally relied on identification and phenotypic antimicrobial susceptibility testing (AST) results, which are typically available within 48 to 72 h of specimen receipt. Several rapid phenotypic (e.g., PBP2a assay) and molecular-based methods have been developed to more rapidly detect a few focused AMR genes or mechanisms (e.g., BioFire panels [Salt Lake City, UT], Luminex Verigene panels [Austin, TX], Cepheid GeneXpert cartridges [Sunnyvale, CA], GenMark ePlex panels [Carlsbad, CA], etc.). Currently available molecular panels often target a single or a narrow range of AMR genes, such as mecA/C in Staphylococcus aureus, vanA/B in enterococci, or β-lactamase genes (extended-spectrum β-lactamases [ESBL] or carbapenemases) among Gram-negative bacilli (7, 8). Research-use-only multiplex molecular assays have been described that focus on detecting a broader array of beta-lactamase genes, such as the ARM-D beta-lactamase multiplex PCR kit (Streck, Inc., Omaha, NE), the microarray Check-MDR CT103XL assay (Check-Points BV, Wageningen, Netherlands), and the loop-mediated isothermal amplification (LAMP) eazyplex SuperBug carbapenem-resistant Enterobacteriaceae (CRE) test (AmplexDiagnostics GmbH, Germany), among others (9). Furthermore, whole-genome sequencing (WGS) methods are becoming more commonplace, as they can detect an unlimited number of AMR genes based on the database being queried, and the data generated can be applied for multiple purposes (e.g., surveillance, assessing genetic relatedness, organism identification, virulence factors, etc.). However, WGS methods are still cumbersome and require significant molecular and bioinformatics expertise. Thus, automated or semiautomated, broad AMR panels with built-in analysis programs are an alternative considered by clinical laboratories. Until recently, there had yet to be a U.S. Food and Drug Administration (FDA)-cleared commercially available molecular panel that could detect a broad array of AMR mechanisms to several antimicrobial classes (e.g., beta-lactams, aminoglycosides, fluoroquinolones, sulfonamides) to help guide patient management. The Acuitas AMR gene panel helps to address these requirements as the first FDA-cleared diagnostic for the detection of 28 AMR markers, as well as to associate AMR markers with a phenotypic not susceptible (NS; i.e., intermediate or resistant) interpretation to specific antimicrobial agents depending on the gene and organism identified.

The Acuitas AMR gene panel, performed on the Qiagen EZ1 advanced XL and the Applied Biosystems QuantStudio 5 instruments, is a qualitative nucleic acid‐based in vitro diagnostic test for the detection and differentiation of 28 AMR markers associated with NS results to one or more antimicrobial agents. The test utilizes multiplex, real-time PCRs in a 96-well format and is conducted on isolated colonies of Pseudomonas aeruginosa, Enterococcus faecalis, or select members of the Enterobacterales grown on blood agar (BA) or MacConkey agar (MAC) plates. The 28 AMR genes are associated with NS results to 9 antimicrobial classes/subclasses—aminoglycosides, carbapenems, cephalosporins, fluoroquinolones, penicillins, sulfonamides, trimethoprim, vancomycin, and polymyxins—depending on the species (Table 1). Results from the Acuitas AMR gene panel are then combined with an identification method (e.g., molecular or matrix-assisted laser desorption ionization–time of flight mass spectrometry [MALDI-TOF MS]) to interpret the results.

TABLE 1.

FDA-cleared antimicrobial resistance gene markers detected by the Acuitas AMR gene panel for each bacterial species

Organism	Reported AMR gene marker(s)
Citrobacter freundii complexa	CTX-M-1, CTX-M-9, KPC, NDM, OXA-48
Citrobacter koseri	KPC, OXA-48
Enterobacter cloacae complexb	CTX-M-1, CTX-M-9, KPC, TEMd
Enterococcus faecalis	vanA
Escherichia coli	AAC, ANT, CMY, CTX-M-1, CTX-M-2, CTX-M-9, DFR, gyrA mutantc, KPC, MCR-1e, OXA-1, OXA-9, SHVd, Sul1, Sul2, TEMd
Klebsiella aerogenes	CTX-M-1, CTX-M-9, KPC, NDM, OXA-48
Klebsiella michiganensis	CTX-M-1, CTX-M-9, KPC, NDM, OXA-48
Klebsiella oxytoca	CTX-M-1, CTX-M-9, KPC, NDM, OXA-48
Klebsiella pneumoniae	AAC, AAD, APH, CMY, CTX-M-1, CTX-M-9, DFR, DHA, IMP, KPC, NDM, OXA-1, OXA-9, OXA-48, RMT, Sul1, Sul2, TEMd
Klebsiella quasipneumoniae	CTX-M-1, CTX-M-9, KPC, NDM, OXA-48
Klebsiella variicola	CTX-M-1, CTX-M-9, KPC, NDM, OXA-48
Morganella morganii	CTX-M-1, KPC, NDM, OXA-48
Proteus mirabilis	AAC, ANT, APH, armA, CMY, CTX-M-1, CTX-M-2, CTX-M-9, DFR, KPC, NDM, OXA-1, OXA-9, OXA-48, Sul2, TEMd, VEB, VIM
Providencia rettgeri	NDM
Providencia stuartii	NDM
Pseudomonas aeruginosa	AAC, ANT, CTX-M-1, CTX-M-2, gyrA mutantc, KPC, NDM, OXA-1, PER, SHVd, TEMd, VEB, VIM
Raoultella ornithinolytica	KPC, NDM, OXA-48
Raoultella planticola	KPC
Serratia marcescens	CTX-M-1, CTX-M-9, KPC, NDM, OXA-48

^aCitrobacter freundii complex = C. freundii, C. braakii, C. werkmanii, and C. youngae.

^bEnterobacter cloacae complex = E. asburiae, E. cloacae, E. hormaechei, E. kobei, and E. ludwigii.

^cPCR assays associated with fluoroquinolone resistance detect and differentiate wild-type and mutant variants of DNA gyrase A at amino acid position 87 for E. coli and position 83 for P. aeruginosa.

^dPCR assays for SHV and TEM detect several sequence variants for the two genes, respectively, at amino acid positions 156 and 104 associated with wild-type penicillin resistance and mutations associated with ESBL phenotypes.

^eThe panel includes an assay for the detection of the mobilized colistin genetic determinant MCR-1 in E. coli.

The purpose of this multicenter study was to establish the performance characteristics of the Acuitas AMR gene panel compared to a combined reference standard, including WGS, organism identification, and AST results, for the detection of AMR markers from pure culture growth of select Enterobacterales, P. aeruginosa, and E. faecalis.

MATERIALS AND METHODS

Study design and bacterial isolates.

The study was conducted at 4 geographically diverse investigational sites (Johns Hopkins University [JHU] School of Medicine, Baltimore, MD; Wadsworth Center [WC], New York State Department of Health [NYSDOH], Albany, NY; University Hospitals Cleveland Medical Center [UHCMC], Cleveland, OH; and International Health Management Associates [IHMA], Inc., Schaumberg, IL) and included two arms to the study, a retrospective arm using previously collected clinical stock isolates and a prospective arm using prospectively collected clinical isolates.

The retrospective clinical stock isolates were contrived from 584 unique bacterial strains, which covered 31 AMR targets initially evaluated on the Acuitas AMR Gene Panel (28 FDA-cleared AMR markers as defined in Tables 1 and 2 and 3 AMR makers that did not meet FDA criteria see Table 3). A total of 1,224 deidentified stocks were created from the 584 unique isolates. In certain circumstances where AMR genes were rare, replicates of the same isolate were distributed to the sites for testing (e.g., 5 unique strains with IMP were available with 71 replicates performed across the sites; Table 2). The Acuitas AMR marker reported result names do not follow traditional gene nomenclature, as the assay targets may represent gene families or multiple variants (see Table S1 in the supplemental material; e.g., the sul1 gene is reported as Sul1). Here, we apply the Acuitas AMR gene panel reportable name. The frozen stocks were previously molecularly characterized and were obtained from a combination of biobanks, including the CDC and FDA Antibiotic Resistance Isolate Bank, IHMA, University Hospitals Cleveland Medical Center, Cleveland, and OpGen’s biorepository. They were shipped to 3 of the testing sites (JHU, WC, and IHMA) in the form of frozen glycerol stocks. The number of isolates tested at each site ranged from 268 to 556.

TABLE 2.

Clinical performance (retrospective and prospective samples) for FDA-cleared Acuitas AMR gene panel targets compared to whole-genome sequencing

AMR gene target	PPA			NPA			No. or total unique strains	No. of total replicatesq	Positive for AMR targets by WGS		Negative for AMR target by WGS
	TP/(TP+FN) (n)r	%	95% CI	TN/(TN+FP) (n)r	%	95% CI			No. of unique strains	No. of replicatesq	No. of unique strains	No. of replicatesq
AAC	610/622a,b,c	98.1	96.66–98.89	536/545d,e	98.3	96.89–99.13	577	732	315	386	262	346
AAD	128/130	98.5	94.56–99.58	192/199f	96.5	92.92–98.29	185	189	98	54	87	135
ANT	203/205	99.0	96.51–99.73	628/633g	99.2	98.16–99.66	392	543	64	168	328	375
APH	39/40	97.5	87.12–99.56	443/444	99.8	98.74–99.96	263	280	31	12	232	268
armA	8/8	100.0	67.56–100.00	147/147	100.0	97.45–100.00	78	91	4	8	74	83
CMY	126/128h	98.4	94.48–99.57	688/691	99.6	98.73–99.85	422	489	55	84	367	405
CTX-M-1	264/273	96.7	93.85–98.26	929/938i	99.0	98.19–99.49	621	732	162	143	459	589
CTX-M-2	35/35	100.0	90.11–100.00	801/803	99.8	99.10–99.93	392	543	21	27	371	516
CTX-M-9	73/74	98.6	92.73–99.76	781/782	99.9	99.28–99.98	459	489	58	23	401	466
DFR	167/169	98.8	95.79–99.67	646/650	99.4	98.43–99.76	422	489	90	96	332	393
DHA	36/36	100.0	90.36–100.00	293/293	100.0	98.71–100.00	185	189	33	6	152	183
E. coli gyrA mutant	160/163	98.2	94.73–99.37	167/168	99.4	96.71–99.89	155	209	81	101	74	108
IMP	72/72	100.0	94.93–100.00	257/257	100.0	98.53–100.00	185	189	5	71	180	118
KPC	75/77	97.4	91.02–99.28	1130/1134	99.6	99.10–99.86	621	732	63	21	558	711
MCR-1	51/54j	94.4	84.89–98.09	281/281	100.0	98.65–100.00	159	209	14	48	145	161
NDM	56/57	98.2	90.71–99.69	801/805k	99.5	98.73–99.81	448	523	47	17	401	506
OXA-1	240/249	96.4	93.27–98.09	910/918	99.1	98.29–99.56	577	732	112	161	465	571
OXA-9	58/58	100.0	93.79–100.00	760/761	99.9	99.26–99.98	422	489	47	21	375	468
OXA-48	59/62	95.2	86.71–98.34	448/452	99.1	97.75–99.66	293	280	48	27	245	253
PER	81/82	98.8	93.41–99.78	265/266	99.6	97.90–99.93	155	243	9	81	146	162
P. aeruginosa gyrA mutant	265/279l	95.0	91.75–96.99	67/68	98.5	92.13–99.74	154	243	103	216	51	27
RMT	31/32	96.9	84.26–99.45	297/297	100.0	98.72–100.00	185	189	27	10	158	179
SHV	12/12	100	75.75–100.00	668/671	99.6	98.69–99.85	314	452	10	4	304	448
Sul1	420/424	99.1	97.60–99.63	232/240m	96.7	93.56–98.30	344	398	226	249	118	149
Sul2	489/501	97.6	95.86–98.62	307/318n	96.5	93.91–98.06	422	489	212	331	210	158
TEM	600/609o	98.5	97.22–99.22	559/572p	97.7	96.15–98.67	591	732	277	391	314	341
vanA	57/57	100.0	93.69–100.00	36/36	100.0	90.36–100.00	43	54	8	52	35	2
VEB	89/89	100.0	95.86–100.00	411/414	99.3	97.89–99.75	233	334	24	72	209	262
VIM	91/93	97.8	92.49–99.41	409/410	99.8	98.63–99.96	233	334	22	80	211	254

^aOne FN result attributed to the presence of an aac(3)-IIa gene variant that had no valid alignment with the primers/probe of the AAC assay harbored by a single K. pneumoniae unique isolate.

^bTwo FN results due to testing of two replicates of a single unique E. coli isolate.

^cTwo FN results due to testing of two replicates of a single unique P. aeruginosa isolate.

^dOne FP result attributed to the presence of a truncated aac(3)-II gene harbored by a single unique E. coli isolate.

^eOne FP result attributed to the presence of a truncated aac(3)-Ib gene in a single unique K. pneumoniae isolate.

^fThree FP results attributed to an aadA15 gene variant harbored by three K. pneumoniae isolates with high numbers (≥3) of mismatches in the reverse primer of the AAD assay, with two isolates tested as replicates of a single unique strain. Two additional FP results from K. pneumoniae isolates demonstrated alignment of the AAD assay primers/probe with high numbers of mismatches (≥3) in the reverse primer, but no attributable gene variant was detected in the AR database used for analysis.

^gOne FP result attributed to high PCR baseline drift and not true amplification of the ANT target assay in one unique P. aeruginosa isolate.

^hTwo FN results attributed to the presence of CMY gene variants with high numbers of mismatches (≥3) to the primers of the CMY assay in two E. coli isolates. One isolate harbored a blaCMY-2 gene variant, and one isolate harbored a blaCMY-42 gene variant.

ⁱSeven FP results attributed to the presence of a blaCTX-M-27 gene variant with perfect alignment to the primers/probe of the CTX-M-1 assay for 7 replicates of one unique E. coli isolate that was not originally identified by WGS analysis.

^jThree FN results due to testing of three replicates of a single E. coli isolate.

^kFour FP results due to testing of four replicates of a single K. pneumoniae isolate.

^lOne FN result attributed to a negative result for the P. aeruginosa ID assay for a single unique P. aeruginosa isolate. Amplification of the P. aeruginosa gyrA mutant assay was present for this isolate.

^mTwo FP results due to testing of two replicates of one unique E. coli isolate.

ⁿFour FP results due to testing for four replicates of a single unique K. pneumoniae isolate.

^oTwo FN results due to testing of two replicates of a single unique E. coli isolate.

^pFour FP results due to testing for four replicates of a single unique K. pneumoniae isolate.

^qReplicates are the total number of samples for unique isolates tested multiple times. For example, replicates would equal five if three unique isolates were, respectively, tested in singlate, duplicate, and triplicate.

^rTP, true positive, TN, true negative, FP, false positive, FN, false negative.

TABLE 3.

Performance characteristics of AMR markers not included in the FDA-cleared assay by organism group compared to whole-genome sequencing^a

AMR gene target	Organism group	PPA			NPA
		%	TP/(TP+FN) (n)	95% CI	%	TN/(TN+FP) (n)	95% CI
AAC	Morganella morganii	75.00	3/4	30.06–95.44	100.00	3/3	43.85–100.00
AAD	Enterobacter cloacae complex	90.90	10/11	62.26–98.38	100.00	3/3	43.85–100.00
AAD	Enterococcus faecalis	0.00	0/1	0.00–79.35	100.00	92/92	95.99–100.00
AAD	P. aeruginosa	61.30	68/111	51.97–69.80	99.60	236/237	97.65–99.93
AAD	Serratia marcescens	50.00	1/2	9.45–90.55	NaN
ANT	Klebsiella pneumoniae	91.70	11/12	64.61–98.51	99.40	315/317	97.73–99.83
APH	Escherichia coli	85.70	12/14	60.06–95.99	99.70	320/321	98.26–99.94
armA	E. cloacae complex	50.00	1/2	9.45–90.55	100.0	12/12	75.75–100.00
armA	K. pneumoniae	92.30	48/52	81.83–96.97	98.60	273/277	96.35–99.44
CMY	Citrobacter freundii complex	100.0	12/12	75.75-100.00	0.00	0/2	0.00–65.76
CMY	E. cloacae complex	NaN			92.90	13/14	68.53–98.73
CTX-M-2	K. pneumoniae	93.30	14/15	70.18–98.81	99.70	313/314	98.22–99.94
DFR	E. cloacae complex	NaN			92.90	13/14	68.53–98.73
DHA	M. morganii	100.00	6/6	60.97–100.00	0.00	0/1	0.00–79.35
GESb	K. pneumoniae	87.50	7/8	52.91–97.76	99.70	320/321	98.26–99.94
GESb	P. aeruginosa	90.50	38/42	77.93–96.23	100.00	306/306	98.76–100.00
GyrA mutant	K. pneumoniae	90.60	144/159	85.02–94.20	100.00	167/167	97.75–100.00
IMP	E. coli	71.40	5/7	35.89–91.78	100.00	328/328	98.84–100.00
IMP	P. aeruginosa	93.80	45/48	83.16–97.85	99.00	297/300	97.10–99.66
MCR-1	K. pneumoniae	20.00	1/5	3.62–62.45	100.00	324/324	98.83–100.00
MCR-2b	E. coli	100.00	50/50	92.87–100.00	100.00	285/285	98.67–100.00
MCR-2b	K. pneumoniae	NaN			100.00	329/329	98.85–100.00
NDM	E. cloacae complex	66.70	2/3	20.77–93.85	100.00	11/11	74.12–100.00
NDM	E. coli	90.00	18/20	69.90–97.21	99.40	313/315	97.71–99.83
OXA-48	E. cloacae complex	0.00	0/1	0.00–79.35	92.30	12/13	66.69–98.63
OXA-48	E. coli	92.30	12/13	66.69–98.63	100.0	322/322	98.82–100.00
PER	P. mirabilis	71.40	5/7	35.89–91.78	100.00	148/148	97.47–100.00
RMT	E. cloacae complex	66.70	2/3	20.77–93.85	100.00	11/11	74.12–100.00
RMT	E. coli	88.90	8/9	56.50–98.01	99.70	325/326	98.28–99.95
SHV	K. pneumoniae	98.50	319/324	96.44–99.34	40.00	2/5	11.76–76.93
SPMb	P. aeruginosa	92.90	52/56	83.02–97.19	99.70	291/292	98.09–99.94
Sul1	M. morganii	75.00	3/4	30.06–95.44	100.00	3/3	43.85–100.00
Sul1	P. mirabilis	99.20	123/124	95.57–99.86	87.10	27/31	71.15–94.87
Sul2	C. freundii complex	100.00	5/5	56.55–100.00	88.90	8/9	56.50–98.01
Sul2	E. cloacae complex	100.00	5/5	56.55–100.00	88.90	8/9	56.50–98.01
Sul2	M. morganii	50.00	1/2	9.45–90.55	100.00	5/5	56.55–100.00
VEB	K. pneumoniae	87.50	7/8	52.91–97.76	99.40	319/321	97.76–99.83

^aNaN, not-a-number/divide-by-zero. TP, true positive, TN, true negative, FP, false positive, FN, false negative.

^bGES, SPM, MCR-2 did not meet acceptance criteria and were not included in the FDA-cleared version of the assay.

The prospective clinical isolates from various sources (e.g., blood, urine, wound, and respiratory) were enrolled at 2 sites (JHU and UHCMC) and included 83 fresh bacterial isolates that were deidentified and saved to glycerol stocks within 7 days of isolation. The isolates were enrolled if they were one of the target species with documented NS results (i.e., intermediate or resistant), excluding intrinsic resistance, determined by AST to one of the nine antimicrobial classes/subclasses for which the Acuitas panel results are associated with NS results based on AMR gene detection, where appropriate. A maximum of eight consecutively collected isolates per species were enrolled at each site. A glycerol stock was sent to a reference laboratory for reference method testing (see below).

Acuitas AMR gene panel testing.

The Acuitas AMR gene panel (catalog [cat.] no. KT03U01) includes PCR master mix and two 96-well PCR plates (4 isolates/plate) with proprietary dried primers and probes for testing up to 8 isolates. Table 1 summarizes the FDA-cleared AMR genes or family of genes detected by the panel based on the species. Inclusivity of targets is defined in Table S1. A research-use-only (RUO) version of the assay that is technically identical to the FDA-cleared version of the assay was used for the study.

At the completion of successful training and competency assessment, sites began testing. Each retrospective stock isolate was subcultured twice from glycerol stock to BA, while prospective clinical isolates were subcultured for purity to BA and incubated for 18 to 24 h at 35 to 37°C in ambient air. From pure culture growth, a direct colony suspension equivalent to a 0.5 McFarland standard was prepared from 3 to 5 well-isolated colonies in sterile saline (10). These suspensions underwent DNA extraction on the EZ1 advanced XL instrument (Qiagen) followed by real‐time detection of bacterial genes on the QuantStudio 5 real‐time PCR system (Thermo Fisher) using multiplex PCR as described in the Acuitas AMR gene panel package insert (Fig. 1). Briefly, 200 μL ATL buffer and 200 μL of the 0.5 McFarland standard were added to an assay tube (provided in kit) and vortexed for 30 to 60 s. Up to 14 isolates and appropriate reagents were then loaded on the EZ1 advanced XL instrument for DNA extraction (1-h, 5-min protocol). Using a 200-μL pipette, 140 μL of DNA extract and 140 μL of mastermix were added to a reagent reservoir trough and mixed with the pipette. Using an 8-channel pipette, 10 μL of the mixture was added to wells of 3 columns of the Acuitas AMR gene panel per isolate for up to 4 isolates per plate. The plate was then sealed, centrifuged for 30 s to remove air bubbles, and loaded on the QuantStudio 5 real‐time PCR system using the preprogrammed 45-min PCR protocol. When the run was complete, results were exported to an Excel file and opened in the Acuitas AMR gene analysis software, where the software generated a report with the species and resistance genes reported as “detected” or “not detected.” If the target was detected, a crossing-point (cycle threshold [C_T]) value was provided. The entire process took 2.5 h with 30 min of hands-on time per test run of 4 samples.

FIG 1.

Acuitas AMR gene panel test workflow.

During the extraction step, a lyophilized Bacillus control was reconstituted and introduced into the extraction eluent for each sample and served as the internal control. For each day of testing, a positive external control (PC) and the negative external control (NC), Staphylococcus aureus ATCC 33592, were tested. Five positive controls were cycled through on successive days of testing and included Klebsiella pneumoniae AR Bank #0079, Escherichia coli AR Bank #0346, P. aeruginosa AR Bank #0353, Proteus mirabilis AR Bank #0379, and E. faecalis ATCC BAA-2573.

Reference method.

Performance of the Acuitas AMR gene panel was established using a composite reference method. Gram-negative organism identification (ID) was determined using the bioMérieux Vitek2 compact system or Bruker matrix-assisted laser desorption/ionization–time of flight mass spectrometry (MALDI-TOF MS; using validated non-FDA-cleared MBT Compass Reference Library software version 8468), and antimicrobial susceptibility testing (AST) was performed on the Beckman Coulter MicroScan system (negative MIC 45 panel). For Gram-positive organisms, identification (ID) was performed with Bruker MALDI-TOF MS and AST by reference broth microdilution. AST results were interpreted using Clinical and Laboratory Standards Institute breakpoints (11). The combined ID and AST results were then used as the reference for species ID and antimicrobial susceptibility phenotype.

WGS was utilized as the comparator method for the gene variants detected by the Acuitas AMR gene panel. Validation of WGS as the reference method, including reproducibility studies, was performed and presented to the FDA as part of the 510(k) process (see supplemental material). WGS was performed on every bacterial isolate that was evaluated with the Acuitas AMR gene panel by the Institute for Genome Sciences, University of Maryland School of Medicine. WGS was performed on the Illumina HiSeq 4000 instrument using a 2 × 150-bp paired-end protocol per the manufacturer’s recommendation. Paired-end 150-bp reads were generated with an average of 100× coverage (a minimum of 75× coverage) per isolate. At the completion of WGS, OpGen was provided with the data in the format of FASTQ files for each sequenced isolate containing nucleotide reads and quality information. Ridom SeqSphere+ was used to assemble reads into whole-genome sequences (12). AMR genes from the assembled whole-genome sequences of the isolates were identified through OpGen’s internal bioinformatics pipeline. A database of 2,400 resistance genes, including all genes on the Acuitas AMR gene panel, were compiled from databases such as the Antibiotic Resistance Genes Database (ARDB) (13), the Comprehensive Antibiotic Resistance Database (CARD) (14), Resfinder (15), and the Lahey Clinic (https://externalwebapps.lahey.org/studies/). The NCBI stand-alone software NCBI-blast (https://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/2.7.1/) was accessed in 2019 to query OpGen’s curated resistance gene databases with assembled WGS assemblies.

Medium equivalency study.

To establish equivalency between BA and MAC agar for growth of isolates to be tested by the Acuitas AMR gene panel, a medium equivalency study was performed at OpGen, Inc. Thirteen isolates each of E. coli, K. pneumoniae, P. aeruginosa, and P. mirabilis were cultured on both BA and MAC agar prior to testing with the Acuitas AMR gene panel.

Statistical analysis.

The primary endpoint for both the retrospective stock isolate and prospective isolate studies was ≥95% positive percent agreement (PPA) and negative percent agreement (NPA) of the samples tested for detection of the AMR markers. However, for MCR-1, a PPA of 94.4% was considered acceptable by the FDA for clearance of the marker, and this value is reflected as the minimal acceptable PPA/NPA value listed here. Results of percent agreements were tabulated by individual AMR gene target and by species ID AMR targets. The PPA and NPA were reported with 95% confidence intervals computed using the exact method in the R package binom (https://www.r-project.org/about.html).

PCR results were tabulated against the phenotypic AST MicroScan/broth microdilution results. Resistant (R) and intermediate (I) phenotypes were combined into a not susceptible (NS) category. The number and percentage of positive PCR results in the NS and susceptible (S) categories were tabulated. These tabulations were consistent with the literature except for PER (e.g., an extended-spectrum β-lactamase gene that was associated with carbapenem resistance in P. aeruginosa in this study; see Table 3) (16). Association between a detected AMR marker and an NS result for the antimicrobial agent-organism combination was only considered to be established in cases where (i) at least ≥94.4% PPA and NPA was achieved for detection of an AMR marker and (ii) ≥80% positive predictive value (PPV, true positive/[true positive + false positive] [17]) correlation to phenotypic NS was observed. Of note, PPV depends on the prevalence of each AMR marker; consequently, users may have different PPVs depending on the local prevalence.

Data availability.

The sequence reads of the WGS validation set and prospective arm can be accessed at the NCBI sequence read archive under BioProject PRJNA786742. The sequence reads from the retrospective arm will be made available upon request by contacting customersupport@opgen.com. A nondisclosure and data transfer agreement will be required.

RESULTS

AMR gene target detection.

Results from the retrospective and prospective cohorts were combined in the analysis due to the small number of prospective isolates included (i.e., 83 prospective isolates versus 1,224 retrospective isolates). Table 2 summarizes the number of unique strains and replicates tested per target. The total number of unique strains tested per AMR gene target ranged from 4 (armA) to 277 (TEM) with up to 732 replicates.

The overall performance of FDA-cleared AMR gene targets on the Acuitas AMR gene panel compared with WGS are summarized in Table 2. PPA ranged from 94.4% for MCR-1 to 100% for armA, CTX-M-2, DHA, IMP, OXA-9, SHV, vanA, and VEB. The NPA for the majority of targets was ≥99%, except for AAC, AAD, P. aeruginosa gyrA mutant, Sul1, Sul2, and TEM targets, where they ranged from 96.5% to 98.5%. As GES (PPA, 90%; NPA, 99.8%) and SPM (PPA, 92.9%; NPA, 100%) markers fell below the FDA requirements, they were further excluded as reportable targets (Table 3). MCR-2 was also excluded by the FDA despite 100% PPA and NPA due to the lack of clinical isolates included in the evaluation.

The performance of the detection of AMR gene targets among isolates was further evaluated by bacterial species. The Acuitas AMR gene panel electronic user guide (EUG) available on the OpGen website summarizes the performance of the Acuitas AMR gene panel based on AMR markers and various organism groups/species. For each Gram-negative organism, a minimum of 94.1% PPA and NPA was established to be a reportable AMR target. PPAs of 94.1% for KPC and 94.4% for MCR-1 with Escherichia coli and 94.9% for AAC with P. aeruginosa fell below the initial 95% threshold but were ultimately provided FDA clearance. For ACC and Proteus mirabilis, the PPA was 100%, but the NPA was 92.7% and accepted as a reportable combination. For E. faecalis the PPA and NPA for vanA were 100%. The number of reportable AMR targets varied by species from 1 reportable target for E. faecalis (vanA), Raoultella ornithinolytica (KPC), and Providencia rettgeri/Providencia stuartii (NDM) to 22 AMR targets for Klebsiella pneumoniae and P. mirabilis.

Table 3 summarizes the performance characteristics of AMR markers and organism combinations that were not included in the FDA-cleared assay compared to WGS. For certain combinations, there were very few isolates to comment on the performance of. However, for AMR markers with at least 10 isolates, the majority of the markers demonstrated >90% PPA with the exception of AAD with P. aeruginosa (PPA, 61.4%). The NPA was greater than 94.4% for the great majority of these targets with the exception AAC for P. mirabilis, CMY, DFR, OXA-48, and Sul2 for Enterobacter cloacae complex, SHV for K. pneumoniae, and Sul2 for Citrobacter freundii complex.

Discrepant result analysis.

Discrepant result analysis was performed through the review of WGS results and Acuitas AMR gene panel PCR data to evaluate potential causes for the observed discrepancies. Through these efforts, it was identified that for 9 isolates WGS results identified the incorrect species (e.g., P. mirabilis instead of K. pneumoniae, P. aeruginosa instead of E. coli, etc.), for 4 isolates the WGS comparator method produced inconsistent results (e.g., 1 isolate was sequenced twice, but Sul2 was detected in only 1 of the 2 sequencing runs), and for 49 isolates there was evidence of sample error or mix‐up. In all of these cases, discrepant analysis was not pursued, and therefore these original results are included in the data analysis.

There were several noteworthy findings that came from discrepant result analysis, which are summarized in the footnotes of Table 2. Discrepant results were observed due to variants that did not have valid alignment to the primers/probes for that target [e.g., aac(3)-IIa with AAC], due to the presence of truncated genes [e.g., aac(3)-II or aac(3)-Ib with AAC], or due to a high number of primer/probe mismatches (e.g., aadA15 with AAD, bla_CMY-2 and bla_CMY-42 variants with CMY). One false-positive (FP) result was attributed to high PCR baseline drift not representative of true amplification of the ANT target assay in one unique P. aeruginosa isolate. Furthermore, seven FP results were attributed to the presence of a bla_CTX-M-27 gene variant with perfect alignment to the primers/probe of the CTX-M-1 assay, all resulting from replicate testing of one unique E. coli isolate.

Quality control.

Assay control results for 57 PCs, 57 NCs, and 1,300 test samples were valid. A total of 55 of 57 (96.5%) PCs and 55 of 57 (96.5%) NCs gave the expected Acuitas AMR gene panel results over all days of testing at each testing site. All control failures resolved upon repeat.

Correlation of AMR targets with phenotypic AST results.

Table 4 summarizes the correlation of the FDA-cleared AMR targets with phenotypic AST results for antimicrobial agents that were associated with NS among isolates with at least 80% PPV (i.e., the probability of an organism carrying a resistance marker being NS to expected agents). Any agent that was tested but not associated with NS for an organism did not reach the ≥95% PPA/NPA and ≥80% PPV threshold (both criteria were required). For example, organisms that harbored AAC were tested against amikacin, gentamicin, and tobramycin, but amikacin is not a predicted agent, as it did not meet the minimal threshold associated with NS.

TABLE 4.

Correlation of AMR marker detection with phenotypic AST results associated with not susceptible results among isolates with at least 94.4% positive percent agreement, 94.4% negative percent agreement, and 80% positive predictive value

Organism	Antimicrobial class	AMR gene target(s)	Antimicrobial(s) with associated NS resultsd
Citrobacter freundii complexa	Beta-lactams	CTX-M-1	TZP, FEP, CAZ, and CRO
		CTX-M-9	FEP and CRO
		KPC, NDM, and OXA-48	IPM, ETP, and MEM
Citrobacter koseri	Beta-lactams	KPC, OXA-48	IPM, ETP, and MEM
Enterobacter cloacae complexb	Beta-lactams	CTX-M-1	TZP, FEP, CAZ, and CRO
		CTX-M-9	FEP and CRO
		KPC	IPM, ETP, and MEM
		TEM	TZP
Enterococcus faecalis	Glycopeptide	vanA	VAN
Escherichia coli	Aminoglycosides	AAC, ANT	GEN and TOB
	Beta-lactams	CMY	AMC, SAM, CXM, CAZ, and CRO
		CTX-M-1	SAM, CXM, FEP, CAZ, and CRO
		CTX-M-2, OXA-9	AMC, SAM, CXM, FEP, CAZ, and CRO
		CTX-M-9	SAM, CXM, FEP, and CRO
		KPC	IPM, ETP, and MEM
		OXA-1	AMP, AMC, and SAM
		SHV	AMP, AMC, SAM, and TZP
		TEM	AMP and SAM
	Polymyxins	MCR-1	CST
	Folate pathway inhibitor	DFR, Sul1, and Sul2	SXT
	Fluoroquinolones	GyrA mutant	LVX and CIP
Klebsiella aerogenes	Beta-lactams	CTX-M-1	CXM, FEP, CAZ, and CRO
		CTX-M-9	CXM, FEP, and CRO
		KPC, NDM, OXA-48	IPM, ETP, and MEM
Klebsiella michiganensis	Beta-lactams	CTX-M-1	AMC, SAM, CXM, FEP, CAZ, and CRO
Klebsiella quasipneumoniae		CTX-M-9	SAM, CXM, FEP, and CRO
Klebsiella variicola		KPC, NDM, OXA-48	IPM, ETP, and MEM
Klebsiella oxytoca	Beta-lactams	CTX-M-1	AMC, SAM, CXM, FEP, and CRO
		CTX-M-9	SAM, CXM, FEP, and CRO
		KPC, NDM, and OXA-48	IPM, ETP, and MEM
Klebsiella pneumoniae	Aminoglycosides	AAC, AAD, APH	GEN and TOB
		RMT	AMK, GEN, and TOB
	Beta-lactams	CMY	AMC, SAM, CXM, FEP, and CRO
		CTX-M-1, CTX-M-9, and OXA-9	AMC, SAM, TZP, CXM, FEP, CAZ, and CRO
		DHA	AMC, SAM, TZP, CXM, CAZ, and CRO
		IMP, KPC, NDM, and OXA-48	IPM, ETP, and MEM
		OXA-1 and TEM	AMC, SAM, and TZP
	Folate pathway inhibitor	DFR, Sul1, and Sul2	SXT
Morganella morganii	Beta-lactams	CTX-M-1	SAM, FEP, CAZ, and CRO
		KPC, NDM, OXA-48	ETP and MEM
Proteus mirabilis	Aminoglycosides	AAC, ANT, and APH	GEN and TOB
		armA	AMK, GEN, and TOB
	Beta-lactams	CMY	AMC and SAM
		CTX-M-1 and OXA-9	CXM, FEP, and CRO
		CTX-M-2 and CTX-M-9	SAM, CXM, FEP, and CRO
		KPC, NDM, OXA-48, and VIM	ETP and MEM
		OXA-1	AMP and SAM
		TEM	AMP
		VEB	CXM, FEP, CAZ, and CRO
	Folate pathway inhibitor	DFR and Sul2	SXT
Providencia rettgeri and Providencia stuartii	Beta-lactams	NDM	IMP, ETP, and MEM
Pseudomonas aeruginosa	Aminoglycosides	AAC and ANT	AMK, GEN, and TOB
	Beta-lactams	CTX-M-1	TZP, FEP, and CAZ
		CTX-M-2	TZP and FEP
		KPC, NDM, and PERc, VIM	IPM and MEM
		OXA-1, SHV, and TEM	TZP
		VEB	FEP and CAZ
	Fluoroquinolones	GyrA mutant	LVX and CIP
Raoultella ornithinolytica	Beta-lactams	KPC, NDM, and OXA-48	IPM, ETP, and MEM
Raoultella planticola	Beta-lactams	KPC	IPM, ETP, and MEM
Serratia marcescens	Beta-lactams	CTX-M-1	FEP, CAZ, and CRO
		CTX-M-9	FEP and CRO
		KPC, NDM, and OXA-48	IPM, ETP, and MEM

^aCitrobacter freundii complex = C. freundii, C. braakii, C. werkmanii, and C. youngae.

^bEnterobacter cloacae complex = E. asburiae, E. cloacae, E. hormaechei, E. kobei, and E. ludwigii.

^cPER is an extended-spectrum beta-lactamase. Association with carbapenem resistance in P. aeruginosa is likely due to the combination of mechanisms (e.g., PER and cell wall permeability defects).

^dAMK, amikacin; GEN, gentamicin; TOB, tobramycin; CFZ, cefazolin; CXM, cefuroxime; FEP, cefepime; CAZ, ceftazidime; CRO, ceftriaxone; IPM, imipenem; ETP, ertapenem; MEM, meropenem; AMC, amoxicillin-clavulanic acid; SAM, ampicillin-sulbactam; TZP, piperacillin-tazobactam; IPM, imipenem; ETP, ertapenem; SXT, trimethoprim-sulfamethoxazole; CIP, ciprofloxacin; LVX, levofloxacin; VAN, vancomycin; CST, colistin; AMP, ampicillin.

Medium equivalency study.

The Acuitas AMR gene panel was evaluated for PPA and NPA with the reference method for each combination of organism, AMR gene, and agar medium (BA and MAC) as summarized in Table S2. Isolate agreement across all AMR genes and both types of agar medium ranged from 62% to 92% across the four organisms versus the reference method. Ten isolates were repeated due to user error (i.e., sample mix-up). With incorporation of the repeat results, there was 100% PPA for E. coli, K. pneumoniae, and P. aeruginosa on BA and MAC. For P. mirabilis, the PPA was 92% for BA and MAC. The NPA was 100% for most targets and organisms. The study did not uncover evidence of a medium effect between BA and MAC, both of which are suitable for the Acuitas AMR gene panel.

DISCUSSION

This study presents the results of a multicenter study evaluating the Acuitas AMR gene panel for detection and differentiation of 28 AMR markers that can be associated with NS to one or more antimicrobial agents among select Enterobacterales, P. aeruginosa, and E. faecalis. The Acuitas AMR panel can be performed from cultured growth from both BA and MAC with results available in 2.5 h and hands-on time of 30 min per test run of 4 samples (Fig. 1). This is the first FDA-cleared commercially available diagnostic tool that is capable of detecting a broad range of AMR markers associated with several antimicrobial classes from cultured isolates with ≥94.4% PPA and ≥96.5% NPA compared with a composite reference standard, including WGS.

Recently, the research-use-only version of the Acuitas AMR gene panel was compared to the Streck ARM-D beta-lactamase kit (9 beta-lactamase targets), the Check-MDR CT103XL microarray assay (27 beta-lactamase targets), and WGS (CARD database) as the reference standard for AMR gene detection among Enterobacterales isolates. The study demonstrated that all methods had high concordance compared to sequencing results, with the Acuitas AMR panel achieving the highest concordance due to the larger number and inclusion of diverse AMR targets on the panel. Similar to our study, WGS detected a higher number of aminoglycoside subtypes compared to the Acuitas AMR panel due to the primer/probe design of the targets (9).

Moreover, we demonstrate the ability of the Acuitas AMR gene panel to detect and associate AMR genes with NS results with at least 80% PPV for 9 antimicrobial classes/subclasses among claimed organisms. Associating the AMR markers with NS results using the Acuitas panel is a main differentiating factor compared to other FDA-cleared molecular AMR tests that simply detect the presence/absence of a gene. At a minimum, a single antimicrobial agent is associated with NS results from the Acuitas panel, such as vancomycin with the detection of vanA among E. faecalis or the carbapenems with the detection of carbapenemase genes among Providencia and Raoultella species. A larger set of antimicrobial agents can be associated with NS results among common Enterobacterales (e.g., E. coli, K. pneumoniae, and P. mirabilis) and P. aeruginosa. The associated agents include many of the most commonly prescribed antimicrobial agents, including beta-lactams, fluoroquinolones, trimethoprim-sulfamethoxazole, and aminoglycosides (18). These results have the potential to guide patient care at least a day earlier than traditional phenotypic AST methods (e.g., automated antimicrobial susceptibility testing systems, disk diffusion, or gradient diffusion methods). However, it should be noted that the Acuitas AMR gene panel is an adjunct method and not a replacement for traditional phenotypic AST results. Alternatively, rapid phenotypic AST methods, such as the Accelerate PhenoTest BC kit, have been successfully applied clinically and can be performed from positive blood culture broth to provide an identification and comprehensive AST profile within ∼7 h (19, 20).

Results from the Acuitas AMR gene panel can also be applied to support antimicrobial stewardship and infection control programs. It joins a growing list of molecular tools that were granted FDA clearance for detecting AMR genes either from isolates after cultured growth (e.g., from positive blood culture broth or solid medium) or directly from specimens (e.g., respiratory samples, whole blood, cerebrospinal fluid) for diagnostic and/or surveillance purposes (7, 8). Until recently, the largest AMR marker panel cleared by the FDA was the Unyvero lower respiratory tract (LRT; Curetis, Holzgerlingen, Germany) panel with the ability to detect 10 AMR markers directly from respiratory specimens (21). The Acuitas AMR gene panel more than doubles this number of targets by detecting 28 AMR markers from cultured isolates. Recent Infectious Diseases Society of America treatment guidance for multidrug-resistant Gram-negative bacterial infections highlights how detection of AMR genes or a specific mechanism of resistance can help guide cascade reporting practices for novel antimicrobial agents and tailor therapy for these difficult to treat infections (22). Furthermore, it can help with infection prevention and control initiatives such as patient isolation procedures when multiple isolates with the same AMR profile are detected as an early indication of transmission within a facility or for surveillance of serious or emerging AMR threats (2).

Although the Acuitas AMR gene panel is the first FDA-cleared device to declare the ability to associate NS results based on AMR marker detection, many commercially available tools are applied clinically in this manner (23, 24). Applying rapid diagnostic tools, the presence and/or absence of an AMR marker is associated with an organism to predict phenotypic resistance and/or susceptibility depending on the accuracy of predictions. For Gram-positive organisms, the accuracies of prediction are often higher (98% to 100%) for current AMR targets, as a single mechanism of resistance accounts for most clinically significant resistance (e.g., mecA/mecC for MRSA, vanA/B for VRE), whereas it is more complex among Gram-negative organisms, as mechanisms are heterogeneous, resulting in lower accuracies of predictions, especially for predicting susceptibility (25–28). Overall, many studies applying rapid molecular testing have demonstrated positive impacts on clinical care, including a significant decrease in time to effective therapy, shortened length of stay, and a decrease in mortality risk, especially when combined with antimicrobial stewardship programs (29).

Limitations of this study include the lack of discordant analysis studies. Overall, there were 12 isolates with at least 2 AMR discordant targets that were considered for discordant analysis, but this would have had little impact on the overall performance. As such, discordant analysis was not pursued. Use of the Acuitas AMR gene panel is currently FDA-cleared for the isolated colonies of the claimed organisms. Studies evaluating its utility in a broader array of pathogens and for direct-from-specimen testing (e.g., urine and positive blood culture broth) may be explored in the future. Furthermore, evaluations of targeted testing algorithms for specific populations (e.g., oncology patients) or sources (e.g., isolates recovered from sterile sources), and laboratory workflows and clinical utility of the Acuitas assay still need to be evaluated.

In conclusion, we demonstrate that the Acuitas AMR gene panel is an accurate method for the detection of a broad array of AMR markers among select Enterobacterales, P. aeruginosa, and E. faecalis from cultured isolates, with the ability to associate NS results for 9 antimicrobial classes/subclasses more rapidly by at least a day than traditional phenotypic methods to guide patient management and support antimicrobial stewardship and infection control programs.