Skip to main content
Journal of Clinical Microbiology logoLink to Journal of Clinical Microbiology
. 2018 Mar 26;56(4):e01672-17. doi: 10.1128/JCM.01672-17

Evaluation of the Accelerate Pheno System: Results from Two Academic Medical Centers

Joseph D Lutgring a,b,, Cassiana Bittencourt c,d, Erin McElvania TeKippe d,e, Dominick Cavuoti d, Rita Hollaway d, Eileen M Burd b
Editor: Paul Bourbeau
PMCID: PMC5869849  PMID: 29386262

ABSTRACT

Rapid diagnostic tests are needed to improve patient care and to combat the problem of antimicrobial resistance. The Accelerate Pheno system (Accelerate Diagnostics, Tucson, AZ) is a new diagnostic device that can provide rapid bacterial identification and antimicrobial susceptibility test (AST) results directly from a positive blood culture. The device was compared to the standard of care at two academic medical centers. There were 298 blood cultures included in the study, and the Accelerate Pheno system provided a definitive identification result in 218 instances (73.2%). The Accelerate Pheno system provided a definitive and correct result for 173 runs (58.1%). The Accelerate Pheno system demonstrated an overall sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of 94.7%, 98.9%, 83.7%, and 99.7%, respectively. An AST result was available for analysis in 146 instances. The overall category agreement was 94.1% with 12 very major errors, 5 major errors, and 55 minor errors. After a discrepancy analysis, there were 5 very major errors and 4 major errors. The Accelerate Pheno system provided an identification result in 1.4 h and an AST result in 6.6 h; the identification and AST results were 41.5 h and 48.4 h faster than those with the standard of care, respectively. This study demonstrated that the Accelerate Pheno system is able to provide fast and accurate organism identification and AST data. A limitation is the frequency with which cultures required the use of alternative identification and AST methods.

KEYWORDS: antimicrobial resistance, antimicrobial susceptibility testing, bacteremia, bloodstream infections, diagnostics, rapid diagnostic testing

INTRODUCTION

Sepsis and septic shock are major causes of morbidity and mortality in the United States and globally (14). Prompt diagnosis and initiation of appropriate antimicrobial therapy can improve mortality (57). Rapid diagnostic tests that can accurately identify the pathogen causing an infection and the antimicrobials that are effective against that infection would increase the likelihood that patients are treated appropriately (8). In addition to ensuring patients receive appropriate antibiotics as soon as possible, rapid diagnostic tests can also be used to help clinicians discontinue unnecessary antibiotics or de-escalate broad-spectrum antimicrobial therapy in favor of narrower-spectrum options (9). There is an urgent need for these rapid diagnostic tests given the crisis of antimicrobial resistance (1012).

Many such rapid diagnostic tests currently exist, including matrix-assisted laser desorption ionization–time of flight mass spectrometry (MALDI-TOF MS), Xpert MRSA/SA Blood Culture (Cepheid, Sunnyvale, CA), FilmArray Blood Culture Identification (BCID) Panel (BioFire Diagnostics, LLC, Salt Lake City, UT), Verigene (Luminex Corporation, Northbrook, IL), QuickFISH (AdvanDx, Inc., Woburn, MA), Prove-it Sepsis (Mobidiag, Helsinki, Finland), and the Unyvero BCU application (Curetis, Holzgerlingen, Germany) (13). The U.S. Food and Drug Administration (FDA) has cleared some of these diagnostic tests, and some are available only in Europe. Several of the tests have been shown to decrease mortality, to improve time to effective antimicrobial therapy, to shorten hospital stays, and to decrease health care costs (1417). Several of the tests provide information on the presence or absence of resistance genes, but none of them provides rapid phenotypic antimicrobial susceptibility testing (AST) results. For certain microbes, particularly Gram-negative bacteria, the presence or absence of a resistance gene is not enough information to predict the phenotype (18). Antimicrobial resistance in Gram-negative bacteria is complex and often multifactorial (beta-lactamases, porin mutations, and efflux pumps) (19). For example, in some areas of the United States, the majority of carbapenem resistance in Enterobacteriaceae is not due to Klebsiella pneumoniae carbapenemase (KPC) but is due to non-carbapenemase-producing mechanisms (e.g., the presence of AmpC plus a porin mutation) (20).

The Accelerate Pheno system (Accelerate Diagnostics, Tucson, AZ) is a new diagnostic device that can provide rapid bacterial identification and AST results. The device uses a broad panel of fluorescence in situ hybridization assays for microorganism identification and morphokinetic cellular analysis using time-lapse imaging for AST (2125). The Accelerate PhenoTest BC kit can identify 16 organisms—6 Gram-positive and 8 Gram-negative bacteria, as well as 2 Candida species—directly from positive blood cultures. It is FDA cleared to provide AST data for 6 Gram-positive drugs, 2 Gram-positive resistance phenotype markers, and 12 Gram-negative drugs (Table 1). This study provides a comparison of the performance of the Accelerate PhenoTest BC kit to that of the standards of care at two academic medical centers. Our hypothesis was that the Accelerate Pheno system would provide an identification and AST result from positive blood cultures with accuracy equivalent to that of the result provided by the standard of care, and more quickly.

TABLE 1.

FDA-cleared Accelerate PhenoTest BC kit

Organisma Antimicrobial agentb
AMP CFT DAP ERY LZD VAN MRS MLSb AMK SAM ATM CAZ FEP CRO CIP ETP GEN MEM TZP TOB
AST for Gram-positive bacteria
    CNS X X X X
    EFS X X X X
    EFM X X X X
    SAU X X X X X X
    SLU X X X
    STR
AST for Gram-negative bacteria
    ABA X X
    CIT X X X X X X X X X X X
    ENT X X X X X X X X X X X
    ECO X X X X X X X X X X X X
    KLE X X X X X X X X X X X X
    PRO X X X X X X X X X X X X
    PAE X X X X X X X X
    SMA X X X X X X X X X X X
No antifungal susceptibility testing
    CAL
    CGL
a

CNS, coagulase-negative Staphylococcus spp.; EFS, Enterococcus faecalis; EFM, Enterococcus faecium; SAU, Staphylococcus aureus; SLU, Staphylococcus lugdunensis; STR, Streptococcus spp.; ABA, Acinetobacter baumannii; CIT, Citrobacter spp.; ENT, Enterobacter spp.; ECO, Escherichia coli; KLE, Klebsiella spp.; PRO, Proteus spp.; PAE, Pseudomonas aeruginosa; SMA, Serratia marcescens; CAL, Candida albicans; CGL, Candida glabrata.

b

AMP, ampicillin; CFT, ceftaroline; DAP, daptomycin; ERY, erythromycin; LZD, linezolid; VAN, vancomycin; MRS, methicillin-resistant Staphylococcus; MLSb, macrolide-lincosamide-streptogrammin B resistance; AMK, amikacin; SAM, ampicillin-sulbactam; ATM, aztreonam; CAZ, ceftazidime; FEP, cefepime; CRO, ceftriaxone; CIP, ciprofloxacin; ETP; ertapenem; GEN, gentamicin; MEM, meropenem; TZP, piperacillin-tazobactam; TOB, tobramycin. “X” represents that AST is performed for the given antimicrobial agent when the given organism is identified.

MATERIALS AND METHODS

At Emory University and UTSW, both academic medical centers serving adult patients, 298 positive blood cultures were loaded onto the Accelerate Pheno system within 8 h of the bottle signaling positive between 1 January 2016 and 30 September 2016 (157 at Emory and 141 at UTSW). The results obtained from the Accelerate Pheno system were compared to those of the usual standard of care at each institution. Only the first positive blood culture from a patient was loaded onto the Accelerate Pheno system (duplicate positive cultures were not included in the study). All positive blood cultures included in the study had an organism present on the initial Gram stain. Institutional Review Board approval was obtained at both institutions prior to the conduct of the study.

Standard-of-care arm.

At Emory, the laboratory detection of bacteremia and fungemia was performed using the BacT/Alert 3D system with standard aerobic and anaerobic blood culture bottles (bioMérieux, Durham, NC). At UTSW, blood cultures were performed using the VersaTrek system with Redox 1 and Redox 2 bottles (Trek Diagnostic Systems, Cleveland, OH). The standard-of-care arm was as follows. After Gram staining and incubation on appropriate medium, organism identification was typically performed using the MicroScan WalkAway-96 plus system (Beckman Coulter Diagnostics, Brea, CA) at both institutions. At UTSW, the identification method was switched during the study to MALDI-TOF (Microflex LT; Bruker Daltronics, Germany) starting on 21 May 2016. In certain instances, organism identification was performed using biochemical tests or API (analytical profile index) strips (bioMérieux). At both institutions, antimicrobial susceptibility testing was performed using the MicroScan WalkAway-96 plus system with a Pos Combo 33 panel for Gram-positive organisms and the Neg/Urine Combo 61 panel for Gram-negative organisms.

Experimental arm.

The experimental arm at both institutions involved loading 5 ml of blood from the positive blood culture bottle into the Accelerate PhenoTest BC kit. The kit was then loaded into the Accelerate Pheno system (software version 1.0, research use only: pre-FDA clearance). The FDA-cleared Accelerate PhenoTest BC kit identifies the following Gram-positive and Gram-negative bacteria and yeasts: S. aureus, Staphylococcus lugdunensis, coagulase-negative Staphylococcus species (including S. epidermidis, S. haemolyticus, S. hominis, S. capitis, S. lugdunensis, and S. warneri), Enterococcus faecalis, E. faecium, Streptococcus spp. (including S. mitis, S. oralis, S. gallolyticus, S. agalactiae, and S. pneumoniae), Pseudomonas aeruginosa, Acinetobacter baumannii, Klebsiella spp. (including K. pneumoniae and K. oxytoca), E. coli, Enterobacter spp. (including E. cloacae and E. aerogenes), Proteus spp. (including P. mirabilis and P. vulgaris), Citrobacter spp. (including C. freundii and C. koseri), S. marcescens, Candida albicans, and C. glabrata. AST is performed using 18 drugs and 2 phenotypic resistance markers (Table 1).

Occasionally, the Accelerate Pheno system had a run failure or invalid result. These invalid results occurred when the system identified too few cells for analysis. In these instances, the Accelerate Pheno system did not provide an identification or AST result. With a valid run, the Accelerate Pheno system stated whether there was a positive or negative identification for each of the 16 organisms on the Accelerate PhenoTest BC kit. In addition to a positive or negative identification, the system occasionally gives an indeterminate result. This is given when the quantity present is significantly different from a negative result but does not meet the criteria for a positive result. Finally, the Accelerate Pheno system sometimes identifies off-panel organisms. Off-panel organism identifications occur when the system determines that an organism is present but is not one of the 16 organisms on the Accelerate PhenoTest BC kit. In any of these instances—a run failure, an indeterminate identification result, or the identification of an off-panel organism—the manufacturer recommends performing an alternate testing method for identification and susceptibility results.

Data analysis.

For the purpose of data analysis, the following definitions were used for the organisms listed in Table 1. A true-positive result occurred when the Accelerate Pheno system identified the same microbe as the standard of care. A true-negative result occurred when neither the Accelerate Pheno system nor the standard of care identified an organism. A false-positive result occurred when the Accelerate Pheno system identified an organism that the standard of care did not, and a false-negative result occurred when the Accelerate Pheno system failed to identify an organism that the standard of care did. These labels (true positive, true negative, false positive, and false negative) were given to each organism for each run. Sensitivity, specificity, positive predictive value, and negative predictive value were calculated from these data. When the standard of care identified a microbe as a coagulase-negative Staphylococcus sp. (identification to the species level was not performed if the isolate was a suspected contaminant), this was considered to represent a true-positive result if the experimental arm identified the isolate as a coagulase-negative Staphylococcus sp. This assumption was true as long as the coagulase-negative Staphylococcus sp. was S. epidermidis, S. haemolyticus, S. hominis, S. capitis, S. lugdunensis, or S. warneri. In addition, when the standard of care identified a microbe as alpha-hemolytic streptococcus or viridans streptococcus (identification to the species level was not performed if the isolate was a suspected contaminant), this was not considered a true-positive result if the experimental arm identified the isolate as a Streptococcus sp. This assumption was true unless the viridans streptococcus was actually S. mitis, S. oralis, or S. gallolyticus. This was done because the Accelerate Pheno system Streptococcus sp. result does not include S. mutans, S. salivarius, S. anginosus, and Streptococcus spp.

For the AST analysis, category agreement was defined as the Accelerate Pheno system and the standard of care result having the same susceptible, intermediate, or resistant result using Clinical and Laboratory Standards Institute (CLSI) M100-S26 breakpoints (32). During this study, Emory University and UTSW were using the FDA 2009 breakpoints for cephalosporins and carbapenems, and thus, for some organism-drug combinations, comparison of the standard of care to the Accelerate Pheno system was problematic. For example, there were instances where the standard of care reported an E. coli isolate as susceptible to ceftriaxone (FDA 2009 breakpoint, ≤8 μg/ml) but the Accelerate Pheno system reported the E. coli isolate as resistant to ceftriaxone (CLSI 2016 breakpoint, ≥4 μg/ml). It is possible that both the standard of care and the Accelerate Pheno system were correct in this instance (true MIC, 4 or 8 μg/ml), or it is possible that an error was made. For the purposes of data analysis, this result was considered category agreement, since it was possible that the Accelerate Pheno system result was correct, so category agreement was assumed. Very major errors were defined as the Accelerate Pheno system reporting an isolate as susceptible when the standard of care reported the isolate as resistant. Major errors were defined as the Accelerate Pheno system reporting an isolate as resistant when the standard of care reported the isolate as susceptible. Minor errors were defined as the Accelerate Pheno system or standard of care reporting an isolate as intermediate when the other method reported the isolate as susceptible or resistant. Also, on the rare occasions when 2 organisms that gave the same Accelerate Pheno system result were identified (e.g., S. epidermidis and S. hominis identified from the same blood culture or two different colony morphologies of E. coli), the more resistant phenotype for each pair was used as the comparator to the Accelerate Pheno system.

Discrepancy analysis.

Identification and AST discrepancy analysis were performed at Accelerate Diagnostics, Inc. Study samples were sent to the company as frozen blood aliquots (from the original positive blood culture). The aliquots were thawed, and the blood was plated onto medium. Identifier (ID) discrepancy analysis was performed using an identification method different from the Accelerate Pheno system and the standard of care. This was either MALDI-TOF or Vitek 2 (ID card reference number, GP21342, GN21341, or YST21343). For one isolate, discrepancy analysis was performed at the Georgia Department of Health laboratory because MALDI-TOF is unable to differentiate between E. coli and S. flexneri. AST discrepancy analysis was performed using broth microdilution panels (performed in triplicate, with the mode chosen as the true result). This was performed in all cases of very major errors or major errors.

Discrepancy analysis was needed in the identification portion of the study because the standard-of-care arm did not always report a definitive identification. Isolates were sometimes classified as coagulase-negative Staphylococcus spp. or viridans streptococci, and discrepancy analysis was needed to determine the exact species and whether an identification error had truly occurred. Discrepancy analysis was needed for the AST portion of the study because the standard-of-care instrument (MicroScan WalkAway-96 plus system) has some known limitations, for example, overcalling daptomycin resistance in Enterococcus spp. (33).

RESULTS

Identification analysis.

Of the 298 positive blood cultures in this study, 21 runs were excluded due to Accelerate Pheno system run failures, leaving 277 runs for analysis (no result was given by the Accelerate Pheno system for these 21 runs, and thus, accuracy could not be assessed). In addition, there were 23 runs in which an off-panel organism was identified by the Accelerate Pheno system and 36 runs in which there was an indeterminate result for an organism. In these 59 cases, the manufacturer recommended proceeding with routine organism identification and susceptibility testing using an alternative identification and AST method. A definitive identification result (instances in which the manufacturer did not recommend additional testing) was thus provided by the Accelerate Pheno system for 218 runs (218/298; 73.2%) (Fig. 1). The definitive result was correct in 173 instances (173/218; 79.4%). Incorrect results were due to the presence of false-positive results; false-negative results; or, in 5 instances, a polymicrobial sample in which one organism was correctly identified but the other organism was not part of the Accelerate PhenoTest BC kit. The Accelerate Pheno system thus gave definitive and correct results for 173/298 runs (58.1%).

FIG 1.

FIG 1

Flowchart for isolates available for identification and AST analysis.

For organism identification, of the 277 runs available for analysis, the Accelerate Pheno system gave a definitive and correct identification (no indeterminate results, no off-panel organisms, and no false-positive or false-negative results) 62.5% of the time (173/277). Of these 277 runs, there were 253 instances of monomicrobial bacteremia and 24 instances of polymicrobial bacteremia. For the runs with monomicrobial bacteremia, the Accelerate Pheno system gave a definitive and correct identification 67.2% of the time (170/253). For the runs with polymicrobial bacteremia, the Accelerate Pheno system gave a definitive and correct identification 12.5% of the time (3/24).

For the organisms the Accelerate Pheno system is FDA approved to detect (excluding indeterminate results), the overall sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were 94.7%, 98.9%, 83.7%, and 99.7%, respectively (Table 2). For the monomicrobial bacteremia runs, the Accelerate Pheno system demonstrated an overall sensitivity, specificity, PPV, and NPV of 98.6%, 99.0%, 83.9%, and 99.9%, respectively (see Table S1 in the supplemental material). For the polymicrobial bacteremia runs, the Accelerate Pheno system demonstrated an overall sensitivity, specificity, PPV, and NPV of 68.8%, 98.6%, 81.5%, and 97.2%, respectively (see Tables S2 and S3 in the supplemental material). A list of the organisms identified by the standard of care that are not present on the Accelerate PhenoTest BC kit is provided in Table S4 in the supplemental material.

TABLE 2.

Overall identification performance (pre-discrepancy testing) for the 277 analyzable runs

Speciesa No. true positive No. true negative No. false positive No. false negative No. indeterminate Sensitivity (%) Specificity (%) PPV (%) NPV (%)
CNS 70 179 10 6 12 92.1 94.7 87.5 96.8
EFS 10 265 0 0 2 100 100 100 100
EFM 6 267 3 1 0 85.7 98.9 66.7 99.6
SAU 31 219 11 1 15 96.9 95.2 73.8 99.6
SLU 0 275 0 0 2 N/A 100 N/A 100
STR 15 253 9 0 0 100 96.6 62.5 100
ABA 1 276 0 0 0 100 100 100 100
CIT 3 273 0 1 0 75.0 100 100 99.6
ENT 2 269 0 0 6 100 100 100 100
ECO 46 229 2 0 0 100 99.1 95.8 100
KLE 20 253 0 1 3 95.2 100 100 99.6
PRO 5 271 1 0 0 100 99.6 83.3 100
PAE 10 265 0 2 0 83.3 100 100 99.3
SMA 7 270 0 0 0 100 100 100 100
CAL 3 272 1 1 0 75.0 99.6 75.0 99.6
CGL 2 267 8 0 0 100 97.1 20.0 100
Gram-positive organisms 132 1458 33 8 31 94.3 97.8 80.0 99.5
Gram-negative organisms 94 2106 3 4 9 95.9 99.9 96.9 99.8
All yeasts 5 539 9 1 0 83.3 98.4 35.7 99.8
All organisms 231 4103 45 13 40 94.7 98.9 83.7 99.7
a

CNS, coagulase-negative Staphylococcus spp.; EFS, Enterococcus faecalis; EFM, Enterococcus faecium; SAU, Staphylococcus aureus; SLU, Staphylococcus lugdunensis; STR, Streptococcus spp.; ABA, Acinetobacter baumannii; CIT, Citrobacter spp.; ENT, Enterobacter spp.; ECO, Escherichia coli; KLE, Klebsiella spp.; PRO, Proteus spp.; PAE, Pseudomonas aeruginosa; SMA, Serratia marcescens; CAL, Candida albicans; CGL, Candida glabrata.

For the discrepancy analysis, there were 58 identification errors (see Table S5 in the supplemental material). Nine of these errors were resolved in favor of the Accelerate Pheno system. Six isolates were identified by the standard of care as viridans streptococci (using biochemical tests) and by the Accelerate Pheno system as Streptococcus spp. (false positives). MALDI-TOF confirmed the isolates as Streptococcus mitis/Streptococcus oralis. Three isolates were identified by the standard of care as a coagulase-negative Staphylococcus species, and the Accelerate Pheno system did not identify them (false negatives). MALDI-TOF confirmed the isolates as Staphylococcus caprae, Staphylococcus cohnii, and Staphylococcus lentus. These are all coagulase-negative Staphylococcus spp. not on the Accelerate PhenoTest BC kit panel. Of the remaining 49 identification errors not resolved by discrepancy analysis, there were 39 false-positive results and 10 false-negative results. Most of the false-positive results were due to coagulase-negative Staphylococcus (n = 10), Staphylococcus aureus (n = 11), or Candida glabrata (n = 8).

AST analysis.

For the AST analysis, of the 277 runs available for analysis (with run failures excluded), there were 221 runs that contained one of the 13 organisms for which the Accelerate Pheno system is able to provide AST results. Of these 221 runs, the Accelerate Pheno system provided AST results for 191 runs (86.4%). Reasons for the Accelerate Pheno system not providing an AST result included an Accelerate Pheno system AST failure, the organism being misidentified or incompletely identified (false-negative or indeterminate identification result), and a polymicrobial identification result with similar organisms. Of the 191 runs for which the Accelerate Pheno system provided an AST result, 146 had a standard-of-care AST result. Of these 146 runs, there were 12 very major errors, 5 major errors, and 55 minor errors. The 17 very major errors and 5 major errors occurred in 13 runs. Thus, there were 133 runs with an AST result without a very major error or major error (91.1% of evaluable results [133/146] and 44.6% of all runs [133/298]).

There were 117 runs with true-positive identification results for Gram-positive organisms. No standard-of-care AST was performed in 47 instances, and 11 were excluded because no Accelerate Pheno system AST was provided. This left 59 isolates available for AST analysis (Fig. 1 and Table 3). The category agreement for Gram-positive organisms was 97.1% (4 very major errors, 2 major errors, and 1 minor error). The very major error rate was 7.1% (4 very major errors/56 resistant results), and the major error rate was 1.1% (2 major errors/186 susceptible results). There were 94 true-positive identification results for Gram-negative organisms, and 7 were excluded because no Accelerate Pheno system AST was provided. This left 87 isolates available for AST analysis (Fig. 1 and Table 4). The category agreement for Gram-negative organisms was 93.3% (8 very major errors, 3 major errors, and 54 minor errors). The very major error rate was 7.4% (8 very major errors/108 resistant results), and the major error rate was 0.4% (3 major errors/837 susceptible results). The category agreement for all organisms (Gram positive and Gram negative combined) was 94.1%, with 12 very major errors, 5 major errors, and 55 minor errors. The overall very major error rate was 7.3% (12 very major errors/164 resistant results), and the major error rate was 0.5% (5 major errors/1,023 susceptible results).

TABLE 3.

AST performance for Gram-positive organisms (pre-discrepancy testing) for the 59 analyzable runs

Organisma Drug/resistance phenotypeb Category agreement [no./total (%)] No. of errors
Very major Major Minor
CNS (n = 21) DAP 21/21 (100) 0 0 0
VAN 20/20 (100) 0 0 0
MRS 13/15 (86.7) 1 1 0
MLSb 11/13 (84.6) 1 1 0
EFS and EFM (n = 14) AMP 14/14 (100) 0 0 0
DAP 12/13 (92.3) 1 0 0
LZD 14/14 (100) 0 0 0
VAN 12/14 (85.7) 1 0 1
SAU (n = 24) CFT 0/0 (NA) 0 0 0
DAP 24/24 (100) 0 0 0
ERY 24/24 (100) 0 0 0
LZD 24/24 (100) 0 0 0
VAN 24/24 (100) 0 0 0
MRS 22/22 (100) 0 0 0
All Gram-positive organisms (n = 59) AMP 14/14 (100) 0 0 0
CFT 0/0 (NA) 0 0 0
DAP 57/58 (98.3) 1 0 0
ERY 24/24 (100) 0 0 0
LZD 38/38 (100) 0 0 0
VAN 56/58 (96.6) 1 0 1
MRS 35/37 (94.6) 1 1 0
MLSb 11/13 (84.6) 1 1 0
Overall performance 235/242 (97.1) 4 2 1
a

CNS, coagulase-negative Staphylococcus spp.; EFS, Enterococcus faecalis; EFM, Enterococcus faecium; SAU, Staphylococcus aureus.

b

AMP, ampicillin; CFT, ceftaroline; DAP, daptomycin; ERY, erythromycin; LZD, linezolid; VAN, vancomycin; MRS, methicillin-resistant Staphylococcus; MLSb, macrolide-lincosamide-streptogrammin B resistance.

TABLE 4.

AST performance for Gram-negative organisms (pre-discrepancy testing) for the 87 analyzable runs

Organisma Drugb Category agreement [no./total (%)] No. of errors
Very major Major Minor
ABA (n = 1) AMK 1/1 (100) 0 0 0
CIT, ENT, and SMA (n = 12) AMK 12/12 (100) 0 0 0
ATM 10/12 (83.3) 1 1 0
CAZ 5/12 (41.7) 1 2 4
FEP 12/12 (100) 0 0 0
CRO 10/12 (83.3) 0 0 2
CIP 12/12 (100) 0 0 0
ETP 12/12 (100) 0 0 0
GEN 12/12 (100) 0 0 0
MEM 12/12 (100) 0 0 0
TZP 6/12 (50) 1 0 5
TOB 12/12 (100) 0 0 0
ECO, KLE, and PRO (n = 64) AMK 63/64 (98.4) 0 0 1
SAM 54/62 (87.1) 0 0 8
ATM 60/64 (93.8) 1 0 3
CAZ 59/63 (93.7) 0 0 4
FEP 61/64 (95.3) 2 0 1
CRO 64/64 (100) 0 0 0
CIP 63/64 (98.4) 0 0 1
ETP 64/64 (100) 0 0 0
GEN 64/64 (100) 0 0 0
MEM 62/63 (98.4) 0 0 1
TZP 59/64 (92.2) 2 0 3
TOB 60/63 (95.2) 0 0 3
PAE (n = 10) AMK 10/10 (100) 0 0 0
CAZ 6/10 (60) 0 0 4
FEP 8/10 (80) 0 0 2
CIP 8/10 (80) 0 0 2
GEN 10/10 (100) 0 0 0
MEM 5/10 (50) 0 0 5
TZP 5/10 (50) 0 0 5
TOB 10/10 (100) 0 0 0
All Gram-negative organisms (n = 87) AMK 86/87 (98.9) 0 0 1
SAM 54/62 (87.1) 0 0 8
ATM 70/76 (92.1) 2 1 3
CAZ 70/85 (82.4) 1 2 12
FEP 81/86 (94.2) 2 0 3
CRO 74/76 (97.4) 0 0 2
CIP 83/86 (96.5) 0 0 3
ETP 76/76 (100) 0 0 0
GEN 86/86 (100) 0 0 0
MEM 79/85 (92.9) 0 0 6
TZP 70/86 (81.4) 3 0 13
TOB 82/85 (96.5) 0 0 3
Overall performance 911/976 (93.3) 8 3 54
a

ABA, Acinetobacter baumannii; CIT, Citrobacter spp.; ECO, Escherichia coli; KLE, Klebsiella spp.; PRO, Proteus spp.; PAE, Pseudomonas aeruginosa; SMA, Serratia marcescens.

b

AMK, amikacin; SAM, ampicillin-sulbactam; ATM, aztreonam; CAZ, ceftazidime; FEP, cefepime; CRO, ceftriaxone; CIP, ciprofloxacin; ETP, ertapenem; GEN, gentamicin; MEM, meropenem; TZP, piperacillin-tazobactam; TOB, tobramycin.

For the AST discrepancy analysis, all 12 very major errors and 5 major errors underwent discrepancy testing. There were 7 very major errors and 1 major error that were resolved in favor of the Accelerate Pheno system (see Table S6 in the supplemental material). There was an Enterococcus faecium isolate for which the daptomycin interpretations were nonsusceptible by the standard of care (MIC > 4 μg/ml) and susceptible by the Accelerate Pheno system (MIC = 1 μg/ml); one isolate of Serratia marcescens resistant to ceftazidime, aztreonam, and piperacillin-tazobactam by the standard of care (MICs, >16, >16, and >64 μg/ml, respectively) and susceptible by the Accelerate Pheno system (MICs, 2, 2, and 4 μg/ml, respectively); one Proteus sp. isolate resistant to aztreonam by the standard of care (MIC = 16 μg/ml) and susceptible by the Accelerate Pheno system (MIC = 2 μg/ml); one Klebsiella sp. isolate resistant to cefepime by the standard of care (MIC > 16 μg/ml) and susceptible by the Accelerate Pheno system (MIC = 1 μg/ml); one isolate of Escherichia coli resistant to cefepime by the standard of care (MIC > 16 μg/ml) and susceptible by the Accelerate Pheno system (MIC = 2 μg/ml); and one coagulase-negative Staphylococcus species negative for the methicillin resistance phenotype marker by the standard of care and positive for the methicillin resistance phenotype marker by the Accelerate Pheno system.

After discrepancy analysis, there were 5 very major errors and 4 major errors. The post-discrepancy analysis very major error rate was 3.0% (5 very major errors/164 resistant results), and the major error rate was 0.4% (4 major errors/1,023 susceptible results). These 9 errors occurred in 7 different organisms. There was no discernible pattern to the errors. They occurred with various species (two Enterobacter spp., one E. faecium, one Klebsiella sp., one E. coli isolate, and two coagulase-negative Staphylococcus isolates) and various drugs (one vancomycin, one marker for methicillin-resistant Staphylococcus, two markers for macrolide-lincosamide-streptogramin B resistance, two piperacillin-tazobactam, two ceftazidime, and one aztreonam). See Table S6 in the supplemental material for more details on the nature of the errors.

For the 173 runs (175 organisms) in which the Accelerate Pheno system gave a definitively correct identification result, the mean and median time to identification for the Accelerate Pheno system was 1.4 h. For these 175 organisms, the Accelerate Pheno system produced an identification result a mean of 41.5 h (median, 39.9 h) faster than the standard of care (assuming both the experimental and control arms were started 30 min after a blood culture turned positive). Of these 175 organisms, 126 had an AST result on the Accelerate Pheno system. For these 126 organisms, the mean and median times to an AST result on the Accelerate Pheno system were 6.6 h. The Accelerate Pheno system produced an AST result an average of 48.4 h (median, 40.6 h) faster than the standard of care (assuming both the experimental and control arms were started 30 min after a blood culture turned positive).

DISCUSSION

Rapid diagnostic tests for identification and susceptibility testing of bacteria are urgently needed. There are several types of diagnostic tests that are currently available for the rapid identification of bacteria (2628). Some of these diagnostic tests also provide data on the presence or absence of antibiotic resistance genes (14, 15, 27, 28). However, until recently, there have not been diagnostic tests that can provide rapid phenotypic AST data. These tests are needed because an isolate's phenotypic AST profile, which is required for clinicians to make treatment decisions, cannot always be predicted based on the presence or absence of a limited number of antibiotic resistance genes, especially for Gram-negative bacteria. The Accelerate Pheno system and BC kit can help to provide this information.

Our original hypothesis was that the Accelerate Pheno system would provide an identification and AST result from positive blood cultures with accuracy equivalent to that provided by the standard of care, and more quickly. We found this hypothesis to be true, with caveats. In this study, the Accelerate Pheno system provided an identification and AST result for organisms from a positive blood culture quickly (1.4 h and 6.6 h, respectively). This was faster than the standard of care by more than a day (identification, 39.9 h faster, and AST, 40.6 h faster). The identification performed was accurate (sensitivity, 94.7%; specificity, 98.9%; PPV, 83.7%; and NPV, 99.7% [pre-discrepancy testing]), and the AST result was also accurate (category agreement, 94.1% [pre-discrepancy testing]). There were a significant number of instances however, in which the Accelerate Pheno system recommended pursuing other identification or AST methods (26.8% and 35.9%, respectively).

A limitation of the system concerning accuracy in our study was the significant number of false-positive identification results (n = 39). Some, but not all, of these errors could be identified, as they occur based on a discrepant Gram stain result. The reason for these false positives is unclear. While false-positive results are likely better for the patient than false-negative results (overtreatment rather than undertreatment), false-positive results can still lead to unnecessary exposure to antimicrobial agents and a lack of knowledge about the true pathogen. For example, in our study, there were 11 false-positive S. aureus results (PPV, 73.8%). In 8 of these 11 cases, a coagulase-negative Staphylococcus sp. was misidentified as S. aureus. There were also 8 false-positive C. glabrata results (PPV, 20.0%). These false-positive C. glabrata identifications could lead to unnecessary treatment with antifungal medications, which could have serious implications for the patient due to drug toxicity. The false-negative results were rare but present (n = 10). Another error that was noted was the misidentification of Shigella flexneri as E. coli. This error is also seen with automated biochemical test systems and MALDI-TOF instruments (2931). The relatedness of Shigella and “inactive” E. coli makes them difficult to differentiate. It is unlikely that this error is significant given the rarity of Shigella bacteremia. It should also be noted that this study was conducted on pre-FDA-cleared software (version 1.0). The manufacturer has since updated the software, although the changes were not assessed in this study.

Another limitation of the system in our study was the high number of instances in which routine identification and AST were recommended by the Accelerate Pheno system (26.8% of the time). This was because of technical failures (n = 21), indeterminate results (n = 36), or off-panel organisms that were not identified by the Accelerate Pheno system (n = 23). This implies that the Accelerate Pheno system, as tested in this study, will not be able to replace the standard of care. It will rather serve as an adjunct to the standard of care. It is possible, given the nature of the centers in this study (two large academic referral centers), that the number of off-panel organisms present was higher than would be seen in other settings. Other considerations for clinical microbiology laboratories are cost (of both the instrument and individual kits), laboratory space, workflow in the laboratory (only one sample per unit at a given time, although the system can accommodate up to 4 modules), storage space for the large single-use kits, and waste disposal of the kits.

There are features of this study that may limit generalizability. For example, identification and AST were mostly performed using the MicroScan WalkAway-96 plus system (University of Texas Southwestern [UTSW] did switch to MALDI-TOF during the study). Also, the MicroScan panels were set up and batched every morning (not put on the instrument for 24 h per day). Laboratories that release identification and AST results 24 h per day, use MALDI-TOF immediately, and/or use rapid molecular tests would likely have very different standard-of-care times to identification and AST results. Laboratories using more rapid methods than those employed in this study can compare their identification and AST times to those of the Accelerate Pheno system in this study (1.4 h and 6.6 h, respectively).

The Accelerate Pheno system is a useful tool to help rapidly identify microorganisms and perform phenotypic susceptibility testing on positive blood cultures. It is unlikely to replace current identification and susceptibility testing, but it will likely serve as a supplement to that testing. Future studies are needed to investigate the cost-effectiveness of the technology and the clinical impact of its use and to compare it to other rapid blood culture testing platforms.

Supplementary Material

Supplemental material

ACKNOWLEDGMENTS

Accelerate Diagnostics, Inc., Tucson, AZ, provided the Accelerate Pheno system modules and kits to both study sites. Accelerate Diagnostics had no role in study design, data collection, data interpretation, or the decision to submit the work for publication.

We thank the Accelerate team of diagnostic technicians for their expert technical assistance and the discrepancy analysis. We also thank the medical technologists at William P. Clements, Jr., University Hospital and Emory University Hospital who performed the standard of care workup of all the blood cultures in this study. The study would not have been possible without their excellent work.

Footnotes

Supplemental material for this article may be found at https://doi.org/10.1128/JCM.01672-17.

REFERENCES

  • 1.Angus DC. 2011. Management of sepsis: a 47-year-old woman with an indwelling intravenous catheter and sepsis. JAMA 305:1469–1477. doi: 10.1001/jama.2011.438. [DOI] [PubMed] [Google Scholar]
  • 2.Annane D, Bellissant E, Cavaillon JM. 2005. Septic shock. Lancet 365:63–78. doi: 10.1016/S0140-6736(04)17667-8. [DOI] [PubMed] [Google Scholar]
  • 3.Hotchkiss RS, Karl IE. 2003. The pathophysiology and treatment of sepsis. N Engl J Med 348:138–150. doi: 10.1056/NEJMra021333. [DOI] [PubMed] [Google Scholar]
  • 4.Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, Bellomo R, Bernard GR, Chiche JD, Coopersmith CM, Hotchkiss RS, Levy MM, Marshall JC, Martin GS, Opal SM, Rubenfeld GD, van der Poll T, Vincent JL, Angus DC. 2016. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA 315:801–810. doi: 10.1001/jama.2016.0287. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Ferrer R, Martin-Loeches I, Phillips G, Osborn TM, Townsend S, Dellinger RP, Artigas A, Schorr C, Levy MM. 2014. Empiric antibiotic treatment reduces mortality in severe sepsis and septic shock from the first hour: results from a guideline-based performance improvement program. Crit Care Med 42:1749–1755. doi: 10.1097/CCM.0000000000000330. [DOI] [PubMed] [Google Scholar]
  • 6.Kumar A, Roberts D, Wood KE, Light B, Parrillo JE, Sharma S, Suppes R, Feinstein D, Zanotti S, Taiberg L, Gurka D, Kumar A, Cheang M. 2006. Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock. Crit Care Med 34:1589–1596. doi: 10.1097/01.CCM.0000217961.75225.E9. [DOI] [PubMed] [Google Scholar]
  • 7.Seymour CW, Kahn JM, Martin-Gill C, Callaway CW, Yealy DM, Scales D, Angus DC. 2017. Delays from first medical contact to antibiotic administration for sepsis. Crit Care Med 45:759–765. doi: 10.1097/CCM.0000000000002264. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Tsalik EL, Petzold E, Kreiswirth BN, Bonomo RA, Banerjee R, Lautenbach E, Evans SR, Hanson KE, Klausner JD, Patel R, Diagnostics and Devices Committee of the Antibacterial Resistance Leadership Group. 2017. Advancing diagnostics to address antibacterial resistance: the Diagnostics and Devices Committee of the Antibacterial Resistance Leadership Group. Clin Infect Dis 64:S41–S47. doi: 10.1093/cid/ciw831. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Barlam TF, Cosgrove SE, Abbo LM, MacDougall C, Schuetz AN, Septimus EJ, Srinivasan A, Dellit TH, Falck-Ytter YT, Fishman NO, Hamilton CW, Jenkins TC, Lipsett PA, Malani PN, May LS, Moran GJ, Neuhauser MM, Newland JG, Ohl CA, Samore MH, Seo SK, Trivedi KK. 2016. Executive summary: implementing an antibiotic stewardship program: guidelines by the Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America. Clin Infect Dis 62:1197–1202. doi: 10.1093/cid/ciw217. [DOI] [PubMed] [Google Scholar]
  • 10.Boucher HW, Bakken JS, Murray BE. 2016. The United Nations and the urgent need for coordinated global action in the fight against antimicrobial resistance. Ann Intern Med 165:812–813. doi: 10.7326/M16-2079. [DOI] [PubMed] [Google Scholar]
  • 11.Carlet J, Collignon P, Goldmann D, Goossens H, Gyssens IC, Harbarth S, Jarlier V, Levy SB, N′Doye B, Pittet D, Richtmann R, Seto WH, van der Meer JW, Voss A. 2011. Society's failure to protect a precious resource: antibiotics. Lancet 378:369–371. doi: 10.1016/S0140-6736(11)60401-7. [DOI] [PubMed] [Google Scholar]
  • 12.Murray BE, Jezek A. 2017. The Antibacterial Resistance Leadership Group: first steps. Clin Infect Dis 64:S1–S2. doi: 10.1093/cid/ciw823. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Banerjee R, Özenci V, Patel R. 2016. Individualized approaches are needed for optimized blood cultures. Clin Infect Dis 63:1332–1339. doi: 10.1093/cid/ciw573. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Pardo J, Klinker KP, Borgert SJ, Butler BM, Giglio PG, Rand KH. 2016. Clinical and economic impact of antimicrobial stewardship interventions with the FilmArray blood culture identification panel. Diagn Microbiol Infect Dis 84:159–164. doi: 10.1016/j.diagmicrobio.2015.10.023. [DOI] [PubMed] [Google Scholar]
  • 15.Suzuki H, Hitomi S, Yaguchi Y, Tamai K, Ueda A, Kamata K, Tokuda Y, Koganemaru H, Kurihara Y, Ishikawa H, Yanagisawa H, Yanagihara K. 2015. Prospective intervention study with a microarray-based, multiplexed, automated molecular diagnosis instrument (Verigene system) for the rapid diagnosis of bloodstream infections, and its impact on the clinical outcomes. J Infect Chemother 21:849–856. doi: 10.1016/j.jiac.2015.08.019. [DOI] [PubMed] [Google Scholar]
  • 16.Timbrook TT, Morton JB, McConeghy KW, Caffrey AR, Mylonakis E, LaPlante KL. 2017. The effect of molecular rapid diagnostic testing on clinical outcomes in bloodstream infections: a systematic review and meta-analysis. Clin Infect Dis 64:15–23. doi: 10.1093/cid/ciw649. [DOI] [PubMed] [Google Scholar]
  • 17.Walker T, Dumadag S, Lee CJ, Lee SH, Bender JM, Cupo Abbott J, She RC. 2016. Clinical impact of laboratory implementation of Verigene BC-GN microarray-based assay for detection of gram-negative bacteria in positive blood cultures. J Clin Microbiol 54:1789–1796. doi: 10.1128/JCM.00376-16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Llanes C, Pourcel C, Richardot C, Plésiat P, Fichant G, Cavallo JD, Mérens A, GERPA Study Group . 2013. Diversity of β-lactam resistance mechanisms in cystic fibrosis isolates of Pseudomonas aeruginosa: a French multicentre study. J Antimicrob Chemother 68:1763–1771. doi: 10.1093/jac/dkt115. [DOI] [PubMed] [Google Scholar]
  • 19.Peleg AY, Hooper DC. 2010. Hospital-acquired infections due to gram-negative bacteria. N Engl J Med 362:1804–1813. doi: 10.1056/NEJMra0904124. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Guh AY, Bulens SN, Mu Y, Jacob JT, Reno J, Scott J, Wilson LE, Vaeth E, Lynfield R, Shaw KM, Vagnone PM, Bamberg WM, Janelle SJ, Dumyati G, Concannon C, Beldavs Z, Cunningham M, Cassidy PM, Phipps EC, Kenslow N, Travis T, Lonsway D, Rasheed JK, Limbago BM, Kallen AJ. 2015. Epidemiology of carbapenem-resistant enterobacteriaceae in 7 US communities, 2012–2013. JAMA 314:1479–1487. doi: 10.1001/jama.2015.12480. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Brazelton de Cárdenas JN, Su Y, Rodriguez A, Hewitt C, Tang L, Garner CD, Hayden RT. 2017. Evaluation of rapid phenotypic identification and antimicrobial susceptibility testing in a pediatric oncology center. Diagn Microbiol Infect Dis 89:52–57. doi: 10.1016/j.diagmicrobio.2017.06.014. [DOI] [PubMed] [Google Scholar]
  • 22.Charnot-Katsikas A, Tesic V, Love N, Hill B, Bethel C, Boonlayangoor S, Beavis KG. 2018. Use of the Accelerate Pheno system for identification and antimicrobial susceptibility testing of pathogens in positive blood cultures and impact on time to results and workflow. J Clin Microbiol 56:e01166-17. doi: 10.1128/JCM.01166-17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Douglas IS, Price CS, Overdier KH, Wolken RF, Metzger SW, Hance KR, Howson DC. 2015. Rapid automated microscopy for microbial surveillance of ventilator-associated pneumonia. Am J Respir Crit Care Med 191:566–573. doi: 10.1164/rccm.201408-1468OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Marschal M, Bachmaier J, Autenrieth I, Oberhettinger P, Willmann M, Peter S. 2017. Evaluation of the Accelerate Pheno system for fast identification and antimicrobial susceptibility testing from positive blood cultures in bloodstream infections caused by Gram-negative pathogens. J Clin Microbiol 55:2116–2126. doi: 10.1128/JCM.00181-17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Price CS, Kon SE, Metzger S. 2014. Rapid antibiotic susceptibility phenotypic characterization of Staphylococcus aureus using automated microscopy of small numbers of cells. J Microbiol Methods 98:50–58. doi: 10.1016/j.mimet.2013.12.021. [DOI] [PubMed] [Google Scholar]
  • 26.Lagacé-Wiens PR, Adam HJ, Karlowsky JA, Nichol KA, Pang PF, Guenther J, Webb AA, Miller C, Alfa MJ. 2012. Identification of blood culture isolates directly from positive blood cultures by use of matrix-assisted laser desorption ionization-time of flight mass spectrometry and a commercial extraction system: analysis of performance, cost, and turnaround time. J Clin Microbiol 50:3324–3328. doi: 10.1128/JCM.01479-12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Buchan BW, Allen S, Burnham CA, McElvania TeKippe E, Davis T, Levi M, Mayne D, Pancholi P, Relich RF, Thomson R, Ledeboer NA. 2015. Comparison of the next-generation Xpert MRSA/SA BC assay and the GeneOhm StaphSR assay to routine culture for identification of Staphylococcus aureus and methicillin-resistant S. aureus in positive-blood-culture broths. J Clin Microbiol 53:804–809. doi: 10.1128/JCM.03108-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Bhatti MM, Boonlayangoor S, Beavis KG, Tesic V. 2014. Evaluation of FilmArray and Verigene systems for rapid identification of positive blood cultures. J Clin Microbiol 52:3433–3436. doi: 10.1128/JCM.01417-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.He Y, Li H, Lu X, Stratton CW, Tang YW. 2010. Mass spectrometry Biotyper system identifies enteric bacterial pathogens directly from colonies grown on selective stool culture media. J Clin Microbiol 48:3888–3892. doi: 10.1128/JCM.01290-10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Snyder JW, Munier GK, Johnson CL. 2008. Direct comparison of the BD Phoenix system with the MicroScan WalkAway system for identification and antimicrobial susceptibility testing of Enterobacteriaceae and nonfermentative gram-negative organisms. J Clin Microbiol 46:2327–2333. doi: 10.1128/JCM.00075-08. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Deng J, Fu L, Wang R, Yu N, Ding X, Jiang L, Fang Y, Jiang C, Lin L, Wang Y, Che X. 2014. Comparison of MALDI-TOF MS, gene sequencing and the Vitek 2 for identification of seventy-three clinical isolates of enteropathogens. J Thorac Dis 6:539–544. doi: 10.3978/j.issn.2072-1439.2014.02.20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Clinical and Laboratory Standards Institute. 2016. Performance standards for antimicrobial susceptibility testing; approved standard; 26th informational supplement. CLSI document M100-S26. Clinical and Laboratory Standards Institute, Wayne, PA. [Google Scholar]
  • 33.Palavecino EL, Burnell JM. 2013. False daptomycin-nonsusceptible MIC results by Microscan panel PC 29 relative to Etest results for Staphylococcus aureus and enterococci. J Clin Microbiol 51:281–283. doi: 10.1128/JCM.01721-12. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental material

Articles from Journal of Clinical Microbiology are provided here courtesy of American Society for Microbiology (ASM)

RESOURCES