Evaluation of Three Rapid Diagnostic Methods for Direct Identification of Microorganisms in Positive Blood Cultures

Abstract

The identification of organisms from positive blood cultures generally takes several days. However, recently developed rapid diagnostic methods offer the potential for organism identification within only a few hours of blood culture positivity. In this study, we evaluated the performance of three commercial methods to rapidly identify organisms directly from positive blood cultures: QuickFISH (AdvanDx, Wolburn, MA), Verigene Gram-Positive Blood Culture (BC-GP; Nanosphere, Northbrook, IL), and matrix-assisted laser desorption ionization–time of flight mass spectrometry (MALDI-TOF MS) with Sepsityper processing (Bruker Daltonics, Billerica, MA). A total of 159 blood cultures (VersaTREK Trek Diagnostic Systems, Cleveland, OH) positive for Gram-positive and Gram-negative bacteria as well as yeast were analyzed with QuickFISH and MALDI-TOF MS. In all, 102 blood cultures were analyzed using the BC-GP assay. For monomicrobial cultures, we observed 98.0% concordance with routine methods for both QuickFISH (143/146) and the BC-GP assay (93/95). MALDI-TOF MS demonstrated 80.1% (117/146) and 87.7% (128/146) concordance with routine methods to the genus and species levels, respectively. None of the methods tested were capable of consistently identifying polymicrobial cultures in their entirety or reliably differentiating Streptococcus pneumoniae from viridans streptococci. Nevertheless, the methods evaluated in this study are convenient and accurate for the most commonly encountered pathogens and have the potential to dramatically reduce turnaround time for the provision of results to the treating physician.

INTRODUCTION

Bloodstream infections (BSIs) are associated with high mortality, longer hospital stays, and increased costs of care (1). Timely diagnosis of the etiology of infection is imperative, with mortality increasing nearly 8% for every hour of inappropriate antimicrobial therapy administered to patients with sepsis (2). Although patients with suspected BSI are typically treated empirically with broad-spectrum antimicrobial agents (3), these are not necessarily always the most appropriate therapy for the infecting organism. Furthermore, previous studies have shown improved patient outcomes in patients switched to narrower-spectrum therapy (4, 5). Early organism identification and detection of antimicrobial resistance directly from blood cultures can therefore facilitate earlier optimization of treatment.

The current standard of practice for diagnosing BSI continues to be blood culture. Once a culture is positive for growth, definitive organism identification and antimicrobial susceptibility testing require additional growth in vitro. Thus, definitive results of positive blood cultures are generally not available to the treating physician for approximately 24 to 72 h after culture positivity is first noted (6). Because traditional microbiological methods do not necessarily provide results in a clinically optimal time frame, a number of technologies have been developed that enable rapid organism identification directly from positive blood cultures. These include both amplification (i.e., PCR)-based and non-amplification-based methodologies (6–10). Importantly, previous studies have shown that rapid organism identification can positively impact patient care by shortening length of stay, lowering mortality rates, and reducing overall hospital-associated costs (11–14).

Peptide nucleic acid fluorescence in situ hybridization (PNA FISH) is a targeted approach detecting specific organisms that are commonly isolated from blood cultures. A number of different assays have been developed, each capable of detecting up to a maximum of 3 different species per assay. More recently, this method has been modified to incorporate a self-reporting probe design (QuickFISH), thereby eliminating the washing step and reducing the assay time to less than 30 min (15, 16). The Verigene Blood Culture Gram-Positive (BC-GP) test uses array-based nanoparticle technology capable of detecting 12 different Gram-positive organisms from positive blood cultures in a single assay. The assay has a run time of ∼2.5 h and is capable of detecting the genes encoding methicillin (mecA) and vancomycin (vanA and vanB) resistance (17, 18). In contrast to QuickFISH and the BC-GP assay, matrix-assisted laser desorption ionization–time of flight mass spectrometry (MALDI-TOF MS) theoretically has the potential to identify any organism from a positive blood culture. Nevertheless, this method has shown variable results in this regard (19–23). Importantly, blood culture samples require processing prior to MALDI-TOF MS analysis to remove nonbacterial proteins such as serum proteins and hemoglobin (24–27) and a variety of such methods have been published (28–30).

The comparative performances of QuickFISH, the BC-GP assay, and MALDI-TOF MS have not been previously established; neither has the performance of QuickFISH for the detection of Gram-negative bacilli or Candida species been ascertained. The goal of this study was to compare the performances of these methods for the direct identification of microorganisms from positive blood cultures.

(This work was presented in part as a poster at the American Society for Microbiology 113th General Meeting [Denver, CO]).

MATERIALS AND METHODS

Specimens.

Blood cultures were collected in VersaTREK Redox 80-ml aerobic and anaerobic blood culture bottles as part of routine clinical care and incubated on a VersaTREK instrument (Trek Diagnostic Systems, Cleveland, OH). Aerobic blood cultures (n = 159) were prospectively enrolled between August 2012 and December 2012 at the University of Washington Medical Center (UWMC) clinical microbiology laboratory, which serves a 450-bed tertiary care hospital, Seattle Cancer Care Alliance, and multiple outpatient clinics. The VersaTREK Redox 2 blood culture broth is not compatible with the QuickFISH assays; therefore, anaerobic blood cultures were excluded from the study. Only one blood culture per patient episode was included (i.e., no duplicate specimens were included). Specimens were used and data analyses performed in accordance with procedures approved by the University of Washington Institutional Review Board (IRB approval no. 29860).

Routine blood culture processing.

At the time of blood culture positivity, 3 ml of the culture broth was aseptically transferred to a sterile polystyrene round-bottom tube. Following Gram stain analysis and reporting of results to the treating physician, subculture to the following media was performed where appropriate: blood agar (tryptic soy agar with 5% sheep blood), chocolate agar, brucella agar, and MacConkey lactose agar (all from Remel, Lenexa, KS). If yeast were noted in the Gram stain, subculture to HardyCHROM Candida and inhibitory mold agars was also performed (Hardy Diagnostics, Santa Maria, CA). Isolates were definitively identified using a combination of MALDI-TOF MS (Bruker Daltonics, Billerica, MA), standard biochemical tests, API-20E, and Vitek 2 (both from bioMérieux SA, Marcy l'Etoile, France). In cases where the isolate could not be identified using these tests, 16S rRNA gene sequencing was performed as described previously (33). Antimicrobial susceptibility testing was performed using custom broth microdilution panels (TREK Sensititre; Trek Diagnostic Systems) and interpreted according to CLSI guidelines.

Study blood culture processing.

Leftover positive blood culture specimens were processed for the study immediately following the completion of clinical processing. When Gram-positive cocci were observed, a 1-ml aliquot was frozen at −70°C for subsequent testing with the BC-GP assay (see “Verigene Gram-Positive Blood Culture test” below). All specimens were subjected to immediate analysis with the QuickFISH assay appropriate to the Gram stain result (see “QuickFISH” below). A 1-ml aliquot was also transferred to a microcentrifuge tube for Sepsityper processing and stored at room temperature until processing (see “Sepsityper processing and MALDI-TOF analysis” below).

QuickFISH.

The following QuickFISH tests (AdvanDx) were used in this study: (i) Staphylococcus QuickFISH for Gram-positive cocci in clusters, which differentiates Staphylococcus aureus (green) from coagulase-negative staphylococci (red); (ii) Enterococcus QuickFISH for Gram-positive cocci in pairs and chains, which differentiates Enterococcus faecalis (green) from other Enterococcus spp., including E. faecium (red); (iii) Gram-negative QuickFISH for Gram-negative bacilli, which differentiates Escherichia coli (green) from Pseudomonas aeruginosa (red) and Klebsiella pneumoniae (yellow); and (iv) Candida QuickFISH kit for yeast, which differentiates Candida albicans (green) from C. glabrata (red) and C. parapsilosis sensu stricto (yellow).

All reagents were used according to the manufacturer's instructions. QuickFISH slides were placed on a SlideStation 55°C heat block, and 10 μl of blood culture media was added to the sample well of the slide. A single drop of QuickFIX-1 reagent was quickly added and thoroughly mixed over the entire well using a sterile disposable needle. The sample was allowed to air dry (1 to 3 min), after which time 2 drops of QuickFIX-2 were added and allowed to dry (∼1 min). The appropriate QuickFISH reagent (Staphylococcus, Enterococcus, Gram-negative bacteria, or Candida) was selected, and a drop each of PNA Yellow and PNA Blue was added to the center of a coverslip and mixed until a uniform green color was achieved. The coverslip was placed on the slide, and after a 15-min hybridization at 55°C, the slide was immediately visualized using the 50× and/or 100× oil objective of a fluorescence microscope (Nikon Instruments Inc., Melville, NY). Sample results were interpreted as negative when no fluorescence was observed.

Verigene Gram-Positive Blood Culture test (BC-GP).

The targets detected by the Verigene BC-GP assay (Nanosphere) are as follows: Staphylococcus spp., S. aureus, S. epidermidis, S. lugdunensis, Streptococcus spp., S. agalactiae, S. anginosus group, S. pneumoniae, S. pyogenes, Enterococcus faecalis, E. faecium, and Listeria monocytogenes, as well as the mecA, vanA, and vanB resistance genes. The test was performed according to the manufacturer's instructions using frozen blood culture specimens that were allowed to thaw at room temperature prior to testing. All consumables were loaded into an SP processor, the sample was mixed using a vortex device, and 350 μl was transferred to the sample well of the extraction tray. The SP processor drawer was closed, and the assay was initiated. Upon completion of the procedure, the test cartridge was removed from the processor and placed on its side to dry. Next, the reagent pack and protective tape were removed from the substrate holder, the barcode was scanned, and the substrate holder was placed into the reader for analysis. In the event of a “No Call” error, the specimen was retested.

Sepsityper processing and MALDI-TOF analysis.

All samples were processed within 8 h of culture positivity using the Bruker Sepsityper kit (Bruker Daltonics). Briefly, 200 μl of lysis buffer was added to the blood culture specimen, mixed using a vortex device for 10 s, and subjected to centrifugation (10,000 × g, 1 min). The pellet was washed twice in washing buffer and subsequently suspended in 75% ethanol. Because of staffing considerations during the study, ethanol extracts were stored at room temperature for a maximum of 5 days until analysis by MALDI-TOF MS occurred; this time frame is known not to affect analysis by MALDI-TOF MS (Gongyi Shi, Bruker Daltonics, personal communication). At the time of analysis, samples were mixed by inversion and centrifuged (10,000 × g, 2 min) and the pellet was extracted using equal volumes of 70% acetonitrile and formic acid. The extraction was centrifuged (10,000 × g, 2 min), and final supernatants were spotted onto a 96-well polished stainless steel target plate and overlaid with 1 μl of α-cyano-4-hydroxycinnamic acid matrix (both from Bruker). After drying, samples were subjected to analysis using a Bruker Microflex LT system according to the manufacturer's recommendations. The resulting spectra were analyzed with Bruker Biotyper 3.0 software as described previously and a reference spectrum library using blood-culture-specific parameters that excluded mass peaks with m/z ratios of <4,000 (21). Identification scores of 1.6 to <1.8 were considered valid to the genus level, with scores of ≥1.8 to 3.0 considered valid to the species level. Scores of <1.6 were considered invalid.

Resolution of discordant BC-GP results.

One sample (BC61) repeatedly produced an internal control error when analyzed with the BC-GP assay. Approximately 10² CFU of the E. faecium strain recovered from BC61 was inoculated into an aerobic VersaTREK Redox 80-ml bottle containing 5 ml of blood and incubated until positive for growth. At the time of culture positivity, a 1-ml aliquot was removed and analyzed with the BC-GP assay. A second specimen (BC169) showed discordant results for the detection of vancomycin resistance. The E. faecium strain isolated from BC169 was subjected to vancomycin and teicoplanin susceptibility testing by Etest (bioMérieux) and interpreted according to CLSI guidelines. In addition, PCR amplification of the vanR, vanS, and vanA genes was performed using previously published primers (34, 35). Genomic DNA was extracted using an UltraClean Microbial DNA isolation kit (MoBio, Carlsbad, CA). Briefly, each 50 μl PCR mixture consisted of 5 μl of extracted DNA, 27.5 μl H₂O, 0.5 μM (each) primer, 0.2 mM deoxynucleoside triphosphates (dNTPs), 1.5 mM MgCl₂, 5× buffer, and 2.5 U of AmpliTaq polymerase (Roche Applied Science, Indianapolis, IN). PCR conditions were as follows: 95°C for 5 min; 94°C for 30 s, 47°C for 30 s, and 72°C for 30 s for 30 cycles; and 72°C for 10 min. Amplified DNA was visualized on a 1% agarose gel stained with 0.1% ethidium bromide.

Data analysis.

The results obtained with QuickFISH, the BC-GP assay, and MALDI-TOF MS were compared to those obtained by routine culture identification and susceptibility testing methods. The following results were considered correctly identified: (i) an organism detectable by the test method that was appropriately detected (i.e., true positive) or (ii) an organism not detectable by the test method that was appropriately not detected (i.e., true negative). Concordance with routine culture methods was calculated as the number of correct organism identifications divided by the total number of organisms identified. For MALDI-TOF MS, concordance with culture results was calculated to both the genus and species levels (i.e., score values of >1.6 to <1.8 and ≥1.8, respectively).

RESULTS

Multitest evaluation of QuickFISH, the BC-GP assay, and MALDI-TOF MS.

In order to assess the relative performances of rapid blood culture identification methods, we simultaneously evaluated the performances of QuickFISH, the BC-GP assay, and MALDI-TOF MS. In all, 159 positive aerobic blood cultures were analyzed by both QuickFISH and MALDI-TOF MS. Of these 159 cultures, 102 were positive for Gram-positive bacteria by Gram stain and were also analyzed with the BC-GP assay. A total of 174 organisms were isolated from these 159 positive aerobic blood cultures (Table 1), with monomicrobial blood cultures accounting for 91.8% of cultures (n = 146). We did not detect vanB, Listeria monocytogenes, or S. lugdunensis during this study. As shown in Table 2, QuickFISH and the BC-GP assay both showed 97.9% concordance with routine methods for monomicrobial cultures (143/146 and 93/95, respectively). By comparison, MALDI-TOF MS showed genus and species level concordance with routine methods for 87.7% (128/146) and 80.1% (117/146) of monomicrobial cultures, respectively.

TABLE 1.

Distribution of microorganisms isolated^a

Definitive ID	No. of monomicrobial isolates	No. of polymicrobial isolates	Total no. of isolates (%)
Gram-positive bacteria
CoNS	45	5	50 (28.7)
Staphylococcus aureus	13	2	15 (8.6)
Enterococcus faecium	9	2	11 (6.3)
Enterococcus faecalis	7	0	7 (4.0)
Enterococcus casseliflavus/E. gallinarum	1	0	1 (0.6)
Viridans streptococci	7	1	8 (4.6)
Streptococcus agalactiae	1	0	1 (0.6)
Streptococcus anginosus group	2	1	3 (1.7)
Streptococcus dysgalactiae (group G)	1	0	1 (0.6)
Streptococcus pneumoniae	1	0	1 (0.6)
Streptococcus pyogenes	2	0	2 (1.1)
Micrococcus spp.	3	2	5 (2.9)
Gemella haemolysans	2	0	2 (1.1)
Granulicatella adiacens	1	1	2 (1.1)
Subtotal	95	14	109 (62.6)
Gram-negative bacteria
Escherichia coli	23	0	23 (13.2)
Klebsiella pneumoniae	5	2	7 (4.0)
Enterobacter cloacae	3	1	4 (2.3)
Pseudomonas aeruginosa	2	1	3 (1.7)
Serratia marcescens	1	2	3 (1.7)
Klebsiella oxytoca	0	2	2 (1.1)
Enterobacter asburiae	0	2	2 (1.1)
Chryseobacterim indologenes	1	1	2 (1.1)
Acinetobacter ursingii	1	1	2 (1.1)
Acinetobacter baumannii complex	1	0	1 (0.6)
Burkholderia cepacia complex	1	0	1 (0.6)
Campylobacter jejuni	1	0	1 (0.6)
Citrobacter freundii	0	1	1 (0.6)
Citrobacter koseri	1	0	1 (0.6)
Haemophilus influenzae	1	0	1 (0.6)
Haemophilus parainfluenae	1	0	1 (0.6)
Morganella morganii	1	0	1 (0.6)
Pseudomonas stutzeri	1	0	1 (0.6)
Raoultella sp.	1	0	1 (0.6)
Rhizobium radiobacter	0	1	1 (0.6)
Stenotrophomonas maltophilia	1	0	1 (0.6)
Subtotal	46	14	60 (34.5)
Yeast
Candida parapsilosis	2	0	2 (1.1)
Candida kefyr	1	0	1 (0.6)
Saccharomyces cerevisiae	1	0	1 (0.6)
Cryptococcus neoformans	1	0	1 (0.6)
Subtotal	5	0	5 (2.9)
Total	146	28	174

^aID, identification; CoNS, coagulase-negative staphylococcus.

TABLE 2.

Evaluation of QuickFISH, BC-GP, and MALDI-TOF MS for 146 monomicrobial cultures

Definitive ID	No. of cultures	No. (%) of cultures correctly identified by:
		MALDI-TOF MS (%)		QuickFISH (%)	BC-GP (%)
		Genus (>1.6)	Species (>1.8)
Staphylococci	58	57 (98.3)	51 (87.9)	57 (98.3)	58 (100)
CoNSa	45	44 (97.8)	38 (84.4)	44 (97.8)	45 (100)
S. aureus	13	13 (100)	13 (100)	13 (100)	13 (100)
mecA	3				3 (100)
Enterococci	16	16 (100)	16 (100)	16 (100)	16 (100)
E. faecium	9	9 (100)	9 (100)	9 (100)	9 (100)
E. faecalis	7	7 (100)	7 (100)	7 (100)	7 (100)
vanA	6				4 (66.7)
Streptococci	14	7 (50)	6 (42.9)	14 (100)	12 (85.7)
Viridans streptococci	7	1 (14.3)	1 (14.3)	NDb (100)	5 (71.4)
S. pyogenes	2	2 (100)	2 (100)	ND (100)	2 (100)
S. anginosus group	2	2 (100)	1 (50)	ND (100)	2 (100)
S. pneumoniae	1	0 (0)	0 (0)	ND (100)	1 (100)
S. agalactiae	1	1 (100)	1 (100)	ND (100)	1 (100)
S. dysgalactiae (group G)	1	1 (100)	1 (100)	ND (100)	1 (100)
Other	7	5 (71.4)	4 (57.1)	7 (100)	7 (100)
Micrococcus spp.	3	1 (33.3)	1 (33.3)	ND (100)	ND (100)
Gemella haemolysans	2	2 (100)	1 (50)	ND (100)	ND (100)
Granulicatella adiacens	1	1 (100)	1 (100)	ND (100)	ND (100)
E. casseliflavus/E. gallinarum	1	1 (100)	1 (100)	ND (100)	ND (100)
Enterobacteriaceae	35	34 (97.1)	32 (91.4)	33 (94.3)	—
E. coli	23	22 (95.7)	22 (95.7)	23 (100)	—
K. pneumoniae	5	5 (100)	5 (100)	3 (60)	—
E. cloacae	3	3 (100)	3 (100)	ND (100)	—
C. koseri	1	1 (100)	1 (100)	ND (100)	—
M. morganii	1	1 (100)	0 (0)	ND (100)	—
Raoultella sp.	1	1 (100)	0 (0)	ND (100)	—
S. marcescens	1	1 (100)	1 (100)	ND (100)	—
Non-Enterobacteriaceae	11	6 (54.5)	5 (45.5)	11 (100)	—
P. aeruginosa	2	2 (100)	2 (100)	2 (100)	—
A. baumannii complex	1	1 (100)	1 (100)	ND (100)	—
A. ursingii	1	0 (0)	0 (0)	ND (100)	—
B. cepacia complex	1	1 (100)	1 (100)	ND (100)	—
C. jejuni	1	0 (0)	0 (0)	ND (100)	—
C. indologenes	1	0 (0)	0 (0)	ND (100)	—
H. influenzae	1	0 (0)	0 (0)	ND (100)	—
H. parainfluenzae	1	0 (0)	0 (0)	ND (100)	—
P. stutzeri	1	1 (100)	0 (0)	ND (100)	—
S. maltophilia	1	1 (100)	1 (100)	ND (100)	—
Yeast	5	3 (60)	3 (60)	5 (100)	—
C. parapsilosis	2	1 (50)	1 (50)	2 (100)	—
C. kefyr	1	1 (100)	1 (100)	ND (100)	—
S. cerevisiae	1	0 (0)	0 (0)	ND (100)	—
C. neoformans	1	1 (100)	1 (100)	ND (100)	—
Total	146	128 (87.7)	117 (80.1)	143 (97.9)	93 (97.9)

^aCoNS data include the following species: S. epidermidis (37 cultures), S. hominis (4 cultures), S. haemolyticus (2 cultures), S. pasteuri (1 culture), and S. simulans (1 culture).

^bND, not detected; —, not done.

Gram-positive cocci.

All three methodologies displayed 100% concordance with routine methods for monomicrobial blood cultures positive for S. aureus (13/13) and enterococci (17/17) (Table 2). For coagulase-negative staphylococci, we observed 97.8% concordance with routine methods for QuickFISH (44/45), 100% (45/45) concordance for the BC-GP assay, and 84.4% species-level (38/45) concordance for MALDI-TOF MS. The BC-GP assay accurately detected mecA in S. aureus (3/3) with an initial concordance of 66.7% (4/6) for the detection of the vanA gene in E. faecium. The BC-GP assay and MALDI-TOF MS showed 85.7% (12/14) and 42.9% (6/14) concordance with routine methods, respectively, for the identification of streptococci. Even though the identification of S. pneumoniae and viridans group streptococci was problematic, the remaining streptococcal species in this study were correctly identified by the BC-GP assay and MALDI-TOF MS in 100% (6/6) and 83.3% (5/6) of cases, respectively. There is no specific QuickFISH assay available for the identification of streptococci. However, all (14/14) monomicrobial cultures positive for streptococci were appropriately not detected by the Enterococcus QuickFISH assay.

Gram-negative bacilli and yeast.

Seventeen species of Gram-negative bacteria were isolated during the study (Table 2). The QuickFISH Gram-negative assay showed 95.7% (44/46) concordance with routine methods for monomicrobial cultures compared with 80.4% (37/46) species-level concordance for MALDI-TOF MS. In the case of Enterobacteriaceae, we observed 94.3% (33/35) concordance with routine methods for QuickFISH and 91.4% (32/35) concordance for MALDI-TOF MS. In contrast, only 45.5% (5/11) of monomicrobial cultures positive with non-Enterobacteriaceae species were correctly identified to the species level by MALDI-TOF MS. QuickFISH appropriately did not detect any cultures positive for non-Enterobacteriaceae other than P. aeruginosa (i.e., no false-positive results). All blood cultures positive with yeast were monomicrobial, accounting for only 3.1% (5/159) of cultures (Table 2). Four species of yeast were isolated during the study: Candida parapsilosis (n = 2), C. kefyr (n = 1), Saccharomyces cerevisiae (n = 1), and Cryptococcus neoformans (n = 1). MALDI-TOF MS correctly identified three of the five blood cultures positive for yeast (60%). Both cultures positive for Candida parapsilosis were correctly identified by QuickFISH, with the remaining species (n = 3) appropriately not detected.

Discordant-result analyses.

We observed the highest number of discordant results with MALDI-TOF MS (n = 29). Genus-level confidence scores were obtained for 11 specimens with valid scores (i.e., 1.6 to <1.8). There were 14 specimens that produced invalid scores (i.e., <1.6). These specimens were positive for coagulase-negative staphylococci (n = 1), Micrococcus (n = 2), viridans group streptococci (n = 2), S. pneumoniae (n = 1), E. coli (n = 1), non-Enterobacteriaceae (n = 5), and yeast (n = 2). However, only four of these cultures failed to produce any spectra. We also observed four misidentifications in cultures with species-level identification scores (i.e., ≥1.8), all of which were positive for viridans group streptococci and misidentified by MALDI-TOF MS as S. pneumoniae. Similarly, both discordant identification results obtained with the BC-GP assay were instances of viridans group streptococci misidentified as S. pneumoniae. We observed three discordant results for QuickFISH, the first of which was a culture positive for S. simulans that was not detected (i.e., false negative). The remaining two discrepant results were specimens positive for K. pneumoniae that were misidentified as E. coli.

With regard to the detection of vancomycin-resistant Enterococcus by the BC-GP assay, there were two monomicrobial cultures that showed initial discordance with routine methods. In the first case, we observed repeated internal control errors for specimen BC61, although MALDI-TOF MS and QuickFISH correlated with routine methods for identification of the organism. When the BC61 isolate was inoculated into blood culture bottles and retested with the BC-GP assay upon culture positivity, the organism was correctly identified and the vanA gene appropriately detected, suggesting the presence of an inhibitory substance in the original specimen. For the second specimen (BC169), the presence of the vanA gene was detected by the BC-GP assay but susceptibility to vancomycin was observed in vitro. Even though the resulting isolate was confirmed to be susceptible to both vancomycin and teicoplanin in vitro, we detected the presence of the vanA gene by independent PCR analysis (data not shown). Therefore, upon resolution of discrepant results, we observed 100% accuracy for the detection of vanA.

In order to better understand the nature of this discordant result, we tested the isolate for the presence of the regulatory genes vanR and vanS by PCR. Deletions in either or both of the vanR and vanS genes have been shown to result in vancomycin susceptibility (36, 37). We did not detect the vanS gene in the BC169 isolate but detected vanS in a known vancomycin-resistant, vanA-positive strain (ATCC 51559) (data not shown). However, a PCR product corresponding to the vanR gene was larger in size than expected (∼1,100 bp), suggesting the presence of an insertion within the vanR gene. Thus, although the BC169 isolate harbored the vanA gene, it appeared to have regulatory gene mutations that might have affected expression of a vancomycin-susceptible phenotype.

Polymicrobial cultures.

Polymicrobial cultures accounted for 8.2% (13/159) of cultures during the study (Table 3), with a total of 28 isolates recovered. Identification of at least one of the organisms present was obtained by MALDI-TOF MS for 76.9% (10/13) of these cultures, although none of the cultures had organisms identified in their entirety. However, because we were unable to determine the isolate responsible for a positive test result in four of these cultures (BC85, BC100, BC105, and BC122) (e.g., multiple species of coagulase-negative staphylococci might have been present), these cultures were excluded from the overall analysis. A total of 19 isolates were recovered from the 9 remaining polymicrobial cultures. Of the identifications reported by MALDI-TOF MS, we observed 31.6% (6/19) concordance with routine methods. Only a single culture (BC245) revealed multiple morphologies by Gram stain and was therefore subjected to testing with two different QuickFISH assays. In this case, QuickFISH correctly detected the coagulase-negative staphylococci present and appropriately did not detect the Serratia marcescens strain. The overall concordance of QuickFISH with culture was 87.5% (14/16). We observed a single false-negative result in a culture that was positive for both P. aeruginosa (not detected) and S. marcescens (BC124) and a false-positive result in which Enterobacter cloacae was misidentified as E. coli (BC3). An additional culture was noted in which QuickFISH falsely detected Enterococcus sp., not E. faecalis, in a culture that was positive for S. anginosus group and Granulicatella spp. (BC122). However, because of the similar Gram stain morphologies of these organisms, we were unable to determine which organism was responsible for the false-positive result; this specimen was therefore excluded from the overall analysis. In the case of the BC-GP assay, we observed concordance with routine methods in three of the four cultures analyzed (75%), with a false-negative result for S. epidermidis obtained in a culture that was mixed with viridans group streptococci (BC31).

TABLE 3.

Evaluation of QuickFISH, BC-GP, and MALDI-TOF MS for 13 polymicrobial blood cultures

Blood culture	Definitive ID	Gram stain resulta	Result
			MALDI-TOF MS	QuickFISH	BC-GP
BC245	S. hominis	GPCL	NDb	CoNS	Staphylococcus sp.
	S. marcescens	GNR	S. marcescens	ND	—c
BC115	C. freundii	GNR	C. freundii	ND	—
	Micrococcus sp.	ND	ND	—	—
BC31	Viridans streptococci	GPPC	S. pneumoniae	ND	Streptococcus sp.
	S. epidermidis	ND	ND	—	ND
BC44	E. faecium/vanA	GPPC	ND	Not E. faecalis	E. faecium/vanA
	K. pneumoniae	ND	ND	—	—
BC3	K. pneumoniae	GNR	K. pneumoniae	K. pneumoniae	—
	E. cloacae	GNR	ND	E. coli	—
BC124	S. marcescens	GNR	S. marcescens	ND	—
	P. aeruginosa	GNR	ND	ND	—
BC71	K. oxytoca	GNR	ND	ND	—
	Enterobacter asburiae	GNR	E. asburiae	ND	—
BC77	K. oxytoca	GNR	ND	ND	—
	Enterobacter asburiae	GNR	E. asburiae	ND	—
BC37	C. indologenes	GNR	ND	ND	—
	A. ursingii	GNR	ND	ND	—
	R. radiobacter	GNR	ND	ND	—
BC85d	S. cohnii	GPCL	ND	CoNS	Staphylococcus sp.
	Micrococcus sp.	GPCL	ND	ND	ND
BC105d	S. epidermidis	GPCL	ND	CoNS	ND
	S. hominis	GPCL	ND	CoNS	Staphylococcus sp.
	E. faecium/vanA	ND	E. faecium	—	E. faecium/vanA
BC100d	S. aureus	GPCL	S. aureus	S. aureus	S. aureus
	S. aureus/mecA	GPCL	S. aureus	S. aureus	S. aureus/mecA
BC122d	S. anginosus group	GPPC	S. anginosus group	Not E. faecalis	S. anginosus group
	Granulicatella sp.	GPPC	ND	Not E. faecalis	ND

^aGNR, Gram-negative rod; GPCL, Gram-positive cocci in clusters; GPPC, Gram-positive cocci in pairs and chains.

^bND, not detected.

^c—, not done.

^dExcluded from data analysis.

DISCUSSION

In this study, we assessed the diagnostic accuracy of three commercially available assays for the rapid identification of microorganisms directly from positive blood cultures. Overall, our results show that these assays accurately detected the majority of organisms most commonly isolated from blood cultures. To our knowledge, this is the first head-to-head comparison of QuickFISH, the Verigene BC-GP assay, and MALDI-TOF MS.

The Staphylococcus QuickFISH assay was previously shown to be 97.1% to 99.0% sensitive and 89.5% to 100% specific for the detection of staphylococci (15, 16). More recently, the Enterococcus QuickFISH assay was shown to be 97% sensitive for the detection of non-E. faecalis enterococci (38). Our results are therefore similar to those of previous studies, with observed sensitivity and specificity of 98.3% and 100%, respectively, for the identification of staphylococci and 100% sensitivity and specificity for the detection of Enterococcus species in monomicrobial cultures (data not shown). We observed a single instance of false negativity for the detection of coagulase-negative staphylococci from a monomicrobial culture. This culture was positive for S. simulans, which is not detectable by the assay and is thus a known limitation (15).

One initial report has shown 98.4% concordance with culture for the Gram-negative QuickFISH assay (40). We observed slightly lower concordance with routine methods in our study (95.7%, 44/46). However, both discordant results were noted to have resulted from difficulty in differentiating K. pneumoniae (yellow) from E. coli (green). Subsequent investigation upon completion of the study revealed higher-than-expected levels of background fluorescence with our laboratory's microscope compared to other fluorescence microscopes tested, which might have accounted for these results (data not shown). Of note, although the Candida QuickFISH assay also requires differentiation between green and yellow, we believe that the larger size of the yeast cells may have mitigated the impact of the additional background fluorescence. We therefore believe that use of a better microscope might have allowed us to attain levels of concordance similar to those described previously (40).

Recently published studies have reported 92% to 99% concordance of the BC-GP assay with culture (39, 41–45). Prior to initiating this study, a pilot evaluation of the BC-GP assay was also performed at an affiliate hospital (Harborview Medical Center [HMC]) that included 107 consecutive aerobic and anaerobic blood cultures positive for Gram-positive cocci (see Table S1 in the supplemental material). The two hospitals use identical blood culture bottles and systems (i.e., VersaTREK) as well as routine identification methods (i.e., MALDI-TOF MS using the Bruker Biotyper system). We observed similar levels of overall concordance in the UWMC and HMC studies (97.3% and 96.1%, respectively), with a combined 98% (193/197) concordance for monomicrobial cultures. BC-GP was the only method evaluated in this study that showed 100% concordance with routine methods for the identification of staphylococci directly from positive monomicrobial cultures, although all 3 methods evaluated showed 100% concordance for the detection of S. aureus. Similar to previous studies, the BC-GP test failed to detect coagulase-negative staphylococci in several polymicrobial cultures (18, 39, 43). As with MALDI-TOF MS, we observed that BC-GP misidentified viridans streptococci as S. pneumoniae in our study, an issue that has also been noted in previously published studies. Nevertheless, the BC-GP assay was the only method evaluated in our study that is capable of detecting antimicrobial resistance. Importantly, detection of methicillin resistance in S. aureus and vancomycin resistance in enterococci directly from positive blood cultures has been previously reported to be associated with targeted therapy algorithms (42) and decreased time to initiation of appropriate therapy (43, 46) as well as decreased length of stay and hospital costs (12, 44, 46). Consistent with the findings of Wojewoda and colleagues (39), we observed 100% concordance with routine culture for the detection of methicillin resistance in S. aureus in monomicrobial cultures for both the UWMC and HMC studies (13/13). Although 62/77 (80.5%) of the S. epidermidis isolates were noted to be mecA positive by the BC-GP assay (data not shown), because our laboratories do not routinely perform susceptibility testing on isolates of coagulase-negative staphylococci, we were unable to assess the diagnostic accuracy of mecA detection in S. epidermidis. Although this was a limitation of our study, previous studies have shown excellent performance in this regard (18, 41, 43, 44).

A variety of methods for the extraction of blood cultures for MALDI-TOF MS have been published, although the accuracy of MALDI-TOF MS following Sepsityper processing has ranged between 67% and 100% (19, 21–23, 28–30, 47, 48). Use of blood culture-specific parameters for MALDI-TOF MS analysis was previously shown to improve the rates of organism identification (21, 49). Using such criteria, we observed 87.7% (128/146) genus-level and 80.1% (117/146) species-level concordance with routine methods for monomicrobial cultures. At 72.4% (126/174), the overall correlation of MALDI-TOF MS with culture in our study was similar to that of previously published studies that also employed blood culture-specific analysis parameters (21, 49). The performance of MALDI-TOF MS for the identification of Gram-negative organisms was similar to that observed by Saffert et al. (49), although it was lower than that observed in the study by Buchan and colleagues (21). However, we observed improved performance for the identification of Gram-positive organisms in monomicrobial cultures compared with previous studies (81% versus 57.5% to 63%) (21–23, 28–30, 47, 48). In particular, we observed a higher concordance with routine methods for the identification for coagulase-negative staphylococci (84.4% versus 62.5%) (21, 23). We believe that the improved performance of MALDI-TOF MS that we observed for species-level identification of coagulase-negative staphylococci may have resulted from the inclusion of an additional wash step during Sepsityper processing. Importantly, additional washing prior to specimen processing was previously demonstrated to dramatically improve the performance of MALDI-TOF MS for the identification of yeast directly from positive blood cultures (19).

Even though polymicrobial cultures accounted for only a small percentage of blood cultures in our study as well as in previously published studies, note that none of the methods evaluated were capable of reliably identifying all of the organisms present. Utilization of the “top 10 matched pattern choices” option in the Bruker Biotyper system was recently reported to facilitate identification of the minority species present (23). Nevertheless, it did so for only 5 of 16 polymicrobial cultures. In the case of QuickFISH, we observed a false-negative result in a polymicrobial culture positive for both P. aeruginosa and S. marcescens. Similarly, organism identification by BC-GP from polymicrobial cultures has also previously been shown to be problematic (18, 39, 41, 43, 45), although we did not experience such difficulties in our study. Importantly, inability to identify or detect the presence of multiple organisms in polymicrobial blood cultures also appears to be an issue for other molecular assays capable of simultaneous identification of organisms (e.g., Biofire FilmArray, PCR/electrospray ionization [ESI], etc.) (9, 50, 51). Although QuickFISH appeared to have the highest correlation with routine identification methods for polymicrobial cultures in our study, we believe that laboratories implementing this method for clinical testing are more likely to perform QuickFISH after the initial Gram stain in order to make testing more cost-effective.

There are a number of limitations associated with this study. First, although the total numbers of cultures analyzed by MALDI-TOF MS and QuickFISH were similar to those analyzed in previously published studies, the number of cultures analyzed by each of the QuickFISH assays was relatively small. A second limitation of our analysis of the QuickFISH method was that the majority of blood cultures tested with the Gram-negative QuickFISH assay were positive for E. coli. Thus, we did not have the opportunity to fully assess the sensitivity or specificity of the assay for the detection of other Gram-negative organisms. Similarly, the small number of isolates encountered during our study for several species of bacteria, as well as yeast, made it difficult to adequately assess the performance of both QuickFISH and MALDI-TOF MS for these organisms. Finally, because of the design of our study, we were unable to include an analysis of patient outcomes, as none of the methods evaluated were implemented clinically during the study period. Nevertheless, the strength of our study was the simultaneous testing of blood cultures with multiple methodologies.

The methods evaluated in our study are associated with differing hands-on time requirements as well as reagent and capital equipment costs, but all three offer the potential to substantially reduce the time to a result compared with routine methods (18, 38, 48). Nevertheless, implementation of rapid blood culture identification in a clinical microbiology laboratory will likely not eliminate the need for additional testing to be performed upon recovery of the microorganisms present on subculture. However, although routine susceptibility testing will continue to be required, workflow modifications may be possible. Laboratories may instead decide to use only limited identification methods (e.g., spot tests) in place of a full identification method if the colony morphology is consistent with that of the organism identified by the rapid identification method. Such modifications to workflow may help to offset a portion of the costs associated with these rapid identification methods. However, additional budgetary support at the hospital administration level will likely be required for many laboratories choosing to implement rapid identification from blood cultures.

In summary, all three methods evaluated in this study showed excellent performance characteristics for the identification of the organisms most commonly isolated from blood cultures. Knowledge of the comparative performances of different methodologies may facilitate implementation of a tiered approach to the testing of positive blood cultures based on known local antimicrobial resistance rates. Such an approach may make for a more cost-effective approach to rapid blood culture testing for clinical microbiology laboratories.