Evaluation of the BD Phoenix Automated Microbiology System for Identification and Antimicrobial Susceptibility Testing of Enterobacteriaceae

Abstract

We evaluated the accuracy of the BD Phoenix system for the identification (ID) and antimicrobial susceptibility testing (AST) of 251 isolates of the family Enterobacteriaceae representing 31 species. Organisms were inoculated onto the Phoenix panel according to the manufacturer's instructions. The results from conventional biochemical tests were used for the reference method for ID. Agar dilution, performed according to the CLSI guidelines, was the reference AST method. Essential and categorical agreements were determined. The overall levels of agreement for the genus- and species-level identifications were 95.6% and 94.4%, respectively. Fourteen isolates were incorrectly identified by the Phoenix system; 10 of these were incorrectly identified to the species level. Three of these were Enterobacter (Pantoea) species and four of these were Shigella spp. misidentified as Escherichia coli. For AST results, the essential and categorical agreements were 98.7% and 97.9%, respectively. The very major error, major error, and minor error rates were 0.38%, 0.33%, and 1.8%, respectively. Six isolates (three E. coli isolates and three Klebsiella isolates) were extended-spectrum β-lactamase producers. All six were flagged by the Phoenix system expert rules. The Phoenix system compares favorably to traditional methods for ID and AST of Enterobacteriaceae.

As hospitals face the continuing challenge of treating sicker patients, the burden falls to clinical microbiology laboratories to provide accurate and rapid identification (ID) of the pathogens recovered and, more importantly, to detect antimicrobial resistance. To accomplish this goal, many laboratories rely upon automated microbial identification and antimicrobial susceptibility testing (AST) systems. The newer-generation instruments have more extensive databases; data management tools, including expert systems; and other features unique to each manufacturer. The BD Phoenix automated microbiology system (BD Diagnostic Systems, Sparks, MD) is the newest system to obtain clearance from the Food and Drug Administration.

We evaluated the performance of the Phoenix instrument for the identification and susceptibility testing of challenge and clinical isolates of the family Enterobacteriaceae isolated from a variety of specimen sources in a busy tertiary-care medical center.

MATERIALS AND METHODS

Bacterial strains.

A total of 251 bacterial isolates were used for this evaluation. A total of 76 archived “challenge” gram-negative bacilli, including 19 isolates of Shigella spp. from Bangladesh, and 175 clinical isolates recovered from routine cultures in the Clinical Microbiology Laboratory of the Johns Hopkins Hospital (JHH) were included in this comparison.

Reference identification.

The laboratory's routine method for the identification of gram-negative organisms includes testing with the following conventional biochemicals. For Enterobacteriaceae, the following biochemicals incorporated into an in-house agar system were used: colistin, cefazolin, oxidase, phenylalanine deaminase, urea, hydrogen sulfide, esculin, methyl green DNase, Koser's citrate, lysine, ornithine, glucose (0.4%), sucrose, arabinose (1%), inositol (0.7%), sorbitol, adonitol, and lactose. When the agar system failed to result in an identification, biochemical substrates consisting of macrotubes of lysine, ornithine, arginine, citrate, urea, sorbitol, inositol, adonitol, malonate, and arabinose were set up. All reactions were read at 24 h (3).

Phoenix identification system.

The Phoenix identification method uses modified conventional, fluorogenic, and chromogenic substrates. Research-use-only combination panels (NMIC/ID-26; catalog no. 448026) for both identification and susceptibility testing were used for this comparison. Software versions V3.34A and V3.54A were used for this study. The ID side contains 45 wells with dried biochemical substrates and 2 fluorescent control wells. The ID broth was inoculated with bacterial colonies from a pure culture adjusted to a 0.5 McFarland standard by using a CrystalSpec nephelometer (BD Diagnostics), according to the manufacturer's recommendations. A 25-μl aliquot of this suspension was removed for AST, and the remaining suspension was then poured into the ID side of the Phoenix panel. The specimen was logged and loaded into the instrument within the specified timeline of 30 min. Quality control was performed according to the manufacturer's recommendations.

Reference antimicrobial susceptibility testing.

Agar dilution was performed according to CLSI guidelines (5). The following antibiotic concentrations were tested: amikacin, 4 to 32 μg/ml; gentamicin and tobramycin, 1 to 8 μg/ml; meropenem, 1, 4, and 8 μg/ml; cefepime, 0.5, 2, 8, and 16 μg/ml; ampicillin, 2, 8, and 16 μg/ml; ticarcillin, 8 to 64 μg/ml; trimethoprim sulfamethoxazole, 0.5 and 2 μg/ml; ciprofloxacin, 0.25 to 2 μg/ml; and tetracycline, 2 to 8 μg/ml. Etest was used for confirmation of extended-spectrum β-lactamase (ESBL) production, according to the manufacturer's instructions and in accordance with CLSI guidelines (5).

Phoenix system antimicrobial susceptibility testing.

The AST side of the combination panel contains up to 84 wells with dried antimicrobial panels and 1 growth control well. The assay is a broth-based microdilution test. The system uses a redox indicator for the detection of organism growth in the presence of an antimicrobial agent. The previously described 25 μl of the standardized ID broth suspension was transferred to the AST broth, yielding a final concentration of approximately 5 × 10⁵ CFU/ml. Quality control was performed according to the manufacturer's recommendations.

Data analysis.

All data from both the reference methods and the Phoenix instrument were transcribed into Excel spreadsheets for analysis. For AST, essential agreement (EA) and categorical agreement (CA) were determined. EA was defined as MICs that were within plus or minus 1 doubling dilution between the two systems. CA was defined as susceptible, intermediate, and resistant results that matched between the two systems. The very major error (VME) rate (a false-susceptible result with the Phoenix system), the major error (ME) rate (a false-resistant result with the Phoenix system), and the minor error (mE) rate (one system reported an intermediate result, while the other method reported a resistant or susceptible result) were calculated. The number of resistant strains was used as the denominator for the calculation of the VME rates. For the calculation of ME rates, the number of susceptible strains was used as the denominator.

Discrepancy resolution.

For resolution of discrepant identification results between the two systems, testing was repeated by both methods. Biochemical testing was accepted as the “gold standard.” For susceptibility testing, both methods were likewise repeated for any organism-drug discrepancy between the two systems that resulted in MEs and VMEs. Agar dilution testing was accepted as the “gold standard” for all antimicrobial susceptibility test results.

RESULTS

Identification.

Upon initial testing there were 21 discrepant results between the Phoenix system and the conventional methods. After repeat testing, genus-level agreement occurred for 240/251 of the isolates (95.6%) and species-level agreement occurred for 237/251 (94.4%) of the isolates. The variety and numbers of isolates and the percent agreement after repeat testing are listed in Table 1. The genus and species ID discrepancy details are listed in Table 2. There was no predominating pattern to the incorrect identifications. Four of the discrepancies occurred with members of the genus Enterobacter and former members of that genus that are now in the genus Pantoea. Two E. coli isolates and four Shigella strains were also misidentified. Two strains of Klebsiella pneumoniae were misidentified within the same genus as Klebsiella oxytoca.

TABLE 1.

Enterobacteriaceae tested by JHH classical methods and the Phoenix instrument

Genus and species	No. of isolates correctly identified by:		% Agreement
	JHH method	Phoenix ID system
Citrobacter spp.	11	10	90.9
C. amalonaticus	2	1	50
C. koseri	5	5	100
C. freundii	2	2	100
C. youngae	2	2	100
Enterobacter spp.	25	24	92
E. aerogenes	7	7	100
E. cloacae	16	14	87.5
E. gergoviae	1	1	100
E. sakazakii	1	1	100
Escherichia spp.	89	86	96.6
E. coli	88	86	96.6
E. hermannii	1	1	100
Ewingella americana	2	2	100
Klebsiella spp.	46	46	100
K. oxytoca	11	11	100
K. pneumoniae	35	33	94.3
Leclercia adecarboxylata	1	1	100
Morganella morganii	3	3	100
Pantoea agglomerans	4	2	50
Proteus spp.	23	23	100
P. mirabilis	19	19	100
P. penneri	1	1	100
P. vulgaris	3	3	100
Providencia spp.	6	6	100
P. alcalifaciens	1	1	100
P. rettgeri	2	2	100
P. stuartii	3	3	100
Salmonella spp.	8	8	100
S. enterica serovar Choleraesuis	1	0	0
S. enterica serovar Typhi	1	1	100
Unspecified Salmonella spp.	6	6	100
Serratia marcescens	9	9	100
Shigella spp.	23	19	82.6
S. boydii	1	1	100
S. dysenteriae	5	5	100
S. flexneri	6	5	83.3
S. sonnei	11	8	72.7
Yersinia enteriditis	1	1	100
Total	251
Genus level		240	95.6
Species level		237	94.4

TABLE 2.

Identification discrepancies between reference method and the Phoenix system^a

JHH reference method ID	Phoenix system ID
Citrobacter amalonaticus	Enterobacter cloacae
Pantoea agglomerans	Leclercia adecarboxylata
Pantoea agglomerans	Shigella flexneri
Enterobacter cloacae	Enterobacter sakazakii
Enterobacter cloacae	Klebsiella pneumoniae
Klebsiella pneumoniae	Klebsiella oxytoca
Klebsiella pneumoniae	Klebsiella oxytoca
Escherichia coli	Salmonella enterica subsp. arizonae serovar Choleraesuis
Escherichia coli	Citrobacter youngae
Salmonella enterica serovar	Salmonella sp.
Cholerasuis
Shigella flexneri	Escherichia coli
Shigella sonnei	Escherichia coli
Shigella sonnei	Escherichia coli
Shigella sonnei	Escherichia coli

^aAfter repeat testing.

Antimicrobial susceptibility.

Overall, susceptibility testing was performed by agar dilution with 250 isolates of the Enterobacteriaceae, including 6 ESBL producers. One isolate of Klebsiella pneumoniae failed to grow for either susceptibility method. Initially, there were 28 AST results that required repeat testing. After repeat testing, the overall EA was 98.7% and the overall CA was 97.9%. The mE, ME, and VME rates were 1.8%, 0.33%, and 0.38%, respectively. Table 3 displays the EAs and CAs and lists the number and percentage of mEs, MEs, and VMEs stratified by antibiotic type for 226 isolates, including the 6 ESBL producers. All six ESBL producers, three E. coli isolates and three Klebsiella isolates, were identified by the Phoenix expert rules system.

TABLE 3.

Susceptibility testing results

Antibiotic	Total no. of isolates tested	Percent		No. of isolatesc:			No. (%) of isolates with:
		EA	CA	S	I	R	mE	ME	VME
Amikacin	219a	99.5	97.3	212	3	4	6 (2.7)	0	0
Gentamicin	250	98.0	99.2	224	1	25	2 (0.8)	0	0
Tobramycin	219a	91.8	95.9	188	3	28	8 (3.7)	1 (0.53)	0
Meropenem	219a	100	100	218	1	0	0	0	0
Cefepime	250	NDb	98.4	245	3	2	3 (1.2)	1 (0.41)	0
Ampicillin	250	99.2	96.0	73	15	166	8 (3.2)	2 (2.7)	0
Ticarcillin	219a	99.5	96.3	92	15	112	8 (3.7)	0	0
Trimethoprim-sulfamethoxazole	250	98.8	98.9	187		63	0	2 (2.7)	1 (0.4)
Ciprofloxacin	250	100	99.6	215	1	34	1 (0.4)	0	0
Tetracycline	250	99.2	96.8	156	8	86	7 (2.8)	0	1 (0.4)
Total	2,376	98.7	97.9	1,810	46	520	43 (1.8)	6 (0.33)	2 (0.38)

^a31 Salmonella and Shigella strains were not tested.

^bND, not determined. The EA for cefepime could not be determined because the lowest concentration tested by the Phoenix system was twofold higher than that used for the agar dilution method for the susceptible category.

^cS, susceptible; I, intermediate; R, resistant.

DISCUSSION

The Phoenix system compares favorably to conventional biochemical methods for the identification of isolates of the Enterobacteriaceae. The overall agreements for genus- and species-level identifications were 95.6% and 94.4%, respectively. Three of the errors at the species-level identification were with Enterobacter (Pantoea) species, the most concerning of which was the misidentification of an isolate of Pantoea agglomerans as Shigella flexneri. Also of concern was the misidentification of an isolate of E. coli as a Salmonella sp. All of the Shigella isolates misidentified were called E. coli. This is not unusual even for nonautomated identification systems. Ordinarily, in the usual clinical laboratory work flow, these would be resolved before reporting by using source information, growth on MacConkey agar, and testing for agglutination. Two Klebsiella isolates, one Enterobacter isolate, and one Salmonella isolate were misidentified or not identified at the species level only.

Our results for identification are similar to those reported by Donay et al. (1), Stefaniuk et al. (8), and Eigner et al. (2). All three studies compared the Phoenix system to the API 20E system for the identification of fresh clinical isolates of the Enterobacteriaceae. In the study of Donay et al. (1), 130 isolates were tested, with discordant results obtained for 7 isolates, for an overall concordance rate of 94.6% (1). The overall concordance rate in the study of Stefaniuk et al. (8) was 92.5% (for 120 isolates tested). Six discrepancies were at the species level and three were at the genus level (8). Eigner et al. (2) compared both the Phoenix and the VITEK 2 systems. Four discrepant results were seen with the Phoenix system (concordance rate, 96%) and two were seen with the VITEK 2 system (concordance rate, 98%). No particular patterns in terms of misidentification of a particular genus were obvious from those studies (1, 2, 8). Most of those studies, however, were somewhat limited by the small numbers of isolates tested and the lack of inclusion of diverse species. In addition, unlike our study and that of O'Hara (6), the comparative methods were other commercial systems.

O'Hara (6) challenged the Phoenix system with 507 archived biochemically typical and atypical enteric gram-negative bacilli previously characterized by reference methods with 48 biochemical substrates (6). In that comprehensive evaluation, the Phoenix system did not perform well. Only 89.9% (456/507) of the isolates were correctly identified to the genus and the species levels (6). An additional 20 isolates (3.9%) were identified to the genus level, whereas 29 isolates were incorrectly identified (6). Many of the errors were concentrated in the genus Salmonella (6). Eight of 35 Salmonella strains were misidentified as other genera (6). The two strains each of Salmonella enterica serovars Paratyphi A and Typhi were correctly identified (6). Our study design represents a more balanced approach between the extremes of the two approaches described above (1, 2, 6, 8); that is, we prospectively tested a broader range of clinical isolates likely to be encountered in a clinical microbiology laboratory supplemented with archived strains and used classical biochemical testing as the reference method.

In our evaluation of susceptibility testing comparisons, EA and CA were excellent for all antimicrobials except tobramycin, for which the EA was slightly lower (91.6%) than those for the remaining antimicrobial agents. There was one VME with tetracycline, and there were six MEs (one each with tobramycin and cefepime and two each with ampicillin and trimethoprim-sulfamethoxazole). Most of the errors were minor. Six organisms were ESBL producers, all of which were correctly flagged by the Phoenix system expert rules.

More has been published on the performance of the Phoenix system for susceptibility testing than for identification, especially for the gram-negative bacilli.

Compared to agar dilution, Stefaniuk et al. (8) observed excellent agreement with one VME for the Phoenix system when they tested isolates of the Enterobacteriaceae. Donay et al. (1) compared the Phoenix system to disk diffusion susceptibility testing. An unacceptably high rate of mEs was observed when E. coli was tested with amoxicillin-clavulanic acid (19%) and cephalothin (21%). The authors concluded that this was due to the ability of broth medium to better detect low-level expression of cephalosporinase from E. coli (1).

Three studies to date have extensively evaluated the ability of the Phoenix system to detect ESBLs. In the study by Leverstein-van Hall et al. (4), the Phoenix system was compared to the VITEK 1 and the VITEK 2 systems in an evaluation of 74 clinical isolates of ESBL-producing E. coli and Klebsiella spp. Etest was the comparative method. In addition, 17 genotypically characterized control strains were also included. The accuracies of the methods for the detection of ESBLs were 78%, 83%, and 89% for the VITEK 1, VITEK 2, and Phoenix instruments, respectively (4). Sturenburg et al. (9) likewise compared the VITEK 2 system and the Phoenix system using 34 ESBL-producing challenge strains. The Phoenix system was 100% sensitive compared to the 85% detection rate observed for the VITEK 2. Sanguinetti et al. (7), using both phenotypic and genotypic methods, found that the Phoenix system had 100% sensitivity and 98.9% specificity for ESBL detection among 11 species of Enterobacteriaceae. In the last two studies, the Phoenix system gave false-positive results for K1-hyperproducing isolates of Klebsiella oxytoca (7, 9). Although the study of Sanguinetti et al. (7) demonstrated that the Phoenix system has an excellent ability to detect ESBLs among 76 Proteus mirabilis isolates, the Phoenix system has not been cleared by the Food and Drug Administration for testing of this organism for ESBLs.

In summary, the Phoenix system gave an acceptable performance for the identification and susceptibility testing of 31 commonly encountered species of Enterobacteriaceae in our clinical laboratory. However, at least one published paper to date has reported the system has a poor performance when a combination of unusual strains and those most likely to be encountered in clinical laboratories are tested. As is true for any automated system, users should be aware of the potential for incorrect results and communicate such aberrancies to the manufacturer.