Two-Center Collaborative Evaluation of Performance of the BD Phoenix Automated Microbiology System for Identification and Antimicrobial Susceptibility Testing of Gram-Negative Bacteria

Abstract

The performance of the BD Phoenix Automated Microbiology System (BD Diagnostic Systems, Sparks, MD) was assessed for identification (ID) and antimicrobial susceptibility testing (AST) of the majority of clinically encountered bacterial isolates in a European collaborative two-center trial. A total of 494 bacterial isolates including various species of the Enterobacteriaceae and 110 nonfermentative gram-negative bacteria were investigated: of these, 385 were single patient isolates, and 109 were challenge strains tested at one center. The performance of the Phoenix extended-spectrum β-lactamase (ESBL) test was also evaluated for 203 strains of Escherichia coli, Klebsiella pneumoniae, and Klebsiella oxytoca included in the study. Forty-two antimicrobial drugs were tested, including members of the following drug classes: aminoglycosides, β-lactam antibiotics, β-lactam/β-lactamase inhibitors, carbapenems, cephems, monobactams, folate antagonists, quinolones, and others. Phoenix system ID results were compared to those of the laboratories' routine ID systems (API 20E and API CHE, ATB ID32E, ID32GN, and VITEK 2 [bioMérieux, Marcy l'Etoile, France]); Phoenix AST results were compared to those of frozen standard broth microdilution (SBM) panels according to NCCLS (now CLSI) guidelines (NCCLS document M100-S9, approved standard M7-A4). Discrepant results were repeated in duplicate. Concordant IDs of 98.4 and 99.1% were observed for the Enterobacteriaceae and the nonfermentative group, respectively. For AST results, the overall essential agreement was 94.2%; the category agreement was 97.3%; and the very major error rate, major error rate, and minor error rate were 1.6, 0.6, and 1.9%, respectively. In terms of ESBL detection, Phoenix results were 98.5% concordant with those of the reference system, with 98.0% sensitivity and 98.7% specificity. In conclusion, the Phoenix ID results showed high agreement with results of the systems to which they were being compared: the AST performance was highly equivalent to that of the SBM reference method, and the system proved to be very accurate for the detection of ESBL producers.

Clinical microbiologists are greatly concerned about providing rapid and accurate laboratory data for the diagnosis and treatment of infectious diseases and, moreover, for the control of nosocomial infections. These are caused by multiresistant gram-positive and gram-negative (GN) bacteria, particularly those producing β-lactamases (extended spectrum), or by strains of Pseudomonas aeruginosa and Acinetobacter spp.

The Phoenix automated microbiology system (BD Diagnostic Systems, Sparks, MD) has been developed to provide rapid, reliable, and accurate bacterial identification (ID) and antimicrobial susceptibility test (AST) results for the majority of clinically encountered species.

We investigated the ability of the Phoenix system to accurately perform ID and AST of clinical and challenge isolates in a large two-center trial involving the Section of Microbiology of the Department of Pathology and Laboratory Medicine, University of Parma, Parma, Italy, and the Laboratory Group Heidelberg, Heidelberg, Germany. In this study, gram-negative bacteria were evaluated in comparison to the routine laboratory ID method (API 20E and API CHE, ATB ID32E, ID32GN, and VITEK 2) and to the standard broth microdilution (SBM) procedure for AST according to NCCLS (now CLSI) guidelines (23).

(These findings were partly presented at the 11th European Congress of Clinical Microbiology and Infectious Diseases, Istanbul, Turkey, 1 to 4 April 2001 [abstr. P.1515], and at the 12th European Congress of Clinical Microbiology and Infectious Diseases, Milan, Italy, 21 to 24 April 2002 [abstr. P.1447].)

MATERIALS AND METHODS

The system is comprised of disposable panels, which combine ID and AST, and of an instrument that performs automatic reading at 20-min intervals for the entire duration of the incubation, which takes a maximum of 12 h to obtain an ID and 16 h to complete the AST.

The clinical trial was based on three different phases: proficiency, reproducibility, and accuracy.

Proficiency.

Each technician was required to simultaneously set up 20 strains (provided by the manufacturer) in both the Phoenix and reference AST systems. Proficiency testing was successful if correct results were obtained for 90% or more of the tests performed.

Reproducibility.

The reproducibility phase of the study was performed at one center (Heidelberg). Fifteen strains (including the NCCLS-recommended quality control [QC] strains) provided by the manufacturer were set up on three different days in triplicate in the Phoenix system only. Results were evaluated to determine variability of repeat AST testing. A modal MIC result for each strain and each drug combination and the frequency of MICs within ±1 dilution of this mode were determined to express the reproducibility.

Accuracy. (i) Bacterial isolates.

A total of 494 isolates were investigated, which included 384 strains of various species of the family Enterobacteriaceae and 110 nonfermenting gram-negative rods (NFGNRs). The following species were included: Escherichia coli (102 isolates), Klebsiella pneumoniae (69 isolates), Klebsiella oxytoca (32 isolates), Enterobacter cloacae (37 isolates), Enterobacter aerogenes (14 isolates), Citrobacter freundii (17 isolates), Citrobacter koseri (11 isolates), Citrobacter braakii (1 isolate), Citrobacter youngae (1 isolate), Serratia marcescens (19 isolates), Serratia liquefaciens (3 isolates), Serratia odorifera (1 isolate), Serratia sp. (1 isolate), Hafnia alvei (3 isolates), Proteus mirabilis (34 isolates), Proteus vulgaris (8 isolates), Proteus penneri (2 isolates), Providencia stuartii (4 isolates), Providencia rettgeri (2 isolates), Providencia alcalifaciens (1 isolate), Morganella morganii (13 isolates), Salmonella spp. (2 isolates), Salmonella enterica serovar Typhimurium (1 isolate), Shigella sonnei (2 isolates), Pantoea agglomerans (1 isolate), Leclercia adecarboxylata (2 isolates), Rhanella aquatilis (1 isolate), Pseudomonas aeruginosa (80 isolates), Pseudomonas putida (1 isolate), Stenotrophomonas maltophilia (13 isolates), Burkholderia cepacia (1 isolate), Burkholderia sp. (1 isolate), Acinetobacter baumanii (9 isolates), Acinetobacter sp. (1 isolate), Achromobacter xylosoxidans (1 isolate), Aeromonas hydrophila group (1 isolate), Aeromonas veronii (1 isolate), and Alcaligenes faecalis (1 isolate). Out of these strains, 385 were single patient isolates (188 from Parma and 197 from Heidelberg) and 109 were challenge strains supplied to one of the sites (Parma) by the manufacturer. The challenge set included strains from various sources with well-defined resistance mechanisms (Centers for Disease Control and Prevention, French National Reference Center [SFM], and BD Diagnostic Systems internal collection).

(ii) Phoenix system ID.

The Phoenix panels used were the ID and AST combination type (combo panel), with the identification substrates on the left-hand side of the panel and antimicrobial drugs on the right-hand side. The ID side of the panel for gram-negative bacteria contained a total of 45 dried substrates, including 14 fluorogenic substrates, 16 fermentation substrates, 7 carbon source substrates, 4 chromogenic substrates, esculin, urea, colistin, and polymyxin B, as well as 2 fluorescent controls. Isolates were subcultured twice onto Trypticase soy agar supplemented with 5% sheep blood (TSA II; BD Diagnostic Systems) to ensure viability and purity. The Phoenix ID broth was inoculated with bacterial colonies from a pure culture adjusted to a 0.5 to 0.6 McFarland standard using a CrystalSpec nephelometer (BD Diagnostic Systems), and the suspension was poured into the ID side of the Phoenix panel. Once inoculated, the panel was logged and loaded into the instrument, where kinetic measurements of colorimetric and fluorescent signals were collected every 20 min.

(iii) Reference ID.

The laboratories' routine ID system was set up from the same agar pure culture. At the Section of Microbiology, University of Parma, the gram-negative isolates were tested with the VITEK 2 system (ID-GNB [bioMérieux, Marcy l'Etoile, France]) and with ATB Expression (ID32E and ID32GN [bioMérieux]). At the Laboratory Group Heidelberg, the API system (API 20E or API CHE [bioMérieux]) and ATB were used as comparator ID systems.

(iv) Antimicrobials.

In total, 42 drugs were tested, including members of the drug classes aminoglycosides (5 drugs), β-lactam antibiotics (3 drugs), β-lactam/β-lactamase inhibitors (3 drugs), carbapenems (2 drugs), cephems (15 drugs), monobactams (1 drug), folate antagonists (2 drugs), and quinolones (8 drugs), as well as tetracycline, chloramphenicol, and nitrofurantoin.

NCCLS breakpoints were utilized for all antimicrobial agents except moxifloxacin, which was evaluated using breakpoints suggested by the pharmaceutical's manufacturer (Bayer, Leverkusen, Germany [package insert]).

(v) Phoenix AST.

The AST broth was supplemented with one drop of AST indicator (oxidation-reduction indicator based on resazurin). From the standardized ID suspension, 25 μl was transferred to the AST broth, resulting in a final inoculum density of approximately 5 × 10⁵ CFU/ml. The broth was poured into the AST side of the panel. The combo panel (CT14N) and the three additional AST-only panels (CT21N, CT22N, and CT23N), used to test a total of 42 antibiotics, were sealed, logged, and loaded into the Phoenix instrument. For each antibiotic, a minimum of six concentrations in doubling dilutions were tested with the Phoenix system.

Each Phoenix panel contained an extended-spectrum β-lactamase (ESBL) test which is composed of five single wells each containing a cephalosporin alone or in combination with clavulanic acid. The antibiotic wells include cefpodoxime, ceftazidime, ceftazidime with clavulanic acid, ceftriaxone with clavulanic acid, and cefotaxime with clavulanic acid.

Following incubation and automatic measurement of growth, individual antibiotic MICs were determined and interpreted applying NCCLS or pharmaceutical company breakpoints as elsewhere specified. The results of the ESBL wells were translated into a positive or negative ESBL result; therefore, for each strain, three ESBL test results were obtained. MICs and ESBL test results were based on internal algorithms in the Phoenix software.

(vi) Reference AST.

The frozen SBM panels (four panels) used in the reference method contained the same antimicrobial agents in doubling dilutions as the Phoenix panels, including antibiotics for the ESBL confirmatory test. The reference panels were prepared and tested according to NCCLS standards (23).

QC.

For quality control, 11 ATCC strains were tested for each run at each site (Parma and Heidelberg), resulting in a total of 21 Phoenix and 28 reference panels: Enterococcus faecalis ATCC 29212; Staphylococcus aureus ATCC 29213; Klebsiella pneumoniae ATCC 13883; Enterobacter cloacae ATCC 11061; Escherichia coli ATCC 35218 and ATCC 25922; Serratia rubidaea ATCC 33670; Klebsiella pneumoniae ATCC 700603; and Pseudomonas aeruginosa ATCC 35032, ATCC 27853, and ATCC 11052.

Results for QC strains had to be in control in the Phoenix system, as defined by a U.S. Food and Drug Administration guidance document for AST devices (5, 23).

Data analysis and management.

The Phoenix and reference data were entered into a Microsoft SQL Server 7.0 (SQL Server) database (Microsoft Corporation, Redmond, WA).

Application of NCCLS or pharmaceutical company breakpoints and associated rule recommendations and sensitive (S), intermediate (I), and resistant (R) interpretations were determined electronically in the database for both the reference data and the Phoenix data, ensuring that the same rules were applied for each data set. The Phoenix ID was used in the interpretation of all AST results obtained by both the Phoenix system and the SBM method for each respective isolate.

All AST accuracy reports were generated using SAS software, version 8.0 (SAS Institute, Cary, NC). For each drug, the following measures of accuracy were used: essential agreement (EA), or MICs between systems being within ±1 doubling dilution; and category agreement (CA), or S, I, and R interpretative results matching between the two systems. Errors were classified as very major error (VME), or a false-susceptible Phoenix result; major error (ME), or a false-resistant Phoenix result; and minor error (mE), in which the result in one system was intermediate and the other was susceptible or resistant. In calculating the error rates, the following denominators were used: the number of reference resistant isolates for the VME rate, the number of reference susceptible isolates for the ME rate, and the total number of tests for the mE rate.

Discrepancy resolution.

Those strains which showed ID discrepancies and/or VMEs and MEs in the susceptibility test were repeated in duplicate in both the Phoenix system and the reference method, obtaining three results for each type of test. A majority rule determined the final resolved outcome. For any remaining ID discrepancy, two comparator methods (ATB and the VITEK 2 [bioMérieux]) were set up. In detecting ESBL producers, both tests were also repeated if any discrepancies occurred between the two. In case of discrepancy between the ESBL result obtained with one of the AST panels and the reference method, testing was repeated with both the reference method and the specific panel.

RESULTS

Proficiency.

All laboratory personnel of both centers involved in this study passed the proficiency phase.

Reproducibility.

Of a total of 1,890 single tests, defined as one drug per isolate, reproducibility testing within the expected MIC range showed correct results in 98.0%. Reproducibility within the S, I, and R categories showed correct results in 96.2%.

Identification.

Out of 494 strains tested, a concordant ID to the species level was obtained in 98.6%. The various species of Enterobacteriaceae and of NFGNRs showed concordant species results of 98.4% and 99.1%, respectively. Five strains belonging to Enterobacteriaceae (C. freundii, E. cloacae, K. oxytoca, P. stuartii, and E. coli) and 1 NFGNR (B. cepacia) showed a discordant ID to the respective comparator method; three out of these six strains (P. stuartii, E. coli, and B. cepacia) were challenge isolates (Table 1). These three challenge strains were unidentified or misidentified by the VITEK 2 system and the Phoenix system, respectively, while the ATB system indicated the correct species among two or more species, but with a low level of discrimination. The Phoenix system yielded no identification for one strain of Citrobacter freundii.

TABLE 1.

ID results for gram-negative bacilli

Group	No. tested		No. (%) of ID results:			Phoenix ID (n = 1)	Comparator system ID
	Species	Isolates	Concordant	Discordant	No ID		ATB	VITEK 2
Enterobacteriaceae	27	384	378 (98.4)	5 (1.3)	1 (0.3)	Escherichia coli	C. freundii	C. freundii
						Citrobacter freundii	E. cloacae	E. cloacae
						Enterobacter cloacae	K. oxytoca	K. oxytoca
						Morganella morganii	P. stuartiia	No ID
						Shigella boydii	E. colia	No ID
						No ID	C. freundii	C. freundii
Nonfermentersb	11	110	109 (99.1)	1 (0.9)		Pasteurella multocida	B. cepaciaa	No ID
Total	38	494	487 (98.6)	6 (1.2)	1 (0.2)

^aATB ID had an unacceptable profile or insufficient discrimination.

^bIncluding one fermentative oxidase-positive strain of Aeromonas sp.

Overall AST results.

Single drug failures during QC tests were found in 2.1%. A total of 20,747 single AST results were evaluated for enteric bacteria (17,804 results) and NFGNRs (2,943 results), respectively. The evaluation included 13,717 clinical and 4,087 challenge single test results for enteric bacteria and 2,368 clinical and 575 challenge single test results for NFGNRs. The AST results of all drug classes combined for Heidelberg and Parma are shown in Table 2. The overall EA was 94.2%, the CA was 97.3%, the VME rate was 1.6%, the ME rate was 0.6%, and, finally, the mE rate was 1.9%. For enteric bacteria and NFGNRs, the EAs were 94.4% and 93.2%, the CAs were 97.6% and 95.5%, the VME rates were 1.8% and 0.9%, the ME rates were 0.5% and 1.7%, and the mE rates were 1.7% and 3.3%, respectively.

TABLE 2.

Susceptibility test results for Enterobacteriaceae and nonfermenters from Heidelberg and Parma

Group and center	No. of isolates tested	No. of single tests	No. of single test results in interpretive category		% of results with:
			I	R	EA	CA	mE	ME	VME
Enterobacteriaceae
Parma	224	10,330	419	2,863	95.0	98.2	1.4	0.1	1.7
Heidelberg	160	7,474	311	1,683	93.9	97.0	1.4	0.8	1.8
Total	384	17,804	730	4,546	94.4	97.6	1.7	0.5	1.8
Nonfermentersa
Parma	73	1,972	186	971	96.9	97.8	1.7	0.7	0.6
Heidelberg	37	971	127	308	88.0	92.3	5.6	3.0	1.6
Total	110	2,943	313	1,279	93.2	95.5	3.3	1.7	0.9
All
Parma	297	12,302	605	3,834	95.4	98.1	1.4	0.2	1.4
Heidelberg	197	8,445	438	1,991	93.2	96.5	2.4	1.0	1.8
Total	494	20,747	1,043	5,825	94.2	97.3	1.9	0.6	1.6

^aIncluding one fermentative oxidase-positive strain of Aeromonas sp.

Enterobacteriaceae AST results.

Table 3 shows the results for enteric bacteria of the individual drugs for both centers. Excluded from this table are the drugs for which there are no breakpoints available in either NCCLS (now CLSI), DIN (Deutsches Institut für Normung), or CA-SFM (Comité de l'Antibiogramme de la Société Française de Microbiologie). For the penicillins group (ampicillin, piperacillin, and ticarcillin), the results for EA were greater than 91%, except for piperacillin (86.2%), but overall CA was above 94%. VMEs were only found for ampicillin and ticarcillin (1.7% and 0.4%, respectively); the ME rates in this drug class ranged from 1.4% to 6.8%.

TABLE 3.

Susceptibility test results for Enterobacteriaceae

Drug	No. of isolates	Breakpoint standard	No. of isolates in interpretive category			% of results with:
			S	I	R	EA	CA	mE	ME	VME
Quinolones
Ciprofloxacin	382	NCCLS	320	14	48	99.2	99.5	0.5	0.0	0.0
Gatifloxacin	382	NCCLS	328	13	41	97.9	99.0	1.0	0.0	0.0
Levofloxacin	382	NCCLS	332	12	38	99.2	99.7	0.3	0.0	0.0
Lomefloxacin	382	NCCLS	305	18	59	99.2	99.2	0.8	0.0	0.0
Moxifloxacin	370	Pharma	307	16	47	96.2	99.2	0.8	0.0	0.0
Norfloxacin	381	NCCLS	325	16	40	98.4	99.7	0.3	0.0	0.0
Ofloxacin	382	NCCLS	321	13	48	99.2	99.7	0.3	0.0	0.0
Nalidixic acid	379	NCCLS	277		102	96.0	100.0	0.0	0.0	0.0
Aminoglycosides
Amikacin	382	NCCLS	372	6	4	94.0	99.5	0.5	0.0	0.0
Gentamicin	382	NCCLS	328	10	44	97.4	100.0	0.0	0.0	0.0
Kanamycin	363	NCCLS	297	6	60	97.0	100.0	0.0	0.0	0.0
Netilmicin	381	NCCLS	345	8	28	95.3	99.0	1.0	0.0	0.0
Tobramycin	382	NCCLS	323	18	41	96.3	99.2	0.8	0.0	0.0
β-Lactam penicillins
Ampicillin	382	NCCLS	74	11	297	91.2	93.2	4.2	6.8	1.7
Piperacillin	383	NCCLS	206	29	148	86.2	92.7	6.5	1.5	0.0
Ticarcillin	383	NCCLS	139	21	223	91.9	96.3	2.9	1.4	0.4
β-Lactam/β-lactamase inhibitors
Amoxicillin/clavulanate	382	NCCLS	186	24	172	95.2	96.6	2.4	1.1	1.2
Ampicillin/sulbactam	382	NCCLS	146	58	178	90.6	91.6	6.3	2.1	2.8
Piperacillin/tazobactam	381	NCCLS	286	12	83	90.4	96.3	2.9	0.3	2.4
Carbapenems
Imipenem	379	NCCLS	361	13	5	87.6	98.7	1.3	0.0	0.0
Meropenem	377	NCCLS	373	1	3	96.0	99.5	0.5	0.0	0.0
Cephems
Cefazolin	379	NCCLS	167	12	200	93.3	94.7	2.9	1.2	3.5
Cefdinir	382	NCCLS	208	15	159	89.7	93.2	3.9	3.4	2.5
Cefepime	381	NCCLS	325	2	54	92.7	98.7	0.3	0.6	3.7
Cefmetazole	324	NCCLS	222	12	90	92.2	95.7	2.2	1.4	4.4
Cefoperazone	379	NCCLS	257	6	116	91.5	96.6	2.1	1.6	0.9
Cefotaxime	380	NCCLS	295	11	74	96.4	97.6	1.8	0.0	2.7
Cefotetan	381	NCCLS	291	6	84	93.1	96.6	1.8	0.0	7.1
Cefoxitin	382	NCCLS	219	24	139	92.7	96.1	2.4	1.4	2.2
Cefpodoxime	378	NCCLS	241	27	110	89.3	94.4	4.2	0.0	4.5
Ceftazidime	382	NCCLS	298	5	79	95.8	98.7	1.0	0.0	1.3
Ceftibuten	371	NCCLS	282	3	86	89.4	97.3	1.9	0.4	2.3
Ceftizoxime	377	NCCLS	291	4	82	97.2	98.7	0.5	0.7	1.2
Ceftriaxone	382	NCCLS	290	12	80	94.9	96.1	2.6	0.3	5.0
Cefuroxime	381	NCCLS	213	22	146	94.2	97.4	1.0	1.9	1.4
Cephalothin	381	NCCLS	105	44	232	95.4	97.9	0.8	1.0	1.7
Monobactams
Aztreonam	382	NCCLS	299	6	77	97.3	97.9	1.0	0.3	3.9
Folate antagonists
Trimethoprim	382	NCCLS	290		92	91.9	99.0	0.0	0.0	4.3
Trimethoprim/sulfamethoxazole	382	NCCLS	299		83	97.1	97.6	0.0	1.7	4.8
Others
Tetracycline	379	NCCLS	217	26	136	94.5	97.9	1.6	0.5	0.7
Chloramphenicol	382	NCCLS	297	17	68	94.5	96.6	2.6	1.0	0.0
Nitrofurantoin	377	NCCLS	197	97	83	96.6	98.1	1.6	0.5	0.0

^aPharm, pharmaceutical company.

For the β-lactam/β-lactamase inhibitor combinations, the three formulations tested showed EAs from 90.4% to 95.2%, while the CAs ranged from 91.6% to 96.6%. The VME rates ranged from 1.2% (amoxicillin/clavulanic acid) to 2.8% (ampicillin/sulbactam); ME rates ranged from 0.3% to 2.1%.

For carbapenems, the combined EA and CA were 91.8% and 99.1%, respectively. Only minor errors have been observed (0.9%).

For the 15 cephems reported in this table, the EAs ranged from 91.5% to 97.2%, except for cefdinir (89.7%), ceftibuten (89.4%), and cefpodoxime (89.3%), and the CAs ranged from 93.2% to 98.7%. VME rates ranged from 0.9% to 3.7% for 11 cephems, except for cefotetan (7.1%), cefmetazole (4.4%), cefpodoxime (4.5%), and ceftriaxone (5.0%).

Among the aminoglycosides, the EA was equal to or higher than 94.0% and the CAs ranged from 99.2% to 100.0%. No VMEs and MEs were found.

The EA rate of the quinolones showed a distribution from 96.0% to 99.2%, the CAs ranged from 99.0% to 100.0%, and no VMEs and MEs were found.

Nonfermenting gram-negative rod AST results.

The results for Pseudomonas spp. and other NFGNRs are shown in Table 4 and 5, respectively. Excluded from these tables are the drugs for which there are no breakpoints available in either NCCLS (now CLSI), DIN, or SFM.

TABLE 4.

Susceptibility test results for Pseudomonas spp.^a

Drug	No. of isolates	No. of isolates in interpretive category			% of results with:
		S	I	R	EA	CA	mE	ME	VME
Fluoroquinolones
Ciprofloxacin	78	53	4	21	97.4	98.7	1.3	0.0	0.0
Levofloxacin	74	47	4	23	91.9	96.0	4.1	0.0	0.0
Lomefloxacin	78	37	11	30	96.2	98.7	1.3	0.0	0.0
Norfloxacin	78	53	6	19	96.2	97.4	2.6	0.0	0.0
Ofloxacin	77	45	5	27	97.4	98.7	1.3	0.0	0.0
Aminoglycosides
Amikacin	79	69	1	9	91.1	96.2	2.5	0.0	11.1
Gentamicin	79	53	9	17	100.0	100.0	0.0	0.0	0.0
Netilmicin	78	57	6	15	98.7	100.0	0.0	0.0	0.0
Tobramycin	79	62		17	97.5	100.0	0.0	0.0	0.0
β-Lactam penicillins
Piperacillin	79	57		22	92.3	96.2	0.0	1.8	9.1
Ticarcillin	79	56		23	93.7	98.7	0.0	1.8	0.0
β-Lactam/β-lactamase inhibitors
Piperacillin/tazobactam	79	61		18	92.3	97.5	0.0	0.0	11.1
Carbapenems
Imipenem	77	60	6	11	93.5	94.8	5.2	0.0	0.0
Meropenem	77	67	7	3	90.9	98.7	1.3	0.0	0.0
Cephems
Cefepime	78	55	14	9	91.0	97.4	2.6	0.0	0.0
Cefoperazone	78	46	10	22	96.2	96.2	2.6	2.2	0.0
Cefotaxime	79	5	38	36	91.1	91.1	7.6	20.0	0.0
Ceftazidime	77	58	7	12	94.8	97.4	2.6	0.0	0.0
Ceftizoxime	78	3	25	50	91.0	91.0	7.7	33.3	0.0
Ceftriaxone	79	17	29	33	87.3	88.6	10.1	5.9	0.0
Monobactams
Aztreonam	78	51	8	19	91.0	91.0	7.7	2.0	0.0

^aThe NCCLS breakpoint standard was used for each drug.

TABLE 5.

Susceptibility test results for nonfermentative bacteria other than Pseudomonas spp.^a

Drug	No. of isolates	No. of isolates in interpretive category:			% of results with:
		S	I	R	EA	CA	mE	ME	VME
Fluoroquinolones
Ciprofloxacin	30	8	3	19	96.7	100.0	0.0	0.0	0.0
Levofloxacin	30	15	1	14	90.0	96.7	3.3	0.0	0.0
Lomefloxacin	30	9	5	16	100.0	100.0	0.0	0.0	0.0
Norfloxacin	30	6	2	22	96.7	100.0	0.0	0.0	0.0
Ofloxacin	29	11	4	14	96.6	96.6	3.5	0.0	0.0
Aminoglycosides
Amikacin	29	12		17	89.7	89.7	6.9	0.0	5.9
Gentamicin	30	8	1	21	96.7	96.7	3.3	0.0	0.0
Netilmicin	30	8	1	21	96.7	96.7	0.0	0.0	4.8
Tobramycin	30	8	1	21	93.3	96.7	0.0	0.0	4.8
β-Lactam penicillins
Piperacillin	30	6	7	17	73.3	76.7	23.3	0.0	0.0
Ticarcillin	30	4	8	18	73.3	76.7	20.0	25.0	0.0
β-Lactam-β-lactamase inhibitors
Ampicillin/sulbactam	30	4	4	22	93.3	93.3	6.7	0.0	0.0
Piperacillin/tazobactam	30	10	6	14	76.7	80.0	16.7	10.0	0.0
Carbapenems
Imipenem	30	11		19	96.7	100.0	0.0	0.0	0.0
Meropenem	29	12	1	16	96.6	96.6	3.5	0.0	0.0
Cephems
Cefepime	30	10	5	15	73.3	93.3	6.0	0.0	0.0
Cefoperazone	30	8	2	20	93.3	93.3	3.3	12.5	0.0
Cefotaxime	30	5	3	22	100.0	100.0	0.0	0.0	0.0
Ceftazidime	29	11	3	15	82.7	82.8	13.8	9.1	0.0
Ceftizoxime	30	5	3	22	96.7	96.7	0.0	0.0	4.6
Ceftriaxone	30	6	2	22	90.0	96.7	3.3	0.0	0.0
Monobactams
Aztreonam	30	4	4	22	96.7	96.7	0.0	25.0	0.0
Folate antagonists
Trimethoprim/sulfamethoxazole	30	17		13	90.0	93.3	0.0	11.8	0.0
Others
Tetracycline	30	9	6	15	100.0	100.0	0.0	0.0	0.0
Chloramphenicol	30	13	2	15	96.7	96.7	0.0	7.7	0.0

^aGenera included Achromobacter, Acinetobacter, Alcaligenes, Burkholderia, Stenotrophomonas, and one fermentative oxidase-positive strain of Aeromonas sp. The NCCLS breakpoint was used for each drug.

For Pseudomonas spp. (Table 4), the results for the β-lactam penicillins group (piperacillin and ticarcillin) gave an EA equal to or higher than 92.3%, but the overall CA was greater than 97.0%. Only with piperacillin were VMEs observed at a rate of 9.1%. MEs occurred at a rate of 1.8% for piperacillin and ticarcillin.

For the β-lactam/β-lactamase inhibitor combinations, an EA of 92.3% and a CA of 97.5% for piperacillin/tazobactam were seen. For piperacillin/tazobactam, two VMEs were observed. For imipenem and meropenem, the EAs were 93.5% and 90.9%, while the CAs were 94.8% and 98.7%, respectively, and for both drugs, no MEs or VMEs were observed.

For the six cephems reported in Table 4, the EAs ranged from 87.3% to 96.2% and the CAs ranged from 88.6% to 97.4%. No VMEs were observed, while the overall ME rates covered a wide range on a percentage basis from 0.0% (cefepime and ceftazidime) to 33.3% (ceftizoxime). However, the higher rates resulted from a single error with only a few susceptible strains tested.

Among the aminoglycosides, the EA rate was higher than 97.0% except for amikacin (91.1%). Similarly, the CA was 100.0% for all drugs in this class, except 96.2% for amikacin. There was one VME (1.72%) observed in this antimicrobial class. No MEs were observed with aminoglycosides. The EA and CA rates of the fluoroquinolones showed distributions from 91.9% to 97.4% and from 96.0% to 98.7%, respectively. There were no VMEs and MEs found in this drug class.

For the nonfermentative bacteria other than Pseudomonas spp. (Table 5) for both of the β-lactam penicillins tested (piperacillin and ticarcillin), the results for EA and CA were equal to 73.3% and 76.7%, respectively. There were no VMEs, and only one ME among very few strains susceptible to ticarcillin was observed.

With regards to β-lactam/β-lactamase inhibitor combinations, both EA and CA values were 93.3% for ampicillin/sulbactam, while the EA and CA for piperacillin/tazobactam were 76.7% and 80.0%, respectively. For piperacillin/tazobactam, one ME was found. For imipenem and meropenem, the EA was higher than 96.0%, while the CAs were 100.0% and 96.6%, respectively, and for both no MEs or VMEs were observed.

For the six cephems reported in this table, the EAs ranged from 90.0% to 100.0%, except for 73.3% for cefepime and 82.7% for ceftazidime, while the CAs ranged from 93.3% to 100.0%, except for 82.8% for ceftazidime. One VME was observed for ceftizoxime (4.6%), and two MEs were observed for cefoperazone (12.5%) and ceftazidime (9.1%), respectively.

Among the aminoglycosides, the EA and CA rates ranged from 93.3% to 96.7%, except for 89.7% for amikacin. There were three VMEs observed, giving 3.7% in this antimicrobial class. No MEs were found with aminoglycosides.

The EA and CA rates of the fluoroquinolones ranged from 96.6% to 100%, except for the EA value of 90.0% for levofloxacin. No VMEs and MEs were found in this drug class.

The performance of the Phoenix ESBL test was also evaluated (Table 6) with 203 strains of E. coli, K. pneumoniae, and K. oxytoca included in the study. Fifty-six were challenge strains, containing well-characterized β-lactamases, and 147 were clinical isolates from both sites (Parma and Heidelberg). The Phoenix ESBL test and the reference system identified 50 strains as ESBL producers (32 challenge and 18 clinical isolates) and 151 strains as ESBL negative (22 challenge and 129 clinical isolates, respectively). There were two false-positive isolates (one clinical and one challenge) and one false-negative isolate by the Phoenix ESBL test, and the discrepancy between the two methods remained after repeating the tests. The discrepant strains were two E. coli challenge strains and one clinical Klebsiella oxytoca isolate. The comparison of Phoenix results with the reference system showed 98.0% sensitivity and 98.7% specificity and concordant results for 98.5% of isolates.

TABLE 6.

ESBL results for the E. coli and Klebsiella isolates tested in this study

Isolate type and species	No. of resultsa:					Result (%) forb:
	Total	+ by both tests	PHX+ and REF−	PHX− and REF+	− by both tests	Sensitivity	Specificity	PPV	NPV	Agreement
Clinical
E. coli	71	6	0	0	65	100	100	100	100	100
K. pneumoniae	49	7	0	0	42	100	100	100	100	100
K. oxytoca	27	5	1	0	21	100	95.5	83.3	100	96.3
Challenge
E. coli	31	12	1	1	17	92.3	94.4	92.3	94.4	93.5
K. pneumoniae	20	17	0	0	3	100	100	100	100	100
K. oxytoca	5	3	0	0	2	100	100	100	100	100
Clinical and challenge
E. coli	102	18	1	1	82	94.7	98.8	94.7	98.8	98.0
K. pneumoniae	69	24	0	0	45	100	100	100	100	100
K. oxytoca	32	8	1	0	23	100	95.8	88.9	100	96.9
Total	203	50	2	1	150	98.0	98.7	96.2	99.3	98.5

^aPHX, Phoenix system; REF, reference method. +, positive; −, negative.

^bPPV, positive predictive value; NPV, negative predictive value.

DISCUSSION

This two-center trial is focused on the performance of the Phoenix system with gram-negative bacilli. We have previously reported on the performance of the system with gram-positive cocci (11). Other papers have been published on the performance of the Phoenix system with gram-negative bacteria (3, 8, 10, 13, 22, 24, 28, 31), but our study is the only one that has investigated the performance of both the ID and the AST aspects of the Phoenix system using a large number of strains, antibiotics tested on a broad range of MICs (six concentrations in doubling dilutions), and also specific types of resistance mechanisms—the latter in a percentage which outnumbers that encountered in our routine investigations. We have compared the ID performance of the Phoenix system to commercially available ID methods (API 20E and API CHE, ATB ID32E, ID32GN, and VITEK 2). Six out of 494 gram-negative strains gave a discordant ID result (1.2%), and there was only 1 strain for which the Phoenix system could not determine an ID. The discordant ID results were on a percentage basis nearly equal in Enterobacteriaceae and nonfermenters and were randomly distributed between different genera.

Besides these six strains, eight isolates initially gave discordant results, which were resolved during the discrepancy resolution phase by repeating the IDs in duplicate with the Phoenix system and the comparator methods (ATB and VITEK 2). Among these, interestingly, three strains from Heidelberg gave discordant results, being identified as K. oxytoca by Phoenix and as K. pneumoniae by the ATB system. A report by Freney et al. (12) suggests that a few strains (1/18) of K. oxytoca may be misidentified as K. pneumoniae subsp. pneumoniae in the ATB 32E system if the malonate test well is uninterpretable and the supplemental indole test is interpreted as negative for a weakly positive strain. The ID of these three strains was resolved in favor of the Phoenix system since they were identified as K. oxytoca by the Phoenix system and VITEK 2. Moreover, two challenge isolates were correctly identified to the species level by the Phoenix system and by the VITEK 2 comparator system (one isolate each of K. oxytoca and Acinetobacter baumanii).

Also, one challenge strain of E. cloacae was identified as Enterobacter asburiae by the Phoenix system and by ATB, while VITEK 2 was unable to reach any identification. H. Hoffmann and A. Roggenkamp (15) have clearly demonstrated the genetic heterogeneity of the nomenspecies Enterobacter cloacae and, furthermore, that E. asburiae is closely related to it and is subsumed in the so-called E. cloacae complex. We decided therefore to consider this ID as concordant. Finally, one clinical strain tested in Italy was identified as Citrobacter braakii by both the Phoenix system and VITEK 2 and as Citrobacter freundii by ATB. However, Citrobacter braakii is part of the Citrobacter freundii complex (1, 2), and at the time of our trial, the species was not included in the ATB database and therefore was not considered as a discrepancy.

The Phoenix gram-negative identification system consists of 44 substrates, which allows discrimination of approximately 160 different taxonomic groups (i.e., species, subspecies, groups, etc.) of nonfastidious aerobic gram-negative bacteria without the need for a supplemental test. Our study did not fully challenge the entire spectrum of the identifications available in the Phoenix system. However, our test population was representative of 39 of the most commonly isolated taxonomic groups in our clinical microbiology laboratories. In a reported study, Salomon et al. demonstrated the Phoenix ID system's discriminatory ability with a battery of 1,250 strains including ∼75 fermentative and ∼50 nonfermentative gram-negative bacterial groups (J. Salomon, A. Butterworth, V. Almog, J. Pollit, W. Williams, and T. Dunk, Abstr. 9th Eur. Congr. Clin. Microbiol. Infect. Dis., abstr. P0824, 1999). Similarly, a follow-up study reported an overall accuracy of nearly 97% in an evaluation of including 793 strains distributed over a total of 93 species (V. White, K. Fishbein, T. Dunk, W. Williams, J. Reuben, and J. Solomon, Abstr. 101st Gen. Meet. Am. Soc. Microbiol., abstr. C339, 2001).

Other reports on the ID performance of the Phoenix system have been published. Stefaniuk et al. (28) observed a 96.0% accuracy for gram-negative nonfermenters and 92.5% for members of the Enterobacteriaceae family in a study comparing the Phoenix system to conventional identification methods using 174 gram-negative strains, including eight different species of Enterobacteriaceae and three different species of nonfermentative bacteria. Endimiani et al. (10) evaluated 136 strains of seven species of nonfermentative organisms and reported an overall 95.6% comparability with the ATB ID32GN test (bioMérieux, Marcy l'Etoile, France) and a cumulative agreement of 97.1% when two isolates, concordant at the genus level, were included. Brisse et al. (3) evaluated the Phoenix and VITEK 2 systems for the identification of 134 strains of Burkholderia cepacia or closely related nonfermentative bacteria using genetic/molecular typing as a reference. Although both systems performed unsatisfactorily with many of these very-difficult-to-identify organisms (50% to 53% with the Phoenix system and VITEK 2, respectively), the Phoenix system correctly identified 81.0% of isolates of the most clinically relevant strain, B. cepacia genomovar III.

All of this considered, the major thrust of this study was to evaluate the Phoenix system's accuracy and reliability compared to the NCCLS reference microdilution method. The results show that a large number of antimicrobial agents of different classes had an EA and CA of >90.0%. In particular for the eight quinolones tested in our study, the comparability of the Phoenix system to the reference microdilution method was extremely good, with a CA of >96.0% and no MEs or VMEs. For the Enterobacteriaceae, the mEs were ≤1%, but with Pseudomonas spp. or other nonfermenters, there were a few more mEs, which, however, did not exceed 4.0%. Steward et al. (30) in a comparison of MicroScan and VITEK to the broth microdilution method reported good performance with two quinolones, ciprofloxacin and ofloxacin, with 195 strains of Enterobacteriaceae. This study reported slightly higher rates of major errors in both systems and for both antibiotics (2.7%), with VITEK having a few (2.7%) VMEs with ciprofloxacin. Joyanes (17) reported very similar results with VITEK 2 and the quinolone levofloxacin with Pseudomonas aeruginosa and a variety of other nonfermentative gram-negative bacteria, with no MEs and 2.7% VMEs. It is uncertain whether these differences between other systems and the Phoenix system are significant or not. They may result from differences in the population of organisms tested.

The comparability of the Phoenix system to the broth microdilution method was also very good for Enterobacteriaceae with four aminoglycosides, tetracycline, chloramphenicol, and nitrofurantoin. However, with one aminoglycoside, amikacin, there were two unexpected VMEs observed with nonfermentative organisms. One VME was observed with a strain of P. aeruginosa, and the other was observed with a strain of Stenotrophomonas maltophilia. Endimiani et al. (10) evaluated the Phoenix system with a battery of nonfermenting gram-negative strains, including some highly resistant ones, and they reported no amikacin VMEs when testing a population of amikacin-resistant strains of comparable size. Other than the above discrepancies, the performance of the Phoenix system with aminoglycosides was excellent.

The level of performance with the β-lactams was a little lower, but comparable to those reported for other commercial methods (4, 6, 7, 8, 9, 16, 18, 25, 29, 31). Within the class, the Phoenix system showed excellent performance for the carbapenems, with no VMEs, no MEs, and very few mEs, ranging from 0.5 to 5.2%. This is in contrast with what was reported by Steward et al. (29), who observed a high rate of MEs when using other automated systems and suggested laboratories should consider a second independent antimicrobial susceptibility testing method to validate carbapenem intermediate-resistant results. On the other hand, our results are in accordance with those reported by R. Cantón et al. (4), U. Eigner et al. (9), and P. Giakkoupi et al. (14). On this matter, Pitout et al. have recently recommended that all imipenem-nonsusceptible P. aeruginosa isolates be routinely screened for metallo-β-lactamase production using the EDTA disk screening test and that confirmation by PCR be performed at a regional reference laboratory (25).

Considering the total number of ASTs performed, overall the CA for the β-lactams was 97.3% with 1.9, 0.6, and 1.6 mE, ME, and VME, respectively. In particular, the MEs and VMEs were observed for penicillins, cephems, a folate antagonist, and aztreonam and were almost equally distributed in the different groups of organisms tested. Among the nearly 380 strains of Enterobacteriaceae tested, our results showed VMEs ranging from 1.7 to 7.1%, but these values are referable to a very limited number of strains: two strains with piperacillin, piperacillin/tazobactam, cefotaxime, and cefepime; four strains with cefdinir, cefmetazole, ceftriaxone, trimethoprim, trimethoprim/sulfamethoxazole, and aztreonam; five strains with ampicillin, ampicilllin/sulbactam, and cefpodoxime; six strains with cefotetan; and seven strains with cefazolin. Moreover, MEs occurred only with three strains and ampicillin/sulbactam, with five isolates and ampicillin, and with seven strains and cefdinir. With regard to Pseudomonas spp. and nonfermentative bacteria other than Pseudomonas, VMEs were found for only two strains: one strain (P. aeruginosa) with piperacillin and piperacillin/tazobactam and the other (B. cepacia) with ceftizoxime. Moreover, MEs were observed for one isolate of Pseudomonas sp. and cefotaxime, ceftizoxime, and ceftriaxone and with one other nonfermenting gram-negative rod (S. maltophilia) and piperacillin/tazobactam, cefoperazone, ceftazidime, and aztreonam. A study by Cantón et al. (4) conducted in 52 centers to assess the quality of β-lactam AST using a challenge collection of strains revealed that cefepime, aztreonam, and piperacillin/tazobactam proved to be the antibiotics with the highest percentages of discrepancies, especially when tested against strains characterized by complex β-lactam-resistant phenotypes. In this respect, it is worth noting that a significant number of challenge strains (22.9% of the total tested) with well-defined resistance mechanisms to β-lactams was included in the bacterial population used for this study. Moreover, A. Stefaniuk et al. (28) reported a CA of 80.0% in S. maltophilia for trimethprim/sulfamethoxazole, while we obtained a CA of 93.3%. This difference in results could be explained either by the use of the agar dilution method, chosen by the authors as a reference system, to evaluate the MICs obtained by a broth microdilution method (Phoenix system) or by particular strains tested with MICs around the breakpoints. It has been reported that these discrepancies occurred when S. maltophilia strains have been tested by different methodologies with trimethprim/sulfamethoxazole, which is also characterized by trailing endpoints (B. Turng, V. Towns, H. Lilli, S. Wullff, and T. Wiles, Abstr. 9th Eur. Congr. Clin. Microbiol. Infect. Dis., abstr. LR640, 1999).

Finally, the overall Phoenix system performance for the detection of ESBLs in Enterobacteriaceae was very good. In fact, we observed a sensitivity of 98.0% and a specificity of 98.7%, which is in line with what was reported in other studies (19, 24, 27). Incorrect results were obtained for only 3 strains (out of 203 tested): two ESBL false positives and one false negative. The two false-positive strains consisted of one clinical isolate and one challenge strain. It is of note that one of these two strains was a K. oxytoca isolate which showed elevated MICs for ceftriaxone, cefuroxime, and aztreonam and was resistant to the inhibitor combinations, but was not resistant to ceftazidime. According to Livermore et al. (21), this is a typical phenotype of a K. oxytoca strain hyperproducing the chromosomal K1 enzyme. Other studies have already reported such types of misidentifications with the Phoenix system (27, 31), but this is not exclusive of this automated system (20). In a recent study performed by the group of Livermore (26), it was shown that synergy between clavulanic acid and cefotaxime was observed for 20 of the 25 K. oxytoca isolates lacking ESBLs and hyperproducing K1, when MICs were determined by the agar dilution method and by Etests. The second false positive was a challenge strain known to harbor an OXA-1 enzyme, an enzyme poorly inhibited by clavulanic acid. In this case, only one out of the three AST panels tested (CT23N) incorrectly identified the strain as an ESBL producer, but this discrepancy was not resolved after the test had been repeated, and the reason for this incorrect result remains unclear. The third isolate for which we observed an incorrect result was also a challenge strain known to harbor an SHV-5 β-lactamase. In this case, initially all three AST panels failed to detect the isolate as ESBL positive, and a surprisingly low MIC of cefpodoxime (2 μg/ml) and a relatively low MIC of ceftazidime (16 μg/ml) were also observed with the reference AST method. After the tests were repeated, two of the three Phoenix AST panels correctly identified the strain as an ESBL producer, but the third panel (CT23N) failed to do so. Interestingly, the second set of results obtained with the reference method showed higher MICs of ceftazidime (64 μg/ml) and cefpodoxime (>8 μg/ml). Considering that SHV-5 is one of the most potent ESBLs conferring high resistance to the broad-spectrum cephalosporins, one could speculate that on the first run the strain failed to fully express the enzyme. It remains unclear, though, why the third panel failed to detect the ESBLs also on repeat testing when the isolate showed a more typical ESBL phenotype.

Our current experience suggests that some of the issues described above would have been prevented by the upgraded versions of the built-in rules-based expert system (BDXpert System).

In conclusion, our study demonstrates a satisfactory overall performance of the Phoenix system for ID and AST of a wide variety of bacterial species, such as those most commonly encountered in a microbiology laboratory of a university-based hospital. The system is very accurate in the detection of ESBLs, which are at present one of the most important resistance mechanisms.