Comparison of Phoenix and VITEK 2 Extended-Spectrum-β-Lactamase Detection Tests for Analysis of Escherichia coli and Klebsiella Isolates with Well-Characterized β-Lactamases

Abstract

The VITEK 2 and Phoenix extended-spectrum β-lactamase (ESBL) detection systems, which comprise confirmatory tests and expert systems, were evaluated for their ability to discriminate between 102 well-characterized strains of ESBL-positive or -negative Escherichia coli, Klebsiella pneumoniae, and Klebsiella oxytoca. At least 38 distinct ESBLs were included. The strains were chosen to include some known to cause false-positive and false-negative CLSI ESBL confirmatory test results. Therefore, enzyme characterizations, rather than CLSI tests, were the reference methods for the Phoenix and VITEK 2 evaluations. A third arm of the study was conducted with the Phoenix test using two normally inactive expert rules intended to enhance ESBL detection, in addition to using the currently available software. The Phoenix ESBL confirmatory test and unmodified expert system exhibited 96% sensitivity and 81% specificity for ESBL detection. Activation of the two additional rules increased sensitivity to 99% but reduced the specificity to 58%. The VITEK 2 ESBL confirmatory test exhibited 91% sensitivity, which was reduced to 89% sensitivity by its expert system, while its specificity was 85%. Many of the expert system interpretations of both instruments were helpful, but some were suboptimal. The VITEK 2 expert system was potentially more frustrating because it provided more inconclusive interpretations of the results. Considering the high degree of diagnostic difficulty posed by the strains, both ESBL confirmatory tests were highly sensitive. The expert systems of both instruments require modification to update and enhance their utility.

Extended-spectrum β-lactamases (ESBLs) hydrolyze and cause clinically significant resistance to many β-lactam antibiotics, especially the expanded-spectrum cephalosporins. They are usually associated with resistance to multiple, unrelated antibiotics, such as aminoglycosides, chloramphenicol, trimethoprim-sulfamethoxazole, tetracycline, and fluoroquinolones, leaving few therapeutic choices (5). β-Lactam resistance caused by ESBLs is not detected reliably by routine antibiotic susceptibility tests (13). Therefore, it is necessary to use special ESBL detection tests to avoid the risk of reporting false susceptibility to penicillins, cephalosporins, and aztreonam (4, 13). These tests should be assessed periodically for reductions in accuracy due to the changing epidemiology of ESBL- and other β-lactamase-producing pathogens. The tests may be compromised by new types of ESBLs and by the ability of other β-lactamases to phenotypically mask the presence of ESBLs or cause false-positive tests. For example, the accuracy of some ESBL detection tests has already been impacted by the advent of CTX-M ESBLs, “inhibitor-resistant” ESBLs, and class A carbapenemases; the hyperproduction of SHV-1 or K1 β- lactamases; and the coproduction of AmpC β-lactamases (8, 13, 15).

The VITEK 2 and Phoenix ESBL detection systems involving both an ESBL confirmatory test and an expert system have recently become available in the United States. So far, there is only one published comparison of the two instruments, a study in which both were equipped with expert system rules but lacked ESBL confirmatory tests (17). Because of the lack of a direct comparison of the ESBL confirmatory tests and expert software, a study was designed to evaluate the abilities of the two systems to discriminate between ESBL-positive and -negative strains of Escherichia coli, Klebsiella pneumoniae, and Klebsiella oxytoca.

MATERIALS AND METHODS

Strains and enzymes.

The test organisms comprised both laboratory strains and clinical isolates previously characterized by appropriate biochemical, molecular, and phenotypic procedures to determine their types of β-lactamase production (7). They were chosen to include types of β-lactamase production known to cause false-positive and false-negative Clinical and Laboratory Standards Institute (CLSI) ESBL confirmatory test results, such as high-level production of AmpC, SHV-1, K1 β-lactamases, and class A carbapenemases and the production of multiple β-lactamases (up to five enzymes) (7, 13, 16). Because the CLSI ESBL confirmatory tests were known to be unreliable for some of the organisms, the Phoenix and VITEK 2 results were compared to enzyme characterizations.

Seventy-six strains produced at least 38 distinct ESBLs that were present alone or in combination with other β-lactamases. The definitively identified ESBLs were TEM-3, TEM-4, TEM-5, TEM-6, TEM-7, TEM-8, TEM-9, TEM-10, TEM-12, TEM-15, TEM-16, TEM-24, TEM-26, TEM-28, TEM-43, TEM-47, TEM-50, TEM-52, TEM-61, SHV-2, SHV-3, SHV-4, SHV-5, SHV-6, SHV-7, SHV-9, SHV-10, SHV-12, CTX-M-9, CTX-M-14, CTX-M-16, CTX-M-17, CTX-M-18, CTX-M-19, CTX-M-44 (Toho-1), CTX-M-45 (Toho-2), Toho-3, and PER-1. Some additional ESBLs designated with the affix “like” (e.g., SHV-2-like) were identified by phenotypic, biochemical, and PCR tests, but the gene had not been sequenced.

The 26 ESBL-negative strains produced the enzymes TEM-1, TEM-2, SHV-1, LEN-1, K1, OXA-3, OXA-4, OXA-5, OXA-6, KPC-2, and five AmpC β-lactamases (CMY, DHA-1, MIR-1, MOX-1, and the E. coli chromosomal AmpC). These strains included hyperproducers, producers of multiple β-lactamases, and porin mutants.

Software and cards.

The available U.S. versions of the ESBL confirmatory tests and expert systems were used. The Phoenix card NMIC/ID-108 was run with software version 5.02H/V4.11B. The VITEK 2 card AST-GN13 was run with software version WSVT2-R04.01. In addition, two normally inactive Phoenix expert rules intended to enhance ESBL detection, based on antibiogram testing (rules 345 and 1437), were activated in a separate arm of the study.

Test sites.

Testing occurred at two sites, with the test inoculum for each strain prepared from the same plate culture on the same day. The Phoenix testing was done at Creighton University, and the VITEK 2 testing was performed at Methodist Hospital, Omaha, NE.

Repeated tests.

Tests with some strains were repeated (usually once and occasionally twice) for multiple reasons, including to determine whether errors were reproducible or because growth was insufficient, the test was not finalized, the expert system suggested repeat testing, the ESBL confirmatory test and expert system yielded discrepant results, or an ESBL appeared to have been lost from a strain prior to being tested.

RESULTS

The results were analyzed to assess the accuracy of the ESBL confirmatory tests and the ability of the expert systems to correctly modify or enhance the interpretation of the test results. The initial results were considered more important than those of repeated tests because they represented findings that the clinical laboratory would routinely utilize. The repeated tests included some results that were relevant only to the study and not clinically useful.

Overall performance.

On initial testing, the Phoenix ESBL confirmatory test and unmodified expert system exhibited 96% sensitivity and 81% specificity (see the footnote of Table 1). Activation of Phoenix expert rules 345 and 1437 increased the sensitivity to 99% but reduced the specificity to 54%. The VITEK 2 confirmatory test exhibited 91% sensitivity, which was reduced to 89% by its expert system because a TEM-26-producing strain of E. coli strain was interpreted as ESBL negative by the expert system. Both the VITEK 2 confirmatory test and its expert system exhibited 85% specificity. Both confirmatory tests detected the TEM-50 ESBL, which is poorly inhibited by clavulanate and less readily detected by clavulanate-based ESBL tests. The falsely negative and positive results for each instrument are summarized in Tables 2 and 3, respectively.

TABLE 1.

Overall results of the two systems^a

Systemb	No. (%) of tests that were correct for:
	76 ESBL-producing strains after initial tests		76 ESBL-producing strains after repeat tests		26 ESBL-negative strains after initial tests		26 ESBL-negative strains after repeat tests
	ESBL test	Expert system	ESBL test	Expert system	ESBL test	Expert system	ESBL test	Expert system
Phoenix	73 (96)	73 (96)	74 (97)	73 (96)	21 (81)	21 (81)	20 (77)	19 (73)
Phoenix*	73 (96)	75 (99)	74 (97)	75 (99)	21 (81)	15 (58)	20 (77)	14 (54)
VITEK 2	69 (91)	68 (89)	70 (92)	70 (92)	22 (85)	22 (85)	23 (88)	22 (85)

^aThe sensitivity of ESBL detection was the percentage of ESBL-positive strains detected accurately. The specificity was the percentage of ESBL-negative strains that yielded accurate results. False-positive rates were calculated by subtracting the specificity from 100.

^bPhoenix*, Phoenix system results after activation of two normally inactive Phoenix expert rules (rules 325 and 1437) intended to enhance ESBL detection, based on susceptibility results.

TABLE 2.

Falsely negative confirmatory test results

Organism	Strain	ESBL	β-Lactamase(s) detected		Confirmatory test result
			AmpC	Other	Phoenix	VITEK 2
E. coli	Misc358	TEM-28			−	−
	233	SHV-2-like		TEM-1, KPC-3	+	−
	Misc289	SHV-5			+	−
	Misc382	CTX-M-44			+	−
K. pneumoniae	Coudm621	SHV-4-like	FOX-5	SHV-1 (pI 5.6)	−	−
	01VCH55	SHV-5-like	FOX-like	TEM-1 (pI 5.6)	+	−
	01BH79	SHV-5-like		SHV-1, TEM-1 (pI 5.25), KPC-2	+	−
K. oxytoca	01VCH72	SHV-3-like	DHA-1	K1, TEM-1	−	+

TABLE 3.

Falsely positive confirmatory test results

Organism	Straina	β-Lactamase(s) expression		Confirmatory test result
		AmpC	Other	Phoenix	VITEK 2
E. coli	01LSAI20		High SHV-1	−	+
	01WM15	High chromosomal		+	−
K. pneumoniae	UMM3		KPC-2, TEM-1-like	+	+
	GB91		High SHV-1	+	+
K. oxytoca	01BH59		High K1	+	+
	01EUH162		High K1	+	−

^aAll strains were ESBL negative.

Sensitivity increased slightly for both systems after repeat testing of 28 strains, while specificity increased slightly for the VITEK 2 but decreased slightly for the Phoenix test. The VITEK 2 expert system was slightly more accurate on repeat testing, while the Phoenix expert system was slightly less accurate.

Expert systems.

While many of the expert system interpretations were straightforward and helpful, some were suboptimal and potentially confusing or misleading. A few interpretations indicated conceptual development of expert system software that would be very helpful when perfected. In this category was the ability of the VITEK 2 expert system to categorize one of eight CTX-M-producing strains as a producer of a CTX-M ESBL. The VITEK 2 also correctly overruled a positive ESBL confirmatory test for K1-hyperproducing K. oxytoca 01BH59, which was ESBL negative, correctly identified SHV-1 hyperproduction in one of three strains, and correctly recognized the production of plasmid-mediated AmpC β-lactamases by two of nine strains. The Phoenix expert system performed better than the VITEK 2 system with some strains that produced multiple β-lactamases. For example, it correctly deduced ESBL production in an SHV-5-like-producing K. pneumoniae isolate, 01VCH55, that produced four β-lactamases, including a FOX-like AmpC, whereas the VITEK 2 system initially suggested retesting due to irresolvable correction possibilities and, on retesting, incorrectly suggested carbapenem resistance.

The Phoenix expert system utilizing rules 345 and 1437 increased the ESBL detection rate from 96% to 99% but was associated with an unacceptable 42% false-positive rate (Table 1). The reduced specificity was due mostly to the rules interpreting high-level AmpC production as evidence of an ESBL.

The currently available Phoenix expert system recognized unusually elevated carbapenem MICs in two of three KPC-2-producing K. pneumoniae strains and in a KPC-3-producing E. coli strain, while the VITEK 2 expert system recognized reduced carbapenem susceptibility in only two of the four strains. Neither system suggested that an unusually elevated carbapenem MIC was consistent with possible carbapenemase production.

The currently available Phoenix expert system was in conflict with CLSI recommendations in that for ESBL-producing strains, it converted susceptible and intermediate results for amoxicillin-clavulanate to results indicating resistance (Table 4). It also changed a result indicating susceptibility to cefotaxime to one indicating resistance for a K1-hyperproducing K. oxytoca strain. It did not correct some false-positive ESBL test results associated with KPC or high-level production of AmpC, K1, and SHV-1 β-lactamases. Two strains of K. oxytoca were incorrectly identified as K. pneumoniae, making interpretations relevant to K1 β-lactamase of K. oxytoca impossible. In addition, a K. pneumoniae strain was misidentified as Enterobacter cloacae, which would incorrectly direct the expert system to include the production of a chromosomally mediated AmpC β-lactamase as a possible interpretation.

TABLE 4.

Phoenix expert system issues

Issue	Comment
Changed results indicating susceptibility or intermediate
susceptibility to amoxicillin-clavulanate for
ESBL-producing strains to results indicating resistance	Conflicts with CLSI ESBL recommendations
Changed a result indicating susceptibility to cefotaxime
for a K1-hyperproducing K. oxytoca strain to one
indicating resistance	Conflicts with CLSI recommendations
Did not correct false-positive ESBL tests	Strains listed in Table 3
Incorrectly identified two isolates of K. oxytoca as K. pneumoniae
and one isolate of K. pneumoniae as E. cloacae	Potentially important because some expert system rules are species specific
Did not suggest that unusually elevated carbapenem MICs might
indicate carbapenemase production

The VITEK 2 expert system had more problems than the Phoenix system. It incorrectly overruled a positive ESBL confirmatory test (Table 5). It suggested that 10 strains (8 E. coli and 2 K. oxytoca strains) be retested or that a laboratory decision be made to correct the results. (On retesting, this message was repeated for two strains.) It did not finalize results for six E. coli strains (five were laboratory strains). On retesting, correct, finalized results were obtained for three of the six strains. For the other three strains, although the ESBL was detected correctly and the expert system modified the susceptibility results correctly, a final result was not provided. This was potentially confusing, as it was inconsistent with the handling of most ESBL-positive strains. Two tests with laboratory strains of E. coli were terminated due to insufficient growth. An expert interpretation was not provided for an ESBL-producing strain of E. coli. The expert system suggested that the laboratory should decide if three ESBL-positive K. pneumoniae strains were ESBL positive or ESBL negative. Even though a significantly elevated imipenem MIC of >16 μg/ml was detected for a KPC-2-producing K. pneumoniae strain, the expert system did not suggest the production of carbapenemase. On retesting, the imipenem MIC remained >16 μg/ml, but the expert system changed the MIC to ≤2 μg/ml. Some falsely positive ESBL test results associated with KPC, AmpC, K1, and SHV-1 β-lactamases were not corrected (Table 3). The ESBL test classified correctly the MIR-1 plasmid-mediated AmpC β-lactamase of an E. coli strain as a non-ESBL, but the expert system overrode this result and classified it incorrectly as an ESBL and twice suggested that the test be repeated.

TABLE 5.

VITEK 2 expert system issues

Issue	Comment
Overruled a positive confirmatory test	Reduced the sensitivity of ESBL detection
Incorrectly indicated a CTX-M ESBL in a SHV-5-producing
laboratory strain of E. coli	More importantly, the positive ESBL test was correct
Suggested either retesting or that the laboratory should correct
the results for 10 strains (8 E. coli strains and 2
K. oxytoca strains)	Confusing message was obtained, and for 2 strains the message was repeated after retesting (annoying)
Did not finalize results for 6 E. coli strains (5 were laboratory
strains)	Correct, finalized results were obtained upon retesting of only 3 of the 6 strains
Tests were terminated due to insufficient growth of 2 laboratory
strains of E. coli	Necessitated repeat test
No expert interpretation provided for 1 ESBL-positive laboratory
strain of E. coli	Confusing
Inconclusive, confusing messages for 3 ESBL-positive K. pneumoniae
strains; laboratory was requested to decide if
strains were ESBL positive or negative	The choices offered were ESBL or ESBL + impermeability or SHV-1 hyperproduction for 1 strain and ESBL + impermeability or SHV-1 hyperproduction for 2 strains
Did not correct false-positive ESBL tests	Strains listed in Table 3
Did not suggest that unusually elevated carbapenem MICs
might indicate carbapenemase production
Problem with the MIR-1 AmpC β-lactamase in E. coli; ESBL
test correctly classified it as a non-ESBL, but the expert
system incorrectly changed the interpretation to ESBL
Did not detect the ESBL that was coproduced with the KPC-2
β-lactamase in K. pneumoniae 01BH79

DISCUSSION

Ideally, an automated system that utilizes an ESBL confirmatory test and an expert system should accurately discriminate ESBLs from other β-lactamases and provide interpretations and recommendations that do not require additional input by laboratory personnel (10). Inconclusive or incorrect comments are likely to frustrate and cause confusion and may delay or even prevent the issue of a clinically useful report to the clinician.

The few published studies of the performance of ESBL detection by the Phoenix and VITEK 2 systems are based mostly on European versions of the systems and tend to lack inclusion of strains with the types of β-lactamase production that are known to cause errors with ESBL tests. Spanu et al., using enzyme characterizations as the comparison, reported that the VITEK 2 ESBL confirmatory test and expert system exhibited 98.1% sensitivity and 99.7% specificity for ESBL detection against a panel of 1,129 Italian isolates of Enterobacteriaceae that produced at least 21 different ESBLs. Isolates that produced plasmid-mediated AmpC or carbapenemases of class A or B were not included (13). Sanguinetti et al. evaluated the Phoenix confirmatory test and expert system against 510 characterized clinical isolates of Enterobacteriaceae that produced 12 different ESBLs and reported 100% sensitivity and 98.9% specificity for ESBL detection (11). In the only direct comparison so far, Wiegand et al. tested 147 characterized isolates of 10 species of Enterobacteriaceae (17). Both instruments were equipped with expert system rules but lacked ESBL confirmatory tests. ESBLs were produced by 85 of the isolates, the remaining isolates being hyperproducers of chromosomal AmpC, K1, or SHV-1 β-lactamases or lacking detectable β-lactamase activity. The Phoenix test detected 99% of the ESBLs, whereas the VITEK 2 test detected 86%. The specificities of both systems were poor (52% for Phoenix, 78% for VITEK 2) reflecting the reliance of detection on expert systems in the absence of a confirmatory test. Leverstein-van Hall et al. (6) compared the Phoenix confirmatory test and expert system, VITEK 2 expert system (no confirmatory test), and the VITEK Legacy confirmatory test against 74 uncharacterized multiresistant isolates of E. coli and Klebsiella spp. from The Netherlands and 17 genotypically characterized strains. They concluded that the Phoenix system (89% accuracy) was superior to the VITEK 2 system (78% accuracy) for ESBL detection. However, the VITEK 2 system lacked an ESBL confirmatory test and the reference test for the clinical isolates was the Etest ESBL test (AB Biodisk, Solna, Sweden), not enzyme characterizations (6). Since the clinical isolates were uncharacterized, it is unknown if the Etest was an accurate reference test, how many types of ESBLs were encountered, or if the types of organisms in which it is difficult to detect ESBLs were included. Sturenburg et al. compared the abilities of the Phoenix and VITEK 2 expert systems (in the absence of ESBL confirmatory tests) to detect ESBLs in 34 characterized ESBL-producing clinical isolates of E. coli and Klebsiella spp. that produced eight different ESBLs and reported detection rates of 100% for the Phoenix system and 85% for the VITEK 2 system (14).

The choice of strains for the current study was intended to provide the strongest possible challenge to the capabilities of the two systems. To our knowledge, the study included more ESBLs than previous studies of automated instruments (6, 11, 13, 14). Given the high level of diagnostic difficulty of the strains, the ESBL confirmatory tests of both systems were highly sensitive. The seemingly high percentages of false-positive tests obtained with the ESBL-negative strains reflected the challenging nature of the strains and the high mathematical impact of an incorrect result when only 26 strains were tested. It was, however, necessary to include some laboratory strains of E. coli that produced certain β-lactamases, some of which grew poorly in the VITEK 2 system and contributed to the lower sensitivity of its ESBL confirmatory test. The Phoenix was able to sustain the growth of these strains, suggesting that it may use a more robust growth medium.

Because the Phoenix expert system utilizing rules 345 and 1437 was associated with an unacceptably high number of false-positive ESBL interpretations, this arm of the study is not discussed further. The following comments apply only to the VITEK 2 and currently available Phoenix expert systems. Both provided helpful comments for many strains but were unsatisfactory for others. Modifications are needed to correct errors, eliminate confusing messages, and better address the types of resistance mechanisms encountered in U.S. isolates. For example, it is now essential to indicate the possibility of a carbapenemase when an unusually elevated carbapenem MIC is encountered (1-3, 9, 12, 18). Reduced susceptibility to a carbapenem is an important diagnostic clue that should not be ignored. This is especially important if a positive ESBL test is obtained, as this result may lead to a patient being treated inappropriately with a carbapenem when a carbapenemase is produced by the pathogen.

The VITEK 2 system was better than the Phoenix test at recognizing the hyperproduction of K1 or SHV-1 β-lactamases as causes of false-positive ESBL tests. However, these mechanisms need to be better addressed by both systems. Characteristic differences between these mechanisms and ESBLs have diagnostic utility that could be utilized by expert systems. Although the actual MICs vary with different susceptibility test systems, K1 hyperproduction in K. oxytoca is usually associated with a ceftriaxone MIC at least 3 dilutions higher than the cefotaxime MIC, whereas ESBL production is associated with similar MICs of these drugs. In addition, ceftazidime MICs are often significantly elevated by ESBL production but not by the K1 enzyme (typically <2 μg/ml). Most SHV-1 hyperproducers have low cefpodoxime MICs, whereas ESBL producers typically have significantly elevated cefpodoxime MICs.

In general, the VITEK 2 expert system offered more-complex interpretations and more choices for the user and suggested that more tests be repeated. It also suggested more often that the laboratory should select which resistance mechanism was present. This is likely to cause frustration, particularly in smaller laboratories where a microbiologist with sufficient expertise may not be available to make the required decisions. Frustration is also likely when isolates are encountered for which the software keeps looping back to suggest that the laboratory keep repeating the test.

In conclusion, the ESBL confirmatory tests of both systems exhibited a high capacity to detect a wide range of ESBLs. However, both expert systems require modification to update and enhance their utility. In this regard, the VITEK 2 expert system was considered potentially more frustrating, as it provided more inconclusive interpretations of the results.