ABSTRACT
Early identification of carbapenemase-producing Enterobacteriaceae (CPE) is essential to prevent their dissemination within health care settings. Our objective was to evaluate the accuracy of 11 phenotypic assays for the detection of CPE. Two collections of carbapenem-resistant Enterobacteriaceae (CRE) isolates were evaluated, including 191 retrospective isolates (122 CP-CRE and 69 non-CP isolates) as well as 45 prospective clinical isolates (15 CP-CRE and 30 non-CP-CRE) obtained over a 3-month period. The sensitivity and specificity of each test was determined, with molecular genotype serving as the gold standard. Among the retrospective cohort, sensitivities ranged from 72% for the boronic acid synergy test for the detection of KPC producers to ≥98% for the modified Carba NP, the Rapidec Carba NP, the manual Blue Carba, and the modified carbapenem inactivation method for the detection of any CPE. Sensitivity differed among tests across enzyme classes. All assays had excellent specificity exceeding 95%, with the exception of the boronic acid synergy test (88%) and modified Hodge test (91%). Prospectively, 45 CRE isolates were encountered over a 3-month period, including 15 CPE (33%) and 30 non-CP-CRE (67%). Results from the prospective cohort were similar. However, a decrease in specificity was observed across most tests, likely due to restricted inclusion of non-CP-CRE to assess the specificity of the assays. Overall, accuracy of CPE detection varied across phenotypic tests. Local epidemiology of CP genotypes, turnaround time, and ease of incorporation into the laboratory workflow should be considered when selecting a phenotypic assay for clinical use.
KEYWORDS: carbapenem-resistant organisms, carbapenemase-producing organisms, prevalence
INTRODUCTION
Carbapenemase-producing carbapenem-resistant Enterobacteriaceae (CP-CRE) are associated with significant morbidity and mortality (1, 2). A variety of carbapenemase genes have been described that are either plasmid or chromosomally encoded, including blaKPC, blaSME, blaIMI, blaNDM, blaVIM, blaIMP, and blaOXA-48-type (2). Early and accurate identification of CP-CRE is essential to prevent their dissemination within health care settings.
Detection of CP-CRE in clinical laboratories is challenging, as isolates may only have moderate reductions in susceptibilities to carbapenems (3), and resistance may be mediated by other mechanisms, such as extended-spectrum-β-lactamase (ESBL) and/or AmpC β-lactamase producers with decreased membrane permeability. Molecular methods for the detection of carbapenemase genes are costly, may require significant expertise, and are limited by the targets included. Thus, rapid and affordable phenotypic assays to broadly classify CRE into CP-CRE versus non-CP-CRE have been developed.
Over the last decade, a number of phenotype-based assays have been available, including rapid colorimetric-based assays (manual and commercial versions of the Carba NP) (4), growth-based assays (modified Hodge test [MHT; 5], Etests, and the carbapenem inactivation method [CIM]) (6), matrix-assisted laser-desorption ionization time-of-flight mass spectrometry (MALDI-TOF MS) carbapenem hydrolysis assays (7), and immunochromatogenic assays (8). Our objective was to perform a comprehensive method comparison study to assess the accuracy of 11 carbapenemase detection assays on a collection of well-characterized Enterobacteriaceae isolates.
RESULTS
Accuracy of CPE detection among retrospective isolates.
The sensitivity and specificity of the various assays in detecting carbapenemase production among the retrospective isolates are shown in Table 1. The sensitivity of the assays for the detection of all CPE (excluding the boronic acid synergy test and the metallo-β-lactamase [MβL] Etest) ranged from 84% for the manual Carba NP CLSI method to 99% for the modified Carba NP assay. For the manual Carba NP CLSI method there were 17 (9%) isolates that were read as invalid (tube A, without imipenem [IP], remained red, but tube B, containing imipenem, became red-orange to orange). The invalid results included 8 OXA-48-like producers, 5 KPC producers, 2 NDM producers, 1 VIM producer, and 1 non-CP-CRE. The invalid results were included as negative results to assess sensitivity and specificity. If these isolates were categorized as positive for carbapenemase production, overall sensitivity and specificity for carbapenemase production with the manual Carba NP CLSI method would be 97% and 99%, respectively. The Rapidec Carba NP had the highest sensitivity (98%) for detection of CPE among the commercial rapid chromogenic assays, outperforming the Neo-Rapid Carb screen (89%) and the Rapid Carb Blue screen kits (89%), while all three had a specificity of ≥99%.
TABLE 1.
Accuracy of 11 phenotypic assays for carbapenemase detection using 191 retrospective Enterobacteriaceae isolates
| Drug (n) | Sensitivity or specificityd [% (95% confidence interval)] |
||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Rapidec Carba NP | Neo-Rapid Carb screen | Rapid Carb Blue screen | Manual Carba NP CLSIa | Manual Blue Carba | Modified Carba NP | Boronic acid synergy test | Metallo-β-lactamase Etestb | MHT | CIM | mCIM | |
| KPC (43) | 100 (90–100) | 91 (77–97) | 84 (69–93) | 84 (69–93) | 98 (86–100) | 100 (90–100) | 72 (56–84) | 98 (86–100) | 91 (77–97) | 98 (86–100) | |
| SME (8) | 88 (47–99) | 100 (60–100) | 100 (60–100) | 100 (60–100) | 100 (60–100) | 100 (60–100) | 100 (60–100) | 100 (60–100) | 100 (60–100) | ||
| IMI (2) | 100 (20–100) | 100 (20–100) | 100 (20–100) | 100 (20–100) | 100 (20–100) | 100 (20–100) | 100 (20–100) | 100 (20–100) | 100 (20–100) | ||
| NDM (36) | 100 (88–100) | 94 (80–99) | 97 (84–100) | 92 (76–98) | 97 (84–100) | 97 (84–100) | 86 (70–95) | 86 (70–95) | 92 (76–98) | 100 (88–100) | |
| VIM (11) | 100 (68–100) | 91 (57–100) | 100 (68–100) | 100 (68–100) | 100 (68–100) | 100 (68–100) | 91 (57–99) | 100 (68–100) | 100 (68–100) | 100 (68–100) | |
| IMP (6) | 100 (52–100) | 100 (52–100) | 100 (52–100) | 100 (52–100) | 100 (52–100) | 100 (52–100) | 17 (9–64) | 100 (52–100) | 83 (36–99) | 83 (36–99) | |
| OXA-48-type (15) | 93 (66–100) | 60 (33–83) | 67 (39–87) | 40 (17–67) | 100 (75–100) | 100 (75–100) | 100 (75–100) | 80 (51–95) | 100 (75–100) | ||
| All CPEc (122) | 98 (94–100) | 89 (82–94) | 89 (82–94) | 84 (76–90) | 98 (94–100) | 99 (95–100) | 79 (66–89) | 95 (89–98) | 91 (84–95) | 98 (94–100) | |
| Non-CP (69) | 99 (91–100) | 99 (91–100) | 100 (93–100) | 100 (93–100) | 96 (87–99) | 100 (93–100) | 88 (81–92)e | 98 (93–99)f | 91 (81–96) | 99 (91–100) | 99 (91–100) |
The results shown assume the 17 (9%) invalid results to be negative results. If invalid results are assumed to be positive, the sensitivity results are the following: KPC, 95% (83 to 99%); SME, 100% (60 to 100%); IMI, 100% (20 to 100%); NDM, 97% (84 to 100%); VIM, 100% (68 to 100%); IMP, 100% (52 to 100%); OXA-48-type, 93% (66 to 100%); all CPE, 97% (91 to 99%). The specificity result was 99% (91 to 100%).
These results reflect the inclusion of the 8 (15.1%) nondeterminable results as negative. If they are considered negative, the sensitivities would the following: blaNDM, 94% (80 to 99%); blaVIM, 91% (57 to 99%); blaIMP, 100% (52 to 100%). The specificity of this assay for these three genes would be 57% (48 to 65%).
There was one isolate which produced both a VIM and an OXA-48-type that was included in the “all CPE” calculations but not included in the VIM or OXA-48-type category. This isolate was identified as a carbapenemase producer by all of the described assays except the metallo-β-lactamase Etest.
Values for CP isolates are sensitivities. Values for non-CP isolates are specificities.
Non-KPC carbapenemase producers (including MβLs and OXA-48-types) were included in determining the specificity of the boronic acid test.
Non-MβL carbapenemase producers (including blaKPC and blaOXA) were included in determining the specificity of the MβL Etests.
Overall the rapid chromogenic assays generally performed well for the detection of class A and B carbapenemases, but their sensitivity for detecting OXA-48-types ranged from 40 to 100%. For the growth-based methods, performance varied among the different classes of carbapenemases. The MHT had a lower sensitivity (86%) for the detection of NDM producers and had limited specificity (91%). The CIM had an overall sensitivity and specificity of 91% and 99%, respectively, for carbapenemase detection but was only able to identify 80% of OXA-48-types, 83% of IMP producers, 91% of KPC producers, and 92% of NDM producers. However, with the modifications involved with the modified CIM (mCIM), the overall sensitivity and specificity increased to 98% and 99%, and all OXA-48-types and NDM were detected. A single KPC-producing Serratia marcescens isolate and an IMP-producing Klebsiella pneumoniae isolate yielded false-negative results. With the exception of the MHT (91%) and the manual Blue Carba assay (96%), all assays had a specificity ≥99% for the overall detection of carbapenemase enzymes.
The boronic acid synergy test and the MβL Etest were assessed for their accuracy to detect KPC and MβL producers, respectively. The boronic acid synergy test had a sensitivity of 72% and specificity of 88% for the detection of KPC producers. The MβL Etest had a sensitivity of 79% for detecting class B carbapenemases when the 15% of isolates with nondeterminable results (>256 μg/ml IP/>64 μg/ml imipenem-EDTA [IPI] or <4 μg/ml IP/<1 μg/ml IPI) were considered negative. This improved to 94% when nondeterminable results were considered positive but at the expense of the specificity of the assay, decreasing from 98% to 57%.
Accuracy of CPE detection among prospective clinical CRE isolates.
The accuracy of the 11 phenotypic assays for the detection of CPE among prospectively collected, unique, clinical CRE isolates are summarized in Table 2. Overall, the sensitivity of the assays to detect CPE ranged from 67% for the Rapid Carb Blue screen assay to 100% for the Rapidec Carba NP, manual Carba NP CLSI, the MHT, and the mCIM assays. Sixteen non-CP-CRE isolates (35%) had invalid results using the manual Carba NP CLSI method and were considered negative. If these results were considered positive, the specificity of the assay decreased from 97% to 43%. Results from the rapid chromogenic assays were similar for both groups of isolates, with the notable exception of the Rapid Carb Blue screen, yielding poorer sensitivity at only 67% because of its failure to detect 3 of 11 KPC and 1 of 3 NDM producers. Similarly, the modified Carba NP assay was only able to identify 1 of 3 NDM producers and did not identify 1 KPC producer. The MHT was able to identify all CP-CRE, with a specificity of 87%. Both sensitivity and specificity of the CIM were 93%, but with the mCIM both increased to 100%. The specificity of the assays ranged from 53% for the manual Blue Carba to 100% for the Rapid Carb Blue screen and mCIM assays.
TABLE 2.
Accuracy of 11 phenotypic assays for carbapenemase detection using 45 prospective clinical isolates
| Drug | Sensitivity or specificityf [% (95% confidence interval)] |
||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Rapidec Carba NP | Neo-Rapid Carb screen | Rapid Carb Blue screen | Manual Carba NP CLSIa | Manual Blue Carba | Modified Carba NP | Boronic acid synergy testb | Metallo-β-lactamase Etestc | MHT | CIM | mCIMd | |
| KPC (n = 11) | 100 (68–100) | 100 (68–100) | 73 (39–93) | 100 (68–100) | 100 (68–100) | 91 (57–100) | 91 (57–100) | 100 (68–100) | 100 (68–100) | 100 (68–100) | |
| NDM (n = 3) | 100 (31–100) | 100 (31–100) | 67 (13–98) | 100 (31–100) | 100 (31–100) | 33 (2–87) | 50 (9–91) | 100 (31–100) | 67 (13–98) | 100 (31–100) | |
| All CPEe (n = 15) | 100 (75–100) | 93 (66–100) | 67 (39–87) | 100 (75–100) | 93 (66–100) | 73 (45–91) | 100 (75–100) | 93 (66–100) | 100 (75–100) | ||
| Non-CP CRE (30) | 83 (65–94) | 97 (81–100) | 100 (86–100) | 97 (81–100) | 53 (35–71) | 73 (54–87) | 82 (65–93) | 100 (90–100) | 87 (68–96) | 93 (76–99) | 100 (86–100) |
The results shown assume the 16 (35%) invalid results to be negative results. If the invalid results are assumed to be positive, the sensitivities remain the same but the specificity would be 43% (26 to 62%).
Non-KPC carbapenemase producers (including MβL and OXA-48-types) were included in determining the specificity of the boronic acid test.
Non-metallo-β-lactamase producers (including KPCs and OXA-48-types) were included in determining the specificity of the MβL Etest. The single isolate that produced both an NDM and an OXA-48-type was included in the sensitivity calculation. The results shown assume the 24 (53%) nondeterminable results are negative results. If the nondeterminable results are assumed to be positive, the sensitivity increases to 100% and specificity decreases to 46% (31 to 62%).
The results shown assume the 1 (2%) indeterminate result to be a negative result. If the indeterminate result is assumed to be positive, the sensitivity remains the same but the specificity would be 97% (81 to 100%).
There was one isolate which produced both an NDM and an OXA-48-type that was included in the “all CPE” calculations but not included in the NDM category. This isolate was identified as a carbapenemase producer by the following assays: Rapidec Carba NP, manual Carba NP CLSI, MHT, CIM, and mCIM.
Values for CP-CRE isolates are sensitivities. Values for non-CP-CRE isolates are specificities.
The boronic acid synergy test had a sensitivity of 91% and specificity of 82% for the detection of KPC producers among the clinical isolates. This test produced false-positive results for 6 chromosomal AmpC (cAmpC) producers and failed to identify one KPC-producing K. pneumoniae isolate. The MβL Etest had a sensitivity of 50% and specificity of 100% for detecting class B carbapenemases when nondeterminable (>256 μg/ml IP/>64 μg/ml IPI or <4 μg/ml IP/<1 μg/ml IPI) results (53%) were considered negative. If the nondeterminable results are assumed to be positive, the sensitivity increased to 100% and specificity decreased to 46%.
Table 3 outlines the relative pros and cons of the various phenotypic assays, including the following: (i) accuracy, (ii) anticipated false positives or negatives, (iii) approximate cost per test, (iv) ease of use, (v) special requirements, (vi) interpretation of results, (vii) total time to perform the test, (viii) turnaround time, and (ix) regulatory status. Most of the assays cost $2 or less per test. The cost of the Rapidec Carba NP in U.S. dollars has not been established. Of note, one Turkish institution reports the cost per test at 8.7 euros (9). The Rapidec Carba NP has the benefit of all reagents and supplies being readily available in the kit with results available within about 3 h.
TABLE 3.
Relative pros and cons of various carbapenemase phenotypic assays
| Parameter | Result by test type |
||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Commercially available |
Manual rapid phenotypic |
Growth-based |
|||||||||
| Rapidec Carba NP (9) | Neo-Rapid Carb screen | Rapid Carb Blue screen | Manual Carba NP CLSI (10) | Manual Blue Carba (11) | Modified Carba NP (12) | Boronic acid synergy test (13) | Metallo-β-lactamase Etest | MHT (10) | CIM (6) | mCIM | |
| Accuracy (%; sensitivity, specificity) | 98, 99 | 89, 99 | 89, 100 | 84, 100 | 98, 96 | 99, 100 | 72, 88 | 43, 94 | 95, 91 | 91, 99 | 98, 99 |
| Anticipated false positives or negatives | False negatives with OXA-48-type | False negatives with OXA-48-type | False negatives with OXA-48-type | False negatives with OXA-48-type | False negatives with OXA-48-type | False negatives with OXA-48-type and mucoid isolates | False positives with ESBL/AmpC with outer membrane defects; false negatives with SME, NDM, OXA-types | ||||
| Approximate cost per test (US$) | U.S. pricing not established (∼8.7 euros) | $2.50 | $2.75 | ≥$2.00a | ≥$2.00a | ≥$2.00a | <$1.00 | $6.00 | <$1.00 | <$1.00 | <$1.00 |
| Ease of use | All reagents and supplies (including turbidity standard) includedb | Only diatabs with and without imipenem included in kitb | Only diatabs with and without imipenem included in kitb | Fresh reagents should be prepared daily (10) | Imipenem containing reagent prepared with each use | Fresh reagents need to be prepared often (12) | Most supplies readily available except boronic acid solution | All supplies readily available except MβL Etestb | All supplies readily available | All supplies readily available (6) | All supplies readily available |
| Special requirements | None; all reagents are provided in kit | Protein extraction reagent | pH meter | pH meter | pH meter | pH meter | Boronic acid solution needs to be prepared initially but long shelf-life if refrigerated | None; all standard laboratory supplies | None; all standard laboratory supplies | None; all standard laboratory supplies | None; all standard laboratory supplies |
| Other details | 4 McFarland standard; round-bottom tubes required to interpret results due to diatab dissolving in solution; all supplies other than diatabs not included | Requires prepn of 0.9% NaCl solution (pH 8.5); 2 McFarland standard; all supplies other than diatabs not included; round-bottom tubes required to interpret results due to diatab dissolving in solution | Reagents with imipenem have a short shelf-life | Reagents with imipenem have a short shelf-life; requires 2 solutions of bromothymol blue at different pHs (pH 6 and 7); pH needs to be verified weekly; requires agitation during 2 h of incubation | Reagents with imipenem have a short shelf-life | ||||||
| Interpretation of results | Any color change from red to orange/yellow; color change can be subjective for intermediate results (red to orange) | Any color change from red to orange/yellow; color change can be subjective for intermediate results (red to orange) | Color change from blue to green/yellow; color change can be subjective for intermediate results (blue to green) | Any color change from red to yellow; color change from red to orange considered invalid | Color change from blue to green/yellow; color change can be subjective for intermediate results (blue to green) | Color change from red to orange/yellow; color change can be subjective for intermediate results (red to orange) | Only used for KPC detection; ertapenem disk with boronic acid has ≥5-mm zone diam difference from the ertapenem disk alone | Only used for MβL detection; a ratio of IP/IPI MIC of ≥8 or presence of phantom zone or deformation of ellipse | Indentation of zone diam of E. coli toward ertapenem disk along streaked isolate; reading of zone indentation can be subjective | Zone or no zone of inhibition around meropenem disk; requires initial setup and then plating of disk onto lawn of E. coli following 2 h of incubation | Positive, zone diam of 6–15 mm; indeterminate, 16–18 mm; negative, ≥19 mm; requires initial setup and then plating of disk onto lawn of E. coli following 4 h of incubation |
| Total time to perform the test | Initial 30-min lysis step; 80 to 170 min | Initial 30-min lysis step; 80 to 170 min | Initial 30-min lysis step; 80 to 170 min | Simplified lysis step; 40 (positive at first read) to 130 min | No upfront lysis step; uses a direct colony approach; 40 (positive at first read) to 130 min | Simplified lysis step; 40 (positive at first read) to 130 min | 10-min setup and 2 min to read the following day | 10-min setup and 2 min to read the following day | 15 min for setup and 2 min to read the following day | 5 min for initial setup, 10 min to inoculate plate, and 2 min to read the following day | 5 min for initial setup, 10 min to inoculate plate, and 2 min to read the following day |
| Turnaround time | Same-day result; 30 min to 2 h | Same-day result; 30 min to 1 h | Same-day result; 30 min to 1 h | Same-day result; 30 min to 2 h | Same-day result; 30 min to 2 h | Same-day result; 30 min to 2 h | 18–24 h | 18–24 h | 18–24 h | 8–24 h | 18–24 h |
| Regulatory status | Pending FDAc clearance | RUO | RUO | LDT; CLSI and EUCASTd endorsed | LDT | LDT | LDT | RUO | LDT; CLSI endorsed | LDT | LDT; CLSI endorsed |
The cost per test may be much higher if the imipenem-based reagents are not utilized within 3 days.
According to package insert.
United States Food and Drug Administration.
European Committee on Antimicrobial Susceptibility Testing.
DISCUSSION
Rapid and accurate detection of carbapenemase producers is critical to implementing timely contact isolation and antibiotic treatment decisions, as CP-CRE can have devastating consequences in health care settings (2). Although genotypic tests remain the gold standard for carbapenemase detection, they cannot practically be conducted on all Enterobacteriaceae clinical isolates, as costs associated with these tests can be prohibitively expensive and results are limited by the targets included in assays. Thus, it has become customary for clinical laboratories to implement phenotypic methods to broadly categorize CRE into CP-CRE and non-CP-CRE.
We performed a method comparison of 11 phenotypic tests to detect carbapenemase production among well-characterized Enterobacteriaceae isolates. To the best of our knowledge, this is the most comprehensive published study comparing several phenotypic CPE detection methods. Furthermore, we are the first to report on the comparative performance of the mCIM.
Overall, the accuracy of carbapenemase detection varied across the 11 phenotypic tests evaluated in this study. Among the retrospective cohort, the sensitivity of the assays ranged from 72% for the boronic acid synergy test for the detection of KPC producers to 99% for the modified Carba NP assay for the detection of all CPE. Four tests achieved a sensitivity of ≥98%, including the Rapidec Carba NP, the manual Blue Carba, the modified Carba NP, and the mCIM. Moreover, the sensitivity varied by carbapenemase enzyme group such that most of the rapid colorimetric assays tested and the CIM were less sensitive for the detection of OXA-48-type producers. Interestingly, false-negative results were observed with several of the assays for a few KPC producers with either low carbapenem MICs (low-level expression) or those produced by organisms that were mucoid in nature.
Our results contrast with some studies where the rapid colorimetric assays have sensitivities approaching 100% for KPC producers (14, 15). The KPC isolates included in our study were derived from a variety of geographic sources, identifying sensitivity issues that may not always be evident when testing is limited to a few regional clones. Overall, the assays had excellent specificity, exceeding 95%, with the exception of the boronic acid synergy test (88%) and the MHT (91%). The lack of specificity of the MHT has been previously reported (16, 17). The limited specificity of the boronic acid synergy test is not surprising, as boronic acid is a known inhibitor of other class A carbapenemases (e.g., SME and IMI) and class C β-lactamases, which accounted for the majority of false-positive results we observed (13).
Differences in sensitivity and specificity were observed between the retrospective and prospective cohorts. The most dramatic difference was detected with the modified Carba NP. Its sensitivity and specificity were 99% and 100%, respectively, for the retrospective collection and were reduced to 73% and 73% for the prospective cohort because of 2 false-negative results among NDM producers in the latter. Overall, most tests in the prospective cohort had a decline in specificity, likely due to differences in the characteristics of the isolates included to assess the specificity of the assays between the two cohorts. In the retrospective subset, a notable number of non-CP isolates were carbapenem susceptible, whereas the prospective cohort only included non-CP-CRE. The specificity obtained from the prospective cohort provides a more practical overview of results encountered when implementing these assays, as they would most realistically be used to distinguish CP-CRE from non-CP-CRE in clinical settings. Furthermore, the inclusion of non-CP-CRE in the prospective isolates highlights the subjective nature of interpreting colorimetric assay results with minor color changes. For example, for the manual Carba NP CLSI method, an invalid result occurs when there is a slight color change in the imipenem-containing tube from red-orange to orange. In the retrospective cohort, the 9% of invalid results were mostly observed among carbapenemase producers (94%), whereas in the prospective isolates, 35% of isolates had an invalid result and they were all non-CP-CRE. Invalid results were also witnessed with OXA-48-type producers, necessitating additional testing if invalid manual Carba NP CLSI results occur.
Recently, the CIM was described as a highly sensitive and specific assay for the detection of CP-CRE (6, 9, 18, 19). In contrast to these preliminary reports, we identified limited sensitivity of the CIM for detecting both NDM and OXA-48-type producers. The mCIM was developed by Sanchita Das and colleagues (unpublished work) due to both the limited sensitivity of the CIM for certain enzyme classes and for simplicity (purchasing 2 ml tryptic soy broth [TSB] versus aliquoting 400 μl water). This method has been designated an alternative method to the MHT and Carba NP for carbapenemase detection (10). The modifications made to the CIM allow for improved detection of NDM and OXA-48-like producers, and the overall sensitivity and specificity for all CPE exceeded 98% in both the retrospective and prospective cohorts we evaluated.
Differences in the costs of the assays, ease of use, necessary equipment, time to perform the tests, and turnaround time should all be considered when determining which carbapenemase assay is the most appropriate to incorporate into a clinical microbiology laboratory. The first step in this decision process involves identifying the overall epidemiology of CP-CRE to determine if a phenotypic assay for carbapenemase detection is necessary. If these resistance mechanisms are present in a health care facility, it is important to next evaluate the local molecular epidemiology of carbapenemases to consider the relevance of false-positive and false-negative results observed with the various tests. For example, if OXA-48-types are endemic to a region, then the manual Carba NP CLSI method may not be ideal, as it detected less than half of these resistance enzymes in our study. Similarly, for regions where MβL carbapenemases are present, it may be best to avoid the use of the MHT or the MβL Etest due to the lack of sensitivity for these enzymes. However, the MβL Etest has excellent specificity, so it can be considered a secondary test to detect MβLs following a positive result from a more broad-based CPE test. In addition, testing volumes need to be taken into consideration. A laboratory that infrequently encounters CRE isolates may benefit from commercial rapid colorimetric assays such as the Rapidec Carba NP for same-day results or the mCIM for a next-day result, as manual rapid colorimetric assays can be wasteful because of the need for freshly prepared reagents for each use.
Evaluating these assays globally, our findings suggest that the mCIM is the most practical assay to select. Its sensitivity and specificity approach 100%, it costs less than $1 per test, and all necessary supplies are readily available. Additionally, interpreting the results of this test are not as obscure as procedures for many other phenotypic assays, as it is based on a defined zone diameter. Perhaps its biggest drawback is the time requirement: almost 24 h are needed for results to be available. For same-day results, the Rapidec Carba NP performed the best among the rapid colorimetric assays and is available as a ready-to-use kit. On the other hand, the boronic acid synergy test appeared to be the least advantageous test. It had limited accuracy, and its use for carbapenemase detection is limited to KPC enzymes. Because of international migration and medical tourism, it is increasingly more likely to observe a diversity of carbapenemase resistance mechanisms, and it seems more constructive to implement assays with the ability to identify a diverse range of carbapenemase enzymes.
There were a number of limitations with our study. The prospective isolates only included KPC and NDM enzymes, precluding any discussion of the use of the 11 assays for other carbapenemases in this cohort. Additionally, the limited sample size of the prospective isolates yielded wide confidence intervals. Although the retrospective isolates included a more diverse collection of carbapenemases, there were small numbers of isolates producing several of the carbapenemase enzymes (e.g., SME, IMI, VIM, and IMP), resulting in uncertainty of accuracy estimates. Furthermore, specificity estimates of all assays would be more robust if we had evaluated more non-CP-CRE. Finally, we realize that there are additional tests in development or recently developed that we were unable to include in our method comparison.
Overall, the ability to detect CPE varied across phenotypic tests. The local epidemiology of CP genotypes, accuracy of the assays, turnaround time, and ease of incorporation into the laboratory workflow should all be considered when selecting a phenotypic assay for clinical use.
MATERIALS AND METHODS
Retrospective isolates.
Two separate cohorts were included; a retrospective cohort of 191 Enterobacteriaceae isolates and a prospective cohort of 45 clinical isolates. The retrospective isolates were obtained from (i) the Centers for Disease Control and Prevention and the Food and Drug Administration Antimicrobial Resistance Isolate Bank (n = 138), (ii) International Health Management Associates, Inc. (n = 33) (16), and (iii) the Canadian National Intensive Care Unit (CAN-ICU) Study (n = 20) (20; http://www.can-r.ca/). These isolates were previously molecularly characterized (whole-genome sequencing, Check-Points microarray [Check-Points, Wageningen, Netherlands], or PCR) to identify β-lactamase genes (21; https://www.cdc.gov/drugresistance/resistance-bank/index.html). The retrospective collection included 122 CPE isolates containing the following carbapenemase genes: 43 blaKPC, 36 blaNDM, 15 blaOXA-48-type, 11 blaVIM, 8 blaSME, 6 blaIMP, 2 blaIMI, and 1 blaVIM-4 and blaOXA-48-type. To evaluate the specificity of the assays, 69 non-CP Enterobacteriaceae isolates were tested, including non-CP-CRE (n = 48) and Enterobacteriaceae isolates that were carbapenem susceptible (n = 21). The following organisms were tested: Klebsiella spp. (65), Escherichia coli (58), Enterobacter spp. (25), Citrobacter spp. (13), Serratia marcescens (11), Proteus mirabilis (7), Providencia spp. (4), Morganella morganii (3), Salmonella spp. (2), Shigella sonnei (1), Raoultella ornithinolytica (1), and Kluyvera ascorbata (1).
Prospective isolates.
From January to March 2016, 45 unique CRE were encountered at the Johns Hopkins Hospital (JHH) Medical Microbiology laboratory, representing 0.6% of the 2,674 Enterobacteriaceae clinical isolates processed during this time period. Fifteen (33%) were identified as CPE based on molecular results, including 11 (73%) blaKPC, 3 (20%) blaNDM, and 1 (7%) isolate containing both blaNDM and blaOXA-48-like genes. This collection of CRE included the following organisms: K. pneumoniae (23), Enterobacter cloacae (9), E. coli (4), Citrobacter freundii (4), S. marcescens (3), Enterobacter aerogenes (1), and P. mirabilis (1). MALDI-TOF MS (Bruker Daltonics Inc., Billerica, MA) was used for genus and species identification. The BD Phoenix automated system (BD Diagnostics, Sparks, MD) was used for antimicrobial susceptibility testing (AST). Carbapenem AST results were repeated using the Etest method (bioMérieux, Durham, NC). Carbapenem resistance was determined using CLSI criteria (10). Isolates were frozen at −80°C in glycerol until further testing was performed. β-Lactamase genes were identified using the Check-Points Check-MDR CT103XL kit microarray-based assay (1).
Method comparison study.
Isolates were subcultured from frozen stock to tryptic soy agar (TSA) with 5% blood agar. A second subculture was performed prior to phenotypic testing. All tests were set up in batches on the same calendar day from a common blood agar plate containing a pure culture of the isolate. K. pneumoniae ATCC 1705 and ATCC 1706 were included on a daily basis as positive and negative controls, with the exception of the MβL Etest, where Stenotrophomonas maltophilia ATCC 13636 was used as the positive control.
Eleven phenotypic carbapenemase detection methods were evaluated, including rapid colorimetric assays and growth-based methods. The former included commercial kits, such as Rapidec Carba NP (package insert version 98024-04/03/2016; bioMérieux, Marcy-l'Étoile, France), Neo-Rapid Carb screen kit (package insert version DBV0040F-01/11/2013; ROSCO Diagnostica, Taastrup, Denmark), and Rapid Carb Blue screen (package insert version DBV0-04/09/2014; ROSCO Diagnostica, Taastrup, Denmark), as well as manual methods requiring reagent preparation, such as the manual Carba NP CLSI method (10), manual Blue Carba (11), and modified Carba NP (12). All methods were performed as previously described or by following the package inserts for the commercial-based assays. For consistency, imipenem-based solutions (Sigma-Aldrich, Dorset, United Kingdom) for manual rapid colorimetric assays were made on the same day of testing, as imipenem may hydrolyze with prolonged storage.
Growth-based methods included the boronic acid synergy test (13) using 10-μg ertapenem disks, the MβL Etest containing imipenem and imipenem-EDTA (package insert 16248 2010/11; bioMérieux, Marcy-l'Étoile, France), the modified Hodge test (MHT), the carbapenem inactivation method (CIM; incubated overnight [6]), and the modified carbapenem inactivation method (mCIM) (10).
The mCIM was performed as follows. A 10-μg meropenem disk was added to 2 ml of TSB and a 1-μl loopful of the CRE isolate and incubated for 4 h ± 15 min at 35°C. If the CRE is a CP-CRE, the meropenem is hydrolyzed within 4 h. If the CRE is a non-CP-CRE, the meropenem is incompletely hydrolyzed and will retain activity. Prior to completion of the 4-h incubation, a Muller-Hinton agar (MHA) plate was streaked with a lawn of a carbapenem-susceptible E. coli isolate (ATCC 25922). The meropenem disk was removed from the TSB broth at the end of the 4-h incubation and placed on the MHA plate streaked with susceptible E. coli to determine if the meropenem disk remained active by incubating overnight for 18 to 24 h at 35°C. The mCIM result was interpreted based on the zone diameter of the meropenem disk and considered positive if 6 to 15 mm, indeterminate if 16 to 18 mm, and negative if ≥19 mm.
The investigators performing phenotypic testing were blinded to genus, species, and genotypes of the isolates. Non-CP Enterobacteriaceae isolates were used to assess the specificity of the assays. The sensitivity and specificity of each test was performed, with the molecular genotype serving as the gold standard. For the retrospective isolates, after phenotypic testing was performed, an independent third party compared the phenotypic results to the molecular results. If results were discordant from the anticipated results, testing was repeated by investigators who were still blinded to genus, species, and genotypes and were unaware of the original phenotypic results. For the prospective clinical isolates, if phenotypic and genotypic testing yielded discordant results, testing was not repeated in order to mimic real-world conditions, as microbiologists generally would be unaware of the molecular results in clinical microbiology laboratories.
ACKNOWLEDGMENTS
We acknowledge Tsigereda Tekle for her assistance in clinical isolate collection. Additionally, we thank the International Health Management Associates, Inc. (IHMA), and the Canadian Antimicrobial Resistance Alliance for providing isolates for this study. Rapidec Carba NP kits were kindly provided by bioMérieux, Inc.
The work was supported by funding from the National Institutes of Health (1K23AI127935) and the Inhealth Pilot Project Discovery Program (both awarded to P.D.T.), as well as The Sherrilyn and Ken Fisher Center for Environmental Diseases (awarded to P.J.S.). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
REFERENCES
- 1.Tamma PD, Goodman KE, Harris AD, Tekle T, Robert A, Taiwo A, Simner PJ. 2017. Comparing the outcomes of patients with carbapenemase-producing and non-carbapenemase-producing carbapenem-resistant enterobacteriaceae bacteremia. Clin Infect Dis 64:257–264. doi: 10.1093/cid/ciw741. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Nordmann P, Naas T, Poirel L. 2011. Global spread of carbapenemase-producing Enterobacteriaceae. Emerg Infect Dis 17:1791–1798. doi: 10.3201/eid1710.110655. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Tamma PD, Huang Y, Opene BN, Simner PJ. 2016. Determining the optimal carbapenem MIC that distinguishes carbapenemase-producing and non-carbapenemase-producing carbapenem-resistant Enterobacteriaceae. Antimicrob Agents Chemother 60:6425–6429. doi: 10.1128/AAC.00838-16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Nordmann P, Poirel L, Dortet L. 2012. Rapid detection of carbapenemase-producing Enterobacteriaceae. Emerg Infect Dis 18:1503–1507. doi: 10.3201/eid1809.120355. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Carvalhaes CG, Picao RC, Nicoletti AG, Xavier DE, Gales AC. 2010. Cloverleaf test (modified Hodge test) for detecting carbapenemase production in Klebsiella pneumoniae: be aware of false positive results. J Antimicrob Chemother 65:249–251. doi: 10.1093/jac/dkp431. [DOI] [PubMed] [Google Scholar]
- 6.van der Zwaluw K, de Haan A, Pluister GN, Bootsma HJ, de Neeling AJ, Schouls LM. 2015. The carbapenem inactivation method (CIM), a simple and low-cost alternative for the Carba NP test to assess phenotypic carbapenemase activity in gram-negative rods. PLoS One 10:e0123690. doi: 10.1371/journal.pone.0123690. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Papagiannitsis CC, Studentova V, Izdebski R, Oikonomou O, Pfeifer Y, Petinaki E, Hrabak J. 2015. Matrix-assisted laser desorption ionization-time of flight mass spectrometry meropenem hydrolysis assay with NH4HCO3, a reliable tool for direct detection of carbapenemase activity. J Clin Microbiol 53:1731–1735. doi: 10.1128/JCM.03094-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Glupczynski Y, Evrard S, Ote I, Mertens P, Huang TD, Leclipteux T, Bogaerts P. 2016. Evaluation of two new commercial immunochromatographic assays for the rapid detection of OXA-48 and KPC carbapenemases from cultured bacteria. J Antimicrob Chemother 71:1217–1222. doi: 10.1093/jac/dkv472. [DOI] [PubMed] [Google Scholar]
- 9.Aktas E, Malkocoglu G, Otlu B, Copur Cicek A, Kulah C, Comert F, Sandalli C, Gursoy NC, Erdemir D, Bulut ME. 30 August 2016. Evaluation of the carbapenem inactivation method for detection of carbapenemase-producing Gram-negative bacteria in comparison with the Rapidec Carba NP. Microb Drug Resist. Epub ahead of print. [DOI] [PubMed] [Google Scholar]
- 10.Clinical and Laboratory Standards Institute. 2017. Performance standards for antimicrobial susceptibility testing, 27th ed CLSI supplement M100S. CLSI, Wayne, PA. [Google Scholar]
- 11.Pires J, Novais A, Peixe L. 2013. Blue-carba, an easy biochemical test for detection of diverse carbapenemase producers directly from bacterial cultures. J Clin Microbiol 51:4281–4283. doi: 10.1128/JCM.01634-13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Bakour S, Garcia V, Loucif L, Brunel JM, Gharout-Sait A, Touati A, Rolain JM. 2015. Rapid identification of carbapenemase-producing Enterobacteriaceae, Pseudomonas aeruginosa and Acinetobacter baumannii using a modified Carba NP test. New Microbes New Infect 7:89–93. doi: 10.1016/j.nmni.2015.07.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Doi Y, Potoski BA, Adams-Haduch JM, Sidjabat HE, Pasculle AW, Paterson DL. 2008. Simple disk-based method for detection of Klebsiella pneumoniae carbapenemase-type beta-lactamase by use of a boronic acid compound. J Clin Microbiol 46:4083–4086. doi: 10.1128/JCM.01408-08. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Lutgring JD, Limbago BM. 2016. The problem of carbapenemase-producing-carbapenem-resistant-enterobacteriaceae detection. J Clin Microbiol 54:529–534. doi: 10.1128/JCM.02771-15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Viau R, Frank KM, Jacobs MR, Wilson B, Kaye K, Donskey CJ, Perez F, Endimiani A, Bonomo RA. 2016. Intestinal carriage of carbapenemase-producing organisms: current status of surveillance methods. Clin Microbiol Rev 29:1–27. doi: 10.1128/CMR.00108-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Simner PJ, Gilmour MW, DeGagne P, Nichol K, Karlowsky JA. 2015. Evaluation of five chromogenic agar media and the Rosco Rapid Carb screen kit for detection and confirmation of carbapenemase production in Gram-negative bacilli. J Clin Microbiol 53:105–112. doi: 10.1128/JCM.02068-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Vasoo S, Cunningham SA, Kohner PC, Simner PJ, Mandrekar JN, Lolans K, Hayden MK, Patel R. 2013. Comparison of a novel, rapid chromogenic biochemical assay, the Carba NP test, with the modified Hodge test for detection of carbapenemase-producing Gram-negative bacilli. J Clin Microbiol 51:3097–3101. doi: 10.1128/JCM.00965-13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Tijet N, Patel SN, Melano RG. 2016. Detection of carbapenemase activity in Enterobacteriaceae: comparison of the carbapenem inactivation method versus the Carba NP test. J Antimicrob Chemother 71:274–276. doi: 10.1093/jac/dkv283. [DOI] [PubMed] [Google Scholar]
- 19.Yamada K, Kashiwa M, Arai K, Nagano N, Saito R. 2016. Comparison of the modified-Hodge test, Carba NP test, and carbapenem inactivation method as screening methods for carbapenemase-producing Enterobacteriaceae. J Microbiol Methods 128:48–51. doi: 10.1016/j.mimet.2016.06.019. [DOI] [PubMed] [Google Scholar]
- 20.Zhanel GG, DeCorby M, Laing N, Weshnoweski B, Vashisht R, Tailor F, Nichol KA, Wierzbowski A, Baudry PJ, Karlowsky JA, Lagace-Wiens P, Walkty A, McCracken M, Mulvey MR, Johnson J, Canadian Antimicrobial Resistance Alliance, Hoban DJ. 2008. Antimicrobial-resistant pathogens in intensive care units in Canada: results of the Canadian National Intensive Care Unit (CAN-ICU) study, 2005–2006 Antimicrob Agents Chemother 52:1430–1437. doi: 10.1128/AAC.01538-07. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Taylor F, Baudry P, Zhanel GG, Hoban DJ. Emergence of Escherichia coli and Klebsiella pneumoniae with reduced susceptibilities to carbapenems in Canadian hospitals. Proceedings of the Canadian Association of Clinical Microbiology and Infectious Diseases Annual Meeting. Canadian Association of Clinical Microbiology and Infectious Diseases Annual Meeting, Toronto, ON, Canada. [Google Scholar]