Predicting Probability of Perirectal Colonization with Carbapenem-Resistant Enterobacteriaceae (CRE) and other Carbapenem-Resistant Organisms (CROs) at Hospital Unit Admission

Abstract

Background.

Targeted screening for carbapenem-resistant organisms (CROs), including carbapenem-resistant Enterobacteriaceae (CRE) and carbapenemase-producing organisms (CPOs), remains limited; recent data suggest that existing policies miss many carriers. Our objective was to measure the prevalence of CRO and CPO perirectal colonization at hospital unit admission and use machine learning methods to predict probability of CRO and/or CPO carriage.

Methods.

We performed an observational cohort study of all patients admitted to the medical intensive care unit (MICU) or solid organ transplant (SOT) unit at The Johns Hopkins Hospital between July 1, 2016 and July 1, 2017. Admission perirectal swabs were screened for CROs and CPOs. More than 125 variables capturing pre-admission clinical and demographic characteristics were collected from the electronic medical record (EMR) system. We developed models to predict colonization probabilities using decision tree learning.

Results.

Evaluating 2,878 admission swabs from 2,165 patients, we found that 7.5% and 1.3% of swabs were CRO- and CPO-positive, respectively. There was high organism and carbapenemase diversity among CPO isolates. Despite including many characteristics commonly associated with CRO/CPO carriage or infection, overall, decision tree models poorly predicted CRO and CPO colonization (C-statistics 0.57 and 0.58, respectively). In sub-group analysis, however, models did accurately identify patients with recent CRO-positive cultures who use proton-pump inhibitors as having a high likelihood of CRO colonization.

Conclusions.

In this inpatient population, CRO carriage was infrequent but higher than previously published estimates. Despite including many variables associated with CRO/CPO carriage, models poorly predicted colonization status, likely due to significant host and organism heterogeneity.

INTRODUCTION

Carbapenem-resistant organisms (CROs) are an important cause of healthcare-acquired infections, and are particularly concerning because they are associated with high morbidity and mortality [1–6]. Carbapenem-resistant Enterobacteriaceae (CRE) have received significant attention [7], but glucose non-fermenting (NF) Gram-negatives such as Acinetobacter baumannii and Pseudomonas aeruginosa are an additional and increasingly recognized carbapenem resistance reservoir [8, 9]. Of particular concern among CROs are the subset of carbapenemase-producing organisms (CPOs), for which carbapenem resistance is generally plasmid-mediated and can transfer between organisms and across bacterial species. CPOs have been implicated in high-profile healthcare-associated outbreaks [10] and may be associated with poorer clinical outcomes than non-CPOs [11].

Admission screening for CRO and/or CPO carriage enables prompt implementation of isolation precautions for colonized patients and may provide an opportunity for individualized care, such as targeted empiric antibiotic therapy [12–14]. The Centers for Disease Control and Prevention (CDC) recommends CRE colonization screening in limited instances [15], but most U.S. hospitals do not perform routine CRE or CRO screening. Given limited data on inpatient colonization prevalence in non-outbreak periods and current limitations of CRO and CPO diagnostics, universal screening remains impractical for many acute care settings. Yet, recent CRE data indicate that existing targeted screening policies (e.g., for recent foreign hospitalization, on direct transfer from outside facilities) miss many colonized patients [16, 17].

Better identifying predictors of colonization, and developing algorithms to predict colonization probability, may improve targeted screening approaches. Existing strategies often rely on risk factors (e.g., “independent” variables), but strong risk factors may not necessarily be good predictors. Our objective was to measure the prevalence of CRO and CPO perirectal colonization at hospital unit admission, and to develop machine learning-derived decision trees to predict patients’ probability of organism carriage.

METHODS

Study Setting and Population

This study included patients aged ≥ 16 years admitted to the Johns Hopkins Hospital (JHH) medical intensive care unit (MICU) or solid organ transplant (SOT) unit between July 1, 2016 and July 1, 2017. Both units have longstanding vancomycin-resistant Enterococcus (VRE) surveillance programs and collect patient perirectal Eswabs (COPAN Diagnostics, Murrieta, CA) at unit admission (defined as ≤ 2 calendar days from unit entry) and weekly thereafter. This study was approved by the Johns Hopkins University School of Medicine Institutional Review Board, with a waiver of informed consent.

Microbiology Methods and Outcome Definitions

Residual Amies media from Eswab collection vials was stored at 4°C and, within 4 days of swab collection, 100 μl was streaked for isolation onto a MacConkey agar with ertapenem and meropenem disks [18]. Colonies growing within 27 mm of ertapenem and 32 mm of meropenem were identified by matrix-assisted laser-desorption ionization time-of-flight mass spectrometry (MALDI-TOF MS; Bruker Daltonics). Carbapenem antimicrobial susceptibility testing (ertapenem, meropenem and imipenem) was performed by disk diffusion applying Clinical and Laboratory Standards Institute guidelines [19].

Enterobacteriaceae resistant to ertapenem, meropenem, or imipenem were categorized as CRE. Glucose NF Gram-negative bacilli resistant to meropenem and/or imipenem were categorized as NFCROs. Stenotrophomonas maltophilia was excluded due to its intrinsic carbapenem resistance. All CROs were tested for carbapenemase production by the modified carbapenem inactivation method (mCIM) [20]. CRE and NFCROs positive for carbapenemase production by the mCIM test were defined as CP-CRE and CP-NFCROs, respectively (collectively, CPOs). mCIM-negative isolates were defined as non-CP-CRE and non-CP-NFCROs. CPOs underwent molecular carbapenemase genotype testing using the Check-MDR CT103XL assay (CheckPoints™, Wageningen, Netherlands).

We coded all study laboratory data with a study identifier. We linked laboratory results and clinical data six months after sample collection or following patient discharge. Neither infection control nor clinical staff were aware of patients’ colonization status during the hospital admission.

Clinical Data Collection

Patient data were retrospectively collected using bulk extraction methods from JHH’s electronic medical records (EMR) system, infection control, and administrative databases. EMR data were available for inpatient and outpatient encounters across five Johns Hopkins Health System hospitals across Maryland and the District of Columbia. Extracted patient data included more than 125 variables capturing demographic, pre-existing medical condition, procedure, medication, and other clinical data (Table 1; and Supplemental Material).

Table 1.

Characteristics of Patients in the Medical Intensive Care Unit (MICU) and Solid Organ Transplant (SOT) Unit with Carbapenem-Resistant Organism (CRO) and Carbapenemase-Producing Organism (CPO) Perirectal Colonization at Unit Admission

Variables at or Preceding Unit Admissiona	Total Swabbed Cohort	CRO Colonized	CPO Colonized
	n = 2878	n = 217	n = 36
DEMOGRAPHICS
Age	55 ± 15.4	59 ± 16.1**	59 ± 15.85
Female sex	1325 (46%)	99 (46%)	14 (44%)
Race
White	1317 (46%)	111 (51%)	17 (47%)
Black	1272 (44%)	81 (37%)	12 (33%)
Asian	61 (2%)	5 (2%)	1 (3%)
American Indian, Alaska Native or Native Hawaiian, or Pacific Islander	13 (0.5%)	0 (0%)	0 (0%)
Other	215 (8%)	20 (9%)	6 (17%)
Foreign Permanent Residence	29 (1%)	4 (1.8%)	1 (3%)
ENCOUNTER-LEVEL CHARACTERISTICS
Admission type
Emergency/urgent (non-trauma)	2646 (92%)	210 (97%)	35 (97%)
Trauma	26 (1%)	1 (0.5%)	1 (3%)
Non-urgent/elective	206 (7%)	6 (3%)	0 (0%)
Admission source
ER/Community	2353 (82%)	154 (71%)	27 (75%)
Acute care hospital, direct transfer	434 (15%)	46 (21%)**	7 (19%)
Post-acute care facility (non-acute), direct transfer	74 (3%)	16 (7%)***	2 (6%)
Other/unknown	17 (0.6%)	1 (0.5%)	0 (0%)
ELIXHAUSER COMORBIDITY SCORE AND SELECT PRE-EXISTING MEDICAL CONDITIONS
Elixhauser Score, median (IQR)	4 (2 – 7)	5 (3–7)	5 (3.5–6)
Chronic peptic ulcer disease	81 (3%)	10 (5%)	2 (6%)
Solid tumor without metastasis	468 (16%)	41 (19%)	9 (25%)
Metastatic cancer	197 (7%)	24 (11%)	4 (11%)
Renal failure	1164 (40%)	89 (41%)	14 (39%)
Liver disease	852 (30%)	55 (25%)	13 (36%)
Diabetes	912 (32%)	82 (38%)*	11 (31%)
Iron-deficiency anemia	1203 (42%)	103 (48%)	20 (56%)
Chronic pulmonary disease	630 (22)	49 (23%)	7 (19%)
Paralysis	68 (2%)	12 (6%)***	3 (8%)**
Human Immunodeficiency Virus positive	159 (6%)	8 (4%)	1 (3%)
Immunosuppressed1	772 (27%)	69 (32%)	15 (42%)
INDWELLING HARDWARE OR EXTERNAL DEVICES AT ADMISSION	887 (31%)	89 (41%)**	17 (47%)
Central line2	393 (14%)	42 (19%)*	10 (28%)**
Urologic catheter	631 (22%)	55 (25%)	11 (31%)
Mechanical ventilation	207 (7%)	22 (10%)	1 (3%)
Gastrointestinal upper or lower tube	122 (4%)	10 (5%)	0 (0%)
Fecal management device	8 (0.3%)	2 (0.9%)	1 (3%)
Ostomy pouching system	1 (0.03%)	1 (0.5%)	1 (3%)
INDWELLING HARDWARE OR EXTERNAL DEVICES (< 3 MONTHS)	1148 (40%)	112 (52%)***	24 (67%)**
Central line2	569 (20%)	58 (27%)**	16 (44%)***
Urologic catheter	876 (30%)	76 (35%)*	16 (44%)*
Mechanical ventilation	324 (11%)	39 (18%)*	8 (22%)*
Gastrointestinal upper or lower tube	189 (7%)	27 (12%)**	3 (8%)
Fecal management device	0 (0%)	0 (0%)	0 (0%)
Ostomy pouching system	13 (0.5%)	9 (4%)***	2 (6%)***
INFECTION CONTROL CHARACTERISTICS AT ADMISSION
On Contact Precautions at Admission3	796 (28%)	100 (46%)***	20 (56%)*
Admission Swab Positive for VRE Colonization	315 (11%)	51 (24%)***	12 (33%)***
RECENT MULTIDRUG-RESISTANT ORGANISM HISTORY (COLONIZATION OR INFECTION <6 MONTHS)
Vancomycin-resistant Enterococcus species.	274 (10%)	39 (18%)***	12 (33%)***
Methicillin-resistant Staphylococcus aureus	168 (6%)	28 (13%)***	7 (19%)***
Extended-spectrum β-lactamase (ESBL) or ceftriaxone-resistant Enterobacteriaceae	107 (4%)	30 (14%)***	9 (25%)***
Carbapenem-resistant organism (CRO)	74 (3%)	34 (16%)***	14 (39%)***
Carbapenem-resistant Enterobacteriaceae (CRE)	11 (0.4%)	7 (3%)***	6 (17%)***
Carbapenem-resistant glucose non-fermenting bacilli (NFCRO)	64 (2%)	27 (12%)***	8 (22%)***
Multidrug-resistant Pseudomonas species4	28 (1%)	12 (6%)***	2 (6%)***
Multidrug-resistant Acinetobacter species4	41 (1%)	9 (4%)***	4 (11%)***
RECENT MEDICATION EXPOSURE (< 3 MONTHS)
Immunosuppressive therapy5	620 (22%)	63 (29%)*	14 (39%)
Gastric Acid Suppressants6	611 (21%)	76 (35%)***	17 (47%)**
RECENT ANTIBIOTIC EXPOSURE (<3 MONTHS)
Extended-spectrum penicillin therapy	313 (11%)	43 (20%)***	12 (33%)**
Third and fourth-generation cephalosporin therapy	379 (13%)	37 (17%)	9 (25%)
Aztreonam therapy	21 (0.7%)	6 (3%)**	1 (3%)
Carbapenems	128 (4%)	30 (14%)***	8 (22%)***
Fluoroquinolone therapy	144 (5%)	21 (10%)**	4 (11%)
Aminoglycoside therapy	49 (2%)	14 (7%)***	2 (6%)
Any antibiotics (combined)	607 (21%)	72 (33%)***	14 (39%)
DURATION OF TIME FROM HOSPITAL ADMISSION TO UNIT ADMISSION (DAYS), MEDIAN (IQR)	0 (0 – 1)	0 (0–2)***	0 (0–4.5)***
RECENT INTERNATIONAL EXPOSURE
International Hospitalization (1+ nights, < 6 Months)	30 (1%)	4 (2%)	1 (3%)
International travel, patient or spouse (< 21 days)	18 (0.6%)	3 (1%)	2 (6%)***
OTHER HIGH-RISK HEALTHCARE EXPOSURES (<6 MONTHS)
Inpatient hospitalization	1553 (54%)	132 (61%)*	21 (58%)
Intensive care unit	503 (18%)	60 (28%)***	12 (33%)*
Post-acute care facility	173 (6%)	32 (15%)***	7 (19%)**
Long-term acute care hospital	34 (1%)	8 (4%)**	0 (0%)
Skilled nursing or rehabilitation facility	153 (5%)	29 (13%)***	7 (19%)***
INVASIVE PROCEDURES (< 3 MONTHS)
Endoscopy	330 (12%)	41 (19%)**	6 (17%)
Lower endoscopy	93 (3%)	12 (6%)	2 (6%)
Upper endoscopy	302 (11%)	33 (15%)*	6 (17%)
Bronchoscopy	56 (2%)	3 (1%)	0 (0%)
Surgery	306 (11%)	16 (7%)	4 (11%)
Colorectal surgery	6 (0.2%)	1 (0.5%)	1 (3%)**
Abdominal surgery	282 (10%)	14 (7%)	3 (8%)
Urologic surgery	22 (0.8%)	1 (0.5%)	0 (0%)

^aTable 1 does not include all variables and permutations evaluated in prediction models.

^*Significant at a P-value of ≤ 0.05 (*), ≤ 0.01 (**), or ≤ 0.001 (***), based upon a 2-tailed significance test, in univariable logistic regression with general estimating equations and robust standard errors to account for patient-clustering due to repeat unit admissions.

¹Receipt of chemotherapy or immunosuppressive therapy in the prior 3 months, human immunodeficiency virus (HIV)-positive, and/or documented CBC immunosuppressive abnormalities within 24 hours preceding unit admission (defined as absolute neutrophil counts or total WBC counts less than 500 cells/mm³).

²Defined in reference to the National Healthcare Safety Network (NHSN) 2018 definition of “central line,” available at: https://www.cdc.gov/nhsn/pdfs/pscmanual/pcsmanual_current.pdf.

³Indications for contact precautions are a flagged history of: (1) methicillin-resistant Staphylococcus aureus (MRSA); (2) Vancomycin-resistant Enterococcus (VRE); (3) Clostridioides difficile; (4) Multidrug-resistant Gram-negative (MDRGN) bacteria; (5) CRE (which are classified separately from other MDRGNs at JHH); (6) Respiratory viruses; and (7) Other indications, including “CRE rule-out” for patients recently hospitalized internationally (≤ 6 mos.), enteric pathogens, and contact precautions without associated infection control flag(s).

⁴Resistant to 4 of 5 antibiotic classes tested.

⁵Immunosuppressant or non-topical glucocorticoid.

⁶Proton-pump inhibitors (PPIs) or histamine H2-receptor antagonists (H2-Blockers). These medications were analyzed as a composite category in logistic regression, but were evaluated both individually and as a composite category in predictive models.

Statistical Methods

Data Analysis and Logistic Regression.

Descriptive statistics for patient variables were calculated using mean (standard deviation [SD]), median (interquartile range [IQR]), or frequency count (percentage), as appropriate, with Clopper-Pearson binominal 95% confidence intervals (CIs) for proportions. We compared CRO colonization at admission among MICU patients and SOT unit patients using Fisher’s exact test. The relationship between each covariate and the study outcomes was evaluated using univariable logistic regression with general estimating equations and robust standard errors to account for patient-clustering due to repeat unit admissions. Descriptive and logistic regression analyses were performed in Stata, version 13.0 (StataCorp, College Station, TX).

Machine Learning-Derived Prediction Models and Validation.

Using all collected variables (134), we developed prediction models for the outcomes of CRO, CRE, and CPO colonization at unit admission. We built decision trees applying the classification and regression tree (CART) algorithm [21] using the rpart (Recursive Partitioning and Regression Trees) package, version 4.1–13. To fit our trees, we employed the Gini impurity criterion for splitting rules [22]. Ensemble-based decision tree learning methods were utilized in sensitivity analyses (Supplemental Material). All machine learning models were developed using the R statistical package (version 3.0.5). CART decision trees were internally evaluated using leave-one-out cross-validation [22]. The discrimination of all models, both original (in-sample) and cross-validated (out-of-sample), were assessed through the generation of receiver operating characteristic (ROC) curves and the calculation of C-statistics in R.

RESULTS

Study Population

There were 3,327 unit admissions during the study period: 1,796 (54%) in the MICU and 1,531 (46%) in the SOT unit. Of these encounters, 2,878 (87%), representing 2,165 unique patients, had stored perirectal admission screening swabs that were processed for CROs (Figure 1).

Figure 1.

Study flowchart of carbapenem-resistant organism (CRO) and carbapenemase-producing organism (CPO) colonization at hospital unit admission. Abbreviations: Carbapenem-resistant Enterobacteriaceae – CRE; Non-fermenter carbapenem-resistant organism – NFCRO; Carbapenemase-producing CRE – CP-CRE; Carbapenemase-producing NFCRO – CP-NFCRO.

Patient characteristics are presented in Table 1. In the six months preceding unit admission, 54% of patients had been hospitalized, 17.5% had a prior ICU stay, 6.0% had been discharged to a post-acute care facility, and 1.0% of patients had documented overnight hospitalization in a foreign country. In the prior three months, 21.1% of patients had received antibiotics with Gram-negative coverage, including 4.5% with carbapenems.

CRO and CPO Colonization Admission Prevalence

Overall, 217 swabs (7.5%; 95% CI: 6.6 – 8.5%), from 192 unique patients, tested positive for one or more CROs (Figure 1). Prevalence was higher among MICU admissions than among SOT unit admissions (9.4% vs. 5.0%, p <0.001; respective 95% CIs: 8.0 – 10.9% and 3.9 – 6.4%). Of the CRO-positive swabs, 36 (16.7%) demonstrated carbapenemase production—from 32 unique patients—yielding a CPO colonization admission prevalence of 1.3% (95% CI: 0.9 – 1.7%).

One-hundred-and-twenty-one (121) admission swabs, from 113 unique patients, were positive for CRE(s). The overall prevalence of CRE and CP-CRE perirectal colonization at admission was 4.2% (95% CI: 3.5 – 5.0%) and 0.8% (95% CI: 0.5 – 1.2%), respectively (Figure 1). Twenty percent of CRE isolates were carbapenemase-producers. One hundred and seven (107) admission swabs, from 92 unique patients, tested positive for one or more NFCROs. The overall prevalence of NFCRO and CP-NFCRO perirectal colonization at admission was 3.7% (95% CI: 3.0 – 4.4%) and 0.4% (95% CI: 0.2 – 0.7%), respectively. Eleven percent of NFCRO isolates were carbapenemase-producers. The distribution of CRE and NFCRO organisms by bacterial class and carbapenemase-production status is provided in Figure 2.

Figure 2.

Distribution of organisms by bacterial class and carbapenemase-production status (defined by a positive mCIM test), among 217 perirectal unit admission swabs positive for carbapenem-resistant organism (CRO) colonization. Twenty percent of carbapenem-resistant Enterobacteriaceae (CRE) organisms, accounting for 24 swabs from 22 unique patients, were carbapenemase-producers (CP-CREs). Twelve of 109 NFCROs (11.0%), accounting for 12 swabs from 10 unique patients, were carbapenemase-producers (CP-NFCROs). Eleven admission swabs (0.4%), all from unique patients, were co-colonized with CRE(s) and NFCRO(s). Three of these swabs possessed a carbapenemase-producing organism (CPO), but no admission swabs were CP-CRE and CP-NFCRO co-colonized.

*Aeromonas categorized with glucose non-fermenting Gram-negative bacilli for purposes of this study.

Thirty-three organisms from 32 of 36 CPO-positive swabs (one swab was co-colonized with two CP-CREs), as defined by a positive mCIM test, underwent molecular genotyping. Twenty-three of the 33 (70%) processed mCIM-positive organisms were positive for carbapenemase genes by the Check-MDR CT103XL assay. The gene distribution was as follows: KPC (12, 52%), NDM (2, 9%), VIM (1, 4%), OXA-23 (2, 9%), OXA-24 (2, 9%), OXA-48-like (1, 4%), and NDM + OXA-48-like (3, 13%) (Supplemental Table 1).

Characteristics of Patients with CRO and CPO Colonization at Unit Admission

A large proportion of exposures were associated with CRO colonization (Table 1), including ostomy within three months of unit admission, history of a CRE-positive or NFCRO-positive culture in the prior six months, carbapenem or gastric acid suppressant (proton-pump inhibitor (PPI) or histamine H2-receptor antagonist) use in the prior three months, and post-acute care facility exposure (direct-admission from a skilled nursing/rehabilitation facility, or discharge to a long-term acute care hospital or skilled nursing/rehabilitation facility in the prior six months).

Restricting to the subset of CPO-colonized patients, the preceding variables remained associated with CPO colonization (Table 1). Additional variables were also associated with CPO colonization, including colorectal surgery in the prior three months and foreign travel by the patient or a partner in the preceding 21 days. Nevertheless, only two patients with molecularly-confirmed carbapenemases, a KPC-producing K. pneumoniae and an OXA-48-like-producing K. pneumoniae, had documented foreign travel; none reported recent international hospitalization.

Both CRO- and CPO-colonized patients were significantly more likely than non-carriers to be on contact precautions at unit admission, although overall only 46.1% of CRO carriers were on contact precautions at unit admission. CRO and CPO carriers were also more likely to test positive for VRE colonization during admission screening, i.e., on the same swab that underwent CRO processing. These VRE-colonized patients accounted for 24% and 33% of CRO and CPO colonizations, respectively.

Predicting Probability of Colonization at Unit Admission

We evaluated all collected study variables, including permutations (e.g., varied lookback periods, composite and individual variable categories), for inclusion in decision tree models. These machine learning approaches are well-suited to large EMR datasets because they can accommodate high predictor-to-outcome ratios, variable collinearities, and interaction effects by default [21, 23]. By using branching logic rather than calculations, decision trees are also relatively user-friendly for manual bedside use. We derived models for three alternate outcomes: CRO, CPO, and CRE (Supplemental Material) colonization.

The final decision tree for predicting CRO colonization at unit admission included three study variables (Figure 3). The first question in the tree (“root node”), which is reserved for the most discriminatory variable, asked (1) Did the patient have a CRO-positive culture in the previous six months? If “yes,” the second question queried (2) Did the patient receive 26 or more days (model-derived cut-point) of PPIs in the prior three months? Patients meeting these criteria were classified as CRO-positive with 93% probability (Terminal node 4). In patients with a CRO history but lacking this PPI exposure, the tree questioned (3) Has the patient been hospitalized for 51 or more days (model-derived cut-point) in the prior six months? If “yes,” patients were classified as CRO-positive (Terminal node 3, 80% probability) and if “no” as CRO-negative (Terminal node 2, 74% probability).

Figure 3.

Decision tree for predicting CRO perirectal colonization at hospital unit admission. Gray-shaded terminal nodes indicate that the tree would classify patients as CRO-colonized, and accompanying percentages reflect the probability that patients assigned to a given terminal node are CRO-positive. Terminal node numbering, 1 through 4, is included in parentheses. The tree had an area-under-the-curve (C-Statistic) of 0.57, which was unchanged in cross-validation. Its sensitivity and specificity were 9.8% and 99.9%, respectively, and its positive and negative predictive values were 87.5% and 93.1%, respectively.

For the 2804 patients lacking a recent CRO history, the root node branched left and terminated. Patients lacking this history were classified as CRO-negative (Terminal node 1, 93% probability).

The overall tree possessed a sensitivity of 9.8% and a specificity of 99.9%. The positive and negative predictive values were 87.5% and 93.1%, respectively. Incorporating outcome probabilities based on terminal node impurities, the C-statistic for the final tree trained on the full dataset was 0.57 and unchanged following cross-validation.

The CPO decision tree truncated at a single variable, history of a CRE-positive culture in the prior six months (Figure 4). Its sensitivity was 16.7%, and its specificity was 99.8%. The CPO tree’s discrimination was 0.58 (unchanged following cross-validation).

Figure 4.

Decision tree for predicting CPO perirectal colonization at hospital unit admission. The gray-shaded terminal node indicates that the tree would classify patients as CPO-colonized, and accompanying percentages reflect the probability that patients assigned to a given terminal node are CPO-positive. Terminal node numbering, 1 through 2, is included in parentheses. The tree had an area-under-the-curve (C-Statistic) of 0.58, which was unchanged in cross-validation. Its sensitivity and specificity were 16.7% and 99.8%, respectively, and its positive and negative predictive values were 54.5% and 99.0%, respectively.

To optimize model performance and address possible outcome misclassification, we performed multiple sensitivity analyses: 1) Built prediction models for CRO and CPO colonization with random forests analysis; 2) Refit CART trees to increase sensitivity by imposing a greater “cost” for misclassifying colonized patients as negative; 3) Refit CART trees restricting to first, unique-patient encounters (n=2165); and 4) Re-performed CART and random forests analyses after restricting the CPO outcome to isolates with molecularly-confirmed carbapenemases. With more complicated models in sensitivity analyses 1 and 2, performance improved by approximately 15–20%; performance was similar in analyses 3 and 4 to primary analyses. Results are provided and discussed in the Supplemental Material.

DISCUSSION

Identifying CRO- and CPO-colonized patients at hospital unit admission could facilitate timely infection control interventions, such as implementing prompt contact isolation precautions for colonized patients, in order to limit healthcare-associated transmission. Evaluating patients admitted to MICU and SOT units, we found that 7.5% and 1.3% of patients were peri-rectally colonized with CROs and CPOs, respectively. Among CROs, the distribution of CRE versus NFCROs was roughly similar (54% vs. 46%), with a CRE admission prevalence of 4.2%. This estimate is higher than the proportion of CRE (3.1%) among clinical isolates reported to the National Healthcare Safety Network in 2015 [24], and considerably higher than the 0.5% CRE admission prevalence recently reported at a Chicago tertiary-care hospital (2013 data from ICU populations) [16]. Importantly, the majority (54%) of colonized patients were not on contact precautions at unit admission (for any indication), posing a potential reservoir for transmission during their unit encounter.

Our study included many variables known to be risk factors for CRO and CPO colonization or infection, including MDRO history [25–27], antibiotic exposure overall [28–30] or to carbapenems specifically [31, 32], post-acute care facility stay [33, 34], immunosuppression [28], endoscopy [30, 31, 35], and indwelling hardware [28, 33, 36]. Despite including these known risk factors and more than one hundred other variables, our constructed models did not highly predict CRO and CPO colonization, with C-statistics of 0.57 and 0.58, respectively. Performance improved by approximately 15–20% in sensitivity analyses, but with more complicated models that may be less likely to replicate in other settings and which would be less practical as bedside tools. Despite sub-optimal global performance, however, the CRO decision tree did, with high accuracy, identify certain higher-colonization risk patient populations: patients with recent CRO-positive cultures (≤ 6 months) who had either greater than 26 days of PPI usage in the prior 3 months (93% colonization probability) or greater than 51 days of hospitalization in the prior 6 months (80% colonization probability). This observation comports with recent studies identifying PPI or other gastric acid suppressant use as a significant risk factor for MDRGN carriage [37, 38]. Using these criteria for targeted surveillance would capture 21 of 217 colonized patients while only producing three false-positive screening referrals. Although recent CRO-positive culture combined with either PPI usage or prior hospitalization were highly predictive, however, these criteria would still miss 196 CRO-colonized patients (90%) who did not have these characteristics.

Interestingly, only one CPO-colonized patient had documented recent international hospitalization, the current CDC-recommended exposure for targeted CRE screening [39]. Moreover, although CPO-colonized patients were significantly more likely than non-colonized patients to report foreign travel of themselves or a partner within the 21 preceding days, this variable did not emerge as a strong predictor in decision tree models (likely due to the few patients, only 0.6%, with this exposure).

This study highlights key challenges that may make predicting patients’ CRO/CPO colonization status, and in turn implementing successful targeted screening algorithms, difficult. First, although risk factors are important explanatory variables from an etiologic perspective and can highlight where we may intervene to prevent an outcome, these variables reflect relative risk, not absolute risk; risk factors are not necessarily good at predicting (i.e., distinguishing between) who does or does not have an outcome, particularly where the number of affected patients is small. For example, although a recent CRO-positive culture was a strong risk factor (p < 0.001) for CRO colonization at admission, it only accounted for 34 of 217 cases. Eighty-four percent of CRO-colonized patients did not have a recent CRO-positive culture, and for the majority of our cohort, this variable would therefore not be helpful for predicting CRO status at admission. Second and similarly, high bacterial and genomic diversity among colonizing isolates may contribute to difficulty in predicting carriage by increasing outcome heterogeneity. In particular, CPOs reflected considerable organism and carbapenemase diversity, with 1/3 of CP-CREs encoding carbapenemases other than KPCs, including two genes (bla_NDM and bla_OXA-48-like) in a single organism. Third, although we collected extensive EMR data on healthcare-associated exposures, poor model sensitivity may reflect limitations of EMR data and not absence of true predictive characteristics, particularly if colonization acquisition predated available lookback periods. Finally, although risk factors for CRE and other CROs in U.S. patients have traditionally focused on healthcare settings, increasing reports describe community reservoirs of carbapenem resistance (e.g., porcine farms, retail seafood) [40, 41]. These non-traditional exposures are unlikely to be systematically captured in the EMR.

Notwithstanding these challenges, our results offer actionable information. Recent CRO- or CRE-positive culture was consistently the strongest predictor of admission colonization, and many infection control programs already capture and flag these cultures. Moreover, 24% and 33% of CRO- and CPO-colonized patients, respectively, were co-colonized with VRE detected during routine admission screening. These patients would be placed on contact isolation precautions even without dedicated CRO surveillance. These findings suggest that existing screening policies may have unrecognized benefits, and may justify continued surveillance and/or contact precautions for endemic VRE colonization [42, 43].

Our study has several limitations. First, this was a single-center study, and although we internally validated our models, our results should be validated in other cohorts. Our results may not be generalizable to other, lower-risk hospitalized populations or higher-endemicity areas (e.g., New York City). Second, concordance between phenotypic and molecular carbapenemase assays was lower than expected [20], particularly for E. cloacae; further whole genome sequencing is planned to clarify this discrepancy. Nevertheless, sensitivity analyses restricting the CPO outcome to molecularly-confirmed isolates yielded similar findings. Third, despite gathering extensive demographic and clinical information, there was likely missing exposure data (e.g., outpatient antibiotic use, data that does not interface across hospitals). Many exposures, however, were strongly associated with study outcomes, consistent with other published literature. More importantly, because the prediction models were designed to inform real-world screening decisions, their performance under the practical constraints of potentially incomplete EMR data is arguably relevant.

Overall, in this high-risk inpatient population CRO and CPO carriage was infrequent but higher than previously published estimates, including from other U.S. ICU populations. There was significant organism and resistance mechanism diversity. We molecularly identified carbapenemases in seven different bacterial species, providing an important reminder that many GI-colonizing organisms can serve as carbapenemase gene reservoirs. Despite including many patient characteristics associated with colonization or infection in the literature, overall, neither our machine learning-derived models nor current CDC targeted screening criteria (i.e., recent foreign hospitalization) were highly accurate in predicting whether patients were colonized at admission. An important goal of artificial intelligence and other machine learning applications in healthcare is to capitalize on ‘Big Data,’ despite its imperfections, to improve patient outcomes. Our study demonstrated that currently available EMR data did not meet these targets. We believe this was attributable, in part, to high exposure and microbiological heterogeneity, raising questions about how useful targeted screening strategies will be to identify CRO-colonized patients. Our models did successfully identify certain patient sub-groups who had high probabilities of colonization, however, including those with a recent history of CRO-positive culture(s) who use PPIs. There may be utility in expanding upon existing CDC criteria to include other high-risk sub-groups as efforts continue to optimize CRE and CRO screening policies.