Skip to main content
NCBI home page
Antimicrobial Agents and Chemotherapy logoLink to Antimicrobial Agents and Chemotherapy
. 2022 Jun 21;66(7):e00676-22. doi: 10.1128/aac.00676-22

Validation of Clinical Risk Models for Clostridioides difficile-Attributable Outcomes

Gregory R Madden a,✉, William A Petri Jr a, Deiziane V S Costa a, Cirle A Warren a, Jennie Z Ma b, Costi D Sifri a,c
PMCID: PMC9295569  PMID: 35727061

ABSTRACT

Clostridioides difficile is the leading health care-associated pathogen, leading to substantial morbidity and mortality; however, there is no widely accepted model to predict C. difficile infection severity. Most currently available models perform poorly or were calibrated to predict outcomes that are not clinically relevant. We sought to validate six of the leading risk models (Age Treatment Leukocyte Albumin Serum Creatinine (ATLAS), C. difficile Disease (CDD), Zar, Hensgens, Shivashankar, and C. difficile Severity Score (CDSS)), guideline severity criteria, and PCR cycle threshold for predicting C. difficile-attributable severe outcomes (inpatient mortality, colectomy/ileostomy, or intensive care due to sepsis). Models were calculated using electronic data available within ±48 h of diagnosis (unavailable laboratory measurements assigned zero points), calibrated using a large retrospective cohort of 3,327 inpatient infections spanning 10 years, and compared using receiver operating characteristic (ROC) and precision-recall curves. ATLAS achieved the highest area under the ROC curve (AuROC) of 0.781, significantly better than the next best performing model (Zar 0.745; 95% confidence interval of AuROC difference 0.0094–0.6222; P = 0.008), and highest area under the precision-recall curve of 0.232. Current IDSA/SHEA severity criteria demonstrated moderate performance (AuROC 0.738) and PCR cycle threshold performed the worst (0.531). The overall predictive value for all models was low, with a maximum positive predictive value of 37.9% (ATLAS cutoff ≥9). No clinical model performed well on external validation, but ATLAS did outperform other models for predicting clinically relevant C. difficile-attributable outcomes at diagnosis. Novel markers should be pursued to augment or replace underperforming clinical-only models.

KEYWORDS: outcome, disease severity, Clostridioides difficile, disease severity, prediction model, risk model

INTRODUCTION

Clostridioides difficile infection (CDI) continues to be an often morbid or lethal condition. Despite available treatments, up to a third of hospitalized cases will require intensive care, 3% a colectomy, and 6% will die (1). Evidence suggests that earlier interventions such as colectomy (2) and promising new treatments (e.g., fecal microbiota transplant, monoclonal antibodies [3], SER-109 [4], antisense antibiotics [5]) may prevent severe outcomes and death in selected patients; however, clinicians and researchers lack a valid and reliable method to risk stratify patients for these interventions early, at the time of diagnosis. Conversely, identifying patients at very low risk for serious adverse outcomes could help the significant issue of C. difficile overdiagnosis and overtreatment (6, 7).

A variety of published outcome models (Table 1) have been derived within small populations (e.g., Belmares et al., 102 patients at 1 hospital) (8), utilize trial cohorts with exclusion criteria that limit generalizability (e.g., ATLAS) (9), use variables occurring after severe outcomes have begun to develop (e.g., Im et al., imaging findings up to a week after diagnosis) (10), and/or were calibrated to predict outcome measures that are nonspecific to CDI (e.g., Kulaylat et al., all-cause mortality (1)). The few models that are validated at multiple institutions do not perform well on external validation (11–14). In addition, simplified clinical factors or laboratory cutoffs based on expert opinion including those recommended by the current Infectious Disease Society of America and Society for Healthcare Epidemiology of America (IDSA/SHEA) consensus guidelines (15, 16) do not perform well either (17), especially in patients with disrupted immunity (e.g., neutropenia) or kidney disease (18). As a result, no single model to predict severe outcomes of C. difficile infection has arisen as clearly superior or gained widespread acceptance (15). The primary objective of our study was to evaluate the performance of the leading clinical risk prediction models available for C. difficile infection in hospitalized patients.

TABLE 1.

Existing models for C. difficile infection outcomes and reported area under the receiver operating characteristic curve (AuROC)a

Study (model name) C. difficile outcome(s) Derivation cohort size Single or multicenter AuROC
Belmares et al. (CDD)b ( 8 ) Resolution of diarrhea 102 Single 0.89
Im et al. (10) Inpatient mortality 396 Single 0.87
Kulaylat et al. (1) All-cause mortality 2065 Multicenter 0.82
Lungulescu et al. (CSI)b ( 51 ) All-cause mortality, ICU admission, >10-day hospital stay, or colectomy. 255 Single 0.80
Zilberberg et al. (52) 30-day all-cause mortality 278 Single 0.74
Archbald-Pannone et al. (53) 30-day attributable mortality 362 Single 0.74
Hensgens et al.b ( 16 ) Complicated (all-cause mortality, prolonged ICU stay, or colectomy) 395 Multicenter 0.73
Shivashankar et al. ( 17 ) Severe/complicated (hypotension, shock, sepsis) 1,146 Single 0.71
Miller et al. (ATLAS)b ( 9 ) Response to therapy (“cure”) 1,164 Multicenter 0.71
Li et al. (54) Mortality, ICU admission, or colectomy 1,118 Multicenter 0.69
Na et al. (CDSS)b ( 18 ) Contributable mortality or ICU admission, toxic megacolon, or colectomy 263 Multicenter 0.66 (32)
Zar et al.b ( 15 ) Cure, treatment failure, relapse 150 Single 0.66 (31)
Kassam et al. (CARDS) (55) All-cause in-hospital mortality 77,776 (administrative-only database) Multicenter 0.66
Butt et al. (56) All-cause mortality 213 Single 0.65
Toro et al. (SSI)b (57, 58) Inpatient mortality and/or ICU admission 51 Single 0.64 (31)
Hu et al.b (59) Recurrence 63 Single 0.62
van der Wilden et al. (RSS) (60) 30-day all-cause mortality, ICU admission, or colectomy 746 Single 0.57 (32)
Drew et al. (RUWA)b (61) Mortality, ICU admission, pancolitis on imaging, or colectomy 81 Single Not calculated
Neal et al.b (62) Clinical resolution (of symptoms and WBC) 49 Single Not calculated
Bauer et al. (Hines VA) (63) Treatment failure 1,105 Multicenter Not calculated
Open in a new tab
a

ICU, intensive care unit. Models chosen for external validation shown in bold.

b

Externally validated (using retrospective data) and/or prospectively validated (in small, single-center studies).

RESULTS

Three thousand five hundred and seventy-seven hospitalized cases of C. difficile infection were identified between March 2011 and April 2021 that occurred in 2,928 individual patients (Fig. 1). After excluding cases without treatment, age < 18 years, or > 5 recurrent episodes, the final cohort consisted of 3,327 cases in 2,752 individual patients. Baseline characteristics of our cohort were compared to the validation cohorts for the six clinical models (Table 2). Two hundred sixty-two of 3,327 (7.9%) of cases met one or more of the CDI-attributable primary composite outcomes. Clinician reviews indicated that 130/192 (67.7%) of deaths were attributable to CDI and 22/31 (71.0%) of colectomies or ileostomies were due to CDI. 139/295 (47.1%) of transfers to an ICU following CDI diagnosis were due to sepsis according to the validated definition (19). Baseline lab measurements were not available for 356/3,327 (10.7%) creatinine, 378/3,327 (11.4%) white blood cell count (WBC), and 563/3,327 (16.9%) albumin, measurements. Patients with one unavailable measurement at the time of diagnosis were less likely to develop a severe CDI-attributable outcome compared to patients with all measurements (21/609 [3.4%] versus 201/1444 [8.9%]).

FIG 1.

FIG 1

Open in a new tab

C. difficile infection cohort and composite severe adverse CDI-attributable outcomes. Some patients met >1 exclusion criteria and outcomes were coded as having met ≥1 composite outcome but >1 composite outcomes occurred in some patients.

TABLE 2.

Characteristics of UVA Medical Center cohort and derivation cohorts for C. difficile risk scoresa

Characteristic (cohort size) UVA (n = 3,327) ATLAS (9)
(n = 1164)d 		Zar (15)
(n = 150)		 Hensgens (16)
(n = 395)		 CDSS (18)
(n = 263)		 CDD (8)
(n = 102)		 Shivashankar (17) (n = 1446)
C. difficile cohort setting (yr) Adult (≥18 yr) inpatients at 645-bed tertiary care hospital, Charlottesville, VA (2011–2021). Adults (≥16 yr) enrolled in 2 clinical trials: 62 sites in US/Canada (2006–2008) (33), 86 sites in US/Canada/Europe (2007–2009) (34), total ~36% outpatients. Adult (≥18 yr) inpatients at 200-bed acute care hospital in Chicago, IL (1994–2002). Prospective inpatients of any age admitted to 9 centers in The Netherlands (2006–09). Adult (≥18 yr) prospective inpatients at 1 tertiary care cohort in Boston, MA (2004–06). Inpatients of any age admitted to a 472-bed tertiary care Hines Veterans Affairs hospital in Chicago, IL (2003–04). 1 center retrospective cohort in Rochester MN (2007–2010).
Model development; validation External Validation Cohort Multivariable Logistic Regression derived on first trial cohort that was validated on the second. Final validation on the pooled cohort (n = 1,105). Ad hoc score developed to stratify disease severity for randomized clinical trial. Multivariable Logistic Regression; bootstrapping (n = 200) and shrinkage factor. External validation at 1 center (2009–2011). Multivariable Logistic Regression; Validated using cohorts in Dublin, Ireland (2007–09) and Houston, TX (2006–2010). Empiric score developed based on variables described in the literature, validated using cohort. Multivariable logistic regression; internal validation.
Age (mean) 63.0 62.5 58.5 65.0 (median) 66.5 68.25 62.5 (median)
Male sex 1,686 (50.7) 462/1105 (41.8) 82 (54.6) 220 (56) 131 (49.8) NA 220 (56)
Charlson comorbidity index 1.78 (mean) NA NA 3.28 (mean) 1.68 (mean) NA
 0 1,121 (33.7) 59 (14.9)
 1–2 963 (28.9) 150 (40.0)
 3–4 659 (19.8) 120 (30.4)
 ≥5 584 (17.6) 64 (16.2)
Antibiotic exposureb (<2 mo) 2830 (85.1) NA 150 (100) 336 (85.0) NA NA 614 (42.5) before or after CDI
 Fluoroquinolones 624 (18.8)
 Cephalosporins 1021 (30.7)
 Carboxy/ureidopenicillins 921 (27.7)
 Macrolides/clindamycin 315 (9.5)
 Antistaphylococcal/aminopenicillins 614 (18.4)
Acid suppression (<2 mo) 2,246 (67.5) NA NA 198 (75.2) NA NA
 PPI 1,579 (47.5) 251 (63.5)
 H2-RA 1212 (36.4)
 PPI + H2-RA 545 (16.4)
Immunosuppression  (<6 mo) 721 (21.7) NA NA 172 (43.5) 117 (44.5) NA NA
CDI diagnosis method All patients had
+ Toxin EIA or
+ Cytotoxicity Assay		 All patients had
+ Toxin EIA or
+ Cytotoxicity Assay
 PCR 3251 (97.7) 0 (0) 0 (0) 0 (0) 1446 (100)
 Toxin EIA 76 (2.3) 150 (100) 263 (100) 102 (100) 0 (0)
 Cytotoxicity assay 0 (0) 0 (0) 0 (0) 0 (0) 0 (0)
NHSN classification 746 (64.0) inpatient; 418 (35.9) outpatient NA NA NA
 Hospital onset 1,745 (52.4) 283 (71.6) 25 (24.5) community-onset
 Community onset-HCFA 526 (15.8)
 Community-acquired 1,056 (31.7)
Primary CDI episode 2,740 (82.4) 978 (84.0) NA NA 221 (84.0) 99 (97.1) NA
Ribotype 027 (n/available) 10/97 (10.3)e 292/814 (35.9) NA 17/207 (8.2) 60% of isolates NA
Treatment NA
 Metronidazole (PO or IV) 1,147 (34.5) 0 (0.0) 79 (52.7) 293 (74.2) 247 (93.9) 93 (91.2)
 Vancomycin 1,023 (30.7) 592 (50.9) 71 (47.3) 10 (2.5) 9 (8.8)
 Vancomycin + Metronidazole 1,150 (34.6) 0 (0.0) 0 (0) 47 (11.9) 0 (0)
 Fidaxomicin 7 (0.2) 572 (49.1) 0 (0) 0 (0) 0 (0)
 No Treatment 0 (0.0) 0 (0.0) 0 (0) 47 (11.9) 0 (0)
CDI outcomes
 ICU admission 295 (8.9) NA NA NA NA NA 386 (2.7)
 ICU admission for sepsisc and/or CDI complications 139 (4.2) NA NA 3 (0.8) 32 (12.2) NA NA
 Colectomy/ileostomy 22 (0.7) NA 0 (0) 5 (1.3) 9 (3.4) 31 (2.7)
 30-day all-cause mortality 249 (7.5) 37/623 (5.9) 8 (5.3) 65 (16.5) NA 6 (5.9) 102 (8.9)
 90-day all-cause mortality 389 (11.7%)
 CDI attributed mortality 130 (3.9) NA NA 38 (9.9) 11 (4.2) 2 (2.0) NA
 Complicated CDI outcome (as defined by study) 262 (7.9) 77 (6.6) 14 (9.3) 46 (11.9) NA 9 (8.8) 487 (33.7)
Open in a new tab
a

Data presented as n/total (%) or n/available (%). SD, standard deviation; NA, not available; approximately, approximate; mo, months; PPIs, proton pump inhibitors; H2-RA, Histamine type-2 receptor antagonists; EIA, enzyme immunoassay; GDH, glutamate dehydrogenase; HCFA, health care facility-associated; PO, per os; IV, intravenous.

b

Each patient could have received more than one class of antimicrobials within 2 months of enrollment.

c

In the UVA cohort, defined as ICU transfer due to sepsis (based on validated definition by Rhee et al. (23)), CDI-attributable in-hospital mortality, or in-hospital CDI-attributable colectomy/hemi-colectomy/diverting ileostomy following C. difficile diagnosis.

d

For ATLAS, data for the 967/1164 patients which lacked missing variables and were used for model development were not reported; characteristics of the full cohort is shown.

e

Data from an unpublished convenience sampling of 97 C. difficile isolates from clinical stool specimens at UVA from August 2018 to April 2019 (personal communication, C.A.W.).

Median (interquartile range) scores among the 262 cases with severe attributable outcomes of CDI versus the remaining 3,065 cases for each of the models were: ATLAS (outcomes: 6 [5–7] versus not: 4 [2–5]), IDSA (2 [2–3] versus 1 [1–2]), Hensgens (3 [1–4] versus 1 [1–3]), CDSS (1.5 [1v2] versus 1 [0v1]), Shivashankar (−0.555 [−1.038 to −0.010] versus −1.020 [−1.54 to −1.02]), and CDD (2 [2–3] versus 1 [1–2]). The ATLAS Score achieved the highest AuROC of 0.781 (Fig. 2) with a maximum F1 of 0.326 at a cutoff ≥7 (Table 3). The maximal Youden Index for ATLAS (0.422) occurred at a cutoff ≥6 (Fig. 3), corresponding with a true positive rate (sensitivity) of 63.7% and a false positive (type 1 error) rate of 21.5%. ATLAS performed significantly better than the next highest AuROC (Zar 0.745; 95% confidence interval for the AuROC difference 0.0094–0.0622: P = 0.008). Next, in descending order of AuROC were the IDSA Severity Criteria (0.738), Hensgens score (0.698), CDSS (0.692), Shivashankar (0.678), and CDD (0.675). PCR cycle threshold performed the worst, with an AuROC slightly above 0.5 (0.531).

FIG 2.

FIG 2

Open in a new tab

Receiver operating characteristic (A) and precision-recall curves (B) with area under the curves (AuROC and AUC-PR, respectively) for C. difficile risk models. PCR cycle threshold (CT) data only available for 1,484/3,327 cases.

TABLE 3.

Performance comparison of scoring toolsa

Risk score Cut-off (n patients, %)b # Successes, failures Sens. (%) Spec. (%) Positive predictive value (%) Negative predictive value (%) F1 AuROC (95% CI) P (vs. ATLAS)b AUC-PR
ATLAS 0 (130, 3.9) 0.781
(0.756–0.807)		 0.232
1 (283, 8.5) 390, 2,937 99.6 4.21 8.16 99.2 0.151
2 (433, 13.0) 667, 2,660 98.5 13.3 8.85 99.0 0.162
3 (590, 17.7) 1,088, 2,239 96.2 27.3 10.2 98.8 0.184
4 (553, 16.6) 1,646, 1,681 90.1 46.0 12.5 98.1 0.219
5 (513, 15.4) 2,139, 1,188 78.6 63.1 15.4 97.2 0.258
6 (393, 11.8) 2,574, 753 63.7 78.5 20.2 96.2 0.307
7 (248, 7.5) 2,859, 468 43.1 89.6 26.2 94.9 0.326
8 (126, 3.8) 2,997, 330 22.1 95.9 31.5 93.5 0.260
9 (50, 1.5) 3,051, 276 8.40 98.8 37.9 92.7 0.138
10 (8, 0.24) 3,061, 266 0.76 99.8 25.0 92.2 0.015
Zar 0 (567, 17.0) 0.745
(0.723–0.768)		 0.008 0.192
1 (1167, 35.1) 823, 2,504 98.9 18.4 9.45 99.5 0.171
2 (833, 25.0) 1,902, 1,425 82.1 55.0 13.5 97.3 0.231
3 (469, 14.1) 2,589, 738 54.2 79.8 18.7 95.3 0.278
4 (183, 5.5) 2,908, 419 25.6 92.7 23.0 93.6 0.242
5 (87, 2.6) 3,021, 306 12.2 97.5 29.6 92.9 0.173
6 (21, 0.63) 3,058, 269 2.67 99.5 33.3 92.3 0.049
IDSA/SHEA 0 (620, 18.6) 0.738
(0.12–0.765)		 0.005 0.204
1 (1258, 37.8) 854, 2,473 94.7 19.8 9.16 97.7 0.167
2 (859, 25.8) 2,022, 1,305 77.5 59.3 14.0 96.9 0.237
3 (402, 12.1) 2,733, 594 49.2 85.0 21.9 95.1 0.303
4 (161, 4.84) 2,991, 336 21.8 95.7 30.3 93.5 0.253
5 (27, 0.81) 3,054, 273 3.05 99.4 29.6 92.3 0.055
Hensgens −1 311, 3,016 99.6 1.63 7.97 98.0 0.148 0.738
(0.712–0.765)		 <0.0001 0.158
1 759, 2,568 93.9 16.7 8.79 97.0 0.161
2 2,028, 1,299 74.0 59.8 13.6 96.4 0.230
4 2,814, 513 29.8 89.3 19.2 93.7 0.233
CDSS 0 (1148, 34.5) 0.692
(0.666–0.719)		 <0.0001 0.170
1 (1434, 43.1) 1,338, 1,989 86.3 36.3 10.4 96.9 0.185
2 (609, 18.3) 2,582, 745 50.0 80.0 17.6 94.9 0.260
3 (136, 4.1) 3,005, 322 14.5 96.8 27.9 93.0 0.191
Shivashankar −2.0 519, 2,808 98.9 8.32 8.45 98.8 0.156 0.678
(0.645–0.711)		 <0.0001 0.149
−1.4 1,124, 2,203 87.4 29.2 9.54 96.4 0.172
−0.930 1,969, 1,358 69.1 57.5 12.2 95.6 0.207
−0.4 2,058, 1,269 44.7 79.3 15.6 94.4 0.233
CDD 0 (454, 13.6) 0.675
(0.646–0.704)		 <0.0001 0.163
1 (1154, 34.7) 686, 2,641 94.3 14.3 8.6 96.7 0.158
2 (1061, 31.9) 1,752, 1,575 77.5 50.5 11.8 96.3 0.205
3 (506, 15.2) 2,609, 718 38.5 81.8 15.3 94.0 0.220
4 (134, 4.0) 2,999, 328 16.4 96.4 28.3 93.1 0.208
5 (18, 0.54) 3,059, 268 2.3 99.6 33.3 92.3 0.043
PCR cycle threshold 32.0 402, 1,082 87.2 21.2 9.82 94.4 0.177 0.531
(0.483–0.579)		 <0.0001 0.093
30.0 506, 978 81.2 29.5 10.2 94.1 0.181
28.0 614, 870 70.7 38.5 10.2 93.0 0.178
26.0 708, 776 58.6 46.6 9.76 92.0 0.167
Open in a new tab
a

Sens., sensitivity; Spec., specificity; AuROC, area under the receiver operating characteristic curve; AUC-PR, area under the precision-recall curve; CI, confidence interval.

b

Calculations based on ≥ cutoff, except for PCR cycle threshold (≤ cutoff). Cutoffs that represent the maximum Youden Index are shown in bold.

FIG 3.

FIG 3

Open in a new tab

Youden indices for C. difficile severity score cutoffs. Youden Index is equal to 0 for tests with poor diagnostic accuracy, equal to 1 for a perfect test, and assigns equal weight to sensitivity and specificity.

Two sensitivity analyses were performed: (i) excluding 609 cases without available creatinine, WBC, or albumin measurements at the time of diagnosis (Fig. 4A) and (ii) using a composite outcome without clinical attributions (all-cause inpatient mortality, all-cause ICU transfer, or all-cause colectomy/ileostomy) (Fig. 4B). Both demonstrated similar trends with the exception that Zar, not ATLAS, was the leading model for predicting unattributed outcomes (Zar AUC 0.737; ATLAS 0.718; Hensgens 0.713; IDSA/SHEA 0.693; Shivashankar 0.662; CDD 0.638; CDSS 0.635; PCR cycle threshold 0.541).

FIG 4.

FIG 4

Open in a new tab

Receiver operator curves with area under the receiver operating characteristic curve (AuROC) for C. difficile risk models using the full cohort (A) minus 609 patients with unavailable white blood cell count, creatinine, or albumin measurements and (B) an unattributed composite outcome (all-cause inpatient mortality, all-cause colectomy/ileostomy, and/or all-cause ICU transfer). PCR cycle threshold (CT) data only available for 1,484/3,327 cases.

DISCUSSION

These data support the notion that C. difficile clinical risk scores have a performance ceiling, especially early in the disease course. While ATLAS, Zar, and the IDSA Severity criteria all fell within an AuROC range ≥ 0.7-0.8 that could be considered clinically useful (20), AUC may be misleading in settings such as CDI where outcomes are unbalanced. For example, ATLAS achieved the highest maximal Youden index of 0.422 at a score cutoff of ≥6, however, the positive predictive value (20.2%) at this cutoff was strikingly low. Therefore, the clinical utility of ATLAS to predict attributable outcomes of infection remains unclear.

There are important limitations to this analysis. First, while this is one of the largest single-center cohorts used for C. difficile model validation, it does not necessarily indicate generalizability to other tertiary care settings. ATLAS did not show stable performance across one multi-institution cohort (13) and should be validated in other settings such as community hospitals or outpatient prior to widespread adoption. Second, C. difficile infection occurs inherently in patients that are medically complex (i.e., on antibiotics for other causes, immunosuppressed, multiple comorbidities) and clinician determination of attributable causes for C. difficile complications is often difficult, subjective, and lacks inter-rater reliability (21, 22). ICU transfers meeting a validated electronic definition for sepsis was therefore chosen to minimize the time and subjectivity of clinician reviews. However, this method has not been previously studied in the context of CDI and requires further validation. Third, there were a significant proportion of unavailable measurements that we chose to score with zero points as a practical and inclusive approach to test the “real world” prognostic value of each model. Interestingly, model performance appeared to improve through this imputation method versus omitting cases. Since patients with at least one unavailable measurement had nearly a third the rate of severe outcomes; we hypothesize that the decision to not check a full completement of labs at diagnosis implied an overall favorable prognosis but may have underutilized important prognostic information in some cases. Fourth, electronic-only data gathering did not allow specific components of the CDD (imaging findings) and Hensgens (admission reason) models to be included; however, this did not appear to significantly impact performance compared with other retrospective validation studies (i.e., CDD AUC at UVA: 0.675 versus AUC reported by Perry et al.: 0.620 [13]; Hensgens AUC at UVA: 0.698 versus AUC reported by Beauregard-Paultre and van Beurden et al.: 0.630–0.680 [11, 14]).

Two recent studies attempting to validate these models using smaller, single-center cohorts failed to identify a significantly superior model (11, 13); however, our data showed that ATLAS performed significantly better than other major models. ATLAS was originally derived using two pooled clinical trial cohorts for comparing fidaxomicin and vancomycin treatment (23, 24) that excluded patients with “life-threatening”/fulminant infection or toxic megacolon and approximately a third of the ATLAS cohort were outpatients (25). Despite these limitations, ATLAS appears externally valid to our relatively sicker, inpatient CDI cohort. Similarly, ATLAS performed well at hospitals in Mexico where severe outcomes were more prevalent (i.e., 17.6% 30-day all-cause mortality, 3.9% colectomies) (26). Our ATLAS AuROC was higher than that originally reported by Miller et al. (0.78 versus 0.71), which may be explained by differences in the outcome definition (ATLAS constrained to “clinical failure” defined as a lack of marked reduction of diarrhea and/or need for additional C. difficile therapy based on investigator opinion) (9).

There are several reasons why ATLAS may be superior to other models in our study. ATLAS was derived from one of the largest cohorts and combines 5 robust factors, each of which have repeatedly shown to be important predictors of severe infection in other studies (27). In addition, the ATLAS criteria can be extracted from the electronic medical record with relative ease, objectivity, and fidelity, which was not necessarily the case with the other models, which relied on imperfect billing/coding data (e.g., presence of ileus or megacolon in CDD, IDSA/SHEA) or factors that could not be extracted electronically (CDD, Hensgens).

ATLAS performed significantly worse (AuROC 0.781 versus 0.718) in the post hoc analysis to predict severe outcomes not attributable to CDI. While ATLAS and Zar criteria are very similar, ATLAS has an important feature which Zar lacks: non-CDI antibiotics during CDI treatment. Cases with non-CDI antibiotics during CDI treatment were about twice as likely to develop the composite attributable outcome (189/1888, 10%) compared to cases without additional antibiotics (73/1439; 5.1%). Since concomitant antibiotics is a well-established independent risk factor that is specific to CDI (28, 29), this factor may be particularly important for predicting outcomes attributable to CDI.

While the Youden Index is often used to select an “optimal” cutoff point for diagnostic markers when equal weight is given to sensitivity and specificity, choosing a clinically relevant ATLAS score must take into account its intended use, outcome prevalence (and effects on positive/negative predictive value), and tradeoffs of minimizing false positives or false negatives. For example, if the goal is to correctly identify CDI patients very likely to develop severe complications at the expense of sensitivity/recall (e.g., decisionmaking for an irreversible, morbid intervention such as total colectomy), an ATLAS cutoff ≥9 might be reasonable, which corresponds with a relatively low false positive rate 1.2%. On the other hand, in situations where negative predictive value is of highest importance (e.g., identifying CDI inpatients for early discharge), an ATLAS cutoff of <4 would afford >98% negative predictive value (assuming outcome prevalence 7.9%).

The IDSA/SHEA Guideline definition for severe infection using WBC and creatinine alone (30) has historically been a poor predictor of outcomes (17). The addition of up to 3 empirical “points” for each of hypotension, shock, and ileus/megacolon (≥1 criterion previously termed “complicated” (30) or “fulminant” infection (15, 16)) did have a maximal positive predictive value (30.3% with cutoff ≥4) comparable to ATLAS (37.9% with cutoff ≥9), however, our analysis nonetheless indicates there is potential room for improvement in the Guideline-recommended Severity Classifications.

PCR-based C. difficile testing, now used to diagnose >80% of US cases (31), is highly sensitive but cannot differentiate colonization from infection. The PCR cycle threshold has an inverse correlation with C. difficile organism burden. A low cycle threshold (i.e., high organism burden) correlates with toxin EIA positivity (6), and disease severity at some centers (32, 33) but not all centers (34). Our data showed that PCR cycle threshold was poorly predictive for severe outcomes, which is in keeping with recent work demonstrating that the immune response, not bacterial burden, mediates severity (35). Variable correlation between cycle threshold and severe disease could be explained by the significant variations in C. difficile strain virulence observed at different studies (e.g., binary toxin gene prevalence ranges 0.2% to 48% (36)) and/or variations in quantitative toxin levels (not typically measured).

To augment future iterations of ATLAS or other clinical-only models, evaluating novel biomarkers, either host or pathogen factors, would be a logical next step. For example, the CDI-specific host immune response (37) is recognized to play a central role in pathogenesis (38) and data suggest specific biomarkers could be valuable in conjunction with clinical markers (e.g., with an AUC up to 0.91 in one study) (35). Additionally, C. difficile ribotype 027 and other binary toxin-producing strains independently predict disease severity (39–41) and may be useful adjuncts to clinical risk assessment. Score calculations generated entirely from the electronic medical record using data available within 48 h of diagnosis, and the use of a unique validated electronic definition for sepsis also have important implications on the feasibility of future automated research applications and clinical decision support.

MATERIALS AND METHODS

Study population.

A retrospective cohort of hospitalized adult patients with C. difficile infection was developed at University of Virginia Medical Center, a 645-bed, tertiary care academic hospital. Hospitalized cases were identified based on at least 1 positive C. difficile PCR (PCR; GeneXpert; Cepheid, Sunnyvale, CA) test between March 2011 and April 2021. CDI cases in children were rare (<5%) and were excluded due to differences in testing recommendations and given that measures of clinical status (e.g., Charlson) and the validated sepsis definition were not applicable in patients <18 years. Also excluded were cases with > 5 prior recurrent episodes, and those that did not receive active treatment (oral vancomycin, IV or oral metronidazole) while inpatient. This study received approval from the University of Virginia Institutional Review Board (#20082).

Data collection/risk score calculation.

Twenty models for C. difficile infection severity were reviewed from the literature (Table 1) and six (Age Treatment with Systemic Antibiotics Leukocyte Count Albumin and Serum Creatinine [ATLAS] (9), C. difficile Disease [CDD] [8], Zar et al. [42], Hensgens et al. [43], Shivashankar et al. [44], C. difficile Severity Score (CDSS) [45]; Table 4) were chosen for validation based on their performance, prominence in the literature, derivation cohort size, prior validation, parameters that could be reliably gathered from the electronic medical record at the time of diagnosis, and ≥4 ordinal scores that could be fitted to an ROC curve. For IDSA Severity, one point was empirically assigned for each criterion for Severe and Fulminant infection from the Updated 2017 IDSA/SHEA Guidelines (15, 16). In addition, we were interested if real-time C. difficile PCR cycle threshold data (as an inverse measure of fecal organism burden) could be independently useful for predicting severe outcomes since high burden/low cycle threshold (i.e., ≤28.0) has been shown to correlate with worse outcomes associated with CDI (6, 32, 46).

TABLE 4.

Six major clinical scoring methods to predict severe cdi-attributable outcomesa

Model Clinical criteria (points) Outcome(s) Score range
ATLAS (9) Age, non-CDI systemic antibiotics, creatinine, WBC, albumin (0–2 points for each) “Clinical failure” (lack of marked diarrhea reduction or need for further C. difficile therapy) 0–10
CDD (8) Fever (1), ileus (1), SBP < 100 (1), WBC (2), abdominal imaging findings (2; excluded). Diarrhea resolution ≤6 days after therapy initiation 0–5
Zar (15) Age (1), albumin (1), ICU (2), temp (1), pseudomembranes (2; excluded), WBC (1) Cure, treatment failure, relapse 0–6
IDSA/SHEA (11) WBC (1), creatinine (1), hypotension (1), shock (1), ileus/megacolon (1) Based on “non-severe” vs. “severe” (WBC > 1500 or Cr ≥ 1.5) and “fulminant” IDSA/SHEA definitions (11, 12) 0–5
Shivashankar (17) Age (x0.01), WBC (0.81), narcotic use (0.77), H2-antogonist or PPI (0.63), creatinine > 1.5x baseline (0.52) Severe/complicated (hypotension, shock, sepsis) −2.88–0.61
CDSS (18) Age, creatinine, WBC (1 point each) Contributable mortality, ICU admission, or attributable toxic megacolon/colectomy 0–3
Hensgens (16) Age (3), diagnosis in ICU (3), recent abdominal surgery (−3), hypotension (2), admission for diarrhea (2; excluded) Prolonged ICU admission, attributable colectomy or 30d mortality −3–8
Open in a new tab
a

ICU, intensive care unit.

Baseline clinical and outcome data were gathered electronically from the University of Virginia Clinical Data Warehouse, a database containing billing/coding, clinical, pharmacy, and laboratory data from the Epic electronic medical record. Baseline clinical data included the closest available measurement within ±48 h of the index positive C. difficile PCR specimen collection time. In patients with multiple repeat positives during a hospitalization, time of diagnosis was based upon the initial positive result. If multiple laboratory measurements were available, the maximum white blood cell count (WBC), creatinine, and minimum albumin measurements were used. Unavailable measurements were assigned 0 points (i.e., 0 points assigned for ATLAS albumin criterion if not performed at diagnosis). The Charlson Comorbidity Index and the presence of ileus or megacolon were collected using International Classification of Diseases (ICD) coding data (47). National Healthcare Safety Network surveillance definitions were assigned to each case: hospital-onset CDI (HO-CDI), hospital-onset health care-facility-associated (HO-HCFA), or community onset (CO-CDI) (48). Shock was defined by the need for vasopressors. Immunosuppressive medications were defined as ≥60mg oral daily prednisone or equivalent systemic corticosteroid, azathioprine, rapamycin derivatives, cyclosporine, tacrolimus, or mycophenolate. Antimotility medications were defined as loperamide, diphenoxylate, oral opium, or bismuth subsalicylate.

Risk model scores were calculated based on parameters gathered electronically for each case at the time of diagnosis. The following features could not be reliably gathered from the electronic record and so were omitted from score calculations: specific computed tomography abdominal imaging findings (thickened colonic wall, dilation, or ascites) from the CDD score (8), diarrhea as the reason for admission from the Hensgens score (43), and presence of pseduomembranes on endoscopy from the Zar criteria (42). Cycle threshold values from the GeneXpert (Cepheid, Sunnyvale CA) PCR platform were available from archived data for 1,484 cases that occurred between November 2013 and June 2018. Beginning February 2020, UVA Health transitioned from PCR-only testing to multistep PCR with reflex (if PCR+) to toxin enzyme immunoassay (Alere C. DIFF QUIK CHEK COMPLETE), with both results submitted to the treating clinician.

Outcomes.

Severe adverse outcomes attributable to CDI were defined as ICU transfer due to sepsis, CDI-attributable mortality, or colectomy, hemi-colectomy, or diverting ileostomy due to CDI. Mortality and surgery attributions were determined by an Infectious Diseases specialist with expertise in C. difficile (G.R.M.). ICU transfers due to sepsis were categorized electronically using a validated definition by Rhee et al. (19), based on the Sepsis-3 criteria (49) (evidence of presumed serious infection + acute organ dysfunction). Non-attributable outcomes were also collected including all-cause 30-day and 90-day mortality.

Data analysis.

Using the risk scores calculated based on patient demographics and clinical characteristics, we classified cases into score-specific strata and calculated the standard diagnostic test summary indices (sensitivity, specificity, positive predictive value, and negative predictive value) for each stratum. The area under the receiver operating characteristic curve (AuROC) and area under the precision-recall curve (AUC-PR) for each model were then calculated from these score-specific diagnostic test summary indices. The Youden Index (sensitivity + specificity − 1) was calculated as an overall measure of diagnostic effectiveness and as one method to identify an optimal cutoff that balances sensitivity and specificity (Youden ranges 0–1, with 0 indicating a useless test and 1 indicating no false positives or false negatives) (50).

Delong’s test of variance was used to calculate two-sided statistical comparisons of the highest performing model AuROC against each of the others. F1 scores (harmonic mean of precision (positive predictive value) and recall (sensitivity)) were calculated for each model. Analyses were performed using statistical software R, version 4.1.2 (R Core Team, Vienna, Austria) and the following R packages: dpylr (51), comorbidity (52), ROCit (53), pROC (54), and PRROC (55).

ACKNOWLEDGMENTS

This work was funded by National Institutes of Health (K23AI163368 to G.R.M.), R01 AI152477 and AI124214 (to W.A.P.), R01AI145322 (to C.A.W.) and National Center for Advancing Translational Science (UL1TR003015, KL2TR003016 to G.R.M.).

William A. Petri is a consultant for TechLab Inc., a company that manufactures diagnostic tests for C. difficile toxins. All other authors report no conflicts of interest relevant to this article.

REFERENCES

  • 1.Kulaylat AS, Buonomo EL, Scully KW, Hollenbeak CS, Cook H, Petri WA, Stewart DB. 2018. Development and validation of a prediction model for mortality and adverse outcomes among patients with peripheral eosinopenia on admission for clostridium difficile infection. JAMA Surg 153:1127–1133. doi: 10.1001/jamasurg.2018.3174. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Lamontagne F, Labbé A-C, Haeck O, Lesur O, Lalancette M, Patino C, Leblanc M, Laverdière M, Pepin J. 2007. Impact of emergency colectomy on survival of patients with fulminant Clostridium difficile colitis during an epidemic caused by a hypervirulent strain. Ann Surg 245:267–272. doi: 10.1097/01.sla.0000236628.79550.e5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Cairns CM, van Faassen H, Michael FSt, Aubry A, Henry KA, Rossotti MA, Logan SM, Hussack G, Gisch N, Hogendorf WFJ, Pedersen CM, Cox AD. 2020. Development and characterization of mouse monoclonal antibodies specific for Clostridioides (clostridium) difficile lipoteichoic acid. ACS Chem Biol 15:1050–1058. doi: 10.1021/acschembio.0c00066. [DOI] [PubMed] [Google Scholar]
  • 4.Seres Therapeutics. 2021. Seres therapeutics announces positive topline results from SER-109 phase 3 ECOSPOR III study in recurrent C. difficile infection [Press Release]. https://www.businesswire.com/news/home/20200810005194/en/Seres-Therapeutics-Announces-Positive-Topline-Results-SER-109.
  • 5.Sharma AK, Krzeminski J, Weissig V, Hegarty JP, Stewart DB. 2018. Cationic amphiphilic bolaamphiphile-based delivery of antisense oligonucleotides provides a potentially microbiome sparing treatment for C. difficile. J Antibiot (Tokyo) 71:713–721. doi: 10.1038/s41429-018-0056-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Polage CR, Gyorke CE, Kennedy MA, Leslie JL, Chin DL, Wang S, Nguyen HH, Huang B, Tang Y-W, Lee LW, Kim K, Taylor S, Romano PS, Panacek EA, Goodell PB, Solnick JV, Cohen SH. 2015. Overdiagnosis of clostridium difficile infection in the molecular test era. JAMA Intern Med 175:1792–1801. doi: 10.1001/jamainternmed.2015.4114. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Madden GR, Smith DC, Poulter MD, Sifri CD. 2021. Propensity-matched cost of Clostridioides difficile infection overdiagnosis. Open Forum Infect Dis 8:ofaa630. doi: 10.1093/ofid/ofaa630. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Belmares J, Gerding DN, Parada JP, Miskevics S, Weaver F, Johnson S. 2007. Outcome of metronidazole therapy for Clostridium difficile disease and correlation with a scoring system. J Infect 55:495–501. doi: 10.1016/j.jinf.2007.09.015. [DOI] [PubMed] [Google Scholar]
  • 9.Miller MA, Louie T, Mullane K, Weiss K, Lentnek A, Golan Y, Kean Y, Sears P. 2013. Derivation and validation of a simple clinical bedside score (ATLAS) for Clostridium difficile infection which predicts response to therapy. BMC Infect Dis 13:148. doi: 10.1186/1471-2334-13-148. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Im GY, Modayil RJ, Feuerman M, Lin CT, Balani AR, Katz DS, Grendell JH. 2011. A Prediction model of disease severity in clostridium difficile-associated disease. Gastroenterology 140:S361. doi: 10.1016/S0016-5085(11)61474-1. [DOI] [Google Scholar]
  • 11.Beauregard-Paultre C, Chakra CNA, McGeer A, Labbé A-C, Simor AE, Gold W, Muller MP, Powis J, Katz K, Cadarette SM, Pépin J, Valiquette L. 2019. External validation of clinical prediction rules for complications and mortality following Clostridioides difficile infection. PLoS One 14:e0226672. doi: 10.1371/journal.pone.0226672. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Fujitani S, George WL, Murthy AR. 2011. Comparison of Clinical Severity Score Indices for clostridium difficile infection. Infect Control Hosp Epidemiol 32:220–228. doi: 10.1086/658336. [DOI] [PubMed] [Google Scholar]
  • 13.Perry DA, Shirley D, Micic D, Patel PC, Putler R, Menon A, Young VB, Rao K. 2022. External validation and comparison of Clostridioides difficile severity scoring systems. Clin Infect Dis ciab737. doi: 10.1093/cid/ciab737. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.van Beurden YH, Hensgens MPM, Dekkers OM, Cessie SL, Mulder CJJ, Vandenbroucke-Grauls CMJE. 2017. External validation of three prediction tools for patients at risk of a complicated course of clostridium difficile infection: disappointing in an outbreak setting. Infect Control Hosp Epidemiol 38:897–905. doi: 10.1017/ice.2017.89. [DOI] [PubMed] [Google Scholar]
  • 15.McDonald LC, Gerding DN, Johnson S, Bakken JS, Carroll KC, Coffin SE, Dubberke ER, Garey KW, Gould CV, Kelly C, Loo V, Sammons JS, Sandora TJ, Wilcox MH. 2018. Clinical practice guidelines for clostridium difficile infection in adults and children: 2017 update by the Infectious Diseases Society of America (IDSA) and Society for Healthcare Epidemiology of America (SHEA). Clin Infect Dis 66:e1–e48. doi: 10.1093/cid/cix1085. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Johnson S, Lavergne V, Skinner AM, Gonzales-Luna AJ, Garey KW, Kelly CP, Wilcox MH. 2021. Clinical practice guideline by the Infectious Diseases Society of America (IDSA) and Society for Healthcare Epidemiology of America (SHEA): 2021 focused update guidelines on management of Clostridioides difficile infection in adults. Clin Infect Dis 73:e1029–e1044. doi: 10.1093/cid/ciab549. [DOI] [PubMed] [Google Scholar]
  • 17.Stevens V, Jones M, Nelson RE, Khader K, Samore M, Rubin M. 2018. Validation of the SHEA/IDSA severity criteria to predict poor outcomes among inpatients and outpatients with clostridium difficile infection. Open Forum Infect Dis 5:S181–S181. doi: 10.1093/ofid/ofy210.498. [DOI] [PubMed] [Google Scholar]
  • 18.Wang MS, Evans CT, Rodriguez T, Gerding DN, Johnson S. 2013. Clostridium difficile infection and limitations of markers for severity in patients with hematologic malignancy. Infect Control Hosp Epidemiol 34:127–132. doi: 10.1086/669081. [DOI] [PubMed] [Google Scholar]
  • 19.Zar FA, Bakkanagari SR, Moorthi K, Davis MB. 2007. A comparison of vancomycin and metronidazole for the treatment of clostridium difficile-associated diarrhea, stratified by disease severity. Clin Infect Dis 45:302–307. doi: 10.1086/519265. [DOI] [PubMed] [Google Scholar]
  • 20.Hensgens MPM, Dekkers OM, Goorhuis A, LeCessie S, Kuijper EJ. 2014. Predicting a complicated course of Clostridium difficile infection at the bedside. Clin Microbiol Infect 20:O301–8. doi: 10.1111/1469-0691.12391. [DOI] [PubMed] [Google Scholar]
  • 21.Shivashankar R, Khanna S, Kammer PP, Harmsen WS, Zinsmeister AR, Baddour LM, Pardi DS. 2013. Clinical factors associated with development of severe-complicated Clostridium difficile infection. Clin Gastroenterol Hepatol 11:1466–1471. doi: 10.1016/j.cgh.2013.04.050. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Na X, Martin AJ, Sethi S, Kyne L, Garey KW, Flores SW, Hu M, Shah DN, Shields K, Leffler DA, Kelly CP. 2015. A multi-center prospective derivation and validation of a clinical prediction tool for severe clostridium difficile infection. PLoS One 10:e0123405. doi: 10.1371/journal.pone.0123405. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Senchyna F, Gaur RL, Gombar S, Truong CY, Schroeder LF, Banaei N. 2017. Clostridium difficile PCR cycle threshold predicts free toxin. J Clin Microbiol 55:2651–2660. doi: 10.1128/JCM.00563-17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Kamboj M, Brite J, McMillen T, Robilotti E, Herrera A, Sepkowitz K, Babady NE. 2018. Potential of real-time PCR threshold cycle (CT) to predict presence of free toxin and clinically relevant C. difficile infection (CDI) in patients with cancer. J Infect 76:369–375. doi: 10.1016/j.jinf.2017.12.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Quan H, Li B, Couris CM, Fushimi K, Graham P, Hider P, Januel JM, Sundararajan V. 2011. Updating and validating the Charlson Comorbidity Index and score for risk adjustment in hospital discharge abstracts using data from 6 countries. Am J Epidemiol 173:676–682. doi: 10.1093/aje/kwq433. [DOI] [PubMed] [Google Scholar]
  • 26.National Healthcare Safety Network. 2017. CDC/NHSN surveillance definitions for specific types of infections. https://www.cdc.gov/nhsn/pdfs/pscmanual/17pscnosinfdef_current.pdf.
  • 27.Rhee C, Dantes R, Epstein L, Murphy DJ, Seymour CW, Iwashyna TJ, Kadri SS, Angus DC, Danner RL, Fiore AE, Jernigan JA, Martin GS, Septimus E, Warren DK, Karcz A, Chan C, Menchaca JT, Wang R, Gruber S, Klompas M, Program CPE, CDC Prevention Epicenter Program . 2017. Incidence and trends of sepsis in US hospitals using clinical vs claims data, 2009–2014. JAMA 318:1241–1249. doi: 10.1001/jama.2017.13836. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, Bellomo R, Bernard GR, Chiche J-D, Coopersmith CM, Hotchkiss RS, Levy MM, Marshall JC, Martin GS, Opal SM, Rubenfeld GD, T van der P, Vincent J-L, Angus DC. 2016. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 315:801–810. doi: 10.1001/jama.2016.0287. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Schisterman EF, Perkins NJ, Liu A, Bondell H. 2005. Optimal cut-point and its corresponding Youden Index to discriminate individuals using pooled blood samples. Epidemiology 16:73–81. doi: 10.1097/01.ede.0000147512.81966.ba. [DOI] [PubMed] [Google Scholar]
  • 30.Wickham H, François R, Henry L, Müller K. 2022. dplyr: a grammar of data manipulation. R package version 1.0.7. https://dplyr.tidyverse.org.
  • 31.Gutiérrez-Sacristán A, Bravo À, Giannoula A, Mayer MA, Sanz F, Furlong LI. 2018. comoRbidity: an R package for the systematic analysis of disease comorbidities. Bioinformatics 34:3228–3230. doi: 10.1093/bioinformatics/bty315. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Khan RA. 2020. ROCit: an R package for performance assessment of binary classifier with visualization. https://cran.r-project.org/web/packages/ROCit/vignettes/my-vignette.html.
  • 33.Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez J-C, Müller M. 2011. pROC: an open-source package for R and S+ to analyze and compare ROC curves. Bmc Bioinformatics 12:77. doi: 10.1186/1471-2105-12-77. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Keilwagen J, Grosse I, Grau J. 2014. Area under precision-recall curves for weighted and unweighted data. PLoS One 9:e92209. doi: 10.1371/journal.pone.0092209. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Schummers L, Himes KP, Bodnar LM, Hutcheon JA. 2016. Predictor characteristics necessary for building a clinically useful risk prediction model: a simulation study. BMC Med Res Methodol 16:123. doi: 10.1186/s12874-016-0223-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Hota SS, Achonu C, Crowcroft NS, Harvey BJ, Lauwers A, Gardam MA. 2012. Determining mortality rates attributable to Clostridium difficile infection. Emerg Infect Dis 18:305–307. doi: 10.3201/eid1802.101611. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Gilca R, Frenette C, Thériault N, Fortin E, Villeneuve J. 2012. Attributing cause of death for patients with Clostridium difficile infection. Emerg Infect Dis 18:1707–1708. doi: 10.3201/eid1810.120202. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Louie TJ, Miller MA, Mullane KM, Weiss K, Lentnek A, Golan Y, Gorbach S, Sears P, Shue Y-K, Group O-8-0C, OPT-80-003 Clinical Study Group . 2011. Fidaxomicin versus Vancomycin for Clostridium difficile Infection. N Engl J Med 364:422–431. doi: 10.1056/NEJMoa0910812. [DOI] [PubMed] [Google Scholar]
  • 39.Cornely OA, Crook DW, Esposito R, Poirier A, Somero MS, Weiss K, Sears P, Gorbach S. 2012. Group for the O-80-004 CS. 2012. Fidaxomicin versus vancomycin for infection with Clostridium difficile in Europe, Canada, and the USA: a double-blind, non-inferiority, randomised controlled trial. Lancet Infect Dis 12:281–289. doi: 10.1016/S1473-3099(11)70374-7. [DOI] [PubMed] [Google Scholar]
  • 40.Crook DW, Walker AS, Kean Y, Weiss K, Cornely OA, Miller MA, Esposito R, Louie TJ, Stoesser NE, Young BC, Angus BJ, Gorbach SL, Peto TEA, Teams S 003/004 . 2012. Fidaxomicin versus vancomycin for Clostridium difficile infection: meta-analysis of pivotal randomized controlled trials. Clin Infect Dis Official Publ Infect Dis Soc Am 55:S93–S103. doi: 10.1093/cid/cis499. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Hernández-García R, Garza-González E, Miller M, Arteaga-Muller G, los Santos AMG, Camacho-Ortiz A. 2015. Application of the ATLAS score for evaluating the severity of Clostridium difficile infection in teaching hospitals in Mexico. Braz J Infect Dis 19:399–402. doi: 10.1016/j.bjid.2015.05.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Zhang VRY, Woo ASJ, Scaduto C, Cruz MTK, Tan YY, Du H, Feng M, Siah KTH. 2021. Systematic review on the definition and predictors of severe Clostridiodes difficile infection. J Gastroenterol Hepatol 36:89–104. doi: 10.1111/jgh.15102. [DOI] [PubMed] [Google Scholar]
  • 43.Jin SJ, Seo KH, Wi YM. 2018. The effect of concomitant use of systemic antibiotics in patients with Clostridium difficile infection receiving metronidazole therapy. Epidemiol Infect 146:558–564. doi: 10.1017/S0950268818000390. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Mullane KM, Miller MA, Weiss K, Lentnek A, Golan Y, Sears PS, Shue Y-K, Louie TJ, Gorbach SL. 2011. Efficacy of fidaxomicin versus vancomycin as therapy for Clostridium difficile infection in individuals taking concomitant antibiotics for other concurrent infections. Clin Infect Dis 53:440–447. doi: 10.1093/cid/cir404. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Cohen SH, Gerding DN, Johnson S, Kelly CP, Loo VG, McDonald LC, Pepin J, Wilcox MH, Infectious Diseases Society of America . 2010. Clinical practice guidelines for Clostridium difficile infection in adults: 2010 update by the Society for Healthcare Epidemiology of America (SHEA) and the Infectious Diseases Society of America (IDSA). Infect Control Hosp Epidemiol 31:431–455. doi: 10.1086/651706. [DOI] [PubMed] [Google Scholar]
  • 46.Guh AY, Mu Y, Winston LG, Johnston H, Olson D, Farley MM, Wilson LE, Holzbauer SM, Phipps EC, Dumyati GK, Beldavs ZG, Kainer MA, Karlsson M, Gerding DN, McDonald LC, Emerging Infections Program Clostridioides Difficile Infection Working Group . 2020. Trends in U.S. burden of Clostridioides difficile infection and outcomes. N Engl J Med 382:1320–1330. doi: 10.1056/NEJMoa1910215. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Garvey MI, Bradley CW, Wilkinson MAC, Holden E. 2017. Can a toxin gene NAAT be used to predict toxin EIA and the severity of Clostridium difficile infection? Antimicrob Resist Infect Control 6:127–134. doi: 10.1186/s13756-017-0283-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Rao K, Micic D, Natarajan M, Winters S, Kiel MJ, Walk ST, Santhosh K, Mogle JA, Galecki AT, LeBar W, Higgins PDR, Young VB, Aronoff DM. 2015. Clostridium difficile ribotype 027: relationship to age, detectability of toxins A or B in stool with rapid testing, severe infection, and mortality. Clin Infect Dis 61:233–241. doi: 10.1093/cid/civ254. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Abhyankar MM, Ma JZ, Scully KW, Nafziger AJ, Frisbee AL, Saleh MM, Madden GR, Hays AR, Poulter M, Petri WA. 2020. Immune profiling to predict outcome of Clostridioides difficile infection. mBio 11:e00905-20. doi: 10.1128/mBio.00905-20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Gonçalves C, Decré D, Barbut F, Burghoffer B, Petit JC. 2004. Prevalence and characterization of a binary toxin (actin-specific ADP-Ribosyltransferase) from Clostridium difficile. J Clin Microbiol 42:1933–1939. doi: 10.1128/JCM.42.5.1933-1939.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Kelly CP, Chen X, Williams D, Xu H, Cuddemi CA, Daugherty K, Barrett C, Miller M, Foussadier A, Lantz A, Banz A, Pollock NR. 2020. Host immune markers distinguish Clostridioides difficile infection from asymptomatic carriage and non-C. difficile diarrhea. Clin Infect Dis 70:1083–1093. doi: 10.1093/cid/ciz330. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Madan R, Petri WA. 2012. Immune responses to Clostridium difficile infection. Trends Mol Med 18:658–666. doi: 10.1016/j.molmed.2012.09.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Bacci S, Mølbak K, Kjeldsen MK, Olsen KEP. 2011. Binary toxin and death after Clostridium difficile infection. Emerg Infect Dis 17:976–982. doi: 10.3201/eid/1706.101483. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Barbut F, Gariazzo B, Bonné L, Lalande V, Burghoffer B, Luiuz R, Petit J-C. 2007. Clinical features of Clostridium difficile-associated infections and molecular characterization of strains: results of a retrospective study, 2000–2004. Infect Control Hosp Epidemiol 28:131–139. doi: 10.1086/511794. [DOI] [PubMed] [Google Scholar]
  • 55.Berry CE, Davies KA, Owens DW, Wilcox MH. 2017. Is there a relationship between the presence of the binary toxin genes in Clostridium difficile strains and the severity of C. difficile infection (CDI)? Eur J Clin MicrobiolInfect Dis 36:2405–2415. doi: 10.1007/s10096-017-3075-8 [DOI] [PubMed] [Google Scholar]
  • 56.Butt E, Foster JA, Keedwell E, Bell JE, Titball RW, Bhangu A, Michell SL, Sheridan R. 2013. Derivation and validation of a simple, accurate and robust prediction rule for risk of mortality in patients with Clostridium difficile infection. Bmc Infect Dis 13:316. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Velazquez-Gomez I, Rocha-Rodriguez R, Toro DH, Gutierrez-Nuñez JJ, Gonzalez G, Saavedra S. 2008. A severity score index for Clostridium difficile infection. Infect Dis Clin Prac 16:376–378. [Google Scholar]
  • 58.Toro DH, Amaral-Mojica KM, Rocha-Rodriguez R, Gutierrez-Nuñez J. 2011. An innovative severity score index for Clostridium difficile infection. Infect Dis Clin Prac 19:336–339. [Google Scholar]
  • 59.Hu MY, Katchar K, Kyne L, Maroo S, Tummala S, Dreisbach V, Xu H, Leffler DA, Kelly CP. 2009. Prospective derivation and validation of a clinical prediction rule for recurrent Clostridium difficile infection. Gastroenterology 136:1206–1214. [DOI] [PubMed] [Google Scholar]
  • 60.Wilden GM, van der, Chang Y, Cropano C, Subramanian M, Schipper IB, Yeh DD, King DR, Moya MA de, Fagenholz PJ, Velmahos GC. 2014. Fulminant Clostridium difficile colitis: Prospective development of a risk scoring system. J Trauma Acute Care 76:424–430. [DOI] [PubMed] [Google Scholar]
  • 61.Drew RJ, Boyle B. 2009. RUWA scoring system: a novel predictive tool for the identification of patients at high risk for complications from Clostridium difficile infection. J Hosp Infect 71:93–94. [DOI] [PubMed] [Google Scholar]
  • 62.Neal MD, Alverdy JC, Hall DE, Simmons RL, Zuckerbraun BS. 2011. Diverting loop ileostomy and colonic lavage. Ann Surg 254:423–429. [DOI] [PubMed] [Google Scholar]
  • 63.Bauer MP, Hensgens MPM, Miller MA, Gerding DN, Wilcox MH, Dale AP, Fawley WN, Kuijper EJ, Gorbach SL. 2012. Renal failure and Leukocytosis are predictors of a complicated course of Clostridium difficile infection if measured on day of diagnosis. Clin Infect Dis 55:S149–S153. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Antimicrobial Agents and Chemotherapy are provided here courtesy of American Society for Microbiology (ASM)

RESOURCES