Characterizing Risk Factors for Clostridioides difficile Infection among Hospitalized Patients with Community-Acquired Pneumonia

ABSTRACT

Hospitalized patients with community-acquired pneumonia (CAP) are at risk of developing Clostridioides difficile infection (CDI). We developed and tested clinical decision rules for identifying CDI risk in this patient population. The study was a single-center retrospective, case-control analysis of hospitalized adult patients empirically treated for CAP between 1 January 2014 and 3 March 2018. Differences between cases (CDI diagnosed within 180 days following admission) and controls (no test result indicating CDI during the study period) with respect to prehospitalization variables were modeled to generate propensity scores. Postadmission variables were used to predict case status on each postadmission day where (i) ≥1 additional case was identified and (ii) each model stratum contained ≥15 subjects. Models were developed and tested using optimal discriminant analysis and classification tree analysis. Forty-four cases and 181 controls were included. The median time to diagnosis was 50 days postadmission. After weighting, three models were identified (20, 117, and 165 days postadmission). The day 20 model yielded the greatest (weighted [w]) accuracy (weighted area under the receiver operating characteristic curve [wROC area] = 0.826) and the highest chance-corrected accuracy (weighted effect strength for sensitivity [wESS] = 65.3). Having a positive culture (odds, 1:4; P = 0.001), receipt of ceftriaxone plus azithromycin for a defined infection (odds, 3:5; P = 0.006), and continuation of empirical broad-spectrum antibiotics with activity against P. aeruginosa when no pathogen was identified (odds, 1:8; P = 0.013) were associated with CDI on day 20. Three models were identified that accurately predicted CDI in hospitalized patients treated for CAP. Antibiotic use increased the risk of CDI in all models, underscoring the importance of antibiotic stewardship.

INTRODUCTION

Clostridioides difficile infection (CDI) is a significant complication of antibiotic treatment which is responsible for high morbidity. CDI most commonly presents as new-onset diarrhea among patients having recent antibiotic exposure and rarely presents as ileus and toxic megacolon (1). Pneumonia is the leading infectious cause of death and the second most common cause of hospitalization in the United States (2). Whereas antibiotics are lifesaving in bacterial pneumonia (3, 4), the inappropriate use of broad-spectrum antibiotics among patients with community-acquired pneumonia (CAP) is a major driver of infectious complications, including CDI (5). Specifically, inappropriate antibiotic use in patients with CAP (e.g., excessive duration, empirical antibiotic use for viral etiology, or inappropriate use of broad-spectrum agents) has been linked to incident CDI (6–8). Thus, sensitive and specific clinical decision-making guidelines are needed to identify and lessen the risk of CDI in the heavily antibiotic-exposed CAP population, as no such population-specific strategies have yet been developed.

Optimizing antibiotic prescribing for patients with CAP is hypothesized to improve clinical outcomes and reduce the risk of developing CDI. Patients with lower respiratory tract infections, including CAP, are frequently prescribed broad-spectrum antibiotics inappropriately (9, 10). Inappropriate prescribing is associated with prolonged lengths of hospital stay (11) and greater mortality (12). Likewise, health care-associated outbreaks of CDI have been associated with inappropriate antibiotic prescribing for patients with CAP (13). In the empirical setting, clinicians often lack data regarding the specific etiologic pathogen causing pneumonia, which, in turn, leads to broad-spectrum antibiotic use. Whereas prolonged empirical use of broad-spectrum antibiotics increases CDI risk, CDI has been associated with almost all antibiotic classes used to treat CAP (14). Therefore, an improved understanding of clinical and antibiotic-related factors which contribute to an individual’s risk of developing CDI is needed. Accordingly, in this study, we developed and rigorously tested a population-specific decision rule for identifying the risk of CDI and improving the clinical management of patients with CAP.

RESULTS

Identification and characterization of cases and controls.

Between January 2014 and March 2018, a total of 5,811 patients were admitted to Northwestern Memorial Hospital (NMH; Chicago, IL) and received antibiotics with an indication of CAP. In the same period, 1,542 total LabID events (i.e., CDI cases) were identified. A total of 44 cases of CDI were identified within 180 days following an admission within the population of patients initially treated for CAP.

A total of 200 controls were randomly sampled from the population of patients treated for CAP. One control was identified to be a case (PCR positive) during chart review and was reclassified as a case. This individual was not categorized according to the National Healthcare Safety Network (NHSN) definitions. Overall, 181 were eligible for inclusion as true controls after chart review (19 excluded due to having a negative C. difficile PCR result postadmission).

Within the combined sample, patients had an average age of 62.8 years and a mean body weight of 77.5 kg. Males comprised 55% of cases and 50% of controls. The median time to C. difficile PCR positivity was 50 days among cases. The median lengths of hospitalization were 3 days among controls and 5 days among cases. The median durations of inpatient antibiotic treatment were 2 days among controls and 4 days among cases. Antibiotic duration mirrored hospital length of stay (see Fig. S3 in the supplemental material); therefore, only antibiotic duration was considered in subsequent analyses.

CDI onset.

Among the 44 cases of CDI, 17 were hospital onset (HO), 7 were community onset (CO), and 19 were CO-health care facility associated (CO-HCFA), as per the NHSN definitions. Univariate optimal discriminant analysis (ODA) failed to identify any differences in baseline prehospitalization variables between the three onset classifications having multiple observations (all P values > 0.05). None of the three NHSN classification groups differed in terms of time to PCR positivity (Fig. S4). Therefore, all cases were considered together to maximize statistical power.

Comparing prehospitalization differences between cases and controls.

Table 1 summarizes one-sample (“leave-one-out” [LOO]) analysis-stable prehospitalization variables found to differ between cases and controls. The following variables discriminated between cases versus controls at the experimentwise (Sidak-adjusted P value of <0.05) level: hospitalization for two or more days within 90 days prior to admission, prior methicillin-resistant Staphylococcus aureus (MRSA) colonization or infection, history of severe organ dysfunction, antibiotic use within 90 days prior to admission, and receipt of immunosuppressive agents.

TABLE 1.

Prehospitalization variables that significantly differed between cases and controls in univariate LOO cross-generalizability analyses^a

Prehospital factor differing between groups	Cases (n = 44), %	Control (n = 181), %	LOO ESS,b %	LOO P valuec,d	Absolute SD of unweighted modele	Absolute SD of weighted modele,f
Caucasian	61.0	46.0	15.51	0.04649	0.313	0.365
History of enteral feeding	11.4	1.1	10.26	0.003606	0.430	0.372
History of cerebrovascular accident	14.0	3.9	10.09	0.021217	0.356	0.264
History of chronic renal disease	25.6	8.8	16.74	0.004749	0.451	0.518
History of hospitalization ≥2 days in the last 90 days	45.5	16.0	29.43	0.000067*	0.667	0.430
History of MRSA colonization or infection	11.4	0.0	11.36	0.000236*	0.501	0.513
History of severe organ dysfunction	53.4	23.0	29.73	0.0002*	0.636	0.421
History of nursing home residence	11.6	2.2	9.42	0.014471	0.374	0.463
Receipt of antibiotics within 90 days prior to admission	63.6	28.2	35.46	0.000016*	0.756	0.000
Receipt of immunosuppressive agents	68.4	72.7	24.40	0.001004*	0.093	0.154
Receipt of PPIs or H2RAs	52.3	31.5	20.78	0.008787	0.428	0.577

^aAbbreviations: H2RAs, type 2 histamine receptor antagonists; PPIs, proton pump inhibitors; absolute SD, absolute standardized difference; LOO, leave one out; MRSA, methicillin-resistant Staphylococcus aureus.

^bEffect strength for sensitivity (ESS), for a binary outcome, is equivalent to the corresponding ROC area adjusted for chance (ESS of 0 is accuracy expected by chance, ESS of 100 is perfect prediction, and −100 ≤ ESS < 0 is prediction worse than expected by chance). Any (weighted) ESS values less than 25% indicate a relatively weak effect, 25% to 50% indicate a moderate effect, 50% to 75% indicate a relatively strong effect, and 75% or greater indicate a strong effect (15).

^cP value calculated using Fisher’s exact test, which is identical to the P value obtained by ODA when observations are unit weighted.

^dAsterisks indicate that the comparison was significant at the experimentwise level (i.e., Sidak-adjusted P value of <0.05).

^eAbsolute SD values of >0.25 (bold) to indicate imbalance between classes for the given attribute.

^fPropensity score weights were calculated using a model that was based upon receipt of antibiotics within the last 90 days, and this is graphically depicted in Fig. 1.

SDA to identify the subset of attributes which classify cases and controls with maximum chance-corrected accuracy.

Only two iterative structural decomposition analysis (SDA) stages were required, and the attributes which emerged had moderate effect strength and were stable in LOO analysis. These two attributes were antibiotic use within 90 days prior to admission (effect strength for sensitivity [ESS] = 33.46; area under the receiver operating characteristic curve [ROC area] = 0.6773) and receipt of proton pump inhibitors (PPIs) or H2 receptor antagonists (ESS = 35.46; ROC area = 0.6773). Of these, only prior antibiotic receipt within 90 days was significant at the experimentwise level.

Identifying optimal (maximum-accuracy) propensity score weights.

The variables identified using SDA were next evaluated using classification tree analysis (CTA) to identify the optimal propensity score for cases and controls. A comprehensive search of all statistically viable models which varied as a function of minimum stratum sample size revealed that a two-stratum model parameterized with antibiotic use within 90 days prior to admission was most accurate as well as most parsimonious (ESS = 35.46; distance statistic [D] = 3.64). The observed case and control rates within each stratum of the propensity score model are depicted in Fig. 1. As shown in Table 1, weighting by a propensity score based upon antibiotic use within 90 days prior to admission reduced the absolute standardized difference (SD) to <0.1 for this attribute, indicating that the propensity score balanced cases and controls with respect to prior antibiotic receipt within 90 days.

FIG 1.

CTA model of prehospitalization variable used to define the optimal (maximum accuracy) propensity score weight for discriminating cases and controls (i.e., those with and without Clostridioides difficile infection [CDI]). The propensity score for this model was determined as follows: IF received antibiotics in the prior 90 days = No, PREDICT class = control, OTHERWISE IF received antibiotics in the prior 90 days = Yes, PREDICT class = case. True cases had a propensity score of 0.553, whereas true controls had propensity score of 0.929538. Controls incorrectly predicted to have been cases had a weight of 1.7885, whereas cases incorrectly predicted to have been controls had a weight of 1.245412.

Development of an optimal (maximum-accuracy) propensity score-weighted model of postadmission CDI.

Table 2 summarizes LOO analysis-stable variables found to differ between cases and controls through 180 days postadmission. Of these, the following variables discriminated between cases versus controls at the experimentwise level: unknown pathogen, blood urea nitrogen (BUN) of >29 mg/dl on admission, modified-APACHE II (m-APACHE II) score of >7.5 on admission, total inpatient antibiotic duration of >3.5 days, duration of broad-spectrum antibiotics against Gram-negative organisms (GN antibiotics) of >1.5 days, and receipt of any broad-spectrum GN antibiotics.

TABLE 2.

Variables that significantly differed between cases and controls from admission through 180 days postadmission in weighted univariate LOO analysis, including associated optimal discriminant analysis decision-making rules

Attributea	If attribute =	Then predict	Otherwise predict	Sensitivity (%)	Specificity (%)	ROC areab	LOO ESSc	LOO wESSd	LOO P valuee
Pathogen unknown	No	Case	Control	77.27	57.46	0.6736	34.72	36.84	0.000027*
BUN of >29 mg/dl (admission)	No	Control	Case	31.82	90.06	0.6094	21.88	31.42	0.000516*
m-APACHE II score (admission)	≤7.5	Control	Case	81.82	49.17	0.655	31	27.17	0.000073*
Total inpatient antibiotic duration (in days)	≤3.5	Control	Case	52.27	70.17	0.6122	22.44	26.69	0.004769
Any infectious etiology defined	Yes	Case	Control	38.64	80.66	0.5965	19.3	24.56	0.007341
Duration of broad GN antibiotics (in days)	≤1.5	Control	Case	31.82	93.92	0.6287	25.74	21.95	0.000016*
Definitive diagnosis of CAP	No	Case	Control	59.09	61.33	0.6021	20.42	19.47	0.0114
Receipt of broad GN antibiotics	No	Control	Case	36.36	85.64	0.61	22	19.2	0.001438
Bacterial etiology defined	No	Control	Case	27.27	90.61	0.5894	17.88	16.63	0.003086
Culture-positive CAP	No	Control	Case	22.73	89.5	0.5612	12.24	12.83	0.034346
Altered mental status (admission)	No	Control	Case	15.91	95.58	0.5574	11.48	10.43	0.011518

^aUnless otherwise specified, attributes were assessed from the beginning to the end of admission. Abbreviations: broad GN, antibiotics with broad Gram-negative coverage; BUN, blood urea nitrogen; CAP, community-acquired pneumonia.

^bFor a binary outcome the area under the receiver operating characteristic (ROC) curve ranges between 0.5 (indicating chance) and 1 (indicating perfect prediction) for unit-weighted data and is equivalent to the mean predictive accuracy achieved across classes.

^cESS for a binary outcome is equivalent to the corresponding ROC area adjusted for chance (ESS of 0 is accuracy expected by chance, ESS of 100 is perfect prediction, and −100 ≤ ESS < 0 is prediction worse than expected by chance). In line with prior research, we considered any (weighted) ESS values less than 25% to indicate a relatively weak effect, 25% to 50% to indicate a moderate effect, 50% to 75% to indicate a relatively strong effect, and 75% or greater to indicate a strong effect (15).

^dwESS is weighted ESS, or weighted ROC area adjusted to eliminate the effect of chance.

^eAsterisks indicate that the comparison was significant at the experimentwise level (i.e., Sidak-adjusted P value of <0.05).

All LOO analysis-stable postadmission attributes that discriminated cases versus controls between day 20 and day 165 postadmission were considered for inclusion in multivariable CTA models (Table S1). Table 3 gives the model performance characteristics for three CTA models which met the a priori statistical power requirements, including a minimum node denominator of 15. A model predicting CDI status on day 20 postadmission had the lowest weighted D (wD; 3.19), the highest wESS (65.27), and the largest wROC area (0.8264) among all three models and was therefore accepted as the globally optimal CTA model.

TABLE 3.

CTA models predicting CDI within indicated days postadmission which met the minimum sample size requirement and were LOO analysis stable^a

Dayb	Smallest stratum no.	Control no.	Case no.	Model stratum no.c	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)	ROC area	wROC area	ESS	D	wESS (%)	wD
20	16	206	17	6	82.35	76.21	22.22	98.12	0.793	0.826	58.57	4.24	65.27	3.19
117	15	188	36	5	77.78	69.68	32.94	94.24	0.737	0.761	47.46	5.54	52.23	4.57
165	15	184	40	5	75	70.11	35.29	92.81	0.726	0.763	45.11	6.08	52.56	4.51

^aAbbreviations: CTA, classification tree analysis; D, distance statistic, calculated using unweighted ESS and stratum number; wD, weighted D, calculated using weighted ESS and stratum number; ESS, effect strength for sensitivity calculated from the ROC area adjusting for the number of model outcomes; wESS, weighted ESS calculated from the weighted ROC area; LOO, leave-one-out jackknife analysis; NPV, unweighted negative predictive value; PPV, unweighted positive predictive value; ROC area, area under the receiver operating characteristic curve, equal to mean predictive accuracy for a binary outcome; wROC area, weighted ROC area.

^bNo. of days postadmission on which case or control status was modeled. Each model predicts case or control status up through the specified postadmission day.

^cModel stratum numbers are the numbers of unique model outcome prediction groups wherein the smallest stratum number was observed.

Figure 2 presents a graphical depiction of the attributes and strata that comprised the day 20 model. Attributes retained in the CTA model included whether or not any culture was positive (n = 223 classified [two subjects were missing data on this attribute]), a pathogen was known (n = 194 classified), an infectious etiology was identified (n = 87 classified), any broad-spectrum GN antibiotics were selected (n = 107 classified), or the patient received ceftriaxone plus azithromycin therapy (n = 36 classified). The model suggested that among CAP patients with a positive culture, the odds of being a case were 1:4. On the other hand, among patients with a negative culture having no pathogen identified and no infectious etiology defined, the odds of being a case were 1:24. In patients for whom an infectious etiology was defined who received ceftriaxone plus azithromycin, the odds of being a case were 3:5; however, the odds of being a case within this endpoint were significantly lower if an alternative treatment was selected (1:19; P = 0.006). Finally, among patients for whom no pathogen was identified but empirical treatment was continued, the odds of being a case were less than 1:89 when broad-spectrum GN antibiotics were avoided, but the odds of being a case were significantly greater if broad-spectrum GN antibiotics were selected (1:8; P = 0.013). Table S2 displays the component attributes and associated decision rules for each of three models shown in Table 3.

FIG 2.

Weighted classification tree analysis (CTA) model predicting CDI on postadmission day 20. Broad-spectrum Gram-negative (GN) antibiotics included cefepime, meropenem, and piperacillin-tazobactam. Culture-positive CAP (community-acquired pneumonia) was classified for patients having any culture or molecular test showing a pneumonia pathogen. All included model attributes were considered predictors of case status up through but not beyond 20 days postadmission. Abbreviations: AZM, azithromycin; CRO, ceftriaxone.

The classification matrix for the weighted day 20 model is given in Table 4. This model correctly classified 98% of controls before weighting (99% after weighting) and correctly predicted 22% of cases on day 20 prior to weighting (20% after weighting). The ESS of the unweighted model was 58.6 and increased to 65.3 after weighting. Thus, weighting by a propensity score that balanced cases and controls with respect to antibiotic receipt prior to admission increased the effect strength (i.e., chance corrected accuracy) of the day 20 model. A similar effect of propensity score weighting was observed in the day 117 and the day 165 models (data not shown).

TABLE 4.

Observed versus predicted C. difficile infections from the globally optimal model^a shown in Fig. 2

Day 20 model cross-classification tableb	Matrix parameter	No. of patients with predicted outcome			Classification accuracyd
		Control	Case	Margin	% sensitivity	% wSensitivity
No. of patients with observed outcome	Control	157	49	206	76.2	76.8
	Case	3	14	17	82.4	88.4
	Margin	160	63	223
Predictive performancec
% PV		98.1	22.2
% wPV		99.0	20.8

^aThe globally optimal weighted CTA model yielded a weighted ESS that was stable in a LOO analysis and remained statistically significant (P = 0.000002).

^bThe ROC area for the model was 0.7928, the area of the ROC weighted by the propensity score (wROC) (i.e., antibiotic use within 90 days prior to admission) was 0.8264, the ESS was 58.6%, and the weighted ESS was 65.3%.

^cAbbreviations: PV, predictive value; wPV, PV weighted by the propensity score.

^dAbbreviation: wSensitivity, sensitivity for the class of case or control weighted by the propensity score.

Evaluation of propensity score weighted CTA models predicting postadmission CDI.

Two evaluations were conducted to determine the robustness of model predictions.

First, unweighted model predictions were generated using the CTA model attributes and decision rules for each of the models shown in Table 3. These predictions were used to classify case or control status within the entire sample using a propensity score-weighted ODA with LOO analysis. The LOO analysis confirmed that after weighting, each of the three models was stable and significant at the experimentwise level. Therefore, individual observation variation did not impact the accuracy of model predictions.

Second, weighted predictions for the three models shown in Table 3 were generated. These predictions were then subjected to bootstrap resampling to estimate the 90% predictive interval (PI) bounds of the weighted model’s performance. Next, the weighted predictions underwent Fisher’s randomization relative to the class variable (i.e., scrambling) and were then subjected to bootstrap resampling to estimate the 90% PI bounds of chance. Figure 3 provides a visual comparison of the wESS and wROC area bootstrap results for the three models displayed in Table 3. Aggregation of the resampled predictions from the day 20 model and from chance revealed that lower-bound 5th percentile of wESS for the model was 38.5 (wROC = 0.693), whereas the upper-bound 95th percentile of wESS for chance was 29.0 (wROC = 0.645), demonstrating that the 90% PI of the weighted model predictions did not overlap chance. Likewise, weighted predictions from the day 117 and day 165 models had 90% PIs that did not overlap chance.

FIG 3.

Comparison of CTA model performances after bootstrap resampling. Effect strength for sensitivity (ESS) for a binary outcome is equivalent to the corresponding receiver operating characteristic curve (ROC) area adjusted for chance where ESS of 0 is accuracy expected by chance, ESS of 100 is perfect prediction, and −100 ≤ ESS < 0 is prediction worse than expected by chance.

Sensitivity analyses.

After identifying the globally optimal CTA model at day 20, we investigated whether nuisance variables predicting case versus control status changed at any time after day 20. Only the models identified on days 20, 117, and 165 described in Table 2 met our minimum sample size requirements and our robustness evaluation criteria.

DISCUSSION

We identified three models that accurately predicted CDI among hospitalized patients treated for CAP. The globally optimal decision-making model predicted CDI at 20 days after admission with high accuracy (ROC area = 0.7928; ESS = 58.6), and this accuracy was further increased after accounting for antibiotic use within 90 days prior to admission (wROC area = 0.8264; wESS = 65.3). Importantly, the globally optimal weighted model correctly identified 88.4% of cases up to 20 days postadmission. The day 20 model correctly predicted case status (weighted positive predictive value [wPPV] = 20.8%) at a rate nearly 3-fold higher than the base rate (7.6%), and it yielded a high negative predictive value (NPV; wNPV = 99%). The day 20 model predictions were stable and significant in LOO analysis at the experimentwise level (Sidak-adjusted P value of <0.05), suggesting that the model was robust to individual observation variation. Likewise, the accuracy of the model predictions observed after bootstrap resampling yielded a 90% PI (wROC = 0.693 to 0.903) that exceeded that of chance (wROC = 0.362 to 0.645). The point estimate of chance-corrected model accuracy (wESS) on day 20 was 65.3, indicating a relatively strong effect (15). Bootstrap analysis indicates this model would at worst yield a moderate effect (5th percentile wESS = 38.5) (15).

The day 20 model accounted for both prehospitalization and postadmission factors associated with the risk of CDI among patients treated for CAP. After adjustment for antibiotic use within 90 days prior to admission, the following diagnostic and treatment features were associated (per-comparison P < 0.05) with case status: having a positive culture, receiving definitive treatment (infectious etiology defined and known pathogen) with ceftriaxone plus azithromycin, and empirical treatment (no pathogen identified) with broad-spectrum GN antibiotics. Definitive use of ceftriaxone plus azithromycin was strongly associated with case status in our sample (odds, 3:5) compared to alternative therapies (odds, 1:19) within this stratum. Although not consistently observed in prior studies, receipt of cephalosporins has been identified as a risk factor for CDI (14, 16). Additionally, lack of an identifiable pathogen paired with continuation of empirical antibiotics was a major risk factor for CDI in our sample (odds, 1:8 with broad-spectrum GN antibiotics versus <1:89 when broad-spectrum GN antibiotics were avoided). Chalmers et al. found that CDI was significantly associated with empirical use of piperacillin-tazobactam (22.9% versus 2.3%; P < 0.0001) in patients with pneumonia. On the other hand, they found no significant association with CDI among patients receiving narrow-spectrum penicillins (24.6% versus 35.0%; P = 0.09) (6). In our sample, patients having a positive culture were more likely to develop CDI (odds, 1:4), whereas patients for whom no infectious etiology was defined and for whom no pathogen was identified were relatively protected (odds, 1:24). The latter group may represent a patient population in which empirical antibiotics are more easily discontinued or deescalated as alternative diagnoses are made. These findings support the need for antimicrobial stewardship in patients with CAP and underscore the importance of narrowing antibiotics when possible and discontinuing them when not needed.

Our findings have significance in light of the revised pneumonia guidelines (17, 18). Health care-associated pneumonia (HCAP), a major driver of broad-spectrum antibiotic use, is no longer recognized as a valid diagnosis (18). HCAP is associated with more frequent use of broad-spectrum antibiotics targeting nosocomial pathogens (e.g., Pseudomonas aeruginosa) and increases the risk of CDI. Likewise, prior research has shown that antibiotic treatment increases the risk of C. difficile carriage during hospitalization, and the risk of carriage increases with longer treatment durations (8, 16, 19). The observation that patients experience net harm after 3 to 5 days of treatment, paired with a lack of evidence of benefit for longer treatment durations for the majority of patients with community-acquired pneumonia, underscores the clinical paradigm that “shorter is better” (20).

Our study has several limitations. First, it was a single-center retrospective case-control analysis. Our sample size was relatively small, resulting in power constraints which limited our ability to model CDI prior to day 20. However, statistical power was sufficient to identify three decision-making models (day 20, day 117, and day 165 postadmission), yielding patient classification predictions which were rigorously tested in LOO and bootstrap evaluations. Although we found that a number of prehospitalization variables differed between cases and controls in univariate analysis, the majority of these factors exhibited relatively weak effects (ESS of <25).

To minimize threats to causal inference, we utilized propensity score weighting in model development to balance cases and controls with respect to antibiotic use within 90 days prior to admission. Adjustment for prior antibiotic use is important because it has been associated with both initial incident and recurrent CDI (21, 22). While the median time to PCR positivity was 50 days in our sample, we identified a model with high accuracy (wROC = 0.826) that exhibited a relatively strong effect after accounting for chance (wESS = 65.3) at 20 days postadmission. However, a wD statistic of 3.19 for this model suggests that our work is not complete. That is, at least three additional unrecorded variables with equal explanatory power are required and may exist that together would perfectly classify cases and controls in our sample.

We utilized positive PCR to define clinical CDI within our sample. The diagnosis of CDI relies on both clinical and microbiological evidence of disease. From a strictly analytical perspective, C. difficile PCR nucleic acid amplification (i.e., PCR) of liquid stool specimens is highly sensitive (90%) and specific (96%) (23). In spite of the potential for clinical false positives (24), C. difficile laboratory events remain a required standard for public reporting and tracking of CDI (25).

Additionally, we relied on data available within the electronic health record (EHR) at the time of admission through discharge, limiting our ability to discern all possible time-varying relationships; however, we were able to identify early (i.e., 20 days postadmission), intermediate (i.e., 117 days postadmission), and late (i.e., 165 days postadmission) risk factors for CDI. Larger sample sizes may reveal more granular models that clarify how the importance and geometry of these attributes change over time. Our three preliminary models should be evaluated using external validation as a next step and may warrant further development for prospective use in the EHR.

In summary, we identified three relatively simple and biologically plausible clinical decision models that predict the risk of CDI among hospitalized patients with CAP. Our day 20 model discriminated between cases and controls 30 days earlier than the median disease onset, with a wPPV nearly 3-fold higher than the base rate and a wNPV of 99%. Our models underscore the need for optimization of antimicrobial stewardship for patients with community-acquired pneumonia at risk for C. difficile colonization and subsequent infection. Stewardship programs that provide clinicians with enhanced diagnostic stewardship for respiratory pathogens, a system to optimize antibiotic duration for CAP patients, and a real-time CDI risk assessment may be able to improve patient safety.

MATERIALS AND METHODS

Design and ethical considerations.

We conducted a retrospective case-control study of hospitalized patients treated empirically for CAP. The study took place at Northwestern Memorial Hospital (NMH), an 897-bed tertiary academic medical center in Chicago, IL. The study protocol was reviewed and approved by the institutional review boards of Northwestern University (STU00206507) and Midwestern University (3047).

Inclusion and exclusion criteria.

A convenience sample of cases and controls were eligible for inclusion if the subjects were hospitalized between 1 January 2014 and 3 March 2018, were adults aged 18 years or older, and received systemic antibiotics with an indication listed as CAP. The index admission for cases and controls was taken as the first hospitalization for CAP during the study period. All eligible cases were considered. Case patients were included if they received antibiotics with an indication of CAP and subsequently were diagnosed with CDI. Eligible controls were sampled from the entire cohort of CAP patients targeting a 4:1 ratio of controls for each case to maximize available power (26). Case patients were excluded if they were PCR positive for CDI within the previous 90 days (to capture incident CDI). Additional exclusion criteria were a history of cystic fibrosis, frequent hospitalization (≥3 admissions within prior 30 days) due to the overlap between CAP and hospital-acquired pneumonia (HAP) risk factors in patients with recent prolonged prior hospitalization (27), requirement of initial intensive care unit (ICU) management for severe CAP, and presence of a rapidly fatal underlying condition (i.e., death within 48 h of admission). Inclusion and exclusion criteria for control patients were identical except that controls could not have had any documented positive C. difficile PCR during the study period. Statistical power considerations are discussed in greater detail ahead, under “Identification of postadmission factors associated with CDI.”

C. difficile specimen processing and PCR methods.

As previously described (28), C. difficile specimens at our facility were evaluated using PCR. As a C. difficile PCR-only hospital during the study period, clinicians ordered the test according to agreed-upon clinical criterion of 3 or more stools within 24 h that were new, large, and liquid with no other clear explanation, such as recently ingested laxatives. Briefly, liquid stool specimens were screened and processed by the clinical microbiology laboratory. All formed stools were rejected by the laboratory. Patients could not be tested more frequently than once every 7 days. Evaluable clinical specimens underwent DNA extraction and PCR amplification for the toxin B gene (i.e., tcdB). PCR was performed using one of the following methods according to study year: BD GeneOhm kit (BD Diagnostics) and SmartCycler (years 2012 to 2015) or BD Max kit (BD Diagnostics) (years 2015 to 2018). All PCR conditions were established according to manufacturer protocol.

Identification of cases and controls.

Patients having a positive C. difficile PCR were classified as CDI cases. During the study period, the decision to order the PCR was at the discretion of the patient’s treating clinician. C. difficile carriage for 3 to 6 months after antibiotic exposure has been documented (8, 29). Correspondingly, eligible cases were considered to have occurred within roughly 6 months (i.e., 180 days) following the first hospital day of the index admission. Controls could not have a PCR ordered at any point during the entire study period and were randomly selected (Microsoft Excel) from among all eligible noncase patients treated for CAP.

Data sources and retrieval.

PCR results were extracted from a line list of CDI events reported to the National Healthcare Safety Network (NHSN) database by the NMH Infection Prevention and Healthcare Epidemiology group (C.S.). These data were cross-referenced against the list of patients empirically treated for CAP during the study period (N.J.R.). Patient data were retrieved from the electronic health record (EHR) (Millennium; Cerner) by querying the Northwestern Electronic Data Warehouse (R.M.). The study end date was fixed due to a change in EHR systems starting March 2018. Admissions records were searched for hospital encounters matching the study inclusion criteria. Admission characteristics (i.e., length of stay, antibiotic treatment duration, and antibiotic selection) and patient demographics (i.e., age, weight, sex, and comorbidities) were extracted from the EHR electronically and verified by manual chart review in triplicate (C.C.J., W.J.M., and K.H.).

Data preprocessing.

Data elements were divided into two primary categories: (i) prehospitalization and (ii) postadmission. Prehospitalization variables included comorbidities and other clinical demographics available at or before the index hospitalization. Postadmission variables included component and composite severity of illness measures, including modified-APACHE II (m-APACHE II) score (30) and the pneumonia severity index (PSI) (31), calculated on the day of hospital admission. Antibiotic agents and spectrum were dichotomous, whereas the duration of therapy for each agent was an ordered variable. CAP antibiotics were classified based on spectrum of activity, class, and whether or not they were concordant or discordant with our institution’s empirical CAP treatment algorithm (Fig. S1) (32). Cefepime, meropenem, and piperacillin-tazobactam were considered broad-spectrum Gram-negative (GN) antibiotics. Linezolid and intravenous (i.v.) vancomycin were considered broad-spectrum Gram-positive (GP) antibiotics. Other categorical variables included the presence of radiographic findings consistent with CAP, etiologic assessments of the cause of pneumonia (bacterial, viral, fungal, mycobacterial, or lack thereof), and microbiologic test results (positive sputum or blood cultures containing a CAP pathogen). A complete list of the data elements extracted is included in the supplemental material. Data were stored in REDCap (33).

CDI onset classification.

CDI onset was classified according to NHSN criteria as community onset (CO), CO-health care facility associated (CO-HCFA), or hospital onset (HO). As per the NHSN definitions, HO CDIs were defined as LabID events that occurred between day 4 of the index hospitalization and discharge, and CO-HCFA CDIs were defined as LabID events that occurred within 28 days after discharge from the index admission. CO CDIs were defined as LabID events that occurred within the first 3 days of hospitalization or as events wherein specimens were collected as an outpatient from individuals who were not discharged from an inpatient location within 28 days prior to date of specimen collection. To assess for the presence of time-varying relationships between predictors and outcomes, PCR positivity was evaluated on every postadmission day wherein at least one additional case was observed.

Univariate comparison of differences between cases and controls.

Descriptive statistics were calculated for all variables. Data were analyzed using the ODA package (34) for R (35), which is a user-written front-end interface for the MegaODA and classification tree analysis (CTA) software programs developed by Yarnold and Soltysik (36–38). In univariate analyses, case versus control status was considered as the class (i.e., “dependent”) variable. Prehospitalization attribute (i.e., attributes) variables were considered as “independent” variables to be used in predicting case versus control status. Missing data were coded as missing in all analyses. Differences in prehospitalization categorical and continuous variables between cases and controls were compared at the univariate level using the optimal discriminant analysis (ODA) algorithm (36). Time-to-event data were assessed using Kaplan-Meier curves and 95% confidence intervals generated using the survival package and plotted using the survminer package for R, respectively (39, 40).

Model classification accuracy metrics.

Sensitivity (Sn), specificity (Sp), and positive and negative predictive values were calculated for all models. For both univariate (ODA) and multivariable (CTA) analysis, the effect strength for sensitivity (ESS) index of classification accuracy, which is adjusted to remove the effect of chance, was used to guide model development. For a two-category class variable, ESS = 100 × {[0.5× (Sp + Sn)] − 0.5}/0.5. An ESS of 0 corresponds to the accuracy which is expected by chance (ROC area = 0.5), whereas an ESS of 100 corresponds to perfect accuracy (ROC area = 1) (38, 41). In line with prior research, we considered any (weighted) ESS values less than 25% to indicate a relatively weak effect, between 25% and 50% to indicate a moderate effect, between 50% and 75% to indicate a relatively strong effect, and 75% or greater to indicate a strong effect (15).

The distance statistic [D] was used to adjust ESS to reflect model complexity (38, 42). D is the number of additional effects with equivalent ESS needed to obtain perfect classification of the sample (43), and D is calculated as [100/(ESS/stratum)] – stratum, where stratum is defined as the number of unique model endpoints (groups) into which the sample is partitioned (38, 43).

SDA to identify the subset of prehospitalization variables which optimally classified cases versus controls.

Structural decomposition analysis (SDA) was used to identify the subset of prehospitalization attributes which were subsequently passed to the CTA algorithm to maximize accuracy in discriminating cases versus controls (38). In the first step of SDA, the entire sample is classified separately by conducting ODA analysis for each prehospitalization attribute. The attribute yielding the model having the greatest ESS is retained. Subjects correctly classified by this model are removed from the data set. This process is repeated, eliminating from consideration variables selected in prior steps until no remaining attributes predictive of class category can be identified.

Development of propensity score weights to balance cases and controls.

Prehospitalization attributes discriminating cases versus controls in SDA were used to create propensity score weights in order to mitigate threats to causal inference (44, 45). Briefly, these attributes were initially entered into CTA without specifying a minimum sample size (i.e., “minimum denominator”) required of model endpoints (i.e., “sample strata”). If a statistically viable model was identified, then the “minimum sample size” was set at the minimum denominator in the initial analysis plus one. Continued until no additional models could be identified, this procedure identified all of the statistically viable CTA models which exist for the sample that differ in terms of statistical power (minimum stratum sample size), chance-adjusted accuracy (ESS), and ESS adjusted for complexity (D) (45). The model having the highest ESS and lowest D was considered the most accurate and most parsimonious, respectively.

The propensity score was generated using the observed case and control incidence within each stratum of the CTA model, and this weight was applied to the entire sample (44). Covariate balance was assessed using the user-written COVBAL package (46) for Stata (Statacorp, TX). Absolute SD values of >0.25 suggested imbalance between classes for any given attribute (47).

Identification of postadmission factors associated with CDI.

Similar to the analysis of prehospitalization variables, univariate analyses were conducted using postadmission variables (e.g., severity of illness and antibiotic treatment) as attributes to discriminate case versus control status. CTA models were constructed only for those days wherein statistical power analysis indicated that a sufficient number of cases existed (i.e., ≥15 cases total; see power curves in Fig. S2) (48). Within the sample, statistical power was sufficient to identify a CTA model between days 20 and 165 postadmission. The following postadmission attributes were used to predict case versus control status between days 20 and 165 postadmission: microbiologic findings, etiology of pneumonia, radiographic findings; component and composite elements of severity of illness scores (e.g., modified APACHE II and PSI), each individual antibiotic agent used empirically, duration of each empirically used antibiotic and its spectrum (e.g., broad Gram-negative or broad Gram-positive coverage), and whether the empirical antibiotic(s) which were utilized matched our institutional guidelines.

The final model(s) was selected from among competing models based upon their weighted ESS—referred to as wESS (indicating that observations are weighted with respect to their specific propensity) and wD (weighted D) statistics. The model(s) having the highest wESS and lowest wD was considered the most accurate and the most parsimonious, respectively.

Model evaluation.

In order to maintain rigor and also to minimize the risk of committing type II errors in a relatively small sample, we utilized two thresholds for statistical significance: a strident criterion (i.e., Sidak-adjusted experimentwise P value of <0.05) and a conventional naive criterion (i.e., per-comparison P value of <0.05). The cross-generalizability of the statistical model(s) (when applied to classify an independent random sample) was estimated via a one-sample (“leave-one-out” [LOO]) jackknife analysis (49). In order to maximize reproducibility, only models with classification results which were stable in LOO analysis were considered for further evaluation (38, 50). The reliability of model predictions expressed in terms of wESS and wROC were evaluated using the NOVOboot() function in the ODA package (51). Bootstrap resampling with replacement (n = 25,000 resamples with 50% replacement) was used to estimate an exact discrete 90% predictive interval (PI) for the model(s) evaluated, and this interval was compared to the exact discrete 90% PI for random chance (i.e., predictions scrambled randomly relative to known class membership) (51).

Data availability.

Data will be made available upon reasonable request.