Identification of Population at Risk for Future Clostridium difficile Infection Following Hospital Discharge to be Targeted for Vaccine Trials

Abstract

Background

Efforts to develop a Clostridium difficile vaccine are underway; identification of patients at risk for C. difficile infection (CDI) is critical to inform vaccine trials. We identified groups at high risk of CDI ≥28 days after hospital discharge.

Methods

Hospital discharge data and pharmacy data from two large academic centers, in New York and Connecticut, were linked to active population-based CDI surveillance data from the Emerging Infections Program (EIP). Adult residents of the EIP surveillance area were included if they had an inpatient stay at a study hospital without prior history of CDI. The primary outcome was CDI by either toxin or molecular assay ≥28 days after an index hospitalization. Important predictors of CDI ≥28 days post discharge were initially identified through a Cox proportional hazards model (stepwise backward selection) using a derivation cohort; final model parameters were used to develop a risk score evaluated in the validation cohort.

Results

Of the 35,186 index hospitalizations, 288 (0.82%) had CDI ≥28 days post discharge. After parameter selection, age, number of hospitalizations in the prior 90 days, admission diagnosis, and the use of 3^rd/4^th generation cephalosporin, clindamycin or fluoroquinolone antibiotic classes remained in the model. Using the validation cohort, the risk score was predictive (p<0.001) with a c-score of 0.75. Based on the distribution of scores in the derivation cohort, we divided the patients into low and high risk groups. In the high risk group, 1.6% experienced CDI ≥28 days post discharge compared to 0.3% among our low risk group.

Conclusions

Our study identified specific parameters for a risk score that can be applied at discharge to identify a patient population whose risk of CDI ≥28 days following an acute care hospitalization was 5 times greater than other patients.

Background

Clostridium difficile was estimated to cause nearly half of a million infections in the United States in 2011 and has become the most common pathogen type among healthcare-associated infections.[1, 2] The U.S. hospital discharge rate with C. difficile infection (CDI) listed as any diagnosis increased from 5.6 per 1,000 discharges in 2001 to 11.5 per 1,000 discharges in 2010, and was projected to continue to increase.[3] In the United States, data from vital records showed that C. difficile has become the leading cause of enterocolitis-associated deaths and was estimated to be associated with nearly 29,000 deaths per year in 2011.[2, 4] Furthermore, C. difficile is no longer restricted to hospital settings and approximately 75% of the patients who develop healthcare-associated CDI have their onset outside the hospital settings.[5] Although numerous interventions have been implemented to control the spread of C. difficile in hospitals, this pathogen remains an important cause of healthcare-associated infections in the United States.[5, 6] Efforts to develop a C. difficile vaccine are currently underway and identification of high risk patients to be targeted for vaccine trials is critical to measure vaccine efficacy and immunogenicity.

We conducted a retrospective cohort study of hospitalized patients at two large academic centers, participating in the Emerging Infections Program (EIP) CDI Surveillance, to identify groups at high risk of CDI that could potentially be targeted for vaccine trials.

Methods

Study Settings and Patients

We used data from academic centers in CT with approximately 1,000 beds and 58,000 discharges per year and NY with approximately 700 beds and 40,000 discharges per year. The two academic centers along with their respective state health departments are part of EIP.[7] Active population- and laboratory-based surveillance for CDI is ongoing in both Monroe County, NY and New Haven County, CT since 2009.[7]

For this study, patients were included if they were ≥18 years of age, had an inpatient stay at a study hospital, and were residents of the CT and NY surveillance area.

Data Sources

For this project, we utilized existing data as follows:

Hospital Discharge Data

Each participating institution obtained clinical and demographic information from their hospital discharge data, including ICD-9-CM procedure and diagnostic codes, patient's age, insurance coverage, sex, date of hospital admission and discharge, inpatient unit(s) visited, disposition at discharge, and initial diagnosis on admission. Patient identifiers were obtained to allow for merging of data with pharmacy and EIP surveillance data.

Pharmacy Data

Data on antibiotic class, name, route, and duration for all antibiotics administered during each hospital stay was obtained for hospitalizations in the study period. Antibiotics were grouped into the following classes: aminoglycosides, 1st and 2nd generation cephalosporins, 3rd and 4th generation cephalosporins, lincosamides, fluoroquinolones, macrolides, vancomycin, sulfonamides, beta-lactam/beta-lactamase inhibitor combinations, carbapenems, penicillins, and miscellaneous. Pharmacy data also contained individual patient identifiers to allow merging of pharmacy data with hospital discharge data.

EIP Data

The EIP CDI surveillance provides longitudinal data on CDI among surveillance area residents in the community and healthcare settings. Trained epidemiologists capture all C. difficile-positive reports by either toxin or molecular assay from clinical, reference, or commercials laboratories for residents in the catchment area, along with the personal identifiers. A laboratory report is classified as an incident episode if it was in a patient without a positive C. difficile test in the prior 8 weeks. All incident episodes receive a unique code and are entered on weekly basis into a web-based system without personal identifiers. We utilized EIP CDI data from the beginning of surveillance at each site until December 2012 to identify patients diagnosed with CDI in outpatient and inpatient settings.

Data Linkage and Transfer to CDC

Each site was responsible for generating a linkable patient identification (LPI) code based on patient identifiers that were common across all three datasets. Encounters identified for the same patient received the same LPI code, unique to a specific patient. All patient identifiers were deleted after LPI codes were added. Each site also created a CDI database based on CDC EIP data containing only patients who developed CDI and who appeared in the discharge/pharmacy data, regardless of the timing of CDI onset relative to the hospital admission. The protocol was approved by the institutional review board at each site and CDC.

Analytical Methods

After merging the datasets, the distribution/frequency of all variables was examined, including demographics, frequency and risk of CDI following hospitalization, time from hospitalization to CDI, length of stay (LOS), antibiotic exposure during stay, ICD-9-CM diagnosis and procedure categories, inpatient units, death, disposition on discharge, past hospital stays and antimicrobial exposures, and past episodes of CDI.

Clinical Classification Software (CCS) from the Agency for Healthcare Research and Quality (AHRQ) was used to categorize ICD-9-CM diagnosis and procedure codes into clinically meaningful categories.[8] For each hospital stay, antimicrobials were grouped into twelve classes, described above.

The cohort was created by identifying the hospital stays within a single year for each unique patient at each site, January 2011-December 2011 for CT and July 2011-June 2012 for NY. Among patients with more than one hospitalization, one stay was selected at random and identified as the index hospitalization. Exposures were based on the index hospitalization and past hospitalizations within 90 days. All patients with CDI documented either during index hospitalization or prior to hospitalization based on EIP data and patients receiving oral vancomycin during their index hospitalization were excluded from analysis.

Our outcome was an incident C. difficile episode ≥28 days after discharge from the index hospitalization. The timeframe of ≥28 days after discharge for CDI development was used to take into account a scenario where the vaccine would be administered at the time of hospital discharge, would be a multiple dose regimen, and time would be needed for vaccine recipients to build an immune response. In addition, we hypothesized that the vaccine would be for prevention of primary CDI, and therefore patients with prior CDI were excluded.[9, 10]

Exposures were based on information provided by the hospital administrative data while the outcome was based on the linked EIP CDI surveillance data.

In order to create a CDI risk index, we considered the scenario of enrollment and vaccination at time of hospital discharge to prevent post-discharge incident CDI ≥28 days after discharge. We conducted univariate comparisons of those who developed CDI ≥28 days after discharge to those without CDI using t-tests or Wilcoxon Rank tests for continuous variables and Chi Square or Fisher's Exact tests for categorical variables as applicable.

In the multivariate analysis, we employed a Cox Proportional Hazards model and conducted a backward elimination selection with a significance level to stay criterion of p ≤0.10 to identify important predictors of CDI after discharge. Due to the large number of CCS categories, univariate analysis was used to categorize each of the following CCS categories: discharge diagnosis, procedure and admission diagnosis into four progressive levels depending on the increasing association with CDI ≥28 days post-discharge. Patients were followed until the development of CDI during the post-discharge period, death, or the end of follow-up period. The follow-up period extended to six months following the end of the initial year in which index visits were identified, June 30, 2012 for CT and December 31, 2012 for NY.

After determining the important predictors for CDI development ≥28 days after discharge, we developed a CDI risk index. The risk index was based on parameter estimates from the Cox Proportional Hazards model containing only important predictors. To generate the risk index score, the remaining parameter estimates were divided by the absolute value of the smallest parameter estimate. The integer of that quotient was then declared the number of points to assign for that characteristic. Protective characteristics retained a negative score. Reference categories were assigned a point value of 0. Once points were assigned for each characteristic, all points were summed to calculate a final score for each discharge. Distribution of scores was compared for those with CDI ≥28 days following discharge to those without. A specific score, the first quartile of those with CDI, was selected to divide discharges into high and low risk categories.

In order to validate our CDI risk index, we developed models for each site separately, and then validated the model at the other site. For validation, we employed the same outcome, and conducted logistic regression in the validation site. For validation, we considered 3 measures: area under the receiver operating characteristic (ROC) curve for the risk index score, odds ratio (OR) for the high risk discharges compared to low risk, and the risk in the both the high and low risk discharges.

Further, we conducted three sets of models. The first model (complex scenario) included all information in the multivariate models such as ICD-9-CM discharge and procedure codes, which may not be readily available in medical records at time of patient's discharge. The second model (simple scenario) did not include billing data such as ICD-9-CM discharge and procedure codes and information generated from those codes like Gagne co-morbidity score and the number of ICD-9-CM procedures. For both the complex and simple scenario, models were developed in each site independently and validated at the other site. Finally, a third model (combined scenario) was created based on overlapping predictors from the complex and simple scenarios. For this model, discharge data from the two sites were combined and randomly separated into development and validation cohorts. A Cox model including only the selected overlapping predictors and using the data from the development cohort was used to develop the risk index for the combined model. That risk index was then assessed in the validation cohort.

All analyses were conducted using SAS software (version 9.3; SAS Institute).

Results

A total of 22,069 index hospitalizations were observed in CT from January 1–December 31, 2011, while 14,527 index hospitalizations were identified in NY from July1, 2011–June 30, 2012 among surveillance area residents.

At CT, of the 22,069 index hospitalizations, a total of 660 were sequentially excluded using the following order: prior CDI (n=105), death during hospitalization (n=402), incident CDI during the hospital visit (n=100), and reported oral vancomycin use during the index visit (n=53), leaving 21,409 (97%) index hospitalizations for analysis.

At NY, of the 14,527 index hospitalizations, a total of 750 were sequentially excluded using the following order: prior CDI (n=279), death during hospitalization (n=390), incident CDI during the index hospitalization (n=80), and reported oral vancomycin use during the index visit (n=1), leaving 13,777 (95%) index hospitalizations for analysis.

CDI ≥28 days post-discharge occurred in 164 (0.77%) and 124 (0.90%) of CT and NY patients respectively for a total of 288 (0.82%) CDI cases. Based on univariate analysis across the two sites, older patients, white race, Medicare primary coverage, prolonged LOS, discharge to long term care/skilled nursing facility, higher co-morbidity score [11], prior hospitalization within 90 days, additional days in critical care units, ICD-9-CM diagnosis/procedure/admission diagnosis, prolonged antibiotic therapy and higher number of antibiotic classes received during hospitalization were associated with increased risk of CDI ≥28 days after discharge (Table 1). Certain antibiotic classes such as cephalosporins, fluoroquinolones, intravenous vancomycin, sulfonamides and beta-lactams/beta-lactamase inhibitor combinations were associated with increased CDI risk.

Table 1.

Demographic and Clinical Characteristic of CDI Cases and Non Cases at Two Large Academic Medical Centers

	Cases		Non-Cases		p-value
	288		34898
Site
NY	124		13653
CT	164		21245
Gender					p=0.3910
Female	167	57.99%	21102	60.47%
Male	121	42.01%	13796	39.53%
Age Category					p<0.0001
18-39	22	7.64%	11693	33.51%
40-49	29	10.07%	4912	14.08%
50-64	92	31.94%	8168	23.41%
65-74	44	15.28%	4209	12.06%
75+	101	35.07%	5916	16.95%
Race					p=0.0002
White	214	74.31%	22825	65.40%
Black	59	20.49%	7318	20.97%
Hispanic	13	4.51%	3309	9.48%
Other	2	0.69%	1446	4.14%
Primary Insurance					p<0.0001
Medicare	184	63.89%	12171	34.88%
Medicaid	44	15.28%	6988	20.02%
Private	56	19.44%	14509	41.58%
Other	4	1.39%	1230	3.52%
Number of ICD9 Procedures					p=0.0037
0	100	34.72%	11608	33.26%
1	65	22.57%	7767	22.26%
2	46	15.97%	5904	16.92%
3	25	8.68%	4239	12.15%
4	9	3.13%	2155	6.18%
5+	43	14.93%	3225	9.24%
Length of Stay					p<0.0001
1	39	13.54%	7848	22.49%
2	40	13.89%	7670	21.98%
3	33	11.46%	5751	16.48%
4	29	10.07%	3649	10.46%
5	26	9.03%	2279	6.53%
6-9	54	18.75%	4357	12.48%
10+	67	23.26%	3344	9.58%
Admission from Location					p=0.1521
Acute Care Hospital	10	3.47%	740	2.12%
LTACH	0	0.00%	0	0.00%
Home/Outpatient	269	93.40%	33408	95.73%
LTC/SNF	8	2.78%	501	1.44%
Other	1	0.35%	249	0.71%
Discharge to Location					p<0.0001
Acute Care Hospital	8	2.78%	670	1.92%
LTACH	3	1.04%	131	0.38%
Home/Outpatient	207	71.88%	30016	86.01%
LTC/SNF	65	22.57%	3549	10.17%
Other	5	1.74%	532	1.52%
Gagne Co-Morbidity Score					p<0.0001
0 or less	42	14.58%	6786	19.45%
1	40	13.89%	7227	20.71%
2	55	19.10%	3714	10.64%
3	34	11.81%	2166	6.21%
4+	98	34.03%	4103	11.76%
No score	19	6.60%	10902	31.24%
Past Hospital Visits within 90 days					p<0.0001
0	204	70.83%	31063	89.01%
1	60	20.83%	2960	8.48%
2+	24	8.33%	875	2.51%
Days in critical care					p<0.0001
0	235	81.60%	31325	89.76%
1	10	3.47%	1252	3.59%
2	10	3.47%	825	2.36%
3	6	2.08%	450	1.29%
4	5	1.74%	270	0.77%
5+	22	7.64%	776	2.22%
ICD9 CCS Diagnosis Category					p<0.0001
2-Septicemia	16	5.56%	576	1.65%
237-Complication of device/implant/graft	12	4.17%	560	1.60%
108-CHF nonhypertensive	11	3.82%	603	1.73%
159-UTI	10	3.47%	489	1.40%
50-Diabetes mellitus w complications	9	3.13%	476	1.36%
122-Pneumonia	9	3.13%	568	1.63%
157-Acute Renal Failure	7	2.43%	398	1.14%
109-Acute CVD	6	2.08%	551	1.58%
197-Skin infections	6	2.08%	589	1.69%
238-Complications of surgical procedures/medical care	6	2.08%	390	1.12%
Other	196	68.06%	29698	85.10%
ICD9 CCS Procedure Category					p<0.0001
None	100	34.72%	11634	33.34%
216-Mechanical ventilation	13	4.51%	556	1.59%
61-Other OR procedures on vessels	8	2.78%	332	0.95%
108-Indwelling catheter	8	2.78%	433	1.24%
54-Other vascular catheterization	7	2.43%	282	0.81%
58-Hemodialysis	6	2.08%	225	0.64%
179-CT scan abdomen	6	2.08%	338	0.97%
146-Treatment, fracture, dislocation of hip	6	2.08%	246	0.70%
193-Diagnostic ultrasound of heart	5	1.74%	383	1.10%
205-Arterial blood gases	5	1.74%	278	0.80%
224-Cancer chemotherapy	5	1.74%	112	0.32%
Other	119	41.32%	20079	57.54%
ICD9 CCS Admission Diagnosis Category					p<0.0001
259-Unclassified	16	5.56%	819	2.35%
251-Abdominal pain	12	4.17%	1040	2.98%
133-Other lower respiratory disease	12	4.17%	1005	2.88%
245-Syncope	11	3.82%	685	1.96%
122-Pneumonia	10	3.47%	531	1.52%
197-Skin infections	10	3.47%	632	1.81%
102-Nonspecific chest pain	9	3.13%	1211	3.47%
127-COPD	7	2.43%	213	0.61%
157-Acute Renal Failure	7	2.43%	204	0.58%
Other	194	67.36%	28558	81.83%
Days on antibiotics					p<0.0001
0	130	45.14%	19078	54.67%
1	8	2.78%	3679	10.54%
2	23	7.99%	3143	9.01%
3	11	3.82%	2047	5.87%
4	18	6.25%	1483	4.25%
5	22	7.64%	1021	2.93%
6	4	1.39%	817	2.34%
7	10	3.47%	660	1.89%
8	7	2.43%	525	1.50%
9	5	1.74%	398	1.14%
10	7	2.43%	309	0.89%
11	4	1.39%	253	0.72%
12	5	1.74%	211	0.60%
13	4	1.39%	165	0.47%
14	4	1.39%	148	0.42%
15+	26	9.03%	961	2.75%
		0.00%		0
Antibiotic Class (Ever)
Aminoglycosides	7	2.43%	695	1.99%	p=0.5957
1st and 2nd Gen Cephalosporins	33	11.46%	6378	18.28%	p=0.0028
3rd and 4th Gen Cephalosporins	59	20.49%	2713	7.77%	p<0.0001
Lincosamides	8	2.78%	1151	3.30%	p=0.6222
Fluoroquinolones	68	23.61%	3723	10.67%	p<0.0001
Macrolides	7	2.43%	686	1.97%	p=0.5718
Vancomycin IV	30	10.42%	1586	4.54%	p<0.0001
Sulfa	19	6.60%	952	2.73%	p<0.0001
Beta-lactam/Beta-lactamase inhibitor combinations	61	21.18%	3231	9.26%	p<0.0001
Carbapenems	5	1.74%	441	1.26%	p=0.4207
Penicillins	7	2.43%	1733	4.97%	p=0.0481
Other	45	15.63%	3381	9.69%	p=0.0007
Number of different antibiotic classes					p<0.0001
0	130	45.14%	19078	54.67%
1	58	20.14%	9044	25.92%
2	40	13.89%	4076	11.68%
3	35	12.15%	1747	5.01%
4	20	6.94%	656	1.88%
5+	5	1.74%	297	0.85%
Days on Antibiotics within Past 90 Days of index visit					p<0.0001
0	242	84.03%	33305	95.44%
1	2	0.69%	186	0.53%
2	2	0.69%	219	0.63%
3	9	3.13%	163	0.47%
4	4	1.39%	134	0.38%
5	2	0.69%	105	0.30%
6	0	0.00%	104	0.30%
7	6	2.08%	87	0.25%
8	1	0.35%	74	0.21%
9	0	0.00%	66	0.19%
10+	20	6.94%	455	1.30%
Unit Type Ever
Critical Care (CC)	57	19.79%	5046	14.46%	p=0.0105
Emergency Department (ED)	106	36.81%	9312	26.68%	p=0.0001
Labor and Delivery/Maternity (MAT)	3	1.04%	5651	16.19%	p<0.0001
Speciality Care (SCA)	31	10.76%	1056	3.03%	p<0.0001
Medical Ward (WARD)	146	50.69%	14820	42.47%	p=0.0049
Surgical Ward (SURG)	99	34.38%	10150	29.08%	p=0.0491
Psych	1	0.35%	948	2.72%	p=0.0134
Other	29	10.07%	2809	8.05%	p=0.2099
Inpatient Gerontology Ward (W-GNT)	11	3.82%	781	2.24%	p=0.0716
Inpatient Gynecology Ward (W-GYN)	7	2.43%	2206	6.32%	p=0.0068

A multivariate Cox proportional hazards model was then developed for each site using CDI ≥28 days post discharge as the outcome. The backward elimination procedure was employed starting with all the variables identified in the univariate analysis (Table 1).

For the complex model, after backward selection, the following variables remained associated with increased risk of CDI after discharge for both sites: age, ICD-9-CM diagnosis, procedure and admission diagnosis CCS risk category, number of ICD-9-CM procedure codes, and number of past hospital visits within 90 days. For CT, the number of different antibiotic classes administered during index hospitalization also remained, while at NY, gender, co morbidity score, and maternity and specialty care wards also remained. In addition, each site had specific antibiotic classes associated with increased risk of CDI after discharge, however, none overlapped.

A similar multivariate Cox proportional hazards model was developed for each site, but only variables expected to be readily available at time of discharge were included (simple scenario). After backward selection, ICD-9-CM admission diagnosis CCS risk category, older age, , and previous hospitalization remained in the model at both sites. For CT, number of antibiotic classes, receipt of 3^rd/4^th generation cephalosporin, primary payer, past hospital antibiotic exposure within 90 days of index hospitalization, and admission to specialty care ward also significantly associated with CDI post discharge. For NY, use of clindamycin or a fluoroquinolone during index hospitalization, prolonged LOS, gender, and exposure to wards other than surgical, medical, critical care, labor and delivery, and emergency department also remained in the model.

At each site for both complex and simple models, a risk score was developed. This risk score was then validated at the other hospital site as a cross site validation cohort. In the derivation cohorts, the risk score performed well. However, when the risk score was applied to the cross site validation cohort, performance dropped considerably. For example, in the cross site validation cohorts, the area under the ROC curve typically was estimated to be around 0.7 even though the estimated area was much higher in the derivation cohorts, >0.8 (Table 2). The models developed at each individual site did not perform as well when implemented at another hospital, likely due to inconsistency in important predictors between the two sites.

Table 2.

Results from Predicting CDI ≥28 Days Post Discharge in Derivation and Validation Cohorts at Two Large Academic Centers

Model	ROC Area	OR High Risk	% CDI High Risk	Index Score Denoting High Risk*
Complex
CT Derivation	0.87	14.89	3.34	NA
NY Validation	0.71	2.76	2.08	NA
NY Derivation	0.90	18.86	4.73	NA
CT Validation	0.69	2.71	1.59	NA
Simple
CT Derivation	0.84	9.21	2.60	NA
NY Validation	0.70	3.28	2.13	NA
NY Derivation	0.85	9.96	2.92	NA
CT Validation	0.71	2.77	1.38	NA
Overlap
Derivation	0.78	6.28	2.09	4.00
Validation	0.75	5.37	1.57	4.00
NY Validation	0.75	5.77	1.89	4.00
CT Validation	0.75	5.17	1.39	4.00
Validation on repeated samples	0.75	5.05	1.86	7.00
Validation on repeated samples (LOS Model)	0.74	4.84	1.72	14.00

^*For the final models, the index score denoting the point to divide the population into high risk and low risk cohorts is provided and based on the 25th percentile of the derivation cohort.

Since cross site validation models did not perform well, a third model (combined scenario) based on overlapping variables from the simple model and the complex model was developed. The combined model included ICD-9-CM admission diagnosis CCS risk category, age, past hospitalizations, and 3^rd/4^th generation cephalosporin, clindamycin, or fluoroquinolone use during index hospitalization. For the admission diagnosis risk category, only overlapping risk categories (n=24) were included, Table 4.

Table 4.

Admission Diagnosis (DX) Level and Clinical Classification Software for Use in Final Risk Index Score(CCS) Category

Admission DX Level	CCS Category	Description	ICD-9-CM Codes
2	58	Other nutritional; endocrine; and metabolic disorders	270.0 270.1 270.2 270.3 270.4 270.5 270.6 270.7 270.8 270.9 271.0 271.1 271.2 271.3 271.4 271.8 271.9 272.5 272.6 272.7 272.8 272.9 273.0 273.1 273.2 273.3 273.4 273.8 273.9 275.0 275.01 275.02 275.03 275.09 275.1 275.2 275.3 275.4 275.40 275.41 275.42 275.49 275.5 275.8 275.9 277.1 277.2 277.3 277.30 277.31 277.39 277.4 277.5 277.6 277.7 277.8 277.81 277.82 277.84 277.85 277.86 277.87 277.89 277.9 278.0 278.00 278.01 278.02 278.03 278.1 278.2 278.3 278.4 278.8 783.1 783.2 783.21 783.22 783.3 783.4 783.40 783.41 783.42 783.43 783.5 783.7 783.9 793.91 794.7 795.7 V12.2 V12.21 V12.29 V85.0 V85.21 V85.22 V85.23 V85.24 V85.25 V85.30 V85.31 V85.32 V85.33 V85.34 V85.35 V85.36 V85.37 V85.38 V85.39 V85.4 V85.41 V85.42 V85.43 V85.44 V85.45 V85.51 V85.53 V85.54
2	59	Deficiency and other anemia	280.0 280.1 280.8 280.9 281.0 281.1 281.2 281.3 281.4 281.8 281.9 282.0 282.1 282.2 282.3 282.4 282.40 282.43 282.44 282.45 282.46 282.47 282.49 282.7 282.8 282.9 283.0 283.1 283.10 283.11 283.19 283.2 283.9 284.0 284.01 284.09 284.1 284.11 284.12 284.19 284.2 284.8 284.81 284.89 284.9 285.0 285.21 285.22 285.29 285.8 285.9 282.46 282.47 282.49 282.7 282.8 282.9 283.0 283.1 283.10 283.11 283.19 283.2 283.9 284.0 284.01 284.09 284.1 284.11 284.12 284.19
2	108	Congestive heart failure; nonhypertensive	398.91 428.0 428.1 428.20 428.21 428.22 428.23 428.30 428.31 428.32 428.33 428.40 428.41 428.42 428.43 428.9
2	114	Peripheral and visceral atherosclerosis	440.0 440.1 440.2 440.20 440.21 440.22 440.23 440.29 440.4 440.8 440.9 443.9 557.0 557.1 557.9
2	122	Pneumonia (except that caused by tuberculosis or sexually transmitted disease)	003.22 020.3 020.4 020.5 021.2 022.1 031.0 039.1 052.1 055.1 073.0 083.0 112.4 114.0 114.4 114.5 115.05 115.15 115.95 130.4 136.3 480.0 480.1 480.2 480.3 480.8 480.9 481 482.0 482.1 482.2 482.3 482.30 482.31 482.32 482.39 482.4 482.40 482.41 482.42 482.49 482.8 482.81 482.82 482.83 482.84 482.89 482.9 483 483.0 483.1 483.8 484.1 484.3 484.5 484.6 484.7 484.8 485 486 513.0 517.1
2	146	Diverticulosis and diverticulitis	562.00 562.01 562.02 562.03 562.10 562.11 562.12 562.13
2	159	Urinary tract infections	032.84 590.00 590.01 590.10 590.11 590.2 590.3 590.80 590.81 590.9 595.0 595.1 595.2 595.3 595.4 595.81 595.82 595.89 595.9 597.0 597.80 597.81 597.89 598.00 598.01 599.0
2	197	Skin and subcutaneous tissue infections	020.1 021.0 022.0 031.1 032.85 035 039.0 680.0 680.1 680.2 680.3 680.4 680.5 680.6 680.7 680.8 680.9 681.00 681.01 681.02 681.10 681.11 681.9 682.0 682.1 682.2 682.3 682.4 682.5 682.6 682.7 682.8 682.9 684 685.0 685.1 686.0 686.00 686.01 686.09 686.1 686.8 686.9
2	226	Fracture of neck of femur (hip)	820.00 820.01 820.02 820.03 820.09 820.10 820.11 820.12 820.13 820.19 820.20 820.21 820.22 820.30 820.31 820.32 820.8 820.9 905.3 V54.13 V54.23
2	245	Syncope	780.2
2	251	Abdominal pain	789.0 789.00 789.01 789.02 789.03 789.04 789.05 789.06 789.07 789.09 789.60 789.61 789.62 789.63 789.64 789.65 789.66 789.67 789.69
2	252	Malaise and fatigue	780.7 780.71 780.79
3	49	Diabetes mellitus without complication	249.00 250.00 250.01 790.2 790.21 790.22 790.29 791.5 791.6 V45.85 V53.91 V65.46
3	63	Diseases of white blood cells	288.0 288.00 288.01 288.02 288.03 288.04 288.09 288.1 288.2 288.3 288.4 288.50 288.51 288.59 288.60 288.61 288.62 288.63 288.64 288.65 288.66 288.69 288.8 288.9 289.53
3	105	Conduction disorders	426.0 426.10 426.11 426.12 426.13 426.2 426.3 426.4 426.50 426.51 426.52 426.53 426.54 426.6 426.7 426.81 426.82 426.89 426.9 V45.0 V45.00 V45.01 V45.02 V45.09 V53.3 V53.31 V53.32 V53.39
3	127	Chronic obstructive pulmonary disease and bronchiectasis	490 491.0 491.1 491.2 491.20 491.21 491.22 491.8 491.9 492.0 492.8 494 494.0 494.1 496
3	157	Acute and unspecified renal failure	584.5 584.6 584.7 584.8 584.9 586
3	163	Genitourinary symptoms and ill-defined conditions	599.6 599.60 599.69 599.7 599.70 599.71 599.72 599.8 599.89 599.9 788.1 788.2 788.20 788.21 788.29 788.3 788.30 788.31 788.32 788.33 788.34 788.35 788.36 788.37 788.38 788.39 788.4 788.41 788.42 788.43 788.5 788.6 788.61 788.62 788.63 788.64 788.65 788.69 788.7 788.8 788.9 788.91 788.99 791.0 791.1 791.2 791.3 791.4 791.7 791.9 793.5 794.4 V13.0 V13.00 V13.02 V13.03 V13.09 V41.7 V43.5 V44.5 V44.50 V44.51 V44.52 V44.59 V44.6 V47.4 V47.5 V53.6 V55.5 V55.6
3	237	Complication of device; implant or graft	279.50 279.51 279.52 279.53 414.02 414.03 414.04 414.05 414.07 440.30 440.31 440.32 569.60 569.61 569.69 596.82 596.83 629.31 629.32 996.00 996.01 996.02 996.03 996.04 996.09 996.1 996.2 996.30 996.31 996.32 996.39 996.4 996.40 996.41 996.42 996.43 996.44 996.45 996.46 996.47 996.49 996.51 996.52 996.53 996.54 996.55 996.56 996.57 996.59 996.6 996.60 996.61 996.62 996.63 996.64 996.65 996.66 996.67 996.68 996.69 996.7 996.70 996.71 996.72 996.73 996.74 996.75 996.76 996.77 996.78 996.79 996.80 996.81 996.82 996.83 996.84 996.85 996.86 996.87 996.88 996.89 996.90 996.91 996.92 996.93 996.94 996.95 996.96 996.99 999.31 999.32 999.33
3	653	Delirium, dementia, and amnestic and other cognitive disorders	290.0 290.10 290.11 290.12 290.13 290.20 290.21 290.3 290.40 290.41 290.42 290.43 290.8 290.9 293.0 293.1 294.0 294.1 294.10 294.11 294.20 294.21 294.8 294.9 310.0 310.2 310.8 310.81 310.89 310.9 331.0 331.1 331.11 331.19 331.2 331.82 797
4	2	Septicemia	003.1 020.2 022.3 036.2 038.0 038.1 038.10 038.11 038.12 038.19 038.2 038.3 038.40 038.41 038.42 038.43 038.44 038.49 038.8 038.9 054.5 449 771.81 790.7 995.91 995.92
4	62	Coagulation and hemorrhagic disorders	286.0 286.1 286.2 286.3 286.4 286.5 286.52 286.53 286.59 286.6 286.7 286.9 287.0 287.1 287.2 287.3 287.30 287.31 287.32 287.33 287.39 287.4 287.49 287.5 287.8 287.9 289.81 289.82 289.84 782.7
4	151	Other liver diseases	570 571.5 571.6 571.8 571.9 572.0 572.1 572.2 572.3 572.4 572.8 573.0 573.4 573.5 573.8 573.9 782.4 789.1 789.5 789.59 790.4 790.5 794.8 V42.7
4	248	Gangrene	440.24 785.4

Cohorts from CT and NY were combined then randomly separated into development and validation cohorts. Data from the development cohort was used to create the risk index. As previously, that risk index was assessed in the validation cohort. Compared to the cross site validation cohorts, performance improved in the validation cohort for the combined model. The area under the ROC curve improved to 0.75 and was consistent between both sites, suggesting that simple models using only four consistent variables performed better than previous models.

To develop a final model, the randomization, model estimation, and creation of the risk score index was conducted 500 times. The average area under the ROC curve was 0.75. The final risk index score was based on the repeated 500 samples, Table 3. An additional model using LOS in place of admission diagnosis categories performed nearly as well, c-score=0.74, Table 2 and Table 3.

Table 3.

Final Risk Index Score

Characteristic	Points
Age 40-49	4
Age 50-64	6
Age 65-74	6
Age 75+	8
Admission DX Level 2*	2
Admission DX Level 3*	4
Admission DX Level 4*	5
Fluoroquinolone Ever	1
3rd/4th generation cephalosporin Ever	2
Lincosamide Ever	1
Past Hospital Visits
1	3
2+	5
Alternative Model with LOS
Age 40-49	8
Age 50-64	12
Age 65-74	12
Age 75+	15
LOS (days)
1	0
2-3	1
4-9	4
10+	8
Fluoroquinolone Ever	2
3rd/4th generation cephalosporin Ever	3
Lincosamide Ever	1
Past Hospital Visits
1	6
2+	8

^*Admission DX Level Based on Admission Diagnosis CCS Category. Diagnosis CCS Category definition provided in Table 4.

The cut off point for high vs. low risk groups was arbitrarily set at the 25^th percentile of the derivation cohort for those with CDI, Table 2. However, this point can range and be adjusted. As the score increases, the proportion of the cohort who would be included in the high risk group decreases while the CDI attack rate in that group increases. We further generalized to the situation in which a preventative trial, such as a vaccine trial, is to be designed. We estimated the sample size required from the attack rate and further determined the estimated proportion at high risk to determine the feasibility of the trial. For our example, we assumed a similar hospital with 20,000 index visits, and an efficacy of 50%. We then plotted the necessary sample size and the estimated number at high risk for each risk score (Figure 1). For risk index scores up to 7, the number expected to be at high risk was greater than the estimated sample size.

Figure 1.

In order to determine the feasibility of a vaccine trial, we estimated the required sample size and proportion of patients at high risk available to be included in a vaccine trial for a range of risk scores based on the final risk score developed in our model. For our example, we assumed a hospital with 20,000 annual patient index visits, and a vaccine efficacy of 50%, and we then plotted the necessary sample size and the estimated number at high risk for each risk score. For risk index scores up to 7, the number expected to be at high risk was greater than the estimated sample size.

Discussion

Our study identified specific parameters for a risk index that can be applied at hospital discharge to identify a patient population with increased risk of CDI ≥28 days after discharge that could be targeted for vaccine trials. The CDI risk index performed well in the two academic centers studied, and validation of the risk index observed an CDI attack rate ≥28 days following hospitalization of 1.5%–2.0% with the risk of CDI among high-risk patients being 5 times greater than other patients. The risk index was developed using parameters that are likely to be readily available at the time of discharge.

Previous CDI risk-prediction models only took into consideration CDI developing during the time of hospitalization or were prediction models developed for recurrent CDI.[12-14] Because EIP CDI surveillance is a longitudinal population-based surveillance that includes both inpatient and outpatient laboratories, we are able to capture CDI episodes occurring among the population under surveillance including those occurring in the post-discharge period.

Most of the factors we found to be associated with post-discharge CDI development such as age, LOS, exposure to high risk antibiotics, and past hospital visits are not surprising. However, development of scores associated with those risk factors and the determination of CDI attack rates based on a patient's overall score provide further insights into the design of clinical trials to evaluate interventions for CDI prevention. As demonstrated by our study, using very high risk scores in study designs will substantially decrease the number of patients eligible to be included and could result in study failure as the necessary sample size will unlikely be reached.

There were several limitations in this analysis. First, while administrative data was used in conjunction with other data creating a potential for misclassification,[15, 16] we expect any bias due to misclassification to be non-differential and not substantial.[17] Second, since our combined model used data from both hospitals and certain minority populations may not be well represented, data from additional hospitals would help determine the generalizability of our risk index. However, our risk index has recently been validated using administrative data from a large number of hospitals.[18] In addition, especially for the complex scenario, implementing the risk index in a clinical setting would likely be difficult. Even for the models using only four variables, implementing the index in a clinical setting may not be straight forward due to the admission diagnosis categorization. Using LOS in place of admission diagnosis produced a model that performed nearly as well. Such a model may be more suitable to explore in other hospitals given potential differences between coding practices. Further, choosing an appropriate cut off point to divide cohorts into high and low risk discharges was highly subjective, but that allows the risk index to be applied in other situations depending on the needs of the trial or study. Finally, while our analysis included CDI cases identified in both inpatient and outpatient settings, data on healthcare encounters or antibiotic exposures outside of the hospital, including information on antimicrobials prescribed at discharge, was not obtained. Outpatient encounters and pharmacy data could be important risk factors not accounted for in this analysis, and our final variables included in the risk score may be correlated with certain risk factors outside the hospital. However, while the inclusion of such data could improve our model and identify additional risk factors, the goal of this analysis was to identify a cohort based upon hospital data.

Our study had a number of strengths. First, confirmed CDI cases from EIP surveillance linked to hospital administrative data were used. Therefore, our outcome did not rely on ICD-9-CM codes which do not always reliably identify incident cases of CDI. Further, cases were not limited to those identified only in a hospital setting. There were a large number of index hospitalizations, greater than 35,000, included in the study. In addition, our data included pharmacy data which is not always readily available from large hospital data sets. This novel dataset observed consistent findings with past studies of CDI in a previously reported study observing an increased risk of CDI for patient receiving broad spectrum antibiotics.[19] Finally, by using two large academic hospitals we were able to validate the risk index, and identify predictors consistent at both sites.

In conclusion, we identified specific parameters for a risk index that can be applied at discharge to identify a patient population with increased risk of CDI ≥28 days following an acute care hospitalization. The CDI risk score we developed can provide insight for study designs to evaluate interventions to prevent CDI; a disease that affects millions of individuals and causes thousands of deaths.