A Clostridium difficile infection “intervention”: change in toxin assay results in lower number of Clostridium difficile infection cases without changes in patient outcomes

Abstract

Background

Clostridium difficile infection (CDI) is most commonly diagnosed using toxin enzyme immunoassays (EIA). A sudden decrease in CDI incidence was noted after the EIA was changed at Barnes-Jewish Hospital (BJH) in St. Louis, MO. The objective of the study was to determine if the decrease in CDI incidence related to the change in EIA resulted in adverse patient outcomes.

Methods

Electronic hospital databases were used to collect data on demographics, outcomes and treatment of inpatients who had a C. difficile toxin assay performed from 1/4/09 to 4/3/09 (Period A, pre-assay change) and from 5/21/09 to 8/17/09 (Period B, post-assay change).

Results

Assays were positive in 240 (19.7%) of 1221 patients during Period A and 106 (9.1%) of 1160 patients during Period B (p<0.01). There was no difference in mortality or discharge to hospice between periods (10.3% vs 10.1%, p=0.90). Patients tested in period B were less likely to receive metronidazole or oral vancomycin (p < 0.01).

Conclusions

A new EIA resulted in fewer positive tests and reduced anti-CDI therapy among patients. There was no difference in mortality between the two periods, suggesting the decreased incidence was due to increased assay specificity, not decreased sensitivity.

Clostridium difficile infection (CDI) is the leading cause of nosocomial infectious diarrhea.¹ CDI increases length of hospital stay, readmissions, hospital costs and risk of death. ^2-4 The diagnosis of CDI is typically established by testing a diarrheal stool sample for C. difficile toxins or toxigenic C. difficile. ⁵ Commercial enzyme immunoassays (EIA) that detect both toxins A and B or toxin A alone are used extensively due to their low costs and rapid turnaround time. The sensitivity of these tests ranges from 63-94%, with a specificity of 75-100%. ⁵

A sudden decrease in CDI incidence was noted with a change in toxin EIA assays at a large urban hospital. The initial concern was that the new assay might have a lower sensitivity than the old assay, potentially increasing the number of untreated CDI cases and CDI-associated adverse events. The purpose of this study was to determine if a decrease in CDI incidence related to the change in toxin EIA assays was associated with an increase in adverse patient outcomes in patients who had a negative assay for C. difficile toxins.

METHODS

A before-after cohort study was conducted at Barnes-Jewish Hospital (BJH), a 1250-bed, university-affiliated, tertiary care facility located in St. Louis, MO. Data on demographics, outcomes and treatment information were collected electronically on all inpatients with at least one C. difficile toxin assay performed between 1/4/09 and 4/3/09 [Period A, ProSpecT™ Clostridium difficile Toxin A/B (Remel, Lenexa, KS)] and between 5/21/09 and 8/17/09 [Period B, C. difficile TOX A/B II™ (TechLab, Blacksburg, VA)]. The change in EIA products occurred because the assay from period A was removed from the market due to manufacturing difficulties. Data from 4/4/09 to 5/20/09 were excluded because several different assays were employed during this time period. The microbiology laboratory tests only diarrheal stools for the presence of C. difficile toxin.

All data were collected from electronic hospital databases. Data collected included patient race, gender, age, nadir albumin measured within 48 hours after admission, peak temperature and white blood cell counts measured within 24 hours of specimen collection, CDI-related colectomy rate, length of hospital stay, and discharge disposition. BJH onset CDI was defined as all patients with a positive toxin assay > 48 hours from admission. CDI present on admission was defined as a positive toxin assay ≤ 48 hours from admission. The BJH onset rate was calculated per 10,000 patient days. The CDI present on admission rate was calculated as the number of cases per 1,000 admissions.

The primary outcome investigated was in-hospital mortality or discharge to hospice care. Secondary outcomes included length of hospital stay (LOS) after first test for C. difficile, colectomy for CDI, discharge to a long term care facility, and readmission in 30 days. Long term care facility was defined as any skilled nursing home or rehabilitation center. The number of patients who received metronidazole or oral vancomycin was also compared between the two periods.

The in-hospital crude mortality for BJH patients with a negative test for C. difficile prior to the change in assays was 10% (unpublished data). A priori it was determined that if the crude mortality among patients who tested negative increased to ≥15%, this would require immediate attention. To detect a 5% increase in crude mortality with 80% power and α = 0.05, it would be necessary to have 725 patient who are tested negative for C. difficile during period B. Based on the average number of assays performed per month at BJH, it was decided to include three months of assay results in each time period. Data were analyzed using χ² and logistic regression and Mann-Whitney U tests in SPSS 16.0 (SPSS Inc, Chicago, IL).

RESULTS

There were 1221 patients tested for C. difficile during period A and 1160 during period B. Two hundred and forty (19.7%) patients tested during period A had a positive assay on the first test submitted, compared to 106 (9.1%) during period B (p<0.01, Table 1). The incidence of BJH onset CDI decreased from 23.52 cases per 10,000 patient days to 8.69 (RR=0.37, CI=0.33-0.41). The incidence of CDI on admission also dropped from 6.22 cases per 1000 admissions to 2.86 (RR= 0.46, CI=0.40-0.53) (Figure 1). There was no difference between patients in the different time periods by race, gender, age or length of stay before the first test for C. difficile toxins (Table 1). There were more patients with leukopenia at the time of specimen collection during Period B (18.4% vs. 13.4%, p < 0.01), but no difference was found between the patients based on nadir albumin measured within 48 hours after admission or peak temperature measured within 24 hours of stool specimen collection (Table 1).

Table 1.

Univariate comparison of patient characteristics by assay time period

Patient Characteristics	Period A (n=1221) n (%)	Period B (n=1160) n (%)	p value
Positive on First Test	240 (19.7)	106 (9.1)	<0.01
White	807 (70.5)	791 (72.2)	0.38
Female	604 (49.5)	605 (52.2)	0.19
Age ≥60 years old	598 (49.0)	554 (47.8)	0.55
Time to First Test (days, median (range))	3 (0, 183)	3 (0, 48)	0.90
Albumin ≤ 2.5 g/dl	152 (14.9)	132 (13.7)	0.44
White blood cell count
<4 K cells/mm3	156 (13.4)	206 (18.4)	<0.01
4-12 K cells/mm3	556 (47.9)	522 (46.6)	Reference
>12 K cells/mm3	448 (38.6)	391 (34.9)	0.43
Temperature ≥38 °C	289 (24.0)	277 (24.2)	0.89

Since a new EIA was used in the hospital, the incidence of BJH onset CDI decreased from 23.52 cases per 10,000 patient days to 8.69 (RR=0.37, CI=0.33-0.41). The incidence of CDI upon admission also dropped from 6.22 cases per 1000 admissions to 2.86 (RR= 0.46, CI=0.40-0.53). BJH onset CDI was defined as all patients with a positive toxin assay > 48 hours from admission. CDI present on admission was defined as a positive toxin assay ≤ 48 hours from admission. The BJH onset rate was calculated per 10,000 patient days and the present on admission rate was calculated per 1,000 admissions. The CDI present on admission rate was calculated as the number of cases per 1000 admissions.

Similar proportions of patients during both periods received metronidazole or oral vancomycin before the toxin assays were sent. After the first assay was sent, patients tested in period B were less likely to receive metronidazole or oral vancomycin and more likely to receive anti-diarrheals (p < 0.01 for all) (Table 2). There was no difference in mortality or discharge disposition among the groups, and there was no difference in readmissions (Table 2). The length of stay after the first negative assay was similar in both periods (p=0.07). No patients required colectomy for CDI during either time period.

Table 2.

Treatment and outcome of the patients during the two periods [(Period A (between 1/4/09 and 4/3/09), Period B (between 5/21/09 and 8/17/09)]

	Period A (n=1221) n (%)	Period B (n=1160) n (%)	p value
Outcomes
Discharge Disposition
Home	777 (63.6)	739 (64.1)	Ref
Long Term Care	314 (25.7)	297 (25.6)	0.95
Hospice/Expired	125 (10.3)	117 (10.1)	0.90
Colectomy for CDI	0 (0)	0 (0)	-
Readmission within 30 Days	316 (25.9)	291 (25.1)	0.66
Length of stay after first negative [days, median (range)]	5 (0, 119)	5 (0, 83)	0.07
Patients with more than 1 test	498 (40.8)	548 (47.2)	< 0.01
Number of repeated tests 1[median (range)]	2 (1-15)	2 (1-15)	0.57
Treatments
Metronidazole
Before First Test	160 (13.1)	165 (14.2)	0.43
After First Test 2	504 (41.3)	414 (35.7)	<0.01
Oral Vancomycin
Before First Test	35 (2.9)	22 (1.9)	0.12
After First Test 2	185 (15.2)	85 (7.3)	<0.01
Anti-diarrheals
Before First Test	155 (12.7)	167 (14.4)	0.23
After First Test 2	267 (21.9)	312 (26.9)	<0.01

¹In patients who had a test repeated after an initial negative test

²Regardless of test results

Four hundred and ninety eight (40.8%) patients in period A and 548 (47.2%) patients in period B had repeated tests (p < 0.01, table 2). Among patients with more than one test, the median number of tests ordered until either a positive assay or hospital discharge did not differ between the time periods (median = 2 for both groups, p = 0.57). Among patients with > 1 assay sent, 102 of 498 (20.5%) patients in Period A and 20 of 548 (3.6%) patients in Period B were positive on a subsequent test after the initial negative test (p < 0.01, Table 3). For all patients tested, there was no difference in discharge disposition, length of hospital stay, readmission, or empirical treatment before or after the first assay between the two periods (Table 3). There was an increased length of hospital stay after the first assay for patients with repeated negative assay, but no difference in discharge disposition or empirical treatment in period B compared to period A (Table 3).

Table 3.

Patient outcome with and without repeated tests during the two periods [(Period A (between 1/4/09 and 4/3/09), Period B (between 5/21/09 and 8/17/09)]

	Period A (n=1221) n (%)	Period B (n=1160) n (%)	p value
First assay positive	240 (19.7)	106 (9.1)	< 0.01
Discharge Disposition
Home	142 (59.7)	64 (61.9)	ref
Long term care	69 (29.0)	31 (29.5)	0.94
Hospice/expired	27 (11.3)	9 (8.6)	0.44
Readmission within 30 Days	67 (27.9)	38 (35.8)	0.14
Metronidazole before first assay	30 (12.5)	12 (11.3)	0.76
Metronidazole after first assay	204 (85.0)	91 (85.8)	0.84
Vancomycin before first assay	10 (4.2)	4 (3.8)	0.86
Vancomycin after first assay	83 (34.6)	35 (33.0)	0.78
First assay negative and no repeat test	509 (41.7)	531 (45.8)	0.04
Discharge Disposition
Home	358 (70.5)	370 (69.8)	ref
Long term care	113 (22.2)	118 (22.3)	0.95
Hospice/expired	37 (7.3)	42 (7.9)	0.69
Readmission within 30 Days	135 (26.5)	135 (25.4)	0.69
length of stay after first negative [median days (range)]	3 (0 - 80)	3 (0 - 76)	0.13
Metronidazole before first assay	73 (14.3)	78 (14.7)	0.87
Metronidazole after first assay	84 (16.5)	107 (20.2)	0.13
Vancomycin before first assay	14 (2.8)	8 (1.5)	0.17
Vancomycin after first assay	21 (4.1)	17 (3.2)	0.43
First assay negative and at least one repeat test positive	102 (8.4)	20 (1.7)	< 0.01
Number of repeated tests [median (range)]	2 (1 - 9)	1.5 (1 - 7)	0.58
Discharge Disposition
Home	54 (53.5)	11 (55.0)	ref
Long term care	25 (24.8)	7 (35.0)	0.56
Hospice/expired	22 (21.8)	2 (10.0)	0.32
Readmission within 30 Days	25 (24.5)	3 (15.0)	0.36
length of stay after first negative [median days (range)]	13 (1, 119)	14.5 (2, 59)	0.65
Metronidazole before first assay	13 (12.7)	0 (0.0)	0.99
Metronidazole after first assay	91 (89.2)	16 (80.0)	0.26
Vancomycin before first assay	5 (4.9)	0 (0.0)	0.99
Vancomycin after first assay	50 (49.0)	7 (35.0)	0.26
First assay negative and all repeat tests negative	370 (30.3)	503 (43.4)	< 0.01
Number of repeated tests	2 (1 - 15)	2 (1 - 15)	0.93
Discharge Disposition
Home	223 (60.4)	293 (58.8)	ref
Long term care	107 (29.0)	141 (28.3)	0.99
Hospice/expired	39 (10.6)	64 (12.9)	0.32
Readmission within 30 Days	89 (24.1)	115 (22.9)	0.68
length of stay after first negative [median days (range)]	7 (1, 80)	9 (1, 83)	0.03
Metronidazole before first assay	44 (11.9)	75 (14.9)	0.20
Metronidazole after first assay	125 (33.8)	200 (39.8)	0.07
Vancomycin before first assay	6 (1.6)	10 (2.0)	0.69
Vancomycin after first assay	31 (8.4)	26 (5.2)	0.06

DISCUSSION

The new toxin EIA used during period B resulted in fewer positive tests, decreased exposures to anti-CDI therapy, and an increase in antidiarrheal use. Lack of increases in crude mortality, discharges to long-term care facility, readmissions, and overall length of hospital stay between the two periods suggests the sudden decrease in CDI after implementation of the new assay was most likely due to better specificity, rather than decreased sensitivity, of the assay used during period B versus period A. Prior to collecting the data for this study the directors of medical intensive units, surgical intensive units, and bone marrow transplant were asked about the CDI incidence in their areas and whether they felt there was an increase in false-negative assays. This was done to determine if an urgent intervention was necessary since these are the patient care areas most likely to be impacted by sudden increases in morbidity and mortality due to CDI. None of the directors were aware of the assay change. All of them noticed a decrease in CDI and none of them felt there was an increase in false-negative assays. Their observations were supported by our study, which showed no difference in mortality rates, patient disposition, CDI-related colectomy rates or length of stay during the two periods.

More patients had repeated testing in Period B. Clinicians may consider negative assays to be falsely negative and thus order more tests. However it was found among patients who had more than one test performed, the numbers of tests repeated per patient were similar in both periods. This suggests the clinicians were confident in the results of the repeated assays during period B, despite significantly fewer positive assays on repeat testing during this period. The increase in repeat testing in period B may also be biased due to a larger proportion of patients in period B eligible for repeated testing, since patients were less likely to have an initial positive test compared to period A. If patients with an initial positive test are excluded, 51% of patients with an initial negative test in period A had repeat testing, compared to 52% in period B (p = 0.58). It is also not clear how the common practice at BJH to order “C. diff × 3,” rather than waiting for the test result to be available and reexamining the patient before ordering more tests for C. difficile, may have influenced these results as well.

Empirical treatment in spite of a negative assay could serve as a confounder in evaluating patient outcomes. An increase in empiric treatment for CDI after a negative test during period B would indicate the physicians did not believe the initial negative test. However, on subgroup analysis there was no increase in empirical treatment in patients after the first test was negative during period B (Table 3). Another interesting finding is that the proportion of patients who received medications that treat CDI prior to the result of the first assay did not differ regardless of the result of the first test, whether repeat testing was performed, or the result of the repeat tests (Table 3). This indicates the accuracy of the physicians’ pre-test probability for CDI was poor, and warrants additional study.

Specificity of an assay is important. False positive results can result in unnecessary treatments and contact isolation. Unnecessary treatment for CDI may result in adverse drug events and increase the risk of acquiring vancomycin-resistant enterococci, and increase the risk of CDI after treatment is stopped. ^{6, 7} The use of contact precautions is associated with decreased patient satisfaction, decreased contact with healthcare workers, and increased adverse events. ^8-10 False positive tests increase the observed incidence of CDI for a healthcare facility as well, diverting resources away from other infection prevention and control activities. An increase in CDI rates due to false positive test results can also adversely affect the validity of public reporting of CDI.^{11, 12}

It is possible cases of CDI were missed in period B compared to period A. The length of stay after the first negative assay in patients with multiple negative assays in period B was significantly longer than in period A (7 days versus 9 days, p = 0.03). There are several other potential explanations for these findings. Older, sicker patients are at increased risk for being colonized with C. difficile. ¹³ These patients are also more likely to have longer lengths of stay compared to less sick patients. According to the package inserts, the assay used during period A has a lower limit of detection ( ≥ 0.20 ng/ml toxin A and ≥ 0.61 ng/ml toxin B) than the assay used during period B (≥ 0.8 ng/ml toxin A and ≥ 2.5 ng/ml toxin B). Therefore it is possible the assay used in period A was more sensitive and may have detected C. difficile toxin in patients asymptomatically colonized with toxigenic C. difficile, but with diarrhea due to other reasons. This is supported by there being no difference in empiric treatment for CDI or patient outcomes in this subgroup of patients. It is also possible this finding was due to chance as it was marginally significant and part of a subgroup analysis without control for multiple comparisons.

There are some limitations to this study. Culture was not performed to isolate toxigenic C. difficile. Toxigenic culture is the gold standard for detecting toxin producing C. difficile from stool. ¹⁴ As a retrospective study and because the BJH microbiology lab discards stool specimens after seven days, it was not possible to do toxigenic culture to determine the true sensitivity and specificity of the assays in question. Chart review was not performed to assess patient presentations or outcomes. Because of the urgency to conduct this study to ensure there was no increase in patient harm as a result of the decrease in positive C. difficile assays and the number patients needed with an initial negative assay to have a meaningful result, we did not feel it was prudent to delay the findings of the study due to the time that would be needed to review the charts of over 2,000 patients. In addition, determining attributable outcomes by chart review is subject to significant bias. ¹⁵

The initial concern in regards to the sudden decrease in CDI incidence was the assay in period B had poor sensitivity. No differences were found in baseline patient characteristics, empiric treatment for CDI after an initial negative test for C. difficile, or patient outcomes in period B compared to period A. Although a formal assay comparison is required to know the sensitivity and specificity of the two assays, these findings indicate the decrease in CDI was most likely due to improved specificity of the assay in period B. Investigations into sudden changes in CDI incidence associated with changes in diagnostic assays should assess whether the changes may be related to changes in specificity, in addition to sensitivity.