Comparison of Five Assays for Detection of Clostridium difficile Toxin

Abstract

Performance characteristics of five assays for detection of Clostridium difficile toxin were compared using fresh stool samples from patients with C. difficile infection (CDI). Assays were performed simultaneously and according to the manufacturers' instructions. Patients were included in the study if they exhibited clinical symptoms consistent with CDI. Nonmolecular assays included glutamate dehydrogenase antigen tests, with positive findings followed by the Premier Toxin A and B Enzyme Immunoassay (GDH/EIA), and the C. Diff Quik Chek Complete test. Molecular assays (PCR) included the BD GeneOhm Cdiff Assay, the Xpert C. difficile test, and the ProGastro Cd assay. Specimens were considered true positive if results were positive in two or more assays. For each method, the Youden index was calculated and cost-effectiveness was analyzed. Of 81 patients evaluated, 26 (32.1%) were positive for CDI. Sensitivity of the BD GeneOhm Cdiff assay, the Xpert C. difficile test, the ProGastro Cd assay, C. Diff Quik Chek Complete test, and two-step GDH/EIA was 96.2%, 96.2%, 88.5%, 61.5%, and 42.3%, respectively. Specificity of the Xpert C. difficile test was 96.4%, and for the other four assays was 100%. Compared with nonmolecular methods, molecular methods detected 34.7% more positive specimens. Assessment of performance characteristics and cost-effectiveness demonstrated that the BD GeneOhm Cdiff assay yielded the best results. While costly, the Xpert C. difficile test required limited processing and yielded rapid results. Because of discordant results, specimen processing, and extraction equipment requirements, the ProGastro Cd assay was the least favored molecular assay. The GDH/EIA method lacked sufficient sensitivity to be recommended.

Clostridium difficile, an anaerobic, spore-forming, gram-positive bacillus, has been associated with antibiotic-induced diarrhea since 1974 and with pseudomembranous colitis since 1978.^1,2 Toxigenic strains are responsible for C. difficile infection (CDI). Transmitted via the fecal-oral route, CDI has been historically associated primarily with antibiotic therapy, however, community-acquired CDI has been reported.^3,4 Symptoms include but are not limited to diarrhea, abdominal pain, and leukocytosis.^5–8 Age 65 years or older, immunosuppression, history of gastrointestinal disease, and recent antibiotic therapy have been associated with CDI.^3,5–8 Annually, there are 9000 deaths from hospital-acquired CDI, 3000 postdischarge deaths, and 16,500 deaths from CDI acquired in nursing homes (Overview of Clostridium difficile Infections; http://www.cdc.gov/HAI/organisms/cdiff/Cdiff_infect.html, last accessed April 2010). Methods currently in use for detection of C. difficile toxin include toxigenic culture, cytotoxicity assay, initial screening with glutamate dehydrogenase (GDH) antigen tests with positive screens followed by subsequent assays to detect toxins A and B, and most recently, molecular assays to detect the tcdB gene.^9–11

The microbiology laboratory of the Lifespan network, which encompasses four hospitals in Rhode Island, performs 15,000 C. difficile assays per year. In this study, multiple assays were performed simultaneously using fresh stool specimens from patients who fulfilled the clinical criteria for CDI. Performance characteristics including the cost of each method were compared to determine the appropriate methods for use in each institution. The methods evaluated were the two-step algorithm currently used to detect C. difficile toxin [GDH testing followed by an enzyme immunoassay (EIA) for toxins A and B in GDH-positive specimens (GDH/EIA)] and four other assays including a combined GDH/toxin assay performed in a single cartridge and three molecular methods approved by the US Food and Drug Administration for detection of the tcdB gene. At Lifespan, the microbiology laboratory is mandated to provide C. difficile toxin results within 24 hours, seven days a week.

Materials and Methods

Patients and Specimen Selection

From July 28, 2009, to August 28, 2009, consecutive liquid or soft stool specimens were obtained from hospitalized patients and transported refrigerated to the Lifespan network laboratory for detection of C. difficile toxin. Patient medical records were reviewed retrospectively. Inclusion criteria for entry into the study were the presence of one or more CDI-associated symptoms including diarrhea, abdominal pain, previous CDI infection, recent antibiotic therapy, leukocytosis, fever, loss of appetite, gastrointestinal tract bleeding, and nausea or vomiting.^5–8 Patients who tested positive for CDI less than 1 month before specimen collection were excluded. The study design was approved by the Lifespan Institutional Review Board (#2142–10).

Assays

Nonmolecular Assays

All nonmolecular assays required visual interpretation of the results, and contained a positive control line.

The C. Diff Quik Chek test (TechLab, Inc., Blacksburg, VA), a membrane-bound lateral-flow immunoassay, was used to screen stool specimens for the presence of C. difficile GDH-specific antigens. Samples positive using the C. Diff Quik Check test underwent reflex testing using the Premier Toxin A and B Enzyme Immunoassay (Meridian Bioscience, Inc., Cincinnati, OH) to confirm the presence of C. difficile toxins A and B. For the second nonmolecular method, C. difficile GDH antigen and A and B toxins were simultaneously detected using antibodies specific to those antigens in a single cartridge (C. Diff Quik Chek Complete test; TechLab, Inc.).

Molecular Methods

The BD GeneOhm Cdiff Assay (Becton, Dickinson and Co., Franklin Lakes, NJ) was the first of three molecular methods assessed. If present in the stool, the tcdB gene was amplified using manual lysis and detected via a molecular beacon on emission of a fluorescent signal. Fluorescent emissions were monitored, and data were compiled using the SmartCycler II System (Cepheid, Sunnyvale, CA).

Testing using the second molecular method, Xpert C. difficile test (Cepheid, Sunnyvale, CA), was conducted using a self-contained cartridge that, along with the GeneXpert DX System (Cepheid), automated and integrated sample purification, nucleic acid amplification, and detection of the target sequence using RT-PCR for detection of the tcdB gene.

The final molecular assay assessed was the ProGastro Cd Assay (Prodesse, Inc., Waukesha, WI). After a manual process of stool clarification and nucleic acid extraction and purification using the NucliSENS easyMAG System (bioMérieux SA, Marcy l'Etoile, France), samples were added to the C. difficile master mix, which contained oligonucleotide primers and probes for the tcdB gene. PCR amplification and detection were performed using the SmartCycler II System (Cepheid).

Specimen Processing

On arrival in the laboratory, stool specimens were refrigerated until tested. All five assays were performed within 24 hours. Specimens were fresh and not frozen before processing. Assays were performed simultaneously from fresh specimens to yield optimum results and enable valid comparisons between assays. All assays were performed according to the respective manufacturer's instructions.

Laboratory Personnel

The C. Diff Quik Chek Complete test and subsequent EIAs were performed for clinical diagnostic purposes by the microbiology laboratory. After GDH/EIA was performed, two certified and licensed clinical laboratory scientists who specialize in molecular diagnostics (R.A.D. and F.W.) performed the C. Diff Quik Chek Complete test and all molecular methods.

Invalid Results

In addition to definitive positive and negative results, all analyzed molecular assays had a third test interpretation, that is, invalid results. Methods of resolution for invalid specimens varied between molecular assays; however, all were specified in each manufacturer's package insert. Specimens that initially yielded an invalid test result were reanalyzed, following the manufacturer's recommendation, and final interpretation of the specimen was determined. The rate of invalid results for each assay was determined.

Discrepant Analysis

Specimens with discrepant molecular results were frozen at −70°C after initial testing and sent to a reference laboratory for further testing. Discrepant samples underwent toxigenic culture. Culture-positive specimens were subsequently analyzed using PCR for both the tcdC gene and GDH, and with a toxin A and B enzyme-linked immunosorbent assay (ELISA). The reference laboratory labeled specimens positive according to results of ELISA, and both PCR targets as positive. Samples that were culture-positive, PCR positive for GDH but negative using ELISA toxin A and B, and PCR tcdC-negative were labeled nontoxigenic. The reference laboratory was blinded to the original individual PCR specimen results.

Data Interpretation

Specimens were considered true positive when at least two of the three molecular methods detected the presence of the tcdB gene.

Statistical Analysis

Fisher's exact test was performed to calculate the statistical significance of associated risk factors and sensitivity and specificity for each assay using commercially available software (PRISM, version 5.00 for Windows; GraphPad Software Inc., San Diego, CA). The null hypothesis was rejected at P < 0.05 (two-sided). Matched sample tables and the Youden index, a single characteristic that captures the performance of a test, was applied to each molecular method to determine comparability, as previously described.¹² The Youden index was calculated as follows: (sensitivity + specificity) − 1. Method with the calculated Youden index closest to 1 exhibited comparatively superior performance.

Cost-Effectiveness Analysis

Cost-effective analysis was performed as previously described using the formula: CE ratio = (Cost of molecular assay − Cost of nonmolecular assay) ÷ (Effect of molecular assay − Effect of nonmolecular assay).^13,14 Costs exclusive to the laboratory were included in the analysis. BD GeneOhm Cdiff assay cost was based on an all-inclusive reagent rental program. The BD GeneOhm Cdiff assay kit (48 tests) includes master mix for an additional 16 tests. Thus, unless a high number of invalid results was experienced (>30%), there was no additional cost for retesting of invalid results. Total cost for the Xpert C. difficile test included purchase of the instrument, service program, initial cartridges, and invalid retest cartridges. Cost of the ProGastro Cd assay included kits and service packages for both the easyMAG and SmartCycler II systems. Technologist time was not factored into this cost analysis. Marginal effect, the difference between molecular and nonmolecular assay effects, was defined as the difference between the Youden index values of the methods compared.¹³

Results

Patient Data

Of 89 patients who submitted specimens, 81 patients with symptoms compatible with CDI were included in the study. Their mean (median; range) age was 64.4 (71; 4 to 97) years, and 26 (32%) tested positive for CDI. Signs and symptoms of CDI in the patients evaluated are given in Table 1. The most common symptoms were diarrhea (69.1%), abdominal pain (63.0%), and leukocytosis (39.5%). Physicians requested testing for C. difficile toxins appropriately in 81 of 82 patients (98.8% of test requests), as determined by the presence of signs and symptoms consistent with CDI. Assessment for CDI in patients with known risk or symptoms resulted in detection of CDI 32.1% of the time. While leukocytosis was the only statistically significant independent symptom (P = 0.02; relative risk, 3.3; 95% confidence interval, 1.3 to 8.5), leukocytosis in conjunction with abdominal pain was statistically significant (P = 0.047; odds ratio, 3.2; 95% confidence interval, 1.1 to 9.1). CDI occurred in patients younger than 65 years; however, individuals 65 years or older exhibited a higher incidence of CDI (17 versus 9 cases, respectively). However, there was no statistical difference between the two age groups.

Table 1.

Signs and Symptoms in Population Analyzed

Variable	Patients No. (%)	CDI-Positive No. (%)
Symptom
Diarrhea	56 (69.1)	16 (61.5)
Abdominal pain	51 (63)	17 (65.4)
Leukocytosis⁎	32 (39.5)	15 (57.7)
Nausea	24 (29.6)	10 (38.5)
Loss of appetite	21 (25.9)	4 (15.4)
Vomiting	16 (19.8)	8 (30.8)
GI bleeding	12 (14.8)	4 (15.4)
Fever	11 (22.2)	4 (15.4)
Previous CDI	5 (6.2)	3 (11.5)
Associated risk
Female sex	34 (42.0)	12 (46.2)
Age ≥65 years	46 (56.8)	17 (65.4)
History of colitis, IBS, or other GI disorder	29 (35.8)	10 (38.5)
Previous antibiotic therapy	23 (28.4)	7 (26.9)
Immunocompromise	19 (23.5)	5 (19.2)
Nursing home	9 (11.1)	4 (15.4)

CDI, C. difficile infection; GI, gastrointestinal tract; IBS, irritable bowel syndrome.

^⁎Defined as >10 WBC per cubic millimeter. P = 0.02; relative risk, 3.3; 95% confidence interval, 1.3 to 8.5. No other symptoms were significant.

Performance Data

Performance data for each assay after invalid results and discrepant analysis testing was performed are given in Table 2. Sensitivity of the BD GeneOhm Cdiff assay, Xpert C. difficile test, ProGastro Cd assay, C. Diff Quik Chek Complete test, and two-step GDH/EIA were 96.2%, 96.2%, 88.5%, 61.5%, and 42.3%, respectively. Specificity of the Xpert C. difficile test was 96.4%, and for all other methods was 100%.

Table 2.

Comparison of Methods for Detection of CDI in Patients with Symptoms

Diagnostic method	No. true positive	No. false positive	No. false negative	No. true negative	Invalid results⁎, %	Sensitivity†		Specificity		Predictive value, %		Youden index, %
						%	95% CI	%	95% CI	Positive	Negative
GeneOhm Cdiff assay	25	0	1	55	4.0	96.2	88.0–96.2	100	96.1 to 100	100	98.2	96.2
Xpert C. difficile test	25	2	1	53	1.1	96.2	86.0–99.2	96.4	91.6 to 97.8	92.6	98.1	92.5
ProGastro Cd assay	23	0	3	55	0	88.5	79.3–88.5	100	95.6 to 100	100	94.8	88.5
Quik Chek Complete‡	16	0	10	55	NA	61.5	51.4–61.5	100	95.2 to 100	100	84.6	61.5
GDH/EIA§	11	0	15	55	NA	42.3	32.4–42.3	100	95.3 to 100	100	78.6	57.7

Eighty-one specimens were analyzed using all methods.

CDI, Clostridium difficile infection; CI, confidence interval; NA, not applicable.

^⁎Xpert demonstrated an additional 5.5% “error rate.” At repeat testing, 100% of invalid specimens and errors for both GeneOhm and Xpert were resolved and provided definitive results, which are included in the total.

^†P < 0.03 for all molecular methods versus nonmolecular methods; Quik Chek Complete versus GDH/EIA was not significant.

^‡GDH component was 96.2% sensitive and 81.8% specific.

^§GDH component was 96.2% sensitive and 76.4% specific.

Invalid Results

Tests with results initially determined as invalid were repeated per the manufacturer's instructions. The rates of invalid results for the BD GeneOhm Cdiff assay, Xpert C difficile test, and ProGastro Cd assay were 4.0%, 1.1%, and 0%, respectively. Repeat testing of all specimens with initially invalid results yielded a definitive negative result.

Discrepant Results

Of the 81 stool specimens tested, six were considered discordant between molecular methods. Two of two specimens positive using the BD GeneOhm Cdiff assay and Xpert C. difficile test but negative using the ProGastro Cd assay were resolved as true positive. One of three specimens positive using only the Xpert C. difficile test was resolved as true positive. One of one specimen positive using only the ProGastro Cd assay was resolved as true positive.

Cost-Effectiveness Analysis

Results of cost-effectiveness analysis for each assay evaluated are given in Table 3.

Table 3.

Yearly Cost-Effectiveness of Methods

Diagnostic method	Total cost, $	Effect, %⁎	CE ratio versus GDH/EIA
BD GeneOhm Cdiff assay	367,650.00†	96.2	$5894.81
Xpert C. difficile test	649,662.50‡	92.5	$14,625.36
ProGastro Cd assay	456,004.20§	88.5	$10,237.15
C.Diff Quik Chek Complete test	175,950.00	61.5	$9276.32
GDH/EIA	140,700.00	57.7	NA

Assuming 15,000 assays are performed in 1 year.

CE, cost-effectiveness; EIA, toxins A and B enzyme immunoassay; GDH, glutamate dehydrogenase; NA, not applicable.

^⁎For this analysis, effect was defined as the Youden index.

^†All-inclusive reagent rental program. No additional charge for invalid retesting. Includes cost of SmartCycler II System.

^‡Includes purchase of instrument, service program, initial cartridges, and invalid retest cartridges.

^§Includes purchase of easyMag, SmartCycler II System, service packages, and kits.

Discussion

This is the first study to evaluate simultaneously three Food and Drug Administration–approved molecular methods, the Xpert C. difficile test, the BD GeneOhm Cdiff assay, and the ProGastro Cd assay. Compared with nonmolecular methods, these assays detected 35% to 54% more specimens positive for C. difficile. The better performance of molecular versus nonmolecular methods was significant at P < 0.03 for all methods evaluated. This is consistent with previous studies that suggested that single independent molecular methods are 8.5% to 51.3% more sensitive than nonmolecular methods.^9,10,15–18 However, the data presented herein are in contrast to those of other investigators, who have demonstrated high sensitivity with GDH screening (90% to 100%).^19–21 The differences in GDH sensitivity in these studies relative to molecular assay performance could be due to geographic and genetically varied strains, timeliness of testing, sample condition, and inclusion of patients who recently tested positive. The present study eliminated potential bias related to testing timeliness, duplicate specimens, and patients who previously tested positive.

When comparing all assays using their calculated Youden index values, the BD GeneOhm Cdiff assay was the most efficient (96.2%), followed by the Xpert C. difficile test (92.5%), the ProGastro Cd assay (88.5%), the C. Diff Quik Chek Complete test (61.5%), and the GDH/EIA two-step algorithm (57.7%). Whereas molecular assays were similarly efficient, a significant difference in cost-effectiveness was observed due to great variance in associated laboratory costs.

Although the sample size was not sufficient to support statistical analysis for noninferiority, the results produced using the molecular methods were comparable. In our laboratory, the BD GeneOhm Cdiff assay demonstrated superior performance and cost-effectiveness when compared with both the Xpert C. difficile test and the ProGastro Cd assay. Invalid results occurred more often on initial testing with the BD GeneOhm Cdiff assay (4.0%); however, repeat testing of invalid results could be performed during the same shift from the frozen specimen lysate with no additional cost because each kit contained master mix overage of 33%. Repeat testing provided results 100% of the time, and all results were available within the required 24-hour turnaround time. Compared with the Xpert C. difficile test, the BD GeneOhm Cdiff assay required additional hands-on extraction and setup; however, it was also considerably less expensive. In addition, cost-effectiveness analysis revealed that despite the increased cost of the molecular assay, greater savings were attained with the BD GeneOhm Cdiff assay compared with the C. Diff Quik Chek Complete test and the two-step GDH/EIA method because of greatly enhanced performance. Testing the batch once a day worked well at our high-volume site because of less hands-on time; however, this benefit may not be realized in a laboratory with a smaller volume.

Turnaround time for the Xpert C. difficile test was quick, approximately 50 minutes, and the processing procedure was limited. However, the laboratory cost for reagents was twice that for the other molecular assays. In addition, the hands-on time at our large-volume site would have required a dedicated technologist to perform testing throughout the day, with the 16-bay instrument available. The rate of invalid results was1.1%, and the error rate was 5.5%, secondary to syringe pressure and probe check errors. Careful attention to modules with repeated errors is necessary to differentiate between specimen issues or module malfunction. Of all specimens, 6.6% were determined to be invalid or erroneous. Repeated testing was successful 100% of the time; however, the cost of the test doubled because a new cartridge was necessary for each test.

The ProGastro Cd assay was the least favored of the three molecular assays for the following reasons. According to the Food and Drug Administration–approved product package insert, stool specimens must be kept on ice during processing, and an easyMAG extraction instrument (bioMérieux SA) must be used for specimen processing. While many laboratories have this equipment, the coordination of workflow with other molecular assays that require use of this extraction instrument could be a factor in high-volume sites. In addition, this assay produced the greatest number of discordant results.

Compared with the molecular assays evaluated, both the antigenic C. Diff Quik Chek Complete test and the two-step GDH/EIA assay performed poorly. However, testing using nonmolecular assays may be the only option in certain settings, and has performed adequately as a two-step algorithm with GDH antigen as the preliminary test followed by either PCR or the C. Diff Quik Chek Complete test.^15,18,19,21 Because the C. Diff Quik Chek Complete test detected nearly 20% more specimens correctly in this evaluation, as well as having both the GDH and toxin assay performed simultaneously, this assay is preferred over the two-step GDH/EIA method. Because of the potential severity of CDI and the implications for infection control, institutions that provide nonmolecular test results should alert physicians to performance characteristics of nonmolecular assays and the continued need to consider treatment in patients in whom there is high suspicion of CDI positivity despite negative test results.

While only laboratory costs were considered in this analysis, a turnaround time of results more than once a day may have a positive effect on nursing and housekeeping issues related to patient care, initial placement, transfer, and discharge, and a more rapid test would be appropriate in some settings such as institutions with shared rooms or a limited number of available beds.^22–25 However, more rapid laboratory turnaround time does not always indicate rapid physician response. Delays in reviewing test results are common,²⁶ and delayed response is consistent with findings at our institution when traditional methods were replaced with more rapid procedures.²⁷ To realize the full benefit of rapid molecular techniques, variables other than laboratory turnaround time and physician response are critical to cost-effective implementation.

Based on our large test volume and mandated daily toxin reporting, the BD GeneOhm Cdiff assay was selected for use in operations performed on Monday through Friday, when manual processing and testing can be done within an 8-hour shift. The Xpert C. difficile test was chosen for late Friday afternoon and weekend processing to enable maximum workflow efficiency with limited personnel on weekend shifts while still being able to address the daily toxin result turnaround time requirement. The molecular methods selected for use in the Lifespan laboratory yielded a statistically higher number of positive CDI results. In addition, and important for purposes of instituting the new molecular test, the BD GeneOhm Cdiff assay was more cost-effective than the previously used GDH/EIA method.