Validation of Active Surveillance Testing for Clostridium difficile Colonization Using the cobas Cdiff Test

ABSTRACT

Clostridium difficile infection (CDI) is not declining in the United States. Nucleic acid amplification tests (NAAT) are used as part of active surveillance testing programs to prevent health care-associated infection. The objective of this study was to validate the cobas Cdiff Test on the cobas 4800 System (cobas) within a four-hospital system using prospectively collected perirectal swabs from asymptomatic patients at admission and during monthly intensive care unit (ICU) screening in an infection control CDI reduction program. Performance of the cobas was compared to that of toxigenic culture. Each positive cobas sample and the next following negative patient swab were cultured. The study design gave 273 samples processed by both cobas (137 positive and 136 negative) and culture (one negative swab was not cultured). Discrepant analysis was performed using a second NAAT, the Xpert C. difficile Epi test (Xpert). This strategy was compared to a medical record review for antibiotic receipt that would inhibit growth of C. difficile in colonic stool. None of the cobas-negative samples were culture positive. The cobas positive predictive value was 75.2% (95% confidence interval [CI], 66.9% to 82%) and positive percent agreement was 100% (95% CI, 96.0% to 100%). Overall agreement between cobas and direct toxigenic culture was 87.6% (95% CI, 83.1% to 91%). For the cobas-positive/culture-negative (discrepant) samples, 7 Xpert-positive samples were from patients receiving inhibitory antimicrobials; only 4 of 23 Xpert-negative samples received these agents (P = 0.00006). Our results support use of the cobas as a reliable assay for an active surveillance testing program to detect asymptomatic carriers of toxigenic C. difficile.

INTRODUCTION

Clostridium difficile is an important health care-associated pathogen known to cause C. difficile infection (CDI), which is a spectrum of disease that ranges from moderate and severe diarrhea to pseudomembranous colitis, to toxic megacolon, and death. Frequency and severity of CDI have continued to increase in the United States and worldwide (1, 2), and CDI has now surpassed methicillin-resistant Staphylococcus aureus (MRSA) as the most common cause of health care-associated infection (2). Importantly, uncontrolled CDI has significant morbidity and mortality, with the median risk of ≥1 recurrence being 22% (range, 12% to 64%). Other significant risks include having a complicated disease course (median, 15%; range, 7% to 48%), experiencing treatment failure (median, 21%; range, 5% to 50%), and death (median, 19%; range, 8% to 53%) according to a recent critical review (3).

One of the current, yet unproven, CDI prevention efforts suggests that admission surveillance testing (AST) for C. difficile colonization may be of value (4). There are no Food and Drug Administration (FDA)-approved commercial assays for this application although one report utilized an FDA-cleared real-time PCR test that demonstrated a significant reduction in CDI when admission testing for toxigenic C. difficile was deployed (5). The goal of our study was to validate the cobas Cdiff Test on the cobas 4800 System (cobas), an FDA-cleared diagnostic assay using real-time PCR technology for detection of toxigenic C. difficile (toxin B gene) in stool samples (6), as a reliable method for performing AST for toxigenic C. difficile colonization. Our hypothesis was that this test would be a useful assay for admission surveillance testing. A secondary goal was to evaluate the use of a second nucleic acid amplification test (NAAT) for discrepant analysis when there were discordant results between the assay under study and direct (no broth enrichment) culture for toxigenic C. difficile by using a medical record review to identify those patients receiving antimicrobials likely to interfere with the growth of C. difficile.

MATERIALS AND METHODS

Patient population and specimen collection.

The NorthShore University HealthSystem includes four hospitals located in Evanston, Glenview, Highland Park, and Skokie, Illinois, with 908 inpatient beds and approximately 60,000 annual admissions. This study was conducted at Glenbrook Hospital and at the intensive care units (ICUs) at all four facilities during an infection control quality improvement intervention. Glenbrook Hospital was selected to perform a pilot of admission screening since it had the highest CDI rate at the time, and limited resources permitted the pilot program to be done at only one facility. Including the ICUs at all four hospitals provided samples from across our system. Patients were prospectively enrolled from July 2016 through August 2017. Perirectal (intra-anal) samples were collected from inpatients upon their admission using a premoistened, double-headed BBL culture swab with liquid Amies medium (Becton-Dickinson, Sparks, MD). All adult patients admitted to Glenbrook Hospital and to the intensive care units at all four facilities were eligible for inclusion. This study was done as an infection control quality improvement practice not subject to further Institutional Review Board (IRB) review, per IRB-approved policy.

Real-time PCR procedure.

After receipt of the swab specimens, the cobas Cdiff Test was performed according to the manufacturer's instructions. One swab was removed from the transport container and broken into cobas PCR medium, which was then briefly vortexed. The specimen was loaded directly on the automated cobas 4800 System for processing, PCR amplification, and detection (6). Positive and negative controls were included in each run as provided in the cobas kit.

Culture methods.

The second swab was directly plated onto prereduced anaerobically sterilized (PRAS) selective medium composed of cycloserine-cefoxitin-fructose agar with horse blood and taurocholate (CCFA-HT; Anaerobe Systems, Morgan Hill, CA) for C. difficile culture whenever the cobas test was positive. Only direct plating was used due to resource limitations and the fact that we could compare these results to the FDA clearance trial results where direct plating was also utilized for a comparative measure. The first negative cobas sample from a new patient after each positive patient was also cultured in the same manner. Plates were incubated anaerobically for 5 days, and presumptive C. difficile colonies were confirmed by Gram stain, aerotolerance, and a Pro-disk (Hardy Diagnostics, Santa Maria, CA). To confirm C. difficile colonies as toxigenic, strains were inoculated into anaerobic chopped-meat broth (Anaerobe Systems) that was incubated for 5 to 7 days at 35°C. Supernatant from the anaerobic chopped-meat broth was processed for detection of C. difficile toxin B using cell culture cytotoxicity testing according to the manufacturer's instructions (C. difficile Tox-B test; Techlab, Blacksburg, VA).

Discrepant analysis.

Results that were discrepant between the cobas and the anaerobic culture were evaluated by testing of the remnant swab using the Xpert C. difficile/Epi test (Xpert; Cepheid, Sunnyvale, CA). The samples that were also positive by the Xpert test were considered true-positive samples. We also reviewed the medical records of all patients with a positive cobas but negative culture results. The cobas result was considered a true positive if the patient was receiving oral vancomycin and/or oral metronidazole as therapy for confirmed CDI or intravenous piperacillin-tazobactam (7) and was Xpert positive. No other antimicrobial therapy (e.g., intravenous cefazolin, ceftriaxone, gentamicin, levofloxacin, metronidazole, vancomycin, and oral cephalexin or clindamycin) was considered to have a potential to interfere with a positive culture for toxigenic C. difficile. These data were compared to the results of the Xpert test in order to evaluate the use of this second NAAT as the discrepant reference method.

Statistical analysis.

We selected a convenience set of samples to culture for toxigenic C. difficile, which consisted of all positive specimens plus the first negative swab from a new patient following each positive test, as opposed to culturing all negative samples, due to resource limitations. This restricted the data analysis for the cobas to the positive predictive value (PPV) and percent agreement of the tests. This selection of samples for inclusion on the basis of cobas results (a total of 273 positive and negative tests) influences the calculated sensitivity and specificity by artificially increasing the prevalence of positive results for that test in the population, and so a usual sensitivity and specificity analysis cannot be done in the strict sense. We had already documented the sensitivity and specificity of the cobas assay (6), which suggested that if the PPV and percent agreement are similar to earlier results, then the sensitivity and specificity are expected to be comparable. Since toxigenic culture is not a perfect reference standard, we also calculated the positive percent agreement (PPA), negative percent agreement (NPA), and overall agreement (OA) for these two testing approaches (8). For comparison of use of a medical record review to that of the Xpert test for discrepant analysis, a two-sided Fisher's exact test was used.

RESULTS

The overall rate of cobas-positive tests in our investigation was 7.6%. A total of 273 samples had both real-time PCR (137 positive and 136 negative) and culture performed; one negative swab was not retained and could not be cultured. One real-time PCR sample grew a nontoxigenic C. difficile strain; all others were toxigenic. None of the cobas-negative samples were culture positive. The overall performance of the cobas compared to that of culture is shown in Table 1. The performance of the test after discrepant analysis using the Xpert is shown in Table 2. As can be seen, in both analyses there were no swab samples that were cobas negative and culture positive, giving a positive percent agreement of 100%. There were a total of 27 false-positive tests out of 137 positive assays (19.7%). With a positive test rate of 7.6%, this translates to 1.5% of patients placed into isolation due to a false-positive assay, assuming all admissions are tested.

TABLE 1.

cobas CDiff Test compared to direct toxigenic culture

cobas test resulta	Culture positive (no. of tests)	Culture negative (no. of tests)
Positive	103	34
Negative	0	136

^aPositive predictive value, 75.2% (95% CI, 66.9–82%); positive percent agreement, 100% (95% CI, 96.0–100%); negative percent agreement, 80% (95% CI, 73.4–85.3%); overall percent agreement, 87.6% (95% CI, 83.1–91.0%).

TABLE 2.

cobas Cdiff Test compared to direct toxigenic culture after discrepancy analysis using the Xpert C. difficile/Epi test

cobas test resulta	True positive (no. of tests)	True negative (no. of tests)
Positive	110	27
Negative	0	136

^aPositive predictive value, 80.3% (95% CI, 72.4–86.4%); positive percent agreement, 100% (95% CI, 95.8–100%); negative percent agreement, 83.4% (95% CI, 77.0–88.4%); overall percent agreement = 90.1% (95% CI, 86.0–93.1%).

Table 3 provides data on the use of the Xpert test for the discrepancy analysis. The medical record review confirmed that this was a helpful assay to use for determination of true-positive tests from patients receiving antimicrobials expected to inhibit the growth of C. difficile in anaerobic culture, with seven samples testing positive by the Xpert. Only 4 of 23 Xpert-negative samples received these agents, indicating the reliability of this strategy as a tool for determination of true-positive results where an initial NAAT is positive but culture for toxigenic C. difficile is negative (P = 0.00006).

TABLE 3.

Evaluation of the Xpert C. difficile/Epi test as a tool for discrepant analysis compared to antibiotic receipt active against C. difficile in stool

Treatment	Xpert result with cobas-positive test (no. of tests)a		Significance
	Positive	Negative
Inhibitory antibiotics	7	4	P = 0.00006
No inhibitory antibiotics	0	23

^aTotal number of tests, 34.

DISCUSSION

In this validation of the cobas for use in active surveillance testing on perirectal (intra-anal) swab specimens, we demonstrated that the test had performance similar to that of the FDA-cleared test for detection of toxigenic C. difficile on unformed stool samples. To our knowledge, this is the first report that analyzes the actual performance of a commercial NAAT for validation to use in an AST program, and the fact that it performed as expected is important. We also demonstrated that using a second NAAT from another manufacturer is a reasonable approach for discrepant analysis, based on medical record review for patients receiving antimicrobial therapy that would be expected to inhibit the growth of C. difficile in anaerobic culture. This finding is also helpful for laboratories that plan to undertake a similar validation of a commercial real-time PCR assay detecting C. difficile for use in an active surveillance testing infection control program.

We expected that the cobas Cdiff Test would perform well since the published data using this assay to detect toxigenic C. difficile from unformed stool demonstrated that it was robust (6). Also, the specimens collected for AST of patients potentially colonized with toxigenic C. difficile are very similar to those used in the FDA-cleared diagnostic test. Specifically, the cobas Cdiff Test uses a swab to transfer sample from the collection container to the test system rather than another device, such as an inoculating loop (9). Inserting a swab into the anus is, in a sense, similar to inserting it into a cup containing stool. The internal anal sphincter is involuntary while the external is voluntary (10). Thus, it is typical for some stool and/or mucus from the gastrointestinal tract to be retained in the anus, and sampling this material should provide results for AST of patients colonized with toxigenic C. difficile similar to those of a diagnostic test that incorporates a swab sampling of stool as part of the assay procedure. Finally, perirectal and rectal swabs have been shown to have the same recovery rates of C. difficile when cultured on prereduced cycloserine-cefoxitin-brucella agar containing 0.1% taurocholic acid and lysozyme at 5 mg/ml (11), suggesting that sampling may not be critical as long as the swab enters the anus.

We performed a literature review using PubMed.gov (Medline) with the search terms Clostridium difficile surveillance, active surveillance testing, real-time PCR, and the names of FDA-cleared molecular tests that detect C. difficile to search the medical literature from 2005 through August 2017. We also searched Google using these terms as well as the term anus. In the recent literature there have been two reports suggesting that admission surveillance for toxigenic C. difficile may be useful and that colonized persons are a significant component of transmission within the hospital (12, 13). Mawer and colleagues found that patients with stool harboring a toxigenic C. difficile strain where the actual stool toxin test was negative appeared responsible for 25% of transmission events to others when such transmission could be ascertained (12). Furthermore, they suggested that since no source for most symptomatic CDI was detected, isolation of asymptomatically colonized patients, who are severalfold more numerous than disease patients, may result in a larger reduction in transmission and presumably less disease (12). Caroff and coworkers also reported that most cases of hospital-onset CDI do not have a genetic link to another patient strain and that clinical disease may be due to activation of C. difficile present on admission or to new transmissions from asymptomatic patients, thus leading to recommendations for new control measures, including admission screening (13). Admission surveillance testing is also suggested as a possible, but yet unproven, intervention for lowering the risk of hospital-onset CDI in the current UpToDate document on prevention and control of this disease (4).

Longtin and colleagues reported the impact of C. difficile admission testing, using a commercial NAAT, on the rate of CDI in their facility (5). They found a disease reduction of more than 50% when cases of CDI significantly decreased (P < 0.001) from 0.69 per 1,000 patient days before the intervention to 0.3 cases per 1,000 patient days after screening was implemented (5). However, there was no information provided on the analytical performance of the real-time PCR assay they used when applied to rectal surveillance swabs. Terveer et al. also examined the suitability of PCR to detect asymptomatic colonization (14). In this study, swabs inserted into the stool were tested using a real-time PCR test as well as by an enzyme-linked fluorescent assay (ELFA) detecting glutamate dehydrogenase (14). As expected, the real-time PCR test outperformed the ELFA, with the sensitivity and specificity of the real-time PCR being 96% and 93%, respectively, while the ELFA had a sensitivity and specificity of 87% and 91% (14). The only other study involving the use of perirectal swabs for C. difficile surveillance was that by Curry and coworkers and involved preenrichment of swabs in selective broth, followed by testing using a modified commercial real-time PCR assay (15). The main limitation of this method was the need for preamplification in broth that reduces the rapidity of result reporting.

The cost-effectiveness of admission surveillance testing has also been evaluated. The investigators modeled the cost-effectiveness for admission screening of all patients and found a cost savings for every level of contact precaution compliance when the prevalence of C. difficile colonization exceeded 5% (16), with our testing rate being 7.6%. They used a cost of surveillance testing up to $15.88 (15); but if one were able to perform targeted surveillance and have the same benefit for disease reduction, then the overall cost of testing could be lower, depending on the percentage of patients being screened. One important incentive to control the rate of CDI in U.S. hospitals is the Centers for Medicare and Medicaid Services (CMS) classification of CDI as a hospital-acquired condition (HAC), with potential for fines if rates are too high (17). Each year the HAC reduction program saves CMS $350 million by reducing payments to hospitals, thus incentivizing U.S. health care facilities to reduce these hospital-acquired infections (17). Combining the impact of this CMS program with surveillance data from the Centers for Disease Control and Prevention (CDC) demonstrating a rise in annual total CDI cases (Fig. 1) since 2013 (18), it is not surprising that hospitals are looking at novel interventions to lower CDI rates.

FIG 1.

Annual incidence (cases per 100,000 persons) of CDI based on CDC surveillance data (18).

There are several limitations to our investigation. The primary one is that, because of limited financial resources, we did not culture all samples, which limited the culture detection of toxigenic C. difficile to samples that were positive using the cobas as well as the next following patient with a negative stool sample. This affected calculation of an actual sensitivity and specificity of the test based on the entire population tested. However, it is possible to estimate the sensitivity, specificity, and negative predictive value (NPV) from our data. Since the cobas Cdiff Test assay positive rate was 7.6%, this indicates that the entire population was 1,803 patients. We also know that the total number of false-positive tests (before any discrepancy analysis) was 34. Since the test sensitivity in our sample was 100% (95% confidence interval [CI], 95.5 to 100%), then the greatest number of false-negative tests could be 5. Using these numbers, the estimate for sensitivity was 95.4 to 100% (95% CI, 89.0 to 100%), specificity was 98.0% (95% CI, 97.2 to 98.6%), and NPV was 99.7 to 100% (95% CI, 99.3 to 100%). These results are not different from those found for the cobas FDA-cleared test compared to direct culture (6), with a sensitivity of 97.3% (95% CI, 91.9 to 99.3%), specificity of 94.9% (95% CI, 92.7 to 96.5%), and NPV of 99.4% (95% CI, 98.3 to 99.9%). We were able to calculate the actual PPV and the positive percent agreement (PPA), which were comparable to the performance of this test on stool samples (6). The PPV on the FDA-cleared cobas Cdiff Test assay was 79.1% (95% CI, 71.3 to 85.4%) compared to our surveillance testing results of 75.2% (95% CI, 66.9% to 82%); and a comparison of the PPA values resulted in 97.3% (95% CI, 92.5% to 99.1%) versus 100% (95% CI, 96.0% to 100%). None of these differences were statistically significant (P > 0.1), and these are excellent results for a surveillance test where detecting likely carriers of a potential pathogen is the goal. If one develops a targeted admission surveillance program based on risk factors for C. difficile colonization, which we have done for our system-wide surveillance implementation, one will likely test 1/3 of admissions (our practice), which leads to one unnecessarily isolated patient per 200 admissions, which we consider a minimal number of false positives that is acceptable. Thus, we are confident that the test performed well on our surveillance swab samples. These results are consistent with the most recent CDI quality assessment review indicating that NAAT has the highest clinical diagnostic sensitivity and provides the greatest confidence for absence of disease from C. difficile (19). We also did not perform any culture enrichment to detect samples with low numbers of colonies. In place of this, we tested discrepant samples using a second NAAT (Xpert), and when the results were compared to medical record review, there was an excellent correlation between a positive second NAAT and patients receiving antimicrobial therapy that would be expected to inhibit the growth of C. difficile in anaerobic culture. The Xpert test would not be expected to detect low-colony-count positives that may be detected by enrichment culture that could increase direct culture-based sensitivity by at least 13% (20), but it was helpful in detecting patients receiving antibiotics that inhibit C. difficile growth; and we believe that the majority of true-positive tests were accurately identified.

Useful future research would be to validate additional NAAT assays that could be deployed in AST for toxigenic C. difficile. Should AST become a recommended tool to control the spread of CDI, it would be helpful for commercial vendors supplying these tests to perform needed studies that would permit an FDA claim for use of swab testing in facilities that do not have the capability to perform an in-house validation for this application.

In conclusion, we have demonstrated that the cobas is a reliable assay for detection of toxigenic C. difficile in perirectal samples from asymptomatic patients undergoing active surveillance testing as part of a hospital infection control program. We also demonstrated that the Xpert assay confirmed (P = 0.00006) the presence of C. difficile among patients who had received antibiotics that would have potentially interfered with organism viability in culture.