Ultrasensitive Clostridioides difficile Toxin Testing for Higher Diagnostic Accuracy

Currently available diagnostic tests for Clostridioides difficile infection (CDI) lack specificity or sensitivity, which has led to guideline recommendations for multistep testing algorithms. Ultrasensitive assays for detection of C. difficile toxins provide measurements of disease-specific markers at very low concentrations. These assays may show improved accuracy compared to that of current testing methods and offer a potential standalone solution for CDI diagnosis, although large studies of clinical performance and accuracy are lacking.

ABSTRACT

Currently available diagnostic tests for Clostridioides difficile infection (CDI) lack specificity or sensitivity, which has led to guideline recommendations for multistep testing algorithms. Ultrasensitive assays for detection of C. difficile toxins provide measurements of disease-specific markers at very low concentrations. These assays may show improved accuracy compared to that of current testing methods and offer a potential standalone solution for CDI diagnosis, although large studies of clinical performance and accuracy are lacking.

INTRODUCTION

Clostridioides difficile causes nosocomial and community-acquired gastroenteritis and is the most common pathogen responsible for health care-associated infections (1, 2). C. difficile infection (CDI), ranging in severity from mild to life-threatening diarrhea and colitis, is associated with high morbidity, mortality, and costs and so has been identified as a key challenge (3, 4).

Currently available diagnostic tools for CDI are not optimized in terms of accuracy and/or turnaround time, leading to recommendations for complex (typically algorithmic) testing solutions. Early data suggest that ultrasensitive C. difficile toxin assays could offer a new perspective on the laboratory diagnosis of CDI.

CDI AND THE DIAGNOSTIC LANDSCAPE

C. difficile is an anaerobic, Gram-positive bacillus that exists in a dormant spore form and in a vegetative form with toxin-producing capability (5). CDI is a toxin-mediated disease, and two exotoxins—the enterotoxin toxin A (TcdA) and the cytotoxin toxin B (TcdB)—cause diarrhea and inflammation by cytopathic and cytotoxic effects (6). The majority of toxigenic strains can produce both toxins, and strains predominantly producing either of the toxins have been shown to cause disease (7, 8). Nontoxigenic strains are not pathogenic, and individuals can carry toxigenic and toxin-producing C. difficile without having CDI (colonization) (6, 9).

Crucially, both C. difficile colonization and diarrhea, the cardinal symptom of CDI, are common in at-risk populations, and both outnumber CDI in most patient populations making it imperative to have a clinically specific test. The prevalence of spores in hospitals and long-term-care facilities is relatively high. While 2 to 3% of healthy adults in the general population are colonized with C. difficile, this rate can be up to 25% in hospitalized patients (10). Meanwhile, 12 to 32% of hospital inpatients have diarrhea, increasing to 80% in high-risk groups such as transplant patients (11). A recent large UK study found that on average 1 in 20 hospitalized patients develop diarrhea each day (12). In 85% of these patients with hospital-onset diarrhea, a median of three possible causes for their symptoms could be identified (12). Thus, the potential for noninfectious causes of diarrhea often appears to be underplayed. Taking these rates, alongside the fact that the great majority of fecal samples submitted for testing for CDI are found to be negative, it is clear that accurate clinical diagnosis of CDI is not possible without the appropriate use of laboratory diagnostics, and vice versa, laboratory diagnostics cannot be used accurately without first using appropriate clinical assessment. Unfortunately, however, the wide range of laboratory tests for C. difficile vary considerably, not only in terms of the targets used but also with regard to assay sensitivity and specificity (13–15).

The presence of toxins in a fecal sample better correlates with CDI severity and outcome of disease than does the presence of only toxin gene(s), i.e., toxigenic organisms with the capacity to produce toxins (16, 17). However, conventional toxin enzyme immunoassays (EIAs) have poor sensitivity and can miss CDI cases. In efforts to increase the sensitivity of toxin EIAs, testing with such assays is often performed in combination with glutamate dehydrogenase (GDH), a C. difficile-specific antigen that does not, however, differentiate between toxigenic and nontoxigenic C. difficile (13).

Nucleic acid amplification tests (NAATs) detecting toxin gene(s) were introduced a decade ago and, although associated with higher costs, offered a sensitive and rapid solution. NAATs detect toxigenic organisms but not the presence of free toxins and, therefore, cannot differentiate between colonization and disease and so have poor clinical specificity (9, 16, 17); that is, in the largest study of its type to date, the positive predictive value of an NAAT for CDI was 54% (17). Institutions have reported up to a 67% increase in reported CDI rates after adopting NAATs (18, 19). Use of NAATs, therefore, has considerable potential to result in overdiagnosis, and overtreatment, of CDI (16, 17), which could cause harm to patients and represent a large burden on health care providers.

The cell cytotoxicity neutralization assay (CCNA) and toxigenic culture (TC) are the reference methods for detection of free toxins and toxigenic organisms, respectively, but these are labor intensive, subjective, and have long turnaround times (13, 20). Regulatory studies require comparison of toxin assays with CCNA, an assay that is known to have issues with sensitivity and reproducibility (13, 20, 21).

Based on these diagnostic challenges, testing with multistep algorithms is currently recommended, with the goal of increasing diagnostic accuracy by combining clinically sensitive and specific methods, such as NAAT followed by a toxin EIA or a GDH/toxin EIA arbitrated by NAAT, although this is associated with a longer time to diagnosis (15, 22). An additional problem is that there is not universal agreement on a case definition of CDI. Infectious Diseases Society of America (IDSA) and Society for Healthcare Epidemiology of America (SHEA) guidelines define CDI by the presence of symptoms (usually diarrhea) and a stool test positive for either free toxins (toxin EIA or CCNA) or toxigenic C. difficile (NAAT or TC) (22), while the European Society of Clinical Microbiology and Infectious Diseases (ESCMID) guidelines do not agree on using NAATs alone and also require the exclusion of non-CDI-related causes of diarrhea for diagnosis (15, 23). At the heart of this clinical conundrum is the desire for accurate diagnosis of CDI, and yet the absence of a perfect solution based on currently available laboratory assays means that new options are needed.

ULTRASENSITIVE TOXIN TESTS

Since CDI is a toxin-mediated disease, there is a need for a more sensitive toxin assay that better correlates with disease without missing cases. Two ultrasensitive and rapid assays for TcdA and TcdB have recently been described/reported, both with limits of detection (LoDs) at approximately 1 pg/ml per toxin (24–26). Single-molecule array (Simoa) technology (Quanterix, Inc., Lexington, MA, USA) is based on capture and labeling of single-protein molecules on paramagnetic beads and their detection in arrays of femtoliter-sized wells (24). Single Molecule Counting technology (formerly Singulex, Inc., Alameda, CA, USA), performed on the Clarity system, utilizes a paramagnetic microparticle-based immunoassay that uses single-photon fluorescence detection for analyte measurement (25). A third technology detecting C. difficile toxins, MultiPath (First Light Diagnostics, Chelmsford, MA, USA), uses nonmagnified digital imaging to enumerate microscopic fluorescent particles bound to molecular targets (27). The analytical sensitivity of the MultiPath assay is significantly lower than that of the other two (TcdB LoD, 45 pg/ml; TcdA LoD, not published) (27).

None of these ultrasensitive toxin assays are currently commercially available, but efforts toward regulatory approval are ongoing as well as assay development using other ultrasensitive technologies. For comparison, the toxin assay with the lowest claimed LOD currently on the market is C. Diff Quik Chek Complete (TechLab, Inc., Blacksburg, VA, USA), which detects levels of TcdA at ≥630 pg/ml, TcdB at ≥160 pg/ml, and GDH at ≥800 pg/ml (28), although there is evidence that at least one assay (C. difficile Tox A/B II; TechLab, Inc.) is more sensitive for toxin detection (29). As a result, ultrasensitive toxin tests have been shown to have 27.0% to 39.4% higher analytical sensitivity than toxin EIAs when using CCNA as a reference method (21, 26, 30, 31). In a prospective, multicenter study on 2,000 patients samples, the Clarity assay had 96.3% positive agreement (PA) (“sensitivity”) and 93.0% negative agreement (NA) (“specificity”) with CCNA (although this was after discrepant analysis), while a toxin EIA (C. Diff Quik Chek Complete) had 59.8% PA (21). Simoa reported 88.0 to 84.8% sensitivity and 83.9 to 84.0% specificity for toxin A, 95.5 to 100% sensitivity and 83.3 to 87.0% specificity for toxin B, and 95.5% sensitivity and 79.3% specificity for both toxins combined when compared to CCNA (24, 26). MultiPath technology showed 97.0% sensitivity and 98.3% specificity for a TcdB assay when compared to CCNA (27), although this was in an unblinded training set.

Data suggest that ultrasensitive toxin assays could provide increased clinical specificity compared with that of NAAT and increased sensitivity compared with that of toxin EIAs (7–9, 21, 24–26, 30–37) and with overall higher accuracy than multistep algorithms (30, 33), and the studies have also revealed limitations with other methods, such as the risk of missing cases using assays that only detect toxin B or its gene, the poor reproducibility of CCNA, and the suboptimal performance of NAAT cycle thresholds for prediction of toxins (7, 8, 21, 37). In a study where results from a GDH/toxin EIA, NAAT, and an ultrasensitive toxin assay were compared to those for CCNA, sensitivity and specificity for an individual assay and an algorithm (combining a clinically sensitive and specific test) ending with the same method were identical (30). The sensitivity and specificity for NAAT and for an algorithm, where discordant GDH/toxin EIA results were arbitrated by NAAT, were 97.0% and 79.0%, respectively; for the toxin EIA and an algorithm where NAAT-positive samples were tested with toxin EIA, they were both 57.6% and 100%, respectively, when compared to CCNA. In a prospective multicenter study, samples were tested with the same assays as above, and algorithms did not improve accuracy over single-assay testing (21). Combining a sensitive and a specific test allows negatives to be screened out (using the first test, either NAAT or GDH) and can provide additional information over and above a single test result (for example, if the patient is a potential C. difficile carrier). However, combining tests means that the ultimate sensitivity of the algorithm is a product of the sensitivities of each test (which therefore is lower than the sensitivity of an individual test), and the second test (either NAAT or a toxin test) drives the detection performance of the full algorithm (21, 30).

Multiple comparisons between ultrasensitive toxin testing and standard-of-care algorithms have been performed. In a study from Stanford University on 311 samples, Clarity had 97.7% sensitivity and 100% specificity compared with that of an algorithm utilizing NAAT followed by toxin testing using EIA and CCNA (when EIA negative) (25). Over 1,000 samples were tested in a German study, and the ultrasensitive toxin assay showed improved accuracy compared to that of an algorithm utilizing a GDH/toxin EIA reflexed to NAAT (33). Depending on the comparison test algorithm result, the Clarity assay had high agreement in a study where 211 samples were tested with GDH/toxin EIA and reflexed to a semiquantitative CCNA (which is more sensitive than conventional CCNA) (38). In a study from Mayo Clinic on nearly 500 patients, an ultrasensitive toxin assay had 91.0% sensitivity and 99.1% specificity compared with that of an algorithm where a GDH/toxin EIA reflexed to NAAT (no laboratory reference method utilized) (34). In a UK study, Clarity had high PA with a toxin EIA (96.9%) and PA with multistep algorithms ending with toxin EIA (95.8 to 100%) and high NA with NAAT (89.9%) and a multistep algorithm ending with NAAT (91.7%); the low NA and PA compared with toxin EIA (49.6%) and NAAT (69.4%), respectively, reflected the poor sensitivity of current toxin EIAs and low specificity of NAAT (35). These data suggest that ultrasensitive toxin assays could offer an alternative to conventional testing, including toxin-based algorithms, but further larger studies, in particular with outcome data, are needed to fully understand the clinical meaning of discordant results.

CORRELATION WITH DISEASE: WHAT SPECIFICITY CAN BE ACHIEVED?

Host-response factors play an important role in development of CDI, and asymptomatic individuals can have toxins present in stool, something that has been described previously (39–42). This has also been observed when using an ultrasensitive toxin assay (Simoa), where the presence of toxin or toxin gene could not differentiate an individual with CDI from one with asymptomatic carriage, both as determined by NAAT (9). Toxin concentrations, however, were higher in CDI patients than in carriers but only when CDI was diagnosed by toxin detection (cutoff, 20 pg/ml), which made the authors conclude that toxin detection is more clinically relevant than detection of the toxin gene (9). Indeed, defining CDI/asymptomatic carriage on NAAT may have added confusion to this study.

It is well established that CDI is a clinical diagnosis, i.e., that no test can be used to rule in disease. Instead of implying that a highly imperfect test (NAAT) can only be replaced by a perfect test, the relevant questions to focus on are the following. How much could ultrasensitive toxin assays improve the diagnostic accuracy and clinical specificity compared with NAATs, and what positive predictive value (PPV) is achievable? In a recent U.S. study, nearly 300 patients were tested with NAAT and the Clarity assay, with discordant samples tested with CCNA and results correlated with disease severity and outcome (32). Among the NAAT-positive (NAAT⁺)/Clarity-negative (Clarity⁻) patients, nearly 70% had a non-CDI-related cause of diarrhea compared with less than 22% of NAAT⁺/Clarity⁺ patients—a 3-fold difference. If using one of the guideline CDI case definitions (23), the ultrasensitive toxin test thereby achieved 97.4% clinical specificity and 78.1% PPV while NAAT had 89.0% clinical specificity and 54.7% PPV (32), although larger studies are needed. The obvious counterargument is that using a CDI case definition that does not include an assessment of non-CDI-related causes of diarrhea would increase the specificities. However, the NAAT overdiagnosis rate and a PPV of just over 50%—a statistician’s term for “a flip of a coin”—may be unacceptable to many clinicians when interpreting laboratory tests. Presence of toxins also correlated with outcome; CDI relapse only occurred in Clarity⁺ patients (12.5% of Clarity⁺ patients), and NAAT⁺/Clarity⁺ patients had longer lengths of stay compared to those of NAAT⁺/Clarity⁻ patients (14.2 versus 7.6 days), although this was not statistically different.

Testing criteria for CDI, i.e., who and when to test, are subject to ongoing discussions. IDSA/SHEA agree on using NAAT alone if testing excludes stool specimens from patients receiving laxatives and with less than three unformed stools in 24 h (22). However, in the study evaluating clinical specificity (32), the hospital had previously successfully implemented stringent stool-submission criteria adherent with the IDSA/SHEA guidelines (43) but still observed that two-thirds of NAAT⁺/toxin-negative (toxin⁻) patients had a non-CDI-related cause of diarrhea, indicating that suggested guideline criteria on whom to test are ineffective. Conversely, lack of clinical suspicion for testing, i.e., no test requested, can lead to underdiagnosis of CDI (44). Given that both symptoms and colonization are common, the strategy of limiting testing to those patients with higher disease probability and thereby achieving an increase of the NAAT PPV to acceptable levels seems likely to be problematic and unsuccessful. A consistent finding is that ∼30% to 50% of NAAT⁺ patients have no detectable toxin in stool as measured by ultrasensitive assays (21, 25, 30, 32, 35), indicating that these toxigenic bacteria are not producing toxins, although some might argue that ultrasensitive toxin assays are not sensitive enough. CDI is a toxin-mediated disease, and the lack of toxin in NAAT⁺ samples, therefore, has important implications for diagnostic accuracy and clinical specificity. Diagnosis by NAAT may ultimately lead to use of unnecessary antibiotics and infection-control measures. Although CDI is a clinical diagnosis, physicians often base treatment decisions on laboratory reports, demonstrated by studies where all asymptomatically colonized patients (defined as individuals without clinically significant diarrhea and with positive NAAT) (42, 45) and 95% of patients tested inappropriately (43) were treated when NAAT was used. In addition, there are multiple examples of clinical trials that have failed to meet their endpoints when CDI diagnosis was based on NAAT (46), indicating that NAAT does not accurately define disease.

As ultrasensitive, quantitative toxin assays become available, there is an interest in correlating toxin concentration with disease to improve severity assessment and guide treatment. Toxins in patients with suspected CDI are detected in a wide range, up to 300 ng/ml (21, 24, 25). Higher toxin concentrations have been reported in PCR ribotype 027 than in non-027 strains (25), but there was no difference in toxin concentration between multiple non-027 ribotype strains (35). Although a correlation between toxin concentrations and CDI severity has been observed (47) and high concentrations have been reported in individual patients with severe disease and ileus (32, 36), the lack of such a correlation has also been reported (9, 24, 41, 42). Factors related to host response are important in disease progression (39, 40), and larger observational and interventional studies are needed to understand the role of toxin concentration and disease. A fundamental issue here, however, is that the fluid content/volume of stool/diarrhea is variable in an individual, and so the concentration of toxin measured at any particular time point could be markedly affected. While assay manufacturers need to consider the clinical utility of providing quantitative versus qualitative reporting when developing ultrasensitive toxin assays, at this point, a quantitative readout has not been shown to provide additional value.

RULING OUT DISEASE: WHAT SENSITIVITY IS NEEDED?

The ultrasensitive assays allow for quantification of C. difficile toxins, and assay developers are challenged with optimizing analytical sensitivity and threshold for a qualitative readout. Simoa and Single Molecule Counting technology utilize cutoffs (evaluated compared to CCNA or assay combinations including CCNA) between 12.0 pg/ml of the toxins combined to 29.4 pg/ml per toxin (9, 21, 24–26). In a multicenter study on 2,000 samples, 33.1% (108/326) of Clarity⁺ samples were under 45 pg/ml and 17.8% (58/326) were under 20 pg/ml (21), indicating that an LoD (27) or cutoff (9, 26) in that range may be too high and will lead to missed cases.

CCNA has an estimated LoD of 50 to 100 pg/ml (48), which is significantly higher than the ultrasensitive assays’ detection limits at 1 pg/ml. Indeed, the ultrasensitive assays detected toxins in 22.7% of NAAT⁺/toxin EIA⁻/CCNA⁻ (25) and in 41.3% of GDH⁺/CCNA⁻/NAAT⁺ (26) samples, indicating that Simoa and Single Molecule Counting technologies are more sensitive than CCNA and that their specificity, therefore, may be underestimated in direct comparisons (21, 26). For less sensitive assays, such as the MultiPath assay that has an LoD in a similar range as that of CCNA, an accuracy comparison will look more favorable (27). In the prospective, multicenter study comparing Clarity with CCNA, samples with discrepant results were retested with CCNA when the ultrasensitive toxin result agreed with that of at least one other comparator method (GDH EIA, toxin EIA, or NAAT), and a different CCNA result was reported for as many as 42% of retested samples (21); CCNAs (in which results are read by microscopy) require experienced workers to optimize reproducibility. Lastly, CCNA is impacted by toxin stability and subjectivity (13), something that has not been observed using automated ultrasensitive toxin detection (25, 26). The issues with CCNA reproducibility and sensitivity need to be considered when evaluating new toxin tests. A comparison with TC would not solve this problem, as this method detects toxigenic organisms only, similar to NAAT, and does not provide any information on toxin production in vivo.

When establishing an optimized cutoff compared with CCNA, there is a risk of overestimating an assay’s clinical sensitivity, i.e., wanting to set the cutoff too high, although avoiding setting the cutoff too low to avoid false negatives is also critical to avoid background signal and maintain specificity. If toxin concentration does not correlate with disease severity (9, 24, 42), there might be value in solely providing clinicians with information on toxin detection on the lowest, reproducible level. In cases where NAAT⁺/toxin⁻ patients were deemed to have CDI when retrospectively reviewed by a clinical panel, toxin was present but under the assay’s cutoff (36), although it was not investigated further whether this was background signal. Developers of ultrasensitive toxin assays may want to take this into account when optimizing analytical sensitivity.

FUTURE OF CDI DIAGNOSTICS: BACK TO STANDALONE TOXIN DETECTION?

Ultrasensitive toxin assays may improve the detection of CDI compared to that of current testing methods. However, further studies are required so that recommendations can be formulated on how best ultrasensitive assays, as they become commercially available, can be utilized in clinical practice. In the United States, the Centers for Disease Control and Prevention (CDC) and the National Health Safety Network (NHSN) adjust the C. difficile laboratory-identified event (LabID-CDI event) standardized infection ratio (SIR)—the primary measure used to track health care-associated infections—based on the test used at the facility (NAAT, toxin EIA, or other), and, for multistep algorithms, on the last test that is placed in the patient medical record (49). High SIRs place financial and reputational burdens on health care providers, and the recognition that testing methods impact incidence is important. There are concerns that the CDI SIR risk-adjustment formula used by the CDC and NHSN to take account of diagnostic method may not be sufficient to account for the effects of those testing methods on reported CDI rates. An unintended consequence here could be that hospitals decide on a testing method(s) based on a desire to achieve lower LabID-CDI event rates and SIRs (50, 51). If tests with enhanced sensitivity to detect CDI are to be adopted, a way to overcome the effects of increased reporting will be needed.

Ultrasensitive C. difficile toxin assays provide detection of disease-mediating toxins at very low concentrations. Better CDI diagnostics with higher PPVs could improve antibiotic stewardship efforts and have the potential to make infection-control practices more efficient. It is noteworthy that CDI diagnosis has had several major shifts in the 4 decades since C. difficile was first described as a human pathogen. Toxin detection by immunoassays supplanted culture and/or cytotoxin detection-based methods, and these were followed by a rapid uptake of NAATs in some countries. Implementation of standalone ultrasensitive toxin testing could offer a new way forward in CDI diagnostics.