Multicenter Clinical Evaluation of the Portrait Toxigenic C. difficile Assay for Detection of Toxigenic Clostridium difficile Strains in Clinical Stool Specimens

Abstract

We compared the Portrait Toxigenic C. difficile Assay, a new semiautomated sample-to-result molecular test, to a toxigenic bacterial culture/cell cytotoxin neutralization assay (TBC/CCNA) for the detection of toxigenic Clostridium difficile in 549 stool specimens. Stool specimens were also tested by one of three alternative FDA-cleared molecular tests for toxigenic C. difficile (Xpert C. difficile, Illumigene C. difficile, or GeneOhm Cdiff). The sensitivities and specificities of the molecular tests compared to TBC/CCNA were as follows: 98.2% and 92.8% for the Portrait assay, 100% and 91.7% for the Xpert assay, 93.3% and 95.1% for the Illumigene assay, and 97.4% and 98.5% for the GeneOhm assay, respectively. The majority of Portrait false-positive results (20/31; 64.5%) were also positive for C. difficile by an alternative molecular test, suggesting an increased sensitivity compared to the culture-based “gold standard” method. The Portrait test detected an assay input of 30 CFU in 100% of spiked samples and detected an input of 10 CFU in 96.7% of samples tested.

INTRODUCTION

Clostridium difficile has emerged as a major nosocomial pathogen and is a leading cause of antibiotic-associated diarrhea and pseudomembranous colitis (6). Two toxins, tcdA and tcdB, have been implicated in C. difficile-associated disease; however, the cytotoxin tcdB appears to be primarily responsible for symptoms (10), since strains lacking a functional tcdA gene retain toxigenic potential (8, 12). Toxigenic culture is considered the current gold standard for the identification of toxigenic C. difficile and is based upon the ability of an isolated strain to induce cytopathic effects on cultured cells in the absence of TcdB-neutralizing antibody (3, 11). Unfortunately, this method requires technical expertise and can take 3 to 5 days to return a result. This extended turnaround time (TAT) often results in patients being started on antibiotic therapy for C. difficile infection based upon clinical suspicion rather than a definitive diagnosis.

Rapid antigen tests for the C. difficile “common antigen” glutamate dehydrogenase (GDH) have been used to screen for the presence of C. difficile in stool specimens; however, they lack sensitivity (90% to 93%) when used alone (4, 13). The Society for Healthcare Epidemiology of America (SHEA) and the Infectious Diseases Society of America (IDSA) recommend a multistep approach that includes an initial screen for GDH followed by the detection of toxin using a cell cytotoxin neutralization assay (CCNA) or an enzyme immunoassay (EIA) (3). EIAs directed against C. difficile toxins have reported sensitivities of 25% to 86% (1, 5, 8, 14). Similarly, the reported sensitivities of toxin-based lateral flow tests range from 43% to 73% (3, 5, 14). Given this low sensitivity, it was suggested that toxin EIAs and lateral flow tests lack utility as confirmatory methods for GDH-positive specimens in two-step testing algorithms (5). The cell cytotoxin neutralization assay is another method used for the confirmation of toxigenic C. difficile in GDH-positive specimens; however, when conducted without bacterial enrichment (i.e., directly from stool), the CCNA may be only 54% sensitive (11). Therefore, bacterial culture should be done prior to the CCNA. This slows the reporting of results by 48 to 72 h, which reduces the clinical utility of results and postpones the appropriate management of the patient. Other culture-based detection methods, including chromogenic medium specific for C. difficile, have been developed but also lack sensitivity (1). It is important to note that the SHEA/IDSA recommendations were reported prior to the widespread availability and evaluation of molecular tests and are currently under review.

Molecular methods for the detection and identification of C. difficile are an attractive choice because testing can be conducted directly using stool specimens, and results are available in as little as 45 min. This allows the treating physicians to wait for results prior to initiating targeted antibiotic therapy for C. difficile infection. A number of FDA-cleared nucleic acid tests are currently available for the identification of toxigenic C. difficile in stool: Illumigene C. difficile (Meridian Bioscience, Cincinnati, OH), Xpert C. difficile (Cepheid, Sunnyvale, CA), GeneOhm Cdiff (BD, Sparks, MD), and ProGastro Cd (Gen-Probe, San Diego, CA). All these tests rely on the ability to detect nucleic acid sequences of the tcdA or tcdB gene as a marker of toxigenic strains. The sensitivity (88.5% to 100%) and specificity (97.5% to 100%) of these tests are superior to those of enzyme-linked assays and other direct detection methods (2, 10, 14, 16).

We conducted a multicenter evaluation of the recently FDA-cleared Portrait Toxigenic C. difficile Assay (Great Basin, West Valley City, UT) for the detection of toxigenic Clostridium difficile in clinical stool specimens. This test utilizes the isothermal helicase-dependent amplification of a 78-nucleotide 3′ region of the tcdB gene followed by detection using an immobilized capture probe on a slide array (5). Results were compared to toxigenic bacterial culture (TBC)/CCNA as a gold standard. The Portrait results were also compared to the results of one of three FDA-cleared alternative molecular tests in use at each clinical test site.

MATERIALS AND METHODS

Collection of stool specimens.

Fresh stool specimens (n = 549) from patients suspected of having C. difficile-associated diarrhea were collected and tested at four different study sites, in accordance with site-specific institutional review board (IRB)-approved protocols. Inclusion criteria for this study dictated that testing be performed on fresh (not preserved) specimens obtained from individuals older than 2 years of age who displayed clinical symptoms of C. difficile disease, including persistent soft or liquid stools, abdominal cramping, or colitis. Only one specimen per patient was tested, to avoid duplicate results. Each stool specimen was tested by TBC/CCNA, Portrait, and the standard molecular method used by each respective clinical site. The molecular methods used included the Xpert C. difficile test (2 sites), the Illumigene C. difficile test (1 site), and the GeneOhm Cdiff test (1 site).

Portrait Toxigenic C. difficile Assay.

Stool specimens were tested by using the Portrait Toxigenic C. difficile Assay within 96 h of collection if the sample was maintained at 2°C to 8°C or within 2 h if it was maintained at room temperature (15°C to 30°C), in accordance with the manufacturer's recommendations. Testing was conducted according to the manufacturer's product insert (PI). Briefly, a sterile swab (provided) was used to transfer the stool specimen into a filter syringe device preloaded with extraction buffer (provided). The sample in extraction buffer was agitated by vortexing for 30 s and then passed through a filter (provided) into a sterile 2-ml sample preparation tube to remove particulate matter or mucous in the sample. Following filtration, 180 μl of the filtrate was injected into the sample port on the assay test cartridge by using a standard pipette. Loaded cartridges were then inserted into the Portrait Dx Analyzer instrument for automated thermophilic helicase-dependent (tHDA) amplification and the chip-based detection of a 78-nucleotide sequence of the tcdB gene, which is conserved in 21 C. difficile sequences, including 10 representative ribotype 027 strains, in GenBank (5). Each reaction mixture contained a sample processing control (formalin-fixed Staphylococcus aureus cells in the sample preparation tube), a hybridization control (immobilized oligonucleotide probe complementary to the biotinylated oligonucleotide present in the hybridization buffer), and a detection control (biotinylated probe immobilized onto the chip). If detection criteria for all controls are met, results for the specimen are reported as positive or negative based on the presence or absence of tcdB. The total assay run time was 90 min. Tests with specimens yielding invalid results were repeated once with the original stool specimen.

TBC/CCNA.

The toxigenic culture method used as the gold standard in this study was adapted from methods described previously by Ticehurst et al. (15). Stool specimens were heated for 10 min at 85°C to enrich for spores. Following heat enrichment, each specimen was inoculated into chopped meat glucose (CMG) broth and incubated at 35°C for 48 h prior to plating onto cefoxitin-cycloserine fructose agar with horse blood (CCFA-HB). Inoculated CCFA-HB medium was incubated anaerobically at 35°C and was examined for the presence of colonies with the typical C. difficile morphology. CCFA-HB cultures were held for up to 48 h at 35°C before being called negative. Colonies were analyzed for phenotypic and biochemical properties that would confirm the identification of C. difficile, including a Gram stain yielding a Gram-positive rod, a positive l-proline disk test, a susceptible vancomycin disk test result, and lack of growth on chocolate agar incubated with 5% CO₂. Isolates identified as C. difficile were then inoculated into CMG broth and analyzed for toxin production by using the TechLab C. difficile Tox-B cell cytotoxicity neutralization assay (TechLab, Blacksburg, VA).

Statistical analysis.

Results from the Portrait Toxigenic C. difficile Assay were compared to the results of the TBC/CCNA method as a gold standard. Performance characteristics, including sensitivity and specificity, were calculated by using standard methods. Ninety-five-percent confidence intervals were calculated by using a binomial expansion formula.

RESULTS

Comparison of the Portrait Toxigenic C. difficile Assay to TBC/CCNA.

A total of 549 stool specimens were initially included and tested by using the Portrait Toxigenic C. difficile Assay. Nine of these specimens did not meet study inclusion criteria and were not included in the analysis of the results. These 9 samples included 3 stool specimens (0.5% of the total samples tested) that generated an invalid result by the Portrait assay (0.5%), 1 specimen from an individual <2 years of age, 1 specimen that was not tested by TBC/CCNA, and 4 specimens that were tested outside the acceptable testing time frame defined by the clinical protocol. Compared to TBC/CCNA, the sensitivities of the Portrait assay at four clinical trial sites ranged from 92.9% (13/14) to 100% (44/44), with an overall sensitivity of 98.2% (109/111). The specificities at the four sites ranged from 88.9% (72/81) to 96.9% (127/131), with an overall specificity of 92.8% (398/429) (Table 1). This study included 101 specimens obtained from patients aged 2 to 17 years, a group for which the performance of molecular tests for C. difficile has not been specifically addressed. In this group, the sensitivity (100%; 14/14) and specificity (90.8%; 79/87) of the Portrait test were similar to those of the entire study population.

TABLE 1.

Performance of the Portrait Toxigenic C. difficile Assay compared to TBC/CCNA^a

Clinical test site	No. of specimens at site	No. of specimens with result				% sensitivity (CI)	% specificity (CI)
		TP	FP	TN	FN
1	169	37	4	127	1	97.4 (86–99)	96.9 (92–99)
2	219	44	16	159	0	100 (91–100)	90.9 (85–94)
3	96	15	9	72	0	100 (78–100)	88.9 (79–94)
4	56	13	2	40	1	92.9 (66–99)	95.2 (83–99)
Total	540	109	31	398	2	98.2 (93–99)	92.8 (89–95)

^aTP, true positive; FP, false positive; TN, true negative; FN, false negative; CI, 95% confidence interval.

Both specimens that tested negative by Portrait but positive by TBC/CCNA (i.e., false negative) were reported to be positive by an alternative molecular test (Xpert C. difficile or GeneOhm Cdiff). Of 31 specimens that tested positive by Portrait but negative by TBC/CCNA (i.e., false positive), 20 (64.5%) also tested positive by using an alternative molecular method (15 specimens for Xpert C. difficile, 1 for GeneOhm Cdiff, and 4 Illumigene C. difficile). This finding supports the presence of C. difficile nucleic acid in these specimens and increases the specificity of the Portrait assay to 97.3% for the detection of C. difficile nucleic acid in specimens.

Comparison of the Portrait Toxigenic C. difficile Assay to three alternative molecular tests for detection of toxigenic C. difficile.

Stool specimens tested by the Portrait assay were also tested by one of three FDA-cleared molecular assays for the detection of toxigenic C. difficile (Illumigene C. difficile, Xpert C. difficile, or GeneOhm Cdiff). Compared to TBC/CCNA, the sensitivities of the three comparator tests ranged from 93.3% (Illumigene C. difficile) to 100% (Xpert C. difficile), and the specificities ranged from 91.7% (Xpert C. difficile) to 98.5% (GeneOhm Cdiff) (Table 2). These values are similar to values reported previously (2, 10, 14, 16). The Portrait tests demonstrated a sensitivity of 98.2% and a specificity of 92.8%. The 95% confidence intervals for both sensitivity and specificity overlapped for all four tests.

TABLE 2.

Comparison of four molecular tests to TBC/CCNA for detection of toxigenic C. difficile^a

Test	No. of specimens at site	No. of specimens with result				% sensitivity (CI)	% specificity (CI)
		TP	FP	TN	FN
Portrait	540	109	31	398	2	98.2 (93–99)	92.8 (89–95)
Gene Xpert	275	58	18	199	0	100 (93–100)	91.7 (87–95)
GeneOhm	169	37	2	129	1	97.4 (86–99)	98.5 (94–99)
Illumigene	96	14	4	77	1	93.3 (68–99)	95.1 (87–98)

^aTP, true positive; FP, false positive; TN, true negative; FN, false negative; CI, 95% confidence interval.

The percent agreements between the Portrait results and each the three alternative methods were as follows: 97.5% (n = 275) with Xpert C. difficile, 96.4% (n = 169) with GeneOhm Cdiff, and 93.8% (n = 96) with Illumigene C. difficile (Table 3). Thirteen of nineteen (68.4%) total discrepancies were the result of specimens that tested positive by Portrait but negative by the comparator method. Eleven of these specimens (84.6%) were also negative by TBC/CCNA. Six discrepancies were the result of specimens that tested negative by Portrait but positive by the alternative method. Four of these specimens (66.7%) were confirmed to be negative by TBC/CCNA.

TABLE 3.

Agreement of the Portrait Toxigenic C. difficile Assay with other molecular C. difficile tests

Test	No. of specimens tested	No. of specimens with resulta				Agreement (%)
		P+/M+	P+/M−	P−/M+	P−/M−
Gene Xpert	275	72	3b	4c	196	97.5
GeneOHM	169	37	4d	2e	126	96.4
Illumigene	96	18	6f	0	72	93.8
Total	540	127	13	6	394	96.5

^a“P” is the Portrait result, and “M” is the comparator molecular method result. “+” is positive for C. difficile, and “−” is negative for C. difficile.

^bAll 3 specimens were culture negative.

^cThree specimens were culture negative, and one was culture positive.

^dThree specimens were culture negative, and one was culture positive.

^eOne specimen was culture negative, and one was culture positive.

^fFive specimens were culture negative, and one was culture positive.

Call rate and reproducibility of the Portrait Toxigenic C. difficile Assay.

The Portrait test had a first-run call rate of 95.4% (524/549); however, a large proportion (11/25) of invalid results were from a single study site. Upon retesting, only 3 specimens failed to produce a valid result, raising the final call rate to 99.5%. The 3 specimens giving repeat invalid results included 2 TBC/CCNA-negative specimens and 1 TBC/CCNA-positive specimen. The total test time was approximately 1.75 h, with only 15 min of hands-on time per test. The batching of samples increased the hands-on time required for each additional batched sample; however, if processed in parallel, the hands-on time added per batched sample was <2 min.

A reproducibility study was conducted across three test sites. Stool specimens spiked with a toxigenic C. difficile strain (ATCC 43255) at three different densities were tested in replicates by two operators at each test site on five different days for a total of 270 replicates (30 for each density per site). The “moderate-positive”-density samples contained an assay input of approximately 30 CFU, or 3 times the established limit of detection (LoD) (5). All 30 replicates tested positive by Portrait at two test sites, and 28/30 (93.3%) tested positive at the third site, for an overall positivity rate of 97.8% (Table 4). Repeat testing of the moderate-positive panel at the third test site resulted in 30/30 samples testing positive. The “low-positive”-density (C₉₅ [cell density at which 95% of tests are expected to give a positive result]) samples contained an assay input of approximately 10 CFU, which is equivalent to the assay LoD. At this density, it is expected that approximately 5% of the samples will be called as negative. Indeed, 87/90 (96.7%) “low-positive” samples tested positive by the Portrait test, which is compatible with the expected results. The testing of the “high-negative”-density (C₅ [cell density at which 5% of tests are expected to give a positive result]) samples resulted in 4/90 (4.4%) positive tests, which is consistent with the expected 5% positive samples.

TABLE 4.

Reproducibility of the Portrait Toxigenic C. difficile Assay^e

Study site	% positive Portrait results (no. of specimens with result by Portrait/total no. of specimens)
	Moderate positivea	Low positiveb	High negativec
1	100 (30/30)	100 (30/30)	10.0 (3/30)
2	100 (30/30)	100 (30/30)	3.3 (1/30)
3	93.3 (28/30)d	90.0 (27/30)d	0.0 (0/30)
Total	97.8 (88/90)	96.7 (87/90)	4.4 (4/90)

^aSamples contained a CFU input equal to approximately three times the limit of detection.

^bC₉₅ density. The samples contained a CFU input equal to the assay limit of detection. The positivity rate was expected to be ∼95.0%.

^cC₅ concentration. The samples contained a CFU input approximately 10-fold lower than the assay limit of detection. The positivity rate was expected to be 5.0%.

^dRepeat testing of this reproducibility panel at this site demonstrated 30/30 (100%) positive results.

^eA C. difficile toxinotype O, tcdA- and tcdB-positive strain (ATCC 43255) was used for this study.

DISCUSSION

Timely laboratory results of C. difficile testing can impact decisions regarding antibiotic therapy and infection control measures. This necessitates accurate and rapid methods for the detection of toxigenic C. difficile in symptomatic patients. Molecular and nucleic acid-based tests are increasingly becoming the standard of care for the detection of a wide range of pathogens from various clinical specimens. Characteristics including high sensitivity and specificity compared to those of culture methods, a high throughput, and a rapid turnaround time (TAT) are compelling arguments for the incorporation of molecular testing in the clinical laboratory. Counterbalancing these advantages are considerations including a higher cost per test and the requirement for skilled molecular technologists to conduct testing, which can sometimes include a high-complexity assay setup or instrumentation.

A number of recent studies evaluated the performances of FDA-cleared, commercially available molecular tests, including BD GeneOhm Cdiff, Xpert C. difficile, Illumigene C. difficile, and ProGastro Cd (2, 10, 14, 16). All these test have demonstrated the superior sensitivity of molecular tests compared to alternative methodologies. Compared to the toxigenic culture method, the sensitivities of specific tests were 96.2% to 100% (Xpert), 94.4% to 96.2% (GeneOhm), 83.0% to 86.7% (Illumigene), and 88.5% (ProGastro). The specificity of each test was >97.5%. In addition to the high-sensitivity and high-specificity characteristics, these tests can be completed in 1 to 3 h, with minimal hands-on time.

This multicenter clinical evaluation of the Portrait Toxigenic C. difficile Assay included 543 stool specimens that were compliant with all established inclusion criteria. The prevalence of toxigenic C. difficile was 20.6% (range, 15.6% to 25.0%), as determined by TBC/CCNA. The sensitivity of the Portrait test (98.2%) was higher than those of 2 of 3 of the other evaluated molecular tests. Only the Xpert test had a higher sensitivity (100%); however, the specificity of the Portrait test was higher than that of the Xpert test. Using 95% confidence intervals, these performance characteristics were not statistically different. A significant challenge when using molecular tests is the interpretation of results for specimens that are positive by molecular methods but negative by culture methods, i.e., false-positive results. There were 31/540 (5.7%) such results with the Portrait test. False-positive results can be due to a very low level of C. difficile in the specimen that is not recovered in culture, the presence of nonviable bacteria or residual DNA in the specimen, or external DNA amplicon contamination from prior testing. Twenty (65%) of the Portrait false-positive results were also reported to be positive by a comparator molecular test targeting a different region of the C. difficile chromosome. This finding favors the increased sensitivity of the molecular tests rather than amplicon contamination. Because of the increased sensitivity of Portrait and other molecular tests versus culture techniques, it is critical to establish strict guidelines for the testing of stool specimens for C. difficile. The testing of patients who are not symptomatic or lack risk factors (hospitalization or antibiotic therapy) is more likely to generate a false-positive result by molecular tests than are toxin-based EIAs (3, 7, 11).

A number of other challenges arise when tests are used to detect analytes directly from stool specimens. These challenges include physical characteristics, such as the presence of mucous or particulate matter in a specimen; variable viscosities, ranging from liquid to semisolid; and the presence of blood or other inhibitory substances in this matrix. Depending on the test, these properties can cause the clogging of microfluidic channels or interfere with the PCR amplification of the target sequence(s). The result is indeterminate or unresolved test results that need to be repeated. These barriers can be overcome through the use of mucolytic or other lytic agents, heat, physical homogenization, filtration steps, or the simple dilution of the specimen. The Portrait test relies on the dilution of the specimen in phosphate buffer, filtration, and heat inactivation steps prior to the initiation of the molecular test (5). Using this processing method, only 3/543 (0.5%) samples tested by Portrait failed to produce a definitive result. This indeterminate or unresolved rate is lower than that observed for other comparable tests, such as the GeneOhm Cdiff (6.4%) and Xpert C. difficile (0.9% to 1.7%) tests (9, 16), which can reduce costs associated with repeat testing.

Two strengths of this study include (i) the high number of prospectively included specimens (n = 543), which were collected and tested at four different clinical laboratories, and (ii) the testing of all specimens by TBC/CCNA. Despite the differences in geographic locations and study personnel, the sensitivities (92.9% to 100%) and specificities (88.9% to 96.9%) of the Portrait test were similar across study sites. This result indicates that the Portrait test generates reliable results independent of external variables. Additionally, a multisite reproducibility study demonstrated the reliable detection of the C₉₅ (∼10 CFU) in spiked stool samples. In contrast to similar studies that compared molecular tests, in this study, all stool samples were tested by TBC/CCNA. This approach allowed the identification of all false-positive results, thereby generating a true measure of the test specificity, which is a key characteristic of molecular tests for toxigenic C. difficile.

One weakness of this study was that each test site performed the CCNA reference method on its own stool samples, rather than having a single reference site perform the CCNA on all specimens. While all sites followed the same protocol, differences in familiarity with the technique among sites could slightly affect the recovery rate and thus lead to differences in the sensitivities and specificities of the comparator nucleic acid tests. This did not appear to be a major concern, since the 95% confidence intervals of the sensitivities and specificities for the Portrait test overlapped for all test sites.

This large-scale evaluation of the Portrait Toxigenic C. difficile Assay and comparison to alternative molecular tests further highlight the superb performance and rapid TAT of molecular tests for the detection of C. difficile in stool specimens obtained from symptomatic patients.