Site-Specific Clinical Evaluation of the Luminex xTAG Gastrointestinal Pathogen Panel for Detection of Infectious Gastroenteritis in Fecal Specimens

Abstract

We evaluate the clinical performance of the Luminex xTAG gastrointestinal (GI) pathogen in vitro diagnostic (IVD) assay in a comparison between clinical and public health laboratories. The site reproducibility study showed 98.7% sensitivity with high positive and negative agreement values (96.2% and 99.8%, respectively), while assay performance against confirmatory methods resulted in 96.4% sensitivity with similar positive and negative agreement values (90.1% and 99.5%, respectively). High-throughput detection of multiple GI pathogens improved turnaround time, consolidated laboratory workflow, and simplified stool culture practices, thus reducing the overall cost and number of specimens processed.

TEXT

Gastrointestinal (GI) infections remain a major health problem for morbidity and mortality worldwide (1, 2). Rapid and accurate laboratory diagnosis is critical to avoid further spread of disease due to the complexity of gastroenteritis cases. It also aids in judicious antibiotic therapy use while minimizing costs from presumptive isolation and hospital stay (3–6).

The different conventional microbiological and molecular testing methods are typically performed in clinical laboratories to identify gastrointestinal pathogens. This practice not only limits diagnostic efficiencies due to the failure to test for particular agents, the lack of sensitivity/specificity, the inability to culture organisms, and the longer turnaround time (TAT) but also impacts guidance on clinical decisions (2, 7, 8). Therefore, for timely identification of the pathogens that may cause nosocomial outbreaks, a rapid, multiplex frontline screening assay is highly desirable to improve the current practices of infection control (9–11).

This study reports the clinical evaluation of the xTAG gastrointestinal pathogen panel (GPP) in vitro diagnostic (IVD) assay and compares the assay performances at two different study sites. This parallel evaluation study is the first to assess the feasibility, clinical utility, and acceptability of the xTAG GPP IVD assay for improved sample-to-result turnaround time, treatment measures, and outbreak response using a highly sensitive and specific method with a high degree of reproducibility.

To assess clinical assay reproducibility between testing sites, the Milwaukee Health Department Laboratory (MHDL) and the Le Bonheur Children's Hospital laboratory performed side-by-side GPP analysis of the 167 stool specimens collected from pediatric and adult patients with symptoms of gastroenteritis during February to June 2013. Stool specimens were transported to MHDL and the Le Bonheur laboratory as frozen raw stool in sterile 1.5-ml microtubes according to Institutional Bio-Safety Committee (IBC) guidelines. Each site processed the specimens according to the xTAG GPP IVD package insert guidelines. Analysis of clinical specimen nucleic acid was performed using a Luminex 200 instrument (Luminex Corporation, Austin, TX) with Luminex xPONENT software (version 3.1), while data were exported to xTAG TDAS GPP (version 1.2) software for analysis. Identification of targets was based on elevated relative fluorescence above a predetermined threshold. Positive detection of pathogen targets was confirmed using gold standard methods appropriate for the pathogen detection (Table 1).

TABLE 1.

Summary of reference or gold standard methods

Target	Reference method available at MHDLb	Reference(s)a
E. coli	Culture: MAC with sorbitol; MicroSEQ (16S sequencing)	17, 18
E. coli O157:H7	API20E and ECO157 (slide) + H7 (tube); MicroSEQ (16S sequencing)	17, 18
Shiga toxin	Meridian premier EHEC EIA kit (catalog no. 608096); real-time PCR (stx1 + stx2)	19, 20
Salmonella spp.	Culture: XLD, MAC, SS, BS, BG + GN, Sel; API20E + serology (O and H antigens)	17, 18
Shigella spp.	Culture: XLD, MAC, SS + GN, Sel; API20E + serotyping	17, 18
Campylobacter spp.	Culture: campylobacter agar at 42°C, Gram stain; catalase, oxidase, hippurate, indoxyl acetate, MIDI, MicroSEQ (16S sequencing)	17, 21
C. difficile, toxigenic	Cepheid Xpert C. difficile; MicroSEQ (16S sequencing)
Cryptosporidium spp.	Meridian EIA Crypto/Giardia (catalog no. 250050); microscopy, trichrome stain	22, 23
Giardia lamblia	Meridian EIA Crypto/Giardia (catalog no. 250050); microscopy, trichrome stain	22–24
Rotavirus A	CDC real time RT-PCR	25
Norovirus GI/GII	CDC real time RT-PCR	26
Salmonella and Campylobacter sp.	MALDI-TOF	27

^aAdditional laboratory-validated methods for confirmation are available at MHDL but not referenced in this table.

^bMAC, MacConkey agar; XLD, xylose-lysine-deoxycholate agar; SS, Salmonella-Shigella agar; BG, brilliant green agar; GN, Gram-negative broth; EHEC, enterohepatic Escherichia coli; EIA, enzyme immunoassay; RT-PCR, reverse transcription-PCR; MALDI-TOF, matrix-assisted laser desorption ionization–time of flight; Sel, selenite; MIDI, Sherlock Microbial Identification System (Microbial Identification Inc. [MIDI]).

Tests of discrepant specimens were repeated at both sites by reextraction of the nucleic acid and follow-up testing. Stool specimens were analyzed using available molecular or conventional methods to resolve the discrepancy. Sample inhibition, determined by lack of internal control (IC) detection, was resolved by performing 1:10 dilution of the nucleic acid and retesting. Results from the two sites were compared after discrepancies were resolved, and statistical analysis was performed.

The Le Bonheur laboratory and MHDL analyzed 167 clinical stool specimens with 77 confirmed pathogen or pathogenic toxin-related infections between the two laboratories. Results by organism detected were summarized in Table 2. There were three discrepancies between the sites: two instances of rotavirus missed by each laboratory and one Shigella isolate detected by MHDL and not detected by the Le Bonheur laboratory (Table 2). There were five confirmed coinfections during this study. Three were positive for norovirus GI/GII and Clostridium difficile (toxin A/B), and two were positive for C. difficile (toxin A/B) and rotavirus A.

TABLE 2.

Blinded analysis of clinical stool specimens for GPP IVD site comparison^b

Organism	No. of specimens by laboratory:		No. of discrepant results
	Le Bonheur	MHDL
C. difficile toxin A/B	57	57	0
Rotavirus A	7	9	2
Norovirus GI/GII	6	6	0
Campylobacter spp.	1	1	0
Salmonella spp.	2	2	0
Shigella spp.	1	2	1
Total positive targetsa	74	77	3
Total negative targetsa	1,763	1,760	NAc

^a1,837 total GPP targets (167 specimens × 11 targets per test) were analyzed by the two sites in tandem to determine assay reproducibility.

^bSensitivity [number of true positives/(number of true positives + number of false negatives)] was 98.7% (92 to 99%). Specificity [number of true negatives/(number of true negatives + number of false positives)] was 99.8% (99 to 100%). Positive agreement value was 96.2% (88.6 to 99.0%). Negative agreement value was 99.8% (99.4 to 99.6%).

^cNA, not available.

The rates of inhibition for the Le Bonheur laboratory and MHDL were 19% and 14%, respectively. Each site used an 0.1-ml sterile loop to attempt to sample approximately 0.1 g of stool; however, there is variation in stool consistency as well as in user sampling techniques which affects sample volume. Resolution of inhibition (when all targets and IC remain undetected), according to the GPP package insert, is achieved by performing a 1:10 dilution of the extracted nucleic acid into nuclease-free water and reamplifying/reanalyzing the sample. The final rate of inhibition after performing the repeat testing protocol was 3% for each site.

Statistical analysis indicates that the assay was highly reproducible. The positive agreement value (PAV) was 96.2% (88.6 to 99%), while the negative agreement value (NAV) was 99.8% (99.4 to 99.6%). The assay showed a sensitivity of 98.7% (92 to 99%) with 99.8% (99.4 to 99.6%) specificity, which was determined upon resolving discrepant results between sites (Table 2).

Site reproducibility studies are good collaborative approaches to compare performances of diagnostic techniques. The results of these studies offer a wealth of information by determining clinical performance of new technology, such as the xTAG GPP IVD assay. During the site-to-site comparison, we found high correlations between our laboratories performing the GPP assay. Our data showed a significantly low rate of false negatives/positives, a high degree of sensitivity and reproducibility, and a high degree of positive and negative agreements.

Discrepancies among the methods were resolved using conventional gold standard and molecular techniques when available in order to confirm the results observed. The current FDA recommendation for the use of the xTAG GPP requires confirmatory testing of positive results using an established reference method. This recommendation, although essential for result accuracy when evaluating new technologies, can be time-consuming, cost-prohibitive, and burdensome, especially for smaller laboratories which lack personnel and reference methods. Although we believe this recommendation to be temporary until there are enough data to support the technology, our study indicates a possible way to minimize the confirmation burden while obtaining a statistical level of certainty for the results from the GPP assay.

The xTAG GPP IVD assay was added to the routine microbiology testing algorithm at both sites in order to assess its role as the primary diagnostic method. A total of 211 prospective clinical specimens from symptomatic patients with gastrointestinal (GI) infections were independently tested between the two sites during July to December 2013 during postevaluation studies on clinical and outbreak specimens. Results show a total of 83 confirmed pathogen target detections. The sites identified three false-negative and six false-positive results during the study (Table 3). We wanted to record a snapshot of GI pathogens circulating in our patient populations and to determine the significance of coinfections in clinical and public health settings.

TABLE 3.

Prospective study using xTAG GPP IVD assay as first-line detection method (June 2013 to December 2013)^b

Target	GPP	Confirmed	% occurrencec
C. difficile toxin A/B	23	23	10.9
Norovirus GI/GII	11	11	5.2
Salmonella spp.	22	19	9.0
E. coli Shiga toxin 1/2	3	3	1.4
Shigella spp.	10	7	3.3
Campylobacter spp.	1	1	0.47
Cryptosporidium spp.	11	14	6.6
Rotavirus A	2	2	0.94
Giardia lamblia	3	3	1.4
Total positive targetsa	86	83	39.2
Total negative targetsa	2,236	2,238	NAd

^a2,321 total xTAG GPP IVD targets (211 specimens × 11 targets per test) were independently analyzed at both study sites to evaluate assay performance as a screening method.

^bSensitivity [number of true positives/(number of true positives + number of false negatives)] was 96.4% (89% to 99%). Specificity [number of true negatives/(number of true negatives + number of false positives)] was 99.7% (99.3% to 99.8%). Positive agreement value was 90.1% (81.6% to 95.1%). Negative agreement value was 99.5% (99.1% to 99.8%).

^cOccurrence is based on number of confirmed infections divided by total number of specimens (n = 211).

^dNA, not available.

We confirmed a 39.2% infection rate. The top three pathogen targets detected were C. difficile (toxin A/B), Salmonella spp., and Cryptosporidium spp., accounting for 67% of the targets detected (Table 3). There were five confirmed coinfections during the postevaluation surveillance period: C. difficile (toxin A/B) plus norovirus GI/GII, C. difficile (toxin A/B) plus Salmonella spp., Shigella spp. plus norovirus GI/GII, Shigella spp. plus Cryptosporidium spp., and Cryptosporidium spp. plus norovirus GI/GII. The calculated occurrence of coinfections based on the total number of clinical specimens (n = 211) was approximately 2.3%.

Discrepancies between GPP and conventional methods and inhibited specimens were observed, but similarly to our site reproducibility, the frequency of these events was low. As before, we determined that most inhibition occurred as a result of differences in stool sampling. The dilution of the extracted nucleic acid significantly lowered the inhibition rate. A consideration going forward to maintain low inhibition is to avoid oversampling the stool. We determined the positive and negative agreement values for this assay to be 90.1% (81.6 to 91.5%) and 99.5% (99.1 to 99.8%), respectively.

During this part of the study, we were able to detect common clinical pathogens and small-scale outbreaks of Cryptosporidium hominis and norovirus. Our laboratories were also able to identify a small yet significant occurrence of coinfections. We believe that coinfections will become more relevant in diagnosis as more clinical and public health laboratories start using multiplex methods for GI pathogen testing.

The significance of the xTAG GPP results from this study is to showcase the ability to eliminate multiple testing by consolidating testing into an assay with high sensitivity and improved TAT. Recent studies have demonstrated that replacement of conventional methods with the xTAG GPP has improved sensitivity and TAT (12–15). In an effort to address the workflow, cost benefits, and increased efficiencies of bringing GPP into our laboratories, we compared the costs per specimen, TATs, and hands-on staff times for GI pathogen diagnosis between conventional methods and GPP (Table 4).

TABLE 4.

Laboratory cost and efficiency analysis of xTAG GPP compared to gold standard method

Cost and performance variable	Gold standard method(s)d	IVD GPPe	Efficiencyf (%)
Technician hands-on time (h)a	10	2.5	75
Time to detection (h)	72	5	93
Cost ($) per identification: reagents	50	80	−63
Cost ($) per specimen identification: laborb	250	62.5	75
Method scalability (n)c	10	22	45.5

^aEstimated hands-on time based on a single technician performing all the work.

^bCost estimate is based on hands-on labor for one technician and reagents.

^cLaboratory ability to scale up number of specimens identified without additional labor cost.

^dIdentification of 11 GI infection-causing pathogen targets using detection methods identified in Table 1.

^eDetection method for 11 GI infection-causing pathogen targets in a single stool specimen.

^fPotential laboratory savings by adding xTAG GPP to laboratory algorithm.

We divided laboratory costs into several categories, including reagents, labor cost, and potential scalability of the methods (without additional labor cost). We determined a potential increase in savings across the board, with the exception of reagent costs, which were higher than those for conventional methods. Compared with the overall savings across the other categories, however, we determined that reagent cost was not significant. A recent health economic study specifically looked at the cost savings for infection control due to reduction in the number of isolation days (12). Those findings describe shorter laboratory TAT and a reduction in the number of tests and specimens; however, there is a lack of publications quantifying efficiency benefits in laboratory workflow, materials, labor, and associated cost savings compared to other conventional tests. More outcome-driven data would allow clinical microbiologists and physicians to practice efficient and appropriate testing with an algorithm to assess the true impact on the outcome of patients with infection (16). While more of those studies are yet to come, we believe that the outcome of our study will shed light on comparative levels of effectiveness in clinical GI investigations and outcome measures with respect to individual patients and populations in general.

At the time of this publication, 4 targets available in the RUO panel were not present in the xTAG GPP IVD kit (Vibrio cholerae, adenovirus 40/41, Yersinia enterocolitica, and Entamoeba histolytica). It is unclear if these targets will eventually be included in the IVD version of the assay. For laboratories looking to expand their GI pathogen test panel, high validation costs can be mitigated by collaborations among institutions. The sharing of resources, technical knowledge, and clinical specimens among public health, academic, clinical, and reference laboratory networks can aid these efforts by defraying testing costs and increasing specimen diversity and accessibility.

We see a real added value to the implementation of the xTAG GPP IVD assay to clinical and public health microbiology laboratories. It broadens pathogen detection capabilities with high sensitivity and specificity, while reducing TAT. Detection methods across most of the laboratories are very specific but lack pathogen diversity. Current testing algorithms, shrinking laboratory funding, and lack of technical expertise in conventional culture-based techniques might prohibit the use of multiple detection methods. Use of the GPP assay is likely to offer improved diagnostic capabilities, better patient care, and better physician treatment satisfaction with additional pathogen information not previously or readily available. We anticipate that molecular multiplexing approaches would significantly improve laboratory efficiencies over the current multitiered GI disease diagnostic practices in clinical case management and public health enteric outbreak investigations.