Abstract
Introduction
Conventional diagnostic laboratory algorithms for determining the cause of infectious gastroenteritis include culture, biochemical identification and immunoassays. In addition, multiplex PCR-based testing has advanced into the gastroenterology diagnostic arena in recent years.
Aim
The purpose of this study was to evaluate the performance of a new molecular test (Diagnostics Solutions Laboratory GI-MAP) for the detection of bacterial and parasitic pathogens in stool samples spiked with known organisms.
Methodology
Faeces from a healthy human subject were pooled into a standard matrix and screened for the absence of bacteria, parasites and Helicobacter pylori antigen. Once confirmed negative single faecal aliquots from the matrix were spiked with solely one pathogen-type from a panel of 14 bacterial pathogens or one of 2 parasitic pathogens at a density of 5×106 organisms ml−1. Sixteen spiked samples in appropriate transport media were sent to two testing labs, specifically a reference site using the PCR-based BioFire FilmArray Gastrointestinal Panel, and a second lab using the GI-MAP assay. Seven negative control samples comprised solely of stool matrix were also submitted.
Results
Significant variability was found when the GI-MAP assay was used to test normal stool matrix with and without known bacteria and parasites at densities well within the expected limits of detection. The GI-MAP assay displayed a sensitivity of 80 % and a specificity of only 26 % due to many false positive results. This assay also reported quantitative numbers for pathogens. The BioFire FilmArray Gastrointestinal Panel achieved a sensitivity and specificity of 100 %.
Conclusion
The highly variable results for the GI-MAP assay were unexpected due to the precise pre-spike analysis and the overall maturation of nucleic acid amplification methods within the industry. Problematic to this assay is the poor level of specificity displayed by this assay reporting the presence of several pathogens, which could cause clinicians to treat with antibacterial and/or antiparasitic agents in the absence of any true pathogens.
Keywords: gastrointestinal testing, gastrointestinal pathogens, molecular detection, stool diagnostics
Introduction
According to the World Health Organization, there are 1.7 billion cases of gastroenteritis every year and approximately 1.5 million children deaths [1, 2]. The causes of infectious gastroenteritis are viruses (~70 %), bacteria (10–20 %) and parasites (<10 %) [3, 4]. Historically, the methods used to identify these groups of organisms were relatively slow, labour intensive and expensive, making their diagnostic utility confirmatory, at best. Molecular detection of enteric pathogens can provide a comprehensive and rapid alternative to conventional testing. A number of FDA-cleared multiplexed assays are available for use in clinical laboratories and target a wide range of bacterial, viral, and parasitic pathogens, including: BioFire FilmArray Gastrointestinal Panel (BioFire, Salt Lake City, UT, USA) [5], the xTAG Gastrointestinal Pathogen Panel (Luminex Corporation, Austin, TX, USA) [6], the Luminex Verigene Enteric Pathogen Test [7], and the BD MAX Enteric Bacterial, Viral and Parasite Panels (BD Life Science, Parks, MD, USA) [8–12].
The purpose of this study was to determine the performance of a new test (GI-MAP) for the detection of known bacterial and parasitic pathogens in spiked stool samples.
Methods
An employee of the contributing author’s previous employer (see Author statement) consented to providing a stool matrix sample that was used to construct the experimental specimens for this evaluation. The donor signed an informed consent, which remains on file with the funding agency. The stool matrix from this uninfected healthy subject was screened by an independent reference laboratory using the BioFire FilmArray Gastrointestinal Panel and the Helicobacter pylori antigen detection test by Meridian Bioscience (Newtown, OH, USA). All stool samples were negative for the 16 targets (14 bacterial and 2 parasitic) and the H. pylori antigen. To minimize the effects of pre-analytical variability, individual stool samples were subsequently homogenized and pooled to create a standard matrix, stored continually at 2–8 °C. Characterized control organisms representing the pathogenic panel in the GI-MA test were used to spike the sample matrix, following aseptic protocols to avoid cross-contamination.
Bacteria were received freeze-dried from the American Type Culture Collection (ATCC, Manassas, VA, USA) (see Table 1) and were grown on Trypticase Soy II blood agar (Becton Dickinson, Cockeysville, MD, USA) and CVA (cefoperazone, vancomycin and amphotericin B) plates (Becton Dickinson, Cockeysville, MD, USA) for Campylobacter . Bacterial suspensions were prepared with 0.45 % (w/v) sodium chloride (CareFusion Corporation, San Diego, CA, USA) to a density of approximately 1×108 c.f.u. ml−1 using the DensiCHEK Plus instrument (bioMérieux, Hazelwood, MO, USA).
Table 1.
Results of GI-MAP testing
Organism added to matrix |
GI-MAP results* |
Spike detection |
---|---|---|
Clostridium difficile Toxin A+, B+ATCC 9689 |
C. difficile, Toxin A – High, 1.4×103 c.f.u. gm−1 C. difficile, Toxin B – High, 2.15×107 c.f.u.gm−1 H. pylori† – High, 2.5×105 c.f.u.gm−1 |
TP - FP |
Clostridium difficile Toxin A-, B- ATCC 700057 |
C. difficile, Toxin A – less than detection limit (normal) C. difficile, Toxin B – less than detection limit (normal) |
TN - |
Clostridium difficile Toxin A-, B+ATCC 43598 |
C. difficile, Toxin A – High, 1.9×105 c.f.u. gm−1 C. difficile, Toxin B – High, 6.32×107 c.f.u. gm−1 Enterohemorrhagic E. coli – High, 1.13×103 c.f.u. gm−1 H. pylori– High, 2.8×105 c.f.u. gm−1 Vibrio cholerae – quantity detected but below the normal limit (normal), 5.33×100 c.f.u.gm−1 |
TP TP FP FP Equivocal |
Escherichia coli O157:H7 ATCC 35150 |
Enterohemorrhagic E. coli – High, 5.19×105 c.f.u. gm−1 E. coli O157 – High, 3.4×105 c.f.u.gm−1 Enterotoxigenic E. coli LT/ST – High, 1.35×107 c.f.u. gm−1 Shiga -like Toxin E. coli stx1 – High, 1.56×105 c.f.u. gm−1 Shiga -like Toxin E. coli stx2 – High, 5.18×104 c.f.u.gm−1 H. pylori – High, 1.7×103 c.f.u.gm−1 |
TP - - - - FP |
Shigella boydii ATCC 9207 |
Enteroinvasive E. coli/Shigella – High, 1.14×107 c.f.u.gm−1 |
TP |
Escherichia coli O26:H11 ATCC BAA-2196 |
Enterohemorrhagic E. coli – High, 1.97×107 c.f.u.gm−1 Shiga -like Toxin E. coli stx1 – High, 8.84×106 c.f.u.gm−1 Shiga -like Toxin E. coli stx2 – High, 4.99×105 c.f.u.gm−1 H. pylori – less than detection limits (normal) |
- TP - Equivocal |
Escherichia coli STX1+, STX2- O103:H11 CDC-3008 |
Shiga -like Toxin E. coli stx1 – quantity detected but below the normal limit (normal), 1.56×101 c.f.u.gm−1 |
FN |
Shigella sonnei ATCC 29930 |
Enteroinvasive E. coli/Shigella – High, 4.22×106 c.f.u. gm−1 |
TP |
Salmonella bongori ATCC 43975 |
Salmonella – High, 3.48×108 c.f.u.gm−1 |
TP |
Vibrio cholerae ATCC 25870 |
Vibrio cholerae – quantity detected but below the normal limit (normal), 1.24×103 c.f.u.gm−1 Enterohemorrhagic E. coli – quantity detected but below the normal limit (normal), 2.4×102 c.f.u.gm−1 H. pylori – High, 1.3×104 c.f.u. gm−1 |
FN Equivocal FP |
Salmonella EnteritidisATCC 13076 |
Salmonella – High, 2.89×107 c.f.u. gm−1 Adenovirus – quantity detected but below the normal limit (normal), 9.15×107 c.f.u. gm−1 H. pylori – High, 9.6×103 c.f.u. gm−1 |
TP Equivocal FP |
Yersinia enterocolitica O:8 ATCC 9610 |
Enterohemorrhagic E. coli – quantity detected but below the normal limit (normal), 4.99×101 c.f.u. gm−1 Yersinia enterocolytica - quantity detected but below the normal limit (normal), 2.58×102 c.f.u. gm−1 Giardia – High, 3.64×104 c.f.u.gm −1 H. pylori – High, 3.5×106 c.f.u. gm−1 |
Equivocal FN FP FP |
Campylobacter jejuni ATCC 33560 |
Campylobacter – High, 2.96×106 c.f.u.gm−1 |
TP |
Campylobacter coli ATCC 51729 |
Campylobacter – High, 1.33×106 c.f.u.gm−1 |
TP |
Giardia intestinalis |
Giardia – High, 1.43×106 c.f.u. gm−1 |
TP |
Cryptosporidium parvum |
Cryptosporidium – High, 5.48×105 c.f.u. gm−1 |
TP |
Negative control |
Negative |
TN |
Negative control |
Negative |
TN |
Negative control |
Giardia - quantity detected but below the normal limit (normal), 4.10×103 c.f.u. gm−1 |
TN |
Negative control |
Enteroinvasive E. coli/Shigella – High, 1.53×102 c.f.u. gm−1 H. pylori – High, 1.13×103 c.f.u. gm−1 |
FP FP |
Negative control |
H. pylori – High, 2.0×103 c.f.u. gm−1 |
FP |
Negative control |
H. pylori – High, 9.0×103 c.f.u. gm−1 |
FP |
Negative control |
Enterotoxigenic E. coli LT/ST – High, 6.92×108 c.f.u. gm,−1 |
FP |
*GI-MAP Laboratory report lists pathogens results as:<dl, (normal); a quantity detected with Log10 quantitation but below ‘normal’ quantity; or a quantity detected with Log10 quantitation note as ‘high’.
†Independent reference laboratory using the Meridian H. pylori antigen detection test.
TP, true positive; TN, true negative; FP, false positive; FN, false negative.
Cryptosporidium parvum and Giardia duodenalis (lamblia/intestinalis) were received in phosphate buffered saline (PBS) with antibiotics and 0.01 % (v/v) Tween 20 (Waterborne, New Orleans, LA, USA). C. parvum was supplied at approximately 5×108 oocysts ml−1 and G. duodenalis at approximately 1×108 cysts ml−1.
MCC Para-Fix C and S Medium (Modified Cary Blair) transport media (Medical Chemical Corporation, Torrence, CA) and Para-Pak C and S vials (Meridian Bioscience, Cincinnati, OH, USA) were inoculated in pairs with 5 ml of the homogenized stool matrix then mixed well. These vials were used as negative controls for this study.
Then, 1 ml of Cary Blair transport media was aseptically removed from each vial that was to be spiked in pairs with the bacteria or parasite pathogen. Subsequently 1 ml of the suspension of each of the pathogens listed in Table 1 was aseptically added to the vial, followed directly with 5 ml of the stool matrix for a final pathogen density of approximately 5×106 organism ml−1. This level was chosen to exceed normal limits of detection seen in multiplexed PCR assays (approximately 3×104 organisms ml−1). The volume of homogenized stool added to the negative controls and spiked specimens was the same for all vials.
Twenty three samples were shipped at room temperature by overnight courier to a reference laboratory and were tested within 4 days of preparation using the PCR-based BioFire FilmArray Gastrointestinal Panel to independently confirm expected results. The BioFire was utilized as a qualitative comparator assay, but not intended to serve as a true ‘gold standard’ comparison, even though it has established detection limits and published sensitivity/specificity data from other clinical trials [6, 7]. Paired vials were placed in individual test kits and shipped the same day at room temperature by overnight courier to Diagnostic Solutions Laboratory (Alpharetta, GA, USA) for testing using the GI-MAP assay. This assay, as indicated by this company, was developed and the performance characteristics determined by Diagnostic Solutions Laboratory.
GI-MAP Laboratory reports pathogen results as: ‘<dl’, a quantity detected with Log10 quantitation but below ‘normal’ quantity. A quantity detected with Log10 quantitation above the ‘dl’ is noted as ‘high’. For comparison to inoculated samples, and reference laboratory results, GI-MAP results of <dl and results with a quantity detected with Log10 quantitation but below ‘normal’ quantity were considered negative and results a quantity detected with Log10 quantitation note as ‘high’ were considered positive. At the time of this study, GI-MAP testing reports the detection of 12 bacterial pathogens, three parasitic pathogens and three viral pathogens. The company indicates that the results are reported as c.f.u. per gram of stool as determined by PCR. (https://www.diagnosticsolutionslab.com/tests/gi-map; accessed 2 July 2020).
Results
The results from the GI-MAP test are presented in Table 1. GI-MAP detected the organisms added to the matrix in 12 of 16 spiked samples, with a sensitivity of 80 %. Eleven of 23 total samples were reported as ‘high’, with Log10 quantitation above the normal limits for organisms that were not supplanted to the sample and were not detected by the BioFire assay. Because of multiple organisms being detected by this assay in the 23 samples, an overall specificity of just 27 % was realized. Of the seven samples with no organisms added, two were correctly reported as nothing detected (<dl), four were reported as a pathogen detected with high levels (one sample with two pathogens detected), and one sample as quantity detected but the quantity was below normal limits. All samples spiked with C. difficile were accurately detected by GI-MAP, as were the samples spiked with Salmonella bongori , Shigella sonnei , Campylobacter jejuni, Campylobacter coli, Giardia intestinalis and Cryptosporidium parvum. However, samples spiked with Yersinia enterocolitica and Vibrio cholerae were not detected by GI-MAP, and false positivity was encountered in several aliquots for H. pylori, and in individual negative controls sample aliquots for Enterotoxigenic E. coli and Enteroinvasive E. coli .
The BioFire FilmArray rendered a sensitivity of 100%, no false negatives, and a specificity of 100 %, no false positives (data not shown). Contrastingly, only one virus was reported by GI-MAP in one sample, Adenovirus, at a quantity of 9.15×107 c.f.u. gm−1 (Table 1). The sample matrix was negative for Adenovirus using the BioFire assay.
Discussion
There is a growing demand for faster results for microbiology testing as well as an increasing interest and application of molecular based assays. Studies have reported the successful application of molecular methods for detection of micro-organisms from human gastrointestinal samples [8–14]. To our knowledge, no previous studies have been conducted to evaluate this test. In this study we examined the ability of a novel DNA method for identifying pathogenic micro-organisms in human stool samples.
The highly variable results from this study were unexpected given the protocols employed to minimize pre-analytical variability and maturity of the field of nucleic acid amplification for the detection of stool pathogens. Limits of detection and primer differences are well known to affect PCR testing results but these factors do not explain the number of false-positive results in the absence of added target. The report of H. pylori by GI-MAP in nine samples as ‘high’ by PCR could not be confirmed due to the lack of a reference PCR assay, however, the use of the H. pylori antigen detection test was consistently negative among these same 20 aliquots. See Fig. 1, which illustrates this unanticipated degree of H. pylori variability among 23 aliquots expected to show a negative outcome for this pathogen.
Fig. 1.
Sample burden for H. pylori .
Quantitative multiplex PCR in stool, like that in the GI-MAP DNA Stool Analysis assay, has no established clinical relevance. Quantitation of target in stool as shown in Table 1, and Fig. 1 is extremely variable, as the volume and density of stool is affected by many factors, including diet, liquid consumption, and other medical conditions. This assay reports organism densities in c.f.u. gm−1of stool. However, standardizing this type of assay to the dry weight of stool is technically difficult and would still result in tremendous variability with diet, hydration and fibre intake. In addition, there are a significant number of variables associated with stool such as PCR inhibitors, pH, protein concentrations, primer affinity and other highly variable factors that can affect the reaction. Semi-quantitative PCR assays have recently been developed and tested using specimens where the volume can be controlled such as with bronchoalveolar lavage samples, namely the BioFire FilmArray Pneumonia Panel.
Importantly, the pathogenesis of most enteric pathogens is not dependent on quantity, and any detectable amount of pathogen is indicative of clinical infection. Detection below ‘threshold of normal’ is a result that may be misleading and likely represents assay background. This reporting should not be interpreted as low-level presence of pathogens, as there is no established low-level of pathogenic organisms such as Vibrio, Salmonella or Shigella that is acceptable or not associated with disease. In this study for example, in the sample to which Vibrio cholerae ATCC 25 870 was added to matrix and detected by BioFire, Vibrio cholerae was reported by GI-MAP as detected, but with a level lower than normal, giving a false-negative reading with a concentration of 1.24×103 c.f.u. gm−1 of stool. This is below the provided lower limit of normal: <1.00×105 c.f.u. gm−1. The presence of any amount of V. cholerae in a patient with symptoms is generally thought to be the cause of the disease and warrants treatment. In the sample with Clostridium difficile Toxin A-, B+ATCC 43598, GI-MAP reported the sample as having V. cholerae detected at a level of 5 c.f.u. −1, which is a detection level that is difficult to scientifically justify.
There were limitations in the present study. The initial stool sample/matrix was only tested by the BioFire FilmArray GI panel and the H. pylori antigen detection test by Meridian Bioscience. Both screening assays have specific limits of detection, sensitivity and specificity that may be different from the test assay and therefore testing results may not match. Specifically, there is currently no FDA-approved PCR method for the detection H. pylori , therefore it was decided to use a commonly accepted, FDA-approved method as a comparator. In addition, the design of this study allowed a single consistent matrix and a limited number of replicates. The lack of comparator quantitative PCR assays prevented a direct comparison with the quantitative results of the GI-MAP. Additional data would be needed to calculate positive and negative percent agreement with a gold-standard reference method to validate the false-positive and false-negative finding in the present study. In this study’s qualitative analysis the GI-MAP assay could not attain the sensitivity of the BioFire multiplex assay, and its specificity was surprisingly low. The extreme number of false positives could lead clinicians to treat patients in the absence of any true pathogen.
Although there is a need to develop rapid molecular testing assays for characterization of the gut microbiome, physicians and patients need to be aware that all stool analysis assays may not provide consistent results with both false-positive and false-negative results possible. The clinical implications for diagnosis and treatment of gastroenteritis is potentially significant because of missed diagnoses, and the use of antibacterial or antiparasitic agents in the absence of true infections. The limitations of the GI-MAP method reported here may restrict its applications for the diagnosis of gastroenteritis.
Funding information
Funding was provided to IIT Research Institute by Doctor’s Data, Inc. of St. Charles IL. Jack Maggiore was employed by this entity at the time of the research.
Acknowledgements
Bacteria were received from the American Type Culture Collection (ATCC), Manassas VA. Parasites were obtained from Waterborne Inc., New Orleans LA. The BioFire FilmArray Gastrointestinal Panel testing and the Helicobacter pylori antigen detection testing were conducted by Silver Cross Hospital, New Lenox IL, and the GI-MAP testing was performed by Diagnostic Solutions Laboratory, Alpharetta GA.
Author contributions
B.A.G. performed the investigation process of performing the research and sending out samples to testing labs. He was involved in writing the draft paper and editing the subsequent versions. In addition he was central to the project administration and acquiring funding for the conduct of the study. J.A.M. developed the design of the study and was involved with writing the draft paper and editing the subsequent versions.
Conflicts of interest
The authors declare there are no conflicts of interest.
Ethical statement
The experimental work did not fall under the definition of Human Subjects Research and an exemption by IIT Research Institute’s Institutional Review Board (Ethical Committee) was issued. The human stool matrix was provided by a human donor who entered into an informed consent agreement, which is on file with the funding agency.
Footnotes
Abbreviations: c.f.u., colony forming unit; c.f.u. gm−1, cfu/gm; c.f.u. ml−1, cfu/ml; dl, detection limit.
References
- 1.American Academy of Pediatrics, Committee on Infectious Diseases Red Book: Report of the Committee on Infectious Diseases. Elk Grove Village, IL: American Academy of Pediatrics; 2018. [Google Scholar]
- 2.Elliott EJ. Acute gastroenteritis in children. BMJ. 2007;334:35–40. doi: 10.1136/bmj.39036.406169.80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Kotloff KL, Platts-Mills JA, Nasrin D, Roose A, Blackwelder WC, et al. Global burden of diarrheal diseases among children in developing countries: incidence, etiology, and insights from new molecular diagnostic techniques. Vaccine. 2017;35:6783–6789. doi: 10.1016/j.vaccine.2017.07.036. [DOI] [PubMed] [Google Scholar]
- 4.UNICEF/WHO Diarrhoea: why children are still dying and what can be done. United Nations Children’s Fund/World Health Organization. New York, NY. 2009 https://apps.who.int/iris/bitstream/handle/10665/44174/9789241598415_eng.pdf;sequence=1
- 5.Buss SN, Leber A, Chapin K, Fey PD, Bankowski MJ, et al. Multicenter evaluation of the BioFire FilmArray gastrointestinal panel for etiologic diagnosis of infectious gastroenteritis. J Clin Microbiol. 2015;53:915–925. doi: 10.1128/JCM.02674-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Hammond SP, Gagne LS, Stock SR, Marty FM, Gelman RS, et al. Respiratory virus detection in immunocompromised patients with FilmArray respiratory panel compared to conventional methods. J Clin Microbiol. 2012;50:3216–3221. doi: 10.1128/JCM.00538-12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Rand KH, Rampersaud H, Houck HJ. Comparison of two multiplex methods for detection of respiratory viruses: FilmArray RP and xTAG RVP. J Clin Microbiol. 2011;49:2449–2453. doi: 10.1128/JCM.02582-10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Navidad JF, Griswold DJ, Gradus MS, Bhattacharyya S. Evaluation of Luminex xTAG gastrointestinal pathogen analyte-specific reagents for high-throughput, simultaneous detection of bacteria, viruses, and parasites of clinical and public health importance. J Clin Microbiol. 2013;51:3018–3024. doi: 10.1128/JCM.00896-13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Hyun J, Ko DH, Lee SK, Kim HS, Kim JS, D.H K, Kim JS, et al. Evaluation of a new multiplex real-time PCR assay for detecting Gastroenteritis-Causing viruses in stool samples. Ann Lab Med. 2018;38:220–225. doi: 10.3343/alm.2018.38.3.220. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Anderson NW, Buchan BW, Ledeboer NA. Comparison of the BD max enteric bacterial panel to routine culture methods for detection of Campylobacter, enterohemorrhagic Escherichia coli (O157), Salmonella, and Shigella isolates in preserved stool specimens. J Clin Microbiol. 2014;52:1222–1224. doi: 10.1128/JCM.03099-13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Harrington SM, Buchan BW, Doern C, Fader R, Ferraro MJ, et al. Multicenter evaluation of the BD max enteric bacterial panel PCR assay for rapid detection of Salmonella spp., Shigella spp., Campylobacter spp. (C. jejuni and C. coli), and Shiga toxin 1 and 2 genes. J Clin Microbiol. 2015;53:1639–1647. doi: 10.1128/JCM.03480-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Simner PJ, Oethinger M, Stellrecht KA, Pillai DR, Yogev R, et al. Multisite evaluation of the BD max extended enteric bacterial panel for detection of Yersinia enterocolitica, enterotoxigenic Escherichia coli, Vibrio, and Plesiomonas shigelloides from stool specimens. J Clin Microbiol. 2017;55:3258–3266. doi: 10.1128/JCM.00911-17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Madison-Antenucci S, Relich RF, Doyle L, Espina N, Fuller D, et al. Multicenter evaluation of BD max enteric parasite real-time PCR assay for detection of Giardia duodenalis, Cryptosporidium hominis, Cryptosporidium parvum, and Entamoeba histolytica . J Clin Microbiol. 2016;54:2681–2688. doi: 10.1128/JCM.00765-16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Stokes W, Simner PJ, Mortensen J, Oethinger M, Stellrecht K, et al. Multicenter clinical validation of the molecular BD max enteric viral panel for detection of enteric pathogens. J Clin Microbiol. 2019;57:e00306–00319. doi: 10.1128/JCM.00306-19. [DOI] [PMC free article] [PubMed] [Google Scholar]