Summary
Background
Trained canines are capable of detecting Clostridioides (Clostridium) difficile (CD) in the environment; however, the primary odour of interest on which the dogs alert is unclear.
Aim
To evaluate the inter-rater reliability of two canine detection teams for their ability to discriminate between scent pads containing CD-toxin-positive and -negative odours and their ability to discriminate between clostridial strains.
Methods
During a six-month period, two canine teams were tested weekly for their ability to detect CD-toxin-positive odours and discriminate between these and -negative odours. To further determine the canines' discrimination capability, scent pads impregnated with odours from reference isolates representing common CD toxin types (including toxin-negative CD isolates) or from clinical isolates representing other clostridial species were used.
Results
A total of 264 samples were tested with an overall sensitivity of 94.7% (Team A) and 86.8% (Team B) and specificities of 96.9% and 98.7%, respectively. Inter-rater reliability was very good (Cohen's kappa 0.87). When challenged with toxin- and non-toxin-producing strains, the teams alerted on 96.3% of all CD isolate odours (including nontoxigenic strains) and 46.7% of closely related species.
Conclusions
The canine teams exhibited strong inter-rater reliability on both clinical faecal specimens and reference CD isolates (both toxin and non-toxin producing) but were challenged to discriminate between CD and closely related clostridial species. These findings strongly support the development of scent detection programmes provided dogs and their handlers are properly trained and used in the right context.
Keywords: Clostridioides difficile, Canine, Reliability of results, Environmental microbiology
Introduction
The clinical environment can serve as a reservoir for Clostridioides difficile (CD) (an important cause of nosocomial diarrhoea), in part because of the organism's ability to form spores as well as its resistance to regular hospital cleaner/disinfectants [1]. Furthermore, cleaning practices may be suboptimal, particularly in large and complex healthcare settings. A reliable and efficient environmental detection method for CD would be a useful mitigation strategy to decrease the risk of environmental transmission.
Canine scent detection for CD environmental reservoirs has been implemented since 2016 at a Canadian large adult quarternary care hospital [2]. In a series of examinations that assessed odour recognition and search capability, the canine team demonstrated 85.7% sensitivity and 94.7% specificity [2]. Since then, a second canine team has been certified and deployed at the facility for approximately one year.
Currently, there are only a few canine teams capable of detecting CD; training (including preparation of scent material) and evaluative methods vary [2], [3], [4], [5]. Further, there is limited information on whether the primary odour of interest is one or more toxins produced by CD, the vegetative form of the organism, or a unique scent signature consisting of a mix of volatile organic compounds (VOCs) produced by the altered gut microbiota. This article describes the methods of evaluation and results of inter-rater reliability testing of the two canine teams at our facility. It also details the results of odour discrimination of various American Type Culture Collection (ATCC) CD strains and other clostridial species.
Methodology
Canine team
Handler A is a validated explosives and narcotics canine detection handler and a judge for the sport of nosework under the National Association of Canine Scent Work (Los Angeles, California, USA). Handler B is also a validated handler for narcotic and explosive detection dogs with a diploma in Canine Behavior Science and Technology through the Companion Animal Sciences Institute. Both canines are purebred Springer Spaniels trained to detect CD as described previously [2]. The teams are certified annually for both odour recognition and search capability by an independent validator in the field of canine scent detection.
Preparation of test materials: inter-rater reliability evaluation
Approximately 20 g of fresh faeces from non-repeat patient specimens submitted for polymerase chain reaction (PCR) CD toxin determination were used for this part of the experiment. All specimens were also tested for glutamate dehydrogenase and presence of toxin by C.Diff Quik Chek™ enzyme immunoassay (EIA) (Abbot Diagnostics, Princeton, NJ, USA) as well as cultured to cycloserine-cefoxitin fructose agar (CCFA). PCR- or Quik Chek™-positive samples that failed to grow on CCFA as well as specimens that were PCR negative but had a positive alert by the canine team were also cultured to C Diff Banana Broth™ (Hardy Diagnostics, Santa Maria, CA, USA). All faecal specimens used for this validation were subsequently stored at -70°C for future analysis. The fresh stool aliquots were placed in sterile urine containers and the odours captured on to clean gauze as previously described [2]. Gauzes were exposed to odours for periods between 6 and 36 h depending on time of specimen receipt and used within five days of preparation. A negative faecal odour from a patient specimen was defined as a specimen that was negative by all three diagnostic tests, while a positive faecal odour from a specimen was defined as a sample that was positive by any of the three diagnostic tests.
Preparation of test materials: evaluating discrimination capability of clostridial species
The Clostridium difficile Panel (ATCC® MP-4, Manassas, VA, USA) consisting of eight clinical isolates representing the common CD toxinotypes (types 0, IIIb, IIIc, tcdA-/tcdB-, V, VIII, XII and XXII) was used for this phase of the evaluation. In addition, clinical isolates of C. perfringens, C. ramosum, and two clinical strains each of C. innocuum, C. sordellii were retrieved from the -70°C freezer and subcultured twice on to Brucella agar before use in canine trials. Odours were captured on to gauze as previously described to minimize variation in expression of VOCs; care was taken to ensure that all scent pad odours were captured for 24 h [2].
Use of the scent wheel for inter-rater reliability testing
The study took place between June and November 2018. The Nolan Scent Training Carousel model M5-12p (Tactical-Directional Training Systems, Smithsburg, MD, USA) (Figure 1) was used to evaluate inter-rater reliability for odour recognition. The scent wheel consisted of 12 labelled arms, fully extended and raised approximately 60 cm above the ground. Arm #1 always held a blank container to orient and focus the canine. The other arms held a random assortment of blank canisters or canisters containing gauzes with the odours from PCR-toxin-negative or PCR-toxin-positive fresh clinical faecal specimens. No more than 50% of the canisters held clinical specimens during any evaluation to avoid overloading the carousel with faecal odours and hampering individual odour discrimination. Gloves were changed between placement of blanks, PCR-negative, and PCR-positive odours, and PCR-positive odours were placed last in the carousel to avoid cross-contamination of odours. A minimum of 10 min was required to elapse after canister placement and before canine evaluation to allow odours to stabilize in the headspace of the canisters [5]. Fresh canisters were used for each evaluation and each faecal gauze scent pad was only used once (i.e. clinical specimens were used only once). In addition, canisters and carousel arms were wiped with accelerated hydrogen peroxide between individual canine team runs to minimize the potential for pheromone (a secreted chemical signal) communication between dogs. At the end of each evaluation, used canisters were washed in an industrial washer.
Figure 1.
Scent wheel used for canine team inter-rater reliability testing. The scent wheel has 12 arms that can be set at different distances and the carousel can be set to various heights. The wheel can be rotated. Canisters containing the odours of interest are used once before being washed in an industrial washer.
Canisters were randomly placed by one of the evaluators and then each team was individually asked to search the carousel wheel in any direction they chose beginning with the blank position on arm no. 1. Teams were selected randomly for the start of each evaluation; the canine teams were quality controlled before each evaluation using PCR-positive gauze ‘hides’ as described previously [2]. The handler was given an approximation as to the number of positives in the carousel and the dog was directed to interrogate each container systematically. The handlers were blinded to the number of positive specimens and their positions, and they were not informed of the other handler's results until the end of the evaluation. The handler would announce an alert with true-positive alerts acknowledged by the investigator for reward of the dog. Each evaluation session was recorded digitally. In most carousel runs, two investigators were present, one to independently record results and one to place canisters, and were strategically placed so as not to interfere with the evaluation.
Use of the scent wheel for evaluating clostridial species
For this part of the evaluation, no more than three Clostridioides species (ATCC CD or other clinical isolate) were evaluated during a canister session; the remaining canisters were blank. Each carousel session consisted of three separate runs using the same organisms, however with the numbered arms changed and the scent wheel rotated between each run (i.e. the test was conducted in triplicate for each canine team). The handler would call out an alert and the evaluators (who remained outside the room and the team's visual field) would acknowledge correct results for reward of the dog. The wheel was then wiped down with disinfectant and the second canine team (who remained isolated from the testing process until that time) were asked to interrogate the canisters.
Data was entered into an Excel spreadsheet and the inter-rater reliability was quantified using Cohen's kappa (κ) and the sensitivity and specificity were calculated.
Results
Inter-rater reliability testing
A total of 264 canisters were interrogated by both canine teams of which 115 were blanks and 149 contained odours from clinical faecal specimens. Of the latter, 32 specimens were PCR positive, 34 were EIA (Quik Chek™) toxin positive, 27 grew CD and 38 were positive by any method (Table 1). Table 2 details the individual canine team performance. Team A had more false-positive alerts compared to Team B, but overall sensitivity (using any positive diagnostic method as the criteria) was 94.7% for Team A and 86.8% for team B. Specificity was excellent at 96.9% for Team A and 98.7% for Team B. Inter-rater agreement for all containers was 96.6% with a Cohen's κ of 0.87 (Table 3) indicating very good agreement.
Table I.
Summary of faecal samples used for inter-rater reliability testing
| Positive Clostridioides difficile faecal samples from patients | Number (n) |
|---|---|
| By PCR | 32 |
| PCR CTa range | 23.88–38.4 |
| PCR CT average | 31.5 |
| By Quik Chek™ | 34 |
| By Culture | 27 |
| By PCR and Quik Chek™ | 32 |
| By PCR and culture | 22 |
| By Quik Chek™ and culture | 23 |
| Positive by all three methods | 22 |
| Positive by any method | 38 |
| Negative C. difficile faecal samples from patients | |
| By PCR | 117 |
| By Quik Chek™ | 115 |
| By Culture | 122 |
| By all methods | 111 |
| Blank containers | 115 |
PCR, polymerase chain reaction.
CT, cycle threshold, i.e. the number of cycles for the signal to exceed the background level.
Table II.
Canine team performance
| Parameter | Team A alerts |
Team B alerts |
||
|---|---|---|---|---|
| (N) (%) | (N) (%) | |||
| Total alerts | ||||
| Positive by PCR | 31 | 96.9% | 29 | 90.6% |
| Positive by Quik Chek™ | 33 | 97.1% | 31 | 91.2% |
| Positive by culture | 25 | 92.6% | 24 | 88.9% |
| Positive by any diagnostic test | 36 | 94.7% | 33 | 86.8% |
| False positive alerts | 7 | 6.3% | 3 | 2.7% |
| Total alerts | ||||
| Negative by PCR | 1 | 0.9% | 3 | 2.6% |
| Negative by Quik Chek™ | 1 | 0.9% | 3 | 2.6% |
| Negative by culture | 2 | 1.6% | 3 | 2.5% |
| Negative by any method | 2 | 0.9% | 5 | 2.2% |
| Specificity | 96.9% | 98.7% | ||
| Sensitivity | 94.7% | 86.8% | ||
PCR, polymerase chain reaction.
Table III.
Inter-rater reliability assessment
| Total canisters tested | 264 |
| Total faecal odours tested | 149 |
| Agreement by canine teams (include blanks) | 255 |
| Disagreement by canine teams | 9 |
| % Agreement (includes blanks) | 96.6% |
| % Disagreement (includes blanks) | 3.4% |
| Cohen's kappa agreement | 0.866 |
Evaluation of discrimination capability for clostridial strains
Each of the eight ATCC strains (ATCC MP-4), were tested in triplicate by the canine teams; seven strains produced binary toxin while strain BAA-1801 was toxin negative. ATCC 43598 was tested twice resulting in 48 containers with ATCC CD-toxin-positive odours and six with odours from a non-toxin-producing CD strain. Six Clostridium species (C. perfringens, C. ramosum, C. innocuum (N=2), C. sordellii (N=2)), were evaluated on two different occasions for a total of 36 containers with non-CD clostridial odours. There were 240 blank canisters for this phase of the evaluation. The canine teams had a sensitivity of detection for CD of 92.9% (Table 4). Importantly, they alerted on the non-toxin-producing ATCC strain (BAA-1801; tcdA-, tcdB-) on five of six canister interrogations. C. perfringens, the species tested with the least relatedness to CD never resulted in an alert. However, in 14/30 (46.7%) instances, the dogs alerted on strains closely related to CD.
Table IV.
Results of scent-discrimination testing
| Strain | Toxinotype | Total alerts (both canine teams) |
|---|---|---|
| ATCC CD 1801 | tcdA-, tcdB- | 5/6 |
| ATCC CD 1803 | IIIc | 6/6 |
| ATCC CD 1804 | 0 | 6/6 |
| ATCC CD 1870 | IIIb | 6/6 |
| ATCC CD 1814 | XXII | 6/6 |
| ATCC CD 43598 | VIII | 11/12 |
| ATCC CD 1812 | XII | 6/6 |
| ATCC CD 1875 | V | 6/6 |
| Clostridium perfringens | NA | 0/6 |
| C. innocuum (two strains) | NA | 4/12 |
| C. ramosum | NA | 3/6 |
| C. sordeilli (two strains) | NA | 7/12 |
| Alerts on ATCC strains | 52/54 (96%) | |
| Sensitivity | 92.9% |
Discussion
CD can be a difficult and devastating infection in acute-care facilities and detecting and eliminating environmental reservoirs of this spore-forming pathogen could be a useful mitigation strategy. It is important to emphasize that the goal of this study was not to train CD detection dogs for ‘point of care diagnostic’ purposes, partly because rapid molecular diagnostics are readily available. However, the most important factor in that decision was that it was unclear whether the dogs were alerting on the VOCs associated with the toxin itself, the organism, closely related clostridial species or even a unique faecal gut microbiota signature associated with CD [6], [7]. The exact nature of the VOC alert was not an issue for an environmental search as the presence of any Clostridium sp. in a cleaned area was an unacceptable result. It did, however, preclude the dogs from being used as a bedside diagnostic tool.
A previous article detailed the excellent sensitivity and specificity of a single canine team, however, the training of a second team afforded the opportunity to evaluate inter-rater reliability [2]. The canine teams were evaluated using the same rigour applied to assessing the reliability of a laboratory diagnostic test. The teams were consistent in their alerts as demonstrated by the strong inter-rater reliability κ-value. Sensitivity of detection ranged from 96.9% to 98.7% and specificity was between 86.8% and 94.7%. The strong negative predictive value supported the use of the canine teams to rule out or ‘clear’ rooms/areas recently cleaned and disinfected.
The inter-rater reliability results differed from a recent Canadian study; however, the differences in sample preparation and training protocols for clinical canine scent detection make the studies difficult to compare. There were several important differences in the canine training and the evaluation methods that may account for the observed difference in results [3]. In this study, canister work (the scent carousel) was used exclusively for the inter-rater reliability assessment in order to evaluate only odour recognition and not search capability [5], [8], [9]. In addition, the Vancouver Coastal Health canine teams used scent pads that contained only the odour of the CD-positive faecal specimens or the distractor (CD-negative specimens) rather than the actual faeces. Faeces, particularly when kept for long periods of time, will change their microbial composition, concentrations of the various organisms, and the VOCs generated [10].
Table 1 illustrates that the laboratory tests did not always agree. In a few instances, in retrospect, the dogs alerted on the PCR/QuickChek™ negative specimens that were CD culture positive after a week of incubation but were not rewarded at the time of the testing. This reinforced the investigator's perception that the dogs were in fact not alerting to the toxin per se but rather to the organism itself or a specific VOC signature from faeces, potentially in minute concentrations given the delayed time for the CD to grow. Recent studies have supported the concept of a VOC signature by demonstrating that a CD-positive treated stool could be identified by the detection of a combination of specific VOCs specifically 2-fluro-4-methylphenol, isocaproic acid, and p-cresol by gas chromatography mass spectrometry (HS-SPME-GC-MS) [11].
To further explore the observations and provide additional insight into the dogs' discriminatory ability, the ATCC CD Panel of eight CD toxinotypes along with members of the Clostridium sp. with varying relatedness to CD were used in the second phase of the evaluation. Interestingly, the dogs falsely alerted on odours from C. sordelii and C. innocuum. C. sordelii is a toxin-producing organism with comparable fatty acid profile immunological cross-reactivity to CD that has recently been proposed to be included in the CD genus [12], [13], [14]. C. innocuum has recently been described as a pathogen causing post-antibiotic diarrhoea associated with severe colitis and mortality; it is unknown whether the disruption to the gut microbiome and the volatile organic compound signature is similar to that which occurs with CD [8].
In conclusion, the canine teams exhibited strong inter-rater reliability on both clinical faecal specimens and ATCC CD isolates (both toxin and non-toxin producing). They were challenged to discriminate between CD and closely related Clostridium species. In practical terms, this occasional ‘false alert’ would result in recleaning of a piece of equipment or a clinical area. However, that item or surface is highly likely to be contaminated with a member of the clostridial family and the recleaning would be warranted. Conversely, a negative search provides a high degree of confidence that no reservoirs of CD or other clostridia are present. These findings strongly support the development of scent detection programmes, provided dogs and their handlers are properly trained and used in the right context. Further investigations of the microbiome and the VOC signature of CD-positive faeces is needed to provide further insights into the field of scent detection.
Conflicts of interest
The authors have no conflicts of interest to declare.
Funding
None.
References
- 1.McDonald L.C., Gerding D.N., Johnson S., Bakken J.S., Carroll K.C., Coffin S.E. Clinical Practice Guidelines for Clostridium difficile Infection in Adults and Children: 2017 Update by the Infectious Diseases Society of America (IDSA) and Society for Healthcare Epidemiology of America (SHEA) Clin Infect Dis. 2018 Mar 19;66(7):987–994. doi: 10.1093/cid/ciy149. [DOI] [PubMed] [Google Scholar]
- 2.Bryce E., Zurberg T., Zurberg M., Shajari S., Roscoe D. Identifying environmental reservoirs of Clostridium difficile with a scent detection dog: preliminary evaluation. J Hosp Infect. 2017 Oct;97(2):140–145. doi: 10.1016/j.jhin.2017.05.023. [DOI] [PubMed] [Google Scholar]
- 3.Taylor M.T., McCready J., Broukhanski G., Kirpalaney S., Lutz H., Powis J. Using Dog Scent Detection as a Point-of-Care Tool to Identify Toxigenic Clostridium difficile in Stool. Open Forum Infect Dis. 2018 Aug 22;5(8):ofy179. doi: 10.1093/ofid/ofy179. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Bomers M.K., van Agtmael M.A., Luik H., Vandenbroucke-Grauls C.M., Smulders Y.M. A detection dog to identify patients with Clostridium difficile infection during a hospital outbreak. J Infect. 2014 Nov;69(5):456–461. doi: 10.1016/j.jinf.2014.05.017. [DOI] [PubMed] [Google Scholar]
- 5.Concha A., Mills D.S., Feugier A., Zulch H., Guest C., Harris R. Using sniffing behavior to differentiate true negative from false negative responses in trained scent-detection dogs. Chem Senses. 2014 Nov;39(9):749–754. doi: 10.1093/chemse/bju045. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Angle C., Waggoner L.P., Ferrando A., Haney P., Passler T. Canine detection of the volatilome: a review of implications for pathogen and disease detection. Front Vet Sci. 2016 Jun 24;3:47. doi: 10.3389/fvets.2016.00047. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.El Manouni el Hassani S., Berkhout D.J.C., Bosch S., Benninga M.A., De Boer N.K.H., De Meij T.G.J. Application of fecal volatile organic compound analysis in clinical practice: current state and future perspectives. Chemosensors. 2018;6(3):29. [Google Scholar]
- 8.Chia J.H., Wu T.S., Wu T.L., Chen C.L., Chuang C.H., Su L.H. Clostridium innocuum is a vancomycin-resistant pathogen that may cause antibiotic-associated diarrhoea. Clin Microbiol Infect. 2018 Nov;24(11):1195–1199. doi: 10.1016/j.cmi.2018.02.015. [DOI] [PubMed] [Google Scholar]
- 9.Nevada Police Officer's Standards and training Commission . 2014. Nevada POST K-9 guidelines. [Google Scholar]
- 10.Bosch S., El Manouni El Hassani S., Covington J.A., Wicaksono A.N., Bomers M.K., Benninga M.A. Optimized sampling conditions for fecal volatile organic compound analysis by means of field asymmetric ion mobility spectrometry. Anal Chem. 2018 Jul 3;90(13):7972–7981. doi: 10.1021/acs.analchem.8b00688. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Tait E., Hill K.A., Perry J.D., Stanforth S.P., Dean J.R. Development of a novel method for detection of Clostridium difficile using HS-SPME-GC-MS. J Appl Microbiol. 2014 Apr;116(4):1010–1019. doi: 10.1111/jam.12418. [DOI] [PubMed] [Google Scholar]
- 12.Cundy K.V., Willard K.E., Valeri L.J., Shanholtzer C.J., Singh J., Peterson L.R. Comparison of traditional gas chromatography (GC), headspace GC, and the microbial identification library GC system for the identification of Clostridium difficile. J Clin Microbiol. 1991 Feb;29(2):260–263. doi: 10.1128/jcm.29.2.260-263.1991. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Scaria J., Suzuki H., Ptak C.P., Chen J.W., Zhu Y., Guo X.K. Comparative genomic and phenomic analysis of Clostridium difficile and Clostridium sordellii, two related pathogens with differing host tissue preference. BMC Genomics. 2015 Jun 10;16 doi: 10.1186/s12864-015-1663-5. 448-015-1663-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Yutin N., Galperin M.Y. A genomic update on clostridial phylogeny: Gram-negative spore formers and other misplaced clostridia. Environ Microbiol. 2013 Oct;15(10):2631–2641. doi: 10.1111/1462-2920.12173. [DOI] [PMC free article] [PubMed] [Google Scholar]