Breath versus saliva for lung cancer detection with dogs

Abstract

Background

While some research shows that dogs are able to detect lung cancer at above-chance levels using breath samples, the relative utility of other sample types has not been established. We evaluated the comparative utility of human breath and saliva samples for lung cancer detection using dogs.

Methods

Seven dogs assessed breath and saliva samples from 154 patients attending a general respiratory clinic. Dogs were trained using an automated apparatus to identify samples from patients who were later diagnosed with lung cancer. Sensitivity and specificity measures were used to compare the dogs’ performance with each sample type.

Results

A mixed-methods logistic analysis of accurate responses to breath and saliva samples showed significantly higher detection of lung cancer-positive breath samples (mean 0.78, 95% CI 0.71–0.83) than of lung cancer-positive saliva samples (mean 0.42, 95% CI 0.34–0.50; p<0.001). There were no significant differences in accuracy of classification between non-target breath samples (mean 0.68, 95% CI 0.57–0.77) and non-target saliva samples (mean 0.68, 95% CI 0.56–0.77; p=0.854).

Interpretation

The higher sensitivity of dogs to breath samples than to saliva samples suggests that breath samples have greater utility for canine scent detection of lung cancer. Although these findings support the continued use of breath samples for volatile-based lung cancer detection, with methodological improvements, saliva samples may also have utility for this purpose.

Shareable abstract

Dogs detect lung cancer with greater accuracy using breath than saliva samples. Breath samples may be the better choice for volatile-based diagnostic technology but saliva may still have a role to play in lung cancer diagnostics. https://bit.ly/4gvyo06

Introduction

Lung cancer is the leading cause of cancer-related deaths globally [1]. This is, in part, because lung cancer is typically detected after it has already progressed to later stages when curative treatment is not possible. Early detection of lung cancer is an important determinant of treatment success [2–4], but recently implemented screening methods for early detection have been relatively costly, exploratory in nature or available to only a small subset of the population [5–8]. Results from analytical chemistry and scent-detection dog studies suggest that lung cancer is associated with changes in the volatile organic compound (VOC) profile in human breath samples [9–16]. VOC analysis, either with machines or with animals, may represent a fast, cost-effective and non-invasive screening method for detecting early-stage lung cancer. However, specific comparisons of the relative utility of different sample media (e.g. breath versus saliva) for VOC-based lung cancer screening have not been conducted.

Studies examining the detection of lung cancer using dogs have produced varied results, with sensitivity estimates ranging from 71% to 91% and specificity estimates ranging from 40% to 99% [17–19]. Diverse sample collection processes and training and testing methods have been used in the relevant studies [9–12, 20, 21]. Most studies to date have used breath samples, which have been collected and stored using a range of media (see supplementary file 1). At the same time, a growing body of research suggests that saliva has unique biomarkers that can be used to detect the presence of cancer [22–24]. Two saliva-based cancer detection research projects have used established microbiological enzyme-linked immunosorbent assay processes [25] and Raman spectroscopy [26] with promising results. The efficacy of saliva as a medium for disease detection using animals has not been explored, but there is some evidence that an analysis of VOCs conveying metabolic characteristics of saliva could be used for the diagnostic testing of lung cancer [27]. Both breath and saliva provide access to VOCs that are available in the blood, where reliable disease-associated VOC changes are likely to occur. With breath, this access is due to the gas exchange that occurs at the alveolar–capillary barrier, whereas with saliva, it is due to the rapid blood–saliva diffusion equilibrium that results from the thin salivary gland membranes [28]. Compared with breath samples, saliva has simpler handling and storage requirements and is easier to split into multiple samples, which is beneficial for training. The objective of the present study was to evaluate the relative effectiveness of breath samples (produced by breathing through filter material) and saliva samples (produced by mouth-rinsing with a small quantity of water) for canine detection of lung cancer following best-practice guidelines for scent-detection research [29–31].

Methods

Subjects

Pet dogs were recruited from the local community via word-of-mouth and social media in Hamilton, New Zealand. Dogs were selected for training based on temperament, food drive and independence. The University of Waikato Animal Ethics Committee approved all experimental procedures described herein (protocol 1029).

Apparatus and training procedures

Each dog was trained to work independently using an automated apparatus in the form of a carousel with 17 enclosed compartments. The front panel of the apparatus (1 m²) included a central aperture through which the dog put their nose to access the available sample. An automated feeder was positioned on the ground approximately 1.5 m from the apparatus. The duration of sample sniffing, determined by breakage of an infrared beam across the sample aperture, was used to define a positive indication, with the required sniff duration adjusted as required. A positive indication in the presence of a positive sample triggered the release of up to three pieces of kibble from the food dispenser. If a negative sample was present, positive indications had no effect, and the dogs were required to activate an omnidirectional switch attached to the edge of the front panel [32, 33].

During training, indication thresholds (the period of time each dog was required to remain with their nose in the port for a response to be treated as a positive indication) were individualised and systematically manipulated between 500 ms and up to 4500 ms, depending on the accuracy of each dog's responses. When a dog was performing with high sensitivity but low specificity, the threshold was systematically increased in 100-ms increments until specificity improved without significantly impacting sensitivity. Faster improvements in sample classification accuracy equated to faster increases in the indication duration threshold, so changes in the threshold value are indicative of a dog's rate of target acquisition.

The dogs trained for 2 days each week. On the first training day, the dogs were exposed to 17 breath samples in a randomised order. The following day, saliva samples from the same patients were presented to the dogs, also in a randomised order. After 18 weeks of training in this sequence, breath and saliva sample presentation was reversed to evaluate possible order effects. Further training and methodological details are provided in supplementary files 2 and 3.

Sample information

Samples were provided by human participants recruited from the general respiratory clinic at Waikato Hospital. Human ethics approval was obtained from New Zealand's Health and Disability Ethics Committees (17/NTB/178/AM02). Each participant was undiagnosed at the point of recruitment and was given the opportunity to participate in the project while they waited for their medical clinic appointment. Written consent was gained from all participants. All samples were stored until diagnostic information became available. Lung cancer diagnosis was adjudicated by senior clinicians (CLC and MBJ) using clinical, radiological and pathological data and following a multidisciplinary review.

Four breath samples were collected from each novel participant and sealed in glass tubes packed with polypropylene fibre and cotton fibre. In the same session, one saliva sample was collected using 10 mL of tap water to facilitate sample generation. This was expectorated into a polypropylene cup for later aliquoting. Samples were immediately stored at −80°C. After selection, individual saliva samples were thawed, aliquoted into four samples per participant and then used for training or refrozen. Corresponding breath samples were thawed, opened, used for training and then resealed and refrozen (see supplementary file 3 for additional details).

Data analysis

Sensitivity and specificity were calculated for each sample type for each dog. Sensitivity is the number of positive samples correctly indicated divided by the total number of positive samples. Specificity is the number of correct rejections of negative samples divided by the total number of negative samples. Log d and log B measures were also calculated for each dog and sample type (see supplementary file 4 for formulae). Higher log d values indicate a greater ability to distinguish between positive and negative samples; this is an index of successful target discrimination. Log B is an index of bias or preference in responding. A log B value above zero indicates a dog's tendency to classify samples as positive, regardless of their status. A log B value below zero indicates a dog's tendency to classify samples as negative, regardless of their status.

All calculations were performed on data from the dogs’ first exposure to a novel sample from a novel patient, which is the best indicator of dogs’ acquisition of the target concept [34]. Differences in responding to the two sample types were evaluated using a mixed-methods linear analysis with a logit link function. Individual dog performance was treated as a random factor, with sample type as a fixed factor. A Wilcoxon signed-rank test was applied to the log d and log B measures derived from the dogs’ responses to novel samples.

Additionally, because indication duration thresholds were adjusted according to the dogs’ performance with individual samples, the mean indication threshold times for all dogs for saliva and breath samples were also calculated as a metric of target acquisition. If one set of sample type response durations increased beyond the other, this suggests that the dogs were acquiring that target more quickly.

A period of unusually accurate target acquisition occurred across all six dogs’ responding in the first 15 training days. While this pattern of acquisition may reasonably be expected for simple target acquisition [35], research on complex target acquisition shows that acquisition requires exposure to hundreds of trials with hundreds of unique samples [36]. The uncharacteristic pace of lung cancer target acquisition suggested that it was likely that the dogs were in fact responding to a simple, unintended cue. Consequently, the sample handing processes were thoroughly revised, and the data generated during this period were eliminated from the analysis (see 34 for early acquisition data). The total remaining number of training days completed by the dogs was between 40 and 45 depending on their availability. One dog, Rocky, began training later than the others and after the process revision; his rate of learning was consistent with the anticipated pace of complex target acquisition. This suggests that the revision of sample handling processes had been successful.

Results

Of the 19 dogs recruited for the project, seven were selected to participate in training (table 1). 490 human participants provided samples for the training portion of the project; dogs’ responses to samples from 154 of those participants were used in the present analysis. 41 of these participants (26.6%) were diagnosed with lung cancer following sample collection. 48 participants were smokers at the time of sample collection; 35.4% of smokers were diagnosed with lung cancer compared with 22.6% of non-smokers. At the time of diagnosis, 17 participants had early-stage (I or II) lung cancer and 23 had late-stage (III or IV) lung cancer.

TABLE 1.

Participant dog profiles

Name	Sex#	Age at recruitment	Breed
BJ	Female	10 years	Beagle
Bramble	Female	3 years	English Springer Spaniel
Katie	Female	9 years	Blue Heeler X Border Collie
Onyx	Male	2 years	Labrador
Rocky	Male	4 years	Jack Russell X Beagle
Rylea	Female	6 years	Labrador
Tui	Male	3 years	Border Collie X Huntaway X Kelpie

^#All dogs were neutered.

Responses to novel samples

Sensitivity associated with novel breath samples (mean 0.78, 95% CI 0.71–0.83) was significantly higher than sensitivity scores for novel saliva samples (mean 0.42, 95% CI 0.34–0.50; p<0.001) (Saliva: positive predictive value (PPV) mean 0.34, 95% CI 0.29–0.40; negative predictive value (NPV) mean 0.75, 95% CI 0.75–0.80. Breath: PPV mean 0.47, 95% CI 0.38–0.56; NPV mean 0.86, 95% CI 0.75–0.98) (figure 1).

FIGURE 1.

Mean accuracy with breath versus saliva. Error bars indicate 95% confidence intervals.

A similar analysis of specificity data showed no significant difference between novel breath sample responses (mean 0.68, 95% CI 0.57–0.77) and novel saliva sample responses (mean 0.68, 95% CI 0.56–0.77; p=0.854).

The median log d value associated with breath sample responses was significantly higher than median saliva sample responses (Z=−2.028, p=0.043). This indicates that the dogs were better able to identify the target scent in breath samples than in saliva samples. Differences in dogs’ log B measures with breath and saliva samples were also found to be statistically significant (Z=−2.197, p=0.028) (figure 2). The dogs’ responses to saliva samples showed a conservative bias, while their responses to breath samples showed a liberal bias.

FIGURE 2.

Signal detection measures of responding accuracy, breath versus saliva. Log d and log B values associated with breath and saliva samples. Error bars indicate 95% confidence intervals.

Indication thresholds

The mean indication threshold for responses to all saliva samples during training was 2313 ms. By comparison, the mean indication threshold for responses to all breath samples during training was 2917 ms. This difference is indicative of the dogs’ faster acquisition of the lung cancer target when evaluating breath samples. Figure 3 shows an example of the differentiation in this indicative measure of target acquisition during “retraining” after a 2-month laboratory hiatus.

FIGURE 3.

Mean breath and saliva indication thresholds. Indication thresholds across 8 training days following laboratory closure and reopening. Error bars indicate 95% confidence intervals.

Smokers versus non-smokers

Overall, dogs were more likely to indicate smokers as lung cancer positive than non-smokers, with smoking associated with higher sensitivity (p=0.032) and lower specificity (p<0.001) in the linear model when smoking was included as a factor. When interactions between smoking and sample type were examined, a significant interaction was observed with saliva (p=0.034) and breath (p<0.001) specificity, smoking reducing specificity in both cases. Smokers were associated with higher sensitivity with saliva samples (p=0.017), but no significant interaction was found with breath and smoking status for sensitivity.

Lung cancer stage

There were no significant differences in the dogs’ sensitivity with early- versus late-stage lung cancer with saliva samples (early 0.41, 95% CI 0.31–0.51; late 0.43, 95% CI 0.34–0.51) or with breath samples (early 0.81, 95% CI 0.72–0.88; late 0.74, 95% CI 0.65–0.81).

Group decision criteria

By classifying a sample as “indicated” when most of the dogs had indicated the sample as positive, sensitivity of 0.34 (95% CI 0.2–0.51) and specificity of 0.69 (95% CI 0.60–0.78) were obtained with saliva samples. With breath samples, sensitivity of 0.78 (95% CI 0.62–0.89) and specificity of 0.69 (95% CI 0.60–0.78) were obtained. By extending this analysis to include responses to both saliva and breath samples from the same patient, treating patients as positively indicated when the majority of responses to both types of samples were positive, sensitivity of 0.59 (95% CI 0.42–0.74) and specificity of 0.71 (95% CI 0.62–0.79) were obtained.

Discussion

We compared dogs’ lung cancer detection performance when using breath and saliva samples. The dogs performed with significantly higher sensitivity when working with breath than when working with saliva. There were no differences in specificity between the two sample types. The discriminability index, log d, indicated that dogs could more readily distinguish between individuals with and without lung cancer when using breath than when using saliva. The bias index, log B, indicated a liberal bias towards indicating breath samples as positive, and a conservative bias away from indicating saliva samples as positive. These findings suggest that breath samples may have more utility than saliva for VOC-based lung cancer detection.

A clear differentiation between the dogs’ performance with breath samples and saliva samples was also apparent in the lower mean indication threshold (the minimum response duration required for a response to be classified as a positive indication) associated with the dogs’ training with saliva. The lower mean threshold value was a result of the dogs acquiring the lung cancer target more slowly when working with saliva samples because the indication threshold was raised only when dogs met performance criteria. Because the mean indication threshold was approximately 25% lower with saliva samples than with breath samples, it was less effortful for the dogs to indicate saliva samples as positive. Despite the increased effort required to indicate breath samples as positive, dogs were still more likely to do so than with saliva samples.

The dogs performed with lower specificity and higher sensitivity when evaluating the breath samples of smokers than when evaluating the samples of non-smokers, although a significant effect of smoking status on sensitivity was not found for breath samples. This result suggests that the dogs had a bias toward indicating smokers as lung-cancer positive, despite measures that were taken to prevent the development of such a bias (see supplementary file 3). Notably, there was no difference in sensitivity of detection of early- versus late-stage lung cancers. It is possible that some patients progressed between sample collection and final diagnosis and staging of lung cancer. These findings suggest that VOC-based approaches to early screening for lung cancer may be viable and should be investigated further.

Using group decision criteria allows the sensitivity/specificity trade-off to be adjusted because the cut-off value can be adjusted upwards or downwards, as required. Such an approach has been used successfully for tuberculosis screening using giant African pouched rats [37]. Additionally, this strategy can be used to analyse the data from multiple sample types in combination. In the present study, using a majority group decision criterion, the sensitivity value obtained by considering both breath and saliva sample data together was an intermediate value between the values obtained from each sample type individually. However, the combined specificity value was higher than either of the specificity values obtained from the individual sample types. This suggests that combining data from multiple sample types may improve diagnostic outcomes.

The mean sensitivity and specificity values associated with breath samples of 0.78 and 0.68, respectively, were obtained while the dogs were still learning the target concept, so the dogs may be capable of higher accuracy. These results fall within the range of accuracy values found in other studies of dogs as detectors of lung cancer [9–12, 20, 21]. Additional research is required to determine dogs’ suitability for lung cancer screening under operational conditions before they can be seriously considered as a diagnostic tool [38]. High accuracy values have also been obtained using electronic noses, although many of the relevant studies have limitations that need to be addressed in future research, and there is little research on the utility of saliva for this purpose [39].

With some modifications to the sample collection process, saliva samples may still have a role to play in VOC-based lung cancer screening. Methods of increasing sample material (for example by sampling multiple times) rather than diluting the sample to increase volume could enhance performance. Increases in the surface area or temperature of the sample could also improve performance in future research. With machine-based approaches to saliva analysis, the saliva itself can also be analysed, rather than the headspace VOCs alone [26, 27]. Our findings are primarily applicable to headspace analysis of saliva samples and are not reflective of the utility of saliva for cancer screening more generally.

With respect to breath sampling methodology, the methods used in the present study were based on methods used in previous animal studies, with feasibility a primary consideration. For example, rather than sampling into Tedlar^® bags or Tenax^® tubes, we captured VOCs in fibre, which allowed for prolonged and repeated use of the samples and desorption at room temperature. Nevertheless, some of the best-practice recommendations for breath sampling in analytical chemistry work could be adopted for future breath research with dogs, including using inhalation/exhalation filters to reduce the presence of irrelevant environmental VOCs in the sample [40].

In conclusion, the present findings suggest that dogs respond more accurately to breath samples than to saliva samples but that saliva samples can provide sufficient scent to produce better-than-chance responding. Based on these results, breath samples may be a better choice for training dogs to detect lung cancer, and potentially for VOC analysis using machines. However, with further refinement of the saliva sample collection process, saliva may be a viable substrate for volatile-based lung cancer detection.

Supplementary material

Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.

TABLE S1 Breath Sample Collection and Storage Methods for Canine Scent Detection of Lung Cancer Research

00914-2024.SUPPLEMENT

Standard Operating Procedure for Training Dogs to Use Automated Scent-Detection Apparatus

00914-2024.SUPPLEMENT2

Methods

00914-2024.SUPPLEMENT3

Data analysis

00914-2024.SUPPLEMENT4