Table 2.
Modified scoring system for numerical assessment of study quality of canine scent detection work [24]
| Report* | Ref | Study design [I] | No. of dogs (score) [II] | Relevant information on training given? [III] | Sample characteristics/variability in DTE [IV] | Use of novel samples in DTE? [V] | Sample presentation repetition in DTE? [VI] | Equal sample treatment/preparation in DTE? [VII] | Olfactory transfer between different sample/inactivation types addressed? [VIII] | Randomization in DTE? [IX] | Blinding in DTE [X] | Presentation of results [XI] | Critical discussion of results [XII] | Total/max. score | Total/max. score in % |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Jendrny et al. (2020) | [46] | 0 | 3 | 1 | 1 | 2 | 0 | 1 | 0 | 1 | 2 | 0.5 | 0.5 | 12/17 | 70.6 |
| Grandjean et al. (2020) | [47] | 0 | 3 | 1 | 1 | 2 | 0 | 1 | 0 | 1 | 1 | 0.5 (suppl) | 0.5 | 11/17 | 64.7 |
| Eskandari et al. (2021) (all DTEs) | [48] | 0 | 2 | 0 | 2 | 0 | 1 | 1 | 0 | 0 | 1 | 0.5 | 0.5 | 8/17 | 47.1 |
| Grandjean et al. (2021) | [49] | 0 | 3 | 1 | 2 | 2 | 1 | 1 | 0 | 1 | 1 | 0 | 0.5 | 12.5/17 | 73.5 |
| Essler et al. (2021) (sixth DTE) | [50] | 0 | 3 | 1 | 1 | 2 | 0 | 1 | 0 | 0 | 1 | 0.5 | 0.5 | 10/17 | 58.8 |
| Hag-Ali et al. (2021) | [39] | 1 | 2 | 1 | 3 | 2 | 1 | 1 | 0 | 1 | 2 | 0.5 | 0.5 | 15/17 | 88.2 |
| Mendel et al. (2021) (third DTE) | [51] | 0 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0.5 | 0.5 | 5/17 | 29.4 |
| Angeletti et al. (2021) | [52] | 0 | 2 | 1 | 1 | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 6/17 | 35.3 |
| Sarkis et al. (2022) | [53] | 0 | 2 | 0 | 2 | 2 | 1 | 1 | 0 | 1 | 1 | 0.5 | 0.5 | 11/17 | 64.7 |
| Vlachová et al. (2021) (pre) | [54] | 0 | 2 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 1 | 0.5 | 0 | 5.5/17 | 32.4 |
| Jendrny et al. (2021) (all DTEs) | [55] | 0 | 3 | 1 | 3 | 2 | 0 | 1 | 1 | 1 | 2 | 0.5 | 0.5 | 15/17 | 88.2 |
| Wurtz et al. (2021) (pre) | [41] | 1 | 1 | 0 | 3 | 2 | 1 | 0 | 0 | 1 | 2 | 0.5 | 0 | 11.5/17 | 67.6 |
| Vesga et al. (2021) (third DTE) | [40] | 0 | 3 | 1 | 3 | 2 | 1 | 1 | 0 | 1 | 1 | 0.5 | 0.5 | 14/17 | 82.4 |
| Maia et al. (2021) | [56] | 0 | 2 | 0 | 2 | 0 | 1 | 1 | 0 | 1 | 2 | 0.5 | 0 | 9.5/17 | 55.9 |
| ten Hagen et al. (2021) (first DTE) | [57] | 0 | 3 | 1 | 2 | 2 | 0 | 1 | 1 | 1 | 2 | 0.5 | 0.5 | 14/17 | 82.4 |
| Grandjean et al. (2022) | [58] | 0 | 3 | 1 | 2 | 2 | 1 | 1 | 0 | 1 | 2 | 0.5 | 0.5 | 14/17 | 82.4 |
| Grandjean et al. (2022) | [59] | 0 | 2 | 0 | 1 | 0 | 0 | 1 | 0 | 1 | 1 | 0.5 | 0 | 6.5/17 | 38.2 |
| Devillier et al. (2022) (second DTE) | [60] | 0 | 2 | 1 | 2 | 2 | 1 | 1 | 1 | 1 | 1 | 0.5 | 0.5 | 13/17 | 76.5 |
| Chaber et al. (2022) | [61] | 0 | 3 | 1 (suppl) | 1 | 2 | 1 | 1 | 0 | 1 | 2 | 0.5 | 0.5 | 13/17 | 76.5 |
| Guest et al. (2022) (third DTE) | [62] | 0 | 3 | 1 | 3 | 2 | 1 | 1 | 0 | 1 | 2 | 0.5 | 0.5 | 15/17 | 88.2 |
| Mancilla-Tapia et al. (2022) (first DTE) | [63] | 0 | 2 | 1 | 2 | 2 | 1 | 1 | 1 | 1 | 1 | 0.5 | 0.5 | 13/17 | 76.5 |
| Maurer et al. (2022) (first DTE) | [44] | 0 | 2 | 1 | 3 | 2 | 1 | 1 | 0 | 1 | 2 | 0.5 | 0.5 | 14/17 | 82.4 |
| Kantele et al. (2022) (second DTE) | [43] | 0.5 | 2 | 1 | 3 | 2 | 1 | 1 | 0 | 1 | 2 | 0.5 | 0.5 | 14.5/17 | 85.3 |
| Grandjean et al. (2022) | [42] | 1 | 3 | 1 | 3 | 2 | 1 | 1 | 0 | 1 | 2 | 0.5 | 0.5 | 16/17 | 94.1 |
| Twele et al. (2022) (first DTE) | [64] | 0 | 3 | 1 | 2 | 2 | 0 | 1 | 1 | 1 | 2 | 0.5 | 0.5 | 14/17 | 82.4 |
| ten Hagen et al. (2022) (second to fourth DTE) | [45] | 0.5 | 3 | 1 | 3 | 2 | 1 | 1† | 1 | 1 | 2 | 0.5 | 0 | 16/17 | 94.1 |
| Demirbas et al. (2023) (third DTE) | [65] | 0 | 2 | 1 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 0.5 | 0.5 | 6/17 | 35.3 |
DTE = diagnostic test evaluation; Pre = preprint; Ref = reference; Suppl = supplementary material.
Studies with high quality (14.5–17 points [>85%]), medium quality (9.5–14 points [55%–85%]), and low quality (≤9 points [<55%]) in accordance with the review question.
I Diagnostic accuracy comparative study or cross-sectional study (1 point), cross-sectional study with addition of predefined samples, for example, because of low prevalence (0.5 points), case-controlled study or no information or unclear (0 points).
II ≥5 Dogs (3 points), 2–4 dogs (2 points), and one dog (1 point).
III Relevant training information available (1 point), and no or unclear training information available (0 points).
IV Multiple sample sources (+1 point), symptomatic and asymptomatic patients (+1 point), and patients tested negative with other disease or COVID-19 similar symptoms (+1 point), if conditions do not apply or no information (0 points for each).
V Novel samples for DTE used (2 points), known samples for DTE used or no information (0 points).
VI No sample repetitions in DTEs (1 point), and repetitions present in DTEs or no information (0 points).
VII Positive and negative samples in DTEs treated equally (1 point), samples not treated equally or biological samples tested against nonbiological samples, or no information (0 points) (†Authors showed before that sample inactivation [beta propiolactone] did not impact canine olfaction in terms of detecting noninactivated samples [55]).
VIII Olfactory transfer performance from one type of sample or inactivation method used in training to another type of sample or inactivation method used in the DTE addressed (1 point) and equal sample types or inactivation methods used in training and DTE (0 points).
IX Randomization of sample positions (1 point), and no randomization or no information (0 points).
X Double blinding of sample positions (2 points), single blinding (1 point), and no blinding or no information (0 points).
XI At least sensitivity and specificity available or extractable (0.5 points) and less information available or unclear (0 points).
XII Discussion about limitations and risk of bias (0.5 points), and no critical discussion available (0 points).
*Reports sorted by publication date.