Abstract
Purpose
To analyze the utility of a 5-item odorant test (U-Smell-It™) in determining COVID-19 status in COVID-19 polymerase chain reaction (PCR)-positive and -negative participants.
Methods
Symptoms, COVID-19 status, and 5-item odorant test results were collected from general population COVID-19 testing in Louisiana (n = 1042), and routine COVID-19 screening of healthcare workers in a nursing home in Florida (n = 278) (ClinicalTrials.gov Identifier: NCT04431908).
Results
In the general population COVID-19 testing site, a cutoff point of ≤2 (0, 1, or 2 correct answers out of 5) achieved sensitivity of 40.0% (95% CI: 26.4%–54.8%) and specificity of 89.2% (95% CI: 87.1%–91.1%) in detecting COVID-19 infection. Within this population, analysis of individuals with no self-reported loss of smell/taste and runny/stuffy nose resulted in sensitivity of 38.1% (95% CI: 18.1%–61.6%) and specificity of 92.3% (95% CI: 89.1%–93.4%), while analysis of individuals with self-reported loss of smell/taste and/or runny/stuffy nose resulted in sensitivity of 41.4% (95% CI: 23.5%–61.1%) and specificity of 82.4% (95% CI: 77.7%–86.5%).
Conclusions
The quick turnaround time, low cost, reduced resource requirement, and ease of administering odorant tests provide many advantages as an indicator sign to help flag a molecular diagnostic COVID-19 test with relatively high specificity. Our results suggest that this odorant testing for olfactory dysfunction may be a viable option in pre-screening COVID-19 infection. This tool has the potential to allow for continued monitoring and surveillance, while helping mitigate surges of COVID-19 variants. Further investigation is warranted to observe the extent to which odorant testing might be applied in a serial testing scenario.
Keywords: COVID-19, Olfactory dysfunction, Odorant testing, Rhinology
1. Introduction
Temperature screening and symptom surveys, paired with polymerase chain reaction (PCR)- and antibody-based molecular tests, provide a means for (pre)-screening and diagnosing those at risk of having COVID-19, so as to mitigate infectious spread. However, in many middle- to lower-income countries, molecular testing is expensive and can require specialized intrastructure; as such, there remains a need for convenient, inexpensive, and easy point-of-care (POC) and home self-administered tests with serial testing capacity [1], [2]. This would allow for widespread, continued monitoring, while aiding in preparedness for expected future surges of COVID-19 and its variants [2]. Studies have indicated that odorant testing may be useful to identify COVID-19 patients in need of early treatment or quarantine [3], [4]. Though many varieties of commercially-available odorant tests exist [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], these can be relatively costly (e.g. University of Pennsylvania Smell Identification Test [UPSIT] test costs circa $28–40) and are not easily amenable to large scale and repeated population COVID-19 surveillance.
This study evaluates a new design of an olfactory test card (U-Smell-It™) with 5 odorants on a single card that could be deployed broadly. These ‘scratch and sniff’™ type devices are deemed safe by the FDA as they are non-invasive and non-toxic. The device card proposed here uses the same fragrance and microencapsulation process as used in the NIH Toolbox® tests for loss of smell. Further, the fragrances used in the U-Smell-It™ test kit are compliant with the International Fragrance Association, do not contain oils derived from nuts, wheat, or glutens, and do not use any phthalates or their derivatives. The device takes less than 60 s and the forced choice test has four-to-five odor options and a “no scent” option. We hypothesized that this test can be an objective sign that may help identify infected individuals as part of an enhanced sign/symptom monitoring plan.
2. Methods
2.1. Administering the odorant test
This study was approved by the Yale Human Investigation Committee (IRB# 2000028259; ClinicalTrials.gov Identifier: NCT04431908), and focused on two cohorts: “drive-thru” outpatients seeking COVID-19 tests at a site affiliated with the State of Louisiana Office of Public Health in Central Louisiana (symptomatic group) and asymptomatic healthcare workers (HCWs) in a Florida nursing home undergoing routine PCR screening (asymptomatic group). Testing was performed in September and October of 2020.
Patients and HCWs receiving COVID-19 testing were approached by a research team member to gauge interest, obtain consent, and ask screening questions. The test subjects used pencils to scratch back and forth the first window, sniff the window, then fill in the multiple choice circle (“no scent”, and scent A, B, C or D). This procedure will be repeated for 5 windows. The subject was also asked the following questions:
-
i.
“Do you have any new loss of taste or smell?” (Yes/No)
-
ii.
“Does anything smell different in the last year?” (Yes/No)
-
iii.
“Do you have a stuffy or runny nose?” (Yes/No)
Information gathered included a unique identifier, gender, age, responses to the five window smell test, responses to the three questions above, and RT-PCR results for COVID-19.
2.2. Statistical analysis
Individuals were considered positive on the odorant test for COVID-19 if they got less than or equal to a certain amount of smells correct. Individuals in the general population COVID-19 testing site were split into two groups: symptomatic and asymptomatic. Due to the low prevalence of COVID-19 in the routine HCW testing population, this group was not split. Asymptomatic and symptomatic individuals were herein narrowly defined based on responses to the self-reported symptom questions which included (A) new loss of smell/taste and (B) runny/stuffy nose. Individuals that answered “yes” to at least one of these two questions were considered symptomatic and those that did not report either of these symptoms were considered asymptomatic. SPSS v27.0 (IBM Corp. Armonk, NY) was utilized for descriptive statistics, univariate (t-tests, chi-squared tests, Fischer's exact tests), and multivariable (logistical regression) tests of significance. The Johns Hopkins University web-based calculator for receiver operating characteristic (ROC) curves was used [16]. ROC curve graphs were created using Microsoft Excel (Redmond, CA). Significance was set at p < 0.05. Youden's J statistic (Youden's index; sensitivity+specificity-1) was used in conjunction with ROC curves to help determine the optimal cut-off point for maximizing sensitivity and specificity [17]. Other cut-off points were also analyzed. Other descriptive statistics (e.g. sensitivity, specificity) were obtained using MedCalc [18], [19].
3. Results
3.1. General population COVID-19 testing site characteristics and symptoms
Fifty SARS CoV-2 PCR positive individuals were confirmed in the general population COVID-19 testing site of 1042 (4.8% prevalence; Table 1 ). At time of testing, 12 (24.0%) of these PCR+ individuals reported new loss of either taste or smell, and 25 (50.0%) reported stuffy/runny nose (Table 1). There was no significant differences in age or sex between the PCR+ and PCR- cohorts (Mean [SD] Age: 43.22 [16.48] vs. 45.95 [18.13]; p = 0.297) (24 [48.0%] vs. 404 [40.7%] Males; p = 0.313; Table 1). When compared to the PCR- cohort, there was a significantly greater proportion of individuals in the PCR+ cohort that reported a new loss of either taste or smell (12 [24.0%] vs. 59 [5.9%]; p < 0.001; Table 1) and stuffy/runny nose (25 [50.0%] vs. 274 [27.6%]; p = 0.001; Table 1).
Table 1.
General population COVID-19 testing site characteristics.
| Variable | COVID-19 PCR positive (n = 50) | COVID-19 PCR negative (n = 992) | P-Value | |
|---|---|---|---|---|
| Age, y | 0.297a | |||
| Min-Max (Median) | 18–80 (41) | 17–92 (46) | ||
| Mean (SD) | 43.22 (16.478) | 45.95 (18.125) | ||
| Sex, No. (%) | 0.313b | |||
| Male | 24 (48.0%) | 404 (40.7%) | ||
| Female | 26 (52.0%) | 586 (59.1%) | ||
| Symptoms, No. (%) | ||||
| Loss of smell/taste | 12 (24.0%) | 59 (5.9%) | <0.001b | |
| Stuffy/runny nose | 25 (50.0%) | 274 (27.6%) | 0.001b | |
| Smell test score, No. (%) | ||||
| 0 | 9 (18.0%) | 13 (1.3%) | ||
| 1 | 1 (2.0%) | 21(2.1%) | ||
| 2 | 10 (20.0%) | 73 (7.4%) | ||
| 3 | 8 (16.0%) | 146 (14.7%) | ||
| 4 | 10 (20.0%) | 313 (31.6%) | ||
| 5 | 12 (24.0%) | 426 (42.9%) | ||
| Mean (SD) | 2.90 (1.764) | 4.02 (1.128) | <0.001a |
1 data point missing for age and 2 missing for sex.
Bold indicates statistically significant P-value.
t-test.
Chi-squared test.
3.2. Odorant test utility in general population COVID-19 testing site
There was a significant difference in the mean olfactory score of PCR+ and PCR- individuals (2.90 vs. 4.02 out of 5; p < 0.001; Table 1; Fig. 1 ). The ability for the odorant test to indicate COVID-19 infection was assessed using a ROC curve analysis. The sensitivity and specificity for different number of correct answers (0 to 4 out of 5) were determined, as well at the Younden's index at each cutoff (Table 2 ). For the total general population COVID-19 testing site, Youden's index yielded a maximal value of 0.305 (Sensitivity = 0.56; Specificity = 0.75) at a cutoff point of ≤3 correct answers (i.e., 0, 1, 2, or 3 out of 5 questions correct indicating positive odorant test for olfactory dysfunction, and therefore potentially COVID-19) (Table 2).
Fig. 1.
Number of odors correct on odorant test in general population COVID-19 testing A) PCR- individuals B) PCR+ individuals.
Table 2.
ROC analysis for total and asymptomatic/symptomatic individuals in general population COVID-19 testing site.
| Smell test cutoff (positive if less than or equal to) | Sensitivity | Specificity | Youden's index |
|---|---|---|---|
| All individuals in general population COVID-19 testing (n = 1042; AUC: 0.684) | |||
| 0 of 5 Correct | 0.180 | 0.987 | 0.167 |
| ≤ 1 of 5 Correct | 0.200 | 0.966 | 0.166 |
| ≤ 2 of 5 Correct | 0.400 | 0.892 | 0.292 |
| ≤ 3 of 5 Correct | 0.560 | 0.745 | 0.305 |
| ≤ 4 of 5 Correct | 0.760 | 0.429 | 0.189 |
| Symptomatic in general population COVID-19 testing (n = 313; AUC: 0.708) | |||
| 0 of 5 Correct | 0.222 | 0.972 | 0.194 |
| ≤ 1 of 5 Correct | 0.222 | 0.937 | 0.159 |
| ≤ 2 of 5 Correct | 0.444 | 0.825 | 0.269 |
| ≤ 3 of 5 Correct | 0.630 | 0.706 | 0.336 |
| ≤ 4 of 5 Correct | 0.852 | 0.392 | 0.244 |
| Asymptomatic in general population COVID-19 testing (n = 729; AUC: 0.612) | |||
| 0 of 5 Correct | 0.130 | 0.993 | 0.123 |
| ≤ 1 of 5 Correct | 0.174 | 0.977 | 0.151 |
| ≤ 2 of 5 Correct | 0.348 | 0.919 | 0.267 |
| ≤ 3 of 5 Correct | 0.478 | 0.761 | 0.239 |
| ≤ 4 of 5 Correct | 0.655 | 0.445 | 0.100 |
Asymptomatic individuals are those who reported no loss of smell/taste or runny/stuffy nose.
Bold indicates cutoff which maximizes Youden's index.
Symptomatic individuals were defined as those who reported new loss of either taste or smell and/or stuffy/runny nose. In the symptomatic population (n = 313), Youden's index yielded a maximal value of 0.336 at a cutoff point of ≤3 correct answers (Table 2). In the asymptomatic for loss of smell/taste or runny/stuffy nose population (n = 729), Youden's index yielded a maximal value of 0.267 at a cutoff point of ≤2 correct answers (Table 2).
With our goals of balancing sensitivity and specificity, and maximizing the accuracy and odds ratio, while minimizing falses positives, a cutoff point of ≤2 correct answers was used for all further analysis of the general population COVID-19 testing site due to its overall better performance in all of the aforementioned metrics.
A cutoff point of ≤2 correct answers achieved a sensitivity of 40.00% (95% CI: 26.41%–54.82%) and a specificity of 89.21% (95% CI: 87.12%–91.08%) in screening for COVID-19 (Table 3 ). Overall accuracy was 86.85% (95% CI: 84.65%–88.85%) with an OR of 5.51 (95% CI: 3.03–10.05; p < 0.001) (Table 3). The PPV was 15.75% (95% CI: 11.30%–21.53%), and NPV was 96.72% (95% CI: 95.92%–97.37%) (Table 3). LR+ was 3.71 (95% CI: 2.53–5.44) and LR- was 0.67 (95% CI: 0.54–0.84) (Table 3). Area under the ROC curve was 0.684 (Fig. 2 ; Table 2).
Table 3.
Statistics for asymptomatic and symptomatic individuals within the general population COVID-19 testing site at a cut-off at ≤2 odors correct on olfactory test.
| Total general COVID-19 testing (n = 1042) |
Asymptomatic individuals (n = 706) | Symptomatic individuals (n = 313) |
|
|---|---|---|---|
| Sensitivity | 40.00% (26.41% to 54.82%) |
38.10% (18.11% to 61.56%) |
41.38% (23.52% to 61.06%) |
| Specificity | 89.21% (87.12% to 91.08%) |
92.26% (89.12% to 93.44%) |
82.41% (77.68% to 86.50%) |
| Positive likelihood ratio | 3.71 (2.53 to 5.44) |
4.92 (2.69 to 9.00) |
2.35 (1.43 to 3.86) |
| Negative likelihood ratio | 0.67 (0.54 to 0.84) |
0.67 (0.48 to 0.94) |
0.71 (0.52 to 0.97) |
| Disease prevalence | 4.80% (3.58% to 6.28%) |
2.97% (1.85% to 4.51%) |
8.63% (5.86% to 12.16%) |
| Positive predictive value | 15.75% (11.30% to 21.53%) |
13.11% (7.63% to 21.63%) |
18.18% (11.92% to 26.74%) |
| Negative predictive value | 96.72% (95.92% to 97.37%) |
97.98% (97.20% to 98.55%) |
93.70% (91.61% to 95.30%) |
| Accuracy | 86.85% (84.65% to 88.85%) |
90.65% (88.26% to 92.70%) |
78.87% (74.11% to 83.11%) |
| Odds ratio | 5.5140 (3.0252 to 10.0504) |
7.3382 (2.9199 to 18.4923) |
3.3072 (1.4931 to 7.3255) |
| Odds ratio P-value |
<0.001 | <0.001 | 0.003 |
(Asymptomatic individuals are those who reported no loss of smell/taste or runny/stuffy nose).
(Symptomatic individuals are those who reported loss of smell/taste and/or runny/stuffy nose)
Interval represents 95% CI.
Bold indicates statistically significant P-value.
Fig. 2.
ROC curve for general population COVID-19 testing and routine HCW testing population.
Among asymptomatic for loss of smell/taste or runny/stuffy nose individuals in this group (n = 729), a cutoff point of ≤2 correct answers resulted in a sensitivity of 38.10% (95% CI: 18.11%–61.56%) and specificity of 92.26% (95% CI: 89.12%–93.44%), accuracy of 90.65% (95% CI: 88.26%–92.70%), LR+ of 4.92 (95% CI: 2.69–9.00), LR- of 0.67 (95% CI: 0.48–0.94), PPV of 13.11% (95% CI: 7.63%–21.63%) NPV of 97.98% (95% CI: 97.20%–98.55%), and OR of 7.34 (95% CI: 2.92–18.49; p < 0.001; Table 3). Area under the ROC curve was 0.612 (Fig. 2; Table 2). Although all individuals in this group had no nasal obstruction/runny nose, or self-perception of loss of smell, when using an olfactory test with an olfactory dysfunction cutoff of ≤2, almost 40% actually had olfactory loss that was associated with being positive for SARS CoV-2. These data fit with other reports that up to half of individuals may not notice their olfactory loss [20].
Among symptomatic individuals in this group (those who reported new loss of either taste/smell, and/or stuffy/runny nose; n = 313), a cutoff point of ≤2 correct answers resulted in a sensitivity of 41.38% (95% CI: 23.52%–61.06%), specificity of 82.41% (95% CI: 77.68%–86.50%), accuracy of 78.87% (95% CI: 74.11%–83.11%), LR+ of 2.35 (95% CI: 1.43–3.86), LR- of 0.71 (95% CI: 0.52–0.97), PPV of 18.18% (95% CI: 11.92%–26.74%), NPV of 93.70% (95% CI: 91.61%–95.30%), and OR of 3.31 (95% CI: 1.49–7.33; p = 0.003) (Table 3). Area under the ROC curve was 0.708 (Fig. 2; Table 2).
3.3. Predicting false positives in the general population COVID-19 testing site
To identify what factors contribute to a false positive test result, (COVID-19 PCR-, but positive on the odorant test) univariate and multivariate logistical regression models were created that predicted false positives (relative to true negatives). Univariate analysis showed that relative to age 17–41, age >68 was associated with false positivity (OR: 3.79 [95% CI: 2.40–5.98]; p < 0.001), as was self reported loss of smell/taste (OR: 3.14 [95% CI: 1.64–5.99]; p = 0.001). After adjusting for patient-reported symptoms, both self-reporting a loss of smell/taste (OR 5.99 [95% CI 3.20–11.22]; p < 0.001) and self-reporting a runny or stuff nose (OR 1.74 [95% CI 1.09–2.78]; p = 0.021) were associated with false positivity (Table 4 ). Age >68 remained associated with false positivity (OR 4.41 [95% CI 2.52–7.70]; p < 0.001). False positive rate for the self-reported symptoms was 0.407 and 0.172, respectively.
Table 4.
Univariate and multivariate logistical regression predicting false positive smell test among COVID PCR- participants.
| Variable | Univariate |
Multivariate |
|||
|---|---|---|---|---|---|
| Odds ratio (95% CI) | P-value | Odds ratio (95% CI) | P-value | ||
| Age | 17–41 | 1 | Reference | 1 | Reference |
| 42–67 | 1.27 (0.97–1.67) |
0.079 | 1.38 (0.84–2.27) |
0.202 | |
| 68–93 |
3.79 (2.40 to 5.98) |
<0.001 |
4.41 (2.52–7.70) |
<0.001 | |
| Sex | Female | 1 | Reference | 1 | Reference |
| Male | 0.78 (0.60–1.00) |
0.053 | 1.36 (0.87–2.08) |
0.160 | |
| Loss of smell or taste | No | 1 | Reference | 1 | Reference |
| Yes |
3.14 (1.64–5.99) |
0.001 |
5.99 (3.20–11.22) |
<0.001 | |
| Runny or stuffy nose | No | 1 | Reference | 1 | Reference |
| Yes | 1.18 (0.89–1.56) |
0.257 |
1.74 (1.09–2.78) |
0.021 | |
Bold indicates statistically significant P-value.
3.4. Odorant test utility in routine HCW testing population characteristics
There were only 2 (0.72%) PCR+ individuals in the routine HCW testing population (n = 278) (Supplemental Table 1). One subject, who got 1/5 correct, reported no symptoms and the other, who got 5/5 correct, only reported stuffy/runny nose (Supplemental Table 1). Neither reported a loss of smell of in the questionnaire. Data on particular cutoffs is provided in Supplemental Table 2, although, due to the small number of positives in this group, we were unable to determine a meaningful cutoff.
4. Discussion
Large-scale deployable odorant tests remain relatively experimental in their ability to detect COVID-19 infection. A study in Brazil by Lessa et al. using a U-smell-It™ test showed that assessment of olfactory function could be useful in identifying COVID-19 infection in symptomatic individuals, thus potentially as a rapid and cost-effective screening tool for primary infection [21]. The sensitivity of an olfactory test is considerably lower than that of molecular tests, however lower sensitivity can be compensated for by serial testing [21], [22]. The actual optimal cutoff will depend on the context of screening. If a secondary reflux test is available, higher sensitivity thresholds can be used, at the cost of decreased specificity. It is worth noting that while about 40% of individuals that had no self-reported loss of smell or any kind of nasal obstruction (stuffy or runny nose) actually had, when tested with an olfactory test, an olfactory dysfunction (sensitivity ~38%; cutoff ≤2). Key benefits of our study are: i) inclusion of COVID-19 PCR- individuals, which is critical for balancing sensitivity and specificity, and ii) a very large total sample size (over one thousand total individuals).
In the general population COVID-19 testing site, there was a significantly greater proportion of individuals in the PCR+ cohort that reported new loss of either taste or smell (12 [24.0%] vs. 59 [5.9%]; p < 0.001), and stuffy/runny nose (25 [50.0%] vs. 274 [27.6%]; p = 0.001) when compared to COVID-19 negative cohort. A previous study found smell or taste change to be a strong predictor for a COVID-19 positive test result, thus detection of olfactory dysfunction can be an indicator of COVID-19 infection [4]. Though stuffy/runny nose has only been anecdotally reported as a symptom in COVID-19 positive patients [23], [24], [25], our results indicate that stuffy/runny nose may be a weak indicator for COVID-19 infection, and warrants further evaluation.
At a cutoff point of ≤2 correct, we were unable to reach a >80% sensitivity in the general population COVID-19 testing site, which indicates that this smell test has a moderate ability to discriminate between PCR+ and PCR- patients. However, when the individuals became positive is not known and at the time the population was examined (September–October 2020), the recent surge had passed and cases were dropping – thus it is conceivable that a fraction of people were tested well post their exposure. As raised elegantly by others [26], a PCR tests identifies for the presence of genetic material, but does not identify if the person is still infective. In general, infectivity lasts about 10 days while an individual can be PCR positive for ~30 days. Thus, it is likely that some of the individuals were PCR positive but no longer infectious. As the duration of loss of smell is known to be ~7–14 day [21] it would be expected that some people will recover from anosmia/hyposmia but still be PCR positive. Thus it should be taken that PCR is a reference test and not a gold standard for what matters, which is SARS CoV-2 infectiousness.
In an attempt to find specific cohorts where the test might be more useful, we split the general population COVID-19 testing site into symptomatic and asymptomatic groups (based on loss of smell/taste or runny/stuffy nose). Though results were similar when using a cutoff of ≤2, the sensitivity was slightly higher with the symptomatic group (41.4% vs 38.1% aymptomatic) but specificity was 10% higher with asymptomatic for loss of smell/taste or runny/stuffy nose group (92.3% vs 82.4% symptomatic). This likely reflects that there is some non-specific olfactory dysfunction with a blocked or stuffy nose. Thus, when high specificity is paramount, individuals with a stuffy nose may be excluded at some cost of sensitivity.
Though higher cutoffs such as ≤4 correct were able to achieve sensitivity >80% in the symptomatic group (85.2%), specificity was greatly lower (39.2%), which we found to be associated with increasing age (>67). This raises the possibility that this test might perform better among a cohort of younger individuals, who comprise a greater proportion of the workforce and resource-deficient countries, increasing support for use in these settings. Lower cut-off points, such as ≤2 questions correct, resulted in greater specificity (89.2%), accuracy (86.9%), and OR (5.5), though sensitivity was much lower (40.0%). Though the performance of a 10-odor test was only slightly better than a 5-odor test, as implicated by Lessa et al. [21], there is a possibility that using a lower cut-off point in conjunction with serial testing could help alleviate the reduced sensitivity of a lower cut-off. It has been shown that a low-sensitivity and high-specificity test when used in a high frequency manner was suitable for mitigating outbreaks, especially if the test cost was minimal [22].
HCWs are an interesting and useful population as all reported no COVID-19 related symptoms as part of their daily surveys. Interestingly, one of two COVID-19 positive subjects had severe hyposmia (score of 1/5), although they did not recognize their hyposmia. These data do fit with other reports that asymptomatic COVID-19 positive people can have unrecognized mild-to-severe hyposmia [27]. The weakness of this arm of the study was the low prevalence, exemplifying the challenge in testing asymptomatic people–it will require a large sample size (several thousands) to be statistically robust.
One current rapid mainstream method of screening patients for COVID-19 has been with body temperature measurements of fever (>38 °C) although it has shown very low performance in detecting infection [21]. Lessa et al. reported that 0 of the 44 COVID-19 PCR+ patients in their study had fever [21]. Comparatively, odorant testing is quick (roughly under a minute), easily administrable, and low cost, it may serve as an intermediate for resource allocation decisions in resource deficient areas.
Parma, et al. developed the SCENTinel 1.0 to rapidly evaluate olfactory detection, intensity, and identification, hypothesizing that this could be used as a screening tool for COVID-19 infection [28]. Due to low numerosity, the study was unable to test this endpoint with a reasonable amount of certainty [28]. Though anosmia and olfactory dysfunction have been reported as symptoms of COVID-19 infection [29], [3], our results seem to suggest that an odor identification test has only moderate sensitivity as a single timepoint screening mechanism, although it could be an addition to the Swiss Cheese Model to provide a multilayer mechanism for pandemic defense [30].
Limitations of this study include the self-reporting nature of symptoms, which put the study at risk for recall bias with regards to differentiation of asymptomatic and symptomatic individuals. Only adults were tested and how the results translate to children is unclear, although other reports indicate that ~86% of COVID-19+ children may have hyposmia [31]. Additionally, COVID-19 testing and administration of the odorant test may have been variable across the testing locations. Due to the low prevalence of COVID-19 in the HCW population, this odorant test requires further longitudinal studies and analysis to determine its viability for use in routine screening for HCWs. This was a non-longitudinal study, thus the onset and duration of symptoms were not captured. Sensitivity on a single day of testing does not necessarily equal prevalence of olfactory dysfunction; i.e. if a person was negative one day for olfactory dysfunction, but then turned positive the next day, the prevalence would be higher than the sensitivity based on one day of testing. Furthermore, olfactory testing has its own inherent limitations that may have hindered its effective use.
5. Conclusion
As the pandemic matures and resources devoted to detecting COVID-19 de-intensify, there remains a need for an effective tool to detect infection, especially in resource-deficient settings or those with a low degree of vaccine immunization. Our results suggest that odorant testing to detect olfactory dysfunction may be a viable option in detecting COVID-19 infection and could potentially be an addition to the Swiss Cheese Model to provide a multilayer mechanism for pandemic defense. Though further investigation in longitudinal studies is warranted to observe the extent to which odorant testing could be used to supplement traditional clinical judgement and the potential benefits of serial testing, our preliminary analysis suggests moderate discriminatory ability.
Funding support
The authors declare that there is no funding support.
Presentations
None.
Declaration of competing interest
The authors declare that there is no conflict of interest.
Acknowledgements
The authors would like to thank U-Smell-It™ LLC for kindly providing the olfactory tests. Olfactory tests were provided at no cost by the manufacturer (U-Smell-It LLC). The manufacturer was sponsored through prevision of the test material and mailed it to the test location. No members of the company were involved in the actual study (data collection, analysis, etc.). Additionally, we would like to thank Dr. Basemah Gheith, Dr. David Holcombe, and Dr. Joseph Gannett for their assistance in acquiring data.
Footnotes
Supplementary data to this article can be found online at https://doi.org/10.1016/j.amjoto.2022.103376.
Appendix A. Supplementary data
Routine HCW COVID-19 Testing Population Characteristics, ROC Analysis, and Statistics at a Cut-Off at ≤1 Questions Correct.
References
- 1.Shuren Jeffrey. United States Food and Drug Administration; 2021. Coronavirus (COVID-19) update: FDA takes steps to streamline path for COVID-19 screening tools, provides information to help groups establishing testing programs.https://www.fda.gov/news-events/press-announcements/coronavirus-covid-19-update-fda-takes-steps-streamline-path-covid-19-screening-tools-provides Published March 16. [Google Scholar]
- 2.Ward S., Lindsley A., Courter J., Assa’ad A. Clinical testing for COVID-19. J Allergy Clin Immunol. 2020;146(1):23–34. doi: 10.1016/j.jaci.2020.05.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Moein S.T., Hashemian S.M., Mansourafshar B., Khorram-Tousi A., Tabarsi P., Doty R.L. Smell dysfunction: a biomarker for COVID-19. Int Forum Allergy Rhinol. 2020;10(8):944–950. doi: 10.1002/alr.22587. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Roland L.T., Gurrola J.G., Loftus P.A., Cheung S.W., Chang J.L. Smell and taste symptom-based predictive model for COVID-19 diagnosis. Int Forum Allergy Rhinol. 2020;10(7):832–838. doi: 10.1002/alr.22602. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Doty R.L., Marcus A., Lee W.W. Development of the 12-item cross-cultural smell identification test (CC-SIT) Laryngoscope. 1996;106(3 Pt 1):353–356. doi: 10.1097/00005537-199603000-00021. [DOI] [PubMed] [Google Scholar]
- 6.Doty R.L., Shaman P., Kimmelman C.P., Dann M.S. University of Pennsylvania Smell Identification Test: a rapid quantitative olfactory function test for the clinic. Laryngoscope. 1984;94(2 Pt 1):176–178. doi: 10.1288/00005537-198402000-00004. [DOI] [PubMed] [Google Scholar]
- 7.Hummel T., Sekinger B., Wolf S.R., Pauli E., Kobal G. “Sniffin” sticks’: olfactory performance assessed by the combined testing of odor identification, odor discrimination and olfactory threshold. Chem Senses. 1997;22(1):39–52. doi: 10.1093/chemse/22.1.39. [DOI] [PubMed] [Google Scholar]
- 8.Choudhury E.S. Influences of age and sex on a microencapsulated odor memory test. Chem Senses. 2003;28(9):799–805. doi: 10.1093/chemse/bjg072. [DOI] [PubMed] [Google Scholar]
- 9.Jackman A.H., Doty R.L. Utility of a three-item smell identification test in detecting olfactory dysfunction. Laryngoscope. 2005;115(12):2209–2212. doi: 10.1097/01.mlg.0000183194.17484.bb. [DOI] [PubMed] [Google Scholar]
- 10.Dalton P., Doty R.L., Murphy C., et al. Olfactory assessment using the NIH toolbox. Neurology. 2013;80(11 Suppl 3):S32–S36. doi: 10.1212/WNL.0b013e3182872eb4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Rawal S., Hoffman H.J., Honda M., Huedo-Medin T.B., Duffy V.B. The taste and smell protocol in the 2011–2014 US National Health and nutrition examination survey (NHANES): test-retest reliability and validity testing. Chemosens Percept. 2015;8(3):138–148. doi: 10.1007/s12078-015-9194-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Liu D.T., Besser G., Lang M., et al. Odor mixtures in identification testing using sniffin’ sticks: the SSomix test. Sci Rep. 2020;10(1):8155. doi: 10.1038/s41598-020-65028-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Duff K., McCaffrey R.J., Solomon G.S. The pocket smell test: successfully discriminating probable Alzheimer’s dementia from vascular dementia and major depression. J Neuropsychiatry Clin Neurosci. 2002;14(2):197–201. doi: 10.1176/jnp.14.2.197. [DOI] [PubMed] [Google Scholar]
- 14.Croy I., Zehner C., Larsson M., Zucco G.M., Hummel T. Test-retest reliability and validity of the sniffin’ TOM odor memory test. Chem Senses. 2015;40(3):173–179. doi: 10.1093/chemse/bju069. [DOI] [PubMed] [Google Scholar]
- 15.Kondo H., Matsuda T., Hashiba M., Baba S. A study of the relationship between the T&T olfactometer and the University of Pennsylvania Smell Identification Test in a Japanese population. Am J Rhinol. 1998;12(5):353–358. doi: 10.2500/105065898780182390. [DOI] [PubMed] [Google Scholar]
- 16.Eng John. Johns Hopkins University; 2014. ROC analysis: web-based calculator for ROC curves.http://www.jrocfit.org Published March 19. [Google Scholar]
- 17.Li C., Chen J., Qin G. Partial Youden index and its inferences. J Biopharm Stat. 2019;29(2):385–399. doi: 10.1080/10543406.2018.1535502. [DOI] [PubMed] [Google Scholar]
- 18.MedCalc Software Ltd . Diagnostic test evaluation calculator. MedCalc.
- 19.MedCalc Software Ltd . Odds ratio calculator. MedCalc.
- 20.Agyeman A.A., Chin K.L., Landersdorfer C.B., Liew D., Ofori-Asenso R. Smell and taste dysfunction in patients with COVID-19: a systematic review and meta-analysis. Mayo Clin Proc. 2020;95(8):1621–1631. doi: 10.1016/j.mayocp.2020.05.030. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Lessa M.A., Cotta-Pereira S.M., Ferreira F.A., et al. 2021. The Usefulness of a Quantitative Olfactory Test for the Detection of COVID-19. Infectious Diseases (except HIV/AIDS) [DOI] [Google Scholar]
- 22.Paltiel A.D., Zheng A., Walensky R.P. COVID-19 screening strategies that permit the safe re-opening of college campuses. MedRxiv Prepr Serv Health Sci. 2020 doi: 10.1101/2020.07.06.20147702. Published online July 7. [DOI] [Google Scholar]
- 23.Biadsee A., Biadsee A., Kassem F., Dagan O., Masarwa S., Ormianer Z. Olfactory and oral manifestations of COVID-19: sex-related symptoms-a potential pathway to early diagnosis. Otolaryngol Head Neck Surg. 2020;163(4):722–728. doi: 10.1177/0194599820934380. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Sayin İ., Yaşar K.K., Yazici Z.M. Taste and smell impairment in COVID-19: an AAO-HNS anosmia reporting tool-based comparative study. OtolaryngolHead Neck Surg. 2020;163(3):473–479. doi: 10.1177/0194599820931820. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Speth M.M., Singer-Cornelius T., Oberle M., Gengler I., Brockmeier S.J., Sedaghat A.R. Olfactory dysfunction and sinonasal symptomatology in COVID-19: prevalence, severity, timing, and associated characteristics. OtolaryngolHead Neck Surg. 2020;163(1):114–120. doi: 10.1177/0194599820929185. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Mina M.J., Parker R., Larremore D.B. Rethinking Covid-19 test sensitivity — a strategy for containment. N Engl J Med. 2020;383(22) doi: 10.1056/NEJMp2025631. [DOI] [PubMed] [Google Scholar]
- 27.Bhattacharjee A.S., Joshi S.V., Naik S., Sangle S., Abraham N.M. Quantitative assessment of olfactory dysfunction accurately detects asymptomatic COVID-19 carriers. EClinicalMedicine. 2020;28 doi: 10.1016/j.eclinm.2020.100575. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Parma V., Hannum M.E., O’Leary M., et al. SCENTinel 1.0: development of a rapid test to screen for smell loss. Chem Senses. 2021 doi: 10.1093/chemse/bjab012. Published online March 27. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.World Health Organization (WHO) World Health Organization (WHO); 2020. Coronavirus Disease (COVID-19)https://www.who.int/emergencies/diseases/novel-coronavirus-2019/question-and-answers-hub/q-a-detail/coronavirus-disease-covid-19 Published November 10. [Google Scholar]
- 30.Noh J.Y., Song J.Y., Yoon J.G., Seong H., Cheong H.J., Kim W.J. Safe hospital preparedness in the era of COVID-19: the Swiss cheese model. Int J Infect Dis. 2020;98:294–296. doi: 10.1016/j.ijid.2020.06.094. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Rusetsky Y., Meytel I., Mokoyan Z., Fisenko A., Babayan A., Malyavina U. Smell status in children infected with SARS-CoV-2. The Laryngoscope. 2021 doi: 10.1002/lary.29403. Published online January 14. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Routine HCW COVID-19 Testing Population Characteristics, ROC Analysis, and Statistics at a Cut-Off at ≤1 Questions Correct.