The best COVID-19 predictor is recent smell loss: a cross-sectional study

Abstract

Background:

COVID-19 has heterogeneous manifestations, though one of the most common symptoms is a sudden loss of smell (anosmia or hyposmia). We investigated whether olfactory loss is a reliable predictor of COVID-19.

Methods:

This preregistered, cross-sectional study used a crowdsourced questionnaire in 23 languages to assess symptoms in individuals self-reporting recent respiratory illness. We quantified changes in chemosensory abilities during the course of the respiratory illness using 0-100 visual analog scales (VAS) for participants reporting a positive (C19+; n=4148) or negative (C19−; n=546) COVID-19 laboratory test outcome. Logistic regression models identified singular and cumulative predictors of COVID-19 status and post-COVID-19 olfactory recovery.

Results:

Both C19+ and C19− groups exhibited smell loss, but it was significantly larger in C19+ participants (mean±SD, C19+: −82.5±27.2 points; C19−: −59.8±37.7). Smell loss during illness was the best predictor of COVID-19 in both single and cumulative feature models (ROC AUC=0.72), with additional features providing negligible model improvement. VAS ratings of smell loss were more predictive than binary chemosensory yes/no-questions or other cardinal symptoms, such as fever or cough. Olfactory recovery within 40 days was reported for ~50% of participants and was best predicted by time since illness onset.

Conclusions:

As smell loss is the best predictor of COVID-19, we developed the ODoR-19 tool, a 0-10 scale to screen for recent olfactory loss. Numeric ratings ≤2 indicate high odds of symptomatic COVID-19 (4<OR<10), which can be deployed when viral lab tests are impractical or unavailable.

Introduction

The novel coronavirus SARS-CoV-2 responsible for the global COVID-19 pandemic has left a staggering level of morbidity, mortality, and societal and economic disruption in its wake.¹ Early publications^2-8 indicate that sudden smell and taste loss are cardinal, early and potentially specific symptoms of COVID-19,⁹ including in otherwise asymptomatic individuals.^10-13 While fever and cough are common symptoms of diverse viral infections, the potential specificity of early chemosensory loss to COVID-19 could make it valuable in screening and diagnosis.

Anosmia and other chemosensory disorders have serious health and quality-of-life consequences for patients. However, the general lack of awareness of anosmia and other chemosensory disorders by clinicians and the public, including their association with upper respiratory infections,¹⁴ contributed to an underappreciated role of chemosensory symptoms in the diagnosis of COVID-19. Additionally, the impact of smell loss as a clinical consequence of COVID-19 has not been adequately addressed. Thus, there is an urgent need to better define the chemosensory dysfunctions associated with COVID-19 and to determine their relevance as predictors of this disease. It is critical to develop rapid clinical tools to efficiently and effectively integrate chemosensory assessments into COVID-19 screening and treatment protocols. Information on the duration and reversibility of post-COVID-19 chemosensory impairment is also lacking.

We used binary, categorical and continuous self-report measures to determine the chemosensory phenotype, along with other symptoms and characteristics, of COVID-19-positive (C19+) and COVID-19-negative (C19−) individuals who had reported recent symptoms of respiratory illness. Using those results in logistic regression models, we identified predictors of COVID-19 and recovery from smell loss. Finally, we propose the Olfactory Determination Rating scale for COVID-19 (ODoR-19), a quick, simple-to-use, telemedicine-friendly tool to improve the utility of current COVID-19 screening protocols, particularly when access to rapid testing for SARS-CoV-2 is limited.

Methods

Study design

This preregistered,¹⁵ cross-sectional online study was approved by the Office of Research Protections of The Pennsylvania State University (STUDY00014904); it is in accordance with the revised Declaration of Helsinki, and compliant with privacy laws in the U.S.A. and European Union. Data reported here were collected between April 7 and July 3, 2020 from the Global Consortium for Chemosensory Research (GCCR) core questionnaire (Appendix 1 and https://gcchemosensr.org),⁸ an online crowdsourced survey deployed in 32 languages that used binary response and categorical questions (e.g. Appendix 1, Questions 6,9) and visual analog scales (e.g., Appendix 1, Question 13) to measure self-reported chemosensory ability and other symptoms in adults with current or recent respiratory illness. Data reported here include responses in Arabic, Bengali, Chinese (Simplified and Traditional), Danish, Dutch, English, Farsi, Finnish, French, German, Greek, Hebrew, Hindi, Italian, Japanese, Korean, Norwegian, Portuguese, Russian, Spanish, Swedish, Turkish, and Urdu.

Participants (272)

A convenience sample of 52,334 volunteers accessed the GCCR questionnaire; 25,620 met eligibility criteria (≥19 years old, respiratory illness or suspicion thereof within the past two weeks). After applying pre-registered exclusion criteria, 15,747 participants were included in reported analyses (Figure 1).

Figure 1.

Flow diagram showing the demographics of participants included and excluded in the present analyses. Participants included in the prediction of COVID-19 status are highlighted in blue. Participants included in the smell recovery models are highlighted in green. Participants included in the replication of our prior work are highlighted in orange. N = number of participants; yo = age in years; W = women; M = men. Gender percentages do not include <1% of participants who answered “other” or “preferred not to say”. Participants described in the green boxes are a subset of those described in the boxes listed above them.

Based on the self-reported outcome of a COVID-19 lab test, participants were labeled as either C19+ (positive result) or C19− (negative result). The specific collider bias characterizing this sample (high fraction of C19+ participants and high prevalence of chemosensory disorders in both groups) underestimates the positive correlation between smell loss and COVID-19 (Figure S1). Thus, it represents a conservative scenario to test the hypothesis that smell loss reliably predicts COVID-19 status. We benchmarked the GCCR dataset to the representative samples collected with the Imperial College London YouGov Covid 19 Behaviour Tracker (henceforth, YouGov; countries shared across datasets: Brazil, Canada, Denmark, Finland, France, Germany, Italy, Mexico, Netherlands, Norway, Spain, Sweden, UK, USA; YouGov: N=8,674, GCCR: N=3,962; data publicly available at https://github.com/YouGov-Data/covid-19-tracker). Benchmarking shows the GCCR sample underestimates the positive association between smell loss and C19+ (Figure S1, Table S1). The country-wise fraction of C19+ participants is correlated (r~0.45) when responses from the same calendar week are aligned (Figure S2). These findings are in line with other comparisons between crowdsourced versus representative health data,¹⁶ confirming that trends identified in crowdsourced data reasonably approximate population data. Because the GCCR cohort is not demographically balanced, it should not be used to estimate prevalence. However, the representative YouGov cohort indicates globally ~33% of C19+ individuals report smell loss (Table S1).

Statistical analyses

Statistical analyses were performed in Python 3.7.6 using the pandas,¹⁷ scikit-learn,¹⁸ and statsmodels¹⁹ packages. The data and annotated code is included as supplemental material and will be publicly available on GitHub (http://github.com/GCCR/GCCR002) upon publication. Missing values for covariates used in prediction models of COVID-19 status and smell recovery were imputed as follows: binary features = 0.5, numeric features = median, categorical variables = “Missing”. Prediction targets themselves were never imputed. Responses incompatible with model generalization (e.g., open ended questions) were excluded. A one-hot encoding was applied to all categorical variables to produce binary indicators of category membership. L1-regularized logistic regression (penalty α=1) consistently produced sparse models with comparable cross-validation accuracy and were therefore the prediction test of choice. Model quality was measured using receiver operating characteristic (ROC) area under the curve (AUC). Cross-validation was performed in 100 random splits of 80% training set and 20% test set, and ROC curves are concatenated over each test set. ROC curves are computed on predicted probabilities from each model, circumventing the high-cardinality bias of AUC. For single feature models, AUC is independent of most modeling details, including all rank-invariant decisions. To correctly compute p-values for model coefficients, the normalized data were standardized (mean 0, variance 1) and then coefficients back-transformed to normalized form after fitting.

Results

Chemosensory loss associates with COVID-19

A preregistered replication of our prior study⁸ confirmed that reported smell, taste, and chemesthesis abilities drop significantly in both lab-tested C19+ participants and those diagnosed by clinical assessment (Figure S3, Table S2).

Next, we compared lab-tested C19+ and C19− participants. C19+ participants reported a greater loss of smell (C19+: −82.5±27.2 points; C19−: −59.8±37.7 points; p=2.2e-46, extreme evidence of difference: BF₁₀=8.97e+61; Figure 2A,B; Table S3), taste (C19+: −71.6±31.8 points; C19−: −55.2±37.5 points; p=6.7e-26, extreme evidence of difference: BF₁₀=6.67e+24; Figure 2C,D; Table S3) and chemesthesis ability (C19+: −36.8±37.1 points; C19−: −28.7±37.1 points; p=1.6e-07, extreme evidence of difference: BF₁₀=3182; Figure 2E,F; Table S3). However, both groups reported a similar degree of nasal obstruction (Figure 2G,H; Table S3). Self-reported changes in smell, taste, and chemesthesis were highly correlated within both groups (C19+: 0.71<r<0.83; C19−: 0.76<r<0.87) and orthogonal to nasal obstruction changes (C19+: r=−0.20; C19−: r=−0.13).

Figure 2.

Chemosensory ability and nasal obstruction in C19+ and C19− participants. Self-reported smell (A,B), taste (C,D), chemesthesis (E,F), and nasal obstruction (G,H; formulated as “How blocked was your nose?”) before and during respiratory illness in C19+ (darker shades) and C19− (lighter shades) participants. Ratings were given on 0-100 visual analog scales. Left panels (A,C,E,G) show mean values. Right panels (B,D,F,H) show distributions of the change scores (during minus before). Thicker sections indicate relatively more subjects (higher density of responses). The thick black horizontal bar indicates the median, the shaded area within each violin indicates the interquartile range. Each dot represents the rating of a single participant.

Prediction of COVID-19 status from survey responses

Binary (yes/no) and categorical reported symptoms suggested that COVID-19 is more strongly associated with chemosensory than with non-chemosensory symptoms, including fever, cough, and shortness of breath, cardinal symptoms currently highlighted by the US Centers for Disease Control and Prevention (CDC)(Figure 3A). Using AUC to assess prediction quality (Figure 3B), we found that self-reported smell ability during illness, reported on a continuous scale, was the most predictive survey question for COVID-19 status(AUC=0.71). Changes in smell as a result of illness (the difference between smell ability during and before illness) was similarly predictive (AUC=0.69). Changes in taste ability (assessed via rating) were the next most predictive features (AUC=0.64-0.65) (Figure 3B). Models fit to the same data but with shuffled COVID-19 status consistently produced AUC~0.5 for all features. The most predictive non-chemosensory symptom, sore throat (which was negatively associated with COVID-19) was substantially less predictive (AUC=0.58) than the top chemosensory symptoms. Nasal obstruction was not predictive (AUC=0.52). Responses given on a continuous scale were more predictive (AUC=0.71) than binary responses to parallel questions (e.g., Appendix 1, Question 13 versus 14, FigureS5) (AUC=0.60-0.62), likely because a continuous scale contains a greater amount of diagnostic information (Figure S4).

Figure 3.

Smell loss is the strongest predictor of COVID-19 status. (A) A normalized measure of association (Cramer’s V) between binary or categorical responses on COVID-19 status. V=0 reflects no association between the response and COVID-19 status; V=1 reflects a perfect association; V>0.1 is considered a meaningful association. Features in red are positively associated with C19+ (odds ratio > 1); features in blue are negatively associated with C19+ (odds ratio < 1). (B) Logistic regression is used to predict COVID-19 status from individual features. Top-10 single features are ranked by performance (cross-validated area under the ROC curve, AUC). Chemosensory-related features (bold) show greater predictive accuracy than non-chemosensory features (non-bold). Responses provided on the numeric scale (italic) were more informative than binary responses (non-italic). Red arrows indicate differences in prediction quality (in AUC) between features. (C) Adding features to “Smell During Illness” results in little improvement to the model; only Days Since Onset of Respiratory Symptoms (DOS) yields meaningful improvement. (D) ROC curves for several models. A model using “Smell during illness” (Smell Only, abbreviated “Smell” in figure) is compared against models containing this feature along with DOS, as well as models including the three cardinal CDC features (fever, dry cough, difficulty breathing). “Full” indicates a regularized model fit using 70 dozen survey features, which achieves prediction accuracy similar to the parsimonious model “Smell Only+DOS”.

Next, we examined which simple multi-feature model would best predict COVID-19 status. As some questions have highly correlated responses, the question most complementary to “Smell during illness” is unlikely to be one that carries redundant information. Adding “Days since onset of respiratory symptoms” (DOS) to “Smell during illness” (Smell Only) produced the largest incremental gain in predictive performance (AUC=0.72, +0.01 versus the Smell Only model) (Figure 3C).

We directly compared the Smell Only+DOS model to other candidate models. The Smell Only+DOS model (Figure 3D) yielded an equal or higher AUC than the model including the three cardinal CDC symptoms (AUC=0.55) or the full model using 70 features (AUC=0.72). Because the Smell Only+DOS model exhibits the same AUC as the full model it strikes a good balance between model parsimony and predictive accuracy for C19+. However, the Smell Only model also offers reasonable sensitivity of 0.85 (at specificity=0.51, cutoff=13 on the 100-point VAS) and/or specificity of 0.75 (at sensitivity=0.51, cutoff=1) as desired. By sharp contrast, fever has a sensitivity of only 0.54 with specificity of 0.49 and dry cough has sensitivity of 0.52 and specificity of 0.46.

Recovery from smell loss

Recovery from smell loss was modest (approximately half the initial average loss) in C19+ participants with full or partial resolution of respiratory symptoms. Overall, self-reported, post-illness olfactory ability was still lower for C19+ (39.9±34.7) than C19− (52.2±35.2, p=2.8e-11, Figure S6A). However, the mean recovery of smell (after illness relative to during illness) was greater for C19+ (30.5±35.7) than C19− (24.6±31.9, p=0.0002, Figure S6B). A similar but smaller effect of COVID-19 status on recovery was observed for taste (Figure S6C, D), while little to no association with COVID-19 was observed for recovery of chemesthesis (Figure S6E,F) or nasal obstruction (Figure S6G,H). When illness-induced change in olfactory function (during minus before illness) and recovery of olfactory function (after minus during illness) were evaluated, we identified three respondent clusters: those self-reporting no loss of smell (Intact Smell), those reporting recovery from smell loss (Recovered Smell), and those reporting smell loss without recovery by up to 40 days (Persistent Smell Loss, Figure 4, Table S3). Intact smell was reported by only 8.5% of the participants in the C19+ group but by 27.5% in the C19− group (p=3.8e-31). A greater proportion of C19+ participants were included in both the Recovered Smell group (C19+: 40.9%, C19−: 33.3%; p=4.9e-10) and the Persistent Smell Loss group (C19+: 50.7%, C19−: 39.2%; p=5e-5; Figure 4A, B). C19+ participants in both the Recovered Smell and Persistent Smell Loss clusters reported a similar extent of olfactory loss, irrespective of time since respiratory symptom onset. By contrast, the rate of self-reported smell recovery increased over time, with a plateau at 30 days (Figure 4C). Finally, DOS was the best predictor (AUC=0.62) between the Persistent Smell Loss and the Recovered Smell groups (Figure S6A, Table S3).

Figure 4:

Smell loss, recovery, and time course. (A, B) Joint distribution of smell loss (during minus before illness ratings) and smell recovery (after minus during illness ratings) for C19+ (A) and C19− (B) participants. Darker color indicates a higher probability density; the color map is shared between (A) and (B); dashed lines are placed at a third of the way across the rating scale to aid visualization of the clusters. Severe smell loss that is either persistent (lower left) or recovered (upper left) was more common in C19+ than C19−. n indicates the number of participants in each panel. % indicates the percentage of participants of the given COVID status in each quadrant. (C) In C19+ participants who lost their sense of smell (Recovered Smell + Persistent Smell Loss), the degree of smell recovery (right y axis) increased over ~30 days since onset of respiratory symptoms before plateauing; the degree of reported smell change (left y axis) did not vary in that window of observation. Solid lines indicate the mean of the measure, the shaded region indicates the 95% confidence interval.

Simple screening for COVID-19: the Olfactory Determination Rating scale in COVID-19 (ODoR-19) (136)

Our results indicate that a continuous rating of current olfactory function is the single best predictor of COVID-19 and improves the discrimination between C19+ and C19− over a binary question on smell loss. For example, the Smell Only model can reach a specificity of 0.83 at the low end of the VAS (sensitivity=0.36, cutoff=0). We propose here a numeric variation of the rating scale (0-10), the ODoR-19, that can be administered in person or via telemedicine to improve early COVID-19 screening for individuals without preexisting smell and/or taste disorders. Responses to the ODoR-19 scale ≤2 indicate high odds of COVID-19 positivity (4<OR<10, Figure 5D). An ODoR-19 response of 3 indicates a borderline risk (OR=1.2).

Figure 5.

The odds of a COVID-19 diagnosis as a function of olfactory loss. (A) The solid line indicates the probability of a COVID-19 diagnosis as a function of “Smell during illness” ratings. The shaded region indicates the 95% confidence interval. (B) The solid line expresses the probability of a COVID-19+ diagnosis as a function of “Smell during illness” in odds (p/(1-p)); it is shown on a logarithmic scale. The shaded region indicates the 95% confidence interval. (C) Stylized depiction of change in the odds of a COVID-19 diagnosis and of the odds ratio. (D) The ODoR-19 screening tool. After healthcare providers or contact tracers have excluded previous smell and/or taste disorders such as those resulting from head trauma, chronic rhinosinusitis, or previous viral illness, the patient can be asked to rate their current ability to smell on a scale from 0-10, with 0 being no sense of smell and 10 being excellent sense of smell. If the patient reports a value below or equal to 3, there is a high (red) or moderate (orange) probability that the patient has COVID-19. Values in yellow (ratings above 3) cannot rule out COVID-19.

Discussion

Self-reported smell loss was more common in C19+ than C19− participants, but present in both groups. The use of a VAS to assess olfactory loss better predicted COVID-19 status than using a binary question. We found that the best predictor of COVID-19-associated smell recovery, within the time frame captured by the survey (~40 days), was days since onset of COVID-19.

The SARS-CoV-2 pandemic requires healthcare providers and contact tracers to quickly and reliably assess an individual’s COVID-19 risk, often remotely. Thus, reliable screening tools are critical to assess a person’s likelihood of having COVID-19 and to justify self-quarantine and/or testing recommendations. Indeed, some reports suggest that COVID-19-associated smell loss might be an indicator of disease severity.^2,20 Current symptom criteria (e.g., fever, dry cough) are less specific than severe olfactory loss. Indeed, the value of our ODoR-19 tool lies in the high specificity of values ≤2 for indicating COVID-19 positivity, therefore representing a valuable addition to the current repertoire of COVID-19 screening tools. Those who receive a negative outcome from a COVID-19 viral test, yet report significant idiopathic smell loss, should be considered as high-priority candidates for COVID-19 re-testing.

Our online survey and sampling methodology likely selected participants with a heightened interest in smell and taste and/or their disturbances. This self-selection bias could be viewed as a limitation since the C19− group also showed chemosensory loss. However, finding difference between groups in a sample with a higher barrier for discriminating between C19+ and C19− supports the robustness of this tool when used in a typical clinical population; our collider bias analysis also suggests that our findings are likely conservative estimates (Figure S1, Table S1).

Our results suggest that chemosensory impairment has strong COVID-19 predictive value and is useful when access to viral testing is limited or absent. As with any self-report measure, veracity of self-reports cannot be guaranteed. However, the ability to screen individuals in real-time should outweigh this potential confound.²¹ While objective smell tests are the gold standard for assessing olfactory function,^22,23 they are costly, time-consuming to administer, and can require in-person interactions with potentially infectious patients.^23,24 By contrast, the ODoR-19 is free, quick, and can be administered in person or remotely. We cannot exclude that our C19− sample contains COVID-19 false negatives.²⁵ However, self-reported smell during illness distinguishes between C19+ and C19−, but not between randomly shuffled cases, suggesting that the difference between C19+ and C19−, even in a sample with over-represented chemosensory dysfunction, is substantial and can be captured via self-report.

Approximately half of the participants in the C19+ group recovered their sense of smell within 40 days from the onset of respiratory symptoms. This suggests the presence of at least two subgroups of patients: one that recovers quickly (<40 days, 40.9%) and another that may present a more variable time course of recovery (50.7%). Since these data are collected before the full recovery of all symptoms, we cannot offer a complete picture of recovery from olfactory loss in COVID-19-positive individuals, but they align with other early reports.²⁶ The COVID-19 pandemic will greatly increase the number of patients suffering from anosmia and other chemosensory disorders,²⁷ conditions that significantly affect quality-of-life,^28,29 dietary behavior,³⁰ cardiovascular health,³¹ and mental health.^32,33 Thus, it is necessary to prepare healthcare providers to address the long-term needs of these patients.

Based on our results, we propose the use of the ODoR-19 tool, a quick, free, and effective smell-based screening method for COVID-19. This 0-10 rating scale accurately predicts COVID-19 in individuals without pre-existing smell and taste disorders (e.g., from head trauma, chronic rhinosinusitis³⁴). ODoR-19 combines the utility of a continuous scale with the ease and speed needed for a screening tool. ODoR-19 is safe for remote administration during an illness with high viral spread and can precede and complement viral testing. This tool will improve screening for patients with limited or no access to medical care around the globe.