Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Sep 1.
Published in final edited form as: Neuropsychol Rev. 2019 May 29;29(3):328–337. doi: 10.1007/s11065-019-09406-7

A Quantitative Meta-analysis of Olfactory Dysfunction in Epilepsy

Kiran Khurshid 1, Andrew J D Crow 2, Petra E Rupert 3, Nancy L Minniti 4, Melissa A Carswell 5, Dawn J Mechanic-Hamilton 6, Vidyulata Kamath 7, Richard L Doty 8, Paul J Moberg 2,6,8, David R Roalf 2
PMCID: PMC6766414  NIHMSID: NIHMS1530485  PMID: 31144106

Abstract

Olfactory dysfunction in epilepsy is well-documented in several olfactory domains. However, the clinical specificity of these deficits remains unknown. The aim of this systematic meta-analysis was to determine which domains of olfactory ability were most impaired in individuals with epilepsy, and to assess moderating factors affecting olfactory ability. Extant peer-reviewed literature on olfaction in epilepsy were identified via a computerized literature search using PubMed, MEDLINE, PsycInfo, and Google Scholar databases. Twenty-one articles met inclusion criteria. These studies included a total of 912 patients with epilepsy and 794 healthy comparison subjects. Included studies measured olfaction using tests of odor identification, discrimination, memory, and detection threshold in patients with different types of epilepsy, including temporal lobe epilepsy (TLE), mixed frontal epilepsy (M-F), and mixed epilepsy (MIX). Olfactory deficits were robust in patients with epilepsy when compared to healthy individuals, with effect sizes in the moderate to large range for several olfactory domains, including odor identification (d=−1.59), memory (d=−1.10), discrimination (d=−1.04), and detection threshold (d=−0.58). Olfactory deficits were most prominent in patients with TLE and M-F epilepsy. Amongst patients with epilepsy, sex, age, smoking status, education, handedness, and age of illness onset were significantly related to olfactory performance. Overall, these meta-analytic findings indicate that the olfactory system is compromised in epilepsy and suggest that detailed neurobiological investigations of the olfactory system may provide further insight into this disorder.

Keywords: olfaction, smell, epilepsy, seizures, seizure disorder

INTRODUCTION

Patients with epilepsy often report experiencing olfactory abnormalities. Clinically these abnormalities include olfactory auras (West & Doty, 1995) in which typically malodorous smells are reported although not actually present (Acharya, Acharya, & Lüders, 1998; Chen et al., 2003). While a small proportion of patients report auras (1–30%) (Acharya et al., 1998), olfactory performance deficits are more common (West & Doty, 1995) and include impairments in odor detection (Eskenazi, Cain, Novelly, & Mattson, 1986; Savic, Bookheimer, Fried, & Engel, 1997), memory (Eskenazi et al., 1986; Savic et al., 1997), and identification (Eskenazi et al., 1986). Olfactory dysfunction is reported in both pre- and post-operative cases of epilepsy (West & Doty, 1995). Nonetheless, the clinical specificity of olfactory deficits and the clinical value of olfactory performance testing in epilepsy remains unclear. Arguably, it is possible that brief, accurate, and cost-effective approaches such as olfactory performance testing may aid traditional neuropsychological testing to better elucidate seizure focus and post-surgical outcomes (Abraham & Mathai, 1983; Acharya et al., 1998)

There is considerable evidence identifying the role of the temporal and frontal lobes in olfactory processing and epilepsy (Chen et al., 2003; Eslinger, Damasio, & Van Hoesen, 1982; West & Doty, 1995; Zatorre & Jones-Gotman, 1991). Afferent connections from the olfactory bulb transmit information to the primary olfactory cortex, which includes the piriform cortex, periamygdaloid cortex, entorhinal cortex, and amygdala (West & Doty, 1995). These primary olfactory cortex projections relay information to secondary structures, including the hippocampus, thalamus, orbitofrontal cortex, and insular cortex for higher cortical processing (West & Doty, 1995). Animal models of temporal lobe epilepsy (TLE) indicate that the epileptogenic zone is extensive, and suggest that the substrate for seizure generation is distributed over several temporal lobe structures (Bertram, 1997) including the entorhinal and perirhinal cortices. Surgical data indicate that both unilateral (Zatorre & Jones-Gotman, 1991) and bilateral (Eichenbaum, Morton, Potter, & Corkin, 1983) temporal lobectomies can adversely impact olfactory performance in patients with epilepsy. Moreover, temporal lobe structures (e.g., amygdala) are implicated in olfactory auras (Chen et al., 2003; Eslinger et al., 1982; West & Doty, 1995), which are associated with mesial temporal sclerosis (Acharya et al., 1998). Seizures involving an olfactory experience often elicit affective responses via amygdala activation and processing (Chen et al., 2003; Zald & Pardo, 1997), underscoring the involvement of the amygdala-hippocampal system (Willander & Larsson, 2007). In patients with epilepsy, epileptogenic areas near the amygdala can negatively impact patient performance in odor discrimination and odor recognition (Hudry, Perrin, Ryvlin, Mauguière, & Royet, 2003). As such, olfaction holds a unique role in perception, as major afferent projections from the olfactory bulb project ipsilaterally, not contralaterally, into the cortex and may provide important information about the epileptogenic areas within the brain (Kohler et al., 2001).

While impairments in olfaction in epilepsy are reported in several studies across olfactory task types, a quantitative analysis of these deficits has not yet been completed. Through a comprehensive meta-analysis of existing studies, we examined psychophysical olfactory functioning in patients with epilepsy. This approach allows for the comparison of dysfunction between olfactory domains and subtypes of epilepsy based upon seizure focus. Specifically, we examined and compared olfactory functioning in TLE and non-temporal lobe epilepsy. Additionally, moderating factors, such as sex and age, were examined given their known influence on olfactory functioning (Lehrner, 1993; Ship & Weiffenbach, 1993). We hypothesize that decrements in olfactory performance will be greater in patients with epilepsy than healthy individuals regardless of epilepsy subtype or olfactory domain. Given the role of temporal lobes in olfactory ability, we also posit that patients with temporal lobe epilepsy will demonstrate more profound olfactory deficits than those with other forms of epilepsy.

METHOD

Literature Search Strategy

Articles were identified through a computerized literature search using PubMed, MEDLINE, PsycINFO, and Google Scholar databases to find relevant studies using the search terms “epilep*” or “seizure disorder” and “olfact*” or “smell.” The search was conducted by a board-certified neuropsychologist, postdoctoral fellows in clinical neuropsychology, and advanced graduate and undergraduate research assistants. Studies were limited to English language articles that enrolled human subjects; studies which were not in English were excluded from the study. The last date of search was October 19, 2018 and included both full-text articles and abstracts. A thorough manual review of articles was performed using cross-references from identified original articles and reviews. Studies eligible for inclusion used performance-based measures of olfactory functioning, which provided statistical information that permitted meta-analytical methods to be used. Studies were selected and analyzed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA; Moher et al., 2009).

Methodological Variables

We defined olfactory function broadly, looking at effect sizes across four basic domains including psychophysical tests of: 1) odor identification (k = 38), 2) odor discrimination (k = 27), 3) odor memory (k = 15), and 4) odor detection threshold (k = 25). Assignment of olfactory tests to selected domains was guided by the classifications made in source articles and consensus of the authors. Effects sizes were the main outcome measure. Based on established criteria (Cohen, 1988), effect sizes were considered as small (d = 0.2), medium (d = 0.5), or large (d ≥ 0.8). All variables were entered into and analyzed in Comprehensive Meta-Analysis Software version 2.2 (Englewood, NJ)

Moderator Variables

In the event of significant heterogeneity in effect sizes across studies, categorical moderator analyses were undertaken for: 1) olfactory domain (identification, discrimination, memory, and detection threshold); 2) epilepsy subtype: a) temporal lobe epilepsy (TLE), which consisted of patients with bilateral temporal lobe epilepsy, left temporal lobe epilepsy, right temporal lobe epilepsy, and mixed temporal lobe epilepsy; b) mixed frontal lobe epilepsy (M-F); and c) mixed epilepsy (MIX), which consisted of subtypes which were not clearly specified in the sample; 3) surgery status (non-surgical or post-surgical); 4) lesion side (left, right, bilateral, combined, and other); and 5) laterality of the olfactory presentation method (left, right, or birhinal). Within the epilepsy patient population, the following demographic and clinical moderator variables were coded for meta-regression: 1) sex (% male), 2) smoking (% smokers), 3) age, 4) education, 5) age of seizure onset, 6) duration of illness (years), 7) general cognition (IQ), and 8) handedness (% right-handed). When studies with statistical outliers were noted, analyses were completed with and without outliers. Any change to the result after outlier removal is noted in the results. Pairwise comparisons were corrected for multiple comparisons using Bonferroni correction. Effect-sizes (d), number of effects (k), and heterogeneity (Q) are reported for the overall and moderator analyses. Effects size, number of effects and z-statistics for the weighted average effect size are reported for the meta-regressions within epilepsy patients.

RESULTS

The literature search yielded fifty-two (N=52) potential studies (Figure 1). After evaluation, twenty-one (N=21) studies met eligibility criteria (See “Studies Included in Meta-analysis” in the Supplemental Material), yielding 105 total effects from studies included a total of 912 patients with epilepsy and 794 healthy comparison subjects. Studies that did not provide specific information necessary for calculating effect sizes of olfactory functioning were not included in the analysis; these studies are listed below in the reference section “Studies Excluded from Meta-analysis.”

Figure 1.

Figure 1.

Flowchart of literature search in the meta-analysis.

Overall Meta-Analysis Results

Analysis of effect sizes across olfactory domains and demographic characteristics for the epilepsy sample showed an overall large effect (k = 105, d= −1.16, 95% CI = −1.34 < δ < −0.99) that was significantly heterogeneous (QB[104] = 664.21, p < 0.001). Given significant variability in effect sizes between the patient and healthy comparison groups, further analyses were performed to determine the effect of potential moderator variables.

Publication Bias

Analysis of possible publication bias revealed an asymmetric funnel scatterplot (Figure 2) and significant Begg-Mazumdar (Kendall’s τ = −0.33, p < 0.001) and Egger’s (t[103] = 7.79, p < 0.001) tests. Given these findings, Duval and Tweedie’s “trim and fill” adjustment was used to correct for funnel plot asymmetry and estimate the number of potential missing studies (Duval & Tweedie, 2000). Zero studies were trimmed and results showed that the point estimate of the adjusted value was of similar magnitude as the unadjusted value (d=−0.87). A classic fail-safe N test was also performed to calculate the number of studies that would be required to nullify the observed effect. It was found that an additional 3,202 “null” studies would be needed to nullify the obtained results. Based on these data, it was concluded that publication bias imposed minimal influence on present results.

Figure 2.

Figure 2.

Individual effects by standard error (Cohen’s d). Dashed lines represent standard error of the average standard difference. Funnel lines represent 95% confidence intervals (CI).

Moderator Analysis

Statistical results of the following moderator analyses are reported in Table 2. Briefly, moderator analyses revealed significant heterogeneity among the four olfactory domains (Table 2). Forest plots of these effects by study within each domain is displayed in Figure 3. The greatest deficits were found in odor identification, followed by odor memory, odor discrimination and odor detection threshold. Bonferroni-corrected (adjusted p-threshold<0.003) pairwise comparisons of effects sizes revealed that deficits in these domains were comparable, however, odor identification deficits were greater than odor detection threshold deficits. No other pairwise comparisons by olfactory domain were significant.

Table 2.

Statistical results of the moderator analysis.

k d Q df p
Olfactory Domain 24.23 3 <0.001
Identification 38 −1.59 - - <0.001
Memory 15 −1.10 - - <0.001
Discrimination 27 −1.04 - - <0.001
Detection Threshold 25 −0.58 - - <0.001
Epilepsy Subtype 16.55 2 <0.001
Temporal Lobe (TLE) 92 −1.25 <0.001
Mixed Frontal (M-F) 8 −.088 0.004
Mixed (MIX) 5 0.12 0.72
Surgery Status 3.67 1 0.06
Hemisphere of Lesion 1.69 4 0.64
0.92 1 0.33

Figure 3.

Figure 3.

Forest plots of individual effects with 95% CI by olfactory domain.

The majority (88%) of effects were from patients with TLE. Nonetheless, there was significant heterogeneity among the three diagnostic groups, TLE, M-F, and MIX (Table 2). The TLE and M-F groups showed large olfactory impairment relative to small, non-significant olfactory deficits in MIX. Bonferroni-corrected pairwise analyses (adjusted p threshold <0.017) indicated patients with TLE demonstrated greater impairment in olfactory ability when compared to MIX patients (Q[1] = 15.36, p < 0.001). Yet, the number of observations within the TLE group may have affected this finding. TLE and M-F groups did not differ from one another, nor did M-F and MIX groups. Finally, surgery status, hemisphere of lesion, and laterality of olfactory presentation all showed homogenous effects (Table 2A).

Meta-Regression of Demographic Characteristics on Olfaction in Epilepsy

Statistical results of the following meta-regression analyses within epilepsy patients are reported in Table 3. In general, larger olfactory deficits were associated with being female, older age, right-handedness, a greater frequency of cigarette smoking, fewer years of education, and an older age of illness onset. Duration of illness and general cognition (IQ) were not associated with olfactory performance.

Table 3.

Statistical results of the meta-regression analysis within epilepsy patients

k Z p
Sex 74 6.40 <0.001
Smoking (%) 20 −2.02 0.04
Age 91 −7.02 <0.001
Education 38 2.41 0.02
Age of Illness Onset 3 2.03 0.04
Duration of Illness 6 0.21 0.83
General IQ 26 0.16 0.87
Handedness - <0.01

DISCUSSION

This systematic review of olfactory performance in epilepsy affirms research findings from the extant literature and provides a broad investigation of olfactory functioning in individuals with epilepsy. Consistent with our hypothesis, we found that olfactory deficits were present in patients with epilepsy when compared to healthy individuals, with effect sizes in the moderate to large range for several olfactory domains, including odor identification, memory, discrimination, and detection threshold. Olfactory deficits were most prominent in patients with TLE and M-F epilepsy. Amongst patients with epilepsy, sex, age, education, and handedness were related to olfactory performance. Contrary to expectation, other moderating variables, such as surgery status, lesion laterality, and laterality of presentation method did not moderate the relationship between epilepsy and olfactory impairment.

The overall deficit in olfactory functioning was moderated by olfactory domain, such that patients with epilepsy showed the greatest levels of impairment in odor identification; this observation was primarily driven by patients with TLE. A deficit in odor identification is commensurate with prior research showing identification abilities are often impaired in patients with epilepsy (e.g., prior to a seizure) and that both nonsurgical and temporal lobectomy patients demonstrated deficits in odor identification (Eskenazi et al., 1986; West & Doty, 1995). Thus, based on the considerable contribution of temporal lobe structures in olfaction (Acharya et al., 1998; Eskenazi et al., 1986; Eslinger et al., 1982; West & Doty, 1995; Zald & Pardo, 1997; Zatorre & Jones-Gotman, 1991) and the role of the temporal lobe in semantic knowledge and processing (Doty & Kerr, 2005; Gabrieli, Cohen, & Corkin, 1988; Mummery et al., 2000), our study supports temporal lobe involvement in odor identification. Although there were insufficient effects to allow for the comparison of odor identification abilities in left TLE relative to right TLE patients, the semantic component involved in identifying an odor (Cain, de Wijk, Lulejian, Schiet, & See, 1998) may indicate left greater than right temporal lobe processes in this olfactory domain. In contrast, deficits in odor memory and odor discrimination may, instead implicate right hemisphere and/or frontal structures. This supports prior work which have found that odor memory, odor discrimination, and odor detection threshold are not necessarily localized to the left temporal lobe. For example, excision of the right temporal or right orbitofrontal cortex can affect odor memory discrimination (Jones-Gotman et al., 1997), the right hemisphere/right orbitofrontal cortex has been implicated in odor discrimination (Zatorre & Jones-Gotman, 1991), and odor discrimination requires processing in the right hemisphere (Zatorre & Jones-Gotman, 1990).

Unsurprisingly, older age was associated with a greater magnitude of olfactory deficit in patients with epilepsy, yet this deficit was independent duration of illness. Olfactory deficits are generally common in the older population (Doty & Kamath, 2014) and are prominent in neurodegenerative conditions such as Alzheimer’s disease and Parkinson’s disease (Mesholam, Moberg, Mahr, & Doty, 1998). These age-related deficits are evident across olfactory domains, including odor discrimination (Doty, McKeown, Lee, & Shaman, 1995), identification (Doty, Applebaum, Zusho, & Settle, 1985), detection (Cain & Rabin, 1989), and memory (Doty et al., 1995). The finding that relatively younger patients with epilepsy demonstrate decrements in olfactory functioning reiterates that alterations in either the peripheral or central olfactory system are affected in seizure disorders, including potential changes in primary olfactory reception (olfactory epithelium, olfactory bulb, etc.). Yet, one must consider that such declines may also be caused by non-olfactory (i.e., cognitive) and age-related changes in cortical areas involved in olfactory processing (Doty & Kamath, 2014).

Although epilepsy subtype had an appreciable effect on olfaction, lesion laterality did not. Our findings corroborate extant research implicating temporal and frontal lobe involvement in epilepsy (Eichenbaum et al., 1983) based on our findings that TLE and M-F patients demonstrated greater impairment in olfactory functioning relative to MIX patients. Indeed, we expected TLE to be associated with significant impairment in olfactory functioning given the critical role that subcortical/temporal structures (e.g. amygdala, hippocampus, entorhinal cortex, etc.) (Doty, 2015; Kjelvik, Evensmoen, Brezova, & Håberg, 2012; Kohler et al., 2001; West & Doty, 1995; Willander & Larsson, 2007) play in odor identification (Kjelvik et al., 2012; Willander & Larsson, 2007). M-F epilepsy also showed a large deficit in olfactory function; this result is somewhat expected, given that studies have emphasized the role of frontal structures in olfactory performance, such as the frontal piriform cortex and orbitofrontal cortex (Eslinger et al., 1982). However, these results are juxtaposed by a non-significant finding of lesion laterality. The latter finding may be due to imprecise nature of epilepsy diagnosis. In addition, diagnosis (e.g. TLE) and lesion location are not necessarily synonymous: it is possible for patients to have a seizure disorder without evidence of specific lesions (Kuzniecky et al., 1987) or structural abnormalities (Chen et al., 2003) on neuroimaging (i.e., “cryptogenic” epilepsy). Overall, our results indicate that olfactory dysfunction may provide insight into specific type of epilepsy, but the presence of lesions does not necessarily lead to deficits in olfactory performance.

We also report a counterintuitive sex finding regarding olfactory impairment, where women with epilepsy demonstrate greater olfactory deficit than men. This is surprising, given historical findings that show that women have superior olfactory abilities relative to men, particularly in terms of identification (Brand & Millot, 2001; Doty et al., 1985), sensitivity-detection (Brand & Millot, 2001), and recognition-identification (Brand & Millot, 2001; Ship & Weiffenbach, 1993). Yet, this finding may interact with typical sex related differences in seizure pathology. Previous imaging studies found that in mesial TLE and hippocampal sclerosis, men have a greater likelihood of frontal lobe hypometabolism ipsilateral to seizure onset whereas female patients show decreased metabolism in the contralateral temporal lobe (Savic & Engel, 1998). Thus, it is possible that sex differences may be attributable to differences in seizure pathology, such that men demonstrate less olfactory impairment than women based on reduced temporal involvement in seizure disorder, while women suffer from greater olfactory deficits based on the greater association with temporal lobe structures in epilepsy.

Our study was consistent with previous literature that found mixed evidence for olfactory impairment in temporal lobectomy patients (e.g., impaired olfactory threshold performance but intact odor recognition memory) (Rausch & Serafetinides, 1975), as surgery status approached, but did not meet, significance. Similar findings are evident in unilateral anterior temporal lobe resection patients who demonstrated lower, but statistically insignificant, olfactory performance in detection sensitivity (Eskenazi et al., 1986). Our finding contradicts some of the literature indicating olfactory deficit following surgical intervention for temporal lobe epilepsy. It is possible that variability in the olfactory tests that were used may have affected the reliability of the obtained effects, thereby diminishing a significant statistical finding of olfactory deficit following surgical intervention for epilepsy. This area of research is in need of specific follow-up studies.

Right-handed patients with epilepsy appeared to have greater olfactory deficits relative to non-right-handed individuals. Education appeared to buffer against olfactory deficit in patients with epilepsy, and indeed contributed to better olfactory performance. Rather than being a function of seizure disorder, it is possible that patients who had a higher level of formal academic training achieved higher scores on odor-related tasks that required a broader semantic or vocabulary base. For example, in odor identification tasks, participants are required to correctly identify an odor based on multiple-choice options. Individuals with less education and/or who may be unfamiliar with certain types of odors may obtain a deflated score which underestimates their “true” odor identification abilities by virtue of their limited exposure or knowledge of such stimuli. General cognition (IQ) had no significant effect on olfactory performance within the patient population.

Limitations and Future Directions

The systematic review was limited to English articles; articles which were in another language were not included. In addition, only studies for which qualitative and quantitative data were explicitly indicated or could be interpreted and categorized by the authors were included. Studies which were ambiguous or which did not provide clear data regarding the participants’ olfactory performance were not part of the analyses. There was no uniformity in the types of psychophysical tests used to assess different olfactory domains, which may affect the reliability of the results. Some olfactory domains are tested using more standardized measures than others. For example, identification methods often employ tests such as the University of Pennsylvania Smell Identification Test or the Sniffin’ Sticks, whereas odor threshold testing is often tested using a wide variety of methods. Yet, analyses of publication bias support the notion that these deficits are robust.

There are several moderating factors which, if elucidated in previous studies, would have provided useful information about differential olfactory functioning in patients with epilepsy. First, consistency in the psychophysical approach used (e.g., UPSIT, Sniffin’ Sticks) to assess olfactory dysfunction would enhance the reliability of findings. In addition, many studies fail to report important participant characteristics, such as ethnicity, frequency of smoking, and handedness, which impedes our ability to determine the potential influence of these factors. Incorporating a comprehensive neuropsychological battery and psychiatric evaluation that assessed different domains of cognitive and affective functioning would have allowed for a thorough examination of the association between domain-specific olfactory dysfunction and neurocognitive and psychosocial deficits in epilepsy. Finally, studies that examine the role of antiepileptic drugs (AEDs) on domains of olfactory functioning would provide insight into the effect of pharmacological medications on chemosensory functioning. Although certain AEDs have been explored in treating olfactory problems (Leopold, 2002; Zilstorff, 1966), the evidence regarding their effect has been mixed and the literature is limited. There are case reports indicating that specific AEDs affect olfaction (e.g., topiramate (Ghanizadeh, 2009)) and gustation (e.g., phenytoin (Doty & Bromley, 2004). Preclinical studies of phenobarbital report negative effects on olfactory function (Bath & Scharfman, 2013), but this effect was not replicated in humans (Campanella, 1978). Other AEDs have been shown to have positive (e.g., levetiracetam (Caminiti et al., 2016) or no effect on olfactory performance in epilepsy patients (e.g., ethosuximide, primidone, and sodium valproate;(Campanella, Filla, & De Michele, 1978).

In summary, this is the first comprehensive meta-analysis of olfactory dysfunction in epilepsy. We show that odor identification, odor memory, discrimination and detection threshold are all affected by epilepsy. These deficits are most prominent in TLE and M-F epilepsy. Overall, these findings suggest that the olfactory system is altered in epilepsy, as such detailed neurobiological investigation of the olfactory system may provide insight into this disorder.

Supplementary Material

11065_2019_9406_MOESM1_ESM
11065_2019_9406_MOESM2_ESM

Table 1.

The number of epilepsy cases and healthy controls included for each study by olfactory domain.

Odor Identification
Study Epilepsy Cases Healthy Controls
Carroll 1993 30 10
Desai 2015 25 25
Doty 2018 71 71
Eskenazi 1983 17 46
Haehner 2012 35 35
Hudry 2014 61 60
Hummel 2013 20 20
Jones-Gotman 1997 70 40
Juran 2016 31 14
Kohler 2001 32 25
Lehrner 1997 4 31
Martinez 1993 21 33
Rausch 1975 12 10
Savage 2017 55 50
Total 470
Odor Memory
Study Epilepsy Cases Healthy Controls
Dade 2002 40 21
Doty 2018 69 69
Eskenazi 1983 17 46
Hudry 2003 38 40
Martinez 1993 21 33
Pouliet 2008 19 19
Rausch 1977 14 10
Total 218 238
Odor Detection Threshold
Study Epilepsy Cases Healthy Controls
Campenella 1978 48 50
Doty 2018 71 71
Haehner 2012 35 35
Kohler 2001 32 25
Lehrner 1997 4 31
Rausch 1975 12 10
Total 202 222
Odor Discrimination
Study Epilepey Cases Healthy Controls
Abraham 1919 94 95
Eskenazi 1919 17 46
Haehner 2020 35 35
Martinez 1919 21 33
Zatorre 1991 106 20
Total 273 229

Acknowledgments

Funding Sources: This work was supported by K01 MH102609 (Roalf), the Brain and Behavior Foundation NARSAD Young Investigator grant program (Roalf), and the John Hopkins Clinical Research Scholars Program KL2TR001077 (Kamath).

Role of the Funding Sources: The funding sources had no role in the design, collection, analysis or manuscript preparation for this study.

Footnotes

Publisher's Disclaimer: This Author Accepted Manuscript is a PDF file of a an unedited peer-reviewed manuscript that has been accepted for publication but has not been copyedited or corrected. The official version of record that is published in the journal is kept up to date and so may therefore differ from this version.

Ethical Publication Statement: We affirm that we have read the Neuropsychology Review’s position on issues involved in ethical publication and affirm that this report is consistent with those guidelines.

Conflicts of Interest Disclosures: The authors declare that they have no conflict of interest.

Portions of this study were presented in a poster session at the 46th Annual Meeting of the International Neuropsychological Society, Washington, DC, February 2018.

REFERENCES

Uncategorized References

  1. Abraham A, & Mathai K (1983). The effect of right temporal lobe lesions on matching of smells. Neuropsychologia, 21(3), 277–281. [DOI] [PubMed] [Google Scholar]
  2. Acharya V, Acharya J, & Lüders H (1998). Olfactory epileptic auras. Neurology, 51(1), 56–61. [DOI] [PubMed] [Google Scholar]
  3. Bertram EH (1997). Functional anatomy of spontaneous seizures in a rat model of limbic epilepsy. Epilepsia, 38(1), 95–105. [DOI] [PubMed] [Google Scholar]
  4. Brand G, & Millot J-L (2001). Sex differences in human olfaction: between evidence and enigma. The Quarterly Journal of Experimental Psychology: Section B, 54(3), 259–270. [DOI] [PubMed] [Google Scholar]
  5. Cain WS, de Wijk R, Lulejian C, Schiet F, & See L-C (1998). Odor identification: perceptual and semantic dimensions. Chemical Senses, 23(3), 309–326. [DOI] [PubMed] [Google Scholar]
  6. Cain WS, & Rabin MD (1989). Comparability of two tests of olfactory functioning. Chemical Senses, 14(4), 479–485. [Google Scholar]
  7. Caminiti F, De Salvo S, Nunnari D, Bramanti P, Ciurleo R, Granata F, & Marino S (2016). Effect of the antiepileptic therapy on olfactory disorders associated with mesial temporal sclerosis. Neurocase, 22(4), 357–361. [DOI] [PubMed] [Google Scholar]
  8. Campanella G, Filla A, & De Michele G (1978). Smell and taste acuity in epileptic syndromes. European Neurology, 17(3), 136–141. [DOI] [PubMed] [Google Scholar]
  9. Chen C, Shih YH, Yen DJ, Lirng JF, Guo YC, Yu HY, & Yiu CH (2003). Olfactory auras in patients with temporal lobe epilepsy. Epilepsia, 44(2), 257–260. [DOI] [PubMed] [Google Scholar]
  10. Cohen J (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, New Jersey: Erlbaum. [Google Scholar]
  11. Doty RL (2015). Olfactory dysfunction and its measurement in the clinic. World Journal of Otorhinolaryngology-Head and Neck Surgery, 1(1), 28–33. doi: 10.1016/j.wjorl.2015.09.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Doty RL, Applebaum S, Zusho H, & Settle RG (1985). Sex differences in odor identification ability: a cross-cultural analysis. Neuropsychologia, 23(5), 667–672. [DOI] [PubMed] [Google Scholar]
  13. Doty RL, & Bromley SM (2004). Effects of drugs on olfaction and taste. Otolaryngologic Clinics of North America, 37(6), 1229–1254. [DOI] [PubMed] [Google Scholar]
  14. Doty RL, & Kamath V (2014). The influences of age on olfaction: a review. Frontiers in Psychology, 5, 20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Doty RL, & Kerr K-L (2005). Episodic odor memory: influences of handedness, sex, and side of nose. Neuropsychologia, 43(12), 1749–1753. [DOI] [PubMed] [Google Scholar]
  16. Doty RL, McKeown DA, Lee WW, & Shaman P (1995). A study of the test-retest reliability of ten olfactory tests. Chemical Senses, 20(6), 645–656. [DOI] [PubMed] [Google Scholar]
  17. Duval S, & Tweedie R (2000). Trim and fill: a simple funnel‐plot–based method of testing and adjusting for publication bias in meta‐analysis. Biometrics, 56(2), 45–463. [DOI] [PubMed] [Google Scholar]
  18. Eichenbaum H, Morton TH, Potter H, & Corkin S (1983). Selective olfactory deficits in case HM. Brain, 106(2), 459–472. [DOI] [PubMed] [Google Scholar]
  19. Eskenazi B, Cain WS, Novelly RA, & Mattson R (1986). Odor perception in temporal lobe epilepsy patients with and without temporal lobectomy. Neuropsychologia, 24(4), 553–562. [DOI] [PubMed] [Google Scholar]
  20. Eslinger PJ, Damasio AR, & Van Hoesen GW (1982). Olfactory dysfunction in man: anatomical and behavioral aspects. Brain and Cognition, 1(3), 259–285. [DOI] [PubMed] [Google Scholar]
  21. Gabrieli JD, Cohen NJ, & Corkin S (1988). The impaired learning of semantic knowledge following bilateral medial temporal-lobe resection. Brain and Cognition, 7(2), 157–177. [DOI] [PubMed] [Google Scholar]
  22. Ghanizadeh A (2009). Loss of taste and smell during treatment with topiramate. Eating and Weight Disorders-Studies on Anorexia, Bulimia and Obesity, 14(2–3), e137–e138. [DOI] [PubMed] [Google Scholar]
  23. Hudry J, Perrin F, Ryvlin P, Mauguière F, & Royet JP (2003). Olfactory short‐term memory and related amygdala recordings in patients with temporal lobe epilepsy. Brain, 126(8), 1851–1863. [DOI] [PubMed] [Google Scholar]
  24. Jones-Gotman M, Zatorre R, Cendes F, Olivier A, Andermann F, McMackin D, …Wieser H (1997).Contribution of medial versus lateral temporal-lobe structures to human odour identification. Brain: a journal of neurology, 120(10), 1845–1856. [DOI] [PubMed] [Google Scholar]
  25. Kjelvik G, Evensmoen HR, Brezova V, & Håberg AK (2012). The human brain representation of odor identification. Journal of Neurophysiology, 108(2), 645–657. [DOI] [PubMed] [Google Scholar]
  26. Kohler CG, Moberg PJ, Gur RE, O’connor MJ, Sperling MR, & Doty RL (2001). Olfactory dysfunction in schizophrenia and temporal lobe epilepsy. Cognitive and Behavioral Neurology, 14(2), 83–88. [PubMed] [Google Scholar]
  27. Kuzniecky R, de la Sayette V, Ethier R, Melanson D, Andermann F, Berkovic S, … Feindel W (1987). Magnetic resonance imaging in temporal lobe epilepsy: pathological correlations. Annals of Neurology:Official Journal of the American Neurological Association and the Child Neurology Society, 22(3), 341–347. [DOI] [PubMed] [Google Scholar]
  28. Lehrner JP (1993). Gender differences in long-term odor recognition memory: verbal versus sensory influences and the consistency of label use. Chemical Senses, 18(1), 17–26. [Google Scholar]
  29. Leopold D (2002). Distortion of olfactory perception: diagnosis and treatment. Chemical Senses, 27(7), 611–615. [DOI] [PubMed] [Google Scholar]
  30. Mesholam RI, Moberg PJ, Mahr RN, & Doty RL (1998). Olfaction in neurodegenerative disease: a meta-analysis of olfactory functioning in Alzheimer’s and Parkinson’s diseases. Archives of Neurology, 55(1), 84–90. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/9443714 [DOI] [PubMed] [Google Scholar]
  31. Mummery CJ, Patterson K, Price CJ, Ashburner J, Frackowiak RS, & Hodges JR (2000). A voxel‐based morphometry study of semantic dementia: relationship between temporal lobe atrophy and semantic memory. Annals of Neurology, 47(1), 36–45. [PubMed] [Google Scholar]
  32. Rausch R, & Serafetinides E (1975). Specific alterations of olfactory function in humans with temporal lobe lesions. Nature, 255(5509), 557. [DOI] [PubMed] [Google Scholar]
  33. Savic I, Bookheimer SY, Fried I, & Engel J (1997). Olfactory bedside test: a simple approach to identify temporo-orbitofrontal dysfunction. Archives of Neurology, 54(2), 162–168. [DOI] [PubMed] [Google Scholar]
  34. Savic I, & Engel J (1998). Sex differences in patients with mesial temporal lobe epilepsy. Journal of Neurology, Neurosurgery and Psychiatry, 65(6), 910–912. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Ship JA, & Weiffenbach JM (1993). Age, gender, medical treatment, and medication effects on smell identification. Journal of Gerontology, 48(1), M26–M32. [DOI] [PubMed] [Google Scholar]
  36. West SE, & Doty RL (1995). Influence of epilepsy and temporal lobe resection on olfactory function. Epilepsia, 36(6), 531–542. [DOI] [PubMed] [Google Scholar]
  37. Willander J, & Larsson M (2007). Olfaction and emotion: The case of autobiographical memory. Memory and Cognition, 35(7), 1659–1663. [DOI] [PubMed] [Google Scholar]
  38. Zald DH, & Pardo JV (1997). Emotion, olfaction, and the human amygdala: amygdala activation during aversive olfactory stimulation. Proceedings of the National Academy of Sciences, 94(8), 4119–4124. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Zatorre RJ, & Jones-Gotman M (1990). Right-nostril advantage for discrimination of odors. Perception and Psychophysics, 47(6), 526–531. [DOI] [PubMed] [Google Scholar]
  40. Zatorre RJ, & Jones-Gotman M (1991). Human olfactory discrimination after unilateral frontal or temporal lobectomy. Brain, 114(1), 71–84. [PubMed] [Google Scholar]
  41. Zilstorff K (1966). Parosmia. The Journal of Laryngology & Otology, 80(11), 1102–1104. [DOI] [PubMed] [Google Scholar]

Studies Included in Meta-analysis

*See associated Supplemental File*

  • 1.Abraham A, & Mathai K (1983). The effect of right temporal lobe lesions on matching of smells. Neuropsychologia, 21(3), 277–281. [DOI] [PubMed] [Google Scholar]
  • 2.Campanella G, Filla A, & De Michele G (1978). Smell and taste acuity in epileptic syndromes. European Neurology, 17(3), 136–141. [DOI] [PubMed] [Google Scholar]
  • 3.Carroll B, Richardson J, & Thompson P (1993). Olfactory information processing and temporal lobe epilepsy. Brain and Cognition, 22(2), 230–243. [DOI] [PubMed] [Google Scholar]
  • 4.Dade LA, Zatorre RJ, & Jones‐ Gotman M (2002). Olfactory learning: convergent findings from lesion and brain imaging studies in humans. Brain, 125(1), 86–101. [DOI] [PubMed] [Google Scholar]
  • 5.Desai M, Agadi J, Karthik N, Praveenkumar S, & Netto A (2015). Olfactory abnormalities in temporal lobe epilepsy. Journal of Clinical Neuroscience, 22(10), 1614–1618. [DOI] [PubMed] [Google Scholar]
  • 6.Doty RL, Tourbier I, Neff JK, Silas J Turetsky B, Moberg P, … Detre JA (2018). Influences of temporal lobe epilepsy and temporal lobe resection on olfaction. Journal of Neurology, 265(7), 1654–1665. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Eskenazi B, Cain WS, Novelly RA, & Friend KB (1983). Olfactory functioning in temporal lobectomy patients. Neuropsychologia, 21(4), 365–374. [DOI] [PubMed] [Google Scholar]
  • 8.Haehner A, Henkel S, Hopp P, Hallmeyer-Elgner S, Reuner U, Reichmann H, & Hummel T (2012). Olfactory function in patients with and without temporal lobe resection. Epilepsy & Behavior, 25(4), 477–480. [DOI] [PubMed] [Google Scholar]
  • 9.Hudry J, Perrin F, Ryvlin P, Mauguière F, & Royet JP (2003). Olfactory short-term memory and related amygdala recordings in patients with temporal lobe epilepsy. Brain, 126(8), 1851–1863. [DOI] [PubMed] [Google Scholar]
  • 10.Hudry J, Ryvlin P, Saive A-L, Ravel N, Plailly J, & Royet J-P (2014). Lateralization of olfactory processing: Differential impact of right and left temporal lobe epilepsies. Epilepsy & Behavior, 37, 184–190. [DOI] [PubMed] [Google Scholar]
  • 11.Hummel T, Henkel S, Negoias S, Galván JR, Bogdanov V, Hopp P, … Haehner A (2013). Olfactory bulb volume in patients with temporal lobe epilepsy. Journal of Neurology, 260(4), 1004–1008. [DOI] [PubMed] [Google Scholar]
  • 12.Jones-Gotman M, Zatorre R, Cendes F, Olivier A, Andermann F, McMackin D, … Wieser H (1997). Contribution of medial versus lateral temporal-lobe structures to human odour identification. Brain: A Journal of Neurology, 120(10), 1845–1856. [DOI] [PubMed] [Google Scholar]
  • 13.Juran SA, Lundström JN, Geigant M, Kumlien E, Fredrikson M, Åhs F, & Olsson MJ (2016). Unilateral resection of the anterior medial temporal lobe impairs odor identification and valence perception. Frontiers in Psychology, 6, 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Kohler CG, Moberg PJ, Gur RE, O’connor MJ, Sperling MR, & Doty RL (2001). Olfactory dysfunction in schizophrenia and temporal lobe epilepsy. Cognitive and Behavioral Neurology, 14(2), 83–88. [PubMed] [Google Scholar]
  • 15.Lehrner J, Baumgartner C, Serles W, Olbrich A, Pataraia E, Bacher J, … Deecke L (1997). Olfactory prodromal symptoms and unilateral olfactory dysfunction are associated in patients with right mesial temporal lobe epilepsy. Epilepsia, 38(9), 1042–1044. [DOI] [PubMed] [Google Scholar]
  • 16.Martinez BA, Cain WS, de Wijk RA, Spencer DD, Novelly RA, & Sass KJ (1993). Olfactory functioning before and after temporal lobe resection for intractable seizures. Neuropsychology, 7(3), 351–363. [Google Scholar]
  • 17.Pouliot S, & Jones-Gotman M (2008). Medial temporal-lobe damage and memory for emotionally arousing odors. Neuropsychologia, 46(4), 1124–1134. [DOI] [PubMed] [Google Scholar]
  • 18.Rausch R, & Serafetinides E (1975). Specific alterations of olfactory function in humans with temporal lobe lesions. Nature, 255(5509), 557. [DOI] [PubMed] [Google Scholar]
  • 19.Rausch R, Serafetinides E, & Crandall PH (1977). Olfactory memory in patients with anterior temporal lobectomy. Cortex, 13(4), 445–452. [DOI] [PubMed] [Google Scholar]
  • 20.Savage SA, Butler CR, Milton F, Han Y, & Zeman AZ (2017). On the nose: Olfactory disturbances in patients with transient epileptic amnesia. Epilepsy & Behavior, 66, 113–119. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Zatorre RJ, & Jones-Gotman M (1991). Human olfactory discrimination after unilateral frontal or temporal lobectomy. Brain, 114(1), 71–84. [PubMed] [Google Scholar]

Studies Excluded from Meta-analysis

  • 1.Bernhardt BC, Kim H, & Bernasconi N (2013). Patterns of subregional mesiotemporal disease progression in temporal lobe epilepsy. Neurology, 81(21), 1840–1847. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Berry DT, Wetter MW, Baer RA, Youngjohn JR, Gass CS, Lamb DG, … & Buchholz D (1995). Overreporting of closed-head injury symptoms on the MMPI-2. Psychological Assessment, 7(4),517. [Google Scholar]
  • 3.Buchanan TW, Tranel D, & Adolphs R (2003). A specific role for the human amygdala in olfactory memory. Learning & Memory, 10(5), 319–325. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Caminiti F, De Salvo S, Nunnari D, Bramanti P, Ciurleo R, Granata F, & Marino S (2016). Effect of the antiepileptic therapy on olfactory disorders associated with mesial temporal sclerosis. Neurocase, 22(4), 357–361. [DOI] [PubMed] [Google Scholar]
  • 5.Ciumas C, Lindström P, Aoun B, & Savic I (2008). Imaging of odor perception delineates functional disintegration of the limbic circuits in mesial temporal lobe epilepsy. Neuroimage, 39(2), 578–592. [DOI] [PubMed] [Google Scholar]
  • 6.Crunelli V, & Carmignoto G (2013). New vistas on astroglia in convulsive and non‐convulsive epilepsy highlight novel astrocytic targets for treatment. The Journal of physiology, 591(4), 775–785. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Doty RL (2008). Temporal lobe epilepsy and temporal lobe resection: Influences on olfactory function.Chemical Senses, 33, S1–S175. doi: 10.1093/chemse/bjn065 [DOI] [Google Scholar]
  • 8.Doty R, Tourbier I, & Neff J (2014). Influences of temporal lobe epilepsy and temporal lobe resection on olfactory function. Neurology, 82, S59–007. [Google Scholar]
  • 9.Dulay MF, Gesteland RC, Shear PK, Ritchey PN, & Frank RA (2008). Assessment of the influence of cognition and cognitive processing speed on three tests of olfaction. Journal of Clinical and Experimental Neuropsychology, 30(3), 327–337. [DOI] [PubMed] [Google Scholar]
  • 10.Eichenbaum H, Sauvage M, Fortin N, Komorowski R, & Lipton P (2012). Towards a functional organization of episodic memory in the medial temporal lobe. Neuroscience & Biobehavioral Reviews, 36(7), 1597–1608. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Eskenazi B, Cain WS, Novelly RA, & Mattson R (1986). Odor perception in temporal lobe epilepsy patients with and without temporal lobectomy. Neuropsychologia, 24(4). 553–562. [DOI] [PubMed] [Google Scholar]
  • 12.Garganis K, Kokkinos V, & Zountsas B (2013). EEG–fMRI findings in late seizure recurrence following temporal lobectomy: A possible contribution of area tempestas. Epilepsy & Behavior Case Reports, 1, 157–160. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Gordon HW, & Sperry RW (1969). Lateralization of olfactory perception in the surgically separated hemispheres of man. Neuropsychologia, 7(2), 111–120. [Google Scholar]
  • 14.Grant AC (2005). Interictal perceptual function in epilepsy. Epilepsy & Behavior, 6(4), 511–519. [DOI] [PubMed] [Google Scholar]
  • 15.Jacek S, Stevenson RJ, & Miller LA (2007). Olfactory dysfunction in temporal lobe epilepsy: A case of ictus-related parosmia. Epilepsy & Behavior, 11(3), 466–470. [DOI] [PubMed] [Google Scholar]
  • 16.Jones-Gotman M, & Zatorre RJ (1988). Olfactory identification deficits in patients with focal cerebral excision. Neuropsychologia, 26(3), 387–400. [DOI] [PubMed] [Google Scholar]
  • 17.Jones-Gotamn M, & Zatorre RJ (1993). Odor recognition memory in humans: Role of right temporal and orbitofrontal regions. Brain and Cognition, 22(2), 182–198. [DOI] [PubMed] [Google Scholar]
  • 18.Kumar G, Juhász C, Sood S, & Asano E (2012). Olfactory hallucinations elicited by electrical stimulation via subdural electrodes: Effects of direct stimulation of olfactory bulb and tract. Epilepsy & Behavior, 24(2), 264–268. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.La Rosa-Prieto D, Moya-Pinilla D, Saiz-Sanchez D, Ubeda-banon I, Arzate DM, Flores-Cuadrado A, … Martinez-Marcos A (2015). Olfactory and cortical projections to bulbar and hippocampal adult-born neurons. Frontiers in Neuroanatomy, 9, 1–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Lee Y-J, & Lee JS (2013). Temporal lobe epilepsy surgery in children versus adults: from etiologies to outcomes. Korean Journal of Pediatrics, 56(7), 275–281. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Poo C, & Isaacson JS (2009). Odor representations in olfactory cortex: “Sparse” coding, global inhibition, and oscillations. Neuron, 62(6), 850–861. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Rami L, Loy CT, Hailstone J, & Warren JD (2007). Odour identification in frontotemporal lobar degeneration. Journal of Neurology, 254(4), 431–435. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Restrepo D, Hellier JL, & Salcedo E (2014). Complex metabolically demanding sensory processing in the olfactory system: Implications for epilepsy. Epilepsy & Behavior, 38, 37–42. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Savic I, Bookheimer SY, Fried I, & Engel J (1997). Olfactory bedside test: A simple approach to identify temporo-orbitofrontal dysfunction. Archives of Neurology, 54(2), 162–168 [DOI] [PubMed] [Google Scholar]
  • 25.Shapiro LA, Ng K, Zhou Q-Y, & Ribak CE (2009). Subventricular zone-derived, newly generated neurons populate several olfactory and limbic forebrain regions. Epilepsy & Behavior, 14(1), 74–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Shimshek DR, Bus T, Kim J, Mihaljevic A, Mack V, Seeburg PH, … Schaefer AT (2005). Enhanced odor discrimination and impaired olfactory memory by spatially controlled switch of AMPA receptors. PLoS Biology, 3(11), 2017–2030. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.van Campen JS, Jansen FE, Kleinrensink NJ, Joëls M, Braun KP, & Bruining H (2015). Sensory modulation disorders in childhood epilepsy. Journal of Neurodevelopmental Disorders, 7(1), 1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Vaughan DN, & Jackson GD (2014). The piriform cortex and human focal epilepsy. Frontiers in Neurology, 5, 1–18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Watanabe S, Hara K, Ohta K, Iino H, Miyajima M, Matsuda A, … & Matsushima E (2013). Aroma helps to preserve information processing resources of the brain in healthy subjects but not in temporal lobe epilepsy. Seizure, 22(1), 59–63. [DOI] [PubMed] [Google Scholar]
  • 30.West SE, & Doty RL (1995). Influence of epilepsy and temporal lobe resection on olfactory function. Epilepsia, 36(6), 531–542. [DOI] [PubMed] [Google Scholar]
  • 31.Ye BS, Cho Y-J, Jang SH, Lee MK, Lee BI, & Heo K (2012). The localizing and lateralizing value of auras in lesional partial epilepsy patients. Yonsei Medical Journal, 53(3), 477–485. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

11065_2019_9406_MOESM1_ESM
11065_2019_9406_MOESM2_ESM

RESOURCES