Spatiotemporal dynamics of odor representations in the human brain revealed by EEG decoding

Significance

To elucidate when and where in the brain different aspects of odor perception emerge, we decoded odors from an electroencephalogram and associated the results with perception and source activities. The odor information was decoded 100 ms after odor onset at the earliest, with its signal sources estimated in and around the olfactory areas. The neural representation of odor unpleasantness emerged 300 ms after odor onset, followed by pleasantness and perceived quality at 500 ms. During this time, brain regions representing odor information spread rapidly from the olfactory areas to regions associated with emotional, semantic, and memory processing. The results suggested that odor perception emerges through computations in these areas, with different perceptual aspects having different spatiotemporal dynamics.

Abstract

How the human brain translates olfactory inputs into diverse perceptions, from pleasurable floral smells to sickening smells of decay, is one of the fundamental questions in olfaction. To examine how different aspects of olfactory perception emerge in space and time in the human brain, we performed time-resolved multivariate pattern analysis of scalp-recorded electroencephalogram responses to 10 perceptually diverse odors and associated the resulting decoding accuracies with perception and source activities. Mean decoding accuracies of odors exceeded the chance level 100 ms after odor onset and reached maxima at 350 ms. The result suggests that the neural representations of individual odors were maximally separated at 350 ms. Perceptual representations emerged following the decoding peak: unipolar unpleasantness (neutral to unpleasant) from 300 ms, and pleasantness (neutral to pleasant) and perceptual quality (applicability to verbal descriptors such as “fruity” or “flowery”) from 500 ms after odor onset, with all these perceptual representations reaching their maxima after 600 ms. A source estimation showed that the areas representing the odor information, estimated based on the decoding accuracies, were localized in and around the primary and secondary olfactory areas at 100 to 350 ms after odor onset. Odor representations then expanded into larger areas associated with emotional, semantic, and memory processing, with the activities of these later areas being significantly associated with perception. These results suggest that initial odor information coded in the olfactory areas (<350 ms) evolves into their perceptual realizations (300 to >600 ms) through computations in widely distributed cortical regions, with different perceptual aspects having different spatiotemporal dynamics.

The human brain rapidly translates sensory inputs into perception through dynamic computations within and across regions. For example, in the visual system, it has been demonstrated that the transition of neural representations of low-level visual features to those of perception (e.g., semantic categories) occurs within several hundred milliseconds after image presentation, during which different combinations of brain regions represent information transiently or persistently (1, 2). Regarding chemical sensations, it has been shown that taste stimuli rapidly activate multiple cortical areas including the primary gustatory area, and simultaneously, neural representations of individual tastes emerge (3, 4). However, in terms of olfaction, integrative knowledge of the temporal and spatial aspects of brain activity is insufficient. Olfaction has multiple perceptual and neural features that differ from those of other senses. Unlike visual stimuli, odors alone rarely form discernible objects (5), and the most salient aspect of olfactory perception has been considered to be pleasantness (6–9). Yet odors are still able to evoke diverse perceptions other than pleasantness, such as fruity and floral perceptions, and a dysfunction of the perceptual system is associated with neurological disorders (10, 11). Peripheral inputs, which are coded through a pattern of differential binding at ∼400 types of olfactory receptors, are directly transmitted to the olfactory bulb (OB) without a thalamic relay and are then sent in parallel to multiple limbic regions, which are collectively referred to as the primary olfactory area (POA). Considering its uniqueness, additional studies on the neural basis of olfactory perception are needed for a more comprehensive understanding of the neural basis of perception.

To date, the involvement of brain regions in olfactory perceptual processing has been examined mainly using functional MRI (fMRI). For example, the representation of odor pleasantness has been reported in the POA including the piriform cortex and amygdala (12, 13) and in the secondary olfactory area (SOA) including the orbitofrontal cortex (OFC) (14). Interestingly, although the relationship with perception was not investigated, a recent fMRI functional connectivity study found that subregions of the POA form dissociable networks with broad regions in the brain, including those that have not been traditionally considered as part of the olfactory cortex, such as Broca’s area and the temporal pole (15). These findings suggest possible early involvement of large brain networks in olfactory processing. However, the temporal resolution of fMRI is too low to examine the time at which each brain region is involved in olfactory perceptual processing.

Magnetoencephalography (MEG) and electroencephalography (EEG) offer high temporal resolutions and may be powerful tools for studying the spatiotemporal dynamics of brain activities underlying odor perception. In fact, by focusing on the traditional olfactory event-related potential/field (OERP/OERF) components, previous studies found that the amplitudes and/or latencies of those components that appeared ∼200 to 1000 ms after odor presentation were modulated by perceptual aspects of the presented odors, such as intensity and pleasantness (16–23). A recent study examined the relationship between oscillatory patterns of scalp-recorded EEG (sEEG) and perceived odor quality in a time-resolved manner (24), and another study examined the source-reconstructed activities of the OB to show that the neural representation of odor pleasantness emerges at the OB within a few hundred milliseconds (25). Although these studies have provided valuable information regarding the temporal aspects of olfactory perceptual processing, there are still many unanswered questions. First, as most of the previous studies on OERP/OERF used few odors, often only one pleasant and one unpleasant, the neural bases of different aspects of perception, such as pleasantness and quality, have been difficult to dissociate. Second, except for the single study that examined activity in the OB (25), source activities associated with perception have yet to be examined in a time-resolved matter, and therefore the spatiotemporal dynamics of odor perceptual representations within the entire brain remain largely unexplored.

In the current study, we investigated the spatiotemporal emergence of odor representations in the human brain based on scalp-recorded OERPs and estimated source activities for 10 odors with diverse perceptual qualities (see Fig. 1). To examine the information coded in the OERP signals, we used time-resolved multivariate pattern analysis (tMVPA), or decoding, on OERPs (see Fig. 2). We then assessed representational similarities between perceptions and brain activities at both sensor and source levels using time-resolved representational similarity analysis (tRSA) (26, 27). By performing tRSA for each time point and/or for each voxel, we identified when and where odor information was represented during olfactory processing (see Figs. 3 and 4). To capture the perceptual characteristics of the presented odors, we used a large set of verbal descriptors and regarded the rated quality as a metric of perceptual structure in the whole olfactory perceptual space, including range of pleasantness, from unpleasant to pleasant, which is the most salient dimension in olfactory perception (6–9). In addition, we obtained perceptual ratings for unipolar pleasantness (neutral to pleasant) and unpleasantness (neutral to unpleasant) and assessed the neural representation for each of these aspects separately. In this way, we examined when and where the neural representations of olfactory perception emerged and how they differed among different aspects of perception.

Fig. 1.

Experimental design, stimulus characteristics, and EEG responses to the odors. (A) The schedule of the overall experimental procedures. (B) The timeline of the experimental tasks for EEG recordings. To maintain attention on odors, subjects rated the bipolar pleasantness in every trial. Odors were presented to the subjects’ right nostrils using an olfactometer, following the recommendations for acquiring OERP (SI Appendix, Fig. S1). Pleasantness ratings were provided by changing numbers by pressing keys (details in SI Appendix, Methods). (C) Ratings of perceived odor quality for each of the 10 odors using semantic descriptors. The size and color of each circle represent the intersubject mean and SD, respectively. Odor abbreviations are shown at the bottom (full names in Materials and Methods). Unipolar pleasantness (D) and unpleasantness (E) ratings for each of the 10 odors. Center lines, box limits, and whiskers represent intersubject medians, interquartile ranges, and 1.5 times the interquartile ranges, respectively. Dots overlaid on the box plots represent ratings of each subject. (F) Analysis approach. Gray words indicate variables used for validation or control. Results of pairwise decoding are shown in Fig. 2, results of (a) are shown in Fig. 3, and results of (b, c) are shown in Fig. 4. (G) Intersubject mean baseline corrected GFP computed across all the 64 electrodes, shown in the inset, for each odor. The bottom horizontal lines correspond to the periods when the GFP was significantly above zero (one-sided, one-sample Student’s t test, P < 0.05, Bonferroni–Holm corrected). A, anterior; P, posterior; L, left hemisphere; R, right hemisphere.

Fig. 2.

Pairwise decoding revealed temporal dynamics of odor information representations. (A) Schematic representation of the tMVPA pairwise decoding. An odor pair, Cyc vs. 4Pe, for time = 200 ms is shown as an example. Decoding models were built for each of the 45 odor pairs for each time point. For a given time point, single-trial event-related potential amplitudes at all 64 scalp electrodes within a preceding time interval of 200 ms were concatenated and used as features. To avoid underestimation of latency, we assigned the accuracy to the last time point in each time window (e.g., the accuracy obtained using OERPs during 0 to 200 ms was assigned to 200 ms). Using a cross-validation protocol, for subject-wise decoding, a random sample of 90% of EEG trials was used for training, and the remaining 10% were used for testing. For cross-subject decoding, EEG trials from 21 subjects were used for training, and those of the remaining one subject were used for testing. The entire procedure was repeated for 60 cross-validations for subject-wise decoding and until all subjects were used as a test set for cross-subject decoding. The grand mean accuracy of subject-wise decoding (B) and cross-subject decoding (C) averaged across 45 odor pairs. Bottom horizontal lines indicate statistical significance (one-sided, one-sample Student’s t test, P < 0.05, Bonferroni–Holm corrected), with numbers indicating their onsets. Shaded areas indicate 95% confidence intervals across subjects (B) or across outer-loop CVs (C). (D) Decoding accuracies of each odor pair at specified time points. The upper triangular matrix corresponds to subject-wise decoding, and the lower one corresponds to cross-subject decoding. Abbreviations of odors are shown at the left and bottom (see Materials and Methods for full names).

Fig. 3.

Temporal evolution of neural representations underlying olfactory perception. (A) Procedures followed for tRSA exemplified for one time point. For each subject (shown as “sub01-03…”), brain representational dissimilarities were measured as pairwise decoding accuracies based on the subject-wise model, and the perceptual representational dissimilarities were measured as pairwise Euclidean distances based on perceptual scores, each resulting in a vector with 45 elements. These vectors were sorted according to the decoding accuracy for the given time point and then were averaged across subjects to obtain the RDVs. We calculated the Spearman’s rank correlation between the brain and perceptual RDVs. This procedure was performed at the same temporal resolution as the decoding analysis (every 50 ms). Results of tRSA with zero–order correlation (B) and with partial correlation (C). For unipolar pleasantness and unpleasantness, intensity was controlled for, and for quality, intensity, unipolar pleasantness, and unpleasantness were controlled for. The bottom horizontal lines indicate the statistical significance (permutation test, 10,000 permutations; P < 0.05, Bonferroni–Holm corrected), with numbers indicating their onsets. Ple, unipolar pleasantness; unp, unipolar unpleasantness; qua, quality; int, intensity. Results for other variables (intensity, pleasantness evaluated during EEG recordings, similarity, and trigeminal) are shown in SI Appendix, Fig. S4A.

Fig. 4.

Spatiotemporal evolution of odor representations in the brain. (A) Procedures used for source-level tRSA. Using the sLORETA algorithm, source activities were estimated for the cortical gray matter that was partitioned into 6,239 voxels. Source-level brain RDVs were constructed for each voxel by computing the pair-wise differences in standardized current source densities for all possible 45 odor pairs, and sensor-level brain RDVs were constructed based on decoding accuracies obtained using the subject-wise models. For both RDVs, odor pairs were sorted according to the source-level dissimilarities of the given time point before averaging across subjects. To relate the source activities with olfactory perception, RDVs based on perceptual scores were used instead of sensor-level brain RDVs. tRSA was performed at the same temporal resolution as the decoding analysis (every 50 ms). (B–E) Results of source–sensor tRSA (B) and source-level tRSA for unipolar pleasantness while controlling for intensity (C), unipolar unpleasantness while controlling for intensity (D), and quality while controlling for intensity, unipolar pleasantness, and unipolar unpleasantness (E). For ease of visualization, for each analysis, time points with similar brain maps were grouped based on cluster analysis, and the mean (partial) correlation coefficients within each time window were shown according to the color bar. Only significant voxels (permutation test, 10,000 permutations; P < 0.05, FDR corrected for all the voxels and time points) for at least one time point within a given time window were colored. Results of all the time points and source–quality tRSA are shown in SI Appendix, Fig. S5, and brain slices from the POA are shown for source–sensor tRSA at 100 ms in SI Appendix, Fig. S6. Ple, unipolar pleasantness; unp, unipolar unpleasantness; qua, quality; int, intensity; R, right hemisphere; L, left hemisphere.

Results and Discussion

Twenty-two subjects participated in the EEG and sensory testing sessions (Fig. 1A). During the EEG recordings, 10 perceptually diverse odors with similar perceived intensities were presented (Fig. 1B). The perceptual characteristics of the odors were assessed by the same subjects on the days following the EEG recordings (Fig. 1A). While the dimensionality of olfactory perception is not yet known, many studies suggest that it is multidimensional (28, 29), with pleasantness being the most salient dimension (6–9). To capture the perceptual characteristics of the current odors in multidimensional space, we used a large set of verbal descriptors screened for them (Fig. 1C). In the later analysis, we used the rated quality as a metric of perceptual structure in the whole odor perceptual space, including pleasantness. In addition, we obtained ratings of unipolar pleasantness (neutral to pleasant) and unpleasantness (neutral to pleasant) so that we could separately examine each of these neural representations. Ratings on perceptual quality (Fig. 1C), unipolar pleasantness (Fig. 1D), and unpleasantness (Fig. 1E) showed that the 10 odors were indeed perceived differently, although remarkable individual differences were also observed, as have been reported in earlier studies on olfaction (30, 31). The remaining results of sensory tests that were used for either validation (pleasantness ratings obtained during EEG sessions, trigeminal test, and similarity ratings) or as a control variable (intensity) are reported in SI Appendix, Tables S1 and S2. The grand mean global field power and OERP waveform for each odor are shown in Fig. 1G and SI Appendix, Fig. S2. For all odors, typical OERP waveforms and a significant increase of global field powers (GFPs) were observed (one-sample, one-sided Student’s t test; H₀, GFP after odor onset ≤ GFP in baseline period; H₁, GFP after odor onset > GFP in baseline period). Note that odor onset, or time = 0 in time–series plots, indicates the actual time of odor onset after adjusting for the odor delivery delay measured using a photo-ionization detector (PID) (SI Appendix, Fig. S1C).

Multivariate Patterns of OERPs Carry Odor Information 100 ms after Odor Onset.

To assess the temporal dynamics of neural representations of odor information, we conducted tMVPA pairwise decoding for all 45 possible odor pairs (Fig. 2). By examining the decoding accuracies in a time-resolved manner, the analysis allowed us to estimate when OERPs carry enough information to discriminate odors. In addition, as the higher decoding performance of a given odor pair suggests a lower similarity between the neural representations of these odors (32), we could estimate how the representational structure of odors changes over time. Although multivariate pattern analysis (MVPA) often shows higher sensitivity compared to univariate analysis (33, 34), to the best of our knowledge, no studies have applied tMVPA decoding to scalp-recorded OERPs. Considering that previous human OERP studies found differences in amplitudes, latencies, and/or scalp topographies of OERP components between odors (16–18, 20–22, 35), and considering animal studies that suggest the importance of both spatial and temporal patterns of neural activities in odor coding (36, 37), we concatenated single-trial OERP amplitudes at all 64 scalp electrodes for every time point within a 200 ms window to use as features for tMVPA decoding (Fig. 2A); thus, both the spatial and temporal characteristics were included in the decoding models. Decoding analysis was performed every 50 ms, and the statistical significance was tested using a one-sample, one-sided Student’s t test (H₀, decoding accuracy ≤ chance level; H₁, decoding accuracy > chance level) with Bonferroni–Holm correction.

First, we used subject-wise tMVPA (38), where the decoding models were built separately for each subject; then the group-level inference was performed by combining the subject-wise decoding performances of all subjects. The time course of the grand average decoding accuracy across odor pairs is shown in Fig. 2B, and that of each odor pair is shown for the selected time points in Fig. 2D (Upper Triangle). Scalp OERPs allowed the pairwise decoding of individual odors; the across-pair mean decoding accuracies significantly exceeded the chance level (50%) at 100 ms after odor onset (51.2%, t (21) = 3.84, corrected P = 0.012) and reached a maximum at 350 ms (Fig. 2B; 54.6%, t (21) = 4.84, corrected P = 0.0014). Decoding accuracies differed across odor pairs, and the pattern of the across-odor-pair differences also differed over time (Fig. 2D, Upper Triangle). This finding suggests that similarities among the neural representations of the odors differ for each pair of odors and that they change dynamically over time. Furthermore, we conducted multiclass decoding in which the decoder was trained to classify EEG trials into 1 of the 10 possible odors (see SI Appendix, Method for details). Across-odor mean decoding accuracy rose significantly above the chance level (10%) at 150 ms (11.7%, t (21) = 4.27, corrected P = 0.0056) and reached a maximum at 300 ms after odor onset (SI Appendix, Fig. S3 A and B; 13.6%, t (21) = 4.82, corrected P = 0.0016). These results collectively indicate that multivariate patterns of OERPs carry information regarding odors from a very early stage of olfactory processing (∼100 ms), where apparent OERP components do not appear (SI Appendix, Fig. S2), to a relatively late stage (∼1,000 ms), with differences among brain activities evoked by each odor reaching a maximum at approximately 350 ms after odor onset.

In addition to the subject-wise tMVPA, we conducted cross-subject tMVPA, in which the decoding models for each odor pair were generalized across subjects using leave-one-subject-out cross-validation (38). Significant pairwise decoding performance was observed, with the time course of the grand average decoding accuracies (Fig. 2C; peak at 300 ms, 54.7%, t (21) = 5.46, corrected P = 0.00034) and that of each odor pair (Fig. 2D, Lower Triangle) similar to those of the subject-wise-tMVPA. Cross-subject multiclass decoding was also consistent with that of the subject-wise model (SI Appendix, Fig. S3 C and D); significant across-odor mean accuracies began at 150 ms (12.0%, t (21) = 3.71, corrected P = 0.020) and reached a maximum at 300 ms after odor onset (13.5%, t (21) = 4.41, corrected P = 0.011). These results show that there are patterns of OERPs that represent odor information and are consistent across subjects, and on average, the temporal dynamics of odor representations revealed by the cross-subject models (Fig. 2C and Lower Triangle, Fig. 2D) are similar to those revealed by the subject-wise models (Fig. 2B and Upper Triangle, Fig. 2D).

Neural Representations of Different Perceptual Aspects Followed Different Temporal Dynamics.

Pairwise decoding analysis showed remarkable differences in the pairwise dissimilarity of neural representations between odor pairs and across time (Fig. 2D). Next, we asked whether and when such a neural representational structure was related to the perceptual characteristics of odors using the tRSA (Fig. 3A) (26, 27). In olfaction, large individual differences in perception are known (30, 31), which were also observed in our data (Fig. 1 C–E). Given such individual differences, tRSA was conducted using decoding accuracies based on the subject-wise models. Statistical significance was tested using a one-sided permutation test with Bonferroni–Holm correction (H₀, correlation coefficient ≤ 0; H₁, correlation coefficient > 0). We found significant correlations between the brain and perceptual representational dissimilarity vectors (RDVs) in all perceptual measures, except for trigeminal (Fig. 3B and SI Appendix, Fig. S4A). The correlations reached significance at 150 ms with unipolar unpleasantness (rho = 0.41, corrected P = 0.049), which was followed by perceived quality (400 ms, rho = 0.60, corrected P < 0.0001) and unipolar pleasantness (500 ms, rho = 0.46, corrected P = 0.035). Although decoding performance was its maximum 350 ms after odor onset (Fig. 2B), all the perceptual measures reached their maxima after 600 ms (Fig. 3B; 800 ms, rho = 0.66, corrected P < 0.0001 for pleasantness; 600 ms, rho = 0.81, corrected P < 0.0001 for unpleasantness; 600 ms, rho = 0.82, corrected P < 0.0001 for quality). Variables that were analyzed for validation, i.e., perceived similarity, pleasantness obtained during the EEG session (on a bipolar scale), and intensity, also followed similar time courses (SI Appendix, Fig. S4A).

Perceptual variables were correlated with each other (SI Appendix, Table S3). To examine when neural representations underlying specific aspects of perception emerge, we further performed tRSA using partial correlation (Fig. 3C). Previous EEG studies on visual modality suggest that the representation of unipolar unpleasantness emerges earlier than that of unipolar pleasantness (39, 40). Regarding olfaction, three behavioral studies have compared the reaction times to pleasant and unpleasant odors and have reported conflicting results; one showed greater latencies for pleasant odors (41), and two showed mixed results (42, 43). Recently, an EEG study showed that the beta-band activity in the OB in the early time period (50 to 200 ms) was higher for unpleasant odors compared to that for pleasant odors (25). However, electrophysiological examination of whole-brain activities is still lacking. Therefore, we examined the representations of unipolar pleasantness and unpleasantness separately while controlling intensity, which was significantly correlated with unipolar unpleasantness (SI Appendix, Table S3). In addition, while pleasantness is suggested to be the primary dimension of olfactory perceptual space, odors certainly have perceptual aspects other than pleasantness, such as fruity and floral. In the realm of olfactory psychology, the temporal order of processing these perceptual features has been debated. Is olfactory perception “valence-centered,” where humans first recognize the pleasantness of odors before recognizing its quality, or is it “object-centered,” where olfactory quality, e.g., floral or fruity, is recognized before pleasantness (44, 45)? So far, this question has been examined only behaviorally. These behavioral studies have shown that reaction times for pleasantness evaluation tasks are significantly longer than those for quality evaluation tasks (e.g., floral, minty, fuel, fish) (44, 45) and have suggested that odor quality emerges earlier than pleasantness in the odor processing stream. To test this at the neural level, we isolated the temporal dynamics of quality-specific representation by examining the association between the brain and the quality RDVs while controlling for intensity, unipolar pleasantness, and unpleasantness. We then compared the temporal dynamics of the quality-specific representation with those specific to unpleasantness and pleasantness, each controlling for intensity.

Significant partial correlations were observed from 300 ms of odor onset for unipolar unpleasantness (rho = 0.49, corrected P = 0.0027) and from 500 ms for unipolar pleasantness (rho = 0.49, corrected P = 0.0062). These partial correlations continued to be significant for most of the time until the end of the analysis period (Fig. 3C). The perceived odor quality, after removing the effects of (un)pleasantness and intensity, also showed significant partial correlations between 500 and 600 ms of odor onset (Fig. 3C; rho = 0.63, corrected P < 0.0001 for 600 ms). For all these variables, the strongest partial correlations were observed at between 600 and 800 ms (Fig. 3C; 800 ms, rho = 0.70, corrected P < 0.0001 for pleasantness; 600 ms, rho = 0.70, corrected P < 0.0001 for unpleasantness; 600 ms, rho = 0.63, corrected P < 0.0001 for quality), indicating the same latencies where the strongest correlations were observed in the tRSA with zero–order correlation (Fig. 3B). Although the significant time windows were shorter and partial correlations for quality did not reach significance, possibly due to the large interindividual differences in subjective ratings, tRSA based on the decoding accuracies obtained by cross-subject decoding models and intersubject mean perceptual scores also showed similar results (SI Appendix, Fig. S4 E–G).

Taken together, these results suggest that the neural representation underlying olfactory perception emerges as early as 150 ms after odor onset (Fig. 3B), but it takes more time to evolve into a structure closest to the perception, which appears after 600 ms (Fig. 3 B and C). It is worth noting that the peak latency of the decoding occurs much earlier (350 ms, Fig. 2B) than the tRSA peak latencies (≥ 600 ms, Fig. 3 B and C). This precludes the possibility that higher correlations between the brain and perceptual RDVs during the later time period (≥ 600 ms) than those during the earlier time period (∼350 ms) are caused by a higher signal-to-noise ratio (SNR) in the brain RDV during the later period. During the early period, when the decoding accuracy is high but the correlation with the perception remains low, neural activity may represent low-level odor characteristics that are not directly associated with perception, such as response patterns of olfactory receptors. The results of partial correlation analysis also showed that the neural representations of different aspects of olfactory perception follow different time courses, with unipolar unpleasantness emerging earlier than unipolar pleasantness and quality, while unipolar pleasantness and quality emerged simultaneously (Fig. 3C). Thus, the current results support the hypothesis that the neural representation of unipolar unpleasantness emerges earlier than that of unipolar pleasantness, as suggested by previous electrophysiological studies examining vision (39, 40) and olfaction (41). This precedence of unpleasantness may provide adaptive advantages in quickly detecting potential dangers based on olfactory cues (25, 46). Meanwhile, neither the valence-centered hypothesis nor the object-centered hypothesis was fully supported, as onset of the significant partial correlation for quality (500 ms) was later than that for unpleasantness (300 ms), but not that for pleasantness (500 ms). Thus, should we conclude the temporal ordering of pleasantness and quality as unpleasantness first and quality and pleasantness later? Interestingly, the time at which the partial correlation coefficients reached their maxima were similar for all three features (600 ms), with pleasantness trending slightly higher up to 800 ms. The partial correlations for quality rapidly declined compared to those for pleasantness and unpleasantness. Such time courses may indicate that the timings of the “completion” of neural computation are similar, or even later for pleasantness and unpleasantness than quality. As we have noted, previous behavioral studies showed shorter reaction times for quality judgment compared to pleasantness judgment (44, 45). In the current study, we assessed neural processing stages prior to judgments. Future studies that measure both neural and behavioral latencies would lead to a deeper understanding regarding how olfactory perception unfolds. Such an approach could also be applied when studying olfactory processing under more naturalistic conditions, e.g., odors coming with contextual cues like knowledge of odor sources, which may alter the temporal order of olfactory processing.

Olfactory Information Was Dynamically Processed in Widely Distributed Brain Regions.

To gain insights into the cortical generators underlying odor representations, we estimated source activities and analyzed their correlations with sensor-level decoding accuracies and perceptual ratings (Fig. 4A). To identify brain regions representing odor information, for each time point, we examined the correlations between the sensor-level brain RDVs and source-level brain RDVs. This source–sensor tRSA would identify voxels exhibiting representational structures comparable to those computed based on concurrent sensor-level decoding (Fig. 2D, Upper Triangle). However, note that while sensor-level decoding was based on MVPA and could capture the spatial and temporal patterns of all 64 electrodes, the voxel-wise analysis used here would overlook representations comprising activation patterns originating from multiple brain regions or multiple time points.

In addition to source–sensor tRSA, to examine source activities underlying olfactory perception, we also conducted tRSA that related source-level brain RDVs with perceptual RDVs. Correlations between source RDVs and RDVs based on quality ratings were examined to reveal the source activities associated with perception in general (source–quality tRSA). Furthermore, to identify source activities associated with specific aspects of perception, partial correlations between source RDVs and each of the perceptual variables examined in the corresponding sensor-level tRSA (Fig. 3C) were analyzed. When relating source and perceptual RDVs, the analysis was limited to the significant time points in the corresponding sensor-level tRSA (Fig. 3C); otherwise, tRSA procedures were the same as those for source–sensor tRSA. Statistical significance was tested using a one-sided permutation test (H₀, correlation coefficient ≤ 0; H₁, correlation coefficient > 0) with false discovery rate (FDR) correction, and the resulting statistical values (rho and corrected P value for each voxel) were posted on an open database (47).

The resulting time series of the brain maps is shown in SI Appendix, Fig. S5. To save space, in Fig. 4, time points with similar brain maps are collapsed. Significant voxels at 100 ms in the source–sensor tRSA are shown on brain slices from the POA (SI Appendix, Fig. S6A), with a list of Montreal Neurological Institute coordinates and anatomical labels (SI Appendix, Fig. S6B). The brain regions representing odor information changed dynamically (Fig. 4B and SI Appendix, Fig. S5A). At 100 ms of odor onset, i.e., the latency when the sensor-level odor decoding first reached significance (Fig. 2B), localized areas in and around the POA, which is in the medial temporal lobe, and the anterior cingulate cortex (ACC) were significant in both hemispheres (Fig. 4B, 100 to 250 ms; SI Appendix, Fig. S5A, 100 ms; SI Appendix, Fig. S6). At 350 ms after odor onset, i.e., the latency when the highest decoding accuracy was observed in the sensor-level tMVPA (Fig. 2B), relatively small areas in the right OFC and bilateral midcingulate cortex (MCC; Fig. 4B, 300 to 350 ms) were significant, with areas in the OFC extending bilaterally in the next 200 ms (Fig. 4B, 400 to 550 ms; SI Appendix, Fig. S5A). Interestingly, significant areas became largest at 600 to 850 ms after odor onset (Fig. 4B and SI Appendix, Fig. S5A), around the time when sensor-level tRSA showed the highest correlation between the brain and perceptual structures (Fig. 3 B and C). During this period, broad regions known to be involved in emotional (limbic areas, OFC, insular in both hemispheres), semantic (Broca’s area, lateral part of the left temporal lobe), and memory processing (bilateral parahippocampus [PH]) were significant in addition to the POA. After 900 ms, significant areas became right lateralized and shifted posteriorly to include the right fusiform areas and then gradually narrowed (Fig. 4B and SI Appendix, Fig. S5A, 900 to 1,350 ms), with only the voxels around the POA remaining significant at 1,500 ms (SI Appendix, Fig. S5A).

The spatiotemporal dynamics described above were similar to those for perceptual representation revealed by source–quality tRSA, except for the early latencies (SI Appendix, Fig. S5B, < 450 ms). In this early period, while relatively strong correlations were observed in the POA, OFC, and ACC in source–sensor tRSA (Fig. 4B and SI Appendix, Fig. S5A), only a small number of voxels with weak correlations were found in the source–quality tRSA (SI Appendix, Fig. S5B, 400 ms). Therefore, while regions significant in source–sensor tRSA after 450 ms of odor onset likely represent olfactory perception, those before 400 ms are less likely to do so. They may code odor characteristics other than perception, possibly lower-level features, which could not be specified in the present study.

Brain regions associated with specific aspects of perception (Fig. 4 C–E and SI Appendix, Fig. S5 C–E) were mostly within the regions associated with perception in general (source–quality tRSA, SI Appendix, Fig. S5B). For unipolar unpleasantness, only small voxels were significant in the earliest time window (300 to 400 ms), as in the source–quality tRSA. After 450 ms of odor onset, the significant regions became right-lateralized and included the POA, OFC, ACC, posterior cingulate cortex (PCC), PH, insular cortex, ventrolateral prefrontal cortex, and anterior temporal cortex (Fig. 4D and SI Appendix, Fig. S5D). For unipolar pleasantness, significant regions were relatively narrower and included the bilateral OFC and MCC (500 to 800 ms), bilateral PCC and PH (850 to 1,100 ms), and PH (1,150 to 1,500 ms), with small areas in the left temporal cortex throughout the analysis time window (Fig. 4C and SI Appendix, Fig. S5C). Perceptual quality after controlling (un)pleasantness and intensity were associated with Broca’s area (Fig. 4E), with a smaller number of voxels being significant in the right OFC, right temporal lobe, and left insular (SI Appendix, Fig. S5E).

These results show that brain regions that represent odor information change dynamically during olfactory processing. For the early time period, our results indicate that odor information, estimated based on sensor-level decoding accuracies, is represented in and around the POA at 100 to 250 ms (Fig. 4B and SI Appendix, Figs. S5A and S6), followed by those in the SOA (300 to 350 ms; Fig. 4B and SI Appendix, Fig. S5A). This finding is in line with a previous MEG study that found the earliest OERFs in the POA to be within 90 ms after odor onset, followed by those in the SOA, including the OFC and PH, which occur shortly thereafter (48). The early representation of odor information in the POA is also consistent with recent studies using intracranial EEG (iEEG) (49) and sEEG (50), which show that individual odors can be decoded ∼100 ms after odor inhalation from activities in the POA (iEEG) and the connectivity between the OB and the POA (sEEG). However, previous studies have not addressed whether olfactory perception is represented in the POA and SOA. In the current study, although representational structures in the POA and SOA are significantly correlated with those estimated based on the decoding accuracies, they do not show a significant correlation with perception in either the POA or SOA until 450 ms (Fig. 4D and SI Appendix, Fig. S5 B and D). Therefore, our results indicate that the activities in the POA and SOA that we observed before 450 ms possibly represent the initial odor information that is not associated with perception.

For the later time period (≥ 450 ms), the current study shows that brain regions representing olfactory information spread to wider brain regions associated with emotional, semantic, and memory processing (SI Appendix, Fig. S5 B–E and Fig. 4 C–E). Odor pleasantness and unpleasantness are represented in similar areas (Fig. 4 C and D and SI Appendix, Fig. S5 C and D), and a quality-specific representation for a duration of 500 to 600 ms after odor onset was found in and around Broca’s area. To the best of our knowledge, no previous studies have examined the spatiotemporal dynamics of odor representations in the whole brain during this period. Spatially, the regions shown to represent odor information in the current study are known to be involved in odor processing in general (51), the hedonic aspect of odors (52), and odor identification (53), as well as having functional connectivity with the POA (15). The dominance of the right hemisphere seen in our results (Fig. 4 and SI Appendix, Fig. S5) was also consistent with a meta-analysis of human neuroimaging studies on odor hedonic judgment (54, 55). We did not expect the involvement of semantic circuity a priori, as the experimental task during the EEG recordings was not semantic and the quality rating using semantic descriptors was performed on a different day after the EEG recordings (Fig. 1A). However, considering that olfactory perception could implicitly involve semantic processing of odors, such as odor identification or naming, the involvement of a semantic system such as Broca’s area, especially in the quality representation, is reasonable. Thus, our results confirm those from earlier studies on the spatial aspects of neural activity during olfaction and complement them with temporal data showing the ordering of spatial activities representing odor information. Specifically, our study suggests that the initial odor information that is not associated with perception is represented in localized areas in and around the POA 100 ms after odor onset and in the SOA at 300 ms, which evolves into perception while rapidly involving more posterior regions associated with emotion-, semantic-, and memory-related processing over the next several hundred milliseconds, with these brain regions fully representing olfactory perception 600 ms after odor onset and thereafter.

Limitations.

Some caution should be taken in interpreting the current results. First, to compensate for the low SNR of data, we used a fixed-effects model (FFX) for tRSA (56). In olfactory studies, conducting a large number of trials is challenging, as long interstimulus intervals are required to avoid adaptation and a small number of trials results in low SNRs. Therefore, other olfactory studies that conducted RSA have also used FFX (13, 25, 57). To confirm that outliers did not bias tRSA, we replicated the tRSA using rank-transformed RDVs (SI Appendix, Fig. S4B; SI Appendix, Method for details) and obtained very similar results (SI Appendix, Fig. S4 C and D for sensor-level tRSA and SI Appendix, Fig. S7 for source-level tRSA). However, as inferences based on FFX are only valid for the current group of subjects, more studies are required to generalize our findings. Second, in the current study, odors were presented monorhinally to the right nostril. A previous OERP study that examined the effect of odor presentation side (right or left nostril) on source activities found that activated areas were lateralized to the ipsilateral hemisphere in 284 to 388 ms after odor onset (58). Therefore, current results showing right lateralized activities might be due to the side of the nostril, especially in these early latencies. Finally, sEEG has inherently poor spatial resolution. Whereas standardized low-resolution brain electromagnetic tomography (sLORETA) is known to be sensitive to activities at deep sources such as the POA, the estimated source activities are spatially dispersed (59, 60). Thus, further studies using methods such as iEEG are needed to increase spatial resolution.

Conclusion.

In summary, using tMVPA, tRSA, and source estimation on scalp-recorded OERPs, the current study showed how neural representations of odors evolve in time and space. The pairwise decoding accuracy reached significance at 100 ms after odor onset and became a maximum at 350 ms, which suggests a very rapid emergence of neural representation of odor information. The decoding analysis also allowed us to estimate the neural representational structure of odors for every time point (Fig. 2). The tRSA relating the neural and perceptual representational structures revealed that neural representation underlying unipolar unpleasantness started earlier (300 ms) than that underlying unipolar pleasantness and perceived quality (500 ms) and evolved into a structure closest to the perception approximately 600 ms after odor onset (Fig. 3). The tRSA on source activity showed that brain regions that represent odor information changed during the course of olfactory processing, starting with localized areas in the POA at 100 ms, which soon spread to the OFC and cingulate cortex, and then, by the time the sensor-level decoding accuracies were maximally correlated with perception (≥ 600 ms), to larger areas associated with emotional, semantic, and memory processing (Fig. 4). These findings indicate that at the early stage of olfactory processing (< 350 ms), initial odor information is coded in and around the POA and then evolves into perceptual realizations (300 to > 600 ms) through dynamic computations in widely distributed cortical regions, with different perceptual aspects having different spatiotemporal dynamics. Our findings show that a multivariate analysis is effective at extracting rich spatiotemporal information from OERPs. Although olfactory dysfunction and modulation of the brain responses to odors have been reported as early harbingers of neurodegenerative diseases (e.g., Parkinson’s and Alzheimer’s diseases) (10, 11, 61), the spatiotemporal characteristics of brain activities associated with pathologies, and the impaired information contents, are still not well understood. Our adopted approach is potentially useful in studies conducted to further elucidate the neural basis of olfaction, where spatiotemporal data have been limited.

Materials and Methods

An overview of the methods is provided herein, with details of all sections provided in SI Appendix.

Subjects.

Twenty-six subjects (age: 19 to 28 y; 13 females) participated in the study with monetary compensation for their participation. Among them, two subjects who did not respond to a task or sense odor in more than half of the trials (15 trials) for at least one odor and two subjects who did not complete the experiment due to physical conditions were excluded from the analysis. Consequently, 22 subjects remained (age: 19–28 y; 11 females). Written informed consent was obtained from the subjects before conducting the experiments. The study was approved by the ethics committee of the University of Tokyo and in accordance with the Declaration of Helsinki.

Odor Selection.

We selected 10 odors from 138 monomolecular odors listed on a database compiled by Dravnieks, in which the applicability of 146 semantic descriptors based on ratings of 120 to 140 panelists was reported for each odor (62). Consequently, the following odors were selected: allyl caproate (Ally; 10%; TCI), fructone (Fru; 10%; TCI), citral (Cit; 1%; TCI), linalool (Lin; 1%; Santa), vanillin (Van; 8%; TCI), acetophenone (Ace; 1%; TCI), hexanal (Hex; 1%; TCI), alpha-pinene (Pin; 10%; Sigma), cyclodithalfarol (Cyc; 1%; TCI), and 4-pentanoic acid (4Pe; 10%; TCI). The odors were diluted in propylene glycol (T. Hasegawa Company, Tokyo, Japan).

Odor Delivery.

Odors were presented in a computer-controlled setup using an olfactometer (OL022; Burghart Messtechnik GmbH; SI Appendix, Fig. S1 A and B). Odor onset time (time = 0) was defined as the time point at which odor concentration measured at the nose outlet using a PID reached 10% of the maximum value, which was 60 ms after valve-switching (SI Appendix, Fig. S1C).

Experimental Procedures.

The experiment consisted of five sessions conducted over four days (Fig. 1A): an EEG session, an intensity rating session, a pairwise similarity rating session, a trigeminal test session, and a quality and unipolar (un)pleasantness rating session (SI Appendix, Methods). The order of the sessions was as shown in Fig. 1A, and the same across subjects. The intersubject mean ± SD of the interval between day 1 and day 4 was 18.0 ± 7.0 d.

EEG Acquisition.

The EEG signals were obtained using 64 active Ag–AgCl electrodes placed according to the international 10–20 system (BioSemi Active Two, BioSemi GmbH, Amsterdam, NL). A standard BioSemi cap was used for electrode placement. In addition, BioSemi active electrodes were placed at the following four locations: the right and left mastoids for later offline referencing, and below and above the left eye to obtain the electrooculogram. The signals were recorded at a sampling rate of 2,048 Hz.

EEG Preprocessing.

EEG data were analyzed using EEGLAB (version 13.6.5b)—an open-source toolbox for EEG data analysis (63)—and custom-written MATLAB scripts unless otherwise stated. For details, see SI Appendix, Methods.

Source Reconstruction.

We estimated the electrical activity in the brain using the sLORETA software. Brain activity was estimated as the standardized current density power for each of the 6,239 isotropic (5 × 5 × 5 mm) voxels for every time point after odor onset.

Statistical Hypothesis Testing.

All statistical tests were performed using MATLAB functions (Statistics and Machine Learning Toolbox, R2021b), with the number of observations equal to the number of subjects (n = 22), unless otherwise stated. We report the specific method for statistical testing, multiple testing correction, and alpha level used for each analysis in the legends of figures where corresponding results are presented.

tMVPA.

To assess the temporal dynamics of the neural representations of odor information, we performed tMVPA decoding of individual odors from EEG time–series data (Fig. 2A and SI Appendix, Fig. S3). An ${\mathrm{\ell }}^2$-regularized linear least-squares classifier was used for pairwise decoding, and an ${\mathrm{\ell }}^2$-regularized multinomial logistic regression classifier was used for multiclass decoding. We employed two approaches, one based on a subject-wise model, and the other based on a cross-subject model. In the former approach, the decoding models were built separately for each subject and then group-level inference was performed by combining the subject-wise decoding performances of all the subjects. This is the most commonly used approach in cognitive neuroscience (38, 64) and has the advantage of allowing the flexibility to conduct individually tailored analysis. In contrast, in the cross-subject approach, the decoding models were generalized across subjects. Therefore, whether and when brain signatures of odors common across subjects exist can be addressed using this approach.

tRSA.

To examine how the sensor-level, source-level, and perceptual representations of odors relate to each other, we conducted an RSA (26, 27) in a time-resolved manner (Fig. 1F). In the RSA framework, a representational structure of a given concept (e.g., odor) in a given space (e.g., neural space) can be defined as an RDV, which consists of pairwise distances of all pairs of samples of that concept (e.g., odors used in the experiment) in that space. Then, similarities between different representational structures, such as odors in perceptual space vs. those in neural space, are examined as correlations between RDVs.

In the current study, representational structures were defined based on the data for all the 45 possible odor pairs, and thus each RDV comprised 45 elements (specific variables used are explained in SI Appendix, Methods). When RDVs were constructed based on decoding accuracies or trigeminal test scores, we replaced the values under the chance level (50%) with 50%. As recommended for RSA (27, 56), relationships between representational structures were examined using Spearman’s rank correlation.

Data Availability

EEG data, subjective ratings, code, and scripts have been deposited in Zenodo (DOI: 10.5281/zenodo.6387085).

Change History

May 24, 2022: Due to a production error, Fig. 4 appeared incorrectly; Fig. 4 has been updated.