Contextual modulation of sound processing in the auditory cortex

Abstract

In everyday acoustic environments, we navigate through a maze of sounds that possess a complex spectrotemporal structure, spanning many frequencies and exhibiting temporal modulations that differ within frequency bands. Our auditory system needs to efficiently encode the same sounds in a variety of different contexts, while preserving the ability to separate complex sounds within an acoustic scene. Recent work in auditory neuroscience has made substantial progress in studying how sounds are represented in the auditory system under different contexts, demonstrating that auditory processing of seemingly simple acoustic features, such as frequency and time, is highly dependent on co-occurring acoustic and behavioral stimuli. Through a combination of electrophysiological recordings, computational analysis and behavioral techniques, recent research identified the interactions between external spectral and temporal context of stimuli, as well as the internal behavioral state.

At the first stage in the auditory processing cascade, the cochlea decomposes the incoming sound waveform into electrical signals for distinct frequency bands, creating a frequency-delimited organization that persists throughout the central auditory processing centers. The inferior colliculus, the auditory thalamus and the auditory cortex all exhibit tonotopic organization through a systematic neuronal best frequency gradient across space. Therefore, tonotopy has been considered a fundamental feature of auditory processing and, historically, auditory neuroscientists used pure frequency tones to systematically characterize the response properties of the auditory system. However, while pure tones are useful for determining the tuning properties of individual cells and the tonotopic arrangement of different brain regions, they ultimately do not capture the complex spectral profile of many natural sounds. Natural acoustic stimuli like speech, conspecific vocalizations, and environmental sounds, are comprised of signals with power across multiple frequency bands. Encoding complicated spectral profiles is behaviorally important, as these types of sounds provide cues for identifying different speakers and call types and for sound localization. However, how a complex sound is encoded is not immediately evident by looking at the responses to individual frequency components: rather, responses to distinct spectral components of sounds interact with each other in frequency and time. From moment to moment, a neuron’s response does not necessarily reflect only the frequency band it is best tuned to, but also depends on nonlinear integration of stimulus power across the spectral and temporal bands. Furthermore, behavioral engagement or task can modify this representation. Here, we review recent investigations on how spectral, temporal and behavioral contexts affects sound representation in the auditory cortex (Figure 1).

Figure 1.

Schematic of auditory context effects. Spectral context. The effects of spectral energy in near and distant frequency bands on characteristic frequency responses, as demonstrated with two-tone suppression and harmonic facilitation. Temporal context. The effects of preceding tones on a probe stimulus, as demonstrated by forward suppression and related to SSA. Spectrotemporal context. The joint effects of energy distributed across frequency and time, often resulting in adaptation of nonlinear response properties to suit persistent environmental statistics. Behavioral context. The effects of reward contingency on auditory responses.

Modulation of Auditory Processing by Spectral Context

Indeed, in the central auditory pathway, neural response properties to spectrally complicated stimuli are not well predicted by their tuning to pure tones. In the periphery, auditory nerve fibers typically transmit a linear, narrow-band representation of pure tone stimuli that is determined by their frequency tuning [1]. However, when presented with pairs of pure tones, auditory nerve fiber responses at best frequency are often suppressed by the presence of a second tone, a well-studied phenomenon called two-tone suppression (Figure 1, 2A; for review, see [2]), which arises from nonlinearities in the mechanics of the basilar membrane of the cochlea [3–5]. Many cortical neurons also nonlinearly integrate spectral components, showing multi-peaked tuning [6–9], two-tone suppression and facilitation (Figure 2A) [10–12], or combination sensitivity [13,14] when presented with sounds composed of multiple frequencies. This selectivity for complex spectral stimuli is thought to arise from a combination of excitation and lateral inhibition, as belied by the suppressive effects of multiband stimuli on single-peaked neuronal responses [10,15]. Thus, rather than combining responses to inputs at different frequencies in an additive fashion, the auditory system facilitates non-linear interactions across spectral bands.

Figure 2.

Examples of spectral and temporal context. A) Two-tone suppression. Top: schematic of stimuli used in two-tone suppression experiments, consisting of a reference tone presented at characteristic frequency (CF; S1, black bar) presented alone and in the presence of a competing tone (S2, grey bars). Bottom: Change in firing rate of an example neuron between presentations of S1 alone (S1: 1.47 kHz, 50 dB SPL) relative to S1 presented with S2 stimuli of varying frequencies (S2:.12–5.88 kHz, 70 dB SPL). Note suppressive effects at nearby frequencies, but facilitative effects near the first harmonic (3 kHz). Dotted and dashed lines represent the response to S1 alone, and the characteristic frequency, respectively. Figure adapted from Kadia and Wang, 2003 [12]. B) Forward suppression. Top: schematic of stimuli used in forward suppression experiments, consisting of masker tones of varying frequency (grey bars) followed by a probe tone typically presented at CF (black bar) at variable delays. Bottom: Frequency response areas (FRAs) of an example neuron in response to the masker (left) and the probe (right) as a function of masker frequency relative to CF and masker level. Note the suppression of spiking in response to the probe when preceded by masker tones that elicited strong responses, such that forward suppression roughly resembles the FRA of the neuron. Figure adapted from Scholl et al., 2008 [30].

Sensitivity to spectral context is useful for encoding sounds composed of several distinct frequencies, a feature common to many mammalian vocalizations [12,13]. Many communication sounds contain harmonic components, a broadband acoustic feature that is highly perceptible by many mammalian species [16–18]. Indeed, harmonic features are perceptually useful, and can be used to discriminate between different sound sources or speakers [19] or to hear vocalizations in noisy environments [20], indicating that harmonicity is a prominent acoustic feature for auditory processing. In auditory cortex, single- and multi-peaked neurons are often suppressed or facilitated by harmonically-spaced tone pairs (Figure 2A) [12] and can be selective for higher order harmonic sounds [21–23] demonstrating that auditory cortex is highly sensitive to the harmonic content of natural stimuli, possibly through a harmonic arrangement of alternating excitatory and inhibitory inputs [23]. These studies demonstrate that spectral processing in the auditory system combines a linear, tonotopic representation of frequency with a nonlinear representation, which creates sensitivity to features of the spectral context outside of a neuron’s best frequency.

Modulation of Auditory Processing by Temporal Context

Just as the spectral context outside of the best frequency is integrated in cortical neurons, the temporal history of an acoustic waveform also impacts neural responses to proceeding stimuli. Sensitivity to temporal context is important for identifying auditory objects, allowing sequences of auditory stimuli to be perceptually grouped or separated based on their temporal properties [24,25] or for detecting novel or rare sounds by decreasing responsivity to redundant sounds [26,27].

In the auditory cortex, responses to a probe tone are suppressed by a preceding masking tone, a phenomenon known as forward suppression (Figure 2B) [11,28–31]. The magnitude of forward suppression depends on the frequency and intensity of the masker, creating a suppressive area that matches the frequency response area (FRA) of the neuron, and decays at large delays between the probe and masker, approximately 250ms after masker onset [11,28,29]. The suppressive effect of the masker is released with increasing probe intensities, suggesting a competitive interaction between excitatory responses to the probe, and delayed inhibitory responses to the masker (Figure 2B) [30]. Whole-cell recordings show that inhibitory conductances elicited from the masker last only 50 to 100ms, indicating the involvement GABA-mediated synaptic inhibition at short timescales, but also that long-term synaptic depression may underlie suppression observed at longer delays [32]. Notably, there is considerable diversity in forward masking in awake mice, with mixtures of suppression and facilitation by the masker, and nonlinear relationships between responses to the masker and the probe, further implicating synaptic inhibition mediated by cortical interneurons as a potential mechanism for temporal context sensitivity [31].

However, prolonged stimulus history also affects neural sensitivity. Stimulus specific adaptation (SSA) is one such phenomenon, in which neurons reduce their response to a tone frequently repeated over several seconds, but do not suppress their response to a rarely presented tone of a different frequency [27,33–37]. SSA typically occurs over the course of several seconds [35–37] and can be elicited by tones whose frequency difference is an order of magnitude smaller than typical cortical and auditory nerve tuning widths [33,36]. Curiously, this phenomenon results in frequency hyperacuity in single neurons which matches the psychophysical acuity of untrained humans in a frequency discrimination task [38]. Converging evidence suggests that SSA is mediated through a combination of feedforward synaptic depression and intra-cortical inhibition [27,39]. Parvalbumin-positive (PV) and somatostatin-positive (SOM) interneurons differentially contribute to SSA, with PVs providing non-specific inhibition to the frequent and rare tones, while SOMs selectively inhibit frequent tones in a manner that increases over time [27]. Taken together, these findings outline a key role for interneuron-mediated synaptic inhibition in adaption to temporal context, providing a neural mechanism by which sound sequences and temporal events are encoded in auditory cortex.

Spectrotemporal Context: Adapting to Noisy Environments

So far in this review, spectral and temporal context have been treated separately, although there is a clear interaction between frequency and time in forward suppression [11,28–31]. This interaction is well documented, and can be made explicitly clear by quantifying spectrotemporal receptive fields (STRFs), in which neural responses are characterized by their sensitivity to sound intensity over both time and frequency [40,41]. Notably, incorporating local temporal and spectral stimulus context to an STRF-based model of neuronal responses to sounds not only improves predictions of activity, but suggests that context-specific changes in gain may underlie previously observed context effects such as two-tone or forward suppression [42,43]. This sensitivity to the local effects of frequency and time likely facilitates responses to combinations of spectrotemporal features [13,14]. Indeed, in nearly all natural soundscapes, the auditory background contains high order temporal and spectral characteristics [44,45]. To hear in these complex natural environments, it is necessary to encode the spectrotemporal statistics of the acoustic context to separate auditory streams, or, similarly, hear in the presence of background noise.

As discussed previously, stimulus-specific adaptation allows auditory cortex to reduce sensitivity to redundant stimuli [27,33,36]. Adaptive mechanisms are not limited to pure frequency, but can also account for broadband stimulus statistics over time, including stimulus volume [46] and contrast [47,48]. When presented with stimuli distributed at different volumes, neurons in the inferior colliculus (IC) adjust their rate-level functions to maximize sensitivity to sounds at the presented volume. On a population level, this is reflected by shifts in the mutual information contained in the rate-level responses towards intensities that are most commonly presented [46]. In auditory cortex, neurons adapt not only to stimulus level, but also the dynamic range, or contrast, of the stimulus spectra over time (Figure 3A). To do this, neurons adjust the dynamic range of their response gain to nearly match the dynamic range of the stimulus (Figure 3B) [47,48]. Notably, gain adaptation accounts not only for contrast over spectral space, but contrast over time, in broadband stimuli with varied temporal correlations (Figure 3C, D) [49], creating invariance to changes in temporal modulation rate, commonly seen in speech and other natural sounds. Through these adaptive mechanisms, the auditory system adjusts nonlinearities in its response properties to account for consistent spectrotemporal statistics in broadband stimuli.

Figure 3.

Gain adaptation to spectrotemporal context. A) Spectrograms of dynamic random chord (DRC) stimuli used to manipulate stimulus contrast. σ_L and c denote the width of the uniform distributions from which tone levels were sampled, analogous to the spectrotemporal contrast in each DRC stimulus. Contrast increases from left to right, and the level of each tone in dB SPL is denoted by the color bar on the right. B) Nonlinear fits (grey lines) to predicted linear responses versus the observed rate responses to stimuli presented at low (blue), medium (green), and high (red) contrast levels. X·v denotes the convolution of the corresponding stimulus spectrogram (X) above with the normalized STRF (v) of the neuron. Note that the slopes of the nonlinear fits scale to maximize sensitivity in the range of intensity values present at each contrast level, demonstrating adaptation of neural dynamic range to account for the stimulus dynamic range. C) Spectrograms of DRCs generated with different levels of temporal correlation (TC) across adjacent chords. D) Nonlinear fits (solid lines) to predicted versus observed responses in low (blue) and high TC conditions (red). Response gain is high for stimuli with low TC, and low for stimuli with high TC. A and B adapted from Rabinowitz et al., 2011 [47], C and D adapted from Natan et al., 2017 [49].

Gain adaptation is useful in generating responses that are invariant to persistent statistics of auditory environments, a processing feature which may underlie our ability to hear in the presence of noise [48,50,51]. In the anesthetized ferret, responses to speech in noise become increasingly noise invariant from IC to AC, a phenomenon that correlates with estimates of level and contrast adaptation in each region [48]. Further work explicitly modeling these adaptive mechanisms in terms of subtractive synaptic depression and divisive normalization significantly improve AC response predictions and stimulus reconstructions from population responses to noise-corrupted stimuli compared to static LN models [50,52,53]. These results indicate that, in environments with persistent background statistics, neural adaptation reduces responses to the background to optimally encode stimuli with different spectrotemporal profiles.

It is worth noting that many natural environments, such as a cocktail party or loud crowd, have noise backgrounds that fluctuate, or are a superposition of stimuli with statistics similar to the signal of interest. For complicated sounds, such as vocalizations, sensory representations are modified between primary and secondary auditory areas, generating invariance to acoustic distortions of these complex signals [54,55]. Similarly, in high-level human and avian auditory areas, responses to vocalizations embedded in multi-speaker choruses are background invariant, and strongly reflect behavioral detection of the attended speaker [56,57]. These mixtures of complex background sounds often contain coherently modulated power across different frequency bands [44]. This general property of auditory scenes may underlie co-modulation masking release (CMR), a psychophysical phenomenon in which co-modulated background noise facilitates the detection of embedded signals [58]. When presented with tones in co-modulated noise, auditory cortex strongly locks to the noise modulation envelope, but increases sensitivity to embedded tones by suppressing noise locking during the tone, providing a potential neural substrate for CMR, and behavioral detection of sounds in complex environments [44,59,60].

Modulation of Auditory Processing by Behavioral Context

So far, we discussed how external stimulus context affects auditory coding; however, what we hear in an acoustic scene depends not only on the spectrotemporal properties of the sound reaching our ears, but also on how our movements, attentional state and behavioral goals relate to those auditory inputs. An optimal auditory system generates stimulus representations that facilitate adaptive behaviors; as such, how coding of acoustic stimuli is modulated by behavior is of fundamental importance in our understanding of auditory processing.

The auditory cortex is highly sensitive to behavioral state, showing suppression mediated by PV interneurons during locomotion [61,62], an effect which is likely involved in auditory-motor learning [63] and the suppression of self-generated sounds [64,65]. During auditory tasks, intermediate, but not low or high arousal levels (as assayed by pupillometry, locomotion and hippocampal activity), improves the signal-to-noise ratio of sound-evoked responses [66], providing a neural substrate for the inverted-U relationship between arousal and task performance [67]. These findings suggest that auditory responses rely not only on the external sounds reaching the ear, but also on the behavioral and internal state of the subject.

Recent findings demonstrate that populations of neurons in auditory cortex maintain relatively stable receptive field parameters in response to a wide array of acoustic stimuli [68]. However, during active engagement in a behavioral task, cortical receptive fields change to maximize behavioral outcomes by improving encoding of relevant stimulus features. A popular paradigm to assay the neural effects of task engagement is to record stimulus responses during active (rewarded) and passive (non-rewarded or randomly rewarded) behavioral contexts. Using a trained tone detection task, Fritz and colleagues manipulated reward contingencies between passive and active behavioral contexts while recording from single units in auditory cortex. Estimating STRFs in each context revealed plasticity in the cortical representation of the target stimulus, such that responses to the target frequency were greatly elevated in the active context [69] while responses to non-target stimuli are depressed [70]. These behaviorally-driven changes in auditory coding also generalize to more complex sounds, such as tone sequences [71] and distorted speech [72]. Notably, these effects depend on the reward contingency, with enhanced responses to targets requiring behavioral avoidance, but suppressed responses to targets requiring approach behaviors [71,73], demonstrating considerable cortical adaptability to task demands while conserving neural discrimination of task-relevant frequency information.

While reward contingencies have a clear effect on tuning during frequency discrimination tasks, it was not clear whether these coding properties would hold for more complex stimuli. Using amplitude modulation (AM) discrimination tasks, Niwa and colleagues found that neural responses in AC increase both their firing rate and phase-locking to AM stimuli during the active behavioral context, thus increasing the amount of information about the attended amplitude modulations. While changes in rate and phase-locking are both correlated with behavioral performance, changes in rate are more affected by task engagement and are more correlated with behavior, implicating a multiplexed temporal- and rate-code that is dominated by firing rate changes during behavior (Figure 4) [74,75]. On a population level, neurons with similar AM tuning decrease their noise correlations during active engagement, while noise correlations in neurons with dissimilar AM tuning are unaffected, suggesting that AC selectively modulates population variability to maximize sensory discrimination [76]. Taken together, these results demonstrate that auditory cortex is highly plastic, rapidly adjusting its single-unit and population response properties to optimally encode stimulus features that are relevant to the current behavioral task.

Figure 4.

STRF plasticity during behavior. A) Approach task. During behavior, animals received rewards when licking during a target tone. Left: Population averaged STRF changes between active and passive conditions. STRFs were aligned to the frequency of the target tone. Blue indicates a suppression relative to the passive condition, while red indicates excitation. Note that during the approach task, there is a prominent suppression at the target frequency. Right: Cell counts indicating the population level STRF change at the target frequency. Blue bars indicate suppressed responses while red bars indicate enhanced responses. Filled bars indicate units showing significant modulation by behavioral context. Arrows denote the median change in significant units. B) Avoidance task. During behavior, animals received rewards when licking during reference noise bursts, but not during the target stimulus. Plots as in A). Note that during the approach task, there is significant suppression at the target, while during the avoidance task, there is significant enhancement at the target. Figures adapted from David et al., 2012 [73].

Conclusion

Natural acoustic scenes are highly complex, consisting of stimuli with complex spectral and temporal profiles that can occur in noisy environments and have different behavioral meanings. To handle this considerable complexity, our auditory system evolved considerable sensitivity to spectrotemporal and behavioral context. Beyond a simple spectral representation, the auditory system demonstrates nonlinear sensitivity to temporal and spectral context, often employing network-level mechanisms, such as cross-band and temporally adaptive inhibition, to modulate stimulus responses across time and frequency. Notably, as neuroscience employs more experiments in awake and behaving animals, it is important to consider the effects of behavioral context; arousal state and reward contingency have substantial effects on sensory responses, revealing highly plastic stimulus representations that optimize sensory discrimination depending on the task demands. On a circuit level, it is not yet known how the auditory system modulates responses to sensory and behavioral contexts, providing a rich avenue for future investigation.