Synaptic Plasticity as a Cortical Coding Scheme

Abstract

Processing of auditory information requires constant adjustment due to alterations of the environment and changing conditions in the nervous system with age, health, and experience. Consequently, patterns of activity in cortical networks have complex dynamics over a wide range of timescales, from milliseconds to days and longer. In the primary auditory cortex (AI), multiple forms of adaptation and plasticity shape synaptic input and action potential output. However, the variance of neuronal responses has made it difficult to characterize AI receptive fields and to determine the function of AI in processing auditory information such as vocalizations. Here we describe recent studies on the temporal modulation of cortical responses and consider the relation of synaptic plasticity to neural coding.

Introduction

Natural sounds such as speech and music are composed of acoustic signals that vary over time. Early lesion studies indicated that the auditory cortex is critical for recognition of temporal sequences of auditory stimuli [1,2], supported by newer stimulation studies in behaving rodents [3,4,5••]. However, it is unclear how AI represents and encodes sequences of temporally complex sounds. Relative to the primary visual cortex (V1), for example, less is known about the construction of AI receptive fields and the function of AI in acoustic scene analysis [6–8]. Knowledge of AI receptive field organization and dynamics is essential for understanding the neural basis of auditory perception and vocal communication, and for improvement of training programs and prosthetic devices designed to rehabilitate damaged brains, e.g., for recovery of language comprehension with cochlear implants after hearing loss [9–10].

The functional organization of AI has remained somewhat obscure. In part, this lack of information is due to the shortcomings of current analysis methods, such as reverse correlation and spike-triggered averaging (STA), to produce spectro-temporal receptive fields (STRFs) that accurately predict the responses of AI neurons over a wide range of stimuli (Fig. 1a, b). STA is a standard approach used to determine which features of a continuous, usually white-noise stimulus reliably produced spiking in a given cell [11]. While STRFs extracted from STA techniques using relatively stereotyped stimuli do well at predicting the responses to other similar stimuli such as pure tones or dynamic ripples [12–14,15•], these STRFs do not provide good predictions of responses to natural sounds. STRFs of AI neurons seem to be able to account for ~10–20% of the structure of spike trains evoked by vocalization patterns, even in awake macaques [15•], with a high variance depending on the form of stimuli used. This indicates that the predicted and actual responses to natural stimuli are only weakly correlated (linear correlation coefficient r: ~0.1). In V1, linear predictions are modestly better but still generally account for less than half the variance [16]. Importantly, STRFs are often best at capturing AI responses that slowly vary in time [12].

Figure 1.

STRFs of AI neurons. a, Two examples of STRFs from single units recorded in anesthetized adult cat AI. Recordings were obtained with a multielectrode from separate cortical layers in the same penetration. STRFs were derived via STA of spike trains evoked by dynamic ripple stimuli, with each evoked spike occurring at time 0. Red and blue colors indicate increases and decreases in firing rate, respectively, from the spontaneous rate. Data adapted from [13]. b, STRFs derived from responses to natural sounds, obtained with whole-cell recordings in anesthetized rat auditory cortex. Adapted from [12]. c, STRF prediction from left auditory cortex of awake macaque. Top, prediction spectrogram from convolution of STRF and stimulus spectrogram. Bottom, predicted (black) and actual response (gray), with amplitude of the predicted response scaled to the actual response. Adapted from [15•].

The rest of response variance is presumably due to factors that contribute to receptive field nonlinearity. To predict the neuronal response to sensory stimulation, a given stimulus is convolved with the STRF. Convolution is a linear operation similar to taking the moving average of the sensory stimulus, weighted by the time and frequency components of the STRF (Fig. 1c). If this convolution-based prediction closely matches the observed response, then the receptive field can be called ‘linear’, and different aspects of the stimulus (e.g., frequency and time) are generally independent from each other. Conversely, deviations from linear predictions reflect the presence of nonlinearities, such as interactions between different stimulus components or history-dependent processes.

There are many potential biological sources of nonlinearity for cortical responses. For example, the membrane potential threshold for spike generation is a nonlinear component of any neuronal response. In V1, spike threshold may play a role in determining whether cells exhibit simple or complex receptive field structure [17]. In the auditory midbrain of birds and mammals, threshold can also play a part in shaping feature selectivity [18,19]. Unfortunately, inclusion of spike threshold or other time-invariant (static) nonlinearities does not significantly improve STRF predictions of AI responses to natural stimuli [12,20••]. This suggests that more complex, time-varying nonlinear factors could contribute to tuning properties of cortical neurons [17,21••]. Here we discuss two such nonlinear phenomena- adaptation and plasticity- and their influence over the temporal dynamics of AI receptive fields. We consider three issues in detail: the involvement of short-term synaptic depression in spike train adaptation, the relevance of long-term synaptic modifications for receptive field plasticity, and the relationship between short- and long-term forms of plasticity.

Neuronal adaptation

Cortical neurons adapt in response to repetitive stimulation (Fig. 2a). Depending on the time interval and similarity between two stimuli, the number of spikes evoked by the second stimulus will usually be less than those evoked by the first stimulus. Here we use the term ‘adaptation’ to refer generally to this history-dependent reduction in response, regardless of what sort of response is experimentally measured, e.g., psychophysical detection, electroencephalography (EEG) signals, spike rates, or synaptic strength. Functionally, adaptation represents a fundamental form of physiological nonstationarity at the level of individual neurons, and at least one form of adaptation, contrast normalization of V1 responses, is believed to be a fundamental form of neuronal computation [22]. Neuronal adaptation is stimulus-specific, occurs on a time scale of hundreds of milliseconds, and builds up with an increasing number of stimuli [23–25]. Therefore, neuronal adaptation should be a major nonlinear determinant of the temporal pattern of cortical responses to sensory stimulation.

Figure 2.

Rapid adaptation of AI spiking and synaptic responses. a, Example of neuronal adaptation from cell-attached recording in anesthetized adult rat auditory cortex. Shown are spike rasters and peristimulus time histograms of spiking responses evoked by pairs of pure tones separated by various intervals. The first tone is presented at time 0. The spiking response was reduced at short inter-stimulus intervals, but the response recovered with longer intervals. Adapted from [27]. b, Example RRTF from a single-unit recording in anesthetized cat AI. This unit preferred repetition rates of ~10 Hz, with reduced response at both faster and slower rates of presentation. Adapted from [102]. c, Forward suppression of tone-evoked synaptic currents (top) and conductances (bottom) observed in a whole-cell voltage-clamp recording from rat auditory cortex in vivo. Adapted from [27]. d, Short-term depression of synaptic responses measured with paired whole-cell recordings in young mouse auditory cortical brain slices. Adapted from [103].

Adaptation in AI is usually studied either in the context of forward masking or in terms of measuring repetition rate transfer functions (RRTFs) of cortical neurons. Forward masking occurs when the perception of a second acoustic stimulus is limited or prevented due to interference by the first stimulus [26]. We follow the convention of Wehr and Zador [27] and reserve the term ‘masking’ to refer to psychophysical effects, while forward ‘suppression’ indicates the adaptation of neuronal responses in AI by pairs of auditory stimuli, as measured using electrophysiological techniques. Forward suppression is observed when a second stimulus is presented within a few hundred milliseconds of a first stimulus of similar nature, and thus represents a neural correlate of forward masking. In cat AI, for instance, the duration of forward suppression was highly stimulus-dependent [24]. When the two stimuli were pure tones of the same frequency, suppression lasted 150 msec, but when the tones differed by one octave or more, suppression was observed only for inter-stimulus intervals <50 msec. The input selectivity of suppression is comparable to stimulus-specific forward masking effects in human sensory perception [26].

RRTFs generalize the concept of forward suppression by quantifying the adaptation of spiking evoked by longer trains of stimuli presented at a constant rate (Fig. 2b). While some amount of adaptation occurs at subcortical stages of the auditory pathway, much of the response attenuation seems to be due to cortical filtering. In response to sequences of pure tones, examination of RRTFs revealed that single units in AI follow at considerably slower rates (~10 Hz) than neurons in the auditory thalamus or inferior colliculus (~100–300 Hz), suggesting that most adaptation is cortical in origin [25,27,28]. Perceptually, as forward masking is generally shorter than the time course of forward suppression in AI, rapid recovery from adaptation at subcortical stages may be sufficient to account for the psychophysical results. The functional significance of adaptation in AI at longer time scales is not yet clear, but presumably represents the filtering of redundant information by the cortex. Subcortical areas may be specialized for extraction of basic features of the acoustic environment such as spectral components, while cortical networks might instead process the temporal structure of complex sounds, e.g., those used in vocal communication [5••,15•,20••].

Studies of forward suppression revealed that the response of an AI neuron to an auditory stimulus strongly depends on the response to the preceding stimulus within hundreds of milliseconds. However, the spiking activity of AI neurons is also influenced by the stimulus history over a much longer time scale (on the order of seconds to minutes). The extent of auditory sensory memory can be studied by measuring the responses to ‘oddball’ stimuli. In this paradigm, pure tones of a standard frequency are repetitively presented, while deviant tones of a different frequency (oddballs) are intermittently played with some probability. Classically, these stimuli have been used in human EEG recordings to measure mismatch negativity, a pre-attentive change in cortical activity observable in anesthetized humans and animals [29,30] that indicates the detection of auditory events that occur with low probability [31,32].

In cat AI, standard tones produced weak responses when presented with high probability, but the oddball tones evoked responses of essentially the same vigor from trial to trial when the probability of their occurrence was relatively low [25,33]. The time course of adaptation was then determined by varying the presentation rate of the oddball stimulus. When the oddball probability was 0.5 (at a presentation rate of ~1 Hz), the time constant of adaptation was found to be ~48 seconds, but when the probability of their occurrence was 0.1, no adaptation was observed. For standard tones with presentation probability of 0.9, the time constant of adaptation was ~19 seconds, and included a second, fast component of adaptation that occurred from the first to second trials. These results imply that there are limits of the cortical ability to report stimulus likelihood in this manner, and that beyond a certain rate, stimuli are coded by the cortex as being essentially random [33].

Similar to earlier studies of forward suppression [24], the amount of adaptation observed in these experiments was dependent on the similarity between the standard stimulus and the oddball stimulus. Little adaptation was observed when the oddball and standard tones were separated by half an octave, but responses to the oddball stimulus were reduced as the two tones were moved closer together in frequency. This adaptive process can be considered as a memory trace of a change in sensory input that persists at least seconds to minutes, allowing AI neurons to track stimulus history over long durations. Surprisingly, such stimulus-specific adaptation was observed not only to tones of differing frequencies, but also to tones of different intensities. Using high intensity tones as the standard stimulus did not impede the response to lower intensity oddball tones of the same frequency, when the intensity of the two tones differed by only 16 dB [25]. In other words, AI circuits can treat sound intensity as a separate dimension of acoustic space, varying independently of other dimensions such as frequency [34,35,36••].

Adaptation effects can also be observed with more complex stimuli that more closely mimic the modulations present in natural sounds including human language. Psychophysically, it is more difficult to detect sinusoidal amplitude modulation (SAM) when the target stimulus is preceded by a masking SAM stimulus [37]. We recently recorded responses to such SAM stimuli in AI of awake squirrel monkeys [38•,39•]. AI responses were sensitive to the carrier frequency and responded differentially and heterogeneously to SAM stimuli with pure tone vs. two octave bandwidth noise carriers [38•]. When masking SAM stimuli preceded SAM probe stimuli, firing rates were reduced most strongly for SAM frequency that matched the masker SAM, thus mimicking the psychophysically observed bandpass behavior of modulation masking. In other words, the adaptation produced by more complex SAM stimuli was also highly stimulus-specific. Interestingly, cortical spike-timing synchrony with the modulation envelopes was much less affected than overall firing rate by the SAM masker [39•], indicating that neurons were still driven by specific features in the SAM stimuli, just with a lower probability of firing. Midbrain neurons are much less sensitive to modulation masking [40], indicating that this form of adaptation is at least partially cortical in origin.

Neurological studies indicate that adaptation is altered in several disorders, including autism and dyslexia. Mismatch negativity responses to oddball stimuli, measured with EEG recording, were found to be larger in autistic individuals than in control subjects [41], although other components of the EEG evoked by repetitive auditory stimuli were initially small in amplitude and showed little adaptation [42]. In dyslexics, on the other hand, forward masking was more profound and longer in duration relative to controls [43,44]; this deficit in temporal processing is hypothesized to be a main cause of language impairment in dyslexia and other speech disorders [43].

Short-term plasticity

The dynamics of adaptation are strikingly similar to the characteristics of short-term synaptic depression, a phenomenon common to most neural systems in which synaptic responses such as excitatory and inhibitory postsynaptic potentials (EPSPs and IPSPs ) or currents (EPSCs and IPSCs) decrease in size with repetitive stimulation [45]. Similar to adaptation of spiking, short-term depression occurs rapidly and recovers on the time scale of milliseconds to seconds. Depression is usually measured with intracellular or field potential recording, and is thought to be a consequence of a decrease in presynaptic transmitter release. As such, it is mainly a function of the input statistics and does not usually depend on the postsynaptic response. Short-term depression therefore allows cells to track changes in rate and variance independently for different stimuli [46]. As input rate increases, the size of synaptic responses is decreased, leading to an output rate that increases more slowly than the input rate. Consequently, depression is a regulatory mechanism for cortical gain and dynamic range of input-output transfer functions, enabling neurons to respond to a much broader range of stimulus rates.

The prototypical form of short-term synaptic plasticity is paired-pulse depression, in which the amplitude of the second of two electrically-evoked responses is smaller than the first (Fig. 2c). Some synaptic connections show paired-pulse facilitation instead of paired-pulse depression, where the amplitude of the second response is larger than the first. Whether responses initially facilitate or depress is believed to be a consequence of the initial transmitter release probability, with low probability connections facilitating and high probability connections depressing [45]. However, while synaptic responses can sometimes be facilitated with short trains of stimuli, depression is predominant over longer periods in cortical circuits [46].

Paired-pulse depression is a synaptic phenomenon caused by repetitive presynaptic action potential firing, observable in synaptic connections between pairs of neurons. This synaptic process may have important functional or perceptual consequences, although directly connecting observations at these different levels is challenging. In particular, paired-pulse depression due to repetitive electrical activation (Fig. 2c) may be a primary mechanism for forward suppression produced by repetitive sensory stimulation (Fig. 2d), although it is important to note that forward suppression is a network-level phenomenon that may involve changes to multiple synapses throughout feed-forward and recurrent pathways. Recently, intracellular sharp electrode and whole-cell recordings have been used to examine how stimulus-specific adaptation affected membrane potential responses and synaptic inputs in AI of anesthetized rats [27,47,48•]. In particular, in response to pairs of brief acoustic clicks separated by 64–512 msec, the second click-evoked response was approximately one-third to one-half the amplitude of the first response [27]. Only half of AI neurons showed strong suppression after pairs of clicks, with other AI neurons having facilitated responses to the second click. Regardless of the initial facilitation or depression, the synaptic responses of all neurons depressed after more prolonged sequences of stimulation. To longer sets of stimuli containing oddball/deviant stimuli, some degree of stimulus-specific adaptation was observed to tone-evoked synaptic responses in current-clamp recordings. This suggests that at least part of the adaptation is inherited by cortical circuits. Interestingly, Hershenhoren et al. [48•] reported that adaptation was stronger at the spiking than at the synaptic level, indicating that spike threshold is an intracortical contribution to overall amount of stimulus-specific adaptation. Essentially, small changes in EPSP amplitude can lead to large changes in average spiking activity if EPSPs that were just-suprathreshold become subthreshold. In a related study, Chen et al. [49•] made targeted whole-cell recordings from excitatory and inhibitory neurons of anesthetized mouse auditory cortex. Stimulus-specific adaptation was observed in membrane potential responses for all cell types, including delayed components of the auditory-evoked response. This delayed response in excitatory neurons was sensitive to blockade of NMDA receptors via MK-801 application through the whole-cell pipette, indicating that NMDA receptor activation might also an important nonlinearity for sensory processing and ‘oddball’ stimulus detection.

Even at relatively low repetition rates (~1 Hz), synaptic inputs are strongly suppressed at steady state. However, multiunit recordings in anesthetized rat [50] and single-unit responses to optimized stimuli in awake mouse and primate [36••,51] show that AI neurons can follow stimulation trains at much higher rates (beyond 10 Hz). How is it that neuronal output could adapt at a slower rate than the input? One possibility is that separate subpopulations of synaptic inputs are recruited at different times during a period of ongoing activity, particularly for the ‘best stimuli’ of each neuron. Alternatively, cortical neurons may not require much excitation to reliably follow moderate stimulation rates, as long as inhibition is kept relatively low (i.e., adapts at a faster rate and/or to a lower tonic level relative to the amount of excitation required to produce spikes). Wehr and Zador [27] measured the amplitude of tone-evoked excitatory and inhibitory conductances in rat AI neurons. They found that the extent of forward suppression of inhibitory input was equivalent to or even stronger than the suppression of excitation, leading to a large reduction in inhibitory strength with repetitive stimulation. In young mouse primary somatosensory cortex (S1), inhibition in layer 4 evoked by trains of electrical stimulation were depressed to a much lower level than excitatory responses [52]. Similarly, sharp electrode recordings from layer IV of young adult rat S1 showed that EPSPs recover from sensory adaptation faster than IPSPs [53•].

Adaptation is a fundamental feature of sensory neurons. Neuronal adaptation is apparent at different levels of observation- synapses, spikes, EEG, and behavior- but it remains unclear precisely how a synaptic phenomenon (short-term depression) relates to network-level adaptation (forward suppression) to control sensory perception (forward masking). Adaptation is expressed predominantly in cortex for repetition rates between 1–100 Hz, and may serve to filter out redundant information, heightening the probability of detecting unusual stimuli. The mechanisms of adaptation remain controversial: while short-term synaptic depression is likely to be involved in forward suppression and stimulus-specific adaptation, the role of inhibition is less clear. Regardless of the mechanisms, it may be straightforward to incorporate adaptation into AI receptive field analysis, as the dynamics of short-term depression and stimulus-specific adaptation can be captured in simple models with few free parameters [46,54]. Indeed, incorporating short-term depression into models of AI STRFs improves prediction quality on a range of stimuli including natural vocalizations [21••,55]. As a result, prediction quality was enhanced and explained up to half the variance of responses to vocalization-modulated noise [21••]. Therefore, accounting for rapid temporal dynamics improves predictions of AI coding, although there is seemingly room for further improvement, possibly if slower temporal dynamics are considered.

Long-term plasticity

In addition to neuronal adaptation at a relatively short time scale, cortical responses can be altered over a much longer time period. While forms of short-term plasticity such as paired-pulse depression may contribute to neuronal adaptation and forward suppression, other forms of synaptic plasticity such as long-term potentiation (LTP) and spike-timing-dependent plasticity (STDP) may also play important roles in determining cortical responses. In particular, STRFs might have slower timescales of adaptation on the order of minutes to hours, and long-term synaptic modifications might set the neuronal response patterns to behaviorally-important stimuli such as vocalizations. Thus failure of standard STRF approaches to describe some neuronal responses (or lack of responses) may be due to several factors, including long time-scale stimulus sensitivities, rapidly-induced modifications in cortical receptive fields triggered by test stimuli themselves, context-dependent responses, or responses to specific stimuli that do not have the same behavioral meaning when decomposed into lower level components (e.g., vocalizations consisting of FM sweep sequences).

Whereas repetitive presentation of the same stimulus reduces cortical responsiveness due to neuronal adaptation, certain forms of stimulation can induce receptive field plasticity and lead to long-lasting changes and enhancement of cortical responses. Long-term plasticity can be distinguished from adaptation and short-term plasticity by the mechanistic requirements for induction. Adaptation is thought to shape neuronal responses independently of postsynaptic responses, indicating that adaptive processes are highly input specific and can be expressed by individual neurons or single synapses. Induction of longer-term synaptic modifications, however, usually requires coordination between diverse network processes and the conjunction of pre- and postsynaptic activity, reflecting the involvement of additional circuit elements such as GABAergic inhibition, NMDA receptors, and subcortical neuromodulatory systems [56,57]. NMDA receptor-dependent plasticity has been observed throughout the cortex, and leads to functional and structural changes that seem to account for the longevity of synaptic modification after a transient induction event. In general, there are three main forms of long-term cortical plasticity: 1) pathological or compensatory reorganization after injury or prolonged sensory deprivation, 2) circuit wiring and refinement that occurs during developmental critical periods, and 3) synaptic modifications induced by conditioning or learning [9,57–60]. While the first case is unlikely to play a major role in determination of STRFs under normal conditions, recent experiments from our laboratory and others have shown that large, persistent changes in cortical synaptic strength can be rapidly induced by the latter two mechanisms within minutes depending on the stimulus pattern and activation of neuromodulatory systems.

Neuronal response properties change as cortical circuits mature and are remodeled over development [59,61]. The developmental trajectory of sensory cortex is profoundly influenced by patterned stimulation early in life. For example, the critical period for AI development can be artificially prolonged by exposure to continuous white noise [62], or shortened by exposure to pulsed noise [63] or pure tones [61]. These results indicate that developing cortical networks can reorganize in a way to match the statistics of structured sensory inputs, or remain in a labile state in the absence of such structure. However, given that these forms of passive exposure are usually insufficient to induce modification of the adult cortex, and that exposure regimes usually take place over days to weeks of patterned sensory stimulation, these phenomena are poor candidates for inclusion into STRF models.

Critical period plasticity is often conceived vis-à-vis Hebbian learning, in which temporally correlated spiking leads to long-term changes of synaptic strength and receptive field properties of cortical neurons. This hypothesis has received considerable experimental and theoretical support over the last two decades, with recent studies of STDP emphasizing the importance of precise spike timing in controlling the sign and magnitude of synaptic plasticity. STDP is a protocol for long-term synaptic modification, such that when presynaptic firing precedes postsynaptic spiking by <50 msec, LTP is induced, but when the postsynaptic cell routinely fires first, long-term depression (LTD) is induced. STDP has been observed at many different types of cortical synapses in vitro [57,60,64], in some cases, large changes in synaptic strength can be observed immediately after just one pair or a few pairs of spikes [65].

In vivo, STDP can be readily induced in rat V1 by pairing visual stimuli with postsynaptic spiking [66,67••]. In these studies, postsynaptic spikes are elicited via direct depolarization through the whole-cell recording electrode. Similar to results in brain slices, when the postsynaptic spike follows the stimulus-evoked EPSP, synaptic strength is increased, while when the postsynaptic spike precedes visual stimulation, EPSPs are reduced [66]. Recently, Pawlak et al. [67••] had the insight to examine spike generation as well. Cells that initially had subthreshold responses to visual stimulation could start spiking after LTP induction, while cells that reliably fired before post-before-pre pairing were rendered subthreshold (Fig. 3). Due to the nonlinearity of spike generation, even subtle changes at the synaptic level might then translate into substantial changes in receptive field organization.

Figure 3.

Stimulus-timing-dependent plasticity in V1. Top, subthreshold membrane potential response to flashed bar (grey box) before pairing with a postsynaptic spike after the initial peak (positive timing), and suprathreshold response to same flashed bar stimulus after pairing. Pairing was performed during down states (‘Dn-state stim’). Middle, number of spikes evoked per visual stimulus before and after pairing. Same cell as traces shown in top panel. Bottom, different neuron receiving negative spike timing to induce LTD (postsynaptic spike before the initial peak). Adapted from [67••].

The evidence for STDP in adult S1 and AI in vivo is less clear [35,68,69], perhaps because sensory stimuli tend to evoke strong, co-tuned inhibitory responses with short latency that limit NMDA receptor activation. Conversely, even in the absence of somatic action potentials, an important new study in mouse S1 showed that dendritic NMDA spikes were required for LTP of sensory-evoked synaptic responses [70••]. NMDA spikes were triggered by stimulation of efferents from the POm thalamic nucleus, and interestingly led to a cell-autonomous form of plasticity that was prevented by intracellular NMDA receptor blockade or direct hyperpolarization.

In general, the association of weak or non-preferred stimuli together with strong, preferred stimuli is also effective for altering receptive field structure [64]. One of the first studies of such ‘stimulus-timing-dependent plasticity’ found that repetitive presentation of a pair of oriented gratings, differing in orientation by 15 degrees, reliably shifted neuronal orientation tuning [71]. When the non-preferred orientation was presented first and followed within 40 msec by the preferred orientation, tuning curves were shifted in the direction of the non-preferred stimulus. Likewise, when the preferred orientation was presented first, tuning shifted away from the non-preferred stimulus. Such shifts in receptive field structure indicate that other inputs beyond those directly activated by the paired stimuli are remodeled to accommodate the new change in stimulus tuning; these lateral shifts of tuning properties can be captured by computer simulations incorporating STDP as a network learning rule [72]. Additionally, the timing dependence of these shifts is similar to that for STDP at V1 synapses in vitro and in vivo [64,66]. Other receptive field properties (size and direction selectivity) of V1 neurons were also modifiable by repetitive stimulus pairing. Correspondingly, pairing of visual stimuli at the millisecond timescale was found to affect human psychophysical performance on a visual discrimination task [71,73], suggesting that cortical modifications induced by stimulus pairing might support perceptual learning.

Stimulus pairing also alters frequency tuning curves in the auditory cortex. In anesthetized and awake ferret AI, presentation of a non-preferred tone shortly before a tone at the best frequency led to an increase in response to the non-preferred tone that lasted for several minutes; as in previous studies, the inter-stimulus interval during pairing was tens of milliseconds [74]. Reversing the temporal order, so that the best frequency was presented first, decreased the response to the non-preferred tone. Stimulus pairing was required to shift tuning curves in adult AI: if the non-preferred tone was repetitively presented by itself, no change in tuning was observed.

A general problem with trying to incorporate synaptic plasticity rules into STRF models is that it is difficult or impossible to determine what the relevant pre- and postsynaptic pairing events are for STDP induction in vivo. This is because most recording methods usually permit observation only of the activity of the postsynaptic cell, but a large number of potential presynaptic input patterns might produce a given postsynaptic response [75]. However, a recent study provides insight into what sorts of postsynaptic spike patterns might signify these events. Single-unit recording from cat V1 revealed that repetitive presentation of time-varying natural scenes (movies 3–30 seconds in duration) led to an improvement in the response reliability of cortical neurons, in terms of the cross-correlation between spike trains evoked by the same movie from trial to trial [76]. The increase in response correlation reflected a decrease in inter-trial variability of evoked spike trains, lasted for several minutes or tens of minutes even after presentation of other movies, and was strikingly specific for natural scene stimulation, as changes in spike train correlation were not observed for white noise or drifting grating stimuli.

This specificity seemed to be based on the spike train statistics produced by different sorts of visual stimuli. Natural scenes evoked spike trains that were relatively sparse, but contained bursts of spikes triggered by certain features in the movie [76]. Bursts of postsynaptic spiking (presumably following the requisite presynaptic input that generated these bursts within a few msec) are effective stimuli for inducing STDP. A few minutes of stimulation with natural scenes (which might be used mainly for examining sensory receptive fields) produced enough correlated activity to rapidly, selectively, and persistently modify cortical responses. The persistence of these effects (over minutes to tens of minutes) suggest that they operate through different mechanisms than described above for stimulus-specific adaptation (which operates on the timescale of milliseconds to seconds), and is consistent with recruitment of stimulus-timing-dependent plasticity and STDP. Therefore, although V1 spatio-temporal receptive fields (visual equivalents of auditory STRFs) capture most of the responses to short episodes of visual stimulation, it is not clear how well they will perform on longer stimulus repeats, where mechanisms of long-term plasticity might be engaged. As seemingly innocuous natural stimuli can rapidly induce plasticity and change cortical response properties in a non-trivial manner, these results highlight the need to consider synaptic plasticity as an important nonlinear component of receptive field temporal dynamics.

Thus despite the complexity of rules and mechanisms for long-term synaptic plasticity, studies of STDP have been instrumental in elucidating the precise spiking patterns that lead to changes in the strength of cortical synapses. By providing a basic unit for long-term modification- individual spike pairs- phenomenological models based on STDP have been able to successfully predict the sign and magnitude of changes in synaptic efficacy induced by complex spike trains, including natural patterns of spike trains from in vivo recordings [65,77–79]. In particular, spike bursts seem to be particularly effective for induction of synaptic and receptive field modifications [63]. Much like previous models of adaptation and short-term plasticity, these models of long-term plasticity contain a small number of free parameters, suggesting that they could be integrated with existing STRF models of cortical receptive fields [80].

Synaptic mechanisms of cortical receptive field plasticity

Some responses might be too sparse for any general STRF-based model to capture, especially if there is little correlation between responses to individual features and the overall stimulus. In particular, AI responses to natural sounds and vocalizations can be extremely specific and temporally precise (Fig. 4a), especially for responses to infant distress calls in the maternal auditory cortex (Fig. 4b). While there is considerable heterogeneity in the response to a particular vocalization from cell to cell, many neurons in anesthetized and awake rodents are exquisitely sensitive to aspects of natural sounds like ultrasonic communication signals or infant distress calls [5••,14,20••,81,82,83••,84••]. In rats these responses may be correlated with FM sweep sensitivity and ultrasonic best frequencies [20••], while in mice it seems that neurons sensitive to ultrasonic distress calls can be found throughout maternal AI and the rest of auditory cortex. In these cases, it may not be possible to predict the responses to complex, behaviorally-important stimuli from the responses to pure tones or other more elementary sounds in absence of behavioral context. It has recently been proposed that even at the earliest stages of cortical auditory processing, neuronal responses to these stimuli are categorical, and responses to complex stimuli encoded at the network level rather than by single cells [14].

Figure 4.

Responses to vocalizations in rodent AI. a, Single-unit recording from awake rat AI, playing two vocalizations with similar spectrotemporal characteristics. Top, vocalization waveforms, spectrogram, and power spectrum. Bottom, responses of the same unit to each vocalization. Note the reliable and precise response to vocalization 2 and negligible response to vocalization 3. Adapted from [20••]. b, Cell-attached recordings from anesthetized mother and virgin mouse AI. Top, infant isolation call spectrogram. Middle, three example trials. Bottom, raster of 12 trials to the same call. Spikes in yellow are approximately synchronous within ~10 msec. Adapted from [5••].

How does behavioral relevance shape and transform cortical responses to sensory experience? In general, changes in arousal or behavioral context, events that are surprising or rewarding, and other stimuli or experiences that affect physiological or hormonal state all act via diverse neuromodulatory systems that project extensively throughout the brain including primary sensory cortical areas. Neuromodulation is important for both short-term attentional processes as well as induction of long-term plasticity in vitro and in vivo [57]. Neuromodulators such as acetylcholine, norepinephrine, and peptides hormones like oxytocin each have complex effects on target neurons and networks. These systems are also extensively inter-related: in macaques, the cholinergic nucleus basalis expresses a high level of oxytocin receptors [85••], and a major input to the oxytocin-producing paraventricular nucleus comes from noradrenergic neurons of the brainstem [86]. Thus it is unlikely that these systems have monolithic functions (i.e., acetylcholine level may be more than attentional state, and oxytocin release is not necessarily just related to trust or social significance).

Despite this diversity, a major mode of action of several different neuromodulators is transient disinhibition. Many studies have highlighted the disinhibitory effects of acetylcholine. Metherate and Ashe [87] observed that stimulation of the cholinergic nucleus basalis could enhance EPSPs and reduce IPSPs in AI evoked by thalamic stimulation. Similarly, we observed that nucleus basalis stimulation reduced tone-evoked IPSCs (Fig. 5a), enabling cortical tuning curves to be modified and leading to long-lasting changes in auditory perception in trained rats [35,68]. Letzkus et al. [88] found that foot shock activated cholinergic inputs to layer 1 inhibitory cells in mouse AI, which silenced layer 2/3 interneurons (Fig. 5b) to enhance sensory responses to FM sweeps paired with foot shocks.

Figure 5.

Neuromodulation reduces AI inhibition. a, Nucleus basalis stimulation in adult rat AI rapidly reduces inhibition; excitation more slowly increases afterward. Adapted from [68]. b, Foot shock disinhibits mouse AI. Top, foot shock responses are reduced by nicotinic receptor antagonists. Bottom, foot shock transiently reduces spontaneous inhibitory events. Adapted from [88]. c, Locus coeruleus stimulation reduces ongoing inhibition in rat AI. Adapted from [90•]. d, Voltage-clamp recording from anesthetized virgin mouse AI. Pup call stimuli initially evoke imbalanced EPSCs and IPSCs. Pairing oxytocin with pup calls first reduced IPSCs, then strengthened EPSCs, and finally call-evoked inhibition balanced excitation after 30+ minutes. Adapted from [5••].

Kruglikov and Rudy [89] surveyed the action of ten different modulators in slices of mouse S1. Four of the modulators (muscarine to modulate muscarinic acetylcholine receptors, serotonin, adenosine, and baclofen to modulate GABA_B receptors) each selectively reduced GABAergic inhibition by reducing transmitter release from presynaptic inhibitory neurons onto excitatory cells. Perhaps surprisingly, norepinephrine did not reduce evoked inhibition in the same way, despite some functional similarities and interrelations between the cholinergic and noradrenergic systems. Instead, either electrical or optogenetic stimulation of rat locus coeruleus reduced tonic inhibition rather than stimulus-evoked inhibition (Fig. 5c), leading to much larger and longer-lasting changes in AI tuning curves [90•]. Therefore noradrenergic modulation can also act to control cortical inhibition in a fundamentally different and more general manner than acetylcholine. Specifically, acetylcholine reduced stimulus-evoked inhibition but may leave spontaneous inhibitory events unchanged, whereas noradrenalin reduced spontaneous inhibition but if anything, stimulus-evoked inhibitory currents were enhanced.

Building off these previous studies, we asked if modulatory-based plasticity could produce the sparse and specific responses to infant distress calls observed in maternal mouse AI. We recorded responses to ultrasonic infant distress calls in virgin mice that did not have prior experience with pups [5••]. Voltage-clamp recordings from anesthetized animals revealed that virgin AI neurons receive call-evoked EPSCs and IPSCs, but that there was little reliability in the response pattern from trial-to-trial, and the patterns of EPSCs and IPSCs was uncorrelated (Fig. 5d). Oxytocin- either pharmacologically applied or optogenetically released- reduced inhibition and enhanced call-evoked excitation within minutes. This reduction of inhibition led to long-lasting enhancement of call-evoked excitation and inhibition together over approximately an hour. As a result, excitatory and inhibitory responses became more precise and inhibition was more balanced with excitation, essentially transforming these virgin responses into maternal-like responses within minutes to hours [5••]. Similar disinhibitory effects of oxytocin receptor activation have also been observed in hippocampal slices [91•].

In maternal animals, disinhibition can also be produced via sensory cues. Cohen et al. [92] showed that pup odor modulates left AI responses in mouse dams and virgins experienced with pups. In a subsequent study, Cohen and Mizrahi [83••] then performed two-photon guided cell-attached recordings from layer 2/3 neurons, targeting either parvalbumin-positive inhibitory neurons or parvalbumin-negative (putative excitatory) neurons. Pup odor selectively increased the responses of putative excitatory neurons to pup call sounds, while selectively decreased the responses of parvalbumin-positive interneurons to these stimuli. Responses to pure tones were also found to be altered in parvalbumin-positive cells from maternal left AI, with the best frequencies of these inhibitory cells increased by more than one octave in dams relative to the best frequencies of parvalbumin-positive neurons in naïve virgins. As a consequence, the major effect of pup odor presentation was a reduction of maternal left AI inhibitory neuron firing at high (ultrasonic) frequencies, similar to the spectral content of pup distress calls [83••]. These remarkable experiments demonstrate that maternal experience can control state-dependent modulation of cortical responses, enabling auditory and olfactory cues from infants to synergistically enhance sensory processing for social cognition.

We hypothesize that transient, stimulus-specific disinhibition (due to cholinergic, noradrenergic, or oxytocinergic modulation) is permissive for NMDA receptor-dependent synaptic plasticity (e.g., STDP) in a restricted region of the cortical network. When AI neurons are disinhibited, sensory-evoked responses may be large enough to provide sufficient depolarization to activate NMDA receptors, leading to LTP of those inputs specifically engaged by the sensory stimulus. In this way, long-term receptive field modifications are induced, encoding the change in stimulus weight within the AI network. The temporal dynamics of inhibitory transmission would then serve as a synaptic memory trace of the pairing event [5••,68]. The duration of input-selective disinhibition permits reorganization of AI tuning curves to emphasize the new preference for the paired stimulus; under natural conditions, this memory trace would represent episodes or stimuli that have acquired new behavioral meaning. Accordingly, activation of neuromodulatory receptors is important for Hebbian forms of synaptic plasticity in brain slices [93,94•]. Spike-timing-dependent forms of excitatory and inhibitory plasticity can bind co-activated inputs together, normalizing the strength of inhibition relative to excitation in a manner that requires NMDA receptor activation [95••].

Suppression of inhibition therefore may be a general mechanism for induction of NMDA receptor-dependent receptive field modifications in the adult cortex. During developmental critical periods, the increased cortical plasticity may be due to a less-refined inhibitory tone, permissive for alterations of cortical networks by passive stimuli [59,61]. In adult cortex, however, neuromodulator release is required for long-term receptive field plasticity, reflecting the importance of behavioral context in associative learning and memory provided by subcortical systems [57]. Several recent studies have also demonstrated structural analogs to these processes, showing that changes to inhibitory neurons, dendrites and axons can precede detectable changes to excitatory cells. Such forms of rapid inhibitory structural plasticity seem to be important for hippocampal-dependent memory formation and ocular dominance plasticity in the visual cortex, and may also occur during perceptual learning in adults [96–98].

Stimulus-timing-dependent plasticity [73,74], as a model for perceptual learning in adult cortex, might operate by a similar principle. Excitation provided by the non-preferred stimulus might partially activate NMDA receptors, but these receptors would not be fully gated as this weak excitation is not sufficient to depolarize the postsynaptic neuron. This depolarization is instead provided by the subsequent large excitation evoked by the preferred stimulus. The time window for stimulus pairing to be effective would then be initiated by the kinetics of NMDA receptor activation driven by the non-preferred stimulus, possibly in a background of heightened excitability due to adaptation and weakening of intracortical inhibition after repetitive stimulation [27,53] rather than activation of neuromodulatory systems. The full extent of the mechanisms of perceptual learning, in general, and the role of disinhibition and STDP in stimulus-timing-dependent plasticity, in particular, remain to be determined.

Conclusions

AI receptive fields are fundamentally dynamic, tracking perturbations in streams of sensory input via the mechanisms of short-term plasticity, and sensitive to behaviorally relevant experiences over the lifetime of an animal via long-term modification. Plasticity at different levels can therefore be considered as an adaptive coding strategy for cortical circuits, in which receptive fields are adjusted as necessary to stay sensitive to unexpected or behaviorally important stimuli. Together, these findings suggest that careful manipulation of sensory stimuli could be a useful technique for non-invasive modification of cortical circuits, granting the experimenter a high degree of control over the structure of cortical STRFs. Indeed, it seems that a principle reason for the success of neuroprosthetic devices such as the cochlear implant is the ability of the nervous system to be modified around the structured inputs provided by these devices [9,10]. Future technological development of prostheses and training programs should capitalize on these advances in our understanding of the rules and mechanisms for cortical receptive field plasticity. For example, it may be possible to design auditory stimuli or cochlear implant stimulation patterns that recruit mechanisms of STDP or stimulus-timing-dependent plasticity. Alternatively, it may be more promising to use recording techniques and monitor attentional variables or arousal state of subjects (e.g., pupil dilation or skin conductance) during training procedures, or modify behavioral context and/or reward schedules to engage modulatory systems that might enable or boost plasticity in the cortex and throughout the brain [9,10,99–101].

It remains a major challenge to characterize the responses of AI neurons. Linear filters derived from STA methods fail to reliably predict the responses of AI neurons to arbitrary patterns of auditory stimuli. This failure seems to be due mainly to activity-dependent nonstationarity of cortical responses, as addition of static nonlinearities to receptive field models does not greatly improve response prediction by STRFs. Studies of the temporal dynamics of AI neurons are therefore critical for correcting and augmenting these models to produce STRFs that accurately account for cortical responses. Behavioral context is also a major determinant of cortical responsiveness. In vivo intracellular recordings have proved invaluable for determining the synaptic mechanisms of adaptation and plasticity, thereby placing biophysical constraints on the nonlinearities that contribute to AI STRFs. Given that the cortex is critically involved in learning and memory, it is not surprising that history- and context-dependent variables must be included in a complete model of AI function. Such a model, which would delineate the role of temporal processing and the duration of memory traces in AI neurons, is a necessary step towards determining the role of AI in auditory scene analysis and language processing.

Highlights.

• Adaptation occurs throughout the nervous system at different time scales.

• The relation between synaptic depression and forward masking is still unclear.

• Including synaptic depression into STRF models substantially improves predictions.

• STRF models with nonlinearities fail to predict responses to vocalizations.

• AI is highly plastic, and neuromodulation rapidly transforms vocalization responses.