Plasticity of population coding in primary sensory cortex

Abstract

That experience shapes sensory tuning in primary sensory cortex is well understood. But effective neural population codes depend on more than just sensory tuning. Recent population imaging and recording studies have characterized population codes in sensory cortex, and tracked how they change with sensory manipulations and training on perceptual learning tasks. These studies confirm sensory tuning changes, but also reveal other features of plasticity, including sensory gain modulation, restructuring of firing correlations, and differential routing of information to output pathways. Unexpectedly strong day-to-day variation exists in single-neuron sensory tuning, which stabilizes during learning. These are novel dimensions of plasticity in sensory cortex, which refine population codes during learning, but whose mechanisms are unknown.

Introduction

Sensory experience drives robust plasticity of sensory tuning and maps in sensory cortex. This well-studied process drives map development to match sensory statistics, and contributes to sensory perceptual learning [1], [2], [3]. But is there more to sensory cortex plasticity than changes in sensory tuning? Sensory areas use population codes that are based on coordinated spiking across many neurons. Large-scale population imaging and recording enable comprehensive analysis of population coding. Chronic longitudinal imaging allows plasticity to be directly observed, with cellular resolution, as it unfolds [4], [5], [6]. These methods provide new insight into neural coding and how it changes during plasticity. Recent studies confirm changes in sensory tuning, but also reveal plasticity in other aspects of population coding, including response gain and variability, firing correlations, and top-down modulation by task context. Here we review some key findings, which suggest novel sites and mechanisms for sensory cortex plasticity.

Plasticity of sensory tuning

In classical map plasticity, neurons adjust their sensory tuning to represent common or behaviorally relevant (i.e., reinforced) sensory features. This is confirmed by population imaging. In mouse V1, filtering out all but one visual orientation in juveniles increases the proportion of neurons tuned to that orientation [7]. In adults, monocular deprivation causes 60% of active neurons to shift ocular dominance toward the open eye, though a minority shift paradoxically to favor the closed eye [8**]. In S1, a large majority of neurons similarly shift whisker tuning away from deprived whiskers and toward a spared whisker [9]. Hebbian plasticity mechanisms are thought to underlie many of these changes. The ability of Hebbian plasticity to imprint information in sensory cortex has been shown decisively in V1, where optogenetic co-activation of L2/3 pyramidal (PYR) cells induces Hebbian-like ensembles that are spontaneously active, exhibit pattern completion, and are spatially mixed with visual-related ensembles [10*].

Cortical plasticity is also induced by training on sensory tasks. Sensory training often alters sensory tuning in neurons representing relevant sensory features [11], [12], [13], [1], [14], [15**]. These include shifts or expansions in tuning toward trained features [16] ,[1], [14] or sharpened selectivity that improves population-level discrimination [11], [12] (Figure 1A). Training can also shape more complex integrative tuning features, such as tuning for visual contours [17], [13]. In some cases, tuning changes are small or absent in primary cortex, but observable in higher sensory or sensorimotor areas (e.g., [12], [18]), or occur in primary cortex only transiently [16]. Thus, tuning changes in primary cortex are one mechanism, but not the only mechanism, for perceptual learning.

Figure 1. Four ways to adjust a population code in sensory cortex.

A, Systematic changes in sensory tuning by single neurons. Each curve is tuning of a different neuron, along a sensory feature axis. Neurons shift or sharpen their tuning to better represent common or reinforced stimulus features. B, Changes in sensory gain and response reliability. Each circle represents activity of a neuron on a given trial (filled: spiking, open: not spiking). Subpopulations of neurons are tuned for stimulus 1 or 2. Before training, response gain and reliability are relatively low, leading to poor discrimination on the population level. During learning, gain and reliability to the reinforced stimulus increase, increasing reliability of the population code. C, Reduction in day-to-day tuning variability. Under normal sensory conditions, sensory tuning of individual neurons changes from day to day. During learning, this variability decreases, so that population decoding becomes more stable and accurate. D, Routing of information on output pathways from primary sensory cortex. Intermixed subpopulations of neurons project to different output pathways. With learning, one subpopulation increases stimulus selectivity or responsiveness, thus routing information preferentially down one pathway.

Principles of population coding in sensory cortex

Sensation occurs on single trials, despite noisy spike data. Population codes are robust on single trials because they utilize statistical patterns of activity across large numbers of neurons. Both population spike recording and population imaging have been used to characterize population coding in sensory cortex. Here we focus on population imaging, which typically samples more neurons, often with cell type specificity, and has revealed several key features of population coding in sensory cortex.

First, single neurons in rodent cortex have low response probability and high trial-to-trial variability, but populations of ~100 neurons provide robust sensory information on single trials [19], [9], [20]. Second, primary sensory cortex represents not only sensory information, but also non-sensory task variables including movement, anticipation, and behavioral choice [21], [22**], [23], [24*], [15**], which may reflect top-down input. Third, spatially mixed PYR cell subpopulations project to different downstream cortical targets and carry different sensory and non-sensory information [25], [23]. Fourth, longitudinal imaging reveals substantial day-to-day variability in sensory tuning by single neurons, even under nominally constant sensory conditions. In S1, whisker somatotopic tuning is remarkably poorly correlated across days in single neurons [9], though other sensory tuning is more reliable [26], [20]. In V1, Rose et al. (2016) discovered pronounced day-to-day variability in both ocular dominance and orientation tuning. This is not measurement noise, because it exceeds the variability across trials on the same day. Tuning variability is similar over 4-day and 12-day intervals, and thus is not random drift, which would accumulate over days. Instead, it represents constrained variation near each neuron’s mean tuning [8**]. Why these basal dynamics exist is unclear. It could represent plasticity driven by ongoing experience, or random ‘noise’ related to maintaining synapses and circuits. Population coding may filter out this variability to achieve perceptual constancy, e.g. by ignoring the most labile neurons or by using highly redundant population codes [27*], [28]. Alternatively, it may represent active exploration of sensory representations for perceptual learning, similar to variability in motor systems for motor exploration [5], [29], [30].

Recent studies have identified experience-dependent changes in these and other features of population coding in primary sensory cortex.

Learning by changes in response gain and reliability

Sensory training can improve population coding by modulating sensory response gain and reliability (Figure 1B). Poort et al. [15**] trained mice to discriminate vertical from angled gratings in a virtual corridor to guide the decision whether to lick for a reward. Calcium imaging during behavior showed that with training, V1 neurons became more selective for grating orientation, largely driven by increased single-trial reliability and amplitude (gain) to the preferred orientation. This improved population coding of orientation, and correlated strongly with the animal’s behavioral discrimination ability. This learned improvement in population coding was reduced in interleaved trial blocks when orientation was task-irrelevant, or when mice were anesthetized. This suggests that top-down input representing the task-dependent salience of specific stimuli gates these learned changes in response reliability and sensory tuning.

Chen et al. [31**] trained mice to discriminate two textures, and imaged activity in S1 neurons over days of training. Learning gradually increased texture discrimination by S1 populations. For S1 neurons that project to secondary somatosensory cortex (S2p neurons), the learned improvement in stimulus discrimination was not due to shifts in stimulus selectivity by individual neurons, but reflected loss of touch sensitivity by some S2p neurons and gain of touch sensitivity by others across days of learning. This shifted population tuning for rough vs. smooth textures. Thus, learning gated which S2p neurons were active during the task, which may represent another form of gain regulation. When mice were not rewarded for discriminating, the proportion of cells that discriminated textures returned to pre-learning levels, suggesting that top-down task-dependent signals gated the improved population code [31**], similar to V1 [15**].

Learning by changes in firing correlations between neurons

Population coding is strongly impacted by noise correlations, which are co-variations in firing rate between neurons that are not due to shared sensory tuning. Correlated noise in similarly-tuned neurons impairs population coding because it mimics sensory-evoked signals. But when noise correlations are inversely related to tuning similarity, they can increase stimulus information at the population level [32], [33]. Several studies in high-level sensory areas demonstrate that learning can improve stimulus coding by altering noise correlations.

In the first study to examine this issue [34*], macaques were trained to either discriminate heading direction based on optic flow, or to simply fixate on these same visual stimuli. In the medial superior temporal area (a multi-sensory area involved in heading perception), training did not alter neural tuning for optic flow, but did substantially reduce noise correlations.

More recently, Jeanne and Gentner [35**] trained songbirds to discriminate complex song motifs to earn a reward. After learning, single units were recorded in CLM, a higher-order auditory area that encodes song features. For task-relevant motifs, noise correlation was lowest for pairs of neurons tuned to similar song features, and highest for neurons tuned to dissimilar features. The opposite was true for task-irrelevant motifs and novel, untrained motifs. Thus, learning altered noise correlations and their relationship to tuning similarity in a way predicted to specifically improve encoding of relevant motifs. A classifier trained to discriminate motifs from the spiking of cell pairs performed better when noise correlations were intact vs. when they were eliminated, confirming that the learned changes in correlation structure improved sensory coding.

It is unclear whether noise correlations are altered by learning in primary sensory cortex, but they are modulated by attention during discrimination tasks. Ramalingam and Gilbert [36] recorded single unit spiking in V1 in macaques trained to discriminate bar position and perform contour detection. During the task, noise correlations for pairs of neurons were lower when animals attended to stimuli within the receptive fields, compared to when attention was shifted to a different location. This effect only occurred when animals were engaged in the task, and was largest for units with similar visual tuning, which is predicted to maximally increase visual information. Another recent study found different noise correlation structure in V1 when macaques performed different orientation discrimination tasks, arguing that top-down inputs conveying task context determine noise correlations [37].

Thus, sensory training can alter noise correlations in sensory areas to improve coding. This is likely guided by top-down feedback, similar to learned changes in sensory gain [37]. Changes in noise correlations have also been observed in V1 of anesthetized mice after monocular deprivation [8**], but their functional relevance and circuit basis remain unknown.

Learning by reduction in daily tuning variation

In the motor system, variability in movement-related activity is robust, and drives variable motor output that explores the space of useful movements. During motor learning, circuit activity that commands successful movements is reinforced, thus selecting an optimal movement for completing the task. As learning occurs, neural variability decreases [29], [30], [38]. Does the day-to-day variability in single-neuron sensory tuning play a similar role in sensory learning?

Several studies show that day-to-day tuning variability in sensory cortex decreases over the course of learning (Figure 1C). In V1, Poort et al. [15**] examined session-to-session variability in single-neuron selectivity between two oriented gratings, as mice learned the grating discrimination task. Early in training, stimulus selectivity was highly variable for individual neurons from session to session. With learning, day-to-day variability decreased, so that selectivity was more stable across sessions. This stability developed in parallel with improved grating selectivity, population coding and behavioral performance.

Similar changes occur during learning in S1. Chen et al. [31**] tracked whether neurons responded to touch, non-touch cues, or were inactive as mice learned the texture discrimination task. Single neurons showed day-to-day variability in this response category, which decreased as animals became experts. Peron et al. [22**] trained mice to detect a bar using a single whisker, and imaged ~75% of all L2/3 PYR cells during learning. Different cells encoded touch, whisker movement, and other task parameters. While the proportion of cells encoding touch remained constant over the course of learning, which specific cells were touch-responsive changed from session to session. This variability decreased as behavioral performance improved, so that touch-related and whisking-related activity became more predictable day to day.

Interestingly, the increase in tuning stability with learning was associated with improved population coding of the stimulus in the Poort study of visual discrimination [15**], but not in the Peron study of whisker touch detection [22**]. Instead, population coding of touch remained stable throughout learning, and consistently better than the mouse’s behavioral performance. This suggests that the touch information needed for task performance was always present in S1, and that learning was accomplished by improved processing of this information in downstream areas. This supports a general hypothesis that when learning requires fine discrimination between similar stimuli, or detection of a subtle feature from noise, reorganization of population coding in primary sensory cortex occurs to improve the low-level sensory representation of task-relevant features (perceptual learning). But when animals must simply learn to use a robust sensory cue to trigger a specific behavioral response, optimization of sensory coding is not required, and learning occurs by downstream changes that trigger appropriate behavioral responses (sensorimotor learning).

The specific role of tuning variability in sensory learning remains unclear. It may reflect the formation of transient Hebbian ensembles by recent sensory experience patterns, which exist for a brief time but are then reversed by subsequent experience. Alternatively, novel ensembles may be formed stochastically by random circuit variation or noisy patterns of spontaneous activity, and once formed, could provide a basis for selection and reinforcement by sensory activity. Such random variation could help cortex escape from existing ensemble structure to establish new representation patterns [39], [28]. Once learning is complete, stabilization of tuning is likely to allow more reliable readout of task-relevant information by downstream areas for better task performance.

While sensory tuning stabilizes in sensory cortex once learning is complete, single-neuron tuning for task features remains dynamic in higher-order, memory-related cortical and hippocampal regions [40], [41]. For example, in posterior parietal cortex of mice performing a navigation task, single-cell coding of task-relevant visual stimuli reorganizes over days and weeks, even when mice are expert at the task [40]. This suggests that while early sensory areas develop stable representations in consistent behavioral conditions, associative areas maintain more flexible codes that can readily incorporate changing information and behavioral associations.

Learning by changing routing of information down sensorimotor pathways

Learning of sensorimotor associations requires plasticity downstream of sensory cortex to transform sensory signals to appropriate behavioral responses. Recently, Le Merre et al. [42**] tracked macroscopic changes in sensory-evoked population activity along cortical pathways while animals learned to lick in response to whisker deflection. Recording chronically in an array of sensory, associative, and motor areas, they found that sensory-evoked potentials were initially present in S1 and S2, but not medial prefrontal cortex (mPFC) or CA1 hippocampus. During learning, sensory responses stayed constant in S1 and S2, but developed strongly in mPFC, CA1, and M1 in proportion to behavioral learning. Neural activity in mPFC or CA1 was necessary for learned task performance. Thus, changes in sensory routing from sensory to associative and motor cortical areas can be directly observed during sensorimotor learning.

A first stage of this routing may occur in primary sensory cortex itself, by altering information coding in the PYR subclasses that project to different downstream target areas (Figure 1D). In Chen et al.’s study of texture discrimination learning [31**], learning increased the proportion of M1-projecting (M1p) neurons in S1 that encoded touch, due to recruitment of previously inactive cells, and shifted texture tuning within S2p cells. Both M1p and S2p populations became better at discriminating texture stimuli, suggesting that task-relevant information is more effectively relayed to M1 and S2 after learning. Decision-related signals are also found preferentially in S2p cells after training on a whisker detection task [24*]. Thus, learning can affect sensory representations in primary sensory cortex in an output pathway-specific way, which may help transfer task-relevant information to downstream areas related to choice and motor control.

Sites and mechanisms for changing population codes

These changes in population coding are likely to reflect a mix of classical and novel circuit plasticity mechanisms. Hebbian plasticity in local circuits will form and strengthen ensembles of coactive neurons, driven either by bottom-up sensory statistics, top-down inputs, or their interaction. This likely explains shifts and narrowing of sensory tuning for task-relevant sensory features. Pattern completion within Hebbian ensembles could explain increased response reliability. As demonstrated by Carrillo-Reid et al. [10*], coactive ensembles are readily formed within existing structures of connectivity, and layered on pre-existing ensembles. Together, this local Hebbian plasticity will contribute strongly to improved population codes for discriminating relevant stimuli. Top-down input may contribute to reactivating learned ensembles, explaining why learned improvement of population coding is often strongest during task performance.

Other learning-related changes in sensory cortex population codes likely reflect plasticity that takes place in higher sensory, decision, or motor areas, and feeds back to sensory cortex via top-down projections. This may explain how perceptual learning increases choice and reward-related activity in primary sensory cortex [31**], [24*], and how task salience and attention modify the structure of noise correlations in sensory cortex [36], [37]. Learning could alter top-down activation of primary sensory cortex by changing stimulus representation in higher-order areas, thus changing spiking patterns in existing feedback connections, or by driving synaptic plasticity at feedback synapses in primary cortex.

Some population coding changes could also reflect learned changes in sensory strategy, mediated by plasticity in cognitive or motor areas controlling sensory behavior. For example, in both the Peron and Chen studies [22**], [31**], mice learned to optimize whisking behavior and to produce more consistent whisking during learning, presumably to extract relevant touch signals more efficiently. This change in active sensory exploration may have contributed to improved stimulus selectivity, reliability, and decreased day-to-day variability during learning. In Poort and Hofer [15**], however, alterations in eye position, pupil size, or running speed could not explain improvements in sensory coding. Thus, plasticity in motor systems for active sensor control can affect sensory cortex population coding, and it is critical to monitor behavior precisely to assess this possibility.

Together, these findings indicate that experience-dependent changes in sensory cortex population codes reflect a mixture of plasticity in local cortical circuits, in ascending sensory pathways to cortex, in top-down projections carrying cognitive information, and even in distant motor pathways. In this sense, each local sensory area is a window onto large-scale brain plasticity. Monitoring plasticity in the different inputs to sensory areas (e.g., by calcium imaging in specific populations of axon terminals), will be a critical step to unravel these different contributions to sensory cortex plasticity.