Dynamic Salience Processing in Paraventricular Thalamus Gates Associative Learning

Abstract

The salience of behaviorally relevant stimuli is dynamic and influenced by internal state and external environment. Monitoring such changes is critical for effective learning and flexible behavior, but the neuronal substrate for tracking the dynamics of stimulus salience is obscure. We found that neurons in the paraventricular thalamus (PVT) are robustly activated by a variety of behaviorally significant events including novel stimuli, reinforcing stimuli and their predicting cues, as well as omission of the expected reward. PVT responses are scaled with stimulus intensity and modulated by changes in homeostatic state or behavioral context. Inhibition of the PVT responses suppresses appetitive or aversive associative learning and reward extinction. Our findings demonstrate that the PVT gates associative learning by providing a dynamic representation of stimulus salience.

The brain constantly receives streams of complex sensory inputs, and must direct attention to the most important or salient stimulus. The salience of a stimulus is determined by both physical properties and behavioral significance, such as the reward value or novelty (1–5). Although physical properties, such as brightness or color, are fixed attributes of the stimulus, the behavioral significance is a relative property that depends on past experience, current homeostatic state, and behavioral context (1, 2, 5). Therefore, identifying the essential anatomical substrates for tracking the dynamics of stimulus’ behavioral significance is necessary to understand the neural mechanisms underlying proper allocation of attentional resources and to directly examine the contribution of stimulus salience to learning (6–8).

Early studies largely focused on cortical circuitry and identified the frontoparietal attention network for attentional selection of behaviorally relevant stimuli (2, 9–11). Recent work has begun to reveal thalamic contributions to the persistence of frontal cortical activity during motor preparation, working memory and rule representation (12–15). However, the coding of various forms of behavioral relevance in the thalamus has not been systematically studied. It remains unclear whether the thalamus can represent context-dependent dynamics of behavioral significance. If so, it will be important to determine how salience responses in the thalamus contribute to associative learning.

The thalamus is composed of several anatomically and functionally distinct subnuclei. Among them, the paraventricular thalamus (PVT) is uniquely situated for integrating information relevant to behavioral significance (16–24). The PVT is not directly connected with sensory cortices, but is reciprocally connected with regions involved in top-down control, such as the prefrontal and insular cortices. It also receives extensive inputs from the hypothalamus and brainstem which convey signals about motivational arousal and homeostatic states. In turn, the PVT is the only thalamic nucleus that innervate all structures in the extended amygdala system (16, 22, 24–26). Previous lesion and pharmacological silencing studies suggested potential roles of the PVT in both appetitive and defensive behaviors. However, little is known about how PVT neurons engage in behaviors with opposite valence.

PVT encodes multiple forms of salience

We infected PVT neurons (between Bregma −1.06 to −1.58mm) with adeno-associated virus expressing a genetically encoded Ca²⁺ indicator (AAV-GCaMP6m) (27), then used fiber photometry to record population Ca²⁺ signals in the PVT of the head-fixed mice across days of associative learning (fig. S1) (28–30). We first randomly presented water-restricted mice with odors or water (5μl) without pairing them (see methods) (30). Initially, both stimuli robustly activated the PVT, but while PVT responses to free water remained consistent, odor evoked responses were rapidly diminished (Fig. 1A). Habituation of PVT responses to odor was stimulus specific and long-lasting, as subsequent exposure to a different odor still elicited robust PVT responses (fig. S2A), and PVT response to the same odor was still strongly suppressed 2 days later (fig. S2B). Similar novelty responses were also observed across multiple modalities, including visual and auditory stimuli, in the PVT (fig. S2C).

Fig. 1. PVT neurons encode salience irrespective of valence.

(A) Top: Photometric traces of calcium responses in the PVT to 10 repetitions of randomized odor (Left) and water (5μl, Right) stimuli. Dashed line indicates the time of stimulus delivery. Scale bar, 10% ΔF/F, 3 s. Bottom: Left Y axis, quantification of odor (black dot) and free water (red dot) evoked ΔF/F over 10 repetitions. Right Y axis, quantification of free water (orange circle) evoked licks over 10 repetitions. Novel odor: n = 6 mice; Free water n = 12 mice; Licks: n = 12 mice. (B) Trial structure of the Pavlovian conditioning paradigm. ITI, inter-trial interval. CS: conditioned stimulus; US: unconditioned stimulus. (C) Mean lick rate of well-trained animals (n = 7) shows anticipatory licking in appetitive (blue) but not neutral (black) and aversive (red) trials. Gray bar: 1s of CS delivery; Vertical dash line: US delivery. (D) Mean photometric responses of the PVT to CS and US in both appetitive (blue) and aversive (red) but not neutral (black) trials. Shade, SEM across mice, n = 7 mice. (E) Z score heat maps (left) and pie chart (right) for all task-responding neurons identified by in vivo single-unit recording during Pavlovian tasks of well-trained animals. Neurons are separated in five subgroups based on their tuning properties, and are rank-ordered by their response onset times during reward cue stimulation. Each row in the heat maps represents responses from the same neuron to different stimuli. n = 85 neurons from 12 mice. (F, G) Mean photometric responses (F, n = 7 mice) and quantification (G) showing that CS and US response in the PVT are graded to different intensity of reward (left, 5 vs 15 μl water) and punishment (right, air puff vs. tail shock). AUC, area under curve (see methods). Scale bar, 2% (F, left), 4% (F, right) ΔF/F, 1s. Big, small reward and nothing: n = 7,7,7; Big, small punishment and nothing: n = 6,6,5. *P < 0.05, **P < 0.01 (One-way ANOVA, Post-hoc Tukey’s test). Shade, SEM across mice in C, D, F. Data are means ± SEM.

After the odors were no longer novel, we then trained the mice to associate the same set of odors to either appetitive, neutral, or aversive outcomes (17, 31). Each training trial began with a conditioned stimulus (CS, 1s odor), followed by a 2s delay and an unconditioned stimulus (US, the outcome) (Fig. 1B). As training progressed, mice began to display anticipatory licks only during the delay of appetitive (water reward) trials, indicating the establishment of CS-US association (Fig. 1C, fig. S3A) (17, 31). The familiar odors gradually gained behavioral significance as they were associated with reinforcing outcomes. After the mice had fully learnt the task, we performed fiber photometry recording and observed robust task-evoked responses in the PVT of GCaMP6 but not eGFP expressing mice (Fig. 1D, fig. S3B). The PVT responded to both CS and US irrespective of appetitive or aversive outcomes (Fig. 1D), and their averaged response magnitude were graded, reflecting the intensity of reward (5 vs. 15μl water) and punishment (air puff vs. tail shock) (Fig. 1, F and G).

Because fiber photometry records population activities, it is still possible that within the PVT, subpopulations might encode a specific valence. We therefore performed in vivo single unit recording and found majority of recorded PVT neurons (85/115) were responsive to the learned task. Among them, 68% of task-related neurons were excited by CS or US of both appetitive and aversive outcomes, a hallmark of salience coding (Fig. 1E) (32, 33). The other 32% of neurons can encode valence as they responded heterogeneously to the appetitive vs. aversive task (Fig. 1E). Although cross-correlation analysis and lick-triggered spike analysis revealed little correlation between licking with action potential firing in the PVT (fig. S4), the timing of CS responses in appetitive trials was more distributed and tiled the entire delay period (Fig. 1E). This suggests that the PVT activity encodes salience of both CS and US and can reflect level of behavioral engagement.

Prediction error (PE) signals the discrepancy between the expected and actual received outcomes (6). It is encoded by a widespread neuronal network and provides teaching signals for associative learning (31, 34). To test whether the PVT can encode PE, we chronically recorded the PVT across days of associative learning, pairing cues with a water reward or an air puff. Following behavioral training, CS responses in the PVT gradually increased, whereas US responses remained constant in both appetitive and aversive learning (Fig. 2, A and B). Moreover, in well-trained animals, the magnitude of PVT responses to a well-predicted US was similar to that of unexpected delivery of a US (Fig. 2, C to F, fig. S5). Therefore, PVT does not encode PE, because PE encoding neurons gradually decrease their US responses during associative learning, and unexpected events should have bigger US responses.

Fig. 2. Dynamics of salience response in PVT neurons during associative learning.

(A) Representative photometric responses in the PVT across multiple sessions of Pavlovian conditioning (session 1 to 5). Gray bar: 1s of CS delivery; Vertical dash line: US delivery; Horizontal dash line: baseline ΔF/F. Upper panel: appetitive; Middle panel: nothing; Lower panel: aversive. Scale bar: 5% ΔF/F. Each photometric trace is averaged from all 50 trials within single conditioning session. (B) Quantification of CS response (top) and US response (bottom) across 5 training sessions (n = 6). Note the significant increase of CS responses following training, whereas the US responses remain consistent. *P < 0.05, **P < 0.01, ***P < 0.001 (Two-way ANOVA, Post-hoc Bonferroni test). (C) Mean photometric responses of the PVT to expected and unexpected delivery of reward (top, n = 10; Dark blue: expected; Light blue: unexpected) and punishment (bottom, n = 10; Red: expected; Orange: unexpected). Scale bar: 4% ΔF/F. (D) Quantification of C. Wilcoxon signed-rank test: P = 0.32 (Reward); P = 0.49 (Punishment). (E) Mean Z score of single unit responses of the PVT neurons to expected and unexpected delivery of reward (top, n = 31 neurons) and punishment (bottom, n = 22 neurons). (F) Quantification of E. Wilcoxon signed-rank test: P = 0.28 (Reward); P = 0.11 (Punishment). Gray bar: 1s of CS delivery; Vertical dash line: US delivery, Scale bar: 1s in A, C, E. Shade, SEM across mice in C, E. Data are means ± SEM.

PVT activity controls associative learning

Salient stimuli attract attention, which in turn facilitates associative learning. If CS or US evoked responses in the PVT represent stimulus salience, then suppression of PVT responses should decrease the efficiency of CS-US association. To test this, we infected PVT neurons with AAV expressing archaerhodopsin-3 (AAV-ArchT) or eGFP (AAV-eGFP) as a control (fig. S1) (35). We optogenetically inhibited the PVT (in PVT :: ArchT mice) during the cue + delay period or following the US delivery, and examined its effect on appetitive or aversive learning during both conditioning sessions (from D1 to D5) and the no-laser (NL) test (Fig. 3, A to D) (17). We found that PVT inhibition during either CS (Fig. 3, A and B) or US (Fig. 3, C and D) periods during training reduced anticipatory licking in both conditioning sessions and in the NL test. Optogenetic inhibition during the inter-trial interval had no effect (fig. S6A), and PVT inhibition during CS + delay period had no effect on anticipatory licking in well-trained mice (Fig. 3E). Moreover, in a go/no-go task (Fig. 3F), PVT inhibition during cue + delay period reduced licking in go trials but increased licking in no-go trials (Fig. 3G, fig. S7), thus decreasing discriminability (Fig. 3H). Together, CS and US responses in the PVT are required for the formation but not the expression of conditioned reward seeking.

Fig. 3. Photoinhibition of PVT impairs associative learning.

(A, C) Top: representative lick raster plots from PVT :: GFP (left) and PVT :: ArchT mice (right) across 5 conditioning sessions and the NL test when light stimulation was delivered during CS + delay period (A) or after US delivery (C). NL, non-laser. Back lines indicate the start and end time for odor delivery, respectively. Red line indicates water delivery. Green shade indicates laser stimulation. Bottom: representative change of lick rate across 5 conditioning sessions (D1 to D5: Blue, Light blue, Green, Yellow, and Orange) and the NL test (Black). (B, D) Quantification of anticipatory licks of A and C, respectively. B: PVT :: GFP mice, n = 7; PVT :: ArchT mice, n = 7. D: PVT :: GFP mice, n = 6; PVT :: ArchT mice, n = 8. ** P < 0.01 (Two-way ANOVA, Post hoc Bonferroni test). (E) Representative lick raster plots (left) and histograms (right) from well-trained PVT :: ArchT mice with laser off and on. Green, laser on (n = 6). Scale bar: 1s. Wilcoxon signed-rank test: P = 0.84. (F) Schematics of go/no-go task. (G) Anticipatory lick rate of go trials (left) and no-go trials (right) from PVT :: GFP (n = 9) and PVT :: ArchT (n = 9) mice on the last day of training. Mann-Whitney U-test, *P < 0.05. (H) Discriminability of go and no-go trials over training. Discriminability was calculated as (Lick_go-Lick_no-go)/( Lick_go+Lick_no-go). * P < 0.05, ** P < 0.01 (Two-way ANOVA, Posthoc Bonferroni test). Data are means ± SEM.

To examine the impact of PVT inhibition on associative aversive learning, we first trained mice with two different odors both associated with water reward in phase 1, then switched the outcome of odor B to water + tail shock in phase 2 (fig. S8A). The switch caused gradual suppression of odor B elicited anticipatory licking over the next 5 days of training (fig. S8, B and C). PVT silencing had no effect on nociceptive responses to thermal stimuli on the tail (fig. S8C, inset), but optogenetic inhibition of the PVT (in PVT :: ArchT mice) during the cue + delay (fig. S8, B and C) reduced the suppression effect in both conditioning sessions and in the NL test. Interestingly, PVT inhibition during US delivery period had no effect on aversive learning (fig. S8D). Together, these results indicate that PVT activity during the cue period is required for both associative reward and aversive learning, and substantiate the critical role of cue salience in driving associative learning.

PVT tracks context-dependent salience

Besides learning, changes of homeostatic state or external environment also influence the perceived salience of sensory stimuli (2, 7, 10, 36). If PVT activity represents the salience of CS and US, then their evoked activity in the PVT should also be modulated by internal and external factors. Because we used water as the reward during Pavlovian conditioning, we examined the impact of thirsty versus sated state on CS and US evoked PVT activity. The water-predicting cue elicited robust anticipatory licking in well-trained thirsty mice. We then gave these mice free access to 0.6ml of water to drink until sated. As expected, the same odor cue no longer elicited anticipatory licking in sated mice (Fig. 4A). Using fiber photometry, we recorded PVT activity in both thirsty and sated states, and found both CS and US evoked PVT activity were strongly suppressed in sated mice, consistent with a decrease of salience of both the water-predicting cue and of water consumption in sated mice (Fig. 4, B and C). Interestingly, a stronger PVT response to air puff was observed in sated than thirsty mice, indicating that an air puff became more salient when homeostatic needs were met, and supporting the hypothesis that PVT activity represents context-dependent evaluation of salience between different sensory stimuli.

Fig. 4. Effect of homeostatic state on PVT responses.

(A) Mean lick rate after odor cue in thirsty (left, dark blue, n = 7) and sated (right, light blue, n = 7) state. (B, C) Mean photometric traces (B) and quantification (C) of PVT responses in appetitive (left), neutral (middle), and aversive (right) test in thirsty (left, dark blue) and sated (right, light blue) state. CS (top) and US (bottom) response in C. Thirsty: n= 7; Sated: n = 7; *P < 0.05, **P < 0.01, ***P < 0.001 (Mann-Whitney U-test). Shade, SEM across mice in A, B. Gray bar: 1s of CS delivery, vertical dash line: US delivery in A, B. Data are means ± SEM.

To further test this hypothesis, we manipulated the behavioral context by changing the intensity of the aversive stimuli and examined the impact of this change on reward responses in the PVT. We first conditioned the mice for 5 days in a mild aversion context, in which an air puff was used as punishment. On day 6, we switched the punishment from air puff to tail shock (strong aversion context) (Fig. 5A). Switching from a mild to a strong aversion context rapidly suppressed reward responses in the PVT, as revealed by both in vivo calcium imaging and single cell electrophysiological recording (Fig. 5, B to G). Among task-related responders in the PVT, only 76% responded to the aversive condition in the mild aversion context compared to 97% in the strong aversion context, whereas only 80% responded to the reward condition in the latter context compared to 92% in the former (P < 0.001, Chi-squared test, Fig. 1E, 5, E and F). This observation revealed the reallocation of salience from appetitive to aversive stimuli after switching from a mild to a strong aversion context, which is consistent with the notion that individual PVT neurons are tuned to the salience of the sensory stimuli irrespective of its valence. Moreover, the learning rate for the cue-reward association was slower in the strong aversive context (Fig. 5H), consistent with the finding that smaller PVT responses were allocated to reward in the strong aversive context than that in the mild aversive context, further supporting that sensory stimuli evoked PVT response controls the efficiency of associative learning. Together, PVT activity represents the dynamics of stimulus salience following a change of the behavioral context or homeostatic state.

Fig. 5. Context-dependent modulation of salience response in the PVT.

(A) Behavioral procedure for switching from mild to strong aversive context. (B) Mean photometric responses of the PVT to appetitive and aversive test in mild (n = 6) versus strong aversive context (n = 6). (C) Quantification of CS (left) and US (right) response in B. Mann-Whitney U-test, **P < 0.01 (CS); P = 0.15 (US). (D) Rapid suppression of PVT response to water predicting cue after switching from mild to strong aversive context. Note no further reduction observed after 10 trials. (E) Top, Z score heat maps for all task-responding neurons identified by in vivo single unit recording of well-trained animals in strong aversive context. Neurons are separated in four subgroups based on their tuning properties, and are rank-ordered by their response onset times during reward cue stimulation. Each row in the heat maps represents responses from the same neuron to different stimuli. n = 62 neurons from 12 mice. Bottom, Z score quantification of PVT response during Pavlovian tasks. (F) Pie chart shows the tuning of PVT neurons in strong aversive context. (G) Z score quantification of CS (left) and US (right) response in the PVT during appetitive test in mild (n = 85 neurons) versus strong aversive context (n = 62 neurons). Mann-Whitney U-test, *P < 0.05 (CS); **P < 0.01 (US). (H) Left, raster plots illustrate licking behavior across 5 reward conditioning sessions in strong aversive context. . Right, quantification of anticipatory licks during reward conditioning in mild (red, n = 7) versus strong aversive context (orange, n = 7). **P < 0.01, (Two-way ANOVA, Post-hoc Bonferroni test). Red, mild aversive condition; Orange, strong aversive condition, in E, G, H. Shade, SEM across mice in B, E. Gray bar: 1s of CS delivery, vertical dash line: US delivery in B, E. Scale bar: 1s in E, H. Data are means ± SEM.

Reward omission responses in the PVT

The above studied salience responses were all evoked by sensory stimulus. Could PVT activity also represent an emotionally salient state without a sensory stimulus? We thus examined reward omission response in the PVT (18, 37). Although no sensory stimulus is delivered, omission of an expected reward is behaviorally significant. In well-trained mice, we omitted the predicted water reward in a random 10% of trials and recorded PVT activity. Because the CS was delivered before omission, the CS evoked similar PVT responses in both reward trials and reward omission trials (Fig. 6, A and B). Interestingly, the PVT responded very differently to reward delivery and reward omission. Reward omission lacked the immediate response previously observed with water consumption, and instead elicited a delayed long lasting response in the PVT (Fig. 6, A and B). Two possible scenarios might underlie omission responses in the PVT: one might reflect a cognitive state of expectant waiting, since many PVT neurons show an anticipatory response to reward delivery during the delay period between CS and US; the other explanation is that the lack of expected water is a salient stimulus, thus activating the PVT(18, 37). We noticed that omission trial responses generally occurred after licking stopped, and when we aligned the calcium response to the last lick, we observed a rapid increase in PVT activity following the cessation of licking (Fig. 6C). Together these results suggested that PVT activity could also represent behaviorally significant state without sensory stimulus.

Fig. 6. Reward omission response in the PVT.

(A, B) Mean photometric traces (A) and histogram (B) illustrating delayed but long lasting PVT responses to reward omission. Expected reward (black, n = 10); Reward omission (red, n = 10), Wilcoxon signed-rank test, P = 0.19 (CS); **P < 0.01 (US). (C) Left, representative traces of individual omission response (red) superimposed with lick raster plots (black). Right, mean photometric traces (n = 10) after aligning to the last lick in omission trials. Note the rapid increase of calcium signals after licking stops. Scale Bar, 2% ΔF/F, 1s. Gray bar: CS delivery, vertical dash line: US delivery in A, C. (D, E) Top, representative lick raster plots from PVT :: GFP (left) and PVT :: ArchT mice (right) with laser stimulation during reward omission period (D) or CS + delay period (E) of extinction trials. Back lines indicate the start and end time for odor delivery, respectively. Red line indicates water delivery. Scale bar: 1s. The mice received water reward in first 10 trials (black), then water delivery stopped (red) and optogenetic stimulation was on until the end of the trial (green). Bottom: quantification of anticipatory licks in 30 extinction trials. Licks (black dot) are normalized to averaged licks during the first 10 trials. Red line indicates the exponential fit of licks. D: Inset, histogram shows the mean time constants (τ) of extinction from PVT :: GFP (white, n = 6) and PVT :: ArchT (green, n = 10) mice. E: Inset, histogram shows the mean time constants (τ) of extinction from PVT :: GFP (white, n = 9) and PVT :: ArchT (green, n = 10) mice. Mann-Whitney U-test, *P < 0.05. Shade, SEM across mice in A, C. Data are means ± SEM.

Animals use the outcome information of their previous choice to adjust subsequent expectations. How might the reward omission response in the PVT contribute to this behavioral adjustment? Continuous reward omission will extinguish the learned association. Thus, PVT responses to reward omission might serve as a teaching signal for extinction. To test this directly, we first conditioned PVT :: ArchT and PVT :: GFP mice with odor cue and water reward for 5 days, and examined the cued reward seeking behavior for first 10 trials on day 6. We then optogenetically silenced the PVT during the reward omission window in the following extinction trials. Stopping reward delivery caused rapid extinction of cue-evoked anticipatory licking in PVT :: GFP mice, whereas the rate of extinction was significantly slower in PVT :: ArchT mice (Fig. 6D). Since extinction is also a form of learning, the CS response in the PVT should also be important for extinction. Inhibiting the PVT during the cue + delay period did significantly slow the rate of extinction (Fig. 6E). This suggests the function of the PVT CS response is to maintain the salience of the CS which allows for effective learning if the US is changed. In summary, these data together with results in Figure 3 substantiate that salience activity in the PVT controls the rate of multiple forms of associative learning.

Discussion

Here, we show that PVT neurons encode multiple salient features of sensory stimuli, including reward, aversion, novelty, and surprise (reward omission). We further demonstrate that PVT provides dynamic representation of salience by manipulating behavioral significance of stimuli through associative learning, modulation of homeostatic states, and alterations of the behavioral context. Notably, when animals are in the behavior context with a mild aversive stimulus, the majority of responders in the PVT respond to appetitive stimuli (Fig.1E); but in animals where the aversive stimulus is strong, almost all PVT responders respond to aversive stimuli (Fig.5F). This finding demonstrates that individual PVT neuron track the context-dependent dynamics of salience information. These results also suggest that the PVT has a more specific role than promoting general arousal, as PVT reward responses are decreased in the strong aversive context when the animal should be more aroused. How do PVT neurons acquire such response flexibility? Since most thalamic neurons do not have local excitatory connections, we anticipate that PVT inputs play important roles. Further work is required to silence each individual input while examining salience responses in the PVT.

US responses in the PVT are not suppressed by expectation, and reward omission activates the PVT. These results demonstrate that the PVT does not encode PE (6, 30, 31). Moreover, inhibition of PVT activity impairs associative learning of appetitive and aversive outcomes, as well as extinction of an established reward association. Together, our results highlight the importance of stimulus salience in driving learning. Interestingly, silencing PVT activity affects learning but not expression of conditioned behavior, indicating that the function of the PVT is different from other thalamic nuclei such as the mediodorsal thalamus or the thalamus connecting with anterior lateral motor cortex, because silencing these regions disrupts ongoing task performance (12–15). In well-trained mice, silencing PVT CS response has no effect on licking when water is available but slowed extinction when water is not available, suggest that CS responses in well-trained mice are for monitoring potential changes of salience. The critical next step is to determine how salience information in the PVT is communicated to the rest of the brain. Interestingly, axons of PVT neurons show extensive collateralization; therefore, the PVT could simultaneously broadcast salience signals to multiple downstream targets to coordinate their activities (fig. S9) (38). Indeed, PVT terminals in the nucleus accumbens directly interact with dopaminergic fibers from the ventral tegmental area and evoke dopamine efflux, suggesting a direct interaction of salience and reward PE signals in the NAc (39). The impact of these interactions on associative learning needs to be investigated further.