Roles of centromedian parafascicular nuclei of thalamus and cholinergic interneurons in the dorsal striatum in associative learning of environmental events

Abstract

The thalamus provides a massive input to the striatum, but despite accumulating evidence, the functions of this system remain unclear. It is known, however, that the centromedian (CM) and parafascicular (Pf) nuclei of the thalamus can strongly influence particular striatal neuron subtypes, notably including the cholinergic interneurons of the striatum (CINs), key regulators of striatal function. Here we highlight the thalamostriatal system through the CM-Pf to striatal CINs. We consider how, by virtue of the direct synaptic connections of the CM and Pf, their neural activity contributes to the activity of CINs and striatal projection neurons (SPNs). CM-Pf neurons are strongly activated at sudden changes in behavioral context, such as switches in action-outcome contingency or sequence of behavioral requirements, suggesting that their activity may represent change of context operationalized as associability. Striatal CINs, on the other hand, acquire and loose responses to external events associated with particular contexts. In light of this physiological evidence, we propose a hypothesis of the CM-Pf - CINs system, suggesting that it augments associative learning by generating an associability signal and promotes reinforcement learning guided by reward prediction error signals from dopamine-containing neurons. We discuss neuronal circuit and synaptic organizations based on in vivo/in vitro studies that we suppose to underlie our hypothesis. Possible implications of CM-Pf - CINs dysfunction (or degeneration) in brain diseases are also discussed by focusing on Parkinson’s disease.

Introduction

When faced with uncertain environments, humans and other animals exploit knowledge gained from past experience and process currently available cues with heightened attention to determine whether the cues around them predict desirable or undesirable outcomes. Unexpectedly occurred environmental events and reward often causes a “surprise”, i.e., an unexpectedness of the observation based on experience, which helps in making inferences about current state and promotes learning to predict the future by the acquisition of associations strengthened by reinforcement (Roesch et al., 2012, McGuire et al., 2014). As such cues become more reliable predictors through learning, vigilant processing and learning decline. Theories of animal learning and reinforcement learning propose that unexpected reward produces errors of prediction and promotes learning to reduce the prediction error related to reward (‘reward prediction error’ or RPE) and ultimately guides behavior to maximize total expected rewards (Rescorla and Wagner, 1972, Sutton and Barto, 1998). Many experimental studies have delimited neural activity patterns in basal ganglia circuits to support this theory of reinforcement learning: the encoding of errors of reward prediction by midbrain dopamine neurons (Montague et al., 1996, Schultz, 1998b, Satoh et al., 2003, Bayer and Glimcher, 2005, Enomoto et al., 2011), the dopamine-dependent plasticity of corticostriatal synaptic transmission (Reynolds et al., 2001, Calabresi et al., 2007, Shen et al., 2008), and the encoding of reward value of actions in projection neurons in the striatum (SPNs) (Samejima et al., 2005, Lau and Glimcher, 2008). In the reinforcement learning model of the basal ganglia, RPE signals from dopamine-containing neurons directly promote learning by facilitating or suppressing specific action- and cognition-related corticostriatal signals to adjust future predictions and to select behavior (Houk et al., 1995, Montague et al., 1996, Doya, 2000).

Environmental cues also guide us making inferences about the current situation under uncertain circumstances, and promote learning to predict the future. In this mode of learning, unexpected events produce a surprise and enhance processing of the event and thus help to change the association of such events with the current situation or with future rewarding and aversive events by changing the associative strength of events, as part of associative learning about the event (Mackintosh, 1975, Pearce and Hall, 1980). Thus, surprise drives the modulation of the associative strength of the event, which can then influence behavioral learning in terms of rate of learning (Schultz and Dickinson, 2000). Large errors will result in a boost of the associative learning, whereas small errors will delay learning. Studies of neuronal activity of behaving animal and fMRI studies of human brain suggested parietal cortex, lateral prefrontal cortex, and amygdala are involved in attentional processes for decision making (Glascher et al., 2010, Roesch et al., 2010, Roesch et al., 2012), and in interaction between reinforcement learning and attention (Leong et al., 2017). However, available knowledge is still limiting our understanding of the neuronal circuits, encoded signal contents and synaptic organizations responsible for surprise and learning.

Objective of this review is to provide an overview of functional properties of neurons in the centromedian parafascicular (CM-Pf) complex of the thalamus and of the cholinergic interneurons of the striatum (CINs) and to note the potential of the intralaminar thalamostriatal projections as part of a mechanism related to learn and unlearn association of environmental events with action and outcome based on surprise. We first summarize evidence that the neuronal signals generated by neurons of the CM-Pf complex in non-human primate subjects performing tasks include responses to surprise evoked by the unexpected appearance of events in the environment. Surprise, produced by changes in action-outcome contingency and sequencing of task requirements, is critical for such responses. Thus, the activity of the CM-Pf neurons seems to represent processes previously described as attention to specific cue or “associability of cue”, which is equal to the weighted average of prediction errors generated across the past few trials (Mackintosh, 1975, Pearce and Hall, 1980). We next describe the evolution of CIN responses during the acquisition of associations between click sounds and reward in the putamen and caudate nucleus, which are the main targets of thalamostriatal projections from CM-Pf. We consider potential functions of CM-Pf inputs in shaping the activity of striatal CINs and SPNs during behavioral tasks that require a switch in behavioral context. Finally, we propose a hypothetical model in which a surprise elicited by unexpected external events activates brainstem-thalamocortical circuits for attention, CM-Pf circuits for generating associability signals and midbrain dopamine neurons for generating RPEs. We suggest that associability signals of the CM-Pf play roles in weighting external inputs to reduce trial-and-error learning costs depending on their expected associative strength with outcome, whereas RPEs of dopamine neurons promote reinforcement learning of SPNs (as well as CINs and other striatal interneurons) to modulate corticostriatal signals influencing action and cognition signals to update prediction. Thus, surprise promotes two kinds of learning in the striatum, depending on discrete teaching signals, thereby enhancing prediction about the future and guidance of behavior to specific goals. Dysfunction of the system may be involved in a range of basal ganglia disorders affecting movement and cognition.

Neuronal signals in the thalamic CM-Pf nuclei

Neurons in the CM-Pf complex (Fig. 1a) are characterized by their strong responsiveness to multimodal sensory stimuli and the conditional nature of these responses. For example, click sounds occurring near a monkey sitting in a primate chair evoke phasic activation of CM-Pf neurons at short latencies (around 25 ms, SLF neurons) or at long latencies (around 230 ms, LLF neurons) (Fig. 1b) (Matsumoto et al., 2001, Minamimoto and Kimura, 2002). In monkeys, the SLF neurons are predominantly located in the Pf, which is a main origin of fiber projections to the caudate nucleus and the rostral putamen, whereas LLF neurons are predominantly in the CM, which is main origin of the thalamostriatal projection to the putamen (Matsumoto et al., 2001, Smith et al., 2004, Glimcher and Lau, 2005). Robust responses of both SLF and LLF neurons occur when the clicks appear at varied time intervals or in combination with other stimuli such as beep sounds and LED lights. But the responses gradually decline if the click repeatedly appears at fixed time interval with no other stimuli (Fig. 1c). Similarly, robust responses are evoked by handclaps and knocks on the door or walls of the room in which the monkey is sitting. In many instances, the neurons respond to such alerting stimuli only for the first few times, and the stimuli gradually became ineffective in evoking responses if they repeatedly appear. Thus the basic responsiveness of these intralaminar neurons seems to be a surprise or prediction error, which can drive learning (Rescorla and Wagner, 1972, Pearce and Hall, 1980, Sutton, 1988). Midbrain dopamine neurons encode RPEs (Schultz, 1998b, Satoh et al., 2003, Nakahara et al., 2004, Bayer and Glimcher, 2005). The activity of CM-Pf neurons, however, is different from that of dopamine neurons, because it is similar whether or not the unexpected stimuli are associated with reward (Fig. 1b). In a visually cued target detection task, SLF neurons, for example, exhibit responses with much larger magnitude to a visual target appearing on the side opposite to that of the cued location (i.e., invalid) than to the cued target (valid) (Fig. 1d, e). The invalid target works against prediction as evidenced by longer reaction time of arm movement after invalid target than after valid target (Minamimoto and Kimura, 2002).

Fig. 1.

CM-Pf of primate posterior intralaminar nuclei and their neuronal activity during behavioral tasks. (A) Nissl-stained frontal section at the level of CM-Pf. CL: centralis lateralis; MD: mediodorsal nucleus; PFdl: dorsolateral part of Pf, FR: fasciculus retroflexus. Asterisk indicates electrolytic lesion mark made by passing DC current (0.2 mA for 10 min) through a recording elgiloy microelectrode. Scale bar: 2 mm. (B) Ensemble average responses of SLF and LLF neurons in the CM-Pf to Clicks presented to monkeys sitting in a primate chair with or without following water reward. (C) Responses of LLF neurons to Click cues in a 75-trial-long block in which beeps, clicks, and LED flashes appeared in a random order with equal probabilities and at equal interval (left). The same neurons were then examined in a 25-trial-long block in which only clicks were presented (right). (D) Experimental paradigm. Illustration of the valid cue condition, in which the cue and the target appear at the same location (left), and the invalid cue condition, in which the cue and the target appear at different locations (right). (E) Average responses of SLF neurons to a contralateral validly cued target (thin line) as compared with those to a contralateral invalidly cued target (thick line). Figures are modified from Matsumoto et al. (2001) and Minamimoto and Kimura (2002).

Anatomical and physiological evidence is in accord with neuronal circuits of CM-Pf being important for associative learning of events and contexts: the CM-Pf receives information about environmental events coming through multimodal sensory channels, is sensitive to unexpectedness and surprise, and has no specific sensitivity to reward. In contrast to the rostral part of the intralaminar thalamic nuclei that relay multimodal sensory arousal to widespread cortical areas (Mennemeier et al. 1997; Pare´ et al. 1988; Steriade et al. 1994, 1997), the CM and Pf, making up the caudal intralaminar thalamic nuclei, are the main source of the massive thalamostriatal projection (Fig. 1a) (Macchi et al., 1984, Ragsdale and Graybiel, 1991, Groenewegen and Berendse, 1994, Haber and McFarland, 2001, Smith et al., 2004).

Activation at surprise produced by unexpected change of behavioral context as a basic characteristic of CM-Pf neurons

Surprise produced by unexpected events activates multiple discrete neuronal circuits. For example, surprise produces selective concentration of neuronal processing of the event and neglect of other perceivable information, referred to as the allocation of limited processing resources (Pashler, 1998). Surprise produces errors of outcome prediction as well as processing change relative to behavioral context in order to modulate the associative strength of the context with outcome. A strong surprise produced by a large error of prior prediction helps in updating prediction of the future. Neural signals related, respectively, to predicted and unpredicted sensory events can be enhanced or diminished depending on the error of prediction. Certainly, during performance of visually instructed left/right button press task for asymmetric reward by monkeys (Fig. 2a; Hold-GO reward task), SLF neurons exhibit robust phasic responses to GO signals immediately after a switch of action-outcome contingency, and these responses decline within a few trials (Fig. 2b left panel and c). Neuronal activation becomes maximal when the largest change is required in the associative strength of the currently appearing instruction signals (Cue/GO) with outcome. This activation is not likely an encoding of prediction errors of outcome, because the outcome-signaling beep sound that occurs after behavioral responses does not evoke neuronal activation during the first few trials immediately after switch of action-outcome contingency. In addition, instructions associated with large and small reward evoke similar responses (Fig. 2b). Further, selective activation of the neurons also occurs when a sequence of behavioral requirements is suddenly changed. If the Hold-GO reward task (Fig. 2a) is switched to the Hold-Cue-GO reward task (Fig. 2d), the outcome-instructing Cue comes instead of the GO signal that was the correct sequence in the last trials. The GO signal instructs, and is associated with, action and its outcome (behavioral context) during the Hold-GO task, whereas the Cue signal plays this role during the Hold-Cue-GO task. Thus, the alternation of tasks induces a switch in the sequence of task events. At this time, neuronal responses to the Cue become maximal and quickly return to baseline levels (Fig. 2e, f). Thus, the neuronal activity may represent “associability of cue” (Mackintosh, 1975, Pearce and Hall, 1980). It is compatible with the notion of weighting of incoming evidence to reduce costs of trial-and-error search depending on their expected associative strength with outcome (Zhang and Sejnowski, 1999, Butts, 2003, Lacey et al., 2007). In studies of neuronal activity of rats performing behavioral tasks, the basolateral amygdala was suggested to encode a surprise at the change of attention produced by unexpected delivery as well as omission of reward (unsigned error, in contrast to signed reward prediction error posited as the signal from dopamine-containing neurons) (Roesch et al., 2010, Roesch et al., 2012).

Fig. 2.

CM-Pf neurons of SLF type report a “Surprise” induced by an unexpected change in behavioral context. (A) Monkeys depressed either left or right target guided by instruction (green LED) after holding a center (Hold) button for large or small reward. Because the action-outcome contingency was fixed during a block of 60–80 trials, monkeys reacted faster to a large-reward GO signal than to a small-reward GO signal. The contingency was then switched unpredictably after the 60–80 trial block. (B) Pseudocolor plot showing normalized discharge rates for 10 SLF neurons from 4 monkeys during 20 trials before and after switching action-outcome contingency. (C) Spike density histogram (upper panel) and reaction times (z-scores) of GO responses (lower panel) before and after contingency switch. Red asterisks indicate significant change in firing rate and reaction time following the contingency switch (P < 0.05, Wilcoxon signed rank test). (D) Unpredictable switches between the Cue-GO task and the Hold-Cue-GO task. Monkeys depressed an illuminated target immediately after center hold (Hold-GO task) or after a Cue instructing direction and outcome of forthcoming GO signal (Hold-Cue-GO task). (E, F) Pseudocolor plot showing normalized discharge rates for the SLF neurons (E) and population responses to GO and Cue signals before and after switches from Hold-GO task to Hold-Cue-GO task (F). Red asterisks indicate significant difference in firing rate between time before and after the task switch (P < 0.05, Wilcoxon signed rank test). From Yamanaka et al. (2016)

LLF neurons, another class of neurons in the CM-Pf, also encode expected and unexpected transitions of behavioral context, but in a different manner from that of the SLF neurons. During performance of an asymmetrically rewarded GO/NOGO task, LLF neurons were selectively activated when a small-reward action was requested, when a large-reward option had been expected based on an implicitly developed prediction (Minamimoto et al., 2005).

Representation of the associability of cue, as an average of prediction errors generated across the past few trials, is especially suitable for modulation of behavioral responses by CM-Pf neurons in a context-dependent mode. In support of these possibilities, unexpected external events suppress preexisting expectation and response bias (counteracting pre-action bias), whereas the expected demands boost prior expectation and behavioral reaction (Minamimoto et al., 2009, Minamimoto et al., 2014). On the other hand, it is natural to question whether CM-Pf neuron activity reflects mechanisms similar to some more general forms of attention, such as those discussed by Glascher et al. (2010), in which the surprise in facing the unexpected stimulus-action-outcome sequence results in state prediction errors to guide the next action. Further, decision-making in multidimensional environments is facilitated by a bidirectional interaction between attention and trial-and-error reinforcement learning processes (Leong et al., 2017). It is difficult definitively to rule out these alternatives, and some theories of associative learning do use an attention signal to control the assignment of salience to particular stimuli for learning (Pearce and Hall, 1980).

The representation of associability of context by CM-Pf neurons suggested here might not necessarily be a major component, but rather, a specific subcomponent, of a general surprise signal in the brain. However, our proposal about the mechanisms underlying associative learning of behavioral contexts, giving strong weight to the CM-Pf thalamus, is novel and is bolstered by multiple lines of evidence (Fig. 2). Unexpected delivery or omission of reward at the end of a trial may also produce signals of a general surprise and RPE encoded by midbrain dopamine neurons (Schultz, 1998a, Satoh et al., 2003, Nakahara et al., 2004, Bayer and Glimcher, 2005). The event associability signals of the CM-Pf emerge at a time when behavioral learning is about to occur (Fig. 2c). A question, however, is how the associability signals are used in the target brain region for learning prompted by the changes in task context.

Associative learning of events with behavioral context by cholinergic interneurons in the striatum

Thalamostriatal fibers make direct synaptic contacts with SPNs (Sadikot et al., 1992, Lacey et al., 2007, Ellender et al., 2013, Matamales et al., 2016) and with CINs (Lapper et al., 1992, Sidibe and Smith, 1999, Ding et al., 2010, Bradfield et al., 2013). CINs and fast-spiking neurons make up a major class of interneurons in the striatum. Though CINs represent a small population of striatal neurons (less than 3%), their broad arborizations and tiled distribution (Bolam et al., 1984, Wilson et al., 1990, Aosaki et al., 1995) provide dense cholinergic innervation throughout the striatum. Their cell bodies lie mainly in the matrix compartment, many near striosomal borders, and their fine axons innervate both the striosome and matrix compartments (Graybiel et al., 1986, Crittenden and Graybiel, 2016). It should be noted that the thalamostriatal projection, as estimated by molecular markers of neuronal activity, has a stronger impact on CINs than on SPNs (Bradfield et al., 2013). Studies of in vitro slice preparations point to the importance of the Pf-CIN synapses in regulating processing in the striatal circuits by both nicotinic and muscarinic acetylcholine receptors differentially distributed on different elements of striatal circuits and their afferents (Ding et al., 2010, Threlfell et al., 2012). Inactivating neuronal activity in the macaque CM-Pf complex by local infusion of the GABA_A receptor agonist, muscimol, almost completely abolished the responsiveness to behaviorally salient stimuli of the tonically active neurons (TANs), which are though largely to correspond to CINs (Matsumoto et al., 2001). Furthermore, inactivation of Pf has been shown to block an increase of acetylcholine release in the dorsomedial striatum during reversal learning, and to lead to deterioration in the performance of reversal learning (Brown et al., 2010, Bradfield et al., 2013). Selective removal of excitatory input from Pf to CINs in the posterior dorsomedial striatum is reported to reduce firing rates of CINs and to produce deficits in reward-oriented learning especially after changes in action-outcome contingency (Kato et al., 2011, Bradfield et al., 2013). Intriguingly, immunotoxin-mediated elimination of CINs (Kobayashi et al., 1995) from the dorsomedial striatum (DMS) or gene-specific silencing of M4 muscarinic receptor enhances reversal learning (Okada et al., 2014) and impairment of attentional set-shifting (Aoki et al., 2015), suggesting engagement of CINs and M4 receptors in the striatum to acquisition, extinction and relearning of reward contingency (Kato et al., 2011, Okada et al., 2014, Aoki et al., 2015). Thus, the thalamostriatal projections from CM-Pf seem critical for learning associations of behavioral context with outcome at least partly by directly affecting CIN-mediated processing in the striatum under conditions that require a switch in behavioral context.

In the non-human primate striatum, TANs and phasically active neurons are regarded as corresponding to CINs and SPNs (Aosaki et al., 1995, Raz et al., 1996, Shimo and Hikosaka, 2001, Apicella, 2007). The identification of TANs is more difficult in the rodent striatum than in the primate striatum. Electrophysiological, morphological, juxtacellular recording-staining and optogenetic confirmation suggested that the population of neurons identified as TANs is likely to have a large component of CINs (Bennett and Wilson, 1999, Inokawa et al., 2010, Bradfield et al., 2013, Atallah et al., 2014). A biochemical marker particularly sensitive to CIN activity was reported (Brown et al., 2010). The firing of TANs switches from a tonic mode at about 3–8 spikes/s to a transient pause for about 300 ms followed by rebound facilitation after sensory cues for motivationally salient events, such as reward and aversion (Kimura et al., 1984, Apicella et al., 1991, Morris et al., 2004, Yamada et al., 2004, Atallah et al., 2014). The pause is sometimes preceded by brief activation. Electrophysiological studies showed that the pause-rebound facilitation with or without a preceding small activation is generated by extrinsic synaptic drive and/or slowing of intrinsic pace-making (Maurice et al., 2004, Wilson, 2005, Ding et al., 2006, Ding et al., 2010, Goldberg and Reynolds, 2011). Evidence in slice experiments suggests that the CINs can control single spikes of SPNs, that this modulation occurs in striosomes as well as the matrix compartment of the striatum, and that this modulation is vulnerable to control by exogenous sources—thus echoing the highly temporally controlled spike activity of the CINs themselves (Graybiel et al., 1994, Crittenden et al., 2017).

The above mentioned knowledge is a basis of our hypothesis that stimulus associability signals derived from the CM-Pf play indispensable roles in associative learning about the cue that occurs within the striatum, with CINs working as key players in this thalamostriatal mechanism. The activity of CINs has been examined during behavioral learning (Aosaki et al., 1994, Blazquez et al., 2002, Atallah et al., 2014). Aosaki and others directly assessed the associative learning part of the model by recording activity of TANs of monkeys undergoing behavioral conditioning during which particular sensory events (click) became associated with reward (Fig. 3) (Aosaki et al., 1994). Through the conditioning for up to 3 to 4 weeks, population responses gradually developed in both putamen and caudate nucleus, suggesting that TANs acquired new association of click with reward and maintained for weeks as a population. On the other hand, when individual TANs were recorded for extended times during conditioning, a given TAN could acquire the responsiveness in at least as short a time as 10 min. Similarly, individual TANs could lose their responses to the conditioned stimulus after about the same duration of extinction training (Aosaki et al., 1994). Further, when the monkey was trained to receive reward in relation to a new conditioning stimulus, TANs were capable of switching their sensory responses to the new stimulus.

Fig. 3.

Changes in TAN responses to a click cue before, during and after click-reward associative learning in two macaque monkeys. Almost no licks were triggered by the click at the start of training, but by the fifth day, more than 90 % of licks were triggered and the values were nearly 100% for the reminder of the 3-week training period. (A) Among a total of 858 TANs recorded, only a small number (17%) of TANs responded to the click without following reward before conditioning (top). After conditioning, more than two-thirds of TANs responded to the reward-predictive click with a pause and rebound facilitation of sustained firing. In some TANs, the pause was preceded by an initial facilitation (not shown). (B) Ensemble average histograms showing responses of TANs to clicks (upper) recorded in the putamen (left) and caudate nucleus (right) before and during conditioning as indicated at left. Lower histograms show average orofacial muscle activity as a measure of licking. (C) Changes in percentages of TANs in the putamen (filled circles) and caudate nucleus (open circles) responsive to the clicks during training. From Aosaki et al. 1994.

Figure 4 summarizes our hypothesis by highlighting sharp contrast in temporal dynamics of responses to conditioning stimulus between CM-Pf neurons and striatal TANs after an unpredictable change of various behavioral contexts. CM-Pf neurons are selectively activated for very short time (a few trials for 10 to 20 s) instantly after switch of behavioral contexts supporting online associability of cue (Fig. 4a). On the contrary, striatal TANs learn and unlearn association through much longer time range (5 to 15 min of trials) (Fig. 4b). The development of response plasticity is tightly correlated with the probability that a given stimulus would evoke a behavioral response (Blazquez et al., 2002). Indeed, TANs (including optogenetically identified CINs) in the rat ventromedial striatum during a reward-based T-maze learning task showed a striking modulation of activity at the switch of reinforcement contingency (auditory- to tactile-instruction versions) and the reduced activity during overtraining (Atallah et al., 2014). In the same auditory and tactile cuing experiment, by contrast, SPNs were nearly insensitive to contingency switches and gradually lost outcome signaling while maintaining reports at trial start and goal approach, as though more related to stable aspects of task structure (Atallah et al, 2014). The high sensitivity of TANs to switches of action-outcome contingency is further supported by findings that removal of the excitatory input from Pf to the CINs in rat posterior DMS resulted in deficit in goal-directed learning after changes in the action-outcome contingency (Bradfield et al., 2013). Toxicogenetic ablation of CINs displayed behavioral deficits which were same pattern observed upon thalamostriatal denervation supporting the role of the thalamostriatal projection in updating action outcome contingencies (Matamales et al., 2016).

Fig. 4.

Contrasting temporal dynamics of responses after unpredictable switches of action-outcome association and sequence of task events in CM-Pf neurons (a few trials, 10 to 20 s) and striatal TAN (5 to 15 min of trials). (A) Pseudocolor plot showing normalized discharge rates for CM-Pf neurons before and after switching action-outcome contingency (upper panel) and switches from Hold-GO task to Hold-Cue-GO task (same record as Fig. 2B and E). (B) Time course of acquisition and extinction of conditioned response of an individual TAN. At the start of the recording period (left panel), the TAN recorded showed no response to the new stimulus, but it gradually acquired the response, as shown by records at 5 and 15 min. By contrast, the same TAN began the recording period showing a response to the old clicks. The records at 5 and 10 min of recording show the gradual loss of the response of the TAN as the extinction trials continued. From Aosaki et al. (1994). (C) Anatomical relations of primate CM-Pf and striatum (putamen, Put. and caudate nucleus, Cd N.) highlighting transmission of surprise signals from CM-Pf to the striatum where TANs learn and relearn associations of stimuli with outcome. Dopamine-containing innervation of the striatum is drawn as a reference.

It was previously proposed that TANs preferentially encode prediction errors to situational events (Macchi et al., 1984, Apicella et al., 2011) and unpredictably appeared task events (Apicella, 2007, Yamada et al., 2007). But the CM-Pf activity related to stimulus associability with reward as well as no-reward appears to inform TANs that “it is the time for a change” and to induce TAN activity at initial phases of acquisition, extinction and relearning of stimulus outcome associations. The role of the CM-PF - TAN system in weighing incoming evidence may participate in update of beliefs and adjusting behavior. In this sense, it may be reasonable to suppose that stimulus associability-based learning and relearning as instantiated by TANs modulates the learning rates of dopamine-dependent reinforcement learning in the striatum. This idea is consistent with the marked effects of Pf thalamic input on dopamine release in the local striatal regions by either direct action on dopaminergic axons or by way of CINs (Rice et al., 2011, Threlfell et al., 2012). By the same token, aberrations in this process could yield repetitive, overly focused and inflexible behaviors (Alexander et al., 1990, Graybiel, 2008, Crittenden et al., 2017).

Although the synaptic organization of CIN circuits in the striatum underlying behavioral learning, extinction and relearning still remains unclear, recent evidence suggests linking cholinergic signaling to corticostriatal synaptic plasticity through muscarinic receptors. Specifically, in direct pathway SPNs (dSPNs), cholinergic signaling through M4 receptors promotes long-term depression (LTD) of corticostriatal glutamatergic synapses (Girasole and Nelson, 2015, Shen et al., 2015). The M4 receptor-mediated LDT of dSPNs suppressed regulator of G protein signaling type 4 (RGS4) activity and blocked D1 dopamine receptor-dependent long-term potentiation (LTP) (Shen et al., 2015). As described above, selective elimination of CINs and silencing M4 receptors enhanced reversal learning and extinction learning (Okada et al., 2014). These results suggest cholinergic signaling through M4 but not M1 receptors on dSPNs is deeply involved in LTD of corticostriatal synaptic transmission for stabilizing learned reward contingency (Okada et al., 2014), and for replacing with new one through extinction and reversal learning if it is necessary. Furthermore, bidirectional plasticity, not only LTD but also LTP, is induced in dSPNs as well as in indirect pathway SPNs (iSPNs) (Shen et al., 2008, Girasole and Nelson, 2015), while cholinergic control of corticostriatal LTD selectively occurs in dSPNs (Shen et al., 2015). On the other hand, dSPNs and iSPNs are similarly innervated by cortical and thalamic afferents (Fujiyama et al., 2006, Doig et al., 2010, Doig et al., 2014, Huerta-Ocampo et al., 2014, Smith et al., 2014, Matamales et al., 2016, Parker et al., 2016). Thalamic afferents from centrolateral nucleus, rostral intralaminar nucleus, make large-amplitude responses through AMPA receptors, whereas those from Pf, caudal intralaminar nucleus, produce small-amplitude responses and LTD in dSPNs and iSPNs through NMDA receptors (Parker et al., 2016).

Roles of CM-Pf nuclei of the thalamus and cholinergic interneurons of the dorsal striatum in learning and enduring representations of events dependent on behavioral context

We suggested above that the contributions of CINs to the generation of conditional striatal outputs through SPNs and striatal interneuronal networks promote reinforcement learning guided by reward prediction error signals from dopamine neurons. Indeed, outcome-associated instructions in a GO/NOGO task in monkeys evoke pause-rebound facilitation responses of TANs depending on how the instructions are associated with outcome (Hori et al., unpublished data). Encoding trial-unique action-outcome contingencies may help in processing action signals in a conditioned manner. Indeed, the action signals encoded in most of SPNs (75%) are specific to combination of action with outcome (Hori et al., 2009). In addition, initial and peak activations of the SPNs occurred exactly at the pause response of TANs suggesting potential synaptic coupling between TANs and SPNs.

Accumulating evidence suggests the existence of synaptic interactions among CINs, various types of GABAergic interneurons and direct and indirect pathway SPNs. In the striatum, muscarinic receptors do not directly couple to endogenous ion channels but instead indirectly alter excitability through second messenger signaling cascades (Bernard et al., 1992, Calabresi et al., 1998, Shen et al., 2007, Goldberg et al., 2012). The sustained firing of CINs at a rate of 3–8 spikes/s under both in vitro and in vivo preparation and broad axonal arborizations (Bolam et al., 1984, Perez-Rosello et al., 2005) have been thought to create a background tone of acetylcholine that tonically drives muscarinic receptor activation. Some actions of acetylcholine on SPNs are mediated through G-protein-coupled muscarinic receptors and were thought to enhance corticostriatal synaptic transmission (Akins et al., 1990, Galarraga et al., 1999, Shen et al., 2005). GABAergic interneurons receive nicotinic excitation and provide potent inhibition to SPNs (Galarraga et al., 1999, Faust et al., 2015, Mamaligas and Ford, 2016, Ztaou et al., 2016), whereas their suppression largely reduces CIN-evoked inhibition of SPNs (Faust et al., 2015, Faust et al., 2016). This indicates that the majority of CIN-evoked inhibition of SPNs is derived from GABAergic interneurons, although a recent study has reported that the firing of individual CINs resulted in monosynaptic spontaneous inhibitory post-synaptic currents in SPNs through muscarinic receptors (Mamaligas and Ford, 2016, Crittenden et al., 2017). On the other hand, Smith and his colleagues conducted a quantitative ultrastructural analysis of the GABAergic and glutamatergic innervation of CINs in the postcommissural putamen of rhesus monkeys. They showed that 60% of all synaptic inputs to CINs originate from GABAergic terminals, whereas 21% are from putatively glutamatergic terminals that establish asymmetric synapses, and 19% from other (non-GABAergic) sources of symmetric synapses (Gonzales et al., 2013). Thus, CINs receive prominent GABAergic inputs from multiple origins, including a significant component from axon collaterals of direct and indirect pathway SPNs (Akins et al., 1990, Gonzales et al., 2013). Further studies are necessary to understand local circuit mechanisms underlying conditional nature of striatal outputs through dSPNs and iSPNs.

Direct-indirect pathways and striosome-matrix systems are basic processing circuits in the basal ganglia, receiving glutamatergic signals from the cerebral cortex and the thalamus and dopaminergic signals from the midbrain (Graybiel and Ragsdale, 1978, Gerfen, 1984, Albin et al., 1989, Crittenden and Graybiel, 2011, Atallah et al., 2014, Friedman et al., 2015). The projections from CM-Pf neurons reach both dSPNs and iSPNs (Vandermaelen and Kitai, 1980, Ding et al., 2010, Smith et al., 2014) as well as CINs (Ding et al., 2010, Bradfield et al., 2013). They may exert discrete impacts on SPNs of the two systems through CINs: transient presynaptic suppression of cortical input followed by prolonged enhancement of responsiveness of iSPNs suggesting involvement of Pf inputs in attentional shift (Ding et al., 2010). Thalamostriatal projections originating from subnuclei of the intralaminar nuclei have differential impacts: neurons in the rodent centrolateral nucleus produce larger amplitude EPSPs as efficient drivers for SPNs, especially dSPNs, than Pf neurons do (Matamales et al., 2016). Evidence further suggests that identified striosomal SPNs, not only SPNs of the cholinergic-rich matrix compartment, respond to input from the CINs (Crittenden et al., 2017). Neuronal processes of coticostriatal synaptic plasticity through dSPNs and iSPNs in concert with mesostriatal dopamine system and CM-Pf - CIN system are obviously the basis of behavioral learning and relearning. But recent studies suggest that they are much more sophisticated than previously considered. For instance, control of plasticity in corticostriatal signal transmission may not be restricted in D1- and D2-receptor-mediated segregation in different SPN classes: D1-receptor-mediated LTP in dSPNs (Reynolds et al., 2001, Calabresi et al., 2007) and D2-receptor-mediated LTD in iSPNs (Gerdeman et al., 2002, Kreitzer and Malenka, 2007). Both dSPNs and iSPNs may participate in corticostriatal synaptic plasticity in bidirectional (LTP and LTD) manner mediated not solely by D1, D2 receptors but also M4 and A2A receptors (Shen et al., 2008, Girasole and Nelson, 2015, Shen et al., 2015). Thus, although there are obviously many interacting circuits yet to be identified, currently available knowledge is compatible with our hypothetical model that the CINs and their inputs originating from CM-Pf play a critical role in associative learning, extinction and relearning based on salient stimulus.

Possible future research directions are to identify and manipulate signal contents and their plastic changes during associative learning and reversal learning initiated by salient stimulus in individual cell types composing corticostriatal, CM-Pf - CIN and mesostriatal dopamine systems. To this end, cutting-edge technologies including circuit-specific transgenic animals, optogenetics, chemogenetics and endoscopic imaging of activity of hundreds of neurons simultaneously are available in the rodent studies and becoming possible tools in the studies of non-human primates.

Dysfunction of the thalamostriatal system and disease states

CM-Pf neurons die in both animal models of Parkinson’s disease (PD) (Villalba et al., 2014) and in PD patients (Henderson et al., 2000, Henderson et al., 2005, Halliday et al., 2011). Reversal learning in PD patients is particularly impaired as well (Peterson et al., 2009, Buelow et al., 2015). Cell type-specific recording, stimulation and inhibition in PD model animals demonstrated critical involvement of thalamostiatal system. Corticostriatal afferents have received the most attention in PD research, and dysregulation of synaptic plasticity at these inputs is thought to contribute to imbalanced basal ganglia circuit function (Kreitzer and Malenka, 2008, Surmeier et al., 2009). In contrast, fewer studies have examined the thalamostriatal system in PD (Galvan and Smith, 2011, Smith et al., 2014). Recently, Parker et al. found, by using optical stimulation of cortical and thalamic inputs and recording from dSPNs and iSPNs, that thalamostriatal synapses are a site of maladaptive changes in PD mice. After dopamine depletion, relative strength of thalamic, not cortical, inputs reversed: from direct pathway biasing under intact striatum to indirect pathway biasing under PD state. Chemogenetic or optogenetic inhibition of thalamic inputs to striatum restored motor function (Parker et al., 2016). Thus, although our understanding about causal relationship of the circuit and synaptic mechanisms of thalamostriatal system to neurological disorders remains still limited, literatures suggest significant involvement of thalamostriatal system in normal and pathological function of the basal ganglia.

Conclusions

We highlight the thalamostriatal projections from the CM-Pf to CINs in the striatum and describe their substantial involvement in associative learning. We suggest that this thalamostriatal circuit encodes associability signals that promote learning independent of positive or negative reinforcement. We propose a hypothesis about associative learning of environmental events through the CM-Pf - CIN system. Neurons in the CM-Pf are strongly activated at changes in behavioral situations such as altered action-outcome contingencies and transitions in behavioral requirements, suggesting the encoding of the associability of behavioral context, an average of prediction errors generated across the past few trials. We discuss neuronal circuit and synaptic organizations based on in vivo/in vitro studies mostly using rodent subjects that we consider to underlie our hypothesis of associative learning and unlearning of CINs and reinforcement learning by dSPNs and iSPNs that is augmented by online associability signals from the CM-Pf. We examine in this light possible implications of CM-Pf - CIN dysfunction or degeneration of these thalamostriatal circuits in neurologic disorders by focusing on Parkinson’s disease.