How circuits for habits are formed within the basal ganglia

Abstract

Recent findings show that stereotyped movement sequences (habits) need the cortex in the learning phase, but after learning, the cortex can be inactivated, and the movement still be performed flawlessly. The motor program is dependent on the sensorimotor part of the dorsolateral striatum (DLS) and on synaptic plasticity in the thalamostriatal synapses. New findings from several laboratories have revealed a highly precise spatially interactive organization within the basal ganglia [DLS, substantia nigra pars reticulata (SNr), and the thalamostriatal parafascicular nucleus (PF)] and with precise input from the cortex. The DLS-SNr-PF-DLS loop is subdivided into many parallel loops. I now propose that these parallel loops can act to reinforce the activity of the different striatal projection neurons in the DLS that take part and that the synaptic transmission in DLS becomes potentiated each time the motor sequence is performed successfully, if rewarded through a dopamine burst. It is argued that after learning the DLS-SNr-PF-DLS loop can operate in isolation.

How motor actions are learned and stored has remained enigmatic. Recent findings from several independent studies offer a solution for how this may be achieved (1–7). Although the importance of the cortex has mostly been in a central position, the role of the basal ganglia has gained attention both in the context of motor learning and in the selection of behavior. The dorsolateral striatum (DLS) plays a crucial role in the formation of learned motor behaviors that eventually become stereotyped and are often referred to as habits (3). How these motor patterns are formed and called into action, and their dependence on the cortex, has remained unclear. A recent significant finding showed that a specific automatized learned motor program was localized to the DLS (4, 7). While the motor cortex was essential during the learning phase, once the motor behavior was learned, inactivation of the cortex did not impair the motor action, which continued to be performed flawlessly (4, 7). This finding raises the question of how the learning is achieved and where the information is stored!

The word habit, as currently used in this field of research, is broad and includes both very stereotyped movements that are repeated in exactly the same way, each time performed but also standardized movements that are nevertheless adapted to a particular situation, as when opening the door with a key or turning on the light in a dark room (3). Here, we are concerned with the former case. Habitual movements are in general associated with the DLS (3, 8, 9).

In classic primate studies, a relation between different thalamic compartments and different parts of the cortex, and a spatial organization within the striatum was described leading to a concept of parallel thalamocortical loops (10–12). More recently, however, studies in rodents (1, 2, 5, 6) have revealed a highly precise spatial organization from subpopulations of cortical neurons to specific striatal compartments and further to the output level of the basal ganglia, as well as to the different thalamic nuclei and their connectivity back to the striatum and cortex. This organization includes specific and precise connectivity within submodules of the motor cortex, DLS, and the output nucleus of the basal ganglia, the substantia nigra pars reticulata (SNr). Moreover, there are precise projections from the SNr modules to subsets of thalamostriatal neurons in the parafascicular nucleus (PF) (1, 2, 5, 6) that extend further to different compartments within the DLS. These studies mark a significant advancement of our understanding as they offer a new level of granularity, which serves as a game changer by creating a new conceptual framework for the understanding of the operation of the basal ganglia. I will argue below that the striato-nigro-thalamo-striatal loop is subdivided into a number of specific circuits critical for the understanding of how motor memory is formed and stored in the DLS (Fig. 1A).

Fig. 1.

The DLS-SNr-PF-DLS loop during learning and after learning. (A) Three subpopulations of neurons (1–3) within the DLS become activated from the motor cortex (three separate channels) during a movement sequence in the beginning of learning. The DLS neurons in each subpopulation [D1-expressing striatal projection neurons (SPNs)] represented in the scheme are those that directly impinge on three specific substantia nigra pars reticulata (SNr) subpopulations and thereby promoting separate components of the movement (direct pathway). The SNr neurons in turn project to separate subpopulations within the thalamostriatal PF that will excite the DLS neurons and reinforce the activity. Both the SNr and DLS are GABAergic and inhibitory (blue color), and the PF is glutamatergic and excitatory (red color). Dopamine input targeting thalamostriatal synapses are indicated in green. Successful task performance would lead to release of dopamine. The inset shows in a diagrammatic way the pattern of activity of one of the channels (no 1) in the motor cortex, DLS, SNr, PF, and motor centers during performance of a task. (B) During learning, the thalamostriatal synapses have been gradually strengthened as indicated by the breadth of the red axons to the extent that the motor cortex is not needed. The learned behavior can then be performed even without the involvement of the motor cortex. After learning, the DLS-SNr-PF-DLS loop contains the motor programs required to coordinate the movement sequence—that is the interacting subpopulations of neurons within the DLS, SNr, and PF. Abbreviations: DLS, dorsolateral striatum; MC, motor cortex; PF, parafascicular nucleus; SNc, substantia nigra pars compacta; SNr, substantia nigra pars reticulata.

Foster et al. (1) show that projections from the motor cortex to the subcompartments of the somatomotor part of the DLS exhibit a highly specific spatial organization. These subcompartments represent different body regions such as the forelimbs, trunk, and hindlimbs, and this specificity is maintained throughout the basal ganglia, including its output level the SNr. The SNr in turn is composed of many different subpopulations of neurons (40 or more), each projecting to distinct downstream motor centers, as shown by McElvain et al. (6, 13). The SNr can be regarded as a keyboard for movement control, where each key can elicit a specific type of movement. Neurons in each SNr subcompartment also provide efference copy information via collaterals to the specific groups of thalamostriatal neurons within the PF, which then project back to the DLS. The PF is composed of spatially organized subpopulations of neurons representing the input from different parts of the SNr projecting to different microdomains within the DLS (2, 5). The projection pattern from the PF to the DLS is also subdivided into forelimb, trunk, hindlimb, and facial areas that in turn target the appropriate parts of the DLS.

The Ölveczky laboratory (4, 7) has made a series of important findings. They designed a task in which rats were trained to perform two lever presses with a fixed interval (700 ms), a demanding task for a rat. The rats solved this by each making a set of movements taking 700 ms after the first lever press, followed by the second lever press. After learning the task, they performed it in a very precise and reproducible manner (Fig. 1B). When being successful they were duly rewarded. The set of movements performed between the lever presses differed, however, between rats. During the learning period the motor areas of the cortex needed to be intact, but after learning, the frontal areas of the cortex, including the motor cortex (M1 and M2) could be inactivated, and the rats could, to initial surprise, still perform the movements with the same precision. Fig. 1A describes the connectivity envisaged in the learning phase with activity in the motor cortex (see Inset) that activates a subset of neurons within the DLS, that in turn inhibit spontaneously active neurons in the GABAergic SNr that via collaterals will disinhibit neurons in the PF that will thereby increase their activity and further excite the DLS neurons already receiving input from the motor cortex. Dopaminergic input as in reward will potentiate the excitatory synaptic transmission. The DLS contains separate forelimb, trunk, and hindlimb areas, each receiving specific input from the corresponding parts of the cortical motor areas (mouse) (1). Wolff et al. (7) further showed that lesion in the DLS prevented the execution of the learned motor pattern, even though the rats could still perform lever presses and a variety of other motor patterns. Lesions in the dorsomedial striatum (DMS) had no effect.

The projections from the motor cortex are thus required for learning, and their terminals in the striatum are known to undergo synaptic plasticity during motor learning, as do projections from the thalamus (14–16). To test whether plastic changes had taken place in the DLS, Wolf et al. (7) administered the Zeta inhibitory peptide (ZIP), which is known to reverse synaptic potentiation at excitatory synapses in the cortex and the basal ganglia. When ZIP was injected into the DLS, the learned behavior was permanently blocked, while injections into the DMS or motor cortex had no effect in expert rats. This finding shows that, after learning, the motor cortex projections, and any associated synaptic plasticity in the corticostriatal synapses, are no longer necessary for executing the learned motor actions. Interestingly, animals in which the ZIP-induced blockade of the learned behavior occurred were, however, able to relearn the behavior with training, similar to control animals, indicating that no lasting damage occurred in the DLS circuitry. Since the motor cortex is not required for the learned behavior, the most likely candidate for plasticity is the thalamostriatal synapses, which represents a third of the excitatory input to the DLS that predominantly originates from the PF and related intralaminar nuclei (2, 5, 17, 18). A virus-induced blockade of the specific thalamostriatal projections to the DLS indeed causes a disappearance of the learned behavior (7), as did a lesion of the PF and related parts of the thalamus. This also occurs in expert rats, where the motor cortex had previously been removed, yet they were still able to perform the learned motor behavior until the PF projections were subsequently lesioned. The motor program thus can be viewed as stored in the DLS, and the execution depends on the synaptic plasticity in the thalamostriatal synapses.

What about the activity pattern in the SPNs in the DLS during the performance of the task? Individual SPNs became active in different phases of the movement, with an activity pattern corresponding to distinct phases of the movement and was very consistent over many trials (14). The conclusion was that the SPNs active within the DLS can serve as the motor program, projecting to and inhibiting different subpopulations of the SNr (6), which in turn will disinhibit separate downstream motor centers (the GABAergic SNr neurons are tonically active). Each of the SNr neurons, depending on its specific subdivisions, will then provide tonic inhibition of the target neurons in the PF during resting condition. When SNr neurons are inhibited by input from the DLS, they will not only disinhibit downstream motor programs but also disinhibit PF neurons, which will enhance their firing level (2, 6, 19) (Fig. 1, Inset). These activated, PF neurons will then further excite their target SPNs in the DLS. In essence, the activation of specific glutamatergic PF neurons will reinforce the activity of the DLS neurons that initially inhibited the SNr that modified the activity of the PF neurons.

The different subdivisions of the SNr project to distinct areas within the PF, and the different subpopulations of thalamostriatal neurons in turn project to separate parts of the DLS, such as the forelimb area (2) (Fig. 1). The spatial arrangement of axonal terminations from the SNr in the PF overlaps with the location of the thalamostriatal neurons. This suggests, as illustrated in Fig. 1B, a loop from DLS to SNr to PF, and back to DLS. This loop means that each time a group of DLS neurons is activated, the corresponding SNr neurons are silenced, disinhibiting the thalamostriatal PF neurons, which will then excite their target DLS neurons, thereby reinforcing the original activity. In other words, the feedback provided via the SNr-PF loop will thus reinforce the activity of the DLS SPNs originally activated.

During learning, the cortical input to the DLS provides a template in terms of activating the appropriate SPNs with correct timing. In Fig. 1A, the template is represented as three channels of commands from the motor cortex that in turn activate a set of three groups of SPNs in the DLS (marked 1, 2, and 3), controlling different aspects of the movement. If the rat receives a reward each time it successfully performs the task, a dopamine burst will be triggered, which most likely will potentiate the active thalamostriatal synapses during the task (15, 16, 20). The intracellular action of dopamine lasts over 2 s (21) and can thus facilitate a sequence lasting only 0.7 s as is the case here, particularly when the rat predicts reward. This will strengthen or reinforce the motor pattern performed, gradually making it independent of the cortical input. In other words, each time the rat performs the task correctly, it will lead to a strengthening of the thalamostriatal synapses, that ultimately will become independent on the input from the cortex. This can account for the formation of the independent DLS motor program, that during its formation in the learning phase needs the cortical input. It would seem likely that this DLS-SNr-PF-DLS loop in conjunction with dopamine input from the substantia nigra pars compacta (SNc) will form a circuit that will be in operation when learning new motor tasks dependent on the basal ganglia. In other words, a repeated successful task performance will lead to the formation of an independent DLS motor program. While cortical input forms a template that is necessary during the learning phase, the DLS-SNr-PF-DLS loop and the release of dopamine form a circuit that gradually becomes sufficient to elicit the behavior independently of the cortex critical for learning new motor tasks (compare Fig. 1A and 1B). Fig. 1B illustrates this condition showing that the PF projections to the DLS can be strengthened each time the movement sequence is performed successfully (reward/dopamine) through synaptic plasticity induced in the thalamostriatal synapses. When this has occurred a sufficient number of times, the cortical input will not be required any more and the DLS-SNr-PF motor program can operate independently relying on the potentiated thalamostriatal synapses. Thus, each set of SPNs in the DLS can activate a specific movment sequence through the SNr. This signal is forwarded with high specificity via the PF back to the DLS, and if the task is rewarded, a dopamine burst will further potentiate the relevant synapses. The cortical template will contain information not only of which groups of SPNs should become activated (Fig. 1B) but also information regarding the timing between the different groups of SPNs activated in the DLS that form the motor program. The information regarding sequential structure is most likely stored within the striatum (e.g., surround inhibition from, e.g., the first group of SPNs active) or through efference copy mediation of the different subpopulations of SNr neuron (22, 23).

The proposed role of the DLS-SNr-PF-DLS loop rests on the observation that the SNr neurons target the PF neurons, which in turn project to the corresponding parts of the DLS. A marked specificity is required which is fulfilled both with the different projection pattern of different subtypes of SNr neurons with distinct termination areas within the PF (6). In the PF, the projection pattern to the dorsal striatum is spatially organized (2) with separate projections to the DMS and the DLS, which in turn has separate projections to forelimb, hindlimb, and trunk areas of the DLS. The DLS-SNr-PF-DLS loop thus reinforces the motor program each time it is successfully executed. It would seem likely that this is a mechanism operating when learning the many stereotyped motor patterns, often called habits, that are typically dependent on the DLS, rather than the DMS (3). The thalamostriatal link as well as the basic design of the basal ganglia is conserved from the lamprey to primates (24–26) which further supports its evolutionary importance.

Here, I have focused specifically on the thalamostriatal projections from the PF to the DLS, but it should be recalled that the SNr and the globus pallidus interna project back to different parts of the cortex via different thalamic compartments (10). At the time, these loops were assumed to primarily give rise to motor commands, because the direct downstream control of midbrain and brainstem motor centers from the GPi and SNr had not been fully appreciated. Currently, when all SNr neurons are known to have an axonal branch directed upstream via the thalamus and another branch downstream to different motor centers, the upstream branch has been considered to serve at least partially as an efference copy (7). It would signal to cortical centers which motor commands are being issued from the SNr to downstream motor centers at a given moment of time (13). Such an efference copy transmission will of course provide indispensable information for further planning of motor actions at the cortical level.

In conclusion, I propose that the DLS-SNr-PF-DLS loop plays a major role in learning motor subroutines (habits). Thus, a subset of SPNs within the DLS activated in a task, inhibits specific SNr neurons, which in turn disinhibit the excitatory PF neurons, thereby reinforcing the excitation to DLS neurons. If this process is combined with a reward and dopamine release, learning is promoted and enhanced. Through repetition combined with a reward, a stable program is formed in the DLS that no longer is dependent on cortical input.

Data, Materials, and Software Availability

All study data are included in the main text.