Interpreting the role of the striatum during multiple phases of motor learning

Abstract

The synaptic pathways in the striatum are central to basal ganglia functions including motor control, learning and organization, action selection, acquisition of motor skills, cognitive function, and emotion. Here, we review the role of the striatum and its connections in motor learning and performance. The development of new techniques to record neuronal activity and animal models of motor disorders using neurotoxin, pharmacological, and genetic manipulations are revealing pathways that underlie motor performance and motor learning, as well as how they are altered by pathophysiological mechanisms. We discuss approaches that can be used to analyze complex motor skills, particularly in rodents, and identify specific questions central to understanding how striatal circuits mediate motor learning.

Overview of the phases of motor learning

The goals of motor skill acquisition range from a predator learning to adjust speed to intercept a moving prey, a musician learning precise movements for performance, a person learning to ride a bicycle, or more habitual behaviors, such as habitually reaching for a coffee mug in a particular cabinet in the kitchen [1]. These forms of learning require a series of steps, including the selection of a particular action by comparing the expected value of possible actions, executing the chosen action, and evaluating the result of the decision. With experience, we learn to associate sensory cues with rewarding or aversive events and to optimize activity for a preferred outcome.

As these steps can entail very complex behaviors, motor learning occurs through the acquisition of a sequence of simple actions or events (defined as ‘chunks’) necessary to accomplish a specific task and linking the information together into a single executable program [2,3].

Successful motor learning has been suggested to require transitions between four distinct phases [4]. The first phase involves random actions driven by motivation. During this phase, animals perform multiple trials with poor performance related to the outcome. The second phase involves insightful behavior, when a subject links a motor action with a goal, compares appropriate and inappropriate ways to achieve that goal, and begins to repeat the action. In the third phase, motor activity is adjusted to optimize the goal outcome: during this optimization phase, if the reward contingency is altered, the subject will easily learn to change strategies. In the fourth phase, the goal-directed action becomes a skill or a habit: if the reward contingencies are altered at this phase, change in strategies becomes more difficult (Fig. 1) [1,4–7].

Fig. 1.

Schematic of the circuits involved in the four phases of motor learning. (A) In the first phase, any random effort is driven by motivation and supported by convergent cortical, limbic, and dopaminergic signaling in the NAc, which then transfers this information to the SNc. (B) During the second phase (insightful behavior), the motor action is linked to a goal and through repetition, the SNc evaluates the appropriate way to perform the behavior, sending this information to the dorsal striatum, which integrates information from motor output, spatial cues from hippocampus, and value information from the amygdala. (C) The third phase requires optimization primarily through the engagement of the anterior cingulate gyrus and the DMS, which integrates inputs from motor cortex, cerebellum, and limbic structure to refine the motor action, and the involvement of DLS, for changes in strategies or high-demanding tasks. (D) During the fourth and final phases, the action becomes a habit. Activity in the DMS is reduced, and the DLS (particularly the direct pathway) remains engaged for performance. The TS is likely filtering irrelevant sensory stimuli to improve performance. M1, primary motor cortex; mPFC, medial prefrontal cortex; OFC, orbitofrontal cortex; TS, tail of the striatum; DLS, dorsolateral striatum; DMS, dorsomedial striatum; VS, ventral striatum; NAc, nucleus accumbens; Cb, cerebellum; GPe, globus pallidus external; GPi, globus pallidus internal; EP, entopeduncular nucleus; SNr, substantia nigra pars reticulata; SNc, substantia nigra pars compacta; VTA, ventral tegmental area; STN, subthalamic nucleus; DA, dopamine; dSPNs, direct pathway spiny projection neurons; iSPNs, indirect pathway spiny projection neurons.

Identification of specific brain regions in motor learning

From cortex to basal ganglia

The motor cortex has long been considered the main player in motor activity, starting with the discovery by Wilder Penfield and Edwin Boldrey of motor-sensory representation of the entire human body in the cerebral cortex [8]. Through electrical stimulation of different portions of the motor cortex in locally anesthetized, awake patients, Penfield and Boldrey were able to create a brain map visualized as a distorted human-like figure—the homunculus—whose form indicates the amount of cortical area dedicated to motor functions of each body part. Penfield’s work did not directly address the connections with other regions of the brain in the process of motor function [9,10]. Through extensive work on nonhuman primates, Mahlon DeLong and others evaluated the essential role of the basal ganglia in motor function [11,12]. While the involvement of these areas in motor performance has been studied widely, little is known about how these areas work together in motor skill acquisition.

Most of our knowledge on striatum and motor learning comes from the investigation of neurological disorders affecting basal ganglia. The most common is Parkinson’s disease (PD), described by James Parkinson over 200 years ago, and characterized by impairment in movement initiation (akinesia) and reduction in the amplitude and velocity of voluntary movements (bradykinesia), including freezing [13,14].

The dysfunction characteristics of motor disorders suggest that the basal ganglia are not only involved in the performance of the movement, but also involved in the skill acquisition. Indeed, it appears that PD patients have difficulty in acquiring new motor tasks or performing multiple tasks simultaneously, even when previously learned [15,16]. In the parkinsonian subject, the movement durations are more prolonged when simple movements are performed sequentially or simultaneously than when they are performed alone [15,17,18].

A widely used PD model is generated through injection of the compound 1-methyl-4-phenyl-1,2,3,6-tetrahydropyridine (MPTP), an impurity initially reported in a batch of the synthetic opiate fentanyl produced by a young chemistry student in Maryland [19], that later reappeared in California in the 1980s [20]. Users of this product developed Parkinson’s disease-like syndromes associated with destruction of dopamine (DA)-releasing neurons in the substantia nigra pars compacta (SNc) [20]. Monkeys treated with MPTP showed features of motor dysfunction, including rigidity, flexed posture, decreased movement [21], decreased reaction time, reduced movement amplitude, especially when movements were directed away from the body [22], reduction in spontaneous eating or drinking [18,23], and abnormal response to somatosensory stimuli [24,25]. These observations further helped to shift the interest in uncovering pathways of motor learning from a predominant focus on the cortex to include deeper brain structures, in particular the striatum [24,26,27].

From motor performance to motor learning

The first evidence that motor memory is a distinctive type of memory came from studies of amnesic patients, in particular Brenda Milner’s studies of the patient Henry Molaison (known as HM) [28]. HM exhibited severe and intractable epileptic seizures and was operated on by the neurosurgeon William Scoville, who attempted to treat patients with schizophrenic psychosis with lobotomy, a procedure that removed sections of the cortical temporal lobe. In HM’s surgery, in addition to regions of the temporal lobe, Scoville also removed most of both HM’s hippocampi and amygdala, in what he called a ‘frankly experimental operation’, based on his hypothesis that the seizures were initiated in those structures [29]. HM was almost immediately found to have lost normal memory function and was more completely analyzed years later by Milner [29].

Declarative memory is used in everyday language, for example, recognizing faces from the news, or describing a fact or event. Procedural memory refers to skill-based knowledge that develops gradually but with little ability to report what is being learned [30]. Although HM’s long-term memory was severely impaired by the surgery and he could not form new declarative memories, through practice, he was able to learn new procedural memories including motor skills, such as a mirror-tracing task, without awareness of having performed the task previously [29].

Subsequent experiments in rats and cats demonstrated that lesions in some cortical regions do not interfere with the ability to learn motor skills and overall motor behavior [31,32]. In contrast, lesions of different components of the basal ganglia cause a variety of motor symptoms, such as slowness of movements, altered coordination, postural abnormalities, difficulty initiating movements, and impaired acquisition of newly learned motor tasks [33–37], occasionally independently from each other [12,38,39] (additional examples summarized in Table 1).

Table 1.

The most common effects of lesions of components of the basal ganglia circuit. Generally, a final behavioral outcome is similar if upstream structures are affected or when the lesion is more generalized. Disruption of an individual pathway rather than the entire region alters the learning process and performance, although without producing major motor deficits. Corresponding references are in the bottom right corner of each row.

In the next section, we describe the brain structures involved in motor behavior and how they communicate with each other as we learn and perform a motor action.

Striatum in motor learning

Overview of different regions of the striatum

The striatum is often divided into two regions, the dorsal striatum and ventral striatum. Recently, the region posterior to the dorsal striatum, known as the tail of the striatum (TS), has been considered as an additional functionally distinct region [40].

In rodents, the dorsal striatum can be subdivided into a dorsolateral (DLS) and dorsomedial (DMS) regions, corresponding in primates to putamen and nucleus caudate, respectively. The DMS receives afferents mainly from prefrontal and associative cortices, while the DLS from sensorimotor cortical areas (Fig. 1) [40,41]. It is generally thought that the dorsal striatum is mostly involved in movement, particularly in automatized fine skills and micromovements embedded in an action [42,43]. Additional subdivisions of these regions are possible [44], but for the purpose of this review, we use a relatively simple approach that examines broad differences between the DLS and the DMS.

The ventral portion of the striatum (VS), containing the nucleus accumbens (NAc), receives projection from limbic cortices and amygdala and is more broadly involved in goal-related movements, a process by which the animal encodes values to the movement performance (Fig. 1) [45,46].

The TS receives projections primarily from sensory cortices and has been shown to play a role in avoidance and safety learning [47,48], and may also function to filter irrelevant sensory stimuli for goal-directed action [48].

Lesions of the dorsal striatum produce notable motor impairments, depending on the location of the lesion, the lesioning method, the behavioral task, and what is measured (Table 1) [49]. For example, disruption of the putamen and anterior caudate in monkeys causes slowness of movement [50], postural abnormalities [50], and altered acquisition of a motor task [51]. Lesioned monkeys are unable to carry out two concurrent voluntary motor acts, particularly in the response to visual stimulus, without impairments in visual or motor activity itself. Focal cooling of the putamen causes prolonged reaction time, but no changes in performance in a pointing task [52]. In some cases, similar lesions have altered movement velocity, without affecting the reaction time [12]. In cats, lowering the temperature of the caudate slows the reaction time to an auditory cue, followed by cessation of the task performance, without affecting the ability to reach for food [36].

In mice, the DMS appears to be involved in the acquisition of a skill, while the DLS is particularly engaged in mastered motor actions [42,53,54], and possibly facilitates task-relevant motor activity by inhibiting competing motor activity and conducting context-specific movement, as well as switching between motor patterns [50,55].

Dopamine contribution to motor activity

The striatum receives DA axonal inputs from the SNc and the ventral tegmental area (VTA). The DA system has long been implicated in reward and goal-directed behavior, promoting motor actions that lead to rewarding and suppressing actions that have aversive outcomes [56]. However, recent data suggest that DA may also have a direct role in motor function by regulating velocity and accuracy and promoting initiation of motor activity [57,58]. Optogenetic inhibition of SNc DA neurons prior to movement initiation impairs execution of movement [58]. However, inhibition after the movement has been initiated impacts acceleration and the probability of subsequent movements [58]. In mice, termination of spontaneous movement on a wheel correlates with the reduction in DA release in the dorsal striatum, and this function appeared to be independent of reward delivery [59].

Discrepancies in reports on DA’s role in motor performance may often be due to differences in the technique and behavioral setup. For example, when mice are head-fixed during testing, the DA system appears to be more active in the presence of a reward or during continuous locomotion, while in freely moving animals, DA activity correlates mostly with the vigor of an action [60]. Similarly, different striatal contributions in motor learning and performance are expected to depend on the experimental design.

The striatum and the phases of learning

How does the striatum contribute to the different phases of motor learning? It is assumed that the engagement of VS, DLS, DMS, and TS may differ during the acquisition of a goal-directed behavior (Fig. 1) [61].

During the first phase, initial motivation to perform a motor action is supported by convergent cortical and dopaminergic signaling in the NAc [62]. The NAc conveys this information directly or indirectly to the SNc [63]. The SNc then projects to the dorsal portion of the striatum and promotes initiation of motor action.

Motor actions are initially random, but at some point, the second phase begins as the animal gains insight about the outcomes of particular motor actions [58].

During the third phase, the DMS is primarily engaged and facilitates goal-directed action [42,43,51,64–67]. During this phase, both DMS and DLS have similar patterns of activation [42,43].

In the fourth phase, activity in the DMS diminishes and the DLS is primarily involved in the facilitation of habit formation and skill consolidation [53,54,68–74]. In this phase, the TS is likely engaged, filtering irrelevant sensory stimuli and thus improving performance [75].

Interestingly, associative and sensorimotor cortex projections to DMS and DLS appear to be both simultaneously active during skill acquisition, but less so during the execution of a mastered skill [54].

Baladron and Hamker [76] suggest that a combination of a desired goal and environmental cues triggers the selection of an objective or strategy in the DMS, which in turn induces the selection of an appropriate action in the DLS. This would occur during the third phase. Then, in the fourth phase, plasticity within the DLS occurs gradually as a habit is formed, allowing the action to be selected based solely on stimulus information, bypassing the need for goal-directed DMS activity. Single-unit recording during extended training on a lever press task showed a similar activity pattern between DLS and DMS after the skill was acquired [77], suggesting possible differences between recording from individual neurons vs. an ensemble or entire bulk population.

The direct and indirect pathways

Overview

The vast majority of the striatum is composed of spiny projections neurons (SPNs), accounting for up to 95% of all striatal neurons in rodents [78–80] and approximately 75% in primates [81,82]. In addition to SPNs, the striatum is populated by GABAergic and cholinergic interneurons [83]. Striatal SPNs form predominantly two distinct neural pathways [26,84].

The direct pathway (dSPNs) consists of GABAergic neurons expressing substance P and D1 DA receptors (D1R) [85–87]. This pathway receives glutamatergic input from the cortex [88], in particular sensory cortical and limbic structures, and from thalamus. In turn, dSPNs send projections to the motor portions of the globus pallidus internal (GPi) and the substantia nigra pars reticulata (SNr) [26,89].

Broadly, the indirect pathway neurons (iSPNs) are also GABAergic but contain enkephalin and express D2 DA receptors (D2R) [85–87]. The iSPNs receive projections from thalamus and cortex, particularly from motor cortices [88], and are connected to globus pallidus external (GPe) and the subthalamic nucleus (STN), which then project to the GPi and SNr. There is overlap in D1R and D2R expression in some regions of the striatum, particularly the NAc and the TS [88].

The collateral connections between direct and indirect pathways are also important contributors to motor performance, particularly the collateral projections of D1 SPNs to the GPe [90] and strong modulation of D1 SPNs by D2 SPNs [91], and appear to be affected in PD models [91].

Both direct and indirect pathways proceed to the thalamus, thus forming a cortico-basal ganglia loop [26,89]. Both dSPNs and iSPNs, as well as additional striatal neurons and basal ganglia regions including the GPe [92], receive projections from ventral midbrain DA neurons, the serotonergic raphe, and the cholinergic pedunculopontine nuclei [88].

There is increasing evidence that the activity of SPNs decreases as motor learning proceeds [93]. Neural activity is likely more generalized in the early stages of motor learning, but becomes more specific across the corticostriatal pathway as actions are refined over training [93]. Similarly, local DA release onto SPNs decreases as behaviors are incorporated into learned motor sequences [93]. Current theories suggest that with learning, SPNs activity becomes independent of DA control, so that new DA signals can assist complex learning by incorporating the different ‘chunks’ [2,54,56].

The go/no-go model

In the general view, following research directions initiated by recordings of striatal output regions by DeLong and colleagues in monkeys, the activation of dSPNs is understood to pause the activity of pallidal and nigral neurons, releasing the thalamic neurons from inhibition, thus leading to an increase in thalamic activity and promoting movement [11,24,26,94,95]. In contrast, activation of iSPNs inhibits the GPe, which increases the inhibitory action of GPi and SNr to the thalamus, thus suppressing movement [11,24,26,94,95]. Based on these findings, direct and indirect pathways are known by DeLong’s nomenclature as ‘go’ and ‘no-go’, respectively (Fig. 2) [26,84].

Fig. 2.

Simplified schematic of the basal ganglia circuit involved in motor learning in the human brain. Connections to and from the dorsal portion of the striatum are indicated. The caudate and putamen receive glutamatergic projections (red solid arrows) from cortex and thalamus, while sending GABAergic projections (blue dashed lines) to downstream structures. The direct pathway is composed of GABAergic D1R-expressing SPNs from the striatum to the GPi and SNr. The inhibition of these structures by the direct pathway, and therefore disinhibition of the thalamus, promotes movement and is classically referred to as the ‘go’ pathway. In contrast, the indirect pathway via D2R-expressing GABAergic projections inhibits the GPe, which in turn causes disinhibition of the GPi and STN, leading to reduced activity of the thalamus and is known as the ‘no-go’ pathway. DAergic projections from the SNc (green solid line) modulates activity of both caudate and putamen. Location and size of each region are altered for presentation purposes. Figure created with BioRender.com. GPe, globus pallidus external; GPi, globus pallidus internal; SNr, substantia nigra pars reticulata; SNc, substantia nigra pars compacta; STN, subthalamic nucleus; DA, dopamine; SPNs, spiny projection neurons.

In agreement with these functional connections, studies in mice show that optogenetic activation of dSPNs in the DMS produces an overall increase in movement [96–98], while activating iSPNs in the same region results in decreased overall locomotion or freezing [97,98]. When activation of dSPNs or iSPNs is unilateral, mice show an increase in contralateral or ipsilateral turns, respectively [97].

Because dSPNs and iSPNs express different DA receptor types, DA release in the striatum can have different effects. Lesion of SNc DA neurons, similar to that in PD patients, results in overall bradykinesia [24,26,99], while optogenetic stimulation of DA terminals in the striatum increases locomotion [100].

While this model has been enormously helpful in understanding basal ganglia circuitry, new findings are adding significant complexity (see Table 2). For example, in a lever press task in which the mouse has been extensively trained to perform a stereotyped action sequence, optogenetic activation of dSPNs in the DLS results in delayed movement initiation and slower performance as the animal switches to a different task, while iSPN activation halts initiation of those movements entirely, and impairs the performance of ongoing motor activity [101]. Interestingly, photoinhibition of either the direct or indirect pathway in the DLS also affects movement initiation and, particularly with inhibition of dSPNs, increases the probability that the animal disengages from the task [101]. iSPN engagement in motor behaviors appears to be dependent on novelty, as shown by ablation of D2-expressing SPNs in the DMS [102], while ablation of DLS dSPNs has a greater effect on motor performance on high-demand tasks [102].

Table 2.

Comprehensive description of the function of the direct and indirect pathway, organized by type of experiment.

	dSPNs	iSPNs	Reference
Optogenetic activation	Increase in spontaneous locomotion and movement velocity; causes form of dyskinesia that altered overall performance in the lever press task (at least for DLS)	Increase in freezing and decrease movement velocity; ipsilateral turns	[96–98,118]
	Mild delay in initiation of movement	Abolish first half of a motor sequence	[103]
	Slow initiation as the animal switched to other behaviors (and slow performance)	Aborted initiation as the animal switched to other behaviors (and affected performance)	[101]
Optogenetic activation prior to reversal learning	Contralateral bias	Ipsilateral bias	[113]
Optogenetic activation during outcome presentation (in DMS)	Reduced switches following reward	Increase switches after unrewarded trials	[102]
Optogenetic activation during learning	Reinforces behavior and/or spatial preference paired with stimulation; reinforces velocity or other features of trained movements	Reduces performance of stimulation-paired behavior; increases aversion for a spatial location; reduces velocity of movement	[117,118]
Optogenetic inhibition	Affected movement initiation; increased probability that the animal disengages from the task	Affected movement initiation	[101]
	Slows action initiation	No changes in action initiation	[103]
Electrophysiology combined with optogenetic tagging (DLS only)	Sequence related sustained firing activity during action execution	Sequence related inhibited firing activity during action execution	[107]
	Portion of both (~ 40%) active during initiation and/or termination of movements
Electrophysiology combine with optogenetic tagging (DMS only)	Both involved in element-level action execution		[103]
		More active after change in strategy
Fluorescent calcium indicators	Both active during movement and less during immobility (particularly in contralateral inward movements); similarly active during training; both encode velocity; coactive specifically predicting movement initiation		[94,97,99,101–102,105,111]
Simultaneous photometry recording of dSPNs and iSPNs	Both encode the sequences of spontaneous motor behaviors (activation of both pathways necessary for action selection). Both similarly selective for similar actions		[112]
Single-cell recording	Similar activity clustering in regard to spatiotemporal features of the movement		[109]
Lesion of the DLS (and partially DMS)	More likely to signal sequence initiation and termination	Preferentially encodes the transition between sequences	[103]
DLS inhibition during reward-based lever press task	Critical for completion of responses in serial order (without affecting reversal learning)	Behavior not affected, except for a transient improvement on second step of the task	[176]
In DMS	No effect on rotarod performance; stimulates novel object recognition, more active during rewarded presentation; D1 availability correlates with instrumental learning bias from rewarded trials	Affects early training performance on rotarod; inhibits novel object recognition; important to inhibit competing actions; disengages in later stages of learning; modulates dSPNs inhibiting previously learned commands and allow new goal-directed learning; more active during unrewarded outcomes	[96,108,109,113,128]
In DLS	Active in late stages of learning; necessary for performance in early and late learning; necessary to develop novel strategy and habit learning	iMSNs active in early skill learning	[102]
D1R or D2R KO on rotarod	Decreased performance, later improved	Good performance at early training, without any improvement. Poorer performance if animals exposed to high speed first	[133]
D1R antagonist	Lower motor ability at high speed of treadmill, not related to changes in reward sensitivity	No changes

Geddes et al. [103] introduced a custom-written lever press task for mice to obtain a reward. Their task design provides the identification of discrete action sequences, distilling steps in the behavior for better correlation with optogenetic manipulation. They found that inactivation of dSPNs in the DLS led to slower action initiation, while inhibition of iSPNs produced no changes in action initiation. This suggests that the direct pathway is more likely to signal sequence initiation and termination, while the indirect pathway may be preferentially encoding the transition between sequences [103]. It is possible that dSPNs promote selected actions, while iSPNs suppress alternative actions [49].

Other recent studies indicate simultaneous activity and dynamic competition of dSPNs and iSPNs for the control of performance, not only during initiation and/or termination of movements, but also during execution of the single elements of a given task, including adjustments of the velocity of the movements, and in general acquisition and organization of the sequences [43,53,104,105]. These reports suggest that the activation of both pathways is necessary for appropriate action selection and that both are involved in execution of each single element within a given task [103,106–112].

Tai et al. [113] explored the role of DMS direct and indirect pathways during distinct stages of a learning paradigm in which the mouse has to choose between left and right sides to find a water reward. As learning proceeded, the mouse chose the side with the greatest probability of reward. The activation of dSPNs or iSPNs prior to the task generated a contralateral or ipsilateral bias, respectively [113]. Similarly, the activation of dSPNs during outcome presentation led to perseverance in choosing the same side following reward, although the activation of iSPNs increased the rate of switching sides after unrewarded trials [108]. This suggests that the indirect pathway has a higher selectivity for specific actions depending on the value rather than movement per se, while the indirect pathway works independently of reward [108,114–116]. In a place preference task, the activation of dSPNs in the DMS can reinforce motor action and/or the spatial location paired with the stimulation [117], as well as velocity or other features of the trained movement [118]. In contrast, iSPN activation impairs behavioral performance, increases aversion for a spatial location, and decreases movement velocity [117,118]. Overall, these studies call attention to the influence of striatal activity on reward and aversive behaviors.

In a recent study, Matamales et al. [119] demonstrated that iSPNs can inhibit dSPNs. This inhibition disengages dSPNs from previously learned tasks in favor of new tasks, by decreasing activity related to previously learned tasks and allowing new learning. This adds a new active function for iSPNs during learning and an additional layer of complexity.

In summary, it appears that the classic go/no-go model will need to incorporate recent evidence that both dSPNs and iSPNs are simultaneously active during motor learning and execution.

Open questions on the go/no-go hypothesis

dSPN and iSPN populations exhibit overlapping activity during motor behavior, with apparent spatial and temporal clusters [120], and in close relationship with reward [108]. Differences between the classical interpretations of go and no-go circuits and recent findings are likely dependent on technical differences (Table 1). For example, manipulations of brain activity using lesion models, optogenetic activation, or the simultaneous inhibition of multiple classes of neurons, may not precisely recapitulate the activation pattern of specific pathways during motor activity or motor learning. The differing interpretations of the role of direct and indirect pathways may also be dependent on whether individual neurons or the entire ensembles were recorded.

Whether the direct and indirect pathways of DMS are involved in the late stages of learning is still an open question. The direct pathway, at least for the DLS, may rely on reward for the early training during less demanding tasks (phase 2), but with overtraining it becomes more engaged during performance rather than by reward availability (phases 3 and 4, Fig. 1). The indirect pathway may promote learning and task shifting in both early training and late training, inhibiting unwanted movements and previously learned actions. The latter is consistent with studies on PD patients and supports the view that the basal ganglia are involved in the inhibitory control, particularly of antagonist motor actions [49,121,122]. However, the use of reward in the training process or testing appears to activate the two pathways in different manners with respect to spontaneous locomotion [112,114–116]. To interpret the role of the striatum in motor learning, we must explore the roles of striatal interactions with the reward and the limbic system.

Inputs and outputs of the basal ganglia

Dopaminergic modulation of direct and direct pathway

Tonic and phasic DA firing plays important roles in learning from positive (rewarding) or negative (aversive) feedback. Most studies on motor learning use paradigms with both aversive and appetitive stimuli. Both forms of motivation facilitate learning, but aversive and appetitive stimuli engage different striatal circuits: For example, DA release dynamics in the striatum are different for aversive and appetitive states [123]. Aversive stimuli decrease the tonic firing of DA neurons but do not affect phasic release [124–127]. In contrast, rewarding stimuli increase the phasic firing of DA neurons without impacting tonic release [128]. These differences may influence behavioral outcome in subtle ways.

The importance of the DA system in motor performance and learning is evident in PD patients. PD patients in an unmedicated state display impaired learning from positive feedback, but improved learning from negative feedback compared with healthy controls [116]. With L-DOPA, however, patients show improved learning from positive feedback, while presenting impaired learning from negative feedback [116].

The difference is likely due to differential engagement of iSPNs and dSPNs. It is often suggested that D2 receptors have a higher affinity for DA than the D1 receptors [129,130], although this has been a challenging question to study in vivo. Consistent with this possibility, however, blockade of dSPN-mediated synaptic transmission impairs learning of the reward-driven conditioned place preference task, while no effect is observed in aversion-driven inhibitory avoidance tasks. In contrast, synaptic blockade of iSPNs had no effect conditioned place preference, but impairs inhibitory avoidance [131].

Nakamura et al. [132] examined motor leaning in D1R and D2R knockout (KO) mice on a rotarod, a widely used test to assess motor function in rodents. The D1R KO mice initially performed poorly but improved throughout the training, while D2R KOs initially exhibited fair performance with no further improvement. When trained to run on a step-wheel system with an irregular rung pattern to reach a reward, the D1R KO mice improved performance, running closer to the spout for a longer time and missing fewer steps, while D2R KO mice performed similarly to their WT control littermates. The authors noted that the rotarod is aversion-driven, while the step wheel is reward-driven, which may account for differences between the two tasks [132].

A limitation of constitutive KO models is that the complete ablation of the receptor in all regions may have indirect effects on the behavior, as well as possible compensatory mechanisms. When wild-type mice were treated with D1R antagonists, they exhibited a poorer motor ability at high treadmill speed that was unrelated to changes in the value of the reward [133].

Human studies show that D1R availability in the dorsal striatum correlates with instrumental learning bias from rewarded trials, in agreement with studies, suggesting that dSPN architecture plays a major role in motor learning that is highly dependent on reward [134]. This is consistent with the assumption that DA release acts as a teaching signal that modulates corticostriatal synapses [135,136]. Despite differences in the tasks, these data support the contribution of DAergic activity to motor actions in the early phases of learning (phase 1 and phase 2, Fig. 1), particularly in modulating the activity of dSPNs. Then, once DAergic structures disengage during phase 3, dSPNs continue to contribute to performance independently of the presence of reward. The limbic system also plays an important role in motivational aspects of motor learning.

Limbic—striatal roles in motor learning

The limbic system is composed of multiple brain regions including the amygdala, hippocampus, medial prefrontal cortex (mPFC), orbitofrontal cortex, and cingulate cortex [137]. These areas each project to the striatum, especially the NAc [40], and their activity has been shown to correlate with motor learning [138–140]. Similarly to the striatum, the limbic system is activated at various phases of motor learning and can influence the acquisition of motor skills via direct and indirect connections to the striatum (Fig. 1) [4,138,141].

As discussed, during the first phase of motor learning, the NAc processes valence inputs from these limbic structures to generate motivation for motor action [62,142].

In the second phase, the hippocampus becomes more engaged and is likely involved with memory of past performances and spatial location of rewards [143–145]. Information from the hippocampus is conveyed to the amygdala, which assigns positive or negative valence to contexts and sensory stimuli [146]. Both the amygdala and hippocampus project to the medial prefrontal cortex and anterior cingulate cortex, areas involved in goal-directed action selection [147]. Convergent information from these cortical regions, as well as DAergic input from the SNc to the DMS, facilitates learning of action–outcome contingencies [144,148,149].

In the third phase, the anterior cingulate gyrus integrates inputs from the motor cortex and other limbic structures to encode errors that occur during performance. This information is conveyed to the DMS and enables optimization of motor strategy [40,65,150].

In the fourth phase, as an optimal strategy is learned, mPFC input to the DMS diminishes and DLS activity increases, which promotes habitual motor responses required for skilled actions [54,67]. Alterations in the activity of limbic structures by appetitive and aversive stimuli may influence this transition.

mPFC and hippocampus

The mPFC and hippocampus are involved with the acquisition of habits and motor skills. Motivational states can alter synaptic strengths of these inputs to the NAc during learning [138,151].

The NAc is composed of a core and shell regions where phasic DA release is correlated with rewarding stimuli, except in the ventral medial shell region, where phasic DA release correlates with aversive stimuli [123,152]. Plasticity of the projections from mPFC or hippocampus to the NAc core is differentially altered by tonic and phasic DA release. Phasic but not tonic DA release enhances excitatory input from the hippocampus in a D1-dependent manner [153].

In contrast, mPFC inputs are suppressed by tonic but not phasic DA release in a D2-dependent way [153]. Given that tonic and phasic firing is thought to be modulated by aversion and reward, respectively [56,116], it is likely that in the NAc core, hippocampal inputs provide a larger contribution to learning appetitively motivated motor tasks, while PFC inputs provide a larger contribution to aversively motivated motor tasks. In contrast, in the NAc shell, hippocampal excitatory inputs are diminished by phasic DA release in a D1-dependent way. Given that the ventral medial shell region seems to be involved with aversive learning rather than reward, this may indicate that inputs to this region are regulated in opposite directions than the NAc core. Release of DA in the NAc shell also attenuates excitatory inputs from the basolateral amygdala (BLA) [154]. Given the opposite regulation by DA release claimed for hippocampal inputs to shell vs. core, it is likely that BLA inputs to the core region are enhanced by DA release, although this has not been explored. This highlights the possibility that differential dynamics of DA release in the NAc for aversive and appetitive circumstances may impact the contribution of the amygdala, PFC, and hippocampus to motor skill learning, particularly during the second phase of motor learning.

The amygdala

The amygdala plays a key role in the shift from goal-directed to habitual behavior. For example, bilateral ablation of the central nucleus of the amygdala (CeN) prevents formation of habitual responses [70]. The CeN likely exerts this effect via its GABAergic inhibitory indirect projections to the DLS, since combined unilateral ablation of CeN and contralateral ablation of DLS also impairs habit formation [70]. In contrast, ablation of the BLA impairs its excitatory glutamatergic projections and has been shown to impair goal-directed action and promote habit strategies [155,156].

How do appetitive and aversive motivational stimuli impact the amygdala’s contribution? Plasticity at BLA to NAc synapses is enhanced by appetitive and diminished by aversive learning. Stimulation of these terminals has also been shown to promote reward learning and attenuate fear learning. In contrast, BLA-to-medial CeN synapses are enhanced by aversive stimuli and their activation promotes avoidance [157]. Since CeN activity promotes habitual action, this suggests that aversively motivated learning promotes habit learning. Indeed, exposure to conditioned and unconditioned aversive stimuli enhances DLS-dependent habit learning [158–160]. These findings suggest that appetitive and aversive motivational states between the second and the third phase can influence action to habit transitions by differentially engaging amygdala nuclei.

The contributions of limbic structures to motor skill learning vary depending on appetitive vs. aversive motivations, which may cause subtle differences in behavioral performance and skill acquisition. This should be taken into consideration when studying motor learning and researchers need to exercise caution when comparing results of tasks that use different forms of motivation.

Cortico-basal ganglia-cerebellar network

Studies in monkeys using retrograde transneuronal transport of rabies virus indicate that the basal ganglia form reciprocal connections with the cerebellum at both cortical and subcortical levels [161–163]. Multiple lines of evidence indicate significant communication between the cerebellum and the basal ganglia via diverse subcortical pathways, which are independent of the cerebral cortex, that is, the dento-thalamo-striatal and dento-nigral pathways [161,162,164]. The dento-thalamo-striatal pathway connects the deep cerebellar nuclei (DCN), particularly the dentate nucleus, to the contralateral putamen and GPe via intralaminar thalamic nuclei (ILN), while the dento-nigral pathway links the dentate nucleus to both GPi and SNr via the superior cerebellar peduncles [161,162,164].

These findings reveal a complex framework of interactions across functionally distinct regions. How these circuits influence each other to generate an appropriate motor output is an open question. As mentioned, the basal ganglia have an important role in procedural learning. The cerebellum accomplishes the goals that have been evaluated by basal ganglia through supervised learning. For this purpose, it uses information about movement error provided by sensory feedback. The climbing fibers (CFs), originating from the inferior olive neurons, are thought to instruct learning by signaling the occurrence of movement errors and inducing plastic changes (i.e., long-term depression, LTD) at the parallel fiber–Purkinje cell synapse [165–167]. Given the inhibitory input of Purkinje cells to DCN, the error-driven LTD can regulate DCN efferent activity to other brain sites in the direction, which minimizes errors consequent to the movement. The loops through basal ganglia and cerebellum instruct the cortex to perform what has been learned in the subcortical loop.

Using a multisynaptic tracing approach, Xiao et al. [168] recently demonstrated that medial, interposed, and dentate DCN output targets SPNs, as well as cholinergic interneurons, in both DLS and DMS. They further showed that chemogenetic inhibition of a large portion of interpositus/medial DCN neurons affected the reward-driven forced alternation. This effect was replicated in mice following inhibition of the thalamo-striatal axons that arise from thalamic cells innervated by DCN axons, suggesting that DCN outputs can modulate the striatum-dependent behavior. Another study demonstrated that the selective suppression of ILN neurons innervating the striatum affected visual discrimination and the behavioral flexibility, including reversal learning and attentional set shifting of learned motor responses, without impacting the motor skill learning in the accelerating rotarod test [169].

Studies in songbirds have helped unveil the role of the basal ganglia-thalamo-cortical loop in learning and plasticity of a complex sensory-motor task, song learning [170]. In songbirds, a vocalization-related basal ganglia nucleus known as Area X is functionally connected to the DCN via the dorsal thalamic zone; indeed, DCN stimulation provokes a strong response in pallidal-like neurons of Area X that is suppressed by glutamatergic blockers injected into the thalamic zone. Lesion of DCN impairs the development of song timing properties, suggesting that song learning may require the basal ganglia–cerebellum interaction [170].

These findings, together with the demonstration of a short-latency communication (about 10 ms) between the cerebellum and the basal ganglia [162], indicate the cortical–cerebellar basal ganglia system may operate together to maximize the effects of each element of the entire motor action. The cerebellum is important for detecting errors and using those errors to update an internal forward model, which predicts the outcome of motor commands and drives motor learning and skill acquisition [171], likely during the third phase of motor learning. Recently, it has been proposed that the climbing fiber activity can also act as a reward prediction signal, suggesting a cerebellar involvement in reward-based associative learning [172]. This implies that the entire cortical–cerebellar–basal ganglia network may operate at different spatial and temporal scales during goal-directed behavior driven by both motivation and experience. In this context, it is important to understand the contribution of neuromodulators, including not only DA, but also NE, serotonin, and acetylcholine synapses that may drive and shape the activity of the network.

The challenge of studying the synaptic basis of motor behavior

Our current knowledge of motor learning comes from studies on models of motor dysfunction or analysis of simple in-laboratory skills that may not apply to more complex skills. The increased investigation of complex motor skills, given recent improvements in recording techniques, including the ability to generate pose estimates from video recordings, is something to be built upon. Moreover, these approaches may provide a better understanding of a complex continuous brain process that not only encodes the initiation and termination of a movement, but also encodes the velocity, accuracy, value, and expected outcome. Similarly, recording the activity of multiple populations of neurons in brain regions during complex motor activity in moving animals is challenging, but the development of more sensitive imaging techniques has proven helpful.

The experimenter should consider feedback signals such as environmental cues that may affect motor performance and learning, including visual cues, availability of reward or aversive stimuli, and the presence of constraints in the movement itself. This is particularly relevant for investigations of the basal ganglia, which are also decoding sensory stimuli and outcome value.

It is very important for the experimental design to differentiate between goal-directed and habitual behaviors. One way to remove the reward component is to devalue the reinforcing factor and retest each subject [173]. Motor behaviors are often partially goal-directed even after they become habits [173], as could be the case in a study by Vandaele et al. [77], where, in contrast to other studies, the DMS was found to be active even after extensive training.

An additional consideration is whether to record or manipulate larger or smaller populations of neurons. For example, early studies showed differences in behavioral outcomes depending on focal vs. diffuse lesions [49] (Table 1). Similarly, optogenetic activation or inhibition of all MSNs can lead to very different results from those that elicit the specific activation of direct or indirect pathways individually (Table 2). The basal ganglia may operate on a spatiotemporal pattern, rather than an all-or-nothing type of activity, that needs to be specific to enable the accuracy of movements and improved performance [60,93,120]. Even the DA response may vary depending on the interval between trials, shifting from goal-directed to cue-directed response when this interval is extended [174]. Together, these differences related to the specific motor task pose an additional challenge to the study of motor function and skill acquisition.

Where we stand and where we are heading

We have addressed the complexity of multiple striatal connections, each of which is involved in skill acquisition. We have stressed the differences in D1R and D2R roles on SPNs, and how different responses to specific neurotransmitters might account for different activity patterns despite comparable cell numbers between dSPNs and iSPNs. We have outlined a means to organize motor learning into four phases (Fig. 1), each involving multiple brain regions or neuronal populations in specific combination and possibly in a specific temporal pattern. The initial random choice of distinct action in phase 1 is dominated by activity in the limbic circuit. Some actions have no outcome, and others have positive or negative outcomes. During the second phase, an animal begins to recognize that certain actions and outcomes are connected, particularly through the involvement of DAergic inputs, activity of the indirect pathway, and input from the amygdala and hippocampus. Repetition of the action leads to optimization of the action itself, with contribution from cerebellar inputs, as well as motor, premotor, and sensorimotor cortices. Finally, the action becomes more automatic, a well-learned skill or possibly a habit, with strong activity in the DLS, and possibly less activity in the DMS. Nevertheless, a healthy doubt remains on these classifications, as the values collected by available techniques are complicated by factors including total synapse number, surface area of synaptic contacts, and other parameters: Indeed, these factors provide additional challenges for optogenetic, photometry, and in vivo microscopic imaging approaches that do not clearly indicate the number of neurons or synaptic connections involved in a behavior.

During a behavioral task, the number of actions available to the subject at any time varies from study to study, and the difficulty of the task affects learning. A general conclusion from loss-of-function studies is that it is difficult to distinguish ‘learning’ from ‘performance’, since learning is measured by performance. In a related issue, when cues are associated with a behavior, the consistent appearance of a cue relative to the variability in motor action can create a bias for correlations of neural activity with the cue itself. Indeed, in typical experimental designs, the experimenter controls the stimulus, and particularly when the stimulus is salient or familiar, the time delay interval is very short. Nevertheless, for movements in learned behaviors, a wide range of variables can affect timing, body position, velocity of the movement, and motivation. There is also a question of whether specific rodent movements are stereotyped. For example, Kawai et al. [175] showed that well-trained rats exhibit a very consistent motor action when reaching to press a lever for a reward, but this may be specific for very defined conditions. Studying natural behaviors and avoiding the use of rapid reward or aversion may eventually be more informative, but such experiments are challenging to design and interpret.

Continuous measurements of behaviors during training and testing can be analyzed and interpreted with computational methods. Statistical methods can dissociate the neural activity that corresponds to the motor aspect of the task from the sensory components, to resolve the bias of associating a specific cue with the brain activity. Optogenetic or pharmacological manipulations can evaluate the necessity for specific neuronal activity for indicated features, and whether the activity is sufficient for the motor task or can be compensated by the activity of other regions.

While there is experimental support for long-held hypotheses of motor performance and motor learning, these have been challenged by more recent discoveries, suggesting that the classical view may require some modulation. With each technique posing advantages and disadvantages, a combination of approaches can be more informative. Experimenters will need to optimize their design to differentiate contribution of distinct populations of neurons and regions within the basal ganglia, as well as to measure the timing of the contribution of each feature related to the behavioral task in motor performance and skill acquisition.

Given the complexity of the circuits involved in motor learning and the multiple variables of each task itself, extra care is needed when interpreting the animal behavior and the interaction between the different regions. A rigorous experimental setup may provide a better understanding of the motor task itself, and the contribution of different brain regions to the task, leading to a more profound understanding of the basal ganglia involvement in motor activity. This will be useful for the evaluation of early deficits in motor disabilities, from neurodegenerative and neurodevelopmental disease to addiction, and the development of more efficient therapies.

Abbreviations

BLA

basolateral amygdala

Cb

cerebellum

CeN

central nucleus of the amygdala

CF

climbing fibers

DA

dopamine

DCN

deep cerebellar nuclei

DLS

dorsolateral striatum

DMS

dorsomedial striatum

dSPNs

direct pathway spiny projection neurons

EP

entopeduncular nucleus

GPe

globus pallidus external

GPi

globus pallidus internal

HM

Henry Molaison

ILN

intralaminar thalamic nuclei

iSPNs

indirect pathway spiny projection neurons

KO

knockout

LTD

long-term depression

M1

primary motor cortex

mPFC

medial prefrontal cortex

MPTP

1-methyl-4-phenyl-1,2,3,6-tetrahydropyridine

NAc

nucleus accumbens

OFC

orbitofrontal cortex

PD

Parkinson’s disease

SNc

substantia nigra pars compacta

SNr

substantia nigra pars reticulata

STN

subthalamic nucleus

TS

tail of the striatum

VS

ventral striatum

VTA

ventral tegmental area