Are We Ready for a Natural History of Motor Learning?

Abstract

Here we argue that general principles with regard to the contributions of the cerebellum, basal ganglia, and primary motor cortex to motor learning can begin to be inferred from explicit comparison across model systems and consideration of phylogeny. Both the cerebellum and the basal ganglia have highly conserved circuit architecture in vertebrates. The cerebellum has consistently been shown to be necessary for adaptation of eye and limb movements. The precise contribution of the basal ganglia to motor learning remains unclear but one consistent finding is that they are necessary for early acquisition of novel sequential actions. The primary motor cortex allows independent control of joints and construction of new movement synergies. We suggest that this capacity of the motor cortex implies that it is a necessary locus for motor skill learning, which we argue is the ability to execute selected actions with increasing speed and precision.

Biology, like other scientific disciplines, has its model systems. For example, E. coli, C. elegans, and Drosophila are considered simple experimental systems for the discovery of molecular, cellular, and genetic mechanisms that then generalize to untested species. In motor neuroscience we also have various model systems. The assumption that findings in model systems can generalize is implicit to the neuroscientific enterprise in so much that work in multiple model systems is ongoing, funded, and published. It is rare, however, to find any explicit mention of the logic underlying the choice of a particular model system, beyond perhaps its experimental tractability, and even more rare to find overt comparisons made between model systems in the motor learning literature (but see Olveczky, 2011; Shadmehr and Wise, 2005). Choice of model system should be based on judicious use of knowledge of phylogenetic relationships and these chosen model systems should be distributed widely across the tree of life in order to reduce the risk of studying an idiosyncratic species (Krakauer et al., 2011).

Use of the term phylogeny is likely to seem jarring in a review about motor learning and, if so, speaks to the almost complete absence of evolutionary considerations in the mainstream motor control or motor learning literature. This is surprising as a shared natural history provides the opportunity for fruitful generalization: “The observation that all of life shares an evolutionary history imposes enormous regularity on biology in the form of conserved traits amenable to general description and explanation” (Krakauer, 2002). Thus, knowledge of phylogeny can help build more powerful general conceptual frameworks.

In this review, in addition to making a case for a comparative model systems approach, we argue that there is continuing usefulness for decomposition and localization as heuristic strategies in mechanism-based neuroscience research (Bechtel and Richardson, 2010). Specifically, we assume that the motor system is made up of isolable subsystems, each with different capacities. Decomposition is based on the assumption that mechanisms of behavior are made up of component parts and component operations. Localization implies a spatial location for a component part but does not necessarily imply a single contiguous location. There have been two kinds of criticism of the decomposition and localization approach. One has been to say that many properties of a system arise from the hierarchical organization of its components and their nonlinear interactions. The other has been to posit distributed networks in which the connectivity architecture generates the behavior but that this holistic architecture cannot be broken down into separate modules performing recognizable subtasks. This distributed network view is especially prominent when it comes to the study of higher cortical function in cognitive neuroscience (Uttal, 2003). These potential criticisms are mitigated in our view in several ways. First, many of the component parts of motor learning are localized in noncortical areas; the spinal cord, brainstem, basal ganglia, and the cerebellum. These lower-level areas are likely more modular than higher-order cortical areas. Second, these structures are highly conserved phylogenetically, which suggests conserved mechanisms. Third, when it comes to cortex, we will focus exclusively on primary motor cortex (M1), which shows more evolutionary variation than subcortical structures but less than heteromodal cortex. Fourth, we assume that these component parts combine to generate the behavior in question. In the case of the areas discussed in this review; they can still be considered components of a network but in which intrinsic computational operations with message passing between components is emphasized over weight changes in layers of a holistic network. Fifth, we accept the likely possibility that new operations, which no individual component possesses, may arise through interaction between components. Decomposition is just a starting point or null hypothesis, which in our view is more useful than vague statements about the “loop” or the “whole circuit” doing the work with no suggestion as to how this would be proven experimentally or modeled computationally. Finally, here the focus is on learning rather than implementation of that learning via another structure. For example, if the cerebellum is required for learning but the resultant improved performance is expressed via commands from motor cortex, this does not in our view mean that learning is taking place in a “cortico-cerebellar loop.” The three anatomical components that we will discuss in this review are the cerebellum, basal ganglia, and motor cortex (Figure 1).

Figure 1. Cerebellum, Basal Ganglia, and Motor Cortex Analogs in Mouse, Bird, and Primate.

Cerebellum (CB), basal ganglia (BG), and primary motor cortex (M1)in three animal models. In the bird, the robust nucleus of the arcopallium (RA) is considered to be the analog to motor cortex in mammals. LMAN connects BG to RA.

The review, not surprisingly, raises more questions than answers, and if anything should be considered a form of manifesto. The overall purpose is to call attention to the benefits of a comparative approach. First, we hope to show that explicit comparison of motor learning results across the various model systems currently under investigation can help support or refute viewpoints on the role of specific structures. Second, to inspire experimental directions in any given model system that might otherwise not be considered. Finally, given that neurorehabilitation is predicated on motor learning (Krakauer, 2006), taking a closer look at how motor learning itself is accomplished after brain injury and disease in model systems may improve the way that we train patients to gain back their lost motor abilities.

Motor learning is a blanket term for any practice-related change or improvement in motor performance for a defined variable of interest. In this review, we will draw a broad distinction between two learning mechanisms—motor adaptation and skill learning. By motor adaptation we mean the fast changes that return behavior to baseline levels of performance in the setting of perturbations that induce systematic errors, for example, prism adaptation. By skill learning we mean the slower changes that lead to performance improvements that are better than baseline. Such behaviors include learning to ride a bicycle or to play the violin. In addition to these two kinds of motor learning, there is an intermediate category of learning that is more difficult to categorize but can be broadly captured by the idea of action selection. The whole field of reinforcement learning is predicated on the idea that particular actions come to be associated with successful goal completion. For example, completing a maze or learning to press a lever for food at particular intervals. The question is—is this motor skill learning? We would say no because the quality of the motor performance itself is not the metric of interest, instead the motor system is just used to read out whether operant learning has occurred. We will have more to say about this in the course of the review. For now we will restrict ourselves to the comment that it is of interest that many studies of skill have focused on sequence learning, in which the order in which actions must be performed is almost always emphasized over the quality of the execution of the actions themselves.

There are clear preferences with regard to the kind of motor learning studied depending on the effector and model system used. For example, in the case of eye movements, the focus is mostly on adaptation (Schubert and Zee, 2010), indeed it is hard to imagine what a skilled eye movement would be. In rodent models, in contrast, the focus has been either on skilled prehension movements (Whishaw and Pellis, 1990), for which training increases the probability of success, or action selection with fairly coarse movements (Jog et al., 1999). In bird song, the focus has also been on selecting a sequence of vocalizations through comparison with a template (Brainard and Doupe, 2002). In humans, the focus switches back to adaptation—forcefields, visuomotor rotations, and split-treadmills (Bedford, 1989; Cunningham, 1989; Reisman et al., 2005). It is only when one looks across the model systems being studied that one clearly sees these task preferences and can ask what motivates them.

Cerebellum

We begin by discussing the role of the cerebellum in motor learning because in this case we seem to be closest to a unifying hypothesis, precisely because of the consistency of the experimental results across model systems. All vertebrate brains have a cerebellum, some also have additional cerebellar-like structures, with a highly conserved architecture (Bell, 2002; Bell et al., 2008). This conserved architecture is thought to result from historical or phylogenetic homology in the case of the cerebellum, i.e., inherited from a common ancestor and suggests a sustained evolutionary requirement for a specific kind of computation. A large amount of research across many species suggests that the cerebellum can compute estimates of sensory consequences of commands. This cerebellar computation allows for predictive control (simple spike firing tends to lead limb kinematics [Ebner et al., 2011]), improved sensory estimates (Vaziri et al., 2006), and fast feedback corrections at latencies shorter than would be possible with peripheral feedback alone (Xu-Wilson et al., 2011). This predictive capacity of the cerebellum is captured by the idea of a forward model (Wolpert and Miall, 1996). A forward model, however, is only useful for control if it produces unbiased state estimates, which means that it needs to learn in the face of systematic prediction errors. Most of the experiments in humans and model systems that investigate how systematic errors are reduced can be interpreted within the framework of updating of forward models (Shadmehr et al., 2010). Specifically, several recent studies in humans suggest that errors induced by external perturbations are interpreted as sensory prediction errors rather than target errors (Mazzoni and Krakauer, 2006; Wong and Shelhamer, 2011), and these are reduced through a cerebellar-dependent adaptation mechanism (Taylor et al., 2010; Tseng et al., 2007). Learning for all these forms of adaptation is fast, occurs within minutes or hours, is well captured by single or double exponentials, shows prominent aftereffects, and is easily washed out. Very similar learning behavior is seen across multiple model systems and appears to also be cerebellar dependent. In monkeys, lesions of cerebellar cortex severely disrupt adaptation of both Vestibuloocular reflex and saccadic eye movements (Barashet al., 1999; Lisberger et al., 1984). In cats, lesions of the flocculus abolish Vestibuloocular reflex adaptation (Luebke and Robinson, 1994). The rate of rotation adaptation in humans is increased by anodal transcranial direct current stimulation over the ipsilateral cerebellum but not over primary motor cortex (Galea et al., 2011).

Visuomotor adaptation is not disrupted by lesions in the corticospinal tract caused by ischemic stroke in humans (Reisman et al., 2007; Scheidt and Stoeckmann, 2007; Scheidt et al., 2000) and is largely unaffected in Parkinson's disease (PD) (Bédard and Sanes, 2011; Marinelli et al., 2009) and Huntington's disease (Smith and Shadmehr, 2005). Thus, motor cortex, the corticospinal tract, and the basal ganglia do not seem to be necessary structures for visuomotor adaptation. Subtleties and controversies arise, however, because abnormalities in adaptation paradigms have been seen in patients who do not have known cerebellar impairment and patients with cerebellar disease can reduce errors under certain experimental conditions. We shall discuss these in turn and provide potential explanations that show why these exceptions do not disprove the cerebellar hypothesis for adaptation. In two recent studies, patients with PD were able to adapt to a rotation as well as age-matched controls but did not show savings in re-exposure (Bédard and Sanes, 2011; Marinelli et al., 2009). We have recently argued that savings in adaptation paradigms is not due to forward model-based error reduction but is instead attributable to an addition operant process (Huang et al., 2011). Using this new framework, we can explain the result in PD because it is known that operant learning is disrupted in these patients (Knowlton et al., 1996). Patients with stroke in the left superior parietal lobule showed markedly impaired ability to adapt to a visuomotor rotation (Mutha et al., 2011), which would appear to contradict the idea that the cerebellum is the (sole) locus for adaptation. We have recently argued, however, that the parietal cortex receives the output of a cerebellar forward model, which is then integrated with peripheral sensory feedback (Tanaka et al., 2009). Thus, the parietal cortex may be the downstream target of the cerebellum and thus disruption of this target can impair adaptation.

A recent study reported that patients with spinocerebellar ataxia type 6 were able to adapt to an incremental introduced forcefield but not if the forcefield was introduced as a large step (Criscimagna-Hemminger et al., 2010). There are two ways to interpret these data. One is that adaptation to small errors is carried out in a noncerebellar structure. Alternatively, these patients brought down error using a non-adaptation-based mechanism. There is direct and indirect support for the second interpretation. Inamore recent study by the same group, patients who brought down an incremental visuomotor rotation did not show a change in their perceived hand position (an assay for a change in the forward model) nor did they show the pattern of direction generalization that has been described for adaptation to step rotations (personal communication with authors); both results suggest that error reduction was accomplished by aqualitatively distinct learning mechanism. A clue to what this alternative learning mechanism might be was provided by a recent study in which healthy subjects were exposed to an incremental rotation but were provided only with binary reward rather than vector error (Izawa and Shadmehr, 2011). Under these circumstances, subjects showed exploratory trial-and-error behavior rather than typical monotonic adaptation behavior and also did not show a change in perceived hand position. These two sets of results in humans are consistent with the idea that errors can be reduced through cerebellar-independent non-forward model-based processes as long as the errors lie within the envelope of exploratory variability. A study of saccadic gain adaptation in monkeys also showed a small amount of residual adaptation to a gain change after lesions of the oculomotor posterior vermis (Barash et al., 1999). The authors of this study could only speculate as to the locus for this residual capacity to reduce errors, suggesting it might be mediated by the cerebellar nuclei. We would suggest that this result in monkeys is reminiscent of the human reaching studies reported above and that the mechanism might be outside the cerebellum. Support for this idea comes from studies in monkeys, in which intermediate and lateral deep cerebellar nuclei ablations were performed and yet slow recovery of limb ataxia was still seen, which was reversed with lesions to sensory cortex (Mackel, 1987).

Basal Ganglia

Compared to the cerebellum, the precise role of the basal ganglia in motor learning remains unclear and contradictory. Like the cerebellum, both the anatomy and neurotransmitter localization for the basal ganglia (BG) are highly conserved in all vertebrates, again suggesting a preserved form of computation (Reiner et al., 1998). Of particular interest, is the fact that basal ganglia output evolved from principally targeting the tectum in amphibians to also targeting cortical regions in reptiles and in subsequent vertebrates (Reiner et al., 1998). In addition, there is no evidence for either cortical or significant dopaminergic inputs to striatum in amphibians. Amphibians have simpler musculoskeletal systems and execute a simpler repertoire of movements than reptiles; their movements are tectally mediated, stereotypical, and stimulus locked (Reiner et al., 1998). This phylogenetic transition between amphibians and reptiles with respect to the connections of the BG is interesting for a number of reasons. First, it suggests that the BG perform a function that does not have an obligate relationship to cortex. Second, it suggests a parallel between eyes movements in primates and stereotypical movements such as locomotion; both are largely controlled by the brainstem and spinal cord. Third, although a new kind of learning could arise from the new connections between the BG and cortex, the investigation of BG involvement in motor learning should focus first on whether there is a mechanism common to movements under the control of motor cortex, brainstem, or the spinal cord. As stated above, in the section on the cerebellum, adaptation does not seem to be affected by diseases of the BG (Bédard and Sanes, 2011; Marinelli et al., 2009). Surprisingly, while researching this review, we could not find examples of experiments in animal models that investigated the effect of striatal lesions on visuomotor adaptation.

Review of the literature across species suggests instead that the BG are critical for early learning of sequential actions. The challenge is to determine the specific aspect of sequence learning that they contribute to. Confusion arises because, as we have already mentioned above, many studies of the role of the basal ganglia in learning have used motor behavior as a readout of learning of higher-order aspects of the behavior rather than focusing on improvements in the quality of the motor behavior itself. For example, a well-known paradigm in monkeys has them acquire a series of specific sequences of reaches through trial and error learning, but the reaching movements themselves are easy and have no speed-accuracy constraint (Hikosaka et al., 1995). Thus, the movements themselves read out the sequence order. Using such a task, striatal inactivation (using muscimol) has shown to impair the ability to acquire short sequences of button presses in the monkey (Miyachi et al., 1997). In rodents, striatal lesions impair the ability to learn a sequence of nose pokes in a serial reaction time task (Eckart et al., 2010), and learning in a T-maze task (Moussa et al., 2011). Here again, the quality of movements themselves is not emphasized.

It is in the bird song model that the closest look can be taken at the distinction we argue for between knowing a sequence and the quality of its execution. The BG circuit had been shown to be necessary for song formation (Bottjer et al., 1984; Scharff and Nottebohm, 1991). In recent years, LMAN, the cortical target of the BG, has been shown to be the link between the BG and the motor output pathway, and to be crucial for song development in juveniles and for song modification in adults (Kao et al., 2005; Olveczky et al., 2005). Interestingly, one of the functions of this area is to inject variability into song production. This variability presumably allows juvenile birds to acquire a tutor's song through exploration (Olveczky et al., 2005). In the adult bird, the contribution of LMAN to song production is decreased but still apparent when the song is modulated following disruptive auditory feedback (Andalman and Fee, 2009). Variability in the birdsong model is due to exploration, which is different from variability due to planning and execution noise, reductions in which are not the focus of these studies (Tumer and Brainard, 2007). This distinction between exploration of task space and reduction of variability at a chosen location in task space has been nicely demonstrated in a series of studies using a virtual ball and skittle task in humans (Cohen and Sternad, 2009; Müller and Sternad, 2004).

A paradigm recently introduced in adult songbirds induces short-term learning following song disruption (Andalman and Fee, 2009; Tumer and Brainard, 2007; Warren et al., 2011). Specifically, it has been shown that playing white noise to the bird if the frequency of a specific syllable is within a prespecified range lead to song adjustments to avoid the white noise disruption. After learning in this paradigm, LMAN inactivation has shown to partially reverse the song adjustment (Andalman and Fee, 2009; Warren et al., 2011). We would argue that this behavior in birds is similar to error reduction in cerebellar patients (Criscimagna-Hemminger et al., 2010) and when binary reward is provided to healthy human subjects (Izawa and Shadmehr, 2011). In both cases, subjects use reward to select one movement over another but, critically, the newly selected movement is not executed any better than the original one. Similarly, in the songbird, syllable variability at the new frequency is the same, if not increased, compared to the initial frequency (Andalman and Fee, 2009; Warren et al., 2011)—thus syllable production per se at the new frequency has not improved. It is of course possible that improvement in song execution, motor skill, may occur during song acquisition but this has not been shown yet. We predict that this aspect of motor learning will be a property of the song execution circuit rather than the BG circuit and could be investigated by tracking trial-to-trial variability during song practice after LMAN inactivation. Pallidotomy in humans, as a treatment for PD, is consistently associated with an impaired ability to learn new motor sequences (Brown et al., 2003; Obeso et al., 2009). Thus, the unifying principle is that learning of sequential actions proceeds through trial and error, which is aided by the injection of variability by dopaminergic projections to BG, variability then decreases as the chosen successful action automatizes to a stereotypy (Costa, 2011).

Our position so far is that the exploration-to-stereotypy view of sequential learning leaves out improvement in the quality of movement execution itself and that the birdsong literature has not yet shown evidence for the latter. In rodents, however, there is possibly some evidence that BG circuits play a role in task improvement through changes in the quality of movement execution. In the rotarod task, mice improve their ability to run for longer periods of time on an accelerating training wheel and this is associated with potentiation of synaptic strength in the striatum (Costa et al., 2004; Yin et al., 2009). Protein synthesis inhibition in the striatum has been shown to impair early stages of learning of the precision reaching task in rats (Wächter et al., 2010). How to interpret these results? One possibility is that action selection makes a significant contribution to the rotarod and prehension tasks (detailed movement analysis was not performed in these studies). Another possibility is that quality of movement execution is indeed improving in these tasks and that the BG, through their connections to cortex, have evolved to play a role in true skill learning. In support of the latter idea, sequence tasks and initial improvement in the rotarod task have shown to depend on striatal areas that project to the prefrontal cortex (Miyachi et al., 1997; Yin et al., 2005, 2009) whereas improvement across days has shown to be dependent on striatal areas that project to the sensorimotor cortex (Yin et al., 2004, 2009). Thus despite what appears to be a qualitative different kind of motor learning: selection of a sequence of actions versus better execution of the sequence elements, it is possible that both these behaviors depend on similar BG computations but with different cortical targets. While BG reinforces better action selection through its projections to the prefrontal cortex at early stages of learning, BG connections to the motor cortex could enhance selection of better muscle combinations during later stages of training.

Primary Motor Cortex

Sensory and motor neocortex are markedly more developed in mammals compared to amphibians, reptiles, and birds (Butler and Hodos, 2005). In our taxonomy of learning, we have discussed the necessity of the cerebellum for motor adaptation and the basal ganglia for early trial-and-error learning of action sequences. So what about motor cortex? One important clue for answering this question is to realize that, unlike the striatum and the cerebellum, M1 is a controller; it sends commands directly or indirectly (via interneurons) to motorneurons. Many purposeful behaviors can unfold in the absence of descending commands from motor cortex, for example over ground locomotion in rodents (Metz et al., 1998) and treadmill walking in cats (Hiebert et al., 1996). In the case of eye movements, there is no direct equivalent of M1; the frontal eye fields (FEF) do not directly control oculomotor neurons in the brainstem for saccade generation (Hanes and Wurtz, 2001). An interpretation of a lot of data, some of which we describe below, is that motor cortex offers an extra level of limb control that is not provided by the brainstem and spinal cord: flexible combinations of movements that isolate individual joints and allow performance of novel tasks and interaction with novel objects. Such flexibility requires learning throughout life as hardwired stereotyped synergies cannot anticipate ever-changing environmental challenges. In order to control a single joint out of synergy requires knowledge of limb dynamics to compensate for interaction torques across joints. Recent work in primates and humans suggests that M1 has this capacity (Gritsenko et al., 2011; Pruszynski et al., 2011).

Lesions of the corticospinal tract (CST) cause impairments in the execution of over-learned dexterous movements, both of prehension in rodents, cats, and primates (Lawrence and Kuypers, 1968; Martin and Ghez, 1993; Ropper et al., 1979; Whishaw, 2000), and in the ability to make visually guided predictive modifications to the locomotor pattern in cats (Drew et al., 1996). These impairments are in stark contrast to lesions of striatal output, which have surprisingly little effect on execution of well-learned movements when such lesions have been produced in songbirds, monkeys and humans (Desmurget and Turner, 2010; Obeso et al., 2009; Stepanek and Doupe, 2010; York et al., 2007). After lesions of M1 or the CST, rodents (Whishaw et al., 2008), primates (Hoffman and Strick, 1995), and humans compensate with lower-level synergies (Twitchell, 1951). It is interesting to ask whether the ability to find a useful compensatory strategy is itself motor cortex dependent. In anurans (frogs and toads), movements are initiated from the midbrain not the forebrain (Abbie and Adey, 1950). It is notable that despite no significant cortical role in the planning or control of movement, anurans are capable of learning new preycatching behavior after hypoglossal nerve transection through concatenating pre-established synergies—mouth opening, neck extension, and body lunge (Corbacho et al., 2005). It could be conjectured that this process can be accomplished by BG connections with the brainstem.

One of the main contentions of this review is that it is necessary to distinguish between learning “what” from learning “how.” Within this framework, we reserve the term skill for the ability to improve the quality of execution rather than selecting correct actions. For example, faster and more accurate hitting of a particular sequence of piano keys is skill, whereas knowing which sequence of keys you are meant to hit and doing so slowly is not. A large amount of evidence suggests that these improvements in skill are accompanied by plasticity in M1, i.e., skill learning-related changes occur in the same place from which baseline dexterous control originates. In humans, the duration of impairment in dexterous finger movements is correlated with lesion volume (Darling et al., 2009). Improvement in the speed and accuracy of sequential finger movements correlates with increased BOLD activation in M1 (Karni et al., 1995; Stagg et al., 2011), is enhanced by transcranial direct current stimulation over M1 (Classen et al., 1998; Reis et al., 2009; Stagg et al., 2011) and inhibited by repetitive transcranial magnetic stimulation over M1 (Muellbacher et al., 2002). In a recent study it was shown that TMS was much more likely to elicit piano playing-like movements in skilled pianists than in controls, which suggests that M1 can encode representations of novel abilities acquired through practice (Gentner et al., 2010).

Further evidence for the claim that learning of motor skill results from changes in representation in motor cortex comes from experiments in rats. In a specially designed reach to grasp task, performance improvements are accompanied by various structural changes in M1 (Whishaw and Pellis, 1990). It has also been shown that the signal-to-noise ratio in spiking M1 neurons improves with practice on a reach-to-grasp task (Kargo and Nitz, 2004). Recently it has been shown that destroying dopaminergic projections to motor cortex completely abolishes skill acquisition (Hosp et al., 2011), which suggests that a specific kind of learning (skill) needs to take place in M1 directly. Large lesions to motor cortex lead to permanent qualitative changes in skilled reaching, with recovery mediated through compensation (Metz et al., 2005; Whishaw et al., 2008). In contrast, small strokes in motor cortex lead to significant recovery of premorbid prehension kinematics (Gonzalez and Kolb, 2003). This recovery seems to be mediated by plasticity in peri-infarct cortex, with structural changes very similar to those described after reach training in healthy rats. Similar findings have been made in the squirrel monkey (Nudo et al., 1996). Thus M1 is necessary for recovery of previously acquired skills after small cortical lesions and acquisition of new skills, likely using very similar plasticity mechanisms. All these results taken together suggest that if skill is considered the ability to execute better movements of a given type rather than selecting the right sequence of movements without emphasis on their quality, then the motor cortex is necessary if not sufficient. It is notable that simply repeating a movement stereotypically that does not require a skill change does not lead to map changes in motor cortex (Plautz et al., 2000). Finally, it should be emphasized that our contention that M1 is the necessary structure for learning skilled execution does not preclude M1 also being the location for the representation of stereotypies that are learned initially through BG-dependent processes. This “transfer” idea is favored by some investigators and supported by the decreasing LMAN dependence of learned songs in the songbird (Ölveczky et al., 2011).

Conclusions

Here, wehave briefly described experiments across humans and model systems in order to seek unifying functional principles with respect to the roles of the cerebellum, basal ganglia, and primary motor cortex in motor learning. Recently, a similar but more general computational synthesis of these areas has been proposed (Doya, 1999). From an evolutionary perspective, it appears that the structures of the basal ganglia and cerebellum have been highly conserved and predate the development of sensorimotor cortex in mammals, which suggests that the computational role of these subcortical structures may not have changed but their connections evolved to also target cortex and not just the brainstem.

The cerebellum is critical for adaptation, which can be defined as learning of a forward model to reduce sensory prediction errors (Shadmehr et al., 2010). The difference between the role of the cerebellum for limb movements, where it has no access to motor neurons, versus in the case of eye movements, where the cerebellum could also potentially act as a controller, needs further investigation (Medina, 2011). The role of the BG remains contentious but almost all the studies we reviewed tested some kind of sequence task and can be subsumed under the idea of action selection and instrumental conditioning. A current idea is that the BG injects variability for exploration and then as the best movement is converged upon, variability is reduced and stereotypy and automatization ensue (Costa, 2011). Quality of movement execution, i.e., motor skill, is not explained by this framework and has not been the focus of these studies, although in a few studies, striatal lesions have been shown to impair tasks that can be considered tests of motor skill (Costa et al., 2004; Wächter et al., 2010). Motor skill, faster and more precise movements compared to baseline, has been surprisingly understudied, compared to either adaptation or selection of sequential actions, but to the degree that it has been studied, M1 appears to be a necessary structure. The implication of this framework is that skill may be a late development evolutionarily. Adaptation and learning to select the right actions from a hard-wired repertoire of synergies might suffice both for the vast majority of animals and for eye movements in primates.

Where to go from here? One fruitful direction would be for investigators using particular model systems with particular behavioral tasks to take a look across at their colleagues, predicated on the assumption that anatomical homology allows for experimental and conceptual borrowings. Of particular interest is to ask how error-based and reward-based processes combine during motor learning, especially as anatomical connections between the cerebellum and the basal ganglia have recently been described (Hoshi et al., 2005). We finish with a few suggestions for future directions: (1) rodent models could potentially be developed that combine the finer-grained kinematic analysis of the rat reach-and-grasp task (Whishaw et al., 2008) with a sequential action selection requirement. (2) Human and primate studies of sequence learning could pay closer attention to movement quality as well as sequence order, i.e., start to study motor skill with quantitative kinematic analysis. Suggestions (1) and (2) could help characterize the precise nature of the interaction between the BG and M1 during skill learning. (3) Rodent models of limb adaptation could be developed. (4) Rodent models of stroke could test the hypothesis that recovery is motor cortex dependent but that compensation requires exploration of spared movements and so might be BG dependent. In many of these proposed studies, a double lesion approach could be very informative.