Neurocognitive Mechanisms of Error-Based Motor Learning

Introduction

During the initial stages of skill learning, motor performance is cognitively demanding and uncoordinated. Understanding how an individual progresses to the exquisite and automatic processes of proficient performance has not been a trivial process. Here, we review the role that error detection and correction plays in skill learning. We make a distinction between within trial error corrections, or performance adjustments made during the course of a movement that is not achieving its intended goal, and across trial error corrections, or cumulative adjustments that reflect ongoing learning. We review recent debates regarding whether skill learning is a purely implicit or subconscious process, or if it can benefit from explicit instruction.

Learning from errors is one of the basic principles of motor skill acquisition. Current ideas about error-based learning stem from forward model control theories (Diedrichsen et al. 2010; Miall and Wolpert 1996). When movement errors are detected by sensory systems, the information is used to update the motor commands for subsequent actions. However, relying solely on sensory feedback does not allow efficient motor adjustments because of the time delay between the initial motor command and the arrival of sensory feedback. Movement induces continuous changes to state variables such as limb position and velocity. In order to allow accurate movement adjustments, the motor system relies on a forward model that makes predictions of the sensory outcomes (i.e., changes in position and velocity) associated with a given motor command (Bastian 2006; Flanagan et al. 2003). Differences between the predicted and actual sensory outcome serve as the feedback error signal that updates forthcoming motor commands.

When learning a new motor skill such as swinging a golf club, new skills do not have enough of a motor history for an accurate forward model, resulting in large prediction errors. In this case, the process of learning involves updating motor commands through multiple exposures to motor errors and gradually reducing them by refining the forward model (Donchin et al. 2003; Shadmehr et al. 2010).

The mechanisms of error-based learning are often studied using visuomotor adaptation and force field adaptation tasks. Visuomotor adaptation involves distortion of the visual consequences of movement, whereas force field adaptation affects the proprioceptive consequences of motor commands by altering the dynamics of movement (Shadmehr and Mussa-Ivaldi 1994). Error processing under these two paradigms shows extensive neural overlap in the cerebellum, suggesting a common mechanism for error processing and learning (Diedrichsen et al. 2005).

While extensive evidence supports a role for the cerebellum in error-based sensorimotor learning (Criscimanga-Hemminger et al. 2010; Diedrichsen et al. 2005; Ito 2002; Miall and Wolpert 1996; Miall et al. 1993, 2007; Ramnani 2006; Tseng et al. 2007), many neuroimaging studies also provide evidence that brain regions other than the cerebellum may play a role, including the parietal cortex, striatum and anterior cingulate cortex (Clower et al. 1996; Danckert et al. 2008; den Ouden et al.2010). In this chapter, we review the literature supporting the involvement of three distinct brain structures in error processing: (1) the anterior cingulate cortex (ACC), (2) the basal ganglia, and (3) the cerebellum. We also speculate about the specific roles that each structure may play during motor learning, depending on the learning task and context. In an effort to add some structure to the burgeoning literature on this topic, we have organized our review into discussions of both feedforward and feedback learning processes (cf. Wolpert et al. 1998).

Anterior Cingulate/Medial Frontal Cortex Contributions to Error Processing

One of the brain systems that plays a critical role in error processing is the medial prefrontal/ACC region (see Fig. 3.1; Seidler et al. in press). This prefrontal performance monitoring system has been studied extensively by recording the error-related negativity (ERN), an event-related potential (ERP) component that is time locked to an erroneous response (Falkenstein et al. 1995; Gehring et al. 1993, 1995). The ERN is thought to be generated in the anterior cingulate cortex (ACC), which is known to serve cognitive control functions that enable the brain to adapt behavior to changing task demands and environmental circumstances (Botvinick et al. 2001; Ridderinkhof et al. 2004). The cognitive control and performance monitoring functions of this brain region have been predominantly studied using cognitive tasks such as the Stroop word-color interference task and Eriksen’s flanker task (Nachev et al.2008), depicted in Fig. 3.1. These tasks require individuals to minimize interference from irrelevant, conflicting cues, and to monitor incorrect trials to adjust and improve performance. In the following section, we review recent literature demonstrating how the ACC/medial prefrontal system contributes to feedback and feedforward motor learning.

Fig. 3.1.

The left panel illustrates a participant performing the Flanker task, classically employed to study the event related negativity (ERN) associated with error commission. The task goal is to respond to the direction of the center arrow while ignoring the conflicting information represented in the surrounding areas (art by Lauren Wu). The right panel indicates the cingulate gyrus; it is thought that the ERN originates in the anterior cingulate gyrus

Feedback Error Processing in the ACC

The ACC error monitoring system has traditionally been viewed as contributing to the feedback processing of errors. The ERN is a response-locked ERP component that appears after commission of an error, which is monitored by the medial prefrontal/ACC system and contributes to performance improvement (Gehring et al.2011). This is also represented by the reinforcement learning theory that explains the origin of the ERN and the error monitoring role of the medial prefrontal/ACC region (Holroyd and Coles 2002). This theory suggests that the medial prefrontal/ACC receives reward prediction error signals from midbrain dopaminergic cells, which also send the same input to the basal ganglia (described in more detail in the subsequent section). When the error signal is delivered to theACC, this system contributes to performance improvement by exerting greater cognitive control (Gehring et al. 2011). Another theory that explains the mechanism of feedback processing of ERN is the error detection/comparator theory (Gehring et al. 2011). According to this theory, the ERN is a signal representing the mismatch between the actual output of the motor system and the best estimate of the correct response (Falkenstein et al. 1991; Gehring et al. 1993). This mismatch signal is conveyed to the control center for future motor command adjustment as part of the feedback process.

While traditionally ERN feedback processing has been studied with cognitive tasks, recent work also demonstrates a contribution of the medial prefrontal/ACC error processing system to motor control and motor learning (Anguera et al. 2010; Anguera et al. 2009; Danckert et al. 2008; Ferdinand et al. 2008).

We recently tested whether the ERN was sensitive to the magnitude of error experienced during a visuomotor adaptation task and found a larger ERN on trials in which larger motor errors were made (Anguera et al. 2009). ERN magnitude also decreased from the early to the late stages of learning. These results are in agreement with current theories of the ERN and skill acquisition. For example, as the error detection theory proposes (Falkenstein et al. 1991; Gehring et al. 1993), a greater ERN associated with larger errors indicates that the brain was monitoring the disparity between the predicted and actual movement outcomes (Anguera et al.2009).

There is also evidence supporting the notion that error processing in the ACC contributes to motor sequence learning (Berns et al. 1997). The N200 ERP component, which is also localized in ACC, is known to be sensitive to response conflict and cognitive control (Folstein and Van Petten 2008). The N200 component has also been widely studied in the domain of feedback or error monitoring together with the ERN (Folestein and Van Petten 2008; Gehring et al. 2011). Several studies have shown that the N200 is enhanced for a stimulus that violates a learned motor sequence (Eimer et al. 1996). Similarly, when ERN magnitudes were compared between explicit and implicit sequence learners, a larger ERN was found for the explicit learners demonstrating greater involvement of the error monitoring system when individuals are actively searching for the regularity of a sequence (Russeler et al. 2003). A more recent study demonstrated a parametric increase in the magnitude of the ERN during sequence learning as the awareness of the sequential nature and the predictability of the forthcoming sequential element increased (Ferdinand et al. 2008).

Interestingly, a number of EEG source localization studies have suggested that motor regions such as the cingulate motor area and the presupplementary motor area (pre-SMA) are the generator sites of the ERN as opposed to the ACC (Dhar and Pourtois 2011; Hochman et al. 2009; Badgaiyan and Posner 1998; Dehaene et al. 1994; Miltner et al. 1997). These motor regions not only take part in higher order motor executive control and self-initiated movements, but also contribute to sequential movements and adaptive motor learning (Chao et al. 2009; Duann et al.2009; Stuphorn et al. 2010; Chen et al. 2010; Nachev et al. 2005; Cunnington et al.2002; Shima et al. 1996; Chen and Wise 1996; Hikosaka et al. 1996). Clearly further investigation is required to parse out the functions of these individual medial brain regions to error-based learning.

Feedforward Error Processing in the ACC

While the role of the ACC in error processing has historically been more focused on feedback processes, recent work suggests that medial prefrontal regions including the ACC and medial motor areas also serve feedforward functions during motor tasks. One example is shown during motor response inhibition, as measured by the stop signal task. During this task, participants are asked to cancel their prepotent motor response when they see a stop signal that occurs occasionally, is unpredictable, and occurs at various latencies after the appearance of the target stimulus (Logan 1994). Stop signal response time is an estimation of the time an individual needs to inhibit a prepared motor response (Logan 1994). Studies have shown that the pre-SMA together with the inferior frontal gyrus is involved in stop signal task performance (Chao et al. 2009; Duann et al. 2009). In particular, successful motor response inhibition is represented as shorter stop signal response times, and activates the pre-SMA (Chao et al. 2009). The role that this motor region plays in higher order motor control is interpreted as a proactive control system that allows intertrial adjustments to the level of motor readiness based on prior performance and anticipated task requirements (Stuphorn et al. 2010; Chen et al. 2010). This suggests that the medial motor regions also serve a feedforward error processing role during complex motor behaviors. Whether the error processing mechanisms of the cingulate motor area and the pre-SMA are distinct from those of the medial prefrontal/ACC regions during cognitive control tasks is debatable. Considering the relative anatomical closeness of these regions and the evidence suggesting that the cingulate motor area and the pre-SMA may also be generator sites of the ERN signal (Nachev et al. 2008; Dhar and Pourtois 2011; Hochman et al. 2009; Badgaiyan and Posner 1998; Dehaene et al.1994; Miltner et al. 1997), one might argue that these regions serve similar functions. However, it has been proposed that the cingulate motor area and the pre-SMA correct for movement errors in a proactive manner (Isoda and Hikosaka 2007).

In a series of studies, Krigolson et al. have demonstrated a contribution of the ERN to feed forward error processing in a joystick movement motor task (Krigolson and Holroyd 2006, 2007a, 2007b; Krigolson et al. 2008). They found that the ERN began just prior to target tracking errors, indicating that the medial frontal system began to detect the error even before it was fully committed (Krigolson and Holroyd 2006). The authors suggested that this might entail the medial frontal system predicting tracking errors by adopting a predictive mode of control (Desmurget et al. 2000). That is, these results indicate a feedforward role of the medial frontal ERN to motor error processing that is distinct from error feedback processes (Krigolson and Holroyd 2007a; Krigolson et al. 2008).

In conclusion, studies have provided support that the medial prefrontal/ACC system, together with the cingulate motor areas and the pre-SMA play a role in motor error processing and skill acquisition. However, the actual mechanisms whereby these two systems contribute to performance improvements across trials are not well understood. It is also unclear whether this system works independently of or in collaboration with the cerebellum and basal ganglia error processing systems, although it is likely that this system works in concert with the basal ganglia networks as both regions receive similar midbrain dopaminergic inputs as described above.

Basal Ganglia Contributions to Error Processing

Errors play an important role in goal-directed behavior. When the consequences of our behavior are better than expected, associations and response patterns are strengthened (Hebb 1949). Conversely, when outcomes are worse than expected (e.g., when performance errors occur), adjustment often occurs resulting in improved performance (Rabbitt 1966).

Empirical and computational work implicates the midbrain dopamine (DA) system and its most prominent target, the basal ganglia (see Fig. 3.2), as key sub-cortical structures involved in error processing (Schultz 1998, 2001). Dopaminergic projections are most highly concentrated in the dorsal striatum and the ventral bank of the ACC (Goldman-Rakic et al. 1989; Smith and Bolam 1990). Dopamine neurons also receive inputs, primarily from the ventral striatum and caudal orbitofrontal cortex (Eblen and Graybiel 1995). These projections exert control over dopamine output, which can modulate stimulus-response associations (Horvitz 2002). For example, performance errors initiate neural training signals that alter our response tendencies (Holroyd and Coles 2002; Ljungberg et al. 1991). Errors briefly deactivate midbrain dopamine neurons (for approximately 100 ms following error commission), which carry predictive error signals to various parts of the brain for reinforcement learning (Guigon et al. 1995). Changes in basal ganglia activity modulate the thalamus, which then relays information to the cortex.

Fig. 3.2.

The cartoons depict rewarding stimuli (left and middle panels, art by Lauren Wu), while the right panel highlights the basal ganglia, known to be involved in processing error and reward information in reinforcement learning paradigms

There are several functional MRI (fMRI) studies which suggest a role for the human basal ganglia (BG) in regulating many kinematic properties. For example, activity in the putamen and the internal globus pallidus (GPi) is associated with movement velocity and amplitude (Turner et al. 1998, 2003; Spraker et al. 2007), putamen activity also scales with force duration (Vaillancourt et al. 2004; Prodoehl et al. 2008), and GPi and subthalamic nucleus activation is related to force rate of change and amplitude (Spraker et al. 2007; Prodoehl et al. 2008). There is also a large body of fMRI studies showing error-related increases in blood-oxygen level dependent (BOLD) activity in both ventral and dorsal striatum during learning tasks patterned after those used to elicit dopaminergic responses in animals (Delgado et al.2000; McClure et al. 2003; Schönberg et al. 2007; D’Ardenne et al. 2008). Furthermore, imaging studies have shown that activity in structures that modulate dopamine output (caudal orbital frontal cortex and ventral striatum) reflects the magnitude of errors on reward-based learning tasks (Knutson and Cooper 2005).

Motor learning is often broadly classified into two categories: sensorimotor adaptation and sequence learning. Although patients with Parkinson’s disease (PD) and Huntington’s disease (HD) exhibit mild to moderate impairments in kinematic sensorimotor adaptation tasks (cf. Canavan et al. 1990; Laforce and Doyon 2002; Boulet et al. 2005), this finding has generally not been corroborated by the neuroimaging literature. For example, many investigations of kinematic adaptation tasks, such as when subjects make pointing movements while wearing prism lenses or receiving distorted visual feedback of their actions on a computer display, have not reported BG activation (Clower et al. 1996; Inoue et al. 1997; Imamizu et al. 2000). However, several of these studies were not designed to assess early vs. late stages of learning. More recent neuroimaging studies demonstrate activation in the globus pallidus, putamen, and caudate during the early learning phase of sensorimotor adaptation tasks (gain change: Krakauer et al. 2004; feedback rotation: Seidler et al. 2006). In motor sequence learning, increased putamen activity was correlated with performance during the late phases of learning (Seidler et al. 2005). However, it is not clear whether involvement of striatal pathways during these motor learning tasks is related to error processing because the pathways subserve other functions and the two types of learning rely on differing processes.

In the following section, we outline feedforward and feedback models of basal ganglia contributions to error processing during motor tasks. For more complex computational models of the basal ganglia focusing on action selection and the interactions between multiple corticostriatal circuits, we refer readers to a recent issue of Current Opinion in Neurobiology (e.g., Frank 2011; Ito and Doya 2011; and van der Meer and Redish 2011).

Feedforward Error Processing in the Basal Ganglia

Schmidt and Lee (2011) defines two ways that a person can make an error in achieving a task goal. One is an error in motor planning and the other is an error in motor execution. Motor planning errors involve selection of a motor ‘program’ that is inappropriate for the given situation. Correcting a planning error requires perception of the error and selection of a new action plan. Striatal neurons appear to represent predictive information related to movement and reward, and hence could participate in comparing motor output to an internal model or predicted motor outcomes (Guigon et al. 1995). Schmidt estimated that errors in planning require corrections that have approximately 200 ms latency, because attention is required for correcting an error in selection. The involvement of the BG in such movement planning has been indirectly implicated by studies in humans with BG dysfunction. For example, Smith et al. (2000) report that reaching movements of HD patients begin to become irregular 200–300 ms after movement onset, potentially reflecting a deficit in correcting planning errors. Because corrective actions based on visual (Wolpert et al. 1995) and proprioceptive (Cordo and Flanders 1990) information acquired during reaching movements began to take place at about the time at which HD patients’ movements become irregular, the authors suggest that the system that generates these corrective actions is disturbed. In other words, the movement errors seen in HD patients by Smith et al. (2000) might be part of a more general deficit in action planning and selection as opposed to errors in execution or incorporating task feedback.

Further supporting feedforward theories of BG error processing in movement, the dopamine system broadcasts a prediction error signal for reinforcement learning (Schultz 1998; Rangel et al. 2008). Notably, DA neurons in the animal midbrain respond phasically to primary rewards and stimuli that come, via learning, to predict reward (Fig. 3.2). The pattern of these phasic responses resembles a reward prediction error signal derived from formal reinforcement learning models (Dayan and Balleine 2002; Bayer and Glimcher 2005; Daw and Doya 2006; Morris et al. 2006). Neurons in the BG have been shown to predict reward by firing vigorously in advance of reward upon completion of the requirements for reward attainment (Schultz 1998), hinting that predictive capacity may be a general feature of some basal ganglia structures.

Work by Izawa and Shadmehr (2011) suggests that reward prediction error is a significant component of motor adaptation. The authors suggest that the purpose of learning is not merely to estimate the magnitude of a perturbation but to produce motor commands that maximize reward. Based on their theory, an optimal learner utilizes both reinforcement learning for action selection, and state estimation for identifying the sensory consequences of motor commands (Izawa and Shadmehr 2011). Additionally, we (Seidler et al. 2006) have suggested that the left caudate and globus pallidus may contribute to action selection during sensorimotor adaptation tasks as well.

Error signals are relatively preserved in the ventral striatum of early stage PD patients, but impaired in the dorsolateral striatum relative to healthy controls. This pattern reflects the known selective pattern of dopaminergic denervation in PD (Schonberg et al. 2010). Schonberg et al. (2010) showed that prediction error activity in the human striatum of PD patients was differentially affected by disease and was detectably abnormal only in the dorsal putamen, which is innervated by the depleted nigrostriatal pathway. These findings suggest that prediction error signals measured in the human striatum by the BOLD signal likely reflect midbrain phasic DA activity. These results also provide evidence for a deficiency in predictive error signaling in the dorsolateral striatum of PD patients, which may offer an explanation for the deficits observed by these patients in other reward learning tasks. Evidence for an overlapping prediction error signal during learning with juice and money rewards has also been found in a region of dorsal striatum (the caudate nucleus), while prediction error signals in a subregion of ventral striatum were significantly stronger during learning with money but not juice reward (Valentin and O’Doherty 2009). Taken together, these findings fill in a missing link in the puzzle of the role of prediction error signals in reward-related learning and thus provide additional support to the reinforcement-learning hypothesis of dopamine.

Feedback Error Processing in the Basal Ganglia

Feedback error processing can result in corrective and adaptive actions after an (response) error has occurred. Examples of immediate corrective actions include attempts to inhibit the error online, immediate error corrections, and response time slowing for trials following an error (Rabbitt 1990). The main output of the basal ganglia modulates the action of the thalamus, which relays sensory information to the cortex and basal ganglia (McFarland and Haber 2000). This information stream, Smith et al. (2000) suggest, is likely to participate in error feedback control.

The notion of an error-correction dysfunction following basal ganglia damage is not a new one even though the cerebellum has most often been attributed to error correction functions (Flament et al. 1996). For example, Butters and Rosvold (1968) proposed that the caudate nucleus forms part of a neural mechanism for achieving error correction in the motor system, and Angel et al. (1971) attributed some of the motor deficits observed in PD to slowed error correction mechanisms. In a single unit recording study of basal ganglia activity, in which animals learned a motor sequencing task, cells in the caudate fired only following an incorrect button press, supporting a role for the caudate in mechanisms of error correction.

The idea that the BG nuclei may be critical for signaling online motor decisionmaking processes—particularly during feedback rather than feedforward-based motor control—fits well with a long-held view of their role in filtering competing motor programs—by inhibiting unwanted motor programs and disinhibiting desired actions (Houk 2005; Houk et al. 2007; Mink 1996; Mink and Thach 1993). The primary output of the BG is inhibitory (via tonically active GABAergic neurons in BG output structures). Thus, the striatum can disinhibit cortical targets by releasing tonic inhibition and selectively boosting activation of the most salient channel. Therefore the BG do not select the actions themselves but rather facilitate their execution via the ‘direct pathway’ from striatum to BG output structures (Mink 1996). For example, Chevrier and Schachar (2010) show that error detection deactivates the dorsal substantia nigra, dorsal striatum and ventral bank of the ACC. This network is in keeping with observations of phasic suppression of dopamine neurons on error trials (DeLong 1983; Fiorillo et al. 2003; Ljungberg et al. 1991), and the hypothesized role of this pathway in modulating error processes (Amiez et al. 2005; Brown and Braver 2005; Takenouchi et al. 1999). Furthermore, error trials that lead to the greatest response slowing deactivate structures that modulate dopamine output (Eblen and Graybiel 1995) and encode error magnitude (Menon et al. 2007; Murray et al. 2008). For example, the magnitude of ventral striatum deactivation correlates with error magnitude (Murray et al. 2008; O’Doherty et al. 2003).

Gehring et al. (2000) found evidence for exaggerated compensatory behavior in obsessive-compulsive disorder (OCD), and they also concluded that the BG, which might be overactive in OCD, implements action corrections. Such mechanisms also appear relevant to the complex stereotyped behaviors associated with increased dopamine-mediated activity in the striatum (Canales and Graybiel 2000). Cools (1985) applied control systems theory to the results of experimental manipulations of the basal ganglia in animals, which implicates the striatum in arbitrarily programming the ordering and sequencing of behavioral states of varying complexity. On the basis of pharmacological manipulations, Van den Bercken and Cools (1982) concluded that dopamine-mediated activity in the striatum increases the magnitude of error signals, leading to stereotyped behavior.

As mentioned earlier, patients with basal ganglia disorders show little or no deficits in motor adaptation paradigms like force field (Smith and Shadmehr 2005) or visuomotor perturbations (Agostino et al. 1996; Gabrieli et al. 1997). In the typical force field or visuomotor tasks sensory feedback is available, and therefore sensory prediction errors likely play a dominant role in the adaptive process (Izawa and Shadmehr 2011). Learning from sensory prediction errors appears to depend on the integrity of the cerebellum (Synofzik et al. 2008; Tseng et al. 2007), thus the ability of basal ganglia patients to adapt to visuomotor and force field perturbations may provide evidence that changes in motor output in these tasks are primarily driven by sensory prediction errors. Tunik et al. (2009) suggest that the striatum and cerebellum play complementary roles in regulating ongoing actions when precise updating is required. Similar to the symptomatic HD patients, the individuals with cerebellar damage have poorer reaching movement performance than controls, but the decrement in their performance when perturbations are given is generally more like that of controls than HD subjects (Smith et al. 2000). This suggests that subjects with HD generally have greater deficits in error feedback control than do cerebellar patients. Thus the basal ganglia may play a key role in deciding if and when to correct a given movement by initiating corrective submovements, whereas the cerebellum is likely more involved in amplifying and refining the command signals to specify movements with different amplitudes, velocities, and directions.

Cerebellar Contributions to Error Processing

It has long been known that the cerebellum (Fig. 3.3) is important for motor control and learning, based on accounts of patients with lesions to the structure (Holmes 1939). Early theories of cerebellar function proposed that it plays a key role in the learning of new skills (Marr 1969). More specifically, the cerebellum is known to play a role in error-based motor learning (Criscimagna-Hemminger et al. 2010; Ito 2002, Ramnani 2006; Tseng et al. 2007; Miall and Wolpert 1996) and is thought to be important for the formation of internal models of actions (Imamizu et al. 2000; Diedrichsen et al. 2005; for reviews see Ramnani 2006; Ito 2008). The cerebellum may serve as a monitor of motor errors (Marr 1969; Kitazawa et al. 1998; Doya 2000; Desmurget and Grafton 2000; Seidler et al. 2004), and through this process it is involved in the online tuning of movements. It may also use error information to learn and form internal models of action representations.

Fig. 3.3.

The left panel cartoon illustrates a motor error (art by Lauren Wu); both feedback and feedforward processing of such errors have been linked to the cerebellum (right panel)

Most notably, the cerebellum is thought to be an important structure for detecting errors in movement by comparing the predicted with the actual sensorimotor consequences of a movement (Miall et al. 1993; Blakemore et al. 2001; reviewed in Ramnani 2006). Differences between the predicted feedback and the actual feedback are then compared and movements are adjusted accordingly online. Particularly in adaptation learning, the cerebellum is important for detecting errors, but also using those errors to update forward models for online movement control (Kitazawa et al.1998; Imamizu et al. 2000; Tseng et al. 2007; Izawa and Shadmehr 2011). It is through feedback processes that the cerebellum is able to contribute to feedforward models.

Feedback Error Processing in the Cerebellum

Online correction of movements is thought to be reliant upon the cerebellum. The cerebellum is thought to predict the sensory consequences of action with an efferent copy of the motor commands (Blakemore et al. 2001; Miall et al. 1993). By comparing the actual sensory feedback with the predicted sensory consequences, the cerebellum can detect errors and send corrective signals to the motor cortex to allow for smooth and accurate movements. By detecting and correcting errors during the performance of a task, the cerebellum may aid in learning by updating the existing internal models of a particular motor command.

Adaptation of reaching movements seems to be highly dependent upon sensory prediction errors (Izawa and Shadmehr 2011; Tseng et al. 2007). Patients with hereditary cerebellar ataxia are particularly impaired in sensorimotor adaptation (Tseng et al. 2007). Performance on a reach adaptation task was compared between cerebellar ataxia patients and controls. In one condition the participants were allowed to make motor corrections, while in the other condition participants were unable to correct motor errors. Control participants performed equally well with and without online motor corrections. This indicates that sensory prediction errors are key components for motor adaptation, as the addition of online motor corrections provided no additional gains. Similarly, there was also no difference with and without motor corrections in the cerebellar patient group; however, this group generally showed great deficits in adaptation when compared with controls. Taken together, this indicates that adaptation learning is through sensory prediction errors.

The cerebellar-dependent updating of internal models using sensory feedback seems to largely be an implicit process. Mazzoni and Krakauer (2006) investigated implicit learning and explicit strategies in visuomotor adaptation by giving individuals a specific strategy to employ during task performance. They found that although individuals eliminated errors almost immediately when given explicit information, over the course of adaptation there was an increase in performance error (endpoint location of the cursor). They proposed that this was due to ongoing implicit adaptive processes overriding the explicit strategy. To further investigate and dissociate implicit and explicit processes in visuomotor adaptation learning, Taylor et al. (2010) tested individuals with cerebellar ataxia in a similar paradigm. While both patients and controls reduced errors very quickly when given the explicit strategy, the control group showed the expected increase in endpoint error as training progressed whereas the cerebellar group did not. Thus, the inability of cerebellar patients to implicitly learn allowed for good performance due to the explicit strategy, demonstrating that the cerebellum is critical for implicit adaptive learning.

In a related study, Criscimanga-Hemminger et al. (2010) also demonstrated that sensory prediction errors are key to learning reach adaptation and dependent upon the cerebellum. However, they also found that the size of the error is important. Individuals with cerebellar degeneration were unable to learn from large errors resulting from sudden introduction of a perturbation. However, when the perturbation was gradually increased across trials, learning did occur. They suggest that perhaps other brain regions, or more spared regions of the cerebellum, are involved in learning from small errors, which is why learning occurred. Thus, there seems to be a distinction between learning from small versus large errors.

Functional neuroimaging has also provided insight into feedback error processes in the cerebellum. Grafton et al. (2008) investigated the neural correlates of visuomotor tracking and looked at feedback processes using neuroimaging coupled with models of movement. Feedback responses were those with a delay of 150 ms and due to a response to the position or speed of the cursor. Activity in the cerebellum was strongly correlated with both the magnitude of tracking errors and motor corrections. The authors speculate that error processing in the cerebellum may be related to feedback control while not being specific to the formation of an internal model (Grafton et al. 2008).

To investigate feedback control, we manipulated target size in a joystick aiming task (Seidler et al. 2004). Similarly, Ogawa et al. (2006) manipulated the availability of visual feedback during a visual tracing task. In the case of Seidler et al. (2004), small targets required feedback control. In the visual tracing task (Ogawa et al. 2006), when visual information was available, feedback mechanisms would be employed. The cerebellum was found to be more active under feedback control conditions in both studies. Specifically, we (Seidler et al. 2004) noted a decrease in cerebellar activity when movements were made to larger targets, and Ogawa et al. (2006) found greater activity in the cerebellum when visual information was available. Taken together, these studies support a role for the cerebellum in feedback processes of motor control.

Further supporting this notion is work looking at complex spiking patterns in the cerebellar Purkinje cells of monkeys performing reaching movements Kitazawa et al. (1998) noted a spike in firing that occurred at the end of the arm movement. They proposed that this spike is important for encoding errors allowing for learning, supporting the notion that the cerebellum plays a role in feedback processes.

As briefly described above, feedback from errors may be particularly important for driving the formation of internal models (Kawato and Gomi 1992; Kitazawa et al.1998; Imamizu et al. 2000). During the early stages of learning in both visuomotor adaptation tasks (Imamizu et al. 2000) and sequence learning (Jueptner et al. 1997; Doyon et al. 1997; Doyon et al. 2003), the fMRI activation is seen in the cerebellum. Imamizu et al. (2000) noted widespread cerebellar activation early on in learning that was proportional to motor errors, but after learning, smaller areas of activation in posterior regions of the cerebellum remained, perhaps due to the acquisition of a new internal model.

Feedforward Error Processing in the Cerebellum

The motor system relies at least in part on forward models to perform and learn smooth coordinated movements (Miall and Wolpert 1996). The learning of internal models for feedforward monitoring of motor performance seems to rely at least in part on the cerebellum (Miall et al. 1993; Miall and Wolpert 1996; Kitazawa et al. 1998; Imamizu et al. 2000; Doya 2000; Lang and Bastian 2002; Morton and Bastian 2004). Computational theories of motor control first suggested that the cerebellum might be the brain structure where feedforward information is formed into internal models for motor control (Miall et al. 1993; Kawato and Gomi 1992). The cytoarchitecture of the cerebellum is such that it could support a supervised learning system wherein motor commands serve as input, and incoming sensory information (errors) act as a ‘teacher’ to then refine the motor commands, and create internal models allowing for feedforward control (Marr 1969; Doya 2000). Both patient and neuroimaging studies support such a role for the cerebellum in feedforward control of movement.

As described above, Kitazawa et al. (1998) recorded Purkinje cell firing while monkeys made reaching movements. In addition to the complex spiking seen at the end of the movement, there was also complex spiking at the beginning of the movement. The authors suggest that this spiking may represent the destination of the reach to aid in feedforward control. Thus, the cerebellum also seems to play a key role in feedforward processes.

Investigations of force field adaptation in cerebellar patients have revealed that patients with cerebellar damage or degeneration do not adapt (Smith and Shadmehr 2005), and also do not generalize their performance to similar though unpracticed tasks (Smith and Shadmehr 2005; Maschke et al. 2004; Morton and Bastian 2004). Smith and Shadmehr demonstrated that although cerebellar patients modified their performance using within trial corrections, they were unable to use their errors to learn (across trial corrections). Furthermore, they were unable to generalize the task when moving in the opposite direction. Maschke et al. (2004) also demonstrated that cerebellar patients show no evidence of learning in a forcefield adaptation task, and again, found that patients were unable to generalize to unpracticed regions of the workspace. Taken together, it is clear that cerebellar patients have adaptation deficits likely due to their inability to process errors across trials, which allows for the creation and updation of internal models.

Lang and Bastian (2002) tested cerebellar patients as well as healthy controls drawing a figure-of-eight with their arms (in a standing position), under single and dual-task conditions. During the figure-of-eight task, participants were told to optimize performance by increasing the number of eights that were traced in the air. While cerebellar patients did show some improvement on the figure of eight task, they had marked difficulty under dual-task conditions and returned to initial performance levels, while control participants showed no interference. Cerebellar patients were unable to automatically perform the task relying upon feedforward mechanisms, and instead needed to rely on cognitive control.

Diedrichsen et al. (2005) provide further evidence for the role of the cerebellum in learning and forming internal models of reaching. They investigated both target and execution errors in a forcefield adaptation task using functional neuroimaging. Target errors are the errors due to unpredictability in the location of a target, whereas execution errors are the result of a miscalibration of internal models. Activity in cerebellar hemispheres V, VI, and VII was seen during execution errors, supporting that this region is important for feedforward control.

Finally, noninvasive brain stimulation of the cerebellum during target aiming movements also provides support for the role of the cerebellum in feedforward movement control (Miall et al. 2007). Transcranial magnetic stimulation (TMS) was administered to lateral regions of the cerebellum during the reach movement to disrupt its processing. Individuals receiving TMS showed greater errors in their movements relative to those that did not receive stimulation. Interestingly, the movement errors were consistent with out-of-date movement estimations from earlier in the movement. The cerebellum seems thus to be important for estimating the state of the arm during reach, likely through feedforward processes.

Caveats

Numerous studies have demonstrated an important role for error experience in motor learning. Reduction of errors via physical guidance has been shown to hinder learning (Domingo and Ferris 2009) and error augmentation has been shown to aid learning processes (Domingo and Ferris 2010; Wei et al. 2005; Patton et al. 2006; Emken and Reinkensmeyer 2005). However, motor learning is not critically dependent on error detection and correction. For example, Diedrichsen et al. (2010) have reported evidence for use-dependent learning, or improvements via repetition of correct movements. Moreover, Wachter et al. (2009) have shown that procedural learning is enhanced with positive reinforcement. Thus, although error detection and correction processes can be important for motor learning, other mechanisms can be relied upon as well. In addition, although it is self-evident, we wish to emphasize that error based learning can only take place in the presence of errors. That is, the small performance adjustments that take place late in the learning process are likely to rely on differing mechanisms. Finally, although error-based learning can be important to early learning, other cognitive processes such as working memory also appear to make substantial, and perhaps related, contributions (for recent reviews see Seidler et al. under review, in press).

Conclusions and Future Directions

There is substantial evidence favoring a role for the cerebellum, medial prefrontal/ACC, and basal ganglia systems in error corrective processes occurring during motor skill learning. Cerebellar networks have long been implicated in these behaviors. Meanwhile, although traditionally viewed as detecting discrete, binary (present or absent), ‘cognitive’ errors, evidence is accumulating to support a role for medial prefrontal/ACC error processing mechanisms in motor skill learning, including reports that this region scales its activity in a continuous fashion with motor error magnitude. Additionally, data supports a role for basal ganglia pathways in both within trial error corrections and reinforcement-dependent error learning mechanisms. What remains to be resolved is whether these pathways act independently or cooperatively during motor learning, and whether such interactions might vary depending on the task to be learned and the environmental context. Recent studies have reported both structural (Bostan et al. 2010; Bostan and Strick 2010) and functional (Kwak et al. 2010) connectivity between the cerebellum and basal ganglia, supporting the plausibility of multiple interactive error correction systems for motor learning.