Formation of model-free motor memories during motor adaptation depends on perturbation schedule

Abstract

Motor adaptation to an external perturbation relies on several mechanisms such as model-based, model-free, strategic, or repetition-dependent learning. Depending on the experimental conditions, each of these mechanisms has more or less weight in the final adaptation state. Here we focused on the conditions that lead to the formation of a model-free motor memory (Huang VS, Haith AM, Mazzoni P, Krakauer JW. Neuron 70: 787–801, 2011), i.e., a memory that does not depend on an internal model or on the size or direction of the errors experienced during the learning. The formation of such model-free motor memory was hypothesized to depend on the schedule of the perturbation (Orban de Xivry JJ, Ahmadi-Pajouh MA, Harran MD, Salimpour Y, Shadmehr R. J Neurophysiol 109: 124–136, 2013). Here we built on this observation by directly testing the nature of the motor memory after abrupt or gradual introduction of a visuomotor rotation, in an experimental paradigm where the presence of model-free motor memory can be identified (Huang VS, Haith AM, Mazzoni P, Krakauer JW. Neuron 70: 787–801, 2011). We found that relearning was faster after abrupt than gradual perturbation, which suggests that model-free learning is reduced during gradual adaptation to a visuomotor rotation. In addition, the presence of savings after abrupt introduction of the perturbation but gradual extinction of the motor memory suggests that unexpected errors are necessary to induce a model-free motor memory. Overall, these data support the hypothesis that different perturbation schedules do not lead to a more or less stabilized motor memory but to distinct motor memories with different attributes and neural representations.

while for a long time motor adaptation had been considered as a process that learns from errors (Smith et al. 2006), recent studies have suggested that other mechanisms also come into play in order to adapt motor behavior to a new environment dynamics (Della-Maggiore et al. 2015; Shmuelof and Krakauer 2011). Human subjects can use explicit strategies to adapt their new motor behaviors (Mazzoni and Krakauer 2006; Taylor et al. 2010, 2014; Taylor and Ivry 2011, 2012). Reward can also drive a change in motor behavior through reinforcement learning (Abe et al. 2011; Izawa and Shadmehr 2011; Madelain et al. 2011). Finally, repetition of movements can modify motor behavior (Diedrichsen et al. 2010; Leow et al. 2014; Verstynen and Sabes 2011). In addition, if this repetition is combined with a reinforcement of successful actions, it gives rise to model-free learning. That is, subjects learn through exploration and memorize the movements that lead to a reward (Haith and Krakauer 2013). Interestingly, the content of this model-free motor memory can be used to speed up the learning of a subsequent task if the memorized actions are useful to solve the second task. In other words, using the content of the model-free motor memory leads to savings (faster relearning than initial learning), as demonstrated by Huang and colleagues (2011) in the case of visuomotor rotation.

Taylor and colleagues (2014) demonstrated that the availability of visual feedback during movement or end-point error modulated the weight of explicit (i.e., strategic) and implicit (i.e., model based) learning during a visuomotor rotation task. In this study, we wanted to modulate the weight of model-free and model-based learning by varying another task parameter. Namely, here we focused on how the schedule of the perturbation can modulate the weight of model-free learning in different visuomotor rotation paradigms. If, as hypothesized in the original paper (Huang et al. 2011; Haith and Krakauer 2013), model-free learning can be associated with motor cortex plasticity, then the schedule of the perturbation, which modulates the level of motor cortex plasticity during force-field adaptation, should have an effect on the formation of a model-free motor memory. Indeed, motor cortex plasticity appears to be critical for motor adaptation when the perturbation is introduced abruptly but not when it is introduced gradually (Orban de Xivry et al. 2011). In addition, changes in motor cortex that are monitored by premovement motor-evoked potentials are detected during adaptation to an abrupt perturbation but not during adaptation to a gradual perturbation (Orban de Xivry et al. 2013). Therefore this study aimed at testing the hypothesis that a model-free motor memory is formed after abrupt introduction of a perturbation but not after gradual introduction of the same perturbation by looking at rates of relearning after abrupt and gradual introduction of a perturbation. Our prediction was that because gradual introduction of a perturbation does not engage motor cortex plasticity, it should not yield the expression of strong savings upon relearning.

In this study, four different perturbation schedules were used. First, we aimed at confirming that abrupt introduction of a visuomotor rotation led to the formation of a model-free motor memory and to faster relearning than initial learning, as has been shown in Huang et al. (2011). Second, we compared the rate of relearning after gradual and abrupt introduction of visuomotor rotation. Third, we tested the influence of repetition after gradual introduction of the perturbation on the induction of model-free motor memories in order to identify the relationship between repetition-dependent (Diedrichsen et al. 2010) and model-free (Huang et al. 2011) motor memories. Finally, the effect of gradual extinction of the perturbation on the integrity of the model-free motor memory was tested because a fear conditioning study showed that gradual extinction destroys memories (Gershman et al. 2013), which suggests that gradual extinction of a perturbation might erase the motor memory.

METHODS

Subjects.

Forty healthy young subjects (10 subjects per group) were enrolled for the experiment. All subjects had no history of neurological disorders and were right-handed and between 18 and 40 yr old. All of them gave written informed consent. The procedures were approved by the Université catholique de Louvain Ethics Committee and were in accordance with the Declaration of Helsinki.

Methods.

Subjects sat in front of a robotic arm (Endpoint Kinarm, BKin Technologies, Kingston, ON, Canada). They controlled the handle of the robot in order to move a cursor that was displayed on a horizontal mirror positioned above the arm. The cursor and targets of interest were displayed on a screen placed tangentially above the mirror and were reflected by it. Because the mirror was halfway between the handle and the screen, the cursor appeared to be positioned at the same position in space as the hand after horizontal positions were properly calibrated. With this setup, subjects could not see their hand and the displayed cursor was the only available visual feedback of their arm position.

The robot controlled the display through custom-made MATLAB programs uploaded to a real-time computer. It also monitored hand position, velocity, and acceleration at 1,000 Hz. Kinematic and dynamic data were stored on a PC for off-line analysis.

Protocol.

Each trial started with the appearance of a 25-mm² cyan disk (the starting position) that was located in the middle of the screen, 15 cm ahead of the subject. The robot pulled the hand of the subject within the starting target. As soon as the hand cursor was stabilized inside the starting point, a red target that was 6 cm away from the starting position appeared. The seven possible locations for the target (Fig. 1A) were equally distributed between 40° and 100° (counterclockwise). Subjects were instructed to quickly move through the target. When the subjects crossed the target, their hand left a green imprint to show where the hand crossed the target. If the movement was fast enough (<250 ms) the target became yellow, which indicated a correct movement speed. It became blue otherwise. Movement onset was detected online as the time the cursor left the starting area. Movement offset corresponded to the time when the hand had traveled >6 cm. During the movements, a white cursor provided either veridical or rotated online visual feedback (the type of feedback depended on the block).

Fig. 1.

Methods. A: location of the 7 targets (gray disks) is represented with respect to the starting position. Targets T1 (red line) and T2 (blue line) are associated with a counterclockwise and a clockwise 30° visuomotor rotation, respectively. Therefore, in both cases, the 70° direction highlighted by the green line is the correct hand movement for the cursor to reach target T1 with a 30° visuomotor rotation and target T2 with a −30° rotation. B: each group of 10 subjects experienced a different perturbation schedule as illustrated by the evolution of the perturbation angle over the course of the experiment. Each of the schedules was composed of 4 different phases, which are represented by different colors. During the training phase (red), a counterclockwise rotation was introduced while only T1 was presented. The schedule of the perturbation varied across groups. During the relearning phase, a 30° clockwise rotation was abruptly introduced and only T2 was presented.

The first practice block consisted of 85 reaching movements (12 or 13 movements to each target) to pseudorandomly selected targets (baseline, Fig. 1B). Next, the perturbation was introduced differently for the various groups. The perturbation consisted of a counterclockwise rotation of the hand cursor that peaked at 30° for all groups (+30°, Fig. 1B). During this period, only target T1 was presented. That is, when the perturbation was maximal, the hand had to move to the 70° direction (Fig. 1A) for the cursor to move to the 100° direction (target T1, Fig. 1A).

The schedule of the perturbation varied across groups. For the abrupt (ABR) group (top row of Fig. 1B), the perturbation was fully introduced on trial 86 and remained constant for 160 trials and then removed on trial 246. For the gradual (GRA) group (2nd row of Fig. 1B), the perturbation was gradually introduced over 140 trials and then stayed constant for 20 trials before being suddenly removed at the start of the washout period (trial 246). For the LONG group (3rd row of Fig. 1B), the perturbation was also introduced gradually over 140 trials but the perturbation remained constant for 160 trials to match the length of constant perturbation of the ABR group. After the training period, the perturbation was suddenly removed (on trial 386). For the extinction (EXT) group (bottom row of Fig. 1B), the perturbation was introduced abruptly and stayed constant for 160 trials as for the ABR group. However, for this group, the perturbation was gradually decreased over 60 trials and then remained at zero for another 20 trials (but only target T1 was presented for those trials).

During the washout period, all the targets were again pseudorandomly presented for 80 trials (11 or 12 movements to each target) and there was no perturbation. In all groups, savings was then tested by presenting the 40° target with a 30° clockwise rotation angle (−30°, Fig. 1B). During this period, only target T2 at 40° was presented. That is, the hand had to move to the 70° direction (Fig. 1A) for the cursor to move to the 40° direction (target T2, Fig. 1A). In hand space, the solution to the second perturbation (Fig. 1A) was identical to the solution of the first perturbation (direction of movement was 70° in both cases). Across all groups, baseline, washout, and relearning periods (Fig. 1B) were identical. Only the training period was varied across groups.

Data analysis.

We used the exact same dependent variable as Huang et al. (2011). Therefore, the dependent variable of interest was the rate of learning when the subjects were exposed to a perturbation. To compute this rate, the angular error when the subjects crossed the target was computed for each trial and each subject. An exponential function was fitted to the mean data from each group and compared with a permutation test. The exponential function was

$$y\left(k\right)=A\cdot e^{-rate\cdot k}+B$$

where k is trial number within the specific learning epoch and A, B, and rate are three constants that are fitted to the data. The values of y were fitted to the angular error data of the whole learning or relearning periods. Importantly, the learning rate can only be computed during abrupt introduction of the perturbation. Therefore, we cannot measure the savings effect per se in the GRA and LONG groups. Rather, we compare the rates of relearning of the GRA and LONG groups to the rates of relearning of the control (ABR) group. Some subjects overcompensated the visuomotor rotation during the initial learning (with errors now >10° in the direction opposite to the visuomotor rotations). These errors are clearly due to the use of a strategy to try and reduce the errors quickly. This behavior cannot be captured by a single exponential and biases the computation of the learning rate toward higher values. These trials (7 data points from 3 different subjects—2 in ABR and 1 in EXT) were excluded from the analyses.

The authors confirm that no subjects were excluded from the analyses and that no other groups or conditions were tested. Sample size was fixed at 10 subjects per group before the start of the experiment. This number was deliberately chosen higher than in previous experiments on savings (Huang et al. 2011), as we looked at a between-subject effect (comparing the experimental groups) and not only within-subject effects (e.g., savings effect in ABR and EXT).

RESULTS

In these experiments, human subjects were asked to quickly reach to a target while experiencing a visuomotor rotation. That is, the cursor direction was rotated with respect to hand direction by 30° counterclockwise during the training period and 30° clockwise during the relearning period. In all these experiments, the location of the targets was arranged in such a way that the hand direction that brought the rotated cursor to the target was identical during the training and relearning periods, despite opposite visuomotor rotations (see Fig. 1A). Using such target locations, Huang and colleagues demonstrated that the initial abrupt introduction of the perturbation led to a motor memory that enabled faster relearning during the second perturbation thanks to model-free learning (Huang et al. 2011). Here we modified the schedule of the first exposure to the perturbation in order to test whether perturbation schedule modulates the formation of model-free motor memory, which would manifest through the presence of fast relearning during the second exposure.

In the ABR condition, subjects experienced large errors after the sudden introduction of the perturbation but very quickly adapted their hand direction in order to bring the cursor to the target. As a result, most of their hand movements were localized near the ideal hand solution to the visuomotor rotation (Fig. 2A, left). In contrast, the nature of the GRA condition made the subjects experience a whole range of hand movement direction in response to the increasing visuomotor rotation angle (Fig. 2A, right). For these subjects, hand directions gradually converged toward the ideal hand solution for the full perturbation and were therefore equally distributed between the target direction and the ideal hand direction at the end of the training period (Fig. 2A). Despite the differences in the experienced hand trajectories during the training period, both groups learned the visuomotor rotation and achieved small errors by the end of training (Fig. 2B). These errors were slightly larger for the GRA group than for the ABR group, although this difference did not reach significance [t(18) = 1.79, P = 0.1].

Fig. 2.

Comparison of the abrupt (ABR) and gradual (GRA) groups during the training and savings periods. A: heat maps representing the hand movement direction experienced during the training period. Each heat map is normalized separately. Blue colors are associated with directions that are barely followed, while red colors correspond to frequently used directions. Solid and dashed white lines represent the ideal cursor and hand directions under the full visuomotor rotation, respectively. B: angular error of the cursor with respect to the target at the end of the training period (average over last 10 trials). Error bars are SE. C: evolution of the angular error during the relearning period. Solid lines represent the intersubject average for each group. Dashed lines represent the exponential fit to those data. Inset illustrates the evolution of the angular error for the first 10 trials. Error bars are SE. D: rate of relearning for each group. Error bars are SE of the mean obtained by bootstrap.

In this experiment, we hypothesized that model-free learning would be associated with a faster rate of adaptation during the relearning period for the subjects in the ABR condition than in the GRA condition. To test this hypothesis, after washout of the initial learning subjects from both groups experienced a second visuomotor perturbation that could lead to fast relearning if model-free learning was present during the initial learning (Huang et al. 2011, see Fig. 1).

Upon learning of the second visuomotor rotation, subjects from the ABR group learned faster than subjects from the GRA group (Fig. 2C). Figure 2C, inset, illustrates that the two groups experienced similar errors on the first trial but that subjects from the ABR group reduced this error faster. To quantify the rate of relearning, we fitted an exponential function to the angular error data. A permutation test (n = 10,000) was used to compare the rates of relearning across the two groups. This test confirmed that the rates of relearning were faster for the ABR group than for the GRA group (Fig. 2D; P = 0.011). Unfortunately, the savings effect could not be measured per se in the GRA group, as the rate of learning cannot be measured during the gradual introduction of the perturbation.

Influence of repetition.

Abrupt and gradual introduction of the perturbation differed in many ways, and these two conditions alone do not allow us to infer what the necessary conditions for faster relearning are. First of all, the amount of repetition once full adaptation has been reached is dramatically shorter for the GRA condition (∼20 trials) than for the ABR group (at least 80 trials). Therefore, to control for the possible confound of the number of repetitions on the appearance of savings, a third group of 10 subjects participated in the experiment that was similar to the GRA condition except that these subjects trained for an additional 140 trials with full visuomotor rotation before washout (LONG group). That is, these subjects experienced the same small errors during the training period as the GRA group but also repeated the adapted movements many times as the ABR group.

These additional repetitions largely affected the distribution of movement direction experienced during the training phase (Fig. 3A). That is, in addition to having sampled the whole range of visuomotor angles, a peak in the distribution of movement angles appears next to the final hand solution direction. This peak was located close to the hand movement direction reached by the end of training in the ABR condition (compare both panels of Fig. 3A). In addition, while in the GRA group there was a residual error at the end of the introduction of the perturbation (Fig. 2B), this error was much reduced after extensive repetition of the adapted movement (Fig. 3B). The error at the end of the training phase was smaller for the LONG group than for the ABR group [t(18) = −3.09, P = 0.006].

Fig. 3.

Comparison of the abrupt (ABR) and long (LONG) groups during the training and savings periods. A: heat maps representing the hand movement direction experienced during the training period. Each heat map is normalized separately. Blue colors are associated with directions that are barely followed, while red colors correspond to frequently used directions. Solid and dashed white lines represent the ideal cursor and hand directions under the full visuomotor rotation, respectively. B: angular error of the cursor with respect to the target at the end of the training period (average over last 10 trials). Error bars are SE. C: evolution of the angular error during the relearning period. Solid lines represent the intersubject average for each group. Dashed lines represent the exponential fit to those data. Inset illustrates the evolution of the angular error for the first 10 trials. Error bars are SE. D: rate of relearning for each group. Error bars are SE of the mean obtained by bootstrap.

Despite the better performance of the LONG group by the end of the training period, subjects from the ABR group were still learning faster during the relearning period than subjects from the LONG group (Fig. 3C). A permutation test (10,000 permutations) was conducted and confirmed that subjects from the ABR group had faster relearning than subjects from the LONG group (P = 0.042). Again, the savings effect could not be measured directly in the LONG group, as the gradual introduction of the perturbation prevents us from assessing the rate of learning. In addition, the rate of relearning of the LONG group was similar to the rate of relearning of the GRA group (mean ± SE: GRA: 0.2 ± 0.02, LONG: 0.21 ± 0.02, P = 0.6).

Testing for savings.

The experiment was designed to induce faster relearning in groups that experienced large errors during the training periods. Large errors were experienced by the subjects in the ABR condition but also in the fourth group, where the perturbation was suddenly introduced but washed out gradually (EXT condition). The gradual washout aimed at testing the importance of large errors early on during the washout period on the presence of savings. The evolution of errors over the course of learning is illustrated for both groups in Fig. 4.

Fig. 4.

Evolution of the angular error (°) between the cursor and the target over the course of trials during the training and relearning periods (gray and black traces, respectively) for the ABR group (A) and for the EXT group (B). Solid lines represent the intersubject average for each period (training and relearning). Dashed lines represent the exponential fit to the data. Inset illustrates the evolution of the angular error for the first 10 trials. Error bars are SE. Right: rate of learning and relearning for each group. Error bars are SE of the mean obtained by bootstrap.

In both ABR and EXT groups, relearning appears to be faster than initial learning (i.e., during training), as the black curves are below the gray curves (Fig. 4). A permutation test confirmed the existence of savings for the ABR group (P = 0.01) and for the EXT group (P = 0.0045). However, the rate of relearning of the EXT group (0.22 ± 0.025, mean ± SE) was not different from that of the other groups (permutation test, ABR: P = 0.2, GRA: P = 0.84, and LONG: P = 0.74).

As we hypothesized that the presence of faster relearning was due to the abrupt introduction of the perturbation during the training phase, we wanted to make sure that the subjects who experienced an abrupt perturbation during the training period had on average a faster relearning rate than the subjects from the gradual groups. The contrary would seriously hamper our ability to conclude that perturbation schedule during the training period has a significant influence on the rates of relearning, As expected, we found that the 20 subjects who experienced the initial abrupt introduction of the perturbation had a higher relearning rate than the 20 subjects from the two gradual conditions (permutation test, P = 0.028). Obviously, subjects from each of the abrupt groups (ABR or EXT) exhibited savings during the relearning period (permutation test, P < 0.0001). Overall, these results confirmed that savings induced by model-free learning can be observed after abrupt introduction of a visuomotor rotation and that the rates of relearning were reduced after gradual introduction.

Summary of results.

In the four groups, subjects experienced different errors at different moments of the experiment. In the ABR group, large errors were present at the start of the training and washout periods (Fig. 5). One of those two epochs should yield the formation of a motor memory responsible for the savings thereafter. In GRA and LONG groups, large errors were only experienced at the start of the washout period. These large errors did not provide a cue for further faster relearning even though these errors were in the same direction as the errors experienced at the start of the relearning period. Finally, the presence of large errors at the start of the training period was sufficient to induce later savings in the EXT group. These subjects did not experience large errors at the start of the washout period, which are in the same direction and around the same magnitude as the errors experienced at the start of the relearning period. These observations suggest that the presence of large and unexpected errors, even though they are opposite in sign to the error experienced at the start of the relearning period, is critical to the formation of the model-free motor memory that induces savings.

Fig. 5.

Angular error (°) between the cursor and the target at the start of the learning (dark gray), washout (gray), and relearning (light gray) periods. For the learning and washout periods, the error was computed over the first 5 trials.

DISCUSSION

In this study, we tested the hypothesis that perturbation schedule affects the expression of model-free learning through savings in the same way that it modulates reorganization of the motor cortex. Across the 20 subjects who experienced an abrupt introduction of the perturbation (ABR and EXT groups), we found that relearning was faster than initial learning and thus reproduced the result of Huang et al. (2011). In contrast, we found that after gradual introduction of the perturbation relearning was slower than after abrupt introduction, which suggests that the weight of model-free learning is reduced during gradual adaptation to a visuomotor rotation (Huang et al. 2011). Note that these experiments do not allow us to measure the amount of savings in the gradual conditions. Indeed, for the gradual groups, it is not possible to measure the rates of learning during the training period and to compare them to the rates of relearning. In addition, this paradigm prevents the subjects from taking advantage during the relearning period of a strategy that could be built during the training period, as both rotations are in opposite directions. Therefore, any strategy devised during the training period would be counterproductive during the relearning period.

Necessary condition for formation of a model-free motor memory.

The absence of faster relearning after the gradual introduction of the perturbation followed by many repetitions of the adapted hand movement (LONG group) suggests that extensive practice does not facilitate learning upon reexposure (see Huang et al. 2011 for the absence of facilitation when nonadapted movements are repeated). This result is surprising, as there is evidence that movement practice leads to motor cortex plasticity in humans as measured by TMS (Classen et al. 1998; Hayashi et al. 2005) and EEG (Halder et al. 2005) as well as in rodents (Fu et al. 2012). However, it is compatible with the observation that mere repetition of movement that does not involve skill learning does not affect motor cortex activity (Plautz et al. 2000). In the latter study, movement repetition was not accompanied by better repetition or reduction of errors. In contrast, repetition after gradual introduction of the perturbation was associated with an important reduction in movement error. Such accurate performance was insufficient to induce the formation of model-free motor memories that can facilitate subsequent learning.

These experiments also suggest that large errors at the start of the washout period are insufficient to facilitate subsequent relearning even though these errors are in the same direction and around the same magnitude as the errors experienced after the first trials during the relearning period (see Fig. 5). Indeed, the presence of such large errors at the start of the washout period was insufficient to induce fast relearning in the gradual conditions (GRA and LONG groups), while savings was observed when these large errors were suppressed (EXT group). Furthermore, the presence of savings in the EXT group suggests that large errors at the start of the training period are sufficient to induce the formation of model-free motor memories, while the experience of large errors at the start of the washout is not required. This result is reminiscent of the study of Pekny and colleagues (2011), where expression of motor memories was preserved even after gradual extinction, but contrasts with a fear extinction study where gradual extinction of fear conditioning yielded memory erasure (Gershman et al. 2013).

Reward prediction error.

The large errors at the start of the training period were in the direction opposite to the errors experienced at the start of the relearning period. This suggests that the large errors observed at the start of the relearning period do not need to be in the same direction as the errors observed during the learning period in order to observe savings during the relearning period. That is, the direction of the errors is irrelevant but their magnitude is important, as faster relearning was not observed in the gradual conditions. This is fully consistent with the fact that, after extensive training with a clockwise force field, a single trial with a force field in the opposite direction (i.e., the error is in the opposite direction) has the ability to induce the retrieval of the previously learned motor memory (see Fig. 5 of Gonzalez Castro et al. 2014). This suggests that the reward prediction error experienced early in the training phase might be critical for the savings effect. Indeed, reward prediction error only occurs with large errors but is insensitive to their direction. Such a reward prediction error could be used either to build the model-free motor memory through reinforcement learning (Darshan et al. 2014; Izawa and Shadmehr 2011; Madelain et al. 2011) independently of the adaptation of an internal model (Izawa et al. 2012) or to tag the model-free motor memory in order to facilitate its recall when another reward prediction error is experienced (Shmuelof et al. 2012). The involvement of reward prediction would also be consistent with the absence of savings observed in Parkinson's disease patients (Bédard and Sanes 2011; Leow et al. 2012, 2013; Marinelli et al. 2009).

Different mechanisms induce savings.

Three of the groups of the present experiments are incompatible with the idea that only a memory of error could account for the presence of faster adaptation upon reexposure (Herzfeld et al. 2014). Memory of error can account for faster relearning when errors similar to the errors experienced at the start of the relearning period have already been experienced. At the start of the washout period, subjects from the GRA and LONG groups experienced errors that were around the same magnitude and in the same direction as the errors that would be experienced at the start of the relearning period. Despite this experience of errors, subjects from these two groups relearned more slowly than subjects from the ABR group. In addition, subjects from the EXT group did not experience large errors similar to the errors experienced initially during the relearning period. Nonetheless, these subjects exhibited the savings effect. In contrast, model-free learning cannot account for several savings effects observed in the literature, such as meta-learning (Braun et al. 2009; Herzfeld et al. 2014; Turnham et al. 2012), because the observed savings cannot be attributed to the use of similar movements during the training and relearning phases. Therefore, the presence of savings could be due to at least two different mechanisms, one linked to model-free learning and the motor cortex and one linked to memory of errors and presumably the cerebellum.

There is a general agreement that visuomotor rotations and force-field adaptation are governed by similar learning mechanisms even though there are also different error signals underlying the two perturbations (e.g., with regard to sensory errors experienced). Sensitivity to errors is found for both force-field and visual perturbations (Herzfeld et al. 2014). Similarly, the effect of experiencing reward prediction errors induces the retrieval of motor memories for both force-field adaptation (Gonzalez Castro et al. 2014) and visuomotor rotation (present study). However, for both types of perturbations, two different mechanisms can induce savings.

Distinct motor memories are formed after abrupt and gradual perturbation.

Several papers suggest that memories that rely on different neural representations only partially transfer or interfere. In an experiment where left-hand movements, which were perturbed by a force field, were either accompanied by a nonperturbed movement of the right hand (bimanual condition) or not (unimanual condition), Nozaki and colleagues (2006) found limited transfer of learning between these two conditions. Limited transfer of learning was also observed between rhythmic and discrete movements (Ikegami et al. 2010). In addition, distinct motor plans or distinct tools are also associated with limited interference between two opposing perturbations (Cothros et al. 2009; Hirashima and Nozaki 2012). Overall, these studies suggest that transfer or interference between motor memories requires a common neural representation. Interference or transfer studies also suggest that motor memories formed during abrupt and gradual perturbations rely on different neural representation. For instance, interference between consecutive gradual and abrupt perturbations was reduced compared with interference between two consecutive abrupt but opposite perturbations (Pekny et al. 2011). The existence of savings is another way of assessing the transfer of motor memories. Therefore, the absence of faster relearning between initial gradual learning and ensuing abrupt adaptation (i.e., limited transfer) suggests that gradual and abrupt schedules give rise to two different motor memories, which rely on largely nonoverlapping neural representations (Orban de Xivry et al. 2011, 2013; Werner et al. 2014). This implies that the behavioral differences observed after abrupt and gradual adaptation to a perturbation (Abeele and Bock 2001; Berniker and Körding 2008; Buch et al. 2003; Habagishi et al. 2014; Kagerer et al. 1997; Klassen et al. 2005; Kluzik et al. 2008; Michel et al. 2007; Saijo and Gomi 2010; Torres-Oviedo and Bastian 2012; Wong and Shelhamer 2011) do not stem from a single motor memory that acquires different features depending on the schedule but from different memories, with different neural representations and hence different features. Accordingly, the presence of faster relearning after abrupt compared with gradual adaptation cannot be linked to the long-term aspect of motor memories, as it has been shown that motor memories should last longer after a gradual than an abrupt schedule (Huang and Shadmehr 2009; Kagerer et al. 1997).

Model-free learning and the motor cortex.

This difference in neural representations of motor memories formed under gradual and abrupt schedules has been investigated by noninvasive brain stimulation at the level of the motor cortex. Two recent TMS studies on M1 suggested that M1 reorganization took place after abrupt adaptation to a force-field perturbation but not after gradual adaptation (Orban de Xivry et al. 2011, 2013). Originally, these results were interpreted in light of repetition-dependent or use-dependent processes, as repetition of movement alters M1 reorganization (Bütefisch et al. 2000; Classen et al. 1998; Stefan et al. 2005). However, we demonstrate here that extended repetition of the learned motor behaviors after gradual introduction of the perturbation did not restore the faster relearning effect at reexposure. This finding differs from what had been found in a previous study (Orban de Xivry et al. 2011). Therefore, there appears to be a dichotomy between model-free and use-dependent learning, at least in terms of savings effect. Finally, the role of M1 in model-free learning could also explain the deterioration of savings by repetitive TMS of the motor cortex (Richardson et al. 2006).

However, while these studies point to a role of the motor cortex in model-free learning, the role of the cerebellum in the abrupt condition appears to be important as well. For instance, projections from the cerebellum to the motor cortex are modulated during abrupt adaptation but not during gradual adaptation (Schlerf et al. 2012). Nevertheless, contradictory data from cerebellar patients do not allow us to assess the role of the cerebellum in the abrupt and gradual conditions (Criscimagna-Hemminger et al. 2010; Gibo et al. 2013; Izawa et al. 2012; Smith and Shadmehr 2005).

Conclusions.

Perturbation schedule modulates motor memory attributes in the same way that it modulates the amount of reorganization of the motor cortex during motor adaptation (Orban de Xivry et al. 2013). That is, gradual adaptation leads neither to the formation of a model-free motor memory nor to motor cortex reorganization. In addition, the absence of transfer from gradual to abrupt perturbation, as the absence of transfer from unimanual to bimanual motor adaptation, suggests that these memories rely on separate neural representations and actually correspond to distinct motor memories. Finally, the formation of a model-free motor memory requires unexpected errors (namely, reward prediction error), but neither the exact errors experienced nor the repetition of the learned movements appear to be critical.

GRANTS

This work was supported by the Belgian Program on Interuniversity Attraction Poles and PRODEX program initiated by the Belgian Federal Science Policy Office, Actions de Recherche Concertées (French community, Belgium), and the European Space Agency (ESA) of the European Union. J.-J. Orban de Xivry is supported by the Brains Back to Brussels program from the Brussels Region (Belgium).

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the author(s).

AUTHOR CONTRIBUTIONS

Author contributions: J.-J.O.d.X. conception and design of research; J.-J.O.d.X. performed experiments; J.-J.O.d.X. analyzed data; J.-J.O.d.X. interpreted results of experiments; J.-J.O.d.X. prepared figures; J.-J.O.d.X. drafted manuscript; J.-J.O.d.X. and P.L. edited and revised manuscript; J.-J.O.d.X. and P.L. approved final version of manuscript.