Somatosensory Cortex Plays an Essential Role in Forelimb Motor Adaptation in Mice

SUMMARY

Our motor outputs are constantly re-calibrated to adapt to systematic perturbations. This motor adaptation is thought to depend on the ability to form a memory of a systematic perturbation, often called an internal model. However, the mechanisms underlying the formation, storage, and expression of such models remain unknown. Here, we developed a mouse model to study forelimb adaptation to force field perturbations. We found that temporally precise photoinhibition of somatosensory cortex (S1) applied concurrently with the force field abolished the ability to update subsequent motor commands needed to reduce motor errors. This S1 photoinhibition did not impair basic motor patterns, post-perturbation completion of the action, or their performance in a reward-based learning task. Moreover, S1 photoinhibition after partial adaptation blocked further adaptation, but did not affect the expression of already-adapted motor commands. Thus, S1 is critically involved in updating the memory about perturbation that is essential for forelimb motor adaptations.

In Brief

Mathis et al. developed a force field-based motor adaptation task for mice. They found that photoinhibition of forelimb S1 abolished motor adaptation while sparing basic kinematics and their ability to learn a new reward location that required similar motor outputs.

INTRODUCTION

Humans can adapt their motor commands to a plethora of sensory perturbations, like wearing prism glasses, adjusting to external loads, or moving in artificially applied force fields (von Helmholtz, 1867; Shadmehr and Mussa-Ivaldi, 1994; Wolpert et al., 2011; Shadmehr et al., 2010; Herzfeld et al., 2014). It is thought that adaptation critically depends on the animal’s ability to detect the difference between the intended and actual movement. Increasing evidence suggests that motor adaptation is driven by sensory prediction errors, the difference between the predicted and the actual sensory experiences about a movement (Wolpert et al., 1995; Wolpert and Miall, 1996; Hwang and Shadmehr, 2005; Tseng et al., 2007). However, other mechanisms have also been proposed. According to feedback error learning (Albert and Shadmehr, 2016; Ito, 2013; Kawato and Gomi, 1992), the act of making a corrective motor action during the pull generates an error signal. This feedback control signal serves as a “template,” or an error signal, to update future actions. Additionally, it has been observed that reward prediction errors, the difference between predicted and actual reward, may drive motor adaptation in certain conditions (Galea et al., 2015; Huang et al., 2011; Izawa and Shadmehr, 2011), such as when visual feedback is fully or partially absent (Izawa and Shadmehr, 2011).

Studies in humans have provided insights into mechanisms underlying motor adaptation. For example, force field-based and visual rotation-based perturbations have yielded theories about the nature of feedback control (Todorov and Jordan, 2002; Wolpert et al., 2011), the learning rates (Gonzalez Castro et al., 2014), and the underlying anatomy and neural computations (Wolpert et al., 1995; Smith and Shadmehr, 2005; Shadmehr et al., 2010; Shadmehr and Krakauer, 2008). Studies using patients have been instrumental in identifying brain regions implicated in the ability to successfully adapt (Shadmehr and Krakauer, 2008; Smith and Shadmehr, 2005; Tseng et al., 2007). Yet, the exact neural mechanisms remain elusive as many patients have long-standing deficits often spanning multiple domains, such as control and learning (Shadmehr and Krakauer, 2008; Smith and Shadmehr, 2005; Tseng et al., 2007). Furthermore, the different types of error signals are often difficult to dissociate. Thus, the question of which error signals and neural systems contribute to motor adaptation remains debated (Adams et al., 2013; Shadmehr and Krakauer, 2008).

While much attention has been paid to the cerebellum in motor adaptation (Diedrichsen et al., 2005; Gao et al., 1996; Proville et al., 2014; Smith and Shadmehr, 2005; Tseng et al., 2007; Wolpert et al., 1998), there has been less focus on cortical contributions, despite evidence that cortex responds rapidly after the onset of perturbations (Krebs et al., 1998; Omrani et al., 2016). Interestingly, several studies have suggested that primary somatosensory cortex (S1) may have a role in motor learning (Krebs et al., 1998; Pavlides et al., 1993; Vidoni et al., 2010), yet a causal role of S1 in motor adaptation has not been demonstrated.

To address these questions, we developed a mouse model to study motor adaptation by applying force fields to alter paw trajectories. We first show that mice can learn to counteract this force field in this task. We subsequently show that S1 photoinhibition during perturbation abolished motor adaptation. This deficit in motor adaptation occurred with high specificity: S1 photoinhibition did not perturb basic motor patterns, post-force field completion of actions, or the ability of mice to learn from reward feedback. These results provide insights into neural circuits involved in motor adaptation and the type of error signal that drives motor adaptation.

RESULTS

Development of a Forelimb Motor Adaptation Task for Mice

We developed a joystick-based perturbation system for mice inspired by robotic manipulandum-based force field paradigms in humans (Shadmehr and Mussa-Ivaldi, 1994). Mice were trained to reach for, grab, and pull a joystick into a virtual target box (Figure 1A, Movie S1). The task was performed in the dark, therefore the joystick was invisible and their sensory feedback was limited to somatosensation from the forelimb. After the mice mastered the joystick-pulling behavior in an unperturbed environment, we introduced a brief lateral force perturbation using an electromagnet (Figure 1A). In perturbed trials, the force field was activated for the duration of 100 ms immediately after the joystick passed a fixed spatial threshold (Figure 1A) (Fine and Thoroughman, 2006; Sing et al., 2013). In each session (day), mice performed 75 trials without a force field (baseline), 100 trials with a force field (perturbation), and 75 trials without a force field (washout) (Figure 1A). During baseline, the mice had straight trajectories, producing only small perpendicular (i.e., lateral) paw displacements from the vertical pulling direction across the first 75 trials (0.01 ± 0.19 mm, mean ± SD, n = 7 mice, 27 sessions; Figures S1A–S1C). During initial trials of the perturbation phase, the unpredicted force field resulted in leftward perpendicular displacements by ~2 mm from the pull axis (these leftward displacements were denoted as negative values) (Figures 1B–1D, Figures S1A–S1C). After tens of trials, the mice compensated the force field, significantly reducing the perturbation-induced perpendicular displacement by 31.8% ± 0.73% (repeated-measures ANOVA, F_(9,13) = 9.79, p < 0.0001, Figures 1C–1D, Figures S1D and S1E, Figure S2).

Figure 1. Force Field-Based Motor Adaptation Task for Mice.

(A) Diagram of the joystick platform with electromagnet, top-down view with physical dimensions indicating position 1 and 2, start and reward box, and the adaptation task structure: 75 baseline trials, 100 perturbation trials, and 75 “washout” (i.e., null field) trials.

(B) Left: an example session of one mouse showing the average baseline trajectory (black), the first perturbation trial (sea green), average of first ten (green), and the last ten perturbation trials (purple), the first ten washout trials (orange) and the first washout trial (red). The numbers indicate trials. Right: average trajectories for all sessions (n = 27 sessions, from n = 7 mice) with temporally aligned averaging (see STAR Methods). Dashed sea green line is the average first perturbation, and the dashed red line is the average of the first five washout trials. Solid lines are average paw path ± SEM in the direction of the perpendicular deviation, with the same color scheme as the left panel; solid green is the first 25 trials during the perturbation epoch, solid purple is the last 25 trials of the perturbation epoch, and solid red line is the average of the first 25 washout trials.

(C) Average perpendicular displacement in position 2 displayed as mean ± SEM versus trials; n = 27 sessions, from n = 7 mice.

(D) Average perpendicular displacement within indicated trials. Adaptation was measured as a significant change from early to late perturbation trials, and as a significant after-effect during washout (compared to baseline, all washout bins are significant). RM ANOVA p < 0.0001. Post hoc Dunnett’s Multiple Comparison, *p < 0.01. Mean ± SEM.

(E) As in (C) for position 1 (before force onset), showing predictive steering. Light gray shows change in perpendicular displacement inside the start box.

(F) As in (D) for position 1; RM ANOVA p < 0.0001. Post hoc Dunnett’s Multiple Comparison, *p < 0.01. Mean ± SEM.

In addition to reducing motor errors mid-way through the pull, closer inspection of the paw’s paths showed that the mice gradually changed their pull angle near the start of the pull, before the onset of the force field; the mice also shifted their starting position within the start box (position 1 versus start box location compared across the late perturbation trials, Mann-Whitney U test, p < 0.0001, and position 1 normalized for shift in start box location early versus late perturbation, Mann-Whitney U test, p < 0.0001, Figures 1E–1F, Figures S2A and S2E). This pre-emptive steering suggests that the mice learned to expect the force field during the subsequent trial, and expressed compensatory motor commands, which can be confirmed by unexpectedly removing the force field. Indeed, the mice had a significant “aftereffect” during washout, that is, their pulls were significantly deviated in the direction opposing the force field (baseline versus washout bins, repeated-measures ANOVA, F_(7,9) = 11.70, p < 0.0001, Figures 1B–1F, Figure S2). Taken together, these results show that the mice adapted to the force field with predictive compensation. This is evidenced by: (1) an anticipatory change in perpendicular displacement before the force field onset, (2) reduced perpendicular displacement during the force field, and (3) the presence of reliable aftereffects when the force field was unpredictably removed.

Learning from Sensory and Reward Prediction Errors

In some circumstances (Huang et al., 2011; Izawa and Shadmehr, 2011), it has been shown that motor adaptation occurs by increasing the frequency of previously rewarded motor actions, that is, the result of reinforcement learning. We found that within a trial, following the offset of the perturbation, all the mice consistently corrected the joystick position by making an “L”-shape turn, as observed in humans (Shadmehr and Mussa-Ivaldi, 1994). Because of the spatial arrangement of the target and perturbed paw trajectories, these post-perturbation L-turns were enough to hit the target and mice were typically able to obtain reward even in the first perturbation trial they ever experienced (Figure S3A, 6/7 mice). As a result, reward rate remained high during the perturbation block but dropped slightly across the perturbation block in an anti-correlated manner with the reduction in perpendicular displacement (first ten versus last ten perturbation block, Wilcoxon signed rank test, p = 0.010; Spearman’s correlation coefficient: −0.39, p = 7.4 × 10⁻⁵; Figure 2A, Figure S3B). These errors are due to either not pulling far enough toward the reward box (50.3%), or aiming too laterally and/or not stopping long enough in the target box to collect a reward (49.7%; Figures S3B and S3C). The anti-correlation between the adaptation and the reward rate suggests that reward feedback did not positively instruct forelimb movements (adaptation). Consistent with this idea, a standard reinforcement-learning model failed to fit the behavioral data, suggesting that there is no consistent relationship between trial-to-trial changes in motor output to the trial-by-trial pattern of reward feedback (Figure 2B; STAR Methods). These observations suggest that the motor adaptation in our task is not primarily driven by reward prediction errors.

Figure 2. Model-Based Analysis Shows that the Adaptation Can Be Explained by Learning from Sensory Prediction Errors.

(A) Left: average perpendicular displacement (black) and average reward rate (cyan) across the perturbation block (n = 27 sessions, from n = 7 mice). Right: reward rate during indicated trials, mean ± SEM. *p = 0.01 Wilcoxon signed rank test (first ten versus last ten perturbation block).

(B) Best model fit using an actor-critic reinforcement-learning model, demonstrating a poor fit. Data from perpendicular displacement (at position 2) is shown in black (as in Figure 1C). Average best model fit is shown in red. As this model has several sources of noise (see STAR Methods), ten realizations of the model with best parameters are shown in soft lines and the solid line shows the average. Only the perturbation trials are used for fitting.

(C) Schematic of the state estimator model: the efference copy of a motor command is used to generate predicted sensory feedback (internal model), which is contrasted with the actual feedback (SPE), then weighted by a Kalman gain, to then update the state estimate to drive future actions (see STAR Methods).

(D) As in (B) but for the optimal state estimator model.

(E) As in (B) but for hybrid model of an actor-critic reinforcement-learning model and a state estimator model.

(F) The hybrid model’s motor command consists of three components derived from the optimal state estimator (Kalman filter and SPE), the optimal policy estimated by reinforcement-learning (RPE), and exploratory search noise. Left: the magnitude of the first two components corresponding to command signal (orange, RPE-based; purple, SPE-based). Right: quantification of the relative power ratio of contribution of those control signals from the best model fit.

Another class of models posit that motor adaptation relies on sensory prediction errors (Berniker and Kording, 2008; Izawa and Shadmehr, 2011; Shadmehr and Mussa-Ivaldi, 1994; Todorov and Jordan, 2002; Wolpert et al., 1995). In these models, a mismatch between the predicted sensory feedback (based on the efference copy) and the actual sensory feedback indicates the presence of a perturbation to the motor system (Figure 2C). The vector of this error signal can be used as a “teaching signal” to learn the magnitude of the perturbation, and to update a memory (i.e., internal model) of the perturbation. Once this internal model is learned, the agent executes a motor command to counter the estimated perturbation. We found that, indeed, such a state estimator model was able to explain the observed pattern of motor adaptation (Figure 2D). To contrast reward-based and sensory prediction error-based learning, we further studied a hybrid model that could learn from both sensory and reward prediction errors (Todorov and Jordan, 2002; Izawa and Shadmehr, 2011) (Figure 2E, STAR Methods). Consistent with the individual models, we found a dominating role for sensory over reward prediction errors (Figure 2F).

Somatosensory Cortex Is Required to Adapt to a Forelimb Perturbation

The above results suggest that the integration of sensory feedback (either L-turns, or in the form of sensory prediction errors) from the forelimb, but not reward, plays an important role in the motor adaptation we studied. Somatosensory feedback is conveyed from the limb to the central nervous system (CNS) by multiple pathways including the direct projections from the spinal cord to the cerebellum (spino-cerebellar and cuneo-cerebellar pathways) as well as the thalamocortical pathways (Bostan and Strick, 2010; Caligiore et al., 2017). It is, therefore, unclear whether one particular pathway plays a predominant role in motor adaptation. S1 receives feedback from the limb during reaching movements (Hikosaka et al., 1985; Omrani et al., 2016) and projects to important motor control centers such as motor cortex (Petrof et al., 2015) and via brainstem to cerebellum (Bower et al., 1981)—both areas that display rapid responses to S1 stimulation (Brown and Bower, 2002; Petrof et al., 2015). To test the role of S1 during adaptation, we transiently photoinhibited the forelimb area of S1 during movements.

We employed transgenic mice expressing channelrhodopsin-2 (ChR2) in GABAergic neurons, implanted with an optical fiber in forelimb area of S1 contralateral to the forelimb they used during the task (Guo et al., 2014; Zhao et al., 2011, Figure S4, STAR Methods). We then optogenetically activated GABAergic interneurons during the period when the force field was applied to partially inactivate forelimb S1 (n = 3 mice, 17 sessions, Figure 3A). During control sessions (n = 12), the mice significantly adapted to the force field, demonstrated pre-emptive steering, and displayed a significant aftereffect (Figures 3B, 3D, and 3E, Figure S5A; compare to Figures 1C and 1D). In contrast, with S1 photoinhibition the same mice did not exhibit motor adaption as indicated by (1) the lack of reduction in perpendicular deviations (Figures 3C–3D and 3F, Figures S5B–S5D), (2) the lack of predictive steering before force field onset (Figure S5D), and (3) the lack of aftereffect during the washout block (Figures 3C–3D and 3F, Figures S5A and S5B). Specifically, the perpendicular displacement in early trials between control and photoinhibition session were not significantly different (Mann-Whitney U test, p = 0.33). With photoinhibition, there was no difference in perpendicular displacement between the early and late trials during the perturbation block (Mann-Whitney U test, p = 0.50), yet the last ten trials of the perturbation block during photoinhibition and control sessions were significantly different (Mann-Whitney U test, p = 0.001 Figure 3D). The lack of adaptation and aftereffect was not due to general performance deficits (Movies S2 and S3, Figures S5E–S5G). They were still able to perform L-shaped turns and hit the target box, resulting in similar reward rates as in control sessions (62% ± 10% during photoinhibition; 64% ± 13% without; t test, t₍₂₇₎, = 0.12, p = 0.90, Figures 3B and 3C). During photoinhibition sessions, the extent they moved after the termination of the force was slightly longer (~0.15 mm) rather than shorter (Figure S5E). We also obtained no signatures of adaptation when we repeated the same photoinhibition experiments with a lower laser power (Figure S5H). Furthermore, the velocity profile of pulls with or without photoinhibition was comparable between the baseline and early perturbation trials (Figure S6A).

Figure 3. Photoinhibition of Forelimb Somatosensory Cortex Abolishes Adaptation but Not Reward-Based Learning.

(A) Task design: ChR2 was expressed in GABA neurons. The applied neuronal perturbation was achieved by photo-activating GABA neurons through an optical fiber. The adaptation task was performed in three vGAT-ChR2 mice. During perturbation trials, the laser was on concurrently with the force field (force field, 100 ms duration; laser at 50 Hz, 100 ms duration).

(B) Temporally averaged trajectories across all control sessions (n = 12 sessions from 3 mice), coloring as in Figure 1B.

(C) Temporally averaged trajectories across all photoinhibition sessions (n = 17 sessions, across the same 3 mice), coloring as in Figure 1B.

(D) Averaged perpendicular displacement at position 2 across sessions, with (blue) and without (black) photoinhibition (smoothed mean [5 trials] ± SEM shown for visualization), **p = 0.001, (difference in the last ten perturbation trials, t test), ***p < 0.0001, (difference in the first ten washout trials, t test).

(E) Quantification of all the control sessions, *p < 0.05, **p < 0.01, (t test). Mean ± SEM.

(F) Quantification of all the photoinhibition sessions. Mean ± SEM.

(G) Behavioral performance across all sessions in the target-shift task with (blue) or without (black) photoinhibition. All sessions averaged at position 2; there was no significant difference with or without photoinhibition, p = 0.57 (two-way ANOVA). Mean ± SEM.

(H) Quantification of all the control sessions. *p < 0.05, **p < 0.01, ***p < 0.001 (t test). Mean ± SEM.

(I) Quantification of all the photoinhibition sessions, **p < 0.001 (t test). Mean ± SEM.

L-Turns Are Intact with S1 Photoinhibition

According to feedback error learning theory (Kawato and Gomi, 1992; Shadmehr et al., 2010), the act of making L-turns (or the more active act of making motor corrections) guides adaptation. The above results indicate that the ability to counteract the force field by means of online motor control as well as basic kinematic variables was not impaired by S1 photoinhibition. To further test whether S1 photoinhibition affects reflexive motor reactions (i.e., the L-turns), we sought to specifically examine motor control. During the adaptation task, the L-shape turns occurred immediately after the force field was terminated, suggesting that the feedback responses underlying the lateral movement began even before the 100-ms perturbation was complete (allowing the L-shaped movement to take place as soon as the perturbing force was removed). To determine the onset latency of these feedback responses, mice were tested in separate sessions in which force field perturbations of varying durations (drawn from a uniform distribution, range: 10–100 ms) were applied randomly in 50% of trials. We found that during shorter applications of the force field (i.e., 30 ms), the mice made L-turns quickly (Figure S6B). This suggests that the mice are capable of reacting by the end of the force field when it is applied for 100 ms, as in the main adaptation task. They also appeared to correct with a similar magnitude independent of the duration the force field, suggesting that the compensation is not specific to the state of the limb (Fine and Thoroughman, 2006). These results furthermore suggested that the capacity for online correction was not affected by S1 photoinhibition. To test this further, we examined mice in sessions where the force field of varying durations (30, 60, or 90 ms) was applied in a smaller (10%) fraction of randomly chosen trials in order to reduce the effect of adaptation. We again photoinhibited for a fixed duration (100 ms) during 50% of these perturbation trials (i.e., the force field could be active for 30 ms, but photoinhibition was always 100 ms). Consistent with the earlier experiment, mice made the L-shape turns during photoinhibition and the “time to turn” was not significantly different with or without photoinhibition (two-way ANOVA, F_{(2, 189)} = 0.67, p = 0.53 interaction; F_{(1, 189)} = 0.02, p = 0.88, condition; F(_{2, 189)} = 1.4, p = 0.25 force on time; Figure S6C) and had almost identical velocity profiles (Figure S6D). Taken together, these results show that the ability to make an L-turn within 100 ms was not impaired by photoinhibition, demonstrating that motor adaptation can be dissociated from corrective motor control, and suggest that L-turns are not sufficient to drive adaptation in our task.

Learning to Move the Paw to a New Reward Location Is Not Dependent on S1

We next sought to test whether photoinhibition impaired the ability to learn a new reward location. The same mice were tested in a task requiring similar forelimb movements but the location of the reward box was shifted laterally (by 80% of the box width) during the 100 trial “perturbation” block (n = 3 mice: n = 12 control and 13 photoinhibition sessions, Figure 3G). The mice gradually changed their pull direction toward the novel reward location in control sessions (Figure 3G). This behavior was well fit by the actor-critic reinforcement-learning model, but not by the model that learns from sensory prediction errors (Figures S7A and S7B), in stark contrast to the adaptation task. We then applied the same spatiotemporal photoinhibition schedule during this “target-shift task.” We found that their ability to learn in this task with and without photoinhibition was comparable (Figures 3G–3I, Figure S7C, two-way ANOVA F_{(1, 115)} = 0.83, p = 0.37). The reward rate was not different between control and photoinhibition conditions (44% ± 12% reward rate during photoinhibition, 56% ± 12% without; t test, t₍₂₃₎, = 0.71, p = 0.49). These results demonstrate that the same S1 photoinhibition that impaired motor adaptation did not impair their ability to learn in motor learning task based on reward feedback that involves very similar motor patterns as the motor adaptation task. This result further supports the idea that the deficit in motor adaptation during photoinhibition is not due to impairment in reward-based learning.

The Role of S1 in Updating an Internal Model of the Environment

How could S1 be involved in motor adaptation? The above results suggested that sensory prediction errors, and not a motor correction-based error signal or reward prediction errors, drove the adaptation in our task. We reasoned that S1 could be housing a memory of the perturbation (i.e., the internal model of the perturbation), or updating the memory of the perturbation (e.g., computing sensory prediction errors). The former possibility indicates that S1 photoinhibition eliminated the internal model of the force field. The latter possibility indicates that S1 photoinhibition impaired the updating of a motor command that counters the expected perturbations, for example, because sensory prediction errors were eliminated. The above experiments do not allow us to distinguish these possibilities. We reasoned that once the mice partially adapted, these two hypotheses make different predictions. To dissociate these two alternatives, we used a task variant in which, after 75 baseline trials, the mice were presented with 50 perturbation trials, followed by 50 perturbation trials paired with photoinhibition (n = 17 sessions, Figure 4A). The former possibility predicts that S1 photoinhibition should cause a sudden drop back to the full perpendicular deviation measured during initial perturbation trials (Figure 4B). Alternatively, the latter possibility predicts that S1 photoinhibition should only halt updating of a motor command, which would not result in a sudden change in motor command at the transition from un-inhibited to the photoinhibited trials (Figure 4B). The mice exhibited adaptation within 50 trials (Mann-Whitney test, early versus mid-perturbation, p = 0.01, Figure 4C). We found that S1 photoinhibition following this adaptation did not cause an immediate drop in perpendicular displacements (average trial 120–125 versus 126–131, Mann-Whitney test, p = 0.24, Figures 4C and 4D, and trial 125 versus 126 each session, paired t test, t₍₁₆₎, = 0.23, p = 0.82, Figure 4D), which was also true of the predictive steering (Figures 4E and 4F). Over the course of the next few trials with S1 photoinhibition, however, the perpendicular displacement gradually decayed (Figures 4C and 4E, Figure S7D). This result suggests that the S1 photoinhibition impaired updating of the memory of perturbations, but not the expression of already-adapted motor commands based on this memory, which is presumably stored elsewhere in a labile way.

Figure 4. Somatosensory Cortex Is Required for Updating Motor Commands during Adaptation but Not for the Expression of the Adapted Motor Command.

(A) Task structure. During the perturbation block (trials 76–175), photoinhibition was applied only in the second half of trials (126–175).

(B) Model prediction. The best-fit parameters from the control data (Figure 3D) were used. At trial 126, SPE was clamped to zero (cyan) or the perturbation estimate, ( $\widehat{p_k}$) was set to zero (dark blue).

(C) Session average from position 2, showing adaptation up to trial 125 (Mann-Whitney U test, trials 76–85 versus trials 116–125, p = 0.01).

(D) Top: average perpendicular displacement plotted per session of trials 120–125 versus 126–131 (Mann-Whitney U test, p = 0.24). Bottom: by session displacement value for trials early perturbation (P_e, trials 75–80), 124, 125, 126, 127, and late perturbation (P_l, trials 170–175) showing there is no systematic drop from trial 125 to trial 126 (paired t test p = 0.82).

(E) Perpendicular displacement across blocks in position bin 1 (spatially above force onset).

(F) Top: average perpendicular displacement of trials 120–125 versus 126–131. Mann-Whitney U test, p = 0.05. Bottom: same as in (D), showing there is no systematic drop from trial 125 to trial 126 (paired t test, p = 0.73).

DISCUSSION

Our results demonstrate that mice exhibit motor adaptations in forelimb movement that bears similarities to motor adaptations observed in humans (Fine and Thoroughman, 2006; Shadmehr and Mussa-Ivaldi, 1994; Sing et al., 2013) and non-human primates (Li et al., 2001). We used this behavioral paradigm to explore two long-standing questions: which error signals guide motor adaptation, and which neural circuits are required to adapt? Our results demonstrate that S1 photoinhibition impaired updating motor commands but not the execution of already-adapted motor commands. These results indicate that direct projections from the spinal cord to the cerebellum are not sufficient to support motor adaptations in our forelimb motor adaptation task. Instead, the result shows that cortical processing of somatosensory feedback plays a predominant role. Our S1 photoinhibition impaired motor adaptation but the L-turns remained intact. This suggests that the act of making the L-turn (which is thought to produce a “template” [Albert and Shadmehr, 2016]) is not sufficient to drive motor adaptation in this task. Moreover, a model that learns from sensory prediction errors, but not one that learns only from reward prediction errors, was able to explain the learning dynamics of motor adaptation. Furthermore, reward-based learning of a novel target location was not impaired by the S1 photoinhibition. Together, these results suggest that processing of sensory feedback via S1 plays an important role in detecting the discrepancy between predicted and actual sensory experiences, which plays an important part in driving motor adaptation.

Motor Perturbations in Mice as a Tool to Study Sensory-Guided Adaptation

We set out to establish a mouse model to study a forelimb motor adaptation task that has played a pivotal role in motor adaptation studies in humans and non-human primates (Shadmehr and Mussa-Ivaldi, 1994; Fine and Thoroughman, 2006; Sing et al., 2013; Shadmehr et al., 2010). We showed that over the course of 100 trials, the mice began to counter the force field perturbation as measured by the appearance of predictively steering, reduced displacement, and an aftereffect. The predictive steering (and aftereffect) is particularly interesting because it indicates that the mice made a preparatory compensation to an upcoming perturbation, and therefore the mice learned from previous trials.

Although the mice exhibited adaptation similar to those seen in primates, they did not adapt fully back to a straight pull within 100 trials. In most human studies, velocity-dependent force fields are used. In these paradigms, based on the bell-shaped velocity profile of pulls, the force field is gradually increased and then decreased, which may have aided complete adaptations. Even in these conditions, it typically takes more than a hundred trials to fully adapt (Shadmehr and Mussa-Ivaldi, 1994; Wolpert et al., 2011). Nonetheless, incomplete adaptation was also commonly seen in humans (Fine and Thoroughman, 2006; Rabe et al., 2009; Sing et al., 2013; Tseng et al., 2007), suggesting that whether complete adaptations occur depends on various task parameters. For example, when abrupt, pulsed forces were used, humans did not pull straight after more than 200 trials (Fine and Thoroughman, 2006; Sing et al., 2013). Furthermore, using pulsed perturbations, a previous study has suggested that the subject tends to adapt to the direction of perturbation, but not to the detailed limb states during perturbation (Fine and Thoroughman, 2006). How different task parameters affect motor adaptations in our task warrants further investigation.

The measured adaptation was accompanied by a slight decay in the reward rate. This anti-correlation between motor adaptation and reward allowed us to dissociate motor adaptation from reward-based learning. Although the reduced perpendicular deviation is indicative of motor adaptation, the adaptation could be called “maladaptive” with respect to obtaining reward. Several studies have demonstrated that adaptation can occur independent of the “rate” of success in a given task condition (Mazzoni and Krakauer, 2006; Schaefer et al., 2012). For instance, a previous study described a remarkable visuomotor learning task where implicit and explicit processes guiding adaptation were in conflict (Mazzoni and Krakauer, 2006). When subjects “implicitly” adapted, this was accompanied by decay in performance (“drift” or systematic error in reaching a target). These observations suggest that sensory prediction error-driven adaptations can be implicit and can be dissociated from a given goal of a task such as reward or hitting a target.

The Role of Error-Based Learning during Adaptation

We observed L-shape turns in response to force field perturbations similar to those observed in primates (Fine and Thoroughman, 2006; Shadmehr and Mussa-Ivaldi, 1994). These observations suggest that mice quickly counteracted the perturbation. We do not believe that these L-turns reactions are pre-planned because they were observed even when the mice first encountered the perturbation and because similarly oriented trajectories were not observed in washout trials (Figure S1A). Moreover, the mice could be employing both a heuristic strategy or co-contracting muscles (i.e., stiffing in response to the force), which aid both in reducing displacement, and could cause the sudden L-turn once the magnet releases the limb. Furthermore, mice exhibited similar reactions regardless of the duration of perturbations (Figure S6). The mechanism underlying these corrective L-turns remains to be clarified. Because of this reflex-like nature, they could be purely subcortical or spinal. However, we cannot exclude the possibility that the L-turns involve other cortical areas. Furthermore, more complete S1 inactivation might interfere with kinematics or the production of the L-turn.

It has been proposed that the act of making an L-turn (or a more active corrective response) serves as a teaching signal (or a “template”) that drives motor adaptation (Albert and Shadmehr, 2016; Inoue et al., 2016; Kawato, 1996; Thoroughman and Shadmehr, 1999). Consistent with this idea, patterns of muscle activation during corrective motor responses gradually shift to an earlier time point during future movements, providing some evidence supporting its role in motor adaptation (Albert and Shadmehr, 2016; Thoroughman and Shadmehr, 1999). Furthermore, electrical stimulation of the primary motor cortex (M1) that mimicked corrective reactions facilitated motor adaptation (Inoue et al., 2016). However, corrective motor reactions were shown to not aid saccade adaptations in certain conditions (Wallman and Fuchs, 1998), nor did they enhance reaching adaptation in patients with hereditary cerebellar ataxia (Tseng et al., 2007). The result that S1 photoinhibition abolished motor adaptation without compromising the act of making an L-turn indicates that the corrective reaction is not sufficient to drive motor adaptation. However, we cannot exclude the possibility that S1 contributes to error feedback learning, for instance, by relaying motor error signals to other brain regions for learning.

Additionally, it is possible that the mice employed some “cognitive” strategy to aid in compensating the force field. Previous studies have indicated the presence of multiple processes that guide adaptations (Smith et al., 2006; Shadmehr et al., 2010). Moreover, it has recently been proposed that in addition to the “implicit” error-based processes that may update a forward model, an “explicit” process can contribute to the measured adaptation in humans (Mazzoni and Krakauer, 2006; McDougle et al., 2015; Taylor and Ivry, 2012; Taylor et al., 2014). In our task, we characterized motor adaptation based on three metrics of adaptation: predictive steering, reduction in displacement, and aftereffect (Figure 1B, Figure S1A). In addition to these changes, mice also exhibited a small shift rightward in the start box before initiating a pull (Figure 1E). This appears to suggest that the mice employed more than sensory prediction error-guided adaptation. That is, explicit understanding of the force field perturbation might have instructed the animal to counter even before initiating a pull. The ability to apply a cognitive strategy may also have been aided by using perturbations whose profile is not dependent on limb kinematics, as is seen in human studies in which a similar pulsed field resulted in predictive steering (Fine and Thoroughman, 2006). Thus, while our data suggests that S1 is crucial to adapt, it remains unclear how and which specific downstream “systems” S1 may influence.

How Is S1 Involved in Regulating Motor Adaptation?

In total, our results favor that S1 is involved in sensory prediction error-driven motor adaptation. There are multiple possibilities of how our S1 photoinhibition abolished motor adaptation. First, the lack of adaptation can be explained if the photoinhibition eliminated representation and transmission of the sensory prediction errors that update an internal model, which then drives motor adaptation. Second, it is also possible that the CNS learned to ignore sensory feedback due to reduced fidelity in somatosensory feedback signals caused by S1 photoinhibition. Consistent with the former possibility, S1 neurons rapidly respond to unexpected perturbations to a forelimb movement (Fromm and Evarts, 1982; Omrani et al., 2016) and respond differently to externally versus self-generated touching (Blakemore et al., 1999). Although the question of whether these signals encode sensory prediction errors, and if they specifically drive motor adaptation remains to be examined. Another possibility is that S1 relays rather than computes sensory prediction errors to areas that may house the internal model, such as the motor cortex or cerebellum, which are proposed sites of inverse and forward models, respectively. Yet, the abolishment of adaptation by S1 photoinhibition, both at the start of the perturbation block and after partial adaptation (Figures 3 and 4), suggest that S1 is required because it may encode or compute sensory prediction errors.

Previous studies have shown that S1 participates in functional loops with the motor cortex, the basal ganglia, and the cerebellum (Blakemore et al., 1999; Kelly and Strick, 2003; Marini et al., 1982; Petrof et al., 2015), which are thought to be the sites that hold memories about perturbations (often referred to as inverse or forward internal models). When we photoinhibited S1 after partial adaptation (Figure 4), we found that the memory was not immediately (or fully) erased, suggesting that S1 does not exclusively hold the memory of the perturbation. These results are consistent with the idea that S1 is providing the updating signal (such as sensory prediction errors) to another region that houses the internal model. For example, recent work suggests that the cerebellum may contribute to the coordination and adaptation of limb movements (Hoogland et al., 2015; Vinueza Veloz et al., 2015), and thus may house an internal model that could be updated by S1.

The traditional view is that S1 holds multiple sensory representation of the body (i.e., homunculus) (Penfield and Boldrey, 1937; Kaas et al., 1979; Kaas, 1983). It is known that lesions during development alter the cytoarchitectural and functional realization of these bodily representations (Buonomano and Merzenich, 1998; Feldman and Brecht, 2005; Woolsey and Van der Loos, 1970). Furthermore, there is evidence that the representation of areas recently used in motor tasks can shift the map size and location (Braun et al., 2000) and is plastic on the order of minutes (Feldman and Brecht, 2005). Consistent with this view, it has been demonstrated that S1 is critically involved in the perception of object location (Guo et al., 2014) and, together with adjacent somatosensory areas, is involved in perception by active touch (Tomberg and Desmedt, 1999). Thus, it is interesting that in this forelimb-reaching task, our S1 photoinhibition did not hamper motor performance and control (neither in the target-shift task nor in the motor adaptation task was the performance affected by inactivating S1). As the target-shift task involves learning a novel target location, this might seem paradoxical. Yet, in principle, the new location could be learned by reinforcing the actions that were rewarded rather than by associations with explicit sensory cues, which is also supported by our model analysis (Figures 2 and S7). In the motor adaption task, the ability of mice to counteract the novel forces applied by the magnet was also not affected. The accurate perception of those forces, or the calculation of sensory prediction errors as indicated above, however, is crucial for guiding adaption. Thus, photoinhibition of the forelimb region of S1 during a skilled motor behavior suggests that S1 is involved in updating an internal model and that without the flow of information from S1 this memory can decay in a few trials (Figures 4 and S7D).

Conclusions

Although there have been extensive studies addressing the potential mechanisms underlying motor adaptation, the neural circuits and the error signal that regulate it remained elusive. Our behavioral paradigms in mice together with transient S1 photoinhibition allowed us to dissociate sensory prediction errors, template-based learning, and reward feedback. This experimental paradigm will be a powerful platform to probe the contributions of different error signals as well as neural circuits underlying motor adaptation.

STAR ★ METHODS

CONTACT FOR REAGENT AND RESOURCE SHARING

Further information and requests for resources and reagents should be directed to, and will be fulfilled by, the lead contact Mackenzie Weygandt Mathis (mackenzie@post.harvard.edu).

EXPERIMENTAL MODEL AND SUBJECT DETAILS

The behavioral data were collected from 7 adult (P90-P365) male mice whose genotypes are either (1) C57BL/6J [n = 2], (2) Slc6a3^{tm1.1(cre)Bkmn}/J [n = 2], or (3) B6.Cg-Tg(Slc32a1-COP4*H134R/EYFP)8Gfng/J [n = 3; called vGAT-ChR2 in this report]. The same set of vGAT-ChR2 mice were used for photoinhibition experiments. These mice were obtained from the Jackson Laboratory (see Resources Table). For the photo-bleaching assay (Figure S4), we used B6.Cg-Tg(Thy1-YFP)HJrs/J mice (n = 3, Jackson Laboratory). For the electrophysiological confirmation of inhibition (Figure S4), we used 5 additional vGAT-ChR2 mice. All mice were housed on a 12h dark/12h light cycle (dark from 06:00–18:00) and each performed the behavioral task at the same time of day, between 07:00 and 19:00. After surgery they were individually housed. All mice were maintained above 90% of their initial weight, and no mouse was given less than 1.5 mL water per day. All surgical and experimental procedures were in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals and approved by the Harvard Institutional Animal Care and Use Committee. A subset of mice (n = 2) in this study was first tested in a similar adaptation task (unpublished data) prior to participating in this study (but in no more than 8 sessions).

KEY RESOURCES TABLE.

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Experimental Models: Organisms/Strains
Mouse: C57BL/6J	The Jackson Laboratory	Jax # 000664
Mouse: B6.Cg-Tg(Slc32a1COP4*H134R/EYFP)8Gfng/J	The Jackson Laboratory	Jax # 014548
Mouse: B6.SJL-Slc6a3tm1.1(cre)Bkmn/J	The Jackson Laboratory	Jax # 006660
Mouse: B6.Cg-Tg(Thy1-YFP)HJrs/J	The Jackson Laboratory	Jax # 003782
Software and Algorithms
MATLAB (versions used: 2012b 2014a)	Mathworks	http://www.mathworks.com
Python	Python	https://www.python.org/
Prism 5	Graph Pad Software	http://www.graphpad.com
LabView 2013	National Instruments	http://www.ni.com
MClust software	A. David Redish	http://redishlab.neuroscience.umn.edu/MClust/MClust.html
Other
Isoflurane (for anesthesia) IsoSol	Vedco	N/A
Ketoprofen (for analgesia)	Patterson Veterinary	Cat #07-803-7389
Buprenorphine	Patterson Veterinary	Cat #07-850-2280
Dexamethasone	Patterson Veterinary	Cat #07-808-8194
Joystick base	Digi-Key	Cat #679-2501-ND
Solenoid magnet	McMaster-Carr	Cat #69905K25
FlexiForce System: ELF 2 System	Tekscan	Cat #A00675
Mouse platform grip: thin rubber	McMaster-Carr	Cat #8633K52
NI-DAQ card, PCI-e6251	National Instruments	Cat # NI PCIe-6251 – 779512-01
Laser for optogenetics: Opto Engine 473 nm −100 mW	Opto Engine	Cat # MBL-III473/1~100 mW
Fine tetrode wire (nichrome)	Sandvik Kanthal	Cat # PF000591
Digital Lynx 4SX Recording System	Neuralynx	N/A
Open Ephys Acquisition Board and Intan headstage	Open Ephys	http://www.open-ephys.org/store/starter-kit
NeuroNexus 8 shank probe (with tetrode configuration)	NeuroNexus	N/A

METHOD DETAILS

Surgery

Mice were surgically implanted with a custom head plate (Cohen et al., 2012). All surgery was performed under aseptic conditions with animals under isoflurane (1%–2% at 1.0 L/min) anesthesia. Analgesia (ketoprofen, 5 mg/kg, I.P.; buprenorphine, 0.1 mg/kg, I.P.), and an anti-inflammatory (dexamethasone, 5 mg/kg, S.Q.) were administered postoperatively for 48 hr. For the optogenetic experiments, mice were first outfitted with a head plate to determine their preferred paw then they underwent a second surgery to have a 200 μm diameter optical fiber (Thorlabs, 0.39 NA) implanted at 200–400 μm below the dura. The coordinates of the forelimb somatosensory cortex were based on the intrinsic imaging mapping (Sachidhanandam et al., 2013; implant coordinates: −0.1 to +0.1 Bregma, +2 mm lateral (for every mouse), personal communication with Carl Petersen), and at least 1.5 mm away from the forelimb region of motor cortex.

Behavioral apparatus

The mice manipulated a joystick (thumb stick) to obtain reward. The system consists of a handle attached to a joystick base (679-2501-ND, Digi-Key, MN, USA). We modified the joystick base to reduce the force required to move the joystick (0.1–0.2 N from 4N). The joystick had 2 dimensions of freedom, allowing for 360° movements (Figures S8A and S8B). The handle is made of a cylindrical wooden rod of 2 mm diameter at the grip point (and an aluminum base with a small steel ring to interact with the electromagnet). The joystick system was positioned on a separate platform below the main stage where the mouse sat, so it could be adjusted to accommodate each mouse’s reaching distance, if needed. Distance from the mouse with full limb extension to grasp the joystick was ~10.5 mm from the front edge of the platform to the resting, home location of the joystick (vertical). Custom aluminum holders were made to hold a tubular linear solenoid magnet orthogonal to the main pulling axis (74 oz Force, 69905K25, McMaster-Carr, USA). The magnet was hardware triggered for high temporal accuracy (2.2 V applied for 100 ms unless otherwise stated). The system was controlled in a fully closed-loop fashion in-house written software in LabView (National Instruments). We measured the relative force of the joystick during force trials by moving the joystick through a walled-channel, where one side was outfitted with a force sensor (FlexiForce System, Tekscan). We found that relative force on the side of the channel was ~1.5 times the typical pressure exerted on the sidewall. We also measured the effect on the joystick without a mouse by pushing the joystick along the pull axis until it interacted with the magnet. The return trajectories are shown in Figure S8C (with mouse-imposed trajectories overlaid). Additionally, the L-turns are not purely an effect of the joystick’s spring mechanics that exerts a force toward the resting position. This can be deduced from the existence of L-turns away from the magnet even on the positive perpendicular displacement side of the midline pull axis (half-plane), where the spring’s lateral force component aligns with the magnet’s force field direction (Figure S8D).

The mouse’s head plate fit to custom-made holders that reliably positioned the mouse across days. They stood on a solid platform covered in thin rubber for extra grip (8633K52, McMaster-Carr, USA) with no additional cover. Their front limbs could rest comfortably on a slightly elevated (~6 mm), rubber covered rest (Figure 1A). The analog voltage signal from the joystick was recorded in LabView from a National Instruments card (PCI-e6251). We measured the dynamics of the pull using the voltage readout from the joystick (sampled at 10,000 Hz for accurate closed-loop control and down-sampled to 1,000 Hz for analysis).

Behavioral training

After the mice had recovered for more than 7 days post surgery, they were water restricted in their home cage. On the first day, mice were head fixed briefly (as long as could be tolerated, but never longer than 15 min) and given random water reward to encourage licking behavior. The next 3 – 5 days, they were encouraged to grab the joystick, and instantly rewarded if they touched it or pulled it. After the mice had learned to reach for the joystick and move it for reward, a reward box was delineated; here, the mouse was required to allow the joystick to be in the virtual start box, wait > 1 s, then pull ~3mm to a virtual “reward training box” (2 mm width, 6 mm depth) toward the platform, and maintain joystick position for 100 ms inside the box. The behavioral performance was quantified as percentage of pulls to the reward zone per attempted pulls (i.e., reward given). Once a mouse performed with > 60% accuracy for > 2 days, they were shifted to the “Target Box Task” (which is the “baseline” in the main adaptation task), where they had to pull ~7 mm into an invisible box (2 mm in width, 3 mm depth). The mice continued to train (at least 2 weeks, but typically 4–5 weeks) until they were placed into the adaptation task, by which point their reward rate was > 80%. The adaptation task had the same rules as the Target Box Task except the wait time in the target box was 50 ms versus 100 ms, and the session consisted of 75 baseline trials, 100 force field trials, and then 75 washout trials; see Figure 1. This criterion was also used for the Target Shift Task (Figure 3).

Additional motor tasks

For the data collected to determine reaction times to the force field, we tested the mice in separate sessions on a task where they performed 50 baseline trials, followed by a random sequence of perturbation trials (50% probability of occurrence), in which the force field time was drawn from a uniform distribution with range 10–100 ms. For testing L-turns in the optogenetic context, we allowed the mice to do 50 baseline trials followed by a random sequence of perturbation trials (with only a 10% probability of occurrence to limit any adaptation), in which the force was 30, 60 or 90 ms in duration. Then, in 50% of those 10% of perturbation trials, we applied the photoinhibition for 100 ms, starting concurrently with the force field onset. These experiments were performed with the room light on, and an additional blue LED placed close to the mouse’s face for masking.

Optogenetic photoinhibition

Mice were implanted with a optical fiber (as described above) to be sure we target the same region across sessions and to more evenly target the upper and deeper layers (Figure S4). Each mouse also had an aluminum and plastic cone that covered the optical fiber, limiting the light leakage from the laser when attached to the fiber (and protected the fiber from damage; Laser, Opto Engine LLC, 473 nm–100 mW). These experiments were performed with the room light on and an additional blue LED placed close to the mouse’s face for masking. There was no apparent distress or change in behavior caused by the light in any session. The laser power produced ~4 – 9 mW of 473 nm light, as measured through the tip of the optical fiber before implantation, and it was delivered in 5ms pulses at 50 Hz frequency time locked to the force field, or time-locked to the start of the force field (but with a constant duration of 100 ms) when testing various force on lengths (i.e., motor correction tasks). This was performed only during perturbation trials. For the “target shift task,” the light was triggered at the same spatial location and applied in the same manner (but no force was applied).

Photo-bleaching assay

To measure the amount of light spread we performed a photo-bleaching assay as described (Guo et al., 2014). Specifically, we implanted three Thy1-YFP (Line H) with an optical fiber and head plate in the same manner as described above. The mouse was habituated to being head-fixed in a behavior apparatus for > 5 days prior to the photo-bleaching assay. We applied 20 mW of light for 600 s to the area while the mouse quietly rested head-fixed not performing any task. We then immediately transcardially perfused the mice with cold 1x PBS followed by ice cold 4% PFA. Brains were post-fixed in 4% PFA for 24 hr, and then cut into 100 μm sections for analysis. Images of each section were collected on an upright epifluorescent microscope (Zeiss). The extent of photo-bleaching was computed as the mean of the maximum distance across the photo-bleached areas in both the anterior to posterior and medial to lateral axis (Figure S4). To compute the upper and lower layer bounds, we took the sections closest to the fiber tracts in each of the three mice and computed the maximum spread in the medial-lateral axis in 13 spatial bins (from layer 1 to 6).

We additionally measured the amount of inhibition by chronically implanting S1 in vGAT-ChR2 mice (n = 4) with a customized optotetrode (four nichrome wires wound together, Sandvik Kanthal, Palm Coast, Florida); recordings were performed as previously described (Cohen et al., 2015) while the mice were awake but not performing a task. Additionally, we acutely implanted S1 in an anesthetized vGAT-ChR2 mouse with a NeuroNexus 8 shank probe (with tetrode configurations) as previously described (Guo et al., 2014, 2015; Li et al., 2016; Zhao et al., 2011)). The fiber-optic was ~500 μm away from recorded neurons. In brief, optotetrode data was collected using a DigiLynx recording system (Neuralynx, Bozeman, Montana), and probe recording was collected using an OpenEphys system (Cambridge, MA). Signals from each wire were filtered between 0.1 and 9000 Hz and recorded at 32 kHz. To extract the timing of spikes, signals were band pass-filtered between 300 and 6000 Hz. Sorting of individual units was performed offline using MClust 4.4 (D. Redish), and a PSTH (10 ms bins) was computed for each isolated unit, triggered on the first pulse of the 50 Hz laser application (Figure S4). Putative GABA neurons had a spike width less than <0.35 ms (time from peak to trough, as in (Guo et al., 2014, 2015).

Trajectory Analysis

Since general motivation of mice was important, sessions in which the reward rate during baseline was < 60% were aborted and excluded from further study (2/29 sessions were discarded). To analyze the perpendicular displacement within a session and while accounting for differences between the mice (across days), we baseline subtracted using the mean of trials 25–75 at 1 mm above position 2 (variance was verified to be similar across sessions). If a trial did not reach position 2, it was not included in this PD analysis. For perpendicular displacement position analysis, we aimed to analyze a point before the force field onset (position 1), and one within the force field (position 2). The latter bin was chosen to fall in an area where the trajectories are close to parallel to the idealized baseline pull axis to allow reliable estimates of the perpendicular deviation. The results are robust to the specific bin satisfying these conditions (Figure S1E), which can also be appreciated when considering the averaged trajectories (see Figures 1B and 3B–3C).

To determine a pull for post hoc analysis, we analyzed pulls from 150 ms before the crossing of the magnet onset bin (4 mm) to 500 ms post afterward (thus 650 ms in total). Because the mouse had to wait inside the start box (>1 s) to initiate a trial, there was a clear period of little movement. Trajectory averages (Figures 1, 3, S2A, and S5A) are temporal averages computed by aligning the Savitzky-Golay (>20Hz cut-off) filtered pulls at the time when they cross the magnet onset bin (perpendicular line at 4 mm distance from center of start box) and spatially aligning the starting points with respect the perpendicular displacement (to remove any lateral drift of the starting position). The average was then taken over all sessions and the sets of trials as indicated in the respective figure caption (e.g., first ten perturbation trials). Excluding trials that do not reach beyond the upper spatial threshold of the reward box (7 mm vertical distance from the center of the start box) does not greatly alter the appearance of average trajectories (Figure S8E); this criterion excludes about ~10% of all trials. For distance versus time plots (Figures S2D and S2E, Figure S5A), we additionally performed a baseline correction (see Figure S5A). The average speed and acceleration plots (Figures S1B and S1C, Figure S6A) were similarly obtained by aligning the pulls to the magnet onset bin in a temporal fashion (this corresponds to 0 ms in the figures) and then averaging the individual speed and acceleration profiles obtained by taking the derivative of the Savtitzky-Golay filtered pulls. The reward rate was computed by first assigning a 0 or 1 based on reward delivery at the completion of each correctly initialized trial (1 = rewarded trial). Then, this was summed across sessions (for each trial), and divided by the number of sessions. A correctly initiated trial is defined by a pull after they waited long enough (>1 s) in the start box, and then exited the start box.

Generative Model Analysis

As elsewhere (Berniker and Kording, 2008; Izawa and Shadmehr, 2011; Shadmehr and Mussa-Ivaldi, 1994; Todorov and Jordan, 2002; Wolpert et al., 1998), we built a discrete dynamical system that models the scalar perpendicular deviation of the paw and how distinct reward and sensory prediction error driven signals act upon it. Like Izawa and Shadmehr (Izawa and Shadmehr, 2011), we assume that the measured perpendicular displacement of the joystick x_k at a particular position bin and in trial k is the consequence of a motor command u_k, the applied perturbation p_k, and normally distributed motor noise ${\eta }_m\in \mathcal{N}(0,{\sigma }_m^2)$, such that:

$$x_k=u_k+p_k+{\eta }_m.$$ (1)

We further assume that the mouse senses the position of the joystick by proprioception, such that the estimate of its paw position, is given by y_k = x_k + η_s, which is also influenced by sensory noise ${\eta }_s\in \mathcal{N}(0,{\sigma }_s^2)$. We assume that their internal estimate of the perturbation is given by a discrete dynamical system with retention factor r (less or equal to 1; fitted based on data) and memory noise ${\eta }_p\in \mathcal{N}(0,{\sigma }_p^2)$:

$$\widehat{p_{k+1}}=r\widehat{p_k}+{\eta }_p.$$ (2)

Mice received water reward. How should mice pick the motor command u_k, if we assume that they want to maximize reward and minimize motor cost? Given the internal estimate of the perturbation mice can predict their paw position $\widehat{x_k}$ based on the efference copy of their motor signal and their internal estimate of the perturbation $\widehat{x_k}$ by: $\widehat{x_k}=u_k+\widehat{p_k}$. This set of equations describes how motor commands give rise to movement and sensory feedback. Assuming that the mouse wants to minimize costs and maximize reward, the optimal behavior can be derived from the optimal state estimator and the optimal policy (Todorov and Jordan, 2002; Izawa and Shadmehr, 2011; Sutton and Barto, 1998).

Sensory prediction error driven learning

The optimal state estimator is given by the Kalman filter (Berniker and Kording, 2008; Izawa and Shadmehr, 2011; Kalman, 1960; Wolpert et al., 1998). The generative model can be compactly written as a state evolution model:

$$X_k=F{X}_{k-1}+B{u}_k+\eta$$ (3)

with state transition model $F=\left(\begin{array}{cc}r& 0\\ 1& 0\end{array}\right)$, control input model $\mathrm{B}=\left(\begin{array}{c}0\\ 1\end{array}\right)$, process noise η∈𝒩(0, Q); with $Q=\left(\begin{array}{cc}{\eta }_p& 0\\ 0& {\eta }_m\end{array}\right)$ and states $X_k=\left(\begin{array}{c}{p}_k\\ x_k\end{array}\right)$. The observer model is given by: y_k = H X_k + η_s with measurement matrix H = (0 1) and sensory noise ∈ 𝒩(0, R) with $\mathrm{R}={\sigma }_s^2$. In this situation the classical results by Kalman imply that the optimal state estimation is given by recursively iterating (Berniker and Kording, 2008; Kalman, 1960):

$$\begin{array}{c}\widehat{X_k^{-}}=F\widehat{X_{k-1}}+B{u}_k\\ \widehat{X_k}=\widehat{X_k^{-}}+K_k\left(y_k-H\widehat{X_k^{-}}\right).\end{array}$$ (4)

The first equation describes the a-priori prediction of the paw position and perturbation. The subsequent state estimate is a combination of this a-priori estimate and the sensory prediction error (SPE) $y_k-H\widehat{X_k^{-}}$ gated by the Kalman gain matrix, K_k. The Kalman gain is updated according to

$$\begin{array}{c}{P}_k^{-}=F{P}_{k-1}{F}^T+Q\\ K_k=P_k^{-}{H}^T•{(H{P}_k^{-}{H}^T+R)}^{-1}\\ P_k=(I-K_k{H}^T)P_k^{-}.\end{array}$$ (5)

Given this state estimate, the optimal command to compensate for the perturbation is: $-\widehat{p_k}$. We refer to this command as the state estimation driven signal.

Reward prediction error driven learning

For reward-based learning, we employed a simple actor-critic reinforcement-learning algorithm(Sutton and Barto, 1998). The optimal policy’s action, a_k, is based on the value function, which in turn is updated according to the reward prediction error (RPE) (Sutton and Barto, 1998; Izawa and Shadmehr, 2011):

$$\delta =r_k+\gamma \widehat{V_{k+1}}-\widehat{V_k},$$ (6)

with reward/cost term r_k, value function $\widehat{V_k}$ and discounting factor γ. This RPE in turn is used to update the dynamics of the value function and the action based on the temporal difference (TD) rule:

$$\begin{array}{c}\widehat{V_{k+1}}=\widehat{V_k}+{\alpha }_V\delta \\ a_{k+1}=a_k+{\alpha }_a\delta .\end{array}$$ (7)

Both learning rates α_V and α_a are positive. For TD, the reward value function is given by:

$$r_k=\{\begin{array}{l}1-\beta u_k^{2}if\mathit{trial}k\mathit{rewarded}\hfill \\ -\beta u_k^{2}if\mathit{trial}k\mathit{unrewarded}\hfill \end{array}.$$

Thus, the effective reward value of a water drop is modulated by a cost term, which depends on the motor command and a free parameter β.

Motor actions of the learning models

Overall, the motor command (Izawa and Shadmehr, 2011) is given as a sum of the negative perturbation estimate $\widehat{p_k}$ (SPE based) and the action a_k (RPE based) and an exploratory noise term ${\eta }_e\in \mathcal{N}(0,{\sigma }_e^2)$:

$$u_k=-\widehat{p_k}+a_k+{\eta }_e.$$ (8)

This model has several free parameters: four noise variances ( ${\sigma }_s^2,{\sigma }_m^2,{\sigma }_p^2,{\sigma }_e^2$), the retention factor r, the learning rates for the actor-critic model (α_a, α_V) as well as the discounting factor γ and the cost weight β. Given these parameters one can compute the predicted paw position and underlying motor commands. The (purely) actor-critic model refers to the model, where the action is given by u_k = a_k + η_e instead of Equation (8), while the (purely) state estimation based model has motor action $u_k=-\widehat{p_k}$, instead of Equation (8).

Model Fitting

For the adaptation task, we used a signal as induced perturbation p_k that was 0 during baseline and washout and identical to the strongest perpendicular displacement in the first 5 perturbation trials (76–81, averaged over sessions). This estimated perturbation was clamped throughout the perturbation period. For the target-shift task, the induced perturbation p_k was set to zero for all k. During baseline, all motor commands were clamped to zero. We initialized the internal model as zero with vanishing uncertainty, i.e., P = 0, and the value function $\widehat{V_k}$ such that the RPE is on average zero during baseline. We assumed that reward are probabilistic following the average reward rate per task over trial k - i.e., for each simulation any trial was rewarded with the probability given by the average reward rate across sessions in that trial.

We fit the data by minimizing the mean squared differences between the model’s paw position and the averaged measured paw position (x_k) during the perturbation block. As the model is stochastic, we averaged this error over 50 random realizations for which the noise terms and reward were drawn independently. We used the Nelder-Mead simplex algorithm (Nelder and Mead, 1965) to minimize this error function and initialized the optimization at multiple (random) locations. In the main text, we show the motor commands ( $\widehat{p_k}$) and (a_k) that underlie the best fits, as well as the paw position in the model and report their power ratio (SPE versus RPE command ratio): ${\sum }_{k=76}^{175}{\widehat{p_k}}^2/{\sum }_{k=76}^{175}({\widehat{p_k}}^2+a_k^2)$.

Model Predictions

For the separated model predictions, we selectively set either the estimate of the perturbation $\widehat{p_k}$ (SPE-only) or the actor-critic action a_k (RPE only) of the motor command in Equation (8) to zero (results see Figure 4).

For the role of S1, we considered two possibilities:

• Either photoinhibition affects sensory predictions errors, the gating by the Kalman gain or the fidelity of the sensory feedback. This ‘eliminates’ the term $K_k(y_k-H\widehat{X_k^{-}})$ in the updating equation of the Kalman filter (Equation (4)). In this case, the internal estimate evolves according to the a-priori estimate equation. This case suggests that the integration of the two terms in the Kalman filter would happen outside of S1.

• Alternatively, photoinhibition affects the integration of the a-priori estimate and the SPE-based adaptation (right hand side in Equation (4)). In this case the internal estimate immediately returns to the baseline condition upon stimulation. This suggests that S1 would be the locus of the sensory state estimation.

These predictions are presented in (Figure 4) and compared to the mouse behavior in the second half photoinhibition task.

QUANTIFICATION AND STATISTICAL ANALYSIS

The values reported in the text and error bars are the mean ± s.e.m. unless otherwise noted. Behavioral data was analyzed and modeled with MATLAB 2012b or 2014a (Mathworks) or Python. Statistical tests were performed in MATLAB, Python or Prism 5 (GraphPad). Nonparametric tests were used where appropriate and tests were 2-tailed. Alpha was pre-set to 0.05. The first author was not blinded to the experimental conditions, but analysis was blinded to animal and condition identity until complete. Sample size was not predetermined.

DATA AND SOFTWARE AVAILABILTY

The custom LabView code (for control of the behavioral apparatus) and the custom Python code (for the generative model analysis) will be made available upon reasonable request.

Highlights.

• Mice exhibit motor adaptations to forelimb force field-based perturbations

• Optogenetic inhibition of forelimb somatosensory cortex (S1) abolished motor adaptation

• The same photoinhibition did not impair motor control or reward-based learning

• S1 plays an essential role in updating the memory about forelimb motor perturbations