Implications of plan-based generalization in sensorimotor adaptation

The pattern of generalization is thought to reveal how the motor system represents learned actions. Recent work has made the intriguing suggestion that maximum generalization in sensorimotor adaptation tasks occurs at the location of the learned movement plan. Here we support this interpretation, develop a novel model of motor adaptation that incorporates plan-based generalization, and use the model to successfully predict surprising dynamics in the time course of adaptation across several conditions.

Abstract

Generalization is a fundamental aspect of behavior, allowing for the transfer of knowledge from one context to another. The details of this transfer are thought to reveal how the brain represents what it learns. Generalization has been a central focus in studies of sensorimotor adaptation, and its pattern has been well characterized: Learning of new dynamic and kinematic transformations in one region of space tapers off in a Gaussian-like fashion to neighboring untrained regions, echoing tuned population codes in the brain. In contrast to common allusions to generalization in cognitive science, generalization in visually guided reaching is usually framed as a passive consequence of neural tuning functions rather than a cognitive feature of learning. While previous research has presumed that maximum generalization occurs at the instructed task goal or the actual movement direction, recent work suggests that maximum generalization may occur at the location of an explicitly accessible movement plan. Here we provide further support for plan-based generalization, formalize this theory in an updated model of adaptation, and test several unexpected implications of the model. First, we employ a generalization paradigm to parameterize the generalization function and ascertain its maximum point. We then apply the derived generalization function to our model and successfully simulate and fit the time course of implicit adaptation across three behavioral experiments. We find that dynamics predicted by plan-based generalization are borne out in the data, are contrary to what traditional models predict, and lead to surprising implications for the behavioral, computational, and neural characteristics of sensorimotor adaptation.

NEW & NOTEWORTHY The pattern of generalization is thought to reveal how the motor system represents learned actions. Recent work has made the intriguing suggestion that maximum generalization in sensorimotor adaptation tasks occurs at the location of the learned movement plan. Here we support this interpretation, develop a novel model of motor adaptation that incorporates plan-based generalization, and use the model to successfully predict surprising dynamics in the time course of adaptation across several conditions.

the generalization of learning to novel situations plays a central role in perceptual, semantic, motor, and statistical learning and has been aptly termed the first law of psychology (Shepard 1987). In the motor domain, purely “local” learning would inevitably make the learner vulnerable to the curse of dimensionality: One will never make the same movement twice. Generalization in sensorimotor adaptation has been specifically defined: When a subject adapts his/her movements to counter a perturbation in a given movement direction, adaptation generalizes to adjacent directions, following a Gaussian-like function (Poggio and Bizzi 2004; Thoroughman and Shadmehr 2000). However, despite decades of research, questions remain concerning the reference frame for this representation: For instance, is the generalization function couched more in an extrinsic (e.g., world based, target based) or intrinsic (e.g., body based, movement based) reference frame?

One important question concerns the “center” of the generalization function, which refers to the movement direction that displays the maximum degree of adaptation after learning, with the degree of adaptation falling off with adjacent movements. In one study, Gonzalez Castro et al. (2011) showed that generalization is not maximal around the visual target (i.e., the task goal) as previously thought. Instead, a better model of learning centers the generalization function on the actual movements made—that is, the movement executed to counter a perturbing force field. Generalization was most robust at locations tied to the kinematic solution to the task rather than an idealized straight-line path to the target (Gonzalez Castro et al. 2011). These experiments serve as an elegant argument against earlier models that posited target-based generalization. However, in such tasks it has been assumed that the subject’s plan was to always aim toward the target. Recent work, using a method that dissociates explicit aiming processes from implicit motor adaptation, has shown that subjects regularly do not aim toward the target in perturbation tasks (Bond and Taylor 2015; Day et al. 2016; McDougle et al. 2015, 2016; Morehead et al. 2015; Taylor et al. 2014). Using this method, a recent visuomotor rotation study (Day et al. 2016) has provided a different account, showing that the strongest candidate for the center of generalization is the subject’s explicitly reported plan: Generalization was not maximal at the location of the learning target or at the location that was most frequently traversed by the hand but rather at the location the subject most frequently aimed at to achieve task success.

The two accounts (movement centered vs. plan/aim centered) are not necessarily mutually exclusive, and the techniques employed to assay them were different (force fields vs. rotations). However, plan-based generalization is also consistent with recent findings concerning interference: Hirashima and Nozaki (2012) showed that when two different targets are associated with force fields pushing in opposite directions, subjects can succeed in the task by countering both perturbations with the same straight-ahead reach, without showing catastrophic interference. However, interference would be predicted by a purely movement-centered model of generalization, as learning would be canceled out by the opposite signs of the two force fields. This finding suggests that some signal, aside from the movement itself, can train separate neural representations to learn about each force field independent of the movement needed to counter those force fields. Furthermore, the plan-based formulation of generalization may also account for mixed results across different studies, since participants’ aiming behavior can be erratic within an individual, idiosyncratic across individuals, and task and instruction specific (Taylor et al. 2014).

To better understand the dynamics of adaptation, plan-based generalization should be incorporated into the standard state-space model of adaptation. To this end, we first conducted a generalization experiment that parameterized an average generalization function, while also replicating the results of Day et al. (2016). We then simulated the predicted implicit adaptation curves of the plan-based model (and a traditional target-based generalization model) across three different experimental conditions. Finally, we tested those predictions, using three behavioral experiments. Our results support plan-based generalization and confirm one of its strange predictions—that abrupt increases in the magnitude of a sensorimotor perturbation can induce a decrease in observed adaptation, independent of the sign of those errors. Our model also incorporates our recent hypothesis concerning the nature of sensorimotor adaptation, namely that implicit adaptation is homologous with the slow process of the dual-component model of sensorimotor learning (McDougle et al. 2015).

These results provide a plausible computational description of the generalization, time course, and algorithmic features of sensorimotor adaptation and have novel implications for the neural processes and substrates involved. Plan-based generalization of implicit adaptation suggests a model of cooperation between cortical regions involved in high-level movement planning and subcortical regions involved in implicit adaptation, where the former may determine generalization in the latter.

METHODS

Participants.

Sixty-eight right-handed subjects (age range 18–34 yr; 46 women, 22 men) were recruited from the research participation pool maintained by the Department of Psychology at Princeton University in exchange for course credit. Handedness was verified with the Edinburgh Handedness Inventory (Oldfield 1971). Sixteen subjects participated in the generalization experiment, although one subject was excluded for failure to follow task instructions. All analyses in the generalization experiment were conducted on the remaining fifteen subjects. Fifty-two subjects participated in the rebound experiments (see Experimental procedures and analysis). Twenty subjects participated in the first rebound experiment, with ten subjects coming from a previously published data set (McDougle et al. 2015) and the remaining ten added to counterbalance rotation directions. Sixteen subjects participated in the second rebound experiment, and sixteen participated in the third. Rotation signs were counterbalanced in all experiments to account for kinematic biases (Ghilardi et al. 1995). All subjects participated in protocols reviewed and approved by the Princeton University Institutional Review Board and provided written informed consent.

Task and apparatus.

In all experiments, participants made center-out, horizontal reaching movements to visually displayed targets (7-mm radius) with a digitizing tablet (Intuous Pro; Wacom). Movements were recorded with a digital pen, which subjects held in a power grip and moved along the surface of the tablet. The task was controlled by custom software written in Python (https://www.python.org). Reach trajectories were sampled at 100 Hz. Stimuli were shown on a 17-in. LCD computer monitor (Dell) horizontally mounted 25 cm above the tablet. The monitor occluded the subject’s vision of his/her hand. A small cursor (3.5-mm radius) provided feedback during each reach.

At the start of each trial, subjects positioned the digital pen in a central position with the aid of a visual ring that represented the distance between the hand and the starting position. After this position was maintained for 1 s, a green visual target appeared. After leaving the start, the subject’s reach had to cross an invisible ring that contained the target in <500 ms to avoid a “too slow” warning, delivered aurally to the subject by the task software. If the center of the cursor landed within the target, a pleasant “chime” was sounded; otherwise a “buzz” sounded.

A verbal reporting method was used to ascertain subjects’ explicit movement plan on every trial (Fig. 1A). The task instructions in all reporting trials were similar to those of previous work (Bond and Taylor 2015; Day et al. 2016; McDougle et al. 2015; Morehead et al. 2015; Taylor et al. 2014). The target on a given trial was surrounded by a ring of 62 numbered visual landmarks spaced 5.625° apart (Fig. 1A). Subjects were instructed to verbally report, before each reach, the landmark they planned to aim toward to make the cursor terminate within the target. The experimenter manually recorded the reported aiming directions, and we refer to these data as explicit learning. To separate out various learning processes, we quantified implicit learning by subtracting the explicit component from the participant’s movement heading angle on each trial. Thus for every trial we measured 1) the actual movement made by the subject, 2) the subject’s intended movement (“plan”), and 3) the difference between the two, which is interpreted as implicit adaptation (Fig. 1A; Taylor et al. 2014). This method allowed us to analyze each learning process independently.

Fig. 1.

Tasks, explicit/implicit learning. A: participants performed a visuomotor rotation task, where a visual cursor was rotated relative to their reach direction. Explicit plans (blue) were measured via verbal reports, reach movements (purple) were recorded heading angles, and implicit learning (red) is the subtraction of the former from the latter. B: trial protocol for the generalization experiment. Text on left of colored blocks specifies what type of target (learning vs. probe) was presented in the respective block. C: trial protocol for the 3 rebound experiments. Text on left of colored blocks specifies the rotation magnitude and sign for that block (note that rotation signs were counterbalanced in each condition).

Experimental procedures and analysis.

The generalization experiment (Fig. 1B) proceeded as follows: The first 96 trials involved reaches to all 16 target locations (0°, 22.5°, 45°, 67.5°, 90°, 112.5°, 135°, 157.5°, 180°, −157.5°, −135°, −112.5°, −90°, −67.5°, −45°, and −22.5°), which were presented in pseudorandomized blocks of 16 trials. In the first 16 trials subjects received online feedback to get accustomed to the reaching task, followed by 32 trials to get accustomed to end-point feedback and then 48 trials in which subjects received no feedback to measure reach biases at all 16 targets. Veridical end-point feedback was restored for the next 16 trials, directed at a single target location (0°; i.e., the target directly to the right of the start point), and participants were instructed to verbally report their aiming location on each trial before moving (baseline aim report block) so they could get used to the reporting procedure. After this baseline block, for 96 trials (rotation block) participants experienced a 45° rotation (the rotation was −45° for the counterbalanced group) while reaching to a single learning target location (0°); participants were instructed to continue to report their aiming location on each trial. The rotation block was designed to get subjects to fully learn the perturbation at the single learning target and reach asymptote. The following 180 trials constituted the generalization probe block, which was designed to test for generalization to novel target locations: On 50% of trials subjects reached to 1 of the 15 probe target locations, while still reporting their aim but receiving no cursor feedback. Feedback was withheld at the generalization targets to restrict error-driven learning. On the other 50% of trials in the generalization block subjects reached to the learning target and continued to receive rotated end-point cursor feedback, to prevent decay of adaptation at the learning target. Learning trials and generalization trials were alternated to ensure a regular “top-up” of adaptation at the learning target. Finally, the last 48 trials constituted the aftereffect block, where participants were told to reach directly at each target, stopped reporting any aims, and received no feedback. Targets were pseudorandomly presented in three blocks of 16 trials. The aftereffect block was designed to get a measure of the final level of implicit adaptation at each probe location.

Generalization was modeled as a Gaussian function, with a specific height and width (σ). Each subject’s (n = 15) implicit generalization function was determined by calculating the mean implicit learning at each of the 16 locations during the probe trials. A Gaussian was fit to the group data via bootstrapping, resampling our subject pool with replacement 1,000 times, and its parameters (height and width σ) were optimized to provide the lowest root mean square error (rmse) between the fitted curve and the bootstrapped mean function. Fits were optimized with the fmincon function in MATLAB.

The next three experiments (Fig. 1C) used a “rebound” paradigm, which involves learning a first perturbation, briefly learning a second perturbation, and then entering an “error-clamp” block where visual errors are removed and latent states of learning can be observed (Smith et al. 2006). This paradigm is useful for extracting the fast and slow processes of the two-state state-space model: While the brief second perturbation can be adequately learned, the error-clamp phase shows a return to the state trained by the first perturbation, suggesting that one learning process (fast) can quickly learn the second perturbation while another learning process (slow) remains “stuck” in the first perturbation state. In the three rebound experiments, the session proceeded as follows (Fig. 1C): The first 100 trials were baseline trials where online cursor feedback was veridical. On trials 101–300 (R1) the cursor was rotated by 45° (the sign of this rotation was counterbalanced across subjects, but all subsequent descriptions are for the counterclockwise group). On trials 301–320 (R2) the rotation flipped sign to −45° (first rebound condition), was increased to 135° (second rebound condition), or was increased to 75° (third rebound condition). In trials 321–420 (error clamp) a visual error clamp was placed on the cursor, making it move straight to the target regardless of the subject’s hand position orthogonal to the target, and subjects were told to aim directly to the target. In all three rebound experiments, the target appeared at 0° (i.e., to the right of the start point) on all trials.

Note that we chose to provide only end-point feedback in the generalization experiment to preclude online feedback corrections, since feedback corrections can alter the generalization function (Taylor et al. 2013), and to minimize the mismatch between trials with and without any visual feedback. In the subsequent rebound experiments, we chose to provide online cursor feedback to increase the magnitude of implicit learning (Taylor et al. 2014) and thus our ability to detect spontaneous rebound in the error-clamp block.

Generalization model.

The goal of our model was to add plan-based generalization to the slow process of the dual-process state-space model. We have hypothesized that the slow process represents the time course of implicit adaptation (McDougle et al. 2015). We thus used the model to make predictions about how adaptation proceeds in various sensorimotor rebound paradigms. The rebound paradigm was specifically chosen to test our model, since it is thought to reveal the time course of multiple learning processes (Pekny et al. 2011; Smith et al. 2006).

The standard state-space model of adaptation is defined as follows:

$$e_t=x_t-p_t$$ (1)

$$x_{t+1}=A{x}_t+B{e}_t$$ (2)

where e is the error experienced at time t, defined as the difference between the current motor state x and perturbation p. A is the retention factor applied during the update of x, and B is the learning rate.

Smith et al. (2006) outlined a dual-process state-space model that combines a fast component, denoted by an E in Eq. 3, and a slow component, denoted by an I in Eq. 4, to yield the final output x (Eq. 5). This model assigns separate retention factors and learning rates to each process, where A^E < A^I and B^E > B^I.

$$x_{t+1}^{\mathrm{E}}=A^{\mathrm{E}}{x}_t^{\mathrm{E}}+B^{\mathrm{E}}{e}_t$$ (3)

$$x_{t+1}^{\mathrm{I}}=A^{\mathrm{I}}{x}_t^{\mathrm{I}}+B^{\mathrm{I}}{e}_t$$ (4)

$$x_t=x_t^{\mathrm{E}}+x_t^{\mathrm{I}}$$ (5)

Critically, the fast process learns quickly but is flexible, and the slow process learns slowly but is robust. We have recently provided evidence consistent with the idea that the fast and slow processes may map onto, respectively, explicit and implicit components of learning (McDougle et al. 2015, 2016; Taylor et al. 2014). This was made possible by using a task paradigm that measures subjects’ explicit motor plans (angular aim directions) on every trial (Fig. 1), allowing implicit adaptation to be estimated by subtracting subjects’ explicit aiming locations from their actual movement directions. The resulting subtracted process lines up closely with measured aftereffects, consistent with the idea that the implicit subtraction measurement represents true adaptation (Bond and Taylor 2015; Taylor et al. 2014). We found that these two dissociated components can be accurately simulated and fit by the dual-process model (McDougle et al. 2015), further suggesting that the pair of processes (fast/explicit and slow/implicit) are homologous.

Because learning is not perfectly local, it is critical that generalization be incorporated into a complete model of the motor adaptation process. Generalization of adaptation can be explained by a population code, where neurons with distinct tuning functions are sensitive to a range of movement directions (Donchin et al. 2003; Tanaka et al. 2009; Taylor et al. 2013; Thoroughman and Taylor 2005). When learning occurs at one location, it spreads over to adjacent locations. Thus adaptation should be represented as a vector of states corresponding to multiple locations rather than a single location. Generalization was incorporated into the model as follows: The state of implicit adaptation, x^I, was modeled as a 1 × 360 vector ${\vec{x}}^I$, corresponding to a tiled space of independent adaptation states at each integer angular location along the circumference of the two-dimensional circular workspace. Surrounding states are updated via a Gaussian function, where the visited state on trial t was set as the center, µ_t, of the Gaussian with standard deviation σ and height B^I, the latter being equivalent to the learning rate of the slow process:

$${\vec{x}}_{t+1}^{\mathrm{I}}=A^{\mathrm{I}}{\vec{\mathrm{x}}}_t^{\mathrm{I}}+B^{\mathrm{I}}\hspace{0.17em}\exp \left[{\left({\vec{\mathrm{x}}}^{\mathrm{I}}-{\mu }_t\right)}^2/2{\sigma }^2\right]\times e_t$$ (6)

The value used to determine the center of the Gaussian on each trial (µ_t) constitutes the frame of reference for generalization, for instance, the target location, or movement plan, on trial t. In the traditional target-based conception of generalization, the center of generalization on every trial (µ_t) is the target location; in our updated plan-based model, the center of generalization is the current aiming direction (Day et al. 2016). Thus the predicted motor output on trial t (x_t) is the sum of the current explicit state ($x_t^E$) and the implicit state ($x_t^I$) associated with the current model’s “center” of generalization, be it the target (0° state) or the plan ($x_t^E$ state, which models the current aiming direction). We used this updated model to simulate and fit the time course of adaptation in three different “rebound” paradigms, which are designed to help isolate the fast and slow components of learning.

Model fitting.

In the three rebound experiments, models were fit to minimize the combined rmse between the model’s slow-state simulation (x^I) and the subject’s implicit learning data and the model’s fast-state simulation (x^E) and the subject’s explicit learning data (McDougle et al. 2015). Fits were optimized with the fmincon function in MATLAB. The stability of the model fits was tested with a grid search over different starting parameter values. We found that the model fits were consistently stable within our constraints.

RESULTS

Width of generalization.

We first set out to 1) experimentally determine the width of an average generalization function, isolating the generalization of implicit adaptation specifically, and 2) replicate the finding that the movement plan—not the target location or actual movement (kinematic solution)—is the center of generalization (Day et al. 2016). To do this we had subjects learn a 45° perturbation at a single 0° target location and probed their implicit learning state at 15 other locations, both during and after learning (Fig. 2A; see methods for details).

Fig. 2.

Parameterizing the generalization function. A: experimental setup. During the learning block (trials 97–208), subjects were only presented with the learning target (0° target, green) and received rotated feedback, During the probe block, 50% of trials were rotated feedback trials at the 0° target, and 50% of trials were no-feedback trials at the probe locations (black). B: generalization functions are shown for subjects’ hand angles (purple), aim reports (blue), and implicit learning (red). y-Axis represents the angular value of each learning component, at each target location, after being rotated to a common 0° axis for analysis. C: a Gaussian was fit to the bootstrapped mean implicit learning function. Inset: the mean width parameter of the fit Gaussian function, with 95% confidence interval. Shading represents 1 SE.

Subjects displayed the typical Gaussian generalization pattern: Implicit learning at the probe locations tapered off as a function of distance from the original learning location (Fig. 2B). While it is not surprising that maximum adaptation is seen at the learning target, because only those trials had cursor feedback, the pattern shown in Fig. 2B does not specify what aspect of behavior drives maximal implicit adaptation. The maximum adaptation observed at the learning target can reflect at least one of three variables: the learning target’s location (target-based generalization), the movement made during learning trials (movement-based generalization), or the explicit plan during learning trials (plan-based generalization). Thus in Fig. 2B the x-axis is conflated with both subjects’ movement and aiming directions during learning trials. This conflation is addressed in Center of generalization.

Figure 2B depicts the full generalization function of subjects’ movements (purple) and the individual generalization components for explicit (blue) and implicit (red) learning. As previously shown (Heuer and Hegele 2011), explicit generalization was essentially flat across the workspace. Such a pattern could reflect true global generalization, a function of the landmarks and instructions in the task, or a combination of both; critically, our goal in designing the task was to induce global (flat) explicit generalization in order to unmask the “true” implicit generalization function (i.e., if aim directions were highly variable across probe trials, it would yield a highly distorted implicit generalization function). We note that the generalization of explicit learning in an unconstrained experimental setting may not be truly “flat” but would rather have its own nonuniform, perhaps also Gaussian, function. But for the purposes of the present study, we sought to isolate generalization of implicit learning by constraining explicit generalization. Future experiments will need to be conducted to better understand how these two processes fully interact.

Interestingly, global explicit learning causes an effective positive DC shift in the measured generalization function of movements (Fig. 2B, purple). This DC shift is routinely observed in visuomotor generalization data (Krakauer et al. 2000; Taylor et al. 2013), and our results make the novel suggestion that such a shift may be a result of the additive generalization of explicit and implicit learning.

We determined the width of implicit generalization with a Gaussian function, via bootstrapping (see methods); the average width, σ, was 37.76° (Fig. 2C). This width value was used for all subsequent model simulations and fits. We note that the derived value of σ is, qualitatively, in accordance with the width of previously described generalization functions (Krakauer et al. 2000; Mattar and Ostry 2007; Taylor et al. 2013).

Center of generalization.

Our next analysis used the aftereffect block to determine the center of generalization. In the aftereffect block, subjects were told to aim directly to presented targets and all cursor feedback was removed, giving us a measure of implicit learning at each exact target location without the confound of an explicit plan to a location away from the target. If generalization is “target based,” the maximal aftereffects should be seen at the 0° target, which was the only target where subjects received feedback during the task. If generalization is “movement based” (Gonzalez Castro et al. 2011), the peak aftereffect should be seen around the 45° target, as the mean movement direction during learning block trials (0° target trials) was ~43° (µ movement = 42.80 ± 0.50°). Finally, if generalization is plan based, the peak aftereffect should be closest to the 22.5° target, as the mean aiming direction toward the learning target was 26.19 ± 3.50°.

The generalization function during the aftereffect block is shown in Fig. 3A. This function was created by taking subjects’ mean reach direction to each target in the no-feedback aftereffect block and subtracting their mean reach direction to each target in the no-feedback baseline bias block. As predicted by plan-based generalization (Day et al. 2016), peak aftereffects were present near the 22.5° target. This result is especially compelling considering that no feedback was ever received during the rotation block at the 22.5° target trials, whereas at the 0° target location rotated feedback was consistently present through both the learning and probe blocks.

Fig. 3.

Determining the center of the generalization function. A: generalization of aftereffects. Vertical lines show the predicted center of generalization for target-based generalization (red), plan-based generalization (green), and movement-based generalization (blue). The horizontal width of the green and blue lines represents ±1 SE of, respectively, the mean aim direction and hand angle during learning trials. B: difference score analysis. Gaussian functions were fit to each subject’s distribution of aftereffects to determine the peak of generalization for each subject. Difference scores were computed by subtracting the learning target location (0°), the subject’s mean aim/plan, or the subject’s mean movement values during learning trials from their peak generalization value. Error bars represent 1 SE. *P < 0.017, Bonferroni-corrected α value. n.s., Not significant.

Importantly, since each subject had a unique mean aim direction during learning trials, we could perform a more quantitative analysis: First, a Gaussian was fit to each subject’s distribution of aftereffects, with free parameters for the height, offset, standard deviation, and mean value. The fitted mean value (i.e., the “center of generalization,” or the x-coordinate corresponding to the peak/mean of the fitted Gaussian) was labeled as that subject’s peak aftereffect. This fitting procedure allowed us to get smooth values that were not constrained to 1 of the 16 target locations. Second, subjects’ mean aim direction was calculated as their mean aim during all learning target trials (0° target) in both the learning and probe blocks of the generalization experiment, and subjects’ mean movement direction was calculated as their mean hand angle during all learning target trials in the learning and probe blocks. Finally, we computed three separate “difference scores” for each subject, corresponding to three potential centers of generalization: We calculated the difference between the location of their peak aftereffects and 1) the learning target direction of 0°, 2) their mean aiming direction during 0° target trials, and 3) their mean movement direction during 0° target trials.

We performed three one-sample t-tests on the difference scores (Bonferroni corrected) and, as predicted, found no significant difference between the location of subjects’ peak generalization and their mean aiming directions [t₍₁₄₎ = 0.94, P = 0.35] but found significant values for both the target location [t₍₁₄₎ = 4.35, P = 0.0006] and mean movement direction [t₍₁₄₎ = −3.31, P = 0.005; Fig. 3B]. These results support previous findings showing that the movement plan may determine the locus of generalization (Day et al. 2016). Furthermore, our results show that individual differences in generalization may be partly explained by subject-specific aiming behavior during a visuomotor learning task.

Predicting time course of adaptation.

Plan-based generalization makes the important prediction that the direction in which a subject is planning to move selects the internal representation that is to be maximally adapted during learning. This representation can be thought of as a neural population code that corresponds to a certain desired movement direction. These internal representations can be modeled as specific “states” that selectively undergo adaptation, and, in turn, these states can be recast as specific trajectories/directions in the space of the task (i.e., 1°:360°).

Because the direction of a subject’s plan can change trial by trial, the implicit adaptation time course that an experimenter observes may consist of multiple assays of different internal representations/states rather than the progress of a single distinct state. This can be thought of as a “credit assignment problem” (Gonzalez-Castro et al. 2011): Credit for an observed error can be differentially assigned to whichever representation corresponds to the current movement plan.

Based on these assumptions, the plan-based model makes several predictions about how adaptation should proceed in various single-target “rebound” paradigms. Rebound paradigms have been used to expose latent states of learning: In a rebound paradigm, subjects first learn to counter a particular perturbation, then a second perturbation is briefly presented, and finally adaptation is measured in an error clamp. In most cases, subjects learn the second perturbation easily but “rebound” in the error clamp, reaching in a manner that suggests they are still adapted to counter the first perturbation. Such results provide evidence that multiple learning processes exist: a fast process that drives learning of the second perturbation and a slow process that remains adapted to the first perturbation during the error clamp (Smith et al. 2006).

First, the plan-based model predicts that if the perturbation change induces a large change in the enacted plan, the implicit time course should drop to near zero, because a new set of states are now being updated that were only minimally updated during the first perturbation. Second, this drop in observed implicit learning should be generally agnostic to the sign of the perturbation—the main factor is the absolute change in the aim plan. This leads to the novel prediction that an increase in a perturbation, which is followed by a corresponding increase in visuomotor error, can induce a decrease in the degree of observed adaptation. Such a result is not predicted by traditional models of learning. Finally, if the perturbation change induces only a modest aim change, the drop in the degree of observed adaptation should be partial, because states close to the original aim direction may have also been significantly updated during learning.

We simulated the three experimental paradigms with our dual-process plan-based generalization model (Fig. 4). We used the width parameter σ derived from the first experiment (Fig. 2C) and hand-tuned the retention and learning rate parameters using values similar to those previously reported (McDougle et al. 2015), using the same values across all three conditions. As shown in Fig. 4A, in the first rebound condition (45°/−45° rotations) the model predicts a drop in implicit learning during the counterrotation: The center of the generalization function (the aim/plan) moves ~60° during the second rotation, and thus the newly observed implicit state shows little learning (Fig. 4A). Importantly, target-based generalization (dashed line in Fig. 4A) predicts a smaller drop in implicit learning during the second rotation phase, since a target-based model assumes the same state is being updated and observed in all three phases of the experiment. The plan-based model also predicts that the magnitude of adaptation unmasked in the rebound phase lies between the asymptotic value during the first rotation and the near-zero value during the second rotation. This reflects an important prediction of the generalization model: The aiming direction during the error-clamp phase is 0° (subjects are instructed to aim directly for the target), which is less than a standard deviation (in terms of the generalization function) from the mean plan direction in the first rotation block (~22.5°), resulting in significant learning at the 0° state. Thus what appears to be adaptation “rebound” is the observation of a previously adapted state that was “hidden” because the subject had not aimed to the target during the rotation phases.

Fig. 4.

Simulating adaptation with plan-based generalization. A: rebound condition 1, model simulation. The plan-based model predicts an implicit adaptation time course (red) that drops quickly at the onset of the second rotation and shows an intermediate value of rebound. A target-based model predicts an implicit time course (dotted line) that shows a weaker drop and no rebound. Insets: zoomed-in implicit simulation data from the end of the first rotation (trials 290–300), through the counterrotation (301–320), and the beginning of the clamp (321–330). B: rebound condition 2. This schedule is identical to condition 1, except that the second rotation is 135°. The plan-based model predicts an implicit adaptation time course (red) that also drops quickly at the onset of the second rotation (like condition 1), then regains ground, and finally shows robust rebound. Critically, a target-based model predicts an implicit time course (dotted line, inset) that shows an increase instead of a drop upon the second rotation. C: rebound condition 3. This schedule is identical to condition 1, except that the second rotation is 75°. The plan-based model predicts an implicit adaptation time course (red) that drops only subtly at the onset of the second rotation but shows robust rebound. Importantly, a target-based model predicts an implicit time course (dotted line, inset) that shows a slight increase instead of a drop upon the second rotation. Both models make similar predictions due to the more modest change in the aim direction (blue). D–F: heat maps showing learning at each of the 360 states during the 3 conditions with plan-based generalization. Hotter colors denote positive adaptation at those states. Black lines show the state being observed/maximally updated during a given trial (in our model, this is the fast process).

Another implication of our model is that the sign of the error is relatively irrelevant when considering the magnitude of the observed “drop” in implicit learning when the rotation magnitude changes—rather, this drop is primarily driven by the raw distance between the two perturbations (Δ90°). Thus we predicted that when the perturbation was increased by 90° but maintained its sign, the drop in the implicit learning trace would look like that of the sign change condition. In our second rebound condition, the first rotation presented was 45° and the second rotation was 135°. We simulated this paradigm using the same parameter values as the previous simulation. As shown in Fig. 4B, the model again predicts a large drop to around 0° of implicit learning at the onset of the second perturbation. Subsequently, the implicit state starts to positively learn again because the new perturbation has a positive sign. Finally, the model predicts more rebound in the error-clamp phase than the first rebound condition (Fig. 4A), as no negative learning occurs at the 0° state (Fig. 4B).

As shown in Fig. 4, A and B, our model predicts that a large change in the magnitude of a perturbation can vastly alter the observed adaptation. This occurs because a large perturbation change can drive a large change in the movement plan and, as shown in Fig. 3, adaptation is at the mercy of the movement plan. It follows that if a change in the perturbation is modest and lies within the generalization curve, the shift in the state of observed adaptation should be smaller. In our third rebound schedule, we changed the perturbation during the second rotation phase by only 30°, which is less than a single standard deviation of the derived generalization function (Fig. 2C). We simulated the experimental paradigm again using the same parameter values as the previous simulations (Fig. 4C). The model reveals an implicit learning trace that dips briefly during the second rotation block and then shows robust rebound in the error-clamp phase. Heat maps in Fig. 4, D–F, illustrate the simulated degree of learning at each of the 360 states during the task, showing how learning is generalized across states and how changes in the plan alter the locus of maximum adaptation in the map.

We conducted three behavioral experiments with the perturbation schedules highlighted above, using the aim-report method described in the generalization experiment (Fig. 1). We found that the simulated slow/implicit processes of our model (Fig. 4) predicted much of the dynamics of the measured implicit learning time course in the behavioral data (Fig. 5). For the first rebound condition, implicit adaptation dropped in the second rotation phase and rebounded to an intermediate value in the error-clamp phase (Fig. 5A). In the second rebound condition, a similar drop was observed, and rebound was higher than that of the first experiment, as predicted by the model (Fig. 5B). We note, however, that subjects did not positively adapt during the 135° rotation even though the model predicted positive learning in that phase (Fig. 5B, inset); this result may be explained in the context of recent work showing that subjects do not seem to adapt well to especially large rotations (Morehead et al. 2017). Finally, in the third rebound paradigm, as predicted, subjects showed only a slight dip in observed adaptation during the second rotation phase and showed rebound in the clamp phase (Fig. 5C).

Fig. 5.

Behavior. A: data and plan-based model fit for rebound condition 1. B: data and model fit for rebound condition 2. C: data and model fit for rebound condition 3. Insets: zoomed-in implicit learning data from the end of the first rotation (trials 290–300), through the counterrotation (301–320), and the beginning of the clamp (321–330). Shading represents 1 SE.

We fit our generalization model to the data (Fig. 5) by fitting the slow process to the implicit time course and the fast process to the explicit time course (McDougle et al. 2015) and using the actual aiming data and movement data to determine the center of generalization on each trial for the plan- and movement-based models, respectively (see methods). All three models provided a strong fit to the data, and the fitted slow processes resembled the slow processes of the simulations. After collapsing the data across the three experiments, we conducted two model comparison procedures: First, we used the rmse of the fit between the slow process and the implicit data to compare the plan-based generalization model (Fig. 5) to both a target-based generalization model, which used only the target location (0°) as the center of generalization for all trials, and a movement-based generalization model, which used the subject’s movements on each trial as the center of generalization. We note that because the three experiments had only a single target location, the fit of the target-based model is mathematically equivalent to the fit of a model with no generalization. A repeated-measures ANOVA revealed a main effect of model on rmse [F_(2,51) = 28.00, P < 0.0001]. Post hoc tests (Tukey’s HSD) revealed significantly better fits for the plan- vs. target-based models (P < 0.05) and the movement- vs. target-based models (P < 0.05) but no difference between the plan- vs. movement-based models (P = 0.72). In a second analysis, we fit each model to the averaged group data from each experiment and calculated the quality of fit using the Bayesian information criterion (BIC; Berniker et al. 2014). A difference in BIC units of >10 is considered strong support for the model with the lower value. BIC values were summed across experiments and compared across models. Both the plan- and movement-based models outperformed the target-based model (BIC differences > 500), and the plan-based model also outperformed the movement-based model (BIC difference = 57.35).

We note that our behavioral results show an incongruity in the decay of aftereffects in the error-clamp phase across the three rebound conditions, where decay in the error-clamp phase was virtually null in the first rebound experiment (Fig. 5A) but was clearly observed in the other two (Fig. 5, B and C). The model does indeed predict greater rebound in the second two conditions, because in those experiments the second rotation is the same sign as the first and thus does not cause any “unlearning” at any locations. Indeed, an ANOVA revealed a trend in the main effect of condition on early error-clamp trials (first 5 trials) in the error-clamp phases (P = 0.07), with rebound conditions 2 (µ = 16.07°) and 3 (µ = 16.48°) both showing larger early rebound than condition 1 (µ = 10.13°). Late error clamp (last 5 trials), however, was similar across conditions (P = 0.59). Recent research has shown that if error-clamp blocks are conducted up to 24 h after learning, subjects will persistently show some stable degree of aftereffect (Brennan and Smith 2015a). Thus it may be the case that in the second and third rebound conditions subjects are decaying back to some stable value of adaptation, whereas in the first condition the entire error-clamp block represents only that stable value and thus does not decay. However, this interpretation is conjecture, as the effects were not statistically reliable and the present study was not specifically designed to provide any systematic assay of decay rate or decay onset (Brennan and Smith 2015b; Vaswani and Shadmehr 2013). Observed decay could occur through the forgetting parameter in the model (A^I): Forgetting could simply represent imperfect retention of the adapted movement in the recalibration circuit or the action of a consistent proprioceptive error signal that gradually drives the movement back to some prespecified prior concerning the proprioceptive state and the observed visual feedback. Furthermore, given that our error-clamp phase includes an instruction to the subject to aim directly for the target, it is possible that some subjects ignored or only partially followed this instruction, leading to variance in error-clamp decay. Indeed, intersubject variance in a constant 0-error clamp has been reported previously (Vaswani and Shadmehr 2013), and it is possible that this variance is driven by directed exploration. Future research can be designed to address these issues directly.

DISCUSSION

Generalization is a fundamental aspect of learning, allowing behaviors to be flexible and robust considering the ever-changing conditions of the body and environment. Generalization has been shown to be central to human sensorimotor learning, but the precise reference frame for generalization has proven elusive. Recent research (Day et al. 2016; Gonzalez Castro et al. 2011; Krakauer et al. 2006; Novick and Vaadia 2011) has argued against previous models of generalization, which centered the putative generalization function on the task-oriented goal of a movement—that is, the point in space where task success is focused (e.g., a target to be contacted by a cursor). Here we support the theory that centers the reference frame of generalization on the explicitly accessible movement plan (Fig. 3; Day et al. 2016) and formalize a model of generalization that predicts counterintuitive time courses of implicit adaptation in contexts where the movement plan required for task success changes by various signs and magnitudes (Figs. 4 and 5).

The present study not only supports and formalizes plan-based generalization but also shows that generic state-space models of sensorimotor learning, which predict increases in adaptation when the size of a motor error increases during a perturbation task (Fig. 4, B and C), are flawed—if the movement plan is required to deviate to account for large increases in error, the plan-based model makes the correct, though surprising, prediction that measured adaptation will appear to decrease, as an uncharted state is now being updated and observed. Indeed, it may be the case that once implicit learning appears to be asymptotic a significant increase in its value could not be reliably induced. First, in all subjects, a change in the rotation caused a subsequent change in the aiming direction, thus shifting the implicit state being observed to a location that has adapted less. It is likely that only very small changes in the rotation size would prevent the explicit component from shifting, and in such cases implicit learning would remain virtually unchanged. Second, two recent studies have revealed that implicit learning appears to saturate after many trials, suggesting that there may be a ceiling on the level of adaptation independent of questions of generalization (Bond and Taylor 2015; Morehead et al. 2017).

Indeed, Gonzalez Castro et al. (2011) made a similar prediction that the apparent decreased learning rate observed after large vs. small errors (Wei and Körding 2009) could be the result of a “credit assignment” problem: Because the new movement required to counteract a novel large error is distal to the previously learned movement, the observed reduction in adaptation could be the result of movement-based generalization rather than a reflection of error-size-dependent learning rates in the CNS. Our results support the credit assignment interpretation. Indeed, as shown in Figs. 4 and 5, when the required aim/movement is distal to the previously trained aim/movement, decreases in adaptation are predicted by the width parameter of the generalization function, rather than requiring a reformulation of the learning rate. However, recent results (Day et al. 2016) and the generalization experiment described here (Figs. 2 and 3) support the plan as the center of generalization, not the movement itself. Two more pieces of evidence support the plan-based framework: First, previous work has shown that distinct internal models of opposing sensorimotor transformations (rotations) and dynamics (force fields) can be maintained for identical movement kinematics when separate targets are presented for each transformation (Hayashi et al. 2016; Hirashima and Nozaki 2012). Second, a recent study found that interference can be reduced between two opposing force fields applied to the same reach direction when different “follow-throughs” are cued (Sheahan et al. 2016). What’s more, participants do not have to carry out the follow-through but simply must plan it to avoid interference (Sheahan et al. 2016). We argue that the explicit aim, or end goal of the movement plan, provides the context for which states are updated during adaptation. This “context” may act as the “contextual signal” posited in models of sensorimotor learning, such as the MOSAIC model (Haruno et al. 2001).

Distinct learning processes and intrinsic vs. extrinsic coordinate systems.

The pattern of generalization is thought to provide a window into the computational underpinnings of the motor system (Poggio and Bizzi 2004). However, studies of generalization have failed to paint a consistent picture of the reference frame of generalization. Numerous studies have found evidence for an intrinsic (joint based) reference frame (Shadmehr and Mussa-Ivaldi 1994), while others provide evidence for an extrinsic (workspace based) reference frame (Krakauer et al. 2000). More recent work has suggested that the CNS uses both intrinsic (e.g., joint configurations) and extrinsic (e.g., Cartesian points in space) coordinate frames during adaptation (Berniker et al. 2014; Brayanov et al. 2012). This work suggests that the brain learns both the intrinsic motor dynamics needed to reduce sensorimotor error as well as modifying a plan for a reach trajectory in extrinsic space. Thus the CNS may build a composite representation of both internal models. While the present study does not address the question of intrinsic vs. extrinsic coordinate frames during adaptation, some parallels may exist.

One hypothesis is that aiming may reflect explicitly accessible trajectory planning and thus may be related to an extrinsic coordinate frame of learning. Implicit adaptation may be more reliant on an intrinsic reference frame or some combination of the two. Because the generalization of adaptation in visuomotor rotation tasks seems to be closely linked to the locus of the plan (Fig. 3; Day et al. 2016), our data may provide indirect support for the notion of mixed coordinate frames in visuomotor rotation learning (Brayanov et al. 2012) because adaptation appears to be reliant on the plan. One way to disentangle these two coordinate frames would be to systematically shift the workspace and joint configurations while tightly controlling the locus of aiming.

Explicit learning and generalization in a psychological space.

These results also raise questions regarding the pattern of explicit generalization, which has yet to be systematically addressed. Various lines of research have suggested that explicit learning is likely to produce relatively global generalization. First, Bond and Taylor (2015) showed that explicit learning is highly flexible and can respond to varying rotation sizes, number of targets, and, most importantly, whether the aiming landmarks are fixed or rotated with the target. The latter result suggests that subjects have a more abstract representation of the aiming solution rather than just declaratively remembering the appropriate aiming landmark. Heuer and Hegele (2011) showed that subjects’ aftereffects, a hallmark of implicit adaptation, generalize locally but their explicit estimation of the movement required to counteract the perturbation generalizes globally. Moreover, other recent research (Yin et al. 2016) has suggested that the direction-specific, limited generalization observed in adaptation tasks can be eliminated by the priming of a model-free, explicit learning process. Yin et al. (2016) showed that in a rotation task generalization to nonlearned locations can be maximized if subjects perform an irrelevant gain-adaptation task at those locations before a generalization testing phase. One interpretation could be that this additional training acts as a prime on a more explicit learning system.

Another form of generalization, interlimb transfer, has been shown to occur across limbs during the learning of visuomotor rotations (Sainburg and Wang 2002), and it has recently been shown that explicit learning is fully transferred between limbs while implicit learning is only minimally transferred (Poh et al. 2016). The latter result suggests that a relatively abstract, high-level learning process primarily drives effective interlimb transfer.

Critically, our generalization task (Figs. 2 and 3) was designed to encourage subjects to use the same explicit aiming solution (aiming number) at all locations to isolate a smooth implicit generalization function. It is unclear whether the flat global generalization function of explicit aiming depicted in Fig. 2C is solely a result of the task instructions or a true sign of explicit generalization, like that observed in Heuer and Hegele (2011). It has been shown that contextual cues about the movements of specific portions of the limb in extrinsic coordinates appear to contribute to generalization (Krakauer et al. 2006). It appears that these cues act implicitly, suggesting that although plan-based generalization may be measured via explicit reports of “aiming,” as in the present study, explicit knowledge may not be a prerequisite for some plan-based generalization to occur. We hypothesize that the reference frame of generalization is likely an internal representation of the planned movement of the limb in extrinsic coordinates, and that this plan can, in certain contexts, be “read out” explicitly if subjects are required to do so. How abstract the relevant memory is remains an open question.

In sensorimotor adaptation research, the generalization of adaptation is presented as a spatial phenomenon—the generalization metric reflects the degree of adaptation at naive locations in space relative to a reference point at an overtrained location. This leads to the characterization of generalization in physical space, which can be associated with directional or velocity-tuned neurons in motor cortex (Paz et al. 2003) or the cerebellum (Coltz et al. 1999). However, the generalization of explicit planning may not be a question of representing physical space. The shapes of generalization functions from various domains of psychophysics—for instance, in auditory and visual systems—are strikingly similar to each other, suggesting a universal law of generalization across an abstract psychological dimension (Shepard 1987). It may be that while the Gaussian-like generalization of adaptation is constrained by a spatially tuned population code, the factor that may dictate the center of that function, the explicit plan, is a context-sensitive, abstract psychological construct concerning the dynamics of the learning environment and the goals of the learner. Such a relationship between high-level and low-level representations in sensorimotor learning provides an intriguing example of multisystem interaction in the CNS.

GRANTS

This work was supported by the National Science Foundation (Graduate Research Fellowship to S. D. McDougle), the National Institute of Neurological Disorders and Stroke (Grant R01 NS-084948 to J. A. Taylor), and the Princeton Neuroscience Institute’s Innovation Fund (K. M. Bond and J. A. Taylor).

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the authors.

AUTHOR CONTRIBUTIONS

S.D.M. and J.A.T. conceived and designed research; S.D.M. and K.M.B. performed experiments; S.D.M. analyzed data; S.D.M. and J.A.T. interpreted results of experiments; S.D.M. prepared figures; S.D.M. drafted manuscript; S.D.M., K.M.B., and J.A.T. edited and revised manuscript; S.D.M., K.M.B., and J.A.T. approved final version of manuscript.