Feedforward and Feedback Control Share an Internal Model of the Arm's Dynamics

Abstract

Recent work has shown that, when countering external forces, the nervous system adjusts not only predictive (i.e., feedforward) control of reaching but also reflex (i.e., feedback) responses to mechanical perturbations. Here we show that altering the physical properties of the arm (i.e., intersegmental dynamics) causes the nervous system to adjust feedforward control and that this learning transfers to feedback responses even though the latter were never directly trained. Forty-five human participants (30 females) performed a single-joint elbow reaching task and countered mechanical perturbations that created pure elbow motion. In our first experiment, we altered intersegmental dynamics by asking participants to generate pure elbow movements when the shoulder joint was either free to rotate or locked by the robotic manipulandum. With the shoulder unlocked, we found robust activation of shoulder flexor muscles for pure elbow flexion trials, as required to counter the interaction torques that arise at the shoulder because of forearm rotation. After locking the shoulder joint, which cancels these interaction torques, we found a substantial reduction in shoulder muscle activity over many trials. In our second experiment, we tested whether such learning transfers to feedback control. Mechanical perturbations applied to the arm with the shoulder unlocked revealed that feedback responses also account for intersegmental dynamics. After locking the shoulder joint, we found a substantial reduction in shoulder feedback responses, as appropriate for the altered intersegmental dynamics. Our work suggests that feedforward and feedback control share an internal model of the arm's dynamics.

SIGNIFICANCE STATEMENT Here we show that altering the physical properties of the arm causes people to learn new motor commands and that this learning transfers to their reflex responses to unexpected mechanical perturbations, even though the reflex responses were never directly trained. Our results suggest that feedforward motor commands and reflex responses share an internal model of the arm's dynamics.

Introduction

Humans learn new movement patterns in a variety of novel situations (Wolpert et al., 2011). This ability has been long studied by investigating how people change their predictive (i.e., feedforward) motor commands when reaching in viscous force fields (Shadmehr and Mussa-Ivaldi, 1994) or with altered visuomotor mappings (Cunningham, 1989). In these cases, and in many related paradigms, the motor commands required to achieve the goal of the task are accompanied by errors and the resulting errors induce changes in subsequent motor commands; suggesting that the nervous system updates an internal model of the altered environment (Wolpert et al., 1995).

Recent results suggest that such error-based learning, where participants change their feedforward motor commands to compensate for novel force environments, also changes how the motor system responds to sensory feedback following mechanical perturbations (Wang et al., 2001; Wagner and Smith, 2008; Yousif and Diedrichsen, 2012; Cluff and Scott, 2013). For example, when participants are trained to reach in the presence of force fields and occasionally encounter experimentally applied mechanical perturbations over the course of learning, their feedback responses to the perturbations adapt in parallel with their feedforward motor commands (Wang et al., 2001; Wagner and Smith, 2008; Yousif and Diedrichsen, 2012). Cluff and Scott (2013) and Ahmadi-Pajouh et al. (2012) measured muscle activity during this type of learning and showed that adapted feedback responses to mechanical perturbations could be identified in the so called long-latency stretch reflex epoch, starting as early as ∼50 ms following perturbation onset and including cortical, brainstem, cerebellar and spinal circuits, but not in the purely spinal short-latency stretch reflex epoch, which starts ∼20 ms following perturbation onset (Marsden et al., 1976; Rothwell et al., 1980; Darton et al., 1985; Gielen et al., 1988; Palmer and Ashby, 1992; Pruszynski and Scott, 2012; Cluff et al., 2015; Scott, 2016).

Here we tested whether and to what extent feedback responses to mechanical perturbations adapt when people learn novel arm dynamics rather than new properties of the environment. Briefly, participants moved their hand between targets placed along an arc, so that reaches could be accomplished by rotating only the elbow joint. The same pure elbow movements were made with the shoulder free to rotate or with the shoulder locked by the robotic manipulandum. Making pure elbow movements with the shoulder joint free to rotate requires generating torques at both the shoulder and elbow joints to compensate for interaction torques that arise at the shoulder when the forearm rotates. Making pure elbow movements with the shoulder joint locked removes the need to activate shoulder muscles because locking that joint physically cancels these interaction torques. First, we tested whether people learn these altered arm dynamics during reaching movements and we found that participants reduce shoulder muscle activity following shoulder fixation. This learning was much slower than what is typically observed in paradigms where explicit movement errors occur, akin to how participants slowly optimize muscle recruitment after learning to kinematically counter a divergent force field (Franklin et al., 2004), and perhaps explaining why previous work with similar paradigms to our own did not find learning when performing a few trials (Koshland et al., 1991; Debicki and Gribble, 2005). Second, we tested whether learning new feedforward motor commands in this context transfers to feedback control by interspersing mechanical perturbations over the course of learning. If the neural circuits that implement internal models of the arm for feedforward and feedback control are independent of one another, then learning during feedforward control should not transfer to feedback control. However, we found that learning to reduce feedforward shoulder muscle activation transfers to feedback control even though they were never directly trained, suggesting that these neural mechanisms share an internal model of the arm's dynamics.

Materials and Methods

Subjects.

A total of 45 healthy participants (aged 19–47, 30 females) participated in one of two experiments. All participants reported that they were right-handed and had no history of visual, neurological, or musculoskeletal impairments. Participants provided written consent, were paid for their participation, and were free to withdraw from the experiment at any time. The Office of Research Ethics at Western University approved this study.

Apparatus.

Experiments were performed using the KINARM exoskeleton robot (BKIN Technologies). As previously described (Scott, 1999; Pruszynski et al., 2008, 2009), this robot permits flexion and extension movement of the shoulder and elbow joints in a horizontal plane that intersects the hand, and can independently apply torque at both joints. Visual targets and hand feedback were projected in the horizontal plane of the task via an LCD monitor and a semi-silvered mirror. Direct vision of the arm was prevented with a physical barrier. The two segments of the exoskeleton robot (upper arm and forearm) were adjusted to fit each participant's arm and the spaces left were filled with a firm foam to ensure tight coupling with the links of the robot. The robot was then calibrated so that the projected hand cursor was aligned with each participant's right index finger tip.

Experiment 1: single joint elbow reaches with shoulder fixation.

Twenty participants performed 20° elbow flexion and extension movements with the shoulder joint either free to move or with the shoulder fixed by a physical clamp attached to the linkages of the robotic manipulandum. At the beginning of each trial, participants moved their hand to a home target (red circle, 0.6 cm diameter). The home target position corresponded to a participant's hand cursor position when their shoulder and elbow joints were at 10° and 60° (external angles), respectively (Fig. 1, top left). After maintaining their hand at this location for a random period (250–500 ms, uniform distribution), a goal target (white circle: 3 cm diameter) was presented in a location that could be reached with a 20° pure elbow flexion movement. The goal target then turned red after another random period (250–500 ms, uniform distribution), cueing the participant to start their movement. At the same time, the hand feedback cursor was extinguished and remained off for the duration of the movement. Participants were instructed to move to the goal target and to do so within a specific time window. The goal target turned green when movement time (from exiting the home target to entering the goal target) was between 100 and 180 ms, orange if it was too fast (<100 ms) and red if it was too slow (>180 ms). No restrictions were placed on movement trajectories. In addition to timing constraints between targets, participants were instructed to remain at the goal target for an additional 500 ms to finish a trial. After a random period (0–1 s, uniform distribution), the goal target became a new home target (0.6 cm diameter) and the same procedure was repeated but for an extension movement.

Figure 1.

Experimental setup. In Experiments 1 and 2, participants were presented with a peripheral target that could be achieved with 20° of elbow flexion rotation. Participants were instructed to perform fast and accurate reaching movements to this peripheral target and did so with their shoulder joint unlocked and locked (top left column). In Experiment 2, in addition to reaching trials, mechanical perturbations were sometimes applied (probes) to test the sensitivity of feedback responses over the course of learning (bottom left column). Red and blue arrows represent the direction of the multi-joint step-torques applied to the shoulder and elbow joints. Illustrations of the protocols for Experiments 1 and 2 are shown on the right. In Experiments 1 and 2, participants performed 300 baseline trials with the shoulder joint unlocked, 1100 adaptation trials with the shoulder joint locked, and 300 post-adaptation trials with the shoulder joint unlocked. Multi-joint perturbations (probes, red and blue tick marks) were applied in 15% of all trials in Experiments 2.

Participants first completed 300 flexion and extension baseline trials, with the shoulder joint free to move. We then mechanically locked the shoulder joint with a physical clamp and participants repeated the same flexion and extension movements for 1100 trials (adaptation phase). Last, we unlocked the shoulder joint and participants again generated the same flexion and extension movements for 300 trials (post-adaptation phase; Fig. 1, top right).

Experiment 1 lasted ∼2.5 h. Rest breaks were given throughout or when requested. Before data collection participants completed practice trials until they comfortably achieved ∼80% success rates (∼5 min).

Experiment 2: transfer to feedback control.

Twenty participants performed 20° elbow flexion and extension movements with the shoulder joint free to move and with the shoulder fixed via a servo-controller acting at the shoulder joint, and occasionally countered mechanical perturbations that caused pure elbow motion. As in Experiment 1, participants moved their hand to a home target (white circle, 0.6 cm diameter) to start a trial. The home target position corresponded to a participant's fingertip when their shoulder and elbow joints were at 45° and 60° (external angles), respectively (Fig. 1, bottom left). After a random period (250–500 s, uniform distribution), a goal target (white circle, 3 cm diameter) was presented at a location that could be reached with a 20° pure elbow flexion movement. The goal target turned red after another random period (0–1 s, uniform distribution), cueing the participant to start their movement. At the same time, the hand feedback cursor was turned off and remained off for the duration of the movement. As in Experiment 1, participants were instructed to move from the home target to the goal target with a movement time ranging from 100 to 180 ms and remain in the goal target for an additional 400 ms to finish the trial. The goal target turned green when the movement was successful, orange when movement was too fast or red when it was too slow. After a random period (0–1 s, uniform distribution), the goal target became a new home target (0.6 cm diameter) and the same procedure was repeated but for an extension movement.

Participants first completed a total of 300 flexion and extension baseline trials, with the shoulder joint free to move. We then locked the shoulder joint with a servo-controller, and participants repeated the same flexion and extension movements for 1100 trials (adaptation phase). The force channel, implemented as a stiff, viscous spring and damper in the direction orthogonal to the shoulder joint (K = 1000 N/m and B = 250 N/(m/s)), effectively counteracted rotation of the shoulder joint (average maximum absolute deviation, ∼2°) and also clamped reaching trajectories. Last, we unlocked the shoulder joint and participants again generated the same flexion and extension movements for 300 trials (post-adaptation phase; Fig. 1, bottom right).

Perturbation trials occurred in 15% of all trials (Fig. 1, bottom left, probes). In these trials, when the hand cursor entered the home target, the exoskeleton gradually applied (over 2 s) a background torque of (−2/+2 N · m) to the elbow to ensure baseline activation of shoulder and elbow muscles. After maintaining the cursor in the home target for a randomized duration (1.0–2.5 s, uniform distribution), a step-torque (i.e., perturbation) was applied to the shoulder and elbow joints (2 N · m at each joint over and above the background torque), which displaced the participant's hand outside the home target. Critically, the servo-controller was turned off at perturbation onset and we chose this combination of shoulder and elbow loads to minimize shoulder motion (Kurtzer et al., 2008; Maeda et al., 2017). Participants were instructed to quickly counter the load and bring their hand back to the goal target (centered on the home target). If the participant returned to the goal target within 385 ms of perturbation onset, the target circle changed from white to green, otherwise the target circle changed from white to red. In 5% of all trials, the background torques turned on, remained on for the same time period (1.0–2.5 s, uniform distribution), but then slowly turned off, after which participants were still required to perform the reaching movements. These trials ensured that background loads were not always predictive of perturbation trials.

The order of all perturbation, control, and reaching trials was randomized in the baseline and post-adaptation phases and pseudo randomized in blocks of every 22 trials in the adaptation phase. Experiment 2 lasted ∼2.5 h. Rest breaks were given throughout or when requested. Before data collection participants completed practice trials until they comfortably achieved ∼80% success rates (∼5 min).

Five additional participants performed the same version of this Experiment 2 without locking the shoulder joint. This served as a control for both Experiments 1 and 2 to rule out changes in feedforward or feedback changes caused by extensive practice rather than the shoulder locking manipulation.

Kinematic recordings and analysis.

Movement kinematics (i.e., hand position, and joint angles) were sampled at 1000 Hz and then low-pass filtered (12 Hz, 2-pass, fourth-order Butterworth). In Experiment 1, all data were aligned on movement onset in Experiment 1. In Experiment 2, data from reaching trials were aligned on movement onset and data from perturbation trials were aligned on perturbation onset. Movement onset was defined as 5% of peak angular velocity of the elbow joint (Gribble and Ostry, 1999; Maeda et al., 2017). We quantified the adaptation and aftereffects of reaching movements following shoulder fixation using hand path errors relative to the center of the target at 80% of the movement between movement onset and offset (also defined at 5% of peak angular velocity of the elbow joint). This corresponds to 170 ms (SD 15 ms) duration. This window was chosen to select the kinematic traces before any feedback corrections.

EMG recordings and analysis.

We measured electromyographic signals from upper limb muscles using surface electrodes (Delsys Bagnoli-8 system with DE-2.1 sensors). Electrodes were placed on the skin surface overlying the belly of five muscles [pectoralis major clavicular head (PEC), shoulder flexor; posterior deltoid (PD), shoulder extensor; biceps brachii long head (BB), shoulder and elbow flexor, brachioradialis (BR), elbow flexor; triceps brachii lateral head (TR), elbow extensor]. Before electrode placement, the participants' skin was abraded with rubbing alcohol, and the electrodes were coated with conductive gel. Electrodes were placed along the orientation of muscle fibers. A reference electrode was placed on the participant's left clavicle. EMG signals were amplified (gain = 10³), and then digitally sampled at 1000 Hz. EMG data were then bandpass filtered (20–500 Hz, 2-pass, second-order Butterworth) and full-wave rectified.

In Experiment 1, we investigated whether shoulder muscles adapt to novel intersegmental dynamics following shoulder fixation. To compare the changes in amplitude of muscle activity over time and across different phases of the protocol, we calculated the mean amplitude of phasic muscle activity across a fixed time window, −100 to +100 ms relative to movement onset, as has been done previously (Debicki and Gribble, 2005; Maeda et al., 2017). These windows were chosen to capture the agonist burst of EMG activity in each of the experiments, but our results did not qualitatively change with small changes in this averaging window.

In Experiment 2, we investigated whether feedback responses in shoulder muscles also adapt to the novel intersegmental dynamics following shoulder fixation. To test whether the short and long latency stretch response of shoulder flexors account for and adapt to novel intersegmental dynamics, we binned the PEC EMG into previously defined epochs (Pruszynski et al., 2008). This included a pre-perturbation epoch (PRE; −50–0 ms relative to perturbation onset), the short-latency stretch response (25–50 ms), the long-latency stretch response (50–100 ms), and the voluntary response (100–150 ms).

Normalization trials before each experiment were used to normalize muscle activity such that a value of 1 represents a given muscle sample's mean activity when countering a constant 1 N·m torque (Pruszynski et al., 2008; Maeda et al., 2017). Data processing was performed using MATLAB (r2016b, MathWorks). For simplicity, here we only report the results of flexion movements and for feedback responses in shoulder flexors, however, the results are similar for the extension.

Experimental design and statistical analyses.

All statistical analyses were performed using R v3.2.1. We performed different statistical tests (e.g., repeated-measures ANOVA with Tukey tests for multiple comparisons, t test, and regression analysis), when appropriate in each of the two experiments. Details of these procedures are provided in Results. Experimental results were considered statistically significant if the corrected p value was <0.05.

Results

Experiment 1: single-joint elbow reaching with shoulder fixation

Participants (N = 20) had no difficulty learning the imposed speed and accuracy constraints and achieved >90% success within 5 min of practice. Although never instructed to do so, even with the shoulder free to move participants moved from the start target to the goal target by almost exclusively rotating their elbow joint (Fig. 2A). Despite minimal shoulder rotation, we found substantial shoulder flexor muscle activity before movement onset, as required to compensate for the torques that arise at the shoulder when the forearm rotates (Gribble and Ostry, 1999; Maeda et al., 2017; Fig. 2B).

Figure 2.

Compensating for intersegmental dynamics during self-initiated elbow reaches. A, Average kinematics of the shoulder (black) and elbow (blue) joints for elbow flexion movements with the shoulder joint free to move (i.e., baseline trials). Shaded areas represent the standard error of the mean (SEM). Data are aligned on movement onset. B, Solid and dashed lines represent average agonist (PEC) and antagonist (PD) muscle activity during flexion movements, respectively. Shaded areas represent the SEM. EMG data are normalized as described in Materials and Methods. Data are aligned on movement onset.

After the baseline trials, we physically locked the shoulder joint of the KINARM with a physical clamp attached to the robotic manipulandum, a manipulation that eliminated all the torques that needed to be countered by muscles spanning the shoulder joint when the forearm rotated. Locking the shoulder joint did not alter task performance, with participants continuing to demonstrate >90% success rates.

We first tested whether the nervous system adapts to the experimentally imposed intersegmental dynamics by reducing shoulder muscle activity. Figure 3A illustrates mean shoulder flexor muscle activity, in a fixed time window (−100 to 100 ms; see Materials and Methods) relative to movement onset, across trials both before (i.e., baseline epoch) and after (i.e., adaptation epoch) the shoulder was physically locked. The magnitude of shoulder muscle activity appeared to slowly decrease over the course of the adaptation trials. We then removed the shoulder lock and found that shoulder muscle activity quickly returned to baseline levels (Fig. 3A, post-adaptation trials). Indeed, a one-way ANOVA comparing shoulder flexor muscle activity late (last 25 trials) in the baseline, adaptation and post-adaptation phases revealed a reliable effect of phase on shoulder muscle activity (F_(2,38) = 17.103, p < 0.0001). Tukey post hoc tests showed that shoulder flexor muscle activity reliably decreased by 36% relative to baseline (p = 0.0001) when the shoulder was locked and then reliably increased again after the shoulder was unlocked, returning to that seen in baseline trials (p = 0.5; Fig. 3B,C). Note that we found no corresponding changes in monoarticular elbow BR muscle activity as a function of epoch (one-way-ANOVA, F_(2,36) = 1.505, p = 0.236; Fig. 3D–F) and no changes in the biarticular BB muscle activity as a function of epoch (one-way-ANOVA, F_(2,38) = 0.044, p = 0.837; Fig. 3G–I). Moreover, in a control experiment where participants (N = 5) performed the same number of total trials but never with the shoulder locked, we found no reliable decrease in shoulder muscle activity over trials corresponding to the phases of the main experiment (F_(2,8) = 1.307 p = 0.323; Fig. 3A, gray error bars). Indeed, shoulder muscle activity at the end of the adaptation phase in the main experiment was reliably smaller than at the equivalent point in the control experiment (t₍₁₉₎ − 3.7, p = 0.001), indicating that shoulder muscle activity decay is related to shoulder fixation.

Figure 3.

Learning novel intersegmental dynamics following shoulder fixation. A, Average PEC muscle activity in a fixed time window (−100 to 100 ms relative to movement onset). Each data bin is the average of five trials. Shaded areas represent the SEM. EMG data are normalized as described in Materials and Methods. Error bars plotted between the epochs represent the mean and SE of the last five bins of trials in each phase contrasted with the respective bins in the control experiment in gray. B, Time series of PEC normalized muscle activity averaged over the last 25 baseline and adaptation trials. Data are aligned on movement onset. Shaded areas represent the SEM. C, Average PEC muscle activity in a fixed time window (−100 to 100 ms relative to movement onset) associated with these trials late in the baseline, adaptation and post-adaptation phases. Each dot represents data from a single participant. Asterisks indicate reliable effects (p < 0.05; see main text). D–F, Data for elbow BR muscle are shown using the same format as A–C. G–I,Data for BB muscle are shown using the same format as A–C.

In addition to reduced shoulder muscle activity, another indication that participants learned the novel arm dynamics we imposed was the presence of robust after-effects. That is, in the early post-adaptation phase, participants produced substantial reaching errors in the direction that one would predict if they failed to compensate for the unlocked intersegmental dynamics (Fig. 4A). Unlike the introduction of the shoulder lock, the kinematic after-effects present when the shoulder was unlocked resulted in participants not achieving the speed and accuracy constraints imposed in our task; that is, participants made errors when the shoulder was unlocked. We performed a one-way ANOVA to compare reach accuracy (measured as distance from the center of the goal target) of trials late in the baseline phase (last 25 trials), trials early in the post-adaptation phase (first 3 trials), and trials late (last 25 trials) in the post-adaptation phase (Fig. 4B). Note that we chose a smaller bin size early in the post-adaptation because the return to baseline after unlocking the shoulder joint happens quickly (Fig. 3A). We found a significant effect of phase (F_(2,38) = 4.65, p = 0.01). Tukey post hoc tests showed that movement errors increased by 42% (p = 0.03) from baseline to early post-adaptation and returned to baseline levels (p = 0.97) in late post-adaptation trials.

Figure 4.

Movement trajectories after adapting elbow reaches. A, Average hand trajectories late in the baseline (25 trials) and early in the adaptation trials (first 3 trials). Each dot represents data from a single participant. B, Average error between hand position at movement offset to the center of the target in the last 25 trials in the baseline, first 3 trials early in the post-adaptation, and last 25 trials late in post-adaptation phases (p < 0.05; see main text). Asterisks indicate reliable effects. Each dot represent data from a single participant.

Experiment 2: transfer to feedback control

Participants (N = 20) again moved their hand between two targets that required 20° of elbow flexion (Fig. 5A). Participants had no difficulty with the imposed speed and accuracy constraints of the task and achieved >90% of success within 5 min of practice. In addition to these reaching trials, we occasionally applied mechanical perturbations to the arm while participants maintained their hand in the home target. The mechanical perturbations consisted of step torques applied simultaneously to the shoulder and elbow chosen so they caused minimal shoulder motion but different amounts of elbow motion (Fig. 5B).

Figure 5.

Compensating for intersegmental dynamics during self-initiated reaches and when responding to perturbations. A, Average kinematics of the shoulder (dashed) and elbow (solid) joints for elbow flexion trials. Shaded areas represent the SEM. Data are aligned on movement onset. B, Dashed and solid lines represent average agonist (PEC) and antagonist (PD) muscle activity associated with the movement in A. Data are aligned on movement onset. Shaded areas represent SEM. C, Average kinematics of the shoulder (dashed) and elbow (solid) joints following mechanical perturbations. Red and blue traces are from the shoulder/elbow extensor torque and shoulder/elbow flexor torque conditions, respectively. Shaded areas represent SEM. Inset, The amount of shoulder and elbow displacement 50 ms post-perturbation (data are shown for all subjects). D, Normalized shoulder muscle activity associated with C. Shaded region indicates the Long-Latency epoch (LLR). Shaded areas represent SEM.

As shown in Experiment 1 (Fig. 2) and as previously demonstrated, participants completed the reaching trials almost exclusively by rotating their elbow joint and, in so doing, generated a substantial amount of shoulder muscle activity (Fig. 5C; Gribble and Ostry, 1999; Maeda et al., 2017). Also, as previously demonstrated, we found that mechanical perturbations that created pure elbow motion elicited substantial shoulder muscle activity in the long-latency epoch (Kurtzer et al., 2008; Maeda et al., 2017; Fig. 5D), as appropriate for countering the imposed joint torques.

We then tested whether, as in Experiment 1, feedforward commands to shoulder muscles adapt when participants produce the same elbow rotations following shoulder fixation. Consistent with Experiment 1, we found a reliable effect of phase on shoulder muscle activity (F_(2,38) = 19.7 p < 0.0001) with Tukey post hoc tests showing that PEC muscle activity decreased by 41% (p = 0.0001) following shoulder fixation and increased when removing the shoulder clamp by 31% (p = 0.001). We again found no corresponding changes in monoarticular elbow BR muscle activity as a function of epoch (one-way-ANOVA, F_(2,38) = 0.075, p = 2.761) and no changes in biarticular BB muscle activity as a function of epoch (one-way ANOVA, F_(2,38) = 3.04, p = 0.06). Participants again showed evidence of after-effects in the early post-adaptation phase, producing substantial reaching errors in the direction required to compensate for intersegmental dynamics (F_(2,38) = 14.4, p < 0.001) with Tukey post hoc tests revealing that movement errors increased by 50% (p < 0.001) from baseline to early post-adaptation and returned to baseline levels (p < 0.61) in late post-adaptation trials.

The main goal of Experiment 2 was to examine whether learning novel intersegmental dynamics following shoulder fixation during feedforward control also modifies the sensitivity of sensory feedback responses to mechanical perturbations. We tested this idea by occasionally applying mechanical perturbations across all phases of the protocol. If the shoulder joint was locked, we unlocked the shoulder joint and then applied mechanical perturbations at the shoulder and elbow joints that created pure elbow motion. Thus, all joints were free to rotate in perturbation trials across all phases of the protocol such that participants only experienced altered intersegmental dynamics in reaching trials.

Figure 6A illustrates group mean shoulder (PEC) muscle activity in the long-latency epoch over trials in the baseline, adaptation and post-adaptation phases of the protocol. Red and blue traces indicate the shoulder/elbow extensor torque, and shoulder/elbow flexor torque conditions (PEC EMG excitatory and inhibitory loads). We first took the difference between excitatory and inhibitory traces as a metric of sensitivity to intersegmental dynamics. We then used a one-way ANOVA to compare this difference of PEC muscle activity in the long-latency epochs, averaged in last five trials of the baseline, adaptation and post-adaptation phases. We found a reliable effect of phase (F_(2,38) = 9.851, p < 0.0001). Consistent with our prediction, Tukey post hoc tests showed that the difference in PEC muscle activity in the long-latency epoch decreased by 48% (p = 0.001) following shoulder fixation and returned to baseline levels in the post-adaptation phase (p = 0.73; Fig. 6B,C). We performed the same analysis to assess for changes also in short-latency epoch, but we found no reliable differences (F_(2,38) = 0.236, p < 0.791). Importantly, we tested whether there was a change in baseline EMG activity pre-perturbation across phases, which could potentially explain these changes in EMG in the long-latency epoch (gain scaling; Pruszynski et al., 2009). We used a one-way ANOVA to compare the baseline activity of PEC muscle activity pre-perturbation as a function of experimental phase and found no reliable effect (F_(2,38) = 1.649 p = 0.206; Fig. 6B, inset). We also found no corresponding changes in both the long-latency (one-way-ANOVA, F_(2,38) = 0.439, p = 0.648) and short-latency epochs (one-way-ANOVA, F_(2,38) = 0.809, p = 0.453) of the monoarticular elbow BR muscle activity as a function of epoch (Fig. 6D–F). There was also no change in the baseline activity of BR muscle activity pre-perturbation as a function of the learning phases (one-way-ANOVA, F_(2,38) = 0.1695, p = 0.197; Fig. 6E). Moreover, in a control experiment where participants (N = 5) performed the same number of total trials but never with the shoulder locked, we found no decrease in the long-latency epoch over trials corresponding to the phases of the main experiment (F_(2,8) = 0.449 p = 0.653).

Figure 6.

Rapid feedback responses following learning during self-initiated reaching with shoulder fixation. A, Average shoulder EMG (filtered and rectified) in the long latency epoch (50–100 ms) across trials. Red and blue traces indicate the shoulder/elbow extensor torque (excitatory), and shoulder/elbow flexor torque conditions (inhibitory), respectively. Vertical dashed line separates perturbation trials that happened in the baseline, adaptation and post-adaptation phases. B, Five perturbation trials of the difference of PEC muscle activity (excitatory-inhibitory) averaged in late baseline and late adaptation trials. Shaded areas represent SEM. Baseline pre-activity in these trials is shown in the inset. Each data represent a single participant. C, Long-latency epoch of the difference of PEC muscle activity (excitatory-inhibitory) in the baseline, adaptation, and post-adaptation phases. (p < 0.05; see main text). Each data point represents a single participant. D–F, Data for elbow BR muscle are shown using the same format as A–C. Asterisks indicate reliable effects.

Finally, we tested whether there was a correlation between learning effects associated with feedforward motor commands and feedback responses. We found a reliable linear relationship between the decrease in shoulder muscle activity during elbow rotations while the KINARM shoulder joint is fixed, and the decrease in shoulder muscle activity measured in the long-latency epoch of perturbation trials (Slope = 0.45, r = 0.47, p = 0.03; Fig. 7).

Figure 7.

Correlates of reach adaptation and feedback responses. Vertical axis is the percentage change in the long-latency epoch responses (50–100 ms) between baseline and late adaptation phase. Horizontal axis is the percentage change in muscle activity during elbow reaching trials (−100 to 100 ms relative to movement onset). Each data point represents a single participant.

Discussion

In Experiment 1, we tested whether the nervous system learns novel intersegmental dynamics during feedforward control. Consistent with this idea, we found that agonist shoulder muscle activity during elbow rotations slowly decreased after the shoulder was mechanically fixed by a robotic manipulandum. We also found systematic trajectory errors (i.e., after-effects) after removing shoulder fixation. In Experiment 2, we tested whether learning novel intersegmental dynamics during feedforward control transfers to feedback control. Consistent with this idea, we found that shoulder muscle activity in the long-latency stretch epoch slowly decreased after the shoulder was mechanically fixed, even though the long-latency stretch response was never directly exposed to the novel intersegmental dynamics. Together, our results demonstrate that the nervous system learns novel intersegmental dynamics following shoulder fixation, and that this learning transfers from feedforward to feedback control.

Learning novel intersegmental limb dynamics during feedforward control

Intersegmental dynamics complicates the mapping between joint torques and joint motions. One way to study how the nervous system learns and compensates for intersegmental dynamics is by experimentally manipulating these rotational forces during movement. For example, Sainburg et al. (1999) investigated whether the nervous system learns novel intersegmental dynamics in an error-based paradigm by changing the center of mass of the forearm and thus altering intersegmental dynamics during reaching. Consistent with many other error-based learning paradigms (Wolpert et al., 2011), reaching was initially altered, but participants were able to correct their movement trajectories over a relatively small number of trials, and they showed large trajectory errors after removing the added mass (i.e., after-effects). Debicki and Gribble (2005) took another approach. They explored how the nervous system learns novel intersegmental dynamics by asking participants to generate pure elbow flexion movements with the shoulder joint free to move or mechanically fixed. Fixing the shoulder joint removes the interaction torques that arise at the shoulder because of forearm rotation and thus removes the need to activate shoulder muscles. In this paradigm, participants do not make substantial kinematic errors when the shoulder is fixed so reducing shoulder muscle activity is not strictly necessary to achieve the task. Indeed, they report a fixed coupling between shoulder and elbow muscles, indicating that participants did not adapt shoulder muscle activity after it is fixed. Wrist muscle activity also appears to remain unchanged when the wrist joint is fixed during elbow rotation (Koshland et al., 1991). Using the same paradigm, we show that such learning does happen: shoulder muscle activity decreases while the elbow muscle activity remains the same, and people show kinematic after-effects. However, learning unfolds over a timescale much longer than previously examined and what is typical for error-based learning experiments. Even over 550 trials learning was incomplete and shoulder muscle activity did not reach steady-state.

Why was learning incomplete? The most likely possibility is that learning new intersegmental dynamics simply takes place over much longer time scale (e.g., over development) so we did not observe complete learning even with our extended protocol. Indeed, in our results, learning did not reach a clear steady-state suggesting that learning was still ongoing (Fig. 3A). Alternatively, the burst of shoulder muscle activity that is observed just before elbow rotation may partially reflect a hard-wired synergy between shoulder and elbow muscles that makes a complete dissociation impossible (Koshland et al., 1991; Debicki and Gribble, 2005; de Rugy et al., 2012). Insight into these possibilities will ultimately come from long-term learning studies, either by having participants come back over many sessions and exploiting extended periods of practice or by looking at muscle coordination in people who have their joints immobilized for long periods of time (e.g., after bone fracture).

Our finding that learning without kinematic errors is slow is consistent with other studies examining how motor learning, in the context of reaching, evolves in the absence of kinematic errors (Diedrichsen et al., 2010; Vaswani and Shadmehr, 2013). For example, Vaswani and Shadmehr (2013) used a force channel to clamp trajectory errors to zero after participants had learned a force field and showed that memories of the force field fade on a time scale much longer than the original learning. Others have emphasized that motor learning continues to optimize motor programs after kinematic errors have been eliminated, presumably to make the movement more efficient and reduce metabolic cost (Takahashi et al., 2006; Emken et al., 2007; Huang and Ahmed, 2014). From the perspective of the equilibrium point hypothesis (Feldman, 1986), slowly eliminating shoulder muscle activity may reflect the slow realignment of the arm's referent configuration with its actual configuration (Ambike et al., 2016; Parsa et al., 2016). Although learning without kinematic errors usually happens slowly, this is not always the case. For example, Cordo and Nashner (1982) asked participants to push and pull a handle while standing freely and with their trunk stabilized, quite analogous to our own study. They found a reduction in automatic postural responses after only a few trials of trunk stabilization. This may be a special case where the context is very explicit and the switch well practiced (because people often have their trunk stabilized when sitting), and where the potential energy savings associated with not activating core muscles is relatively high.

Although our results show that people adjust their muscle patterns to novel intersegmental dynamics, it is unclear whether they are updating an internal model of their arm, learning a new internal model of the environment as in force-field learning (Shadmehr and Mussa-Ivaldi, 1994), learning a new internal model of the exoskeleton by treating it as a tool (Heald et al., 2018), or some combination of all of these. Several aspects of our experimental approach suggest that participants are updating an internal model of their arm. First, our loads are in a body-centric reference frame. Second, our participants are not grasping the handle of the robot as would be typical for tool use. Third, our approach did not cause kinematic errors and participants learned slowly compared with many previous studies exposing people to new force environments (Shadmehr et al., 2010) and tools. One way of teasing these possibilities apart is with neurophysiological studies in humans and animals. For example, it has been proposed that the cerebellum houses multiple internal models (Wolpert and Kawato, 1998), which may be localized in different parts of the cerebellum (Imamizu et al., 2000, 2003; Imamizu and Kawato, 2012) and differentially govern learning new models of the limb, the environment, and tools.

Learning novel intersegmental limb dynamics during feedforward control transfers to feedback control

There is growing evidence that motor learning can modify how the motor system responds to sensory feedback (Wang et al., 2001; Kimura et al., 2006; Wagner and Smith, 2008; Ahmadi-Pajouh et al., 2012; Yousif and Diedrichsen, 2012; Cluff and Scott, 2013). In previous studies, participants made reaching actions in the presence of force fields that caused kinematic errors and experienced occasional mechanical perturbations to probe feedback responses over the course of learning. These studies convincingly show that, when feedforward motor commands adapt, feedback responses to mechanical perturbations also adapt. Our results reveal that the same is true when people learn new intersegmental dynamics in the absence of systematic kinematic errors. Since we ensured that fast feedback responses were never exposed to the learning context, the transfer we report shows that feedforward and feedback responses have access to a shared internal model of the arm's dynamics.

Internal models are a central concept in motor control, including actions like reaching, grasping and object manipulation (Wolpert et al., 2011). They enable the nervous system to predict the consequences of the motor commands it generates and to determine which motor commands are required to execute a particular action, critical computations for stable and accurate control given various sources of noise, and delays in the sensorimotor system (Wolpert et al., 1995; Harris and Wolpert, 1998). Although internal models have been extensively studied in the context of feedforward motor commands, feedback responses also rely on an internal model that has many of the key features of feedforward motor commands (Lacquaniti and Soechting, 1986; Kurtzer et al., 2008, 2009, 2014; Pruszynski et al., 2011; Crevecoeur and Scott, 2013, 2014; Weiler et al., 2016; Maeda et al., 2017). For example, long-latency stretch responses respond to the expected future kinematic state of the arm by integrating incoming sensory information with prior knowledge about the mechanical perturbations encountered in the environment (Crevecoeur and Scott, 2013).

To our knowledge, no studies have directly addressed the neural mechanisms that underlie shared internal models for feedforward and feedback control and this is an important gap in the literature. However, some of the same neural structures have been implicated in housing internal models for feedforward or feedback control in isolation (for review, see Kurtzer, 2014). One likely locus is the primary motor cortex (M1). In terms of feedforward control, Gritsenko et al. (2011) used transcranial magnetic stimulation applied to human primary motor cortex while participants reached to targets placed at locations that yielded assistive or resistive interaction torques between the arm and forearm. Their results showed that motor-evoked potentials were greater for movement directions that included resistive interaction torques compared with assistive movements, indicating that M1 mediates feedforward compensation for the arm's intersegmental dynamics. In terms of feedback control, Pruszynski et al. (2011) showed that transcranial magnetic stimulation applied to human M1 potentiates shoulder muscle responses following mechanical perturbations that cause pure elbow displacement, indicating that M1 mediates feedback compensation for the arm's intersegmental dynamics. In fact, many of the single neurons in monkey M1 that are activated during feedforward generation of motor commands also respond to mechanical perturbations (Evarts, 1973; Evarts and Tanji, 1976; Wolpaw, 1980; Evarts and Fromm, 1981; Picard and Smith, 1992; Herter et al., 2009; Pruszynski et al., 2011, 2014; Omrani et al., 2014; for review, see Pruszynski, 2014). Although it is unknown whether and how these specific neurons modify their responses in the context of motor learning, primary motor cortex is intimately involved in motor learning so such overlap is plausible if not likely (for review, see Sanes and Donoghue, 2000; Kawai et al., 2015). Another likely locus is the cerebellum, which, at the highest level, is thought to contain the internal models that underlie feedforward motor control (Wolpert et al., 1998). Consistent with this role, damage to the cerebellum yields profound deficits coordinating the joints without affecting the ability to generate the required forces (Holmes, 1939; Goodkin et al., 1993; Bastian et al., 1996, 2000). Cerebellum also contributes to feedback control. Neurons in the dentate and interpositus nuclei of the cerebellum rapidly respond to mechanical perturbations in a goal-dependent manner (Strick, 1983) and long-latency stretch responses are reduced in patients with cerebellar dysfunction (Hore and Vilis, 1984; Kurtzer et al., 2013). Of course, these cerebellum and primary motor cortex do not act in isolation and an important line of future research is precisely delineating their interactions along with other cortical and brainstem contributors.