Impedance control and internal model use during the initial stage of adaptation to novel dynamics in humans

Abstract

This study investigated the neuromuscular mechanisms underlying the initial stage of adaptation to novel dynamics. A destabilizing velocity-dependent force field (VF) was introduced for sets of three consecutive trials. Between sets a random number of 4–8 null field trials were interposed, where the VF was inactivated. This prevented subjects from learning the novel dynamics, making it possible to repeatedly recreate the initial adaptive response. We were able to investigate detailed changes in neural control between the first, second and third VF trials. We identified two feedforward control mechanisms, which were initiated on the second VF trial and resulted in a 50% reduction in the hand path error. Responses to disturbances encountered on the first VF trial were feedback in nature, i.e. reflexes and voluntary correction of errors. However, on the second VF trial, muscle activation patterns were modified in anticipation of the effects of the force field. Feedforward cocontraction of all muscles was used to increase the viscoelastic impedance of the arm. While stiffening the arm, subjects also exerted a lateral force to counteract the perturbing effect of the force field. These anticipatory actions indicate that the central nervous system responds rapidly to counteract hitherto unfamiliar disturbances by a combination of increased viscoelastic impedance and formation of a crude internal dynamics model.

Humans have the ability to adapt relatively quickly to changes in the magnitude or nature of external forces. However, acquiring the same proficiency and skill with a heavier tennis racquet or a more responsive keyboard may require considerable practice. Studies investigating how humans compensate for novel dynamics have shown that compensatory forces are learned as predictive feedforward motor commands to replace reactive feedback motor commands (Lackner & Dizio, 1994; Shadmehr & Mussa-Ivaldi, 1994; Brashers-Krug et al. 1996; Krakauer et al. 1999). It is clear from the way in which compensation is generalized that it does not simply entail memorization of a mirror-image force–time profile, but represents a computational process based on a model of the dynamics of the disturbance (Conditt et al. 1997; Goodbody & Wolpert, 1998; Bhushan & Shadmehr, 1999).

The cerebellum and primary motor cortex appear to be the regions of the brain most directly implicated in adaptation to novel dynamics. In particular, the ipsilateral cerebellum shows changes in regional cerebral blood flow during adaptation to novel dynamics, that appear to be related to changes in motor error (Nezafat et al. 2001). Furthermore, individuals with cerebellar atrophy are less able to adapt to novel dynamics than control subjects (Maschke et al. 2004), and individuals with cerebellar lesions do not update motor commands based on past error (Smith & Shadmehr, 2005) unlike control subjects (Thoroughman & Shadmehr, 2000; Scheidt et al. 2001; Donchin et al. 2003). Evidence for involvement of primary motor cortex is based primarily on single unit recordings from non-human primates which identified a class of neurones that shifted their preferred direction during exposure to novel dynamics and retained this shift during a subsequent washout period with the original dynamics (Li et al. 2001). However, there are strong projections from the cerebellum to primary motor cortex so it is possible that the changes in motor cortex are the consequence of parallel changes in cerebellum. At the molecular level, NMDA receptors and GABAergic inhibition have been implicated in the acquisition of new motor memories (Donchin et al. 2002), although their localization was not possible.

In the case of adaptation to novel dynamics, where movements are mechanically stable, changes in muscle activation patterns closely mirror the adaptive changes in joint torques required to produce the necessary compensatory forces. Nonetheless, there is excess agonist–antagonist muscle cocontraction, particularly in the early stages of learning. As adaptation progresses, this cocontraction decreases (Thoroughman & Shadmehr, 1999; Osu et al. 2002; Franklin et al. 2003a). It would appear that the central nervous system initially uses cocontraction to increase resistance to the disturbing effects of the novel dynamics, and reduces cocontraction as knowledge of the dynamics improves. Improvement in performance during adaptation to novel dynamics is an exponential process (Thoroughman & Shadmehr, 1999; Osu et al. 2003; Franklin et al. 2003a), where the largest increment occurs between the first and second trial. The mechanisms responsible for this dramatic improvement in performance have yet to be examined in detail. We hypothesized that it involved a change in feedforward motor commands to both increase in the neuromuscular impedance of the arm and exert a force that countered the perturbing force of the altered environmental dynamics. To demonstrate this we devised a paradigm in which we intermittently exposed subjects to novel dynamics for three trials at a time so as to focus on the early changes to motor commands while preventing consolidation of these changes.

Two paradigms have been introduced to explore the process of internal dynamics model formation. One paradigm involves reverting to the dynamics that existed prior to adaptation and observing the after effects of the adaptation (Shadmehr & Mussa-Ivaldi, 1994). The other paradigm involves mechanically constraining the movement path to what is presumed to be the desired path and measuring the force generated at the point of interaction between the limb and the constraint (Scheidt et al. 2000). If adaptation occurred primarily by increasing limb mechanical impedance through cocontraction of antagonistic muscle groups, then both after effects and constraint forces should be small. We employed both of these paradigms, but also recorded the electromyogram (EMG) of relevant arm muscles, which provides a more direct measure of adaptation in the control signal. Because of the stochastic nature of processes that contribute to the EMG, an accurate representation of the control signal can only be obtained by averaging over many repetitions of the same condition. This would normally present a problem in investigating adaptive changes that are responsible for the reduction in error occurring during the first few movements after the dynamics of a task change, since averaging successive trials precludes being able to detect successive changes in command signals. By intermittently exposing subjects to the novel dynamics for several trials so that consolidation did not occur, we were able to average the EMG over many trials to reduce its variance. We addressed the questions of how kinematic error is reduced from one trial to the next, how feedback commands are transformed into feedforward commands, and the relative roles of increased mechanical impedance (stiffness) and internal model formation.

Methods

Eight subjects, six male and two female, participated in this study. All subjects gave informed written consent prior to participating in the study. The study was approved by the institutional ethics review committee and conformed to the Declaration of Helsinki.

Protocol

Subjects sat in a chair with a shoulder harness to constrain trunk motion. The forearm and wrist were stabilized by a thermoplastic splint rigidly attached to the handle of the parallel-link direct-drive air and magnet floating manipulandum (PFM). Details of its design and operation have been previously described (Gomi & Kawato, 1996, 1997). The chair's height was adjusted such that the arm moved in the horizontal plane. A circular cursor 0.5 cm in diameter, representing the current hand position, was initially positioned in a 2.5 cm start circle, the centre of which was located 0.31 m directly in front of the shoulder. The cursor, as well as the start and target circles were projected onto an opaque horizontal surface which hid the arm from the subjects' view. The cursor, start and target circles were visible throughout all trials.

Subjects made horizontal point-to-point movements, reaching 0.25 m directly forward to a 2.5 cm diameter target circle. This line defined the y-axis of the coordinate system (Fig. 1). The prescribed movement time of 600 ms was indicated by a series of brief tones that provided synchronizing signals to initiate and terminate the movement. No force acted on the hand until after movement had been initiated, nor was any force applied by the PFM as subjects moved back to the start position prior to commencement of the subsequent trial. The final position was deemed OK if the movement ended in the target circle. The duration was deemed OK if it was within ±100 ms of the prescribed time. Subjects were instructed that their goal was to produce movements that met the OK criteria. Feedback of movement duration (OK, LONG, or SHORT) and final hand position (OK or OUT) were provided as incentives for subjects to improve performance, although all trials were included in the data analysis. Each trial was self-initiated by moving the cursor into the start circle, enabling subjects to rest between trials, if desired. Force and position data were recorded at 500 Hz, beginning 150 ms prior to movement onset for 1500 ms.

Figure 1. Experimental setup.

A, overhead view of apparatus indicating coordinate system. B, graphical representation of force magnitude (length of arrows) and direction as a function of hand velocity. C, forces that would be experienced along a straight trajectory between targets, assuming a bell-shaped velocity profile.

EMG was recorded from six elbow and shoulder muscles. The activity of the brachioradialis, biceps, triceps longus, triceps lateralis, pectoralis and posterior deltoid was recorded. The EMG signals were analog filtered at 25 Hz (high pass) and 1.0 kHz (low pass) using a Nihon Kohden amplifier (MME-3132), and then sampled at 2.0 kHz.

Subjects participated in two experiments in which they were instructed to adapt to a novel dynamic environment. The protocol of Experiment 1 was designed to prevent subjects from completely adapting to the environment by limiting exposure to sets of three sequential trials and dispersing the sets somewhat randomly throughout the training session. During Experiment 2, exposure to the novel dynamics environment was continuous to permit complete adaptation. The objective of Experiment 2 was to characterize kinematic error, joint torque and muscle activation patterns after complete adaptation, relative to a prior null field condition. For four of the subjects, Experiment 2 began several minutes after completion of Experiment 1. For the other four subjects, Experiment 2 was conducted one to four months after Experiment 1.

In Experiment 1, subjects performed 22–23 consecutive movements in a null field (NF), followed by 27–28 sets of three trials in a velocity-dependent force field (VF1, VF2, VF3), each separated by a random number of between four and eight NF trials. The force in the VF was given by:

(1)

with 9 ≤B≤ 15 N s m⁻¹, dependent on the subject's capacity to adapt and where F is the force in N and V is the velocity in m/s. The effect of the force field is schematically illustrated in Fig. 1. It produced a force which assisted motion as well as perturbing the hand to the right. The rightward perturbation was the more noticeable of the two effects, probably because the mechanical impedance of the extended arm is considerably greater in the y direction than the x direction (Milner, 2002). On 10 randomly selected trials, VF3 was replaced by a catch trial, which took the form either of the NF or of a virtual constraint. The virtual constraint (or mechanical channel) consisted of an elastic force applied to the hand whenever it deviated to the right or left of the straight line joining the centres of the start and target zones. The stiffness of the channel was 40 N cm⁻¹. In Experiment 2, subjects performed 20 consecutive NF movements followed by 100 consecutive VF movements during which they completely adapted to the VF.

Analysis

Performance was characterized in terms of two kinematic parameters: the maximum lateral deviation from the straight line joining the movement start and target points, and the absolute hand path error, i.e. the area enclosed by the straight line and the hand path. The changes in elbow and shoulder torques needed to adapt to the VF were determined by comparing the torques for NF movements and for VF movements during adaptation. They were calculated as described in Franklin et al. (2003a). Muscle activity was quantified in terms of the root-mean-squared (rms) EMG computed over two intervals: −100 ms to 100 ms with respect to movement onset, which was considered to include only feedforward commands to muscles (feedforward interval), and 100–600 ms after movement onset, which could also include reflex EMG and voluntary muscle activity associated with on-line error correction (feedback interval). The 100 ms window following movement onset provided a conservative estimate of the earliest latency at which reflex EMG would appear, based on comparison of the EMG on the first trial in the VF and the NF trial which immediately preceded it (described in the Results). Muscle activity during NF trials was used as the reference for quantifying changes during VF trials. For experiment 1, the mean rms EMG in NF trials, which immediately preceded VF1 trials, was used as the reference. The rms EMG in NF and VF trials was normalized by dividing by the corresponding NF reference EMG prior to statistical analysis. This was done separately for the early and late EMG intervals of each muscle of each subject. Thus, the normalized rms EMG represents how much greater the muscle activity was during movements in the VF compared to the NF.

Changes in performance variables and normalized rms EMG between the beginning and end of an experiment were tested for statistical significance by ANOVA with subjects as a random factor. Changes in performance variables and normalized rms EMG between conditions were tested for statistical significance by means of paired t tests. Differences were considered to be statistically significant whenever P < 0.05.

Analysis of mechanical channel trials included calculation of the magnitude and time of occurrence of the peak force and the total force impulse exerted against the channel. The five channel trials were compared to the preceding NF trials and to the final five of the 100 VF adaptation trials using paired t tests, as described above.

Results

Reflex latency

On the first VF trial of Experiment 1, the hand was pushed forward and to the right, beginning about 80 ms after the onset of movement, resulting in shoulder and elbow extension relative to the previous NF trial (Fig. 2). The first change in muscle activity occurred about 40 ms later in the pectoralis, biceps and brachioradialis muscles, i.e. the muscles that would have been stretched relative to their length on the previous NF trial. This indicates that the latency of any reflex EMG was greater than 100 ms from movement onset. In fact, reflex latency should be referenced to perturbation onset, which occurred about 80 ms after movement onset. We chose to set the end of the feedforward interval for EMG at 100 ms after movement onset, which represented an effective reflex latency of 20 ms and thereby excluded the possibility of reflex contribution. This was actually more conservative than necessary because responses at monosynaptic latencies were not observed due to the very gradual nature of the displacement. Lee & Tatton (1982) demonstrated that very slow displacements evoke little or no short-latency (monosynaptic) reflex response. The activity of the three antagonist muscles, which were shortening, increased approximately 130 ms later (250 ms after onset of movement).

Figure 2. Initial effect of VF.

A, mean trajectory in x (lateral) and y (forward) directions and rectified, low-pass filtered EMG (20 Hz cutoff) of the first VF trial of Experiment 2 compared to the NF trial which immediately preceded it, computed for all eight subjects. Broken lines represent NF and solid lines represent VF. B, flexor EMG shown in top panels, from left to right, pectoralis, biceps and brachioradialis. Extensor EMG shown in bottom panels, from left to right, posterior deltoid, triceps longus and triceps lateralis. Vertical lines drawn on EMG records denote 120 ms and 250 ms after movement onset.

Adaptation to the VF (Experiment 2)

The process of gradual adaptation over 100 consecutive VF trials during Experiment 2 is illustrated in Figs 2 and 3. This adaptation was recorded after subjects had completed Experiment 1 so they were no longer naive to the characteristics of the VF. The subject whose data is shown in the figures completed Experiment 1 a month prior to Experiment 2 and did not participate in any experiments in the interim. The principal purpose of Experiment 2 was to determine the final patterns of muscle activation and to relate them to changes in joint torque in comparison to NF movements preceding adaptation. This establishes a baseline against which to gauge the magnitude of changes taking place during the initial stages of adaptation, recorded in Experiment 1. Complete adaptation involved a gradual modification of the torque profiles and straightening of the hand path. The final elbow torque profile was similar in shape to that of NF movements, but biased more in the flexor direction. Thus, the elbow extensor torque early in the movement decreased and the elbow flexor torque later in the movement increased relative to NF movements (Fig. 3). The final shoulder torque profile was more fundamentally altered, with flexor torque being exerted for the entire duration of the movement rather than switching from flexor torque in the first half of the movement to extensor torque in the second half. This adaptation required a marked increase in the total shoulder flexor torque.

Figure 3. Torque profiles during adaptation (Experiment 2).

Flexor torque is positive and extensor torque is negative. Dark solid line with surrounding shaded region represents mean and s.d. of five consecutive VF trials. Dashed line represents mean of final five VF trials during adaptation period. Dotted line represents mean of last five NF trials prior to start of adaptation period. Data are from subject 2.

Subjects required about 50 trials before they were able to produce a relatively straight hand path in the VF (Fig. 4), although even after adaptation to the VF, the hand path tended to be more curved than the NF paths. Comparing the five NF trials prior to onset of the VF with the final five VF trials, we found that the maximum deviation during VF movements was not significantly different from that of NF movements, although the absolute hand path error was significantly larger (P = 0.0006) for VF movements (mean 14 cm²) compared to NF movements (mean 8 cm²).

Figure 4. EMG profiles and hand paths during adaptation (Experiment 2).

Broken lines represent mean hand paths and EMG of sets of five consecutive VF trials, corresponding to torque profiles in Fig. 3. Thick grey lines represent mean hand paths and EMG of the final five VF trials during the adaptation period. Thin black lines represent mean hand paths and EMG of the five NF trials prior to introduction of the VF. Flexor EMG shown in left column, from top to bottom, pectoralis (PEC), biceps and brachioradialis (BRA RAD). Extensor EMG shown in right column, from top to bottom, posterior deltoid (POST DELT), triceps longus (TRI LONG) and triceps lateralis (TRI LAT). Data are from subject 2.

From the changes in net torque between the NF and VF, shown in Fig. 3, the activity of elbow extensors would be expected to decrease during the early portion of VF movements compared to NF movements, and that of elbow flexors would begin earlier and reach higher levels compared to NF movements. In the case of the shoulder, flexor muscle activity in the later part of the movement would be higher for the VF than the NF, whereas the activity of shoulder extensor muscles should remain virtually unchanged after adaptation. The only muscles to show a significant increase in normalized rms EMG in the feedforward interval after adaptation to the VF were the brachioradialis (P = 0.023), which increased its activity by an average of 38%, and the biceps (P = 0.029), which increased its activity by an average of 230% relative to NF movements. Since there was relatively little biceps activity in the feedforward interval during NF movements, this large percentage increase represents a comparatively small increase in flexor torque. The very early increase in the level of elbow flexor activity, and the failure to see any significant decrease in the activity of elbow extensors during the feedforward interval (P > 0.50) suggests that cocontraction was used to stiffen the elbow at the beginning of the movement. In the feedback interval, pectoralis activity increased by an average, of 46% (P = 0.033), biceps activity increased by an average of 230% (P = 0.020), and brachioradialis activity increased by an average of 70% (P = 0.0019), consistent with the increase in shoulder and elbow flexor torque. The activity of the triceps lateralis also increased by an average of 37% (P = 0.046) in the feedback interval, indicating continued stiffening of the elbow. There was no significant difference in the activity of shoulder muscles compared to NF movements (P > 0.40), although there was a tendency for an increase in the activity of the triceps longus (P = 0.07). The subject, whose data is shown in Fig. 3, as well as three other subjects increased the activity of the posterior deltoid in the feedback interval compared to NF movements. As no shoulder extensor activity was required during the movement this would have produced an increase in shoulder stiffness.

Early adaptation (Experiment 1)

The trial-by-trial change in kinematic error and muscle activity was determined for the first three movements in the VF to investigate the early stage of adaptation to the novel dynamics in Experiment 1. On the first VF trial, the absolute hand path error increased by an average of 75 cm² (P < 0.0001), from a mean of 7.7 cm² (s.d. 4.2) on the preceding NF trial to 83 cm² (s.d. 21). The maximum deviation increased by an average of 6.7 cm (P < 0.0001), from a mean of 0.026 cm (s.d. 0.73) to 6.7 cm (s.d. 1.7). On the second VF trial, both absolute hand path error and maximum deviation were reduced, by an average of 51 cm² (P = 0.0005) and 3.9 cm (P = 0.0018), respectively. The absolute hand path error was further reduced on the third VF trial, by an average of 12 cm² (P = 0.035), but the maximum deviation did not change significantly (P = 0.24).

There was no significant change in the activation of any muscle in the feedforward interval on the first VF trial compared to the preceding NF trial (P > 0.14). However, all muscles increased their activation in the feedback interval. Relative to the normalized rms EMG on NF trials, the increase was 150% for the pectoralis (P = 0.0031), 220% for the posterior deltoid (P = 0.01), 270% for the biceps (P = 0.0013), 140% for the triceps longus (P = 0.017), 230% for the brachioradialis (P = 0.0049) and 160% for the triceps lateralis (P = 0.011). On the second VF trial, muscle activity in the feedforward interval increased for all muscles, relative to the first VF trial. Expressed relative to activation on NF trials, the increase was 54% for the pectoralis (P = 0.020), 48% for the posterior deltoid (P = 0.0009), 65% for the biceps (P = 0.018), 61% for the triceps longus (P = 0.014), 38% for the brachioradialis (P = 0.012) and 44% for the triceps lateralis (P = 0.13). There was no significant increase in activity in the feedback interval compared to the first VF trial (P > 0.14) for any muscle except the triceps longus. In the case of the triceps longus, activity increased by an additional 93% relative to NF levels (P = 0.018). On the third VF trial, the activity of the pectoralis increased by an additional 50% (P = 0.022) and that of the triceps lateralis increased by an additional 51% (P = 0.018) in the feedforward interval compared to the second VF trial. However, there was no significant change in the activation of any of the other muscles (P > 0.11) in the feedforward interval. There was no significant change in activity of any muscle in the feedback interval on the third VF trial compared to the second VF trial (P > 0.24).

Intermittent presentation of VF (Experiment 1)

Surface EMG recorded from single trials during rapid adaptation to novel dynamics may not accurately reflect changes in motor commands. This is because the surface EMG is a stochastic representation of motor commands. On any given trial, relatively small changes in the timing and/or location of activated motor units can have a marked effect on the amplitude of the recorded signal. If the stochastic signal could be averaged across many repetitions of the same event, then the mean value would more accurately represent the true motor command. We did this by repeated presentations of the VF for sets of three consecutive trials, interrupted by a randomly selected number of 4–8 NF trials in Experiment 1. The mean and standard deviation of the absolute hand path error for the repeated sets of VF trials are shown in Fig. 5. The data for maximum deviation are not plotted because they are very similar.

Figure 5. Mean and standard deviations of absolute hand path error across subjects for VF sets (Experiment 1), in chronological order.

VF1 and corresponding VF2 trials are compared in the top panel. VF2 and corresponding VF3 trials are compared in the bottom panel. VF2 average values in bottom panel are different from the top panel, owing to exclusion of VF sets containing catch trials in the bottom panel. Data are averaged across subjects.

The element of complete surprise upon presentation of the VF for the first time could not be duplicated with 4–8 NF trials between VF sets. That is, the error on VF1 trials that followed the first VF set was always smaller. The smaller perturbation produced by the VF on VF1 trials after the first set was probably the result of an increase in the stiffness of the arm. We tested this by checking whether the normalized rms EMG of the NF trials interposed between VF sets was greater than that of NF trials prior to the first VF set. We analysed only the final NF trial of each interposed series because it was considered to be most representative of the state of the muscles at the onset of the VF1 trial. The normalized rms EMG of the interposed NF trials was significantly greater (by 25–45%) than that of the NF trials prior to the first VF set for all muscles in the feedforward interval (P < 0.02), and for all muscles except the pectoralis in the feedback interval (P < 0.02). This difference is evident when the average rectified, smoothed EMG across subjects is compared (Fig. 6). The greatest increase appears in the biarticular muscles, which we have previously found to be important in increasing lateral stiffness (Franklin et al. 2003b). Therefore, the arm would have been stiffer after the first VF set, reducing the perturbing effect of the force field and explaining the smaller error.

Figure 6. Increased cocontraction on NF trials.

Mean EMG profiles of the five NF trials prior to the first VF set (thick lines) are compared to NF trials that preceded each of the first five VF sets (thin lines), derived from all eight subjects (Experiment 1), plotted as in Fig. 2. There was an increase in cocontraction on NF trials between VF sets, which was greatest in the biarticular muscles.

Other than the increased stiffness of the arm, there did not appear to be any difference in the adaptation process during the three VF trials of the repeated sets compared to the first VF set. There was a similar pattern of reduction in kinematic error, although the magnitude of the trial-by-trial changes was reduced by the increased stiffness. On VF1 trials, the absolute hand path error increased, on average, from 13 cm² to 51 cm², and the maximum deviation increased from 0.35 cm to 4.1 cm (P < 0.0001). Absolute hand path error and maximum deviation were reduced on VF2 trials by 27 cm² and 2.0 cm, respectively (P < 0.0005). There was no significant change in either the absolute hand path error or the maximum deviation on VF3 trials compared to VF2 trials (P > 0.3). Note, however, that the average absolute hand path error of VF3 trials was the same as that of the third VF trial (20 cm²), whereas the average maximum deviation of VF3 trials was slightly less than that of the third VF trial (1.7 cm compared to 2.1 cm). Thus, the trajectory modifications recorded during repeated VF sets (Fig. 7), although somewhat attenuated compared to the first three VF trials, appear to accurately reflect the adaptive process. On this basis, we averaged the rms EMG across repeated VF sets to confirm that the changes in motor commands inferred from analysis of the first three VF trials were reliable.

Figure 7. Hand paths and force profiles.

A, mean hand paths of subject 2 for NF (black), VF1 (red), VF2 (blue) and VF3 (green) trials of three trial VF sets (Experiment 1). B, mean lateral (x) force exerted on the PFM plotted against y position for hand paths shown in A.

Changes in EMG for repeated VF sets (Experiment 1)

The average changes in muscle activation for the VF sets are shown in Fig. 8. There was no significant difference in the normalized rms EMG of any muscle between preceding NF trials and VF1 trials for the feedforward interval (P > 0.9). However, the perturbation produced by the VF resulted in a significant increase in the normalized rms EMG of all muscles during the feedback interval. Averaged across subjects, the increases were 73% for the pectoralis, 55% for the posterior deltoid, 110% for the biceps, 49% for the triceps longus, 49% for the brachioradialis and 60% for the triceps lateralis.

Figure 8. Mean EMG profiles NF (black), VF1 (red), VF2 (blue) and VF3 (green) trials of three trial VF sets (Experiment 1), plotted as in Fig. 2. Profiles represent means for subject 2.

There was an increase in activation of all muscles during the feedforward interval on VF2 trials relative to VF1 trials (P < 0.025; Fig. 8). The normalized rms EMG increased on average by 29–40%. Except for the triceps longus, the activity of all muscles remained unchanged in the feedback interval on VF2 trials compared to VF1 trials (P > 0.1). In the case of the triceps longus, the normalized rms EMG increased by 45% relative to VF1 trials (P = 0.0083). There was no statistically significant change in the normalized rms EMG of any muscle for either the feedforward or feedback interval on VF3 trials compared to VF2 trials (P > 0.15).

Increased impedance versus change in net force (Experiment 1)

The general increase in feedforward muscle activity indicated that subjects stiffened the arm at the onset of VF2 and VF3 trials compared to the preceding NF trial. However, it is not possible to infer the torque produced by each muscle from its EMG. Therefore, to test whether subjects were compensating for the force field only by increasing the stiffness of the arm or by also generating a net counteracting force, VF3 trials were occasionally replaced by NF trials, which served as catch trials to test for after effects of force compensation. Although catch trials could have replaced VF2 trials, we felt that subjects would be less likely to change their motor commands on catch trials if these were preceded by two VF trials rather than by only one VF trial. Catch trials were introduced in five VF sets chosen randomly. To verify that the feedforward command was not altered by the catch trial we examined the change in rms EMG for the feedforward interval compared to the preceding VF2 trial. For five out of six muscles the change was not significantly different from zero (P > 0.18), although for the triceps longus there was an increase of 26% (P = 0.048). The absolute hand path error and the maximum deviation on these catch trials were significantly greater than on the five NF trials which immediately preceded the VF sequences of the catch trials (Fig. 9). The mean of the hand path error across subjects, taking into account error direction, was 33 cm² to the left on catch trials, compared to 7 cm² to left for the NF trials that preceded the VF sequences (P < 0.0001). The mean of the maximum deviation across subjects was 2.0 cm to the left for catch trials compared to 0.5 cm to the left for the NF trials (P = 0.0002). The larger deviation to the left on catch trials indicates that subjects exerted additional force to the left in anticipation of the expected rightward perturbation by the force field. Note that activation of the triceps longus would counteract leftward deviation of the hand, so greater activation of the triceps longus on catch trials than on VF2 trials does not diminish the significance of the result.

Figure 9. Hand path and force profile of catch trials.

A, mean hand paths of five NF catch trials (thick lines), which replaced VF3 trials, compared to mean of control NF trials (broken lines) for subject 2 (Experiment 1). B, mean lateral (x) force exerted on PFM plotted against y position for five channel trials (thick lines), which replaced VF3 trials, compared to mean of control NF trials (broken lines) for subject 2.

A more direct measure of this force was obtained during randomly selected VF3 trials in which the velocity-dependent force field was replaced by a virtual mechanical channel (Fig. 9). For these trials there was no significant change in the rms EMG of the feedforward interval for any muscle compared to the preceding VF2 trial (P > 0.27). The mean of the peak force perpendicular to the channel (x direction) was −7.38 N (where the negative sign indicates that the force is directed to the left) and the mean of the force impulse was −1.72 Ns, across subjects. We compared this to the force exerted on the PFM in the x direction during the NF trials which preceded the channel trials and after adaptation to the VF. During NF trials, not only were the mean of the peak force (1.23 N) and force impulse (0.071 Ns) significantly smaller than during channel trials (P < 0.0001), but they were also in the opposite direction. The mean of the peak force during VF trials after adaptation (−6.18 N) was somewhat smaller than during channel trials (P = 0.015). However, the force impulse (−2.30 Ns) was significantly larger (P = 0.0058). In addition, both peak force and force impulse were considerably more variable for the five channel trials than for the last five VF trials. The mean coefficient of variation was 0.34 for peak force and 0.38 for force impulse for the channel trials compared to 0.096 and 0.059 for the VF trials.

Latent effects of intermittent perturbation (Experiment 1)

As noted above, we found that subjects increased muscle cocontraction on NF trials interposed between intermittent VF sets compared to NF trials that preceded the first VF set. This cocontraction did not appear to be significantly modified over time. In comparing the first five and last five NF trials that preceded VF sets, we found no significant difference in the normalized rms EMG for any muscle in either the feedforward interval (P > 0.18 for monoarticular muscles and P > 0.92 for biarticular muscles) or the feedback interval (P > 0.1). Furthermore, the four subjects who began Experiment 2 several minutes after completing Experiment 1 retained the elevated cocontraction during the 20 NF trials that preceded the onset of VF trials in Experiment 2. The average rectified, smoothed EMG profiles (similar to Fig. 6) of the final five NF trials that preceded VF sets in Experiment 1 were compared with those of the final five NF trials at the start of Experiment 2. For none of the four subjects was there any clear indication of a reduction in EMG. Thus, although induction of cocontraction during adaptation is a rapid process, its extinction occurs very slowly.

The data in Fig. 5 suggest that kinematic error on VF2 trials may have been gradually reduced over time. Although the slope (excluding the first VF set) was not significantly different from zero for either hand path error (P = 0.081) or maximum deviation (P = 0.18), the mean hand path error for trials in the second half of the VF sets was 3.8 cm² less (P = 0.011), and mean maximum deviation was 0.33 cm less (P = 0.019) than for trials in the first half. This represents about a 10% change relative to the error on the first VF2 trial. In the case of VF1 and VF3 trials, there were no significant differences between trials in the first and second half of the VF sets (P > 0.4). We again compared average smoothed EMG profiles of VF2 trials for early and late VF sets. Although there were differences in the EMG profiles for some subjects, they were quite varied. Comparison of the normalized rms EMG between VF2 trials in early and late sets showed no statistical difference. Furthermore, differences in EMG profiles were only found in the feedback interval.

Discussion

The results of this study demonstrate that two feedforward control mechanisms are involved in the initial stage of adaptation to novel dynamics. Responses to disturbances produced by the novel dynamics on the first trial were feedback in nature, i.e. reflexes and voluntary correction of errors. However, on the second trial, knowledge gained during the first trial was used to change the activation of all muscles in a feedforward fashion. Cocontraction of all muscles was used to increase the viscoelastic impedance of the arm. Simultaneously, subjects counteracted the force field by generating a net force which resisted its perturbing effect, suggesting the use of a crude internal dynamics model of the force field. The changes in muscle activation patterns produced a marked reduction in the hand path error.

Final adapted state

The levels of muscle activation associated with the final adapted state were consistent with the change in shoulder and elbow torque profiles, although there was no decrease in the activity of elbow extensor muscles even though the net elbow extensor torque was markedly reduced in the VF compared to the NF. This suggests that subjects performed the task with some agonist/antagonist cocontraction of elbow muscles even after complete adaptation. We have recently shown that the central nervous system controls muscles to maintain a degree of stability which is approximately equal to the stability achieved during NF movements (Franklin et al. 2004), and have also shown that cocontraction is required to provide the damping necessary for stability in a force field which assists movement in proportion to velocity (Milner, 2004). Although the VF employed in the present study was not purely assistive in nature, it did contribute negatively to damping in the y direction. Therefore, muscle cocontraction would have been necessary to achieve similar overall stability to that of movements in the NF at the target position. We previously found cocontraction of elbow and biarticular muscles after adaptation to a velocity-dependent force field with negative damping (Milner, 2004), but little or no cocontraction after adaptation to a velocity-dependent force field with positive damping (Franklin et al. 2003a). Therefore, we can conclude that all of the observed cocontraction in this study was used to increase damping of the arm to achieve the necessary stability.

Rationale and validity of protocol

The question of how feedforward commands are modified to adapt to novel mechanics has generally been addressed by having subjects adapt over many trials and averaging the EMG over blocks of trials at different points as learning progresses (Thoroughman & Shadmehr, 1999). This is a reasonable approach, which can be considered to have high validity if the learning rate is relatively slow, i.e. if performance is not changing rapidly. However, a number of studies have shown that adaptation to novel mechanical environments occurs in an exponential fashion (Thoroughman & Shadmehr, 1999; Osu et al. 2003; Franklin et al. 2003a), with errors being most rapidly reduced during the first few trials as subjects dramatically alter their patterns of muscle activation (Thoroughman & Shadmehr, 1999; Osu et al. 2003; Franklin et al. 2003a). Having to average the EMG over several successive trials would preclude being able to isolate the processes involved in generating the initial feedforward response to a change in the dynamics of the environment, i.e. the change in the feedforward command between the first and second trials. We first observed the process of adaptation to three trials in a novel velocity-dependent force field for naive subjects. To confirm that our observations were reliable and repeatable, we then repeatedly activated the force field for several trials in succession at random intervals during null field trials. This allowed us to average the surface EMG to obtain a more statistically reliable representation of the change in muscle activation from one trial to the next during the early stage of adaptation to the force field.

Latent effects of intermittent perturbation

One unanticipated effect of intermittently presenting sets of VF trials was an increase in cocontraction during the intervening NF trials. As a result of this cocontraction, the muscles were in a different mechanical state and the α-motoneurones were in a different state of excitability during the second and subsequent VF sets than during the first VF set. The principal effect of the subjects' change in strategy between the first and second VF sets was that the error on VF1 trials was reduced and there was a smaller percentage improvement in performance on VF2 trials compared to the first VF set. This was reflected in the amount by which muscle activation changed, as well. Therefore, the results from the repeated VF sets represent an attenuation of what actually occurs between the first and second trials. It is noteworthy that this increased cocontraction was not noticeably reduced over a set of 20 NF trials for the four subjects who performed Experiment 2 several minutes after completing Experiment 1. This reinforces previous observations (Thoroughman & Shadmehr, 1999; Franklin et al. 2003a) that the decay of excess muscle activation is a very gradual process.

Another unexpected effect was the reduction in error on VF2 trials of later VF sets compared to earlier sets. Since differences in EMG were only found in the feedback interval it is more likely that the reduction in error was due to modified reflex responses or more effective corrective responses than more effective anticipatory (feedforward) compensation for the force field. This also suggests that an accurate internal model of the environmental dynamics cannot be formed by intermittent exposure, since there was no change in feedforward muscle activity of VF2 trials with intermittent repetition.

Initial adaptation by increased impedance and use of an internal model

There was a general increase in the activity of all muscles during the feedback interval on VF1 trials compared to NF trials. In the muscles stretched by the perturbation, this can be attributed to long-latency stretch reflexes. The increased activity of their antagonists occurred considerably later (∼120 ms) and may have represented rapid voluntary muscle activation to increase the stiffness of the arm, or a non-specific triggered response (Crago et al. 1976) that would limit the effect of the perturbation until the CNS was able to take corrective action.

The generalized feedforward or anticipatory increase in activity of all muscles observed on VF2 trials relative to VF1 trials indicates that an increase in viscoelastic impedance was a primary adaptive response. The rapid establishment of a cocontraction level suggests that the central nervous system quickly judges how much viscoelastic impedance is necessary to reduce kinematic error sufficiently for efficient learning of an internal model (Franklin et al. 2003a; Milner, 2004). Our recent studies suggest that cocontraction does not increase for more than one or two trials following initial exposure to novel dynamics. From that point on it tends to decrease. As the dynamics of the task are learned, the margin of stability conferred by cocontraction is reduced (Franklin et al. 2003a; Milner, 2004).

Muscle activation during the feedback interval remained as high on VF2 trials as on VF1 trials, despite considerable reduction in kinematic error. In the case of the triceps longus, it even increased. This suggests that the duration of the anticipatory cocontraction extended well into the feedback interval and that biarticular muscles were used to increase the stiffness of the arm on VF2 trials. The activity of the biceps did not increase in the feedback interval even though less shoulder and elbow flexor torque was required for corrective action because of reduced hand path deviation. Instead, flexor torque was reduced by an increase in triceps longus activity while biceps activity remained unchanged thereby increasing the stiffness of the arm. It is likely that biceps motoneurones received greater synaptic input from descending commands on VF2 trials than VF1 trials. However, this would have been offset by reduced synaptic input from muscle spindles due to the smaller perturbation. At the same time, the descending synaptic input to the triceps longus would have been enhanced due to a decrease in reciprocal inhibition.

In addition to the generalized feedforward cocontraction on VF2 trials, which served to increase the viscoelastic impedance of the arm, subjects began to exert an anticipatory lateral force to counteract the disturbance produced by the VF. The hand path deviations observed when VF3 trials were replaced by NF catch trials, and the lateral forces measured on channel trials indicate that by the third trial subjects were using a crude internal model of the force field. Based on the similarity of the EMG on VF2 and VF3 trials, we can conclude that the internal model was adopted between VF1 and VF2 trials. In the case of channel trials which replaced VF3 trials, our analysis showed that the peak lateral force was greater than that applied to the manipulandum after complete adaptation. However, the lateral force impulse was smaller. This indicates that initial compensation for the force field consisted of the application of a lateral force which was larger and briefer than necessary. This is corroborated by the relatively large hand path deviation which still existed on VF3 trials. This study cannot distinguish between the possibility that this process represents the formation of a new internal model or the selection of some previously learned response that is subsequently refined (Wolpert & Kawato, 1998). However, we have elaborated a set of principles for motor learning that would allow internal model formation to begin on the second trial in a novel mechanical environment (Burdet et al. 2004).

In summary, it appears that the central nervous system is able to glean critical information about the nature of the disturbing force during a relatively brief period of exposure, e.g. a single movement, in a novel mechanical environment, as has also been suggested by Scheidt et al. (2001). In addition to generalized cocontraction, which increases the viscoelastic impedance of the limb, there is an anticipatory increase in the activation of muscles needed to counteract the disturbing force when the second movement is attempted. This contemporaneous adaptation of limb impedance and formation of an internal dynamics model has been suggested by the results of several recent studies (Takahashi et al. 2001; Franklin et al. 2003a; Osu et al. 2003). We are currently investigating how sensory information generated by the initial disturbance is used to initiate the formation of an internal dynamics model of the novel environment.