Optimizing effort: increased efficiency of motor memory with time away from practice

Abstract

In motor tasks, efficiency can be measured via the commands that are produced to accomplish a goal. To maximize efficiency, the nervous system should produce task-relevant motor commands while avoiding behaviors that are task-irrelevant. The current view is that this is achieved through training, i.e., the optimum motor commands are learned by trial and error. However, in contrast to this view, there are numerous examples in which during an experiment, task-irrelevant commands are continuously produced. To address this, we trained human volunteers to reach in a force field. With practice, they learned to produce forces that compensated for the field, generating task-relevant commands that were necessary to achieve success. As expected, training also resulted in generalization, the transfer of learning to other movements. We designed the task so that any forces produced as a result of generalization were unnecessary and therefore task-irrelevant. Importantly, there were no explicit cues to indicate that production of these forces was task-irrelevant. Rather, the only indicator was effort itself. Could this inefficiency of the motor commands be reduced? We found that even with extensive practice, the production of task-irrelevant forces persisted. However, if subjects were given sufficient time away from practice (6 or 24 h but not 3 or 30 min), they spontaneously reduced production of the task-irrelevant forces. Therefore, practice alone was insufficient to allow for increased efficiency of motor output. Time away from practice was a required element for optimization of effort.

in making a movement, there are numerous muscle activation patterns that can produce success. For example, to hit a tennis ball into the service court, some players twist and turn their bodies as they reach to hit the ball, whereas others reach more gracefully. Theoretical approaches suggest that learning of an action should favor production of the less effortful movement, i.e., effort should carry a cost (Salimpour and Shadmehr 2014). Indeed, there is some evidence for this idea. With practice, there is often a reduction in muscle cocontraction (Darainy and Ostry 2008; Franklin et al. 2003; Thoroughman and Shadmehr 1999) and metabolic expenditures (Huang et al. 2012). Furthermore, given a choice between reaching movements that require various levels of effort, people (Cos et al. 2011; Wang and Dounskaia 2012) and monkeys (Pasquereau and Turner 2013) have a preference toward the less effortful reach.

However, there are also examples in which despite practice, more effort is expended during a movement than is necessary. For example, as people reach, muscles about the wrist joint are activated to counter the torques resulting from rotations of the elbow and shoulder. However, when the wrist is mechanically immobilized to eliminate the effects of these “interaction” torques, the wrist activations persist (Koshland et al. 2000). When people train to reach in a field in which a straight point-to-point trajectory requires more force than a very curved trajectory, they choose the straight trajectory even after hours of practice (Kistemaker et al. 2010; but see Izawa et al. 2008 and Uno et al. 1989). Finally, following force field training, experimenters can replace the field with an error-clamp in which the hand is constrained to a straight path to the target and the production of the field-compensating motor commands is no longer necessary. In these error-clamp trials, participants continue to produce field-specific forces (Scheidt et al. 2000) even after hundreds of trials (Pekny et al. 2011; Vaswani and Shadmehr 2013). These examples highlight instances in which there are less effortful motor commands that can produce success, but the brain appears unable or unwilling to select them.

One possibility is that straight reaching movements may be the habitual response in certain conditions, more resistant to change, and therefore a less than ideal paradigm for short-term studies of the processes that may be involved in optimization of effort. Here, we thought to approach the question of effort optimization from a different perspective. We designed a task that involved learning a new motor behavior in which there was a natural tendency to produce inefficient motor commands. We then quantified the conditions that were required for reducing these inefficiencies.

We designed a task in which learning a movement resulted in robust inefficiencies, i.e., production of forces that were unnecessary for success. We then altered the conditions in which the learning took place to uncover the elements that were important for reduction of inefficiencies.

For our study, we considered a standard force field paradigm in which the participants reached to a target and a field was applied to their hand (Shadmehr and Mussa-Ivaldi 1994). To complete the task and hit the target successfully, the participants learned to produce forces that countered the field. However, this learning led to generalization, i.e., extrapolation of training to novel regions of the task space (Hwang and Shadmehr 2005). We measured generalization by asking the subjects to reach to a probe target and measured the forces that they produced during that reach. Importantly, we designed the task in such a way as to make production of these forces unnecessary for success.

To achieve this inefficient but natural behavior, the probe target was always presented in an error-clamp. In these error-clamp probe trials, there were no kinematic consequences of the effortful motor commands, as the reach was clamped to a straight line trajectory to and from the target. Therefore, the production of forces against the clamp had no bearing on success. Without performance errors or explicit knowledge of the task design, the participants were left with an implicit measure of effort as the only cue to indicate the inefficient nature of the forces that they produced. Could these task-irrelevant motor commands be reduced?

Surprisingly, we found that extended practice did not result in a decrease of the task-irrelevant force production. Rather, a critical factor was time away from practice; following a break of 6 or 24 h in duration, but not of 3 or 30 min, the task-irrelevant force production spontaneously decreased. Therefore, whereas increasing the number of practice trials alone did not result in optimization of effort, it was time away from practice that afforded the opportunity to increase the efficiency of behavior.

METHODS

Human volunteers (n = 41, 24.84 ± 5.44 yr old, mean ± 1 SD, including 18 men and 23 women) learned to make out-and-back reaching movements. All volunteers were naïve to the purpose of the experiment, were right-hand dominant, and reported no known neurological disorders at the time of testing. Each participant provided informed, written consent. Our procedures were approved by the Johns Hopkins University School of Medicine Institutional Review Board.

Participants held the handle of a two-joint robotic manipulandum and made out-and-back reaching movements along the body midline from a 1-cm² start target to a 1-cm² goal target located 10 cm away. The goal target appeared at one of two locations, 90° (train target) or 270° (probe target), as shown in Fig. 1A. Participants were instructed to reach to the goal target and then back to the start position in a continuous motion without stopping. Visual feedback was provided at all times via a 5-mm² cursor representing real-time hand position projected onto a screen covering the participant's hand. Reaches were considered successful when the total movement time was within 830–970 ms, peak tangential velocity was within 0.20–0.60 m/s, and participants had their turn-around point within a 6-mm diameter of the center of the goal target. Information regarding success or failure was provided once the hand returned to the start position. The goal target was animated to resemble an explosion in the case of a successful trial.

Fig. 1.

Task setup and experimental groups. A: participants held the handle of a robotic manipulandum and made 10-cm out-and-back reaching movements to 2 different targets. For the train target, a velocity-dependent, curl force field was applied on the outward reach. The field was turned off once the turn-around point of the reach back to start was detected. For the probe target, the reaches were always in error-clamp. In error-clamp trials, the hand was constrained to a straight line between start and target locations, preventing any deviation of the hand but allowing us to measure how much force subjects produced against channel walls. Because of the error-clamps, any forces that subjects produced during the reach to the probe target had no consequences in terms of reach kinematics, resulting in no errors. Sample trajectories for the reach to train and probe targets are shown for a representative subject, taken from trials early in the training protocol. B: perturbation protocols. Participants were provided a short set break (1 min) between blocks (indicated by dashed lines). Each block consisted of 192 trials with 169 train targets and 24 probe targets randomly interspersed.

Targets appeared in one of two locations, as shown in Fig. 1A, and were labeled as train target and probe target. For the train target, a counterclockwise velocity-dependent, curl force field with a gain of 13 N·s/m (Brashers-Krug et al. 1996) was imposed on the hand during the outward phase of the reach. Once the hand reached the train target and a turn-around point was detected, forces were shut off (Fig. 1A). That is, for the train target, the field was present only in the outward reach and not in the return back to start. Furthermore, on randomly selected reaches to the train target (1 out of 8), we imposed an error-clamp. In these trials, the hand was constrained to a straight path both out to the train target and during the return to start. Error-clamp was a channel centered between the start and goal positions and implemented via a stiff 1-dimensional spring (spring coefficient = 2,500 N/m, damping coefficient = 25 N·s/m).

The probe target was presented in 1 out of 8 randomly selected trials (otherwise the train target was presented). Crucially, for the probe target, the reaches were always in error-clamp. Therefore, any forces that subjects produced in the probe trials against channel walls were unnecessary as there was never a force field for the probe target. However, we expected that learning of the train target would be generalized to the probe target. The question was whether the inefficiencies inherent in this generalization could be reduced. Each block was 192 trials. Subjects were given a set break of ∼1 min between each block.

Twenty-four-hour group.

The protocol is summarized in Fig. 1B (24-h group). Participants (n = 10, 27.00 ± 7.05 yr old, mean ± 1 SD, including 5 men and 5 women) completed a baseline block in the null field (192 trials, no perturbation) and were then presented with 2 blocks of practice (192 trials each) in which they reached in a force field to the train target and reached in an error-clamp to the probe target. They then left the laboratory and returned in ∼24 h (24.24 ± 1.86 h, mean ± 1 SD). On day 2, subjects trained in 2 more blocks of trials (192 trials each). Importantly, the 1st 2 trials on day 2 were in error-clamp. At the start of block 3, immediately following the 24-h break, participants 1st reached to an error-clamp in the train direction followed by an error-clamp to the probe direction. This was consistent across all participants and all groups and allowed us to measure the retention of force production following training but before the experience of any further performance errors. This pattern was repeated on days 3 and 4. The sessions were performed at approximately 48 and 72 h after the start of initial training (47.96 ± 1.36 and 71.96 ± 1.60 h, means ± SD) for a combined total of 4 consecutive days.

Three-minute group.

The results of the 24-h group suggested that passage of time may provide the conditions necessary to reduce generalization, as evidenced by overnight reductions in the forces that subjects produced to the probe target. To dissociate between effects of practice vs. effects of time/overnight sleep, we recruited a new group of subjects, the 3-min group (Fig. 1B). Participants (n = 10, 21.20 ± 3.36 yr old, mean ± 1 SD, including 5 men and 5 women) began their training in one block of null and two blocks of field trials, exactly as in the 24-h group. However, they continued their training with two additional blocks of field trials after a 3-min break. This paradigm effectively doubled the amount of practice on the 1st day compared with the 24-h group. This group then returned after 24 h (23.15 ± 2.12 h, mean ± 1 SD) and completed another four blocks of trials. Again, all participants reached through an error-clamp to the train target followed by an error-clamp to the probe target at the start of blocks 3, 5, and 7.

Thirty-minute and six-hour groups.

The results of the 3-min group suggested that increasing the number of trials on day 1 was not sufficient to allow for the reduction of generalization. To dissociate between effects of time vs. leaving the experimental setup and time vs. overnight sleep, we recruited 2 new groups of subjects, a 30-min group (n = 11, 24.45 ± 3.56 yr old, mean ± 1 SD, including 4 men and 7 women) and a 6-h group (n = 10, 26.80 ± 4.05 yr old, mean ± 1 SD, including 4 men and 6 women). Following the initial block of null and then 2 blocks of field training (exactly as in the 24-h and 3-min groups), subjects in the 30-min group left the experimental setup and sat quietly in the laboratory for 30 min before returning to the robotic arm and completing another 2 blocks (Fig. 1B, 30-min group). In contrast, subjects in the 6-h group left the laboratory for at least 6 h (actual wait time was 6 h, 15.0 ± 11.5 min, mean ± SD) and then returned to complete an additional 2 blocks of trials (Fig. 1B, 6-h group). Participants were allowed to perform their normal activities during this 6-h period but were instructed not to sleep, play video games, or exercise. Both groups of subjects returned 24 h later (23.95 ± 2.46 h for 30-min group, 26.12 ± 2.35 h for 6-h group, means ± 1 SD) and performed another 4 blocks of trials. Consistent with the other participants, at the start of blocks 3, 5, and 7, all participants in these groups reached through an error-clamp to the train target followed by an error-clamp to the probe target.

Fig. 6.

The forces that were produced during the downward segment of the reach for the train target as quantified via a force index (Eq. 1). The forces were generally near 0 and did not differ between groups. Error bars represent between-subject SE.

Data analysis.

To quantify the motor commands that subjects learned to generate, a force index was calculated from the forces f(t) that they produced against channel walls during an error-clamp trial. This index, labeled with variable a, reflected a measure of compensation as a function of ideal forces:

$$f\left(t\right)=a\left[13\dot{x}\left(t\right)\right]\mathrm{.}$$ (1)

In Eq. 1, ẋ(t) is hand velocity parallel to the direction of target, and 13 reflects the size of the velocity-dependent, curl force field that we applied to the hand. We found a least-squares estimate of the variable a for each error-clamp trial by using the measured forces f(t) and velocities ẋ(t). To do this, the reach trajectory for a given trial was divided in two parts at the point of maximum extent (reach out and reach back). For the train target, we computed the force index a_t for the reach component from start point to the target. For the probe target, we computed the force index a_p for the reach component from target back to the start point. A probe-to-train ratio a_p/a_t was calculated for each participant by dividing the force index of the probe target by the force index to the train target. This was calculated as a proxy for the percentage of learning that was generalized, as complete compensation for the field is rarely achieved (i.e., a_t < 1).

To measure an individual's reach consistency, pairs of reach trajectories were compared using the time series of velocity vectors, resulting in a correlation coefficient (Shadmehr and Mussa-Ivaldi 1994). To measure changes in correlations with training, all field trials across all sessions were separated into minisets of 7 consecutive trial bins. The cross-correlation of each pair within this 7-trial bin was computed, resulting in 21 correlations. The average of the correlations was then reported as a measure of consistency for that bin of trials for that subject. Reach reaction time was measured by finding the time point at which the tangential velocity of the hand exceeded 0.02 m/s.

All data analysis was completed in MATLAB 7.0.4, and all statistical analysis was performed with SPSS Statistics 22. Individual reach trajectories were excluded from analysis if the participant did not reach the target (within 8 mm of target center). Additionally, reach durations of 2.4 s or more and reaches with force production >150% of necessary force compensation were removed from analysis. Two participants were removed from the data pool and are not included in the overall subject number reported (n = 41) as they had an average a_p/a_t that was outside of the 3 SD window of the population median.

RESULTS

Volunteers (n = 41) made out-and-back reaching movements to a train target and a probe target (Fig. 1A). After baseline training in a null field, all participants completed 2 blocks of trials in a force field (192 trials per block; Fig. 1B).

Learning generalized to the probe target.

For the train target, the field produced forces that were perpendicular to reach direction, pushing the hand away from the goal on the outward reach but not on the inward reach. In response, subjects learned to produce forces perpendicular to the direction of reach on the upward segment of their motion to the train target as illustrated in Fig. 2A. For the probe target, there were never any perturbations; rather, reaches were always in an error-clamp. Nevertheless, as subjects learned to produce forces for the train target, they also produced forces for the probe target. In particular, subjects produced significant forces during the upward segment of their reach in the probe trials (the upward segment is the positive velocity component of the probe target; Fig. 2A).

Fig. 2.

Reaching to the train target resulted in generalization to the probe target. A: the traces show the forces and velocities recorded from the last 20 error-clamp trials in block 2 of training on day 1 for a representative subject (left column) and all subjects (right column). The x-axis, representing hand displacement, was normalized to the maximum extent of each reach (max). Error bars are SD for single subject, SE across subjects (they are very small). The top is motor output (forces produced against channel walls) and velocity for an out-and-back movement to the train target. The forces are the change from forces that the subjects produced in block 0 (null field). Positive velocity indicates a reach to the target. Negative velocity indicates reach back to center. For the train target, the perturbation was present only on the reach toward the target, i.e., positive velocity, and not the reach back. For the train target, subjects learned to produce compensatory forces when velocity was positive. They generalized this learning to the probe target, also producing forces, especially when velocity was positive. B: a force index was used to quantify the amount of force produced for reaches to the train target (upward segment, i.e., positive velocities) and for reaches to the probe target (also positive velocities). The index was the regression of the force profile onto the ideal profile resulting from hand velocity. For the train target, the index was calculated for the reach out to maximum extent. For the probe target, the index was calculated for the reach from maximum extent back to the start target. Probe-to-train ratio is the ratio of the force index for the probe target to the train target and stands as a proxy for percentage generalization. Data are means ± SE plotted for the 2nd half of block 0 and all of blocks 1 and 2.

We used a force index (Eq. 1) to quantify these forces. For the train target, the index a_t was computed as the hand reached from start to the turn-around point (an index of 1 implies exact compensation for the velocity-dependent field). This was our proxy for the task-relevant forces that subjects produced. For the probe target, the index a_p was computed as the hand reached from the turn-around point back to start (for the probe target, an index of 1 implies that the subjects produced forces that were as large as that for the train target). This was our proxy for the task-irrelevant forces that subject produced. We found that a_t increased with practice, reaching a value of 0.89 ± 0.07 by the end of block 2 (mean ± 1 SD across last 5 error-clamp trials; Fig. 2B, top plot). Similarly, a_p increased with practice, reaching an average of 0.33 ± 0.16 (mean ± 1 SD across last 5 error-clamp trials). To compare the rates of increase in the forces for the train and probe targets, we computed a_p/a_t, as shown in the bottom subplot of Fig. 2B. This is a measure of percentage of learning that was generalized to the probe target. We found that a_p/a_t rapidly declined. Of the total change from the beginning to the end of day 1 training, 96.29% of the total drop in a_p/a_t occurred in the 1st 50 trials, remaining relatively constant and decreasing only an additional 3.71% for the subsequent 330 trials. Therefore, participants generalized their experience from the train target to the probe target.

However, because all movements to the probe target were in error-clamp, production of these forces was unnecessary: regardless of the forces produced against channel walls for the reach to the probe target, participants experienced no kinematic performance errors, leaving no explicit cues or errors to indicate that this component of behavior, i.e., this generalization, was resulting in task-irrelevant motor commands. As the data in Fig. 2B illustrate, the forces that subjects produced in the probe trials did not show any evidence of reduction, in terms of both absolute value (top part of Fig. 2B) and relative value (bottom part of Fig. 2B), beyond the 1st 50 trials of training. Could this inefficient behavior become reduced?

Time away from practice was necessary for the reduction of task-irrelevant forces.

After completion of block 2, we separated our subjects into four groups that waited for varying amounts of time until the start of block 3 (Fig. 1B). One group (n = 10) waited 24 h. By the end of training on the 1st day (block 2), these subjects were able to counteract the field, producing a force index of a_t = 0.90 ± 0.02 (mean ± SE, over last 5 error-clamp trials; Fig. 3A). The participants generalized this learning to the probe target, a_p = 0.36 ± 0.04 (mean ± SE, over last 5 error-clamp trials). When the subjects returned at 24 h (block 3), on their first trial, they reached in an error-clamp to the train target. This allowed us to measure recall of the motor memory. We found that on this first trial of day 2, a_t = 0.33 ± 0.05 (mean ± SE), which represented 37.13% of the value achieved at the end of training on day 1. Within a few additional trials, after experiencing errors in the field, performance was statistically indistinguishable from performance 24 h earlier. Indeed, a_t on day 2 measured over block 3 was not significantly different from a_t at the end of day 1 [post hoc pairwise comparison, block 2 vs. block 3, P = 0.207, following repeated-measures ANOVA with a within-subject factor of mean train force index for blocks 1–8, effect of block, F_(7,63) = 15.122, P < 0.001]. However, the forces to the probe target as measured via a_p showed a significant decrease from day 1 to day 2 [post hoc pairwise comparison, block 2 vs. block 3, P = 0.002, following repeated-measures ANOVA within-subject factor of mean probe force index for blocks 1–8, effect of block, F_(7,63) = 4.555, P < 0.001]. Therefore, on day 2, the subjects recalled the forces for the train target but now produced less force for the probe target.

Fig. 3.

Forces produced for the probe target were reduced with time, not practice. A: 24-h group. Training produced generalization. However, when subjects returned at 24 h, they exhibited reduced forces for the probe target. Asterisk denotes the block at which point all subsequent blocks were statistically significant for probe force index according to post hoc analysis with a 0.05 significance level. B: 3-min group. Although a 3-min break was not sufficient to reduce probe forces, after an overnight passage of time the probe forces declined. Asterisk denotes the block at which point all subsequent blocks were statistically significant for probe force index according to post hoc analysis with a 0.05 significance level. C: ratio of probe force index to train force index.

Because we had observed a reduction in the task-irrelevant forces following a 24-h break, we asked the subjects to return on days 3 and 4 for further training, wondering whether this trend of reduced forces for the probe target would continue. During these sessions, participants again exhibited recall on the very first trial of each test day (a_t = 0.43 ± 0.07 for day 3, and a_t = 0.57 ± 0.06 for day 4). There was no significant benefit of time for performance to the train target: a_t as measured over the entire training blocks of days 3 and 4 were not different from a_t at the end of day 2 (all post hoc pairwise comparisons, block 4 vs. blocks 5–8, P > 0.100). However, the ratio a_p/a_t declined significantly as a function of day of training [effect of day, P = 0.004, F_(3,24) = 5.855, comparison of last 5 values per day]. Indeed, we found a within-subject 12.8 ± 3.4% decrease in generalization a_p/a_t from the end of day 1 to the end of day 2, a 2.8 ± 3.0% decrease from the end of day 2 to the end of day 3, and a 5.4 ± 3.0% decrease from the end of day 3 to the end of day 4.

We wondered whether the reduction in the generalization at the 1st 24-h break was due to passage of time, which included a normal night of sleep, or simply increased practice. In the 3-min group (Fig. 1B), participants (n = 10) performed blocks 3 and 4 after a short break following completion of block 2, doubling the amount of training on day 1. The results of the 3-min group are displayed in Fig. 3B. In blocks 2 and 3, the train forces for the 3-min and 24-h groups were comparable: ANOVA with a within-subject repeated-measure of a_t (blocks 2 and 3) and between-subject factor of group revealed no significant interaction, F_(1,18) = 0.41, P = 0.529. However, unlike the 24-h group, the 3-min group did not exhibit a decrease in a_p following block 2 [post hoc comparison of 24-h vs. 3-min group, P = 0.023, following a 1-way ANOVA for difference in a_p between blocks 2 and 3 across all groups, F_(3,40) = 4.074, P = 0.013]. Instead, in the 3-min group, the probe forces in block 3 were not significantly different from those in block 2 [post hoc comparison, block 2 vs. block 3, P > 0.500 following repeated-measures ANOVA within-subject factor of mean a_p in blocks 1–8, effect of block, F_(7,63) = 8.471, P = 0.003]. That is, increased practice was not sufficient to allow for the reduction of the task-irrelevant forces in the probe trials.

After completion of block 4, the 3-min group received a 24-h break. On return, they exhibited recall of the forces for the train target (Fig. 3B). Remarkably, only after this 24-h passage of time did the 3-min group show a reduction in the forces for the probe target (post hoc pairwise comparison, block 4 vs. block 5, P = 0.003). These participants did not experience any further 24-h set breaks. They did, however, practice twice as long on day 2 as the participants in the 24-h group, performing four blocks of trials. We found that forces produced for the probe target continued to decrease with practice in these sessions. Here, we saw that forces due to generalization continued to decrease slightly with practice in these sessions. However, this drop was only statistically significant when comparing the very beginning of the second test session with the end of practice (post hoc comparison, block 5 vs. block 8, P = 0.041).

In summary, we observed that at 24 h following initial practice but not 3 min, subjects reduced the forces that they produced for the probe target. This suggested that following initial acquisition, time away from practice afforded an advantage that practice alone could not provide: the ability to reduce the task-irrelevant generalization.

Sleep was not necessary for reduction of the task-irrelevant forces.

The 24-h group experienced time away from the experimental setup and a normal night of sleep. We wondered which factor was critical. Therefore, we recruited two new groups: one group (n = 11) waited 30 min between blocks 2 and 3, and a second group (n = 10) waited 6 h (30-min and 6-h groups in Fig. 1B). In the 30-min group, subjects left the experiment room after completion of block 2 and waited in another area in the laboratory for exactly 30 min. Once they returned to the experiment room, we observed that generalization had not been reduced significantly [Fig. 4A; post hoc pairwise comparison, block 2 vs. block 3, P = 0.294, following repeated-measures ANOVA with a within-subject factor of mean a_p for blocks 1–8, effect of block, F_(7,70) = 9.688, P < 0.001]. Therefore, 30 min away from practice resulted in no evidence of reduction in probe forces. After completion of block 4, the 30-min group returned 24 h later. We now observed a reduction in a_p (post hoc pairwise comparison, block 4 vs. block 5, P = 0.001).

Fig. 4.

Sleep is not a necessary factor for the reduction of probe forces. A: 30-min group. Although 30 min was not sufficient to reduce the probe forces, after an overnight passage of time these forces declined. Asterisk denotes the block at which point all subsequent blocks were statistically significant for probe force index according to post hoc analysis with a 0.05 significance level. B: 6-h group. After 6 h of time away from practice, probe forces were reduced. Asterisk denotes the block at which point all subsequent blocks were statistically significant for probe force index according to post hoc analysis with a 0.05 significance level. C: summary of data as a function of time. Change in the probe forces was measured by computing the difference between the probe forces in blocks 2 and 3 for each subject. The same procedure was followed for computing the change in ratio of probe to train forces. Data are means ± SE. Asterisk denotes significant difference from the changes observed in the 24-h group at the 0.05 significance level.

In the 6-h group, subjects left the experiment room after completion of block 2 and returned after 6 h of wakefulness. On their return, we observed a significant drop in a_p [Fig. 4B; post hoc pairwise comparison, block 2 vs. block 3, P = 0.028, following repeated-measures ANOVA with a within-subject factor of mean a_p in blocks 1–8, effect of block, F_(7,63) = 5.083, P < 0.001]. We observed no further decrease in the forces to probe target after a 24-h break between blocks 4 and 5 (post hoc pairwise comparison, block 4 vs. block 5, P = 0.827). In fact, the drop following 6 h was indistinguishable from the group who waited 24 h after block 2 [post hoc comparison of 24- vs. 6-h group, P = 0.761, following a 1-way ANOVA for difference in probe force index between block 2 and block 3 across all groups, F_(3,40) = 4.074, P = 0.013]. Therefore, the data from the 6-h group revealed that sleep was not necessary to achieve a reduction in generalization. Rather, 6 h of time away from practice was sufficient.

On day 2, both the 30-min and the 6-h groups returned to the laboratory and were retested (block 5; Fig. 4, A and B). For the 30-min group, on day 1, there had been no reduction in the generalization forces (as in the 3-min group), but now after an overnight period away from practice, they exhibited a reduction (statistics reported above). In contrast, for the 6-h group, there had been a reduction in the forces to the probe target in day 1. Now, after this overnight passage of time, they exhibited no further reductions. This observation reiterates the finding that passage of time, and not sleep, was the critical factor that resulted in the ability to detect inefficient behavior and reduce the probe forces. As in the 3-min group, the 30-min and 6-h participants reached for four blocks on day 2. Unlike the 3-min group, these participants did not show any statistically significant change across these additional blocks (P > 0.100 for all post hoc comparisons, blocks 5–8 within each group).

Figure 4C summarizes some of these results. We have plotted the change in probe forces as a function of time away from practice (from block 2 to 3). ANOVA with a between-subject factor of time yielded a significant effect of group [change in generalization, F_(3,40) = 4.074, P = 0.013]. We found that neither increased practice nor a short amount of rest was sufficient to achieve the reduction (post hoc comparisons, 24 h vs. 3 min, P = 0.0.023 and 24 h vs. 30 min, P = 0.048). Rather, 6 or 24 h were sufficient (post hoc comparisons, 24 vs. 6 h, P = 0.761).

Confounding variables.

Two confounding variables are time of initial acquisition and time of the followup test sessions, which may affect properties of motor memory through circadian rhythms. We approached this problem by first considering whether time of acquisition (initial training) affected the magnitude of the forces that subjects produced for the train and probe targets. We found no evidence that time of acquisition served as a significant factor in predicting the forces at the end of training in block 2 (a_t, t = −0.721, P = 0.474; a_p, t = 1.478, P = 0.146). Next, we considered time of testing (recall) and asked whether the reduction in the forces produced for the probe target was affected by this variable. We found that time of testing did not serve as a significant factor in predicting the reduction in a_p (t = −1.141, P = 0.261). Indeed, only the time away from initial practice proved to be a significant predictor of the reduction in a_p (t = −2.435, P = 0.020). Therefore, time of acquisition and time of testing appeared to have little or no effect on the amount of forces that the subjects produced toward either target. The relevant factor was time away from practice.

Practice, but not passage of time, improved the task-relevant forces.

Finally, we focused on performance for the train target and asked whether time away from practice produced any advantages in performance over and beyond benefits from practice alone. The data for various measures of performance to the train target are shown in Fig. 5. Using the average force index a_t of each training block, we performed a repeated-measures ANOVA with a between-subject factor of time and found improvements across practice [effect of block, F_(7,259) = 24.616, P < 0.001] but no significant effect of group [group × block interaction, F_(21,259) = 1.310, P = 0.168]. This implied that increased practice alone was sufficient to improve task-relevant forces. We performed the same analysis using maximum displacement on the reach out to the train target as the variable of interest and found similar results. Again, we saw a significant difference across blocks of practice [effect of block, F_(7,259) = 62.319, P < 0.001] but no interaction between group and block [F_(21,259) = 1.384, P = 0.126]. When we considered consistency of movements (a measure of correlation between neighboring trials), the results continued this trend: consistency improved with practice [effect of block, F_(7,259) = 83.644, P < 0.001], but no additional benefits were found due to passage of time [group × block interaction, F_(21,259) = 0.897, 0.618]. Finally, we considered reaction time and found a monotonic decrease with practice for the training target [effect of block, F_(7,259) = 7.622, P < 0.001]. However, there were no additional benefits to the changes in reaction time due to passage of time [group × block interaction, F_(21,259) = 0.763, P = 0.763].

Fig. 5.

Changes in speed and accuracy of movements with practice and passage of time. A: data for the 3-min group. The top plot shows maximum displacement of the hand (with respect to a straight line to the target) during the reach out to the train target. The middle plot displays consistency of the reaching movements for the train target. Consistency refers to average correlation between pairs of reach trajectories in bins of 7 trials. The bottom plot displays reaction time, i.e., the time from appearance of the target to start of the reach for the train and probe targets. For the train target, bins of 7 trials were used to calculate the reaction time, and bins of 5 trials were used for the probe target. For all measures, the data are across-subject means ± SE. Each block is 192 trials divided into 24 probe targets and 168 train targets. B: displacement, consistency, and reaction times for the 30-min group. C: displacement, consistency, and reaction times for the 6-h group. D: displacement, consistency, and reaction times for the 24-h group.

A decrease in reaction time was also observed for reaches to the probe target across training, indicating that the improvement in this performance measure was not direction specific [effect of block, F_(7,259) = 14.568, P < 0.001]. As we had observed for the train target, there were no additional benefits due to passage of time for the reaction time of reaches to the probe target [group × block interaction, F_(21,259) = 0.925, P = 0.559]. Noticeably, the reaction time for the probe target was slightly longer than for reaches to the train target. This is presumably due to the infrequent nature of the probe trials.

In summary, practice resulted in an increase of task-relevant forces, a reduction in hand displacement, a reduction of movement variability, and a reduction of reaction times, all of which improved performance. However, the groups that experienced passage of time in between the blocks of practice did not show any benefit in these variables over and above what practice alone provided. Together, it appears that practice (and not time) enhanced the task-relevant component of motor memory, whereas time (and not practice) reduced the task-irrelevant component.

Task-irrelevant forces were minimized if they had kinematic consequences.

The task-irrelevant forces that our participants produced in their reaching movements toward the probe target did not have kinematic consequences as these movements were always in error-clamp. To reduce these forces, time away from practice was necessary. What happens if the production of task-irrelevant forces coincides with kinematic consequences?

Examples of such task-irrelevant forces are those that subjects produced during the return phase of their reach to the train target (Fig. 2A). Using the same force index, we quantified these forces during error-clamp trials and have plotted the results in Fig. 6. We found that participants produced very little force throughout the entire experiment. Indeed, forces on the reach back were not significantly different from 0 by the end of training in block 3 for all groups (P > 0.100 for all t-tests).

DISCUSSION

We designed a task in which with practice, subjects learned to produce task-relevant motor commands, countering a perturbation as they reached to a target. This practice also resulted in generalization, which we measured during reaches to a probe target. Although generalization is often considered a hallmark of learning, in our task these motor commands were task-irrelevant as all reaches to the probe target were in error-clamp, making it so that any forces that were generalized to the probe target were unnecessary. Importantly, production of the task-irrelevant forces did not result in kinematic errors, making it so that the only indication of this inefficiency was an intrinsic measure of effort.

We found that practice alone was not sufficient to reduce the task-irrelevant forces: the more participants practiced reaching to the train target, learning to produce task-relevant forces, the more they produced task-irrelevant forces as they reached to the probe target. However, at 24 h after end of initial practice, but not 3 or 30 min, subjects were able both to maintain production of the task-relevant forces for the train target as well as to reduce the task-irrelevant forces for the probe target. Sleep was not necessary to achieve this feat, as the reductions were also observed following 6 h of time away from practice. Therefore, whereas practice appeared critical in allowing for the improvement in the task-relevant motor commands, it was time away from practice that appeared critical in allowing for the reduction in the task-irrelevant commands.

It seems likely that passage of time away from practice alters motor memory, affording certain measures of stability to its representation: a 6-h time window corresponds with some previous studies of force-field learning that found it to be a critical amount of time necessary to reduce interference from competing tasks (Overduin et al. 2006; Shadmehr and Brashers-Krug 1997; but see Caithness et al. 2004) and the time span over which the neuronal basis of the memory was significantly altered (Shadmehr and Holcomb 1997). Similarly, in sequence-learning tasks, it was found that 6–8 h of time provided protection from interference from a second sequence (Korman et al. 2007; Walker et al. 2003). Here, we found an additional dimension to the benefits of passage of time: time away from practice made it possible to represent the memory more efficiently through reduction of the component that was energetically wasteful.

From a computational perspective, generalization is thought to be a behavioral assay of the width of the receptive fields of the neurons that participate in the process of learning (Hwang and Shadmehr 2005; Shadmehr 2004). One way to view reduction of generalization is via a reduction in the width of these receptive fields, which may occur through weakening of learning-induced potentiated synapses. That is, it is possible that initial practice in the task produces widespread changes in synaptic strength, resulting in wide generalization, and then with passage of time away from practice, some of the changed synapses are returned to near baseline, especially if the generalization produced task-irrelevant behavior. In this hypothesis, the motor memory is made more efficient through a return of the learning-induced synaptic changes toward baseline for those components of the memory that were associated with task-irrelevant motor commands.

What may be the mechanism with which this is achieved? Recent in vivo imaging techniques have allowed for longitudinal tracking of the formation and elimination of specific dendritic spines in mice. Two studies have found that training of a new motor skill in mice leads to increases in the rate of dendritic spine formation (Xu et al. 2009; Yang et al. 2009) with new spines forming within 1 h of initial training (Xu et al. 2009). However, with passage of time and continued practice in additional training sessions, the rate of elimination of spines also increases, bringing the net synaptic changes near baseline. Indeed, it has been hypothesized that a critical function of sleep may be to prune memories that are acquired during wakeful hours (Tononi and Cirelli 2006), an idea that is supported by data in juvenile mice (Maret et al. 2011) and fruit flies (Bushey et al. 2011).

Here, we found that instead of sleep, 6 h of time was sufficient to reduce a large part of the task-irrelevant motor commands while maintaining the task-relevant portion. This independence from sleep appears consistent with other results in motor adaptation tasks. In a number of adaptation tasks, it has been shown that sleep does not add significantly to the benefits of time away from practice (Debas et al. 2010; Donchin et al. 2002; Doyon et al. 2009) and is not necessary for memory consolidation (Overduin et al. 2006; Shadmehr and Brashers-Krug 1997). Possibly, the previously reported wide-scale synaptic changes are responsible for the improved motor efficiency observed here, but unlike other tasks, motor adaptation may not require sleep for this process of synaptic pruning to be initiated.

We interpret our observation of decreased generalization over time as an improvement in the efficiency of the participants' movements, as these forces were an energetically costly byproduct of learning. Although these forces are small in magnitude, they are not completely negligible and incur a metabolic cost (Huang et al. 2012). In our task, there were no kinematic errors or differences in performance feedback to alert the brain that these forces were unnecessary. We posit that the energetic cost of generalization served as a cue to alter these motor commands.

A concern in our study may be regarding the tool that we relied on to measure the task-irrelevant motor commands: error-clamps. A potential limitation is that error-clamps are not innocuous probes of motor memory (Vaswani and Shadmehr 2013). When they are given as a continuous block of trials, the changes that take place in motor commands appear to be partly due to an ability of the nervous system to detect a contextual switch from train trials (in which errors take place) to error-clamp trials (in which errors are eliminated). Here, the reaches to the probe target were always in error-clamp, eliminating the potential problems associated with changes in context.

Our results suggest that only the first break of 6 or 24 h following initial learning contributed significantly to reduced generalization. A subset of our participants who performed the 24-h group paradigm experienced two additional set breaks of 24 h after the first crucial break. Despite a trend toward reduction, there was no statistically significant decrease in generalization across these additional rest periods. The remaining participants did not experience any further set breaks of critical duration but instead experienced additional practice; blocks 5–8 were all performed in the same session. Although the participants in the 3-min group did display a slight decrease across this long practice session, the subjects in the 30-min and 6-h groups did not show a change in forces due to generalization. Therefore, we did not find strong evidence that additional set breaks of critical duration or additional practice after the first set break could further reduce the inefficient motor behavior. Although neither of these interventions appeared to improve performance, future experiments could benefit from systematically controlling these two variables and investigating the possible benefits of combining practice and additional long break durations.

With future work, we hope to understand whether generalization that results in task-irrelevant force production could be completely eliminated. Although here we observed benefits from passage of time, at the end of the 4-day experiment subjects still produced significant task-irrelevant forces. Perhaps additional passage of time, for example 48 or 72 h, could provide a larger drop in generalization. However, we believe that passage of time may not be the only critical factor but that the coupling of the train targets and probe targets may play a role. Reaching to the probe target provides the subject with the necessary experience to identify that this effort is unnecessary, but additional practice to the train target may act to counter this behavior. Therefore, altering the probability of the probe target with respect to the train target may affect the patterns of generalization. Finally, we note that the largest drops in generalization occurred after large durations of rest, which led to more forgetting and a larger initial error in subsequent training sessions. As training progressed, there was less forgetting across rest breaks, and generalization was not decreased further. It is possible that greater relearning at the start of each block, which could be achieved by introducing null blocks or catch trials into our paradigm, could also serve to reduce the task-irrelevant generalization.

In summary, we studied a motor task in which learning produced a memory that included both task-relevant and task-irrelevant motor commands. With practice, the task-relevant motor commands improved, and this improvement did not benefit from time away from practice. However, with time away from practice, the production of the task-irrelevant motor commands was reduced, resulting in a more efficient control of movements. The crucial factor necessary for this reduction was time away from practice, as additional training was not sufficient to improve efficiency. Therefore, our work illustrates a previously unknown property of motor memory: practice (and not time) improves the task-relevant component of motor memory, whereas time (and not practice) makes the task-irrelevant component eligible for reduction, a process that makes the motor memory more efficient.

GRANTS

This work was supported by grants from the National Institute of Neurological Disorders and Stroke (NS-078311) and the Human Frontier Science Program.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the author(s).

AUTHOR CONTRIBUTIONS

S.E.P. and R.S. conception and design of research; S.E.P. performed experiments; S.E.P. analyzed data; S.E.P. and R.S. interpreted results of experiments; S.E.P. prepared figures; S.E.P. drafted manuscript; S.E.P. and R.S. edited and revised manuscript; S.E.P. and R.S. approved final version of manuscript.