Abstract
Movement error is a driving force behind motor learning. For motor learning with discrete movements, such as point-to-point reaching, it is believed that the brain uses error information of the immediately preceding movement only. However, in the case of continuous and repetitive movements (i.e., rhythmic movements), there is a ceaseless inflow of performance information. Thus, an accurate temporal association of the motor commands with the resultant movement errors is not necessarily guaranteed. We investigated how the brain overcomes this challenging situation. Human participants adapted rhythmic movements between two targets to visuomotor rotations, the amplitudes of which changed randomly from cycle to cycle (the duration of one cycle was ∼400 ms). A system identification technique revealed that the motor adaptation was affected not just by the preceding movement error, but also by a history of errors from the previous cycles. Error information obtained from more than one previous cycle tended to increase, rather than decrease, movement error. This result led to a counterintuitive prediction: providing visual error feedback for only a fraction of cycles should enhance visuomotor adaptation. As predicted, we observed that motor adaptation to a constant visual rotation (30°) was significantly enhanced by providing visual feedback once every fourth or fifth cycle rather than for every cycle. These results suggest that the brain requires a specific processing time to modify the motor command, based on the error information, and so is unable to deal appropriately with the overwhelming flow of error information generated during rhythmic movements.
Introduction
Motor learning is thought to proceed through iterative corrections of motor commands, based on movement error (Shadmehr et al., 2010). For motor learning involving discrete movements, a mathematical model, based on the assumption that the brain uses information from only the immediately preceding movement, has successfully explained trial-by-trial behaviors that are observed when a novel environment is imposed (Thoroughman and Shadmehr, 2000; Donchin et al., 2003; Cheng and Sabes, 2006; Nozaki and Scott, 2009). System identification studies, examining more general trial-dependent relationships between movement errors and perturbations (Scheidt et al., 2001, 2012), have indicated that although the movement correction may be influenced by information obtained beyond a single trial, the effect of the immediately preceding movement error is dominant.
In theory, using movement error to drive motor learning requires proper association between each movement error and a corresponding motor command. This requirement is generally guaranteed for motor learning with discrete movements, in which the intertrial interval (ITI) time is sufficiently long for the process to modify the motor command based on the error information. However, the brain receives a ceaseless flow of error information for rhythmic movements, which are another fundamental and extensive class of behaviors (Hogan and Sternad, 2007), so an accurate temporal association between the motor commands and the resultant movement errors becomes a challenging task for the brain to perform.
We have previously reported that the performance of motor adaptation to a visual rotation deteriorated with regard to rhythmic reaching movements, compared with a discrete reaching movement with ITIs of >1 s with respect to the rate of error reduction and converged plateau level (Ikegami et al., 2010). This implies that the impaired motor learning for rhythmic movement may be attributed to the degraded temporal association between the motor commands and the resultant movement errors. To examine this possibility, we used a system identification technique to clarify how the visual error information obtained from a cycle of rhythmic movements was used to correct the movements of subsequent cycles during visuomotor adaption of rhythmic reaching movements. The identified model for visuomotor learning demonstrated that the correction of a movement was based on the error information from not only the previous cycle, but also from earlier cycles. Furthermore, the error information of more than one previous cycle increased, rather than decreased, movement error. This finding inevitably led to a counterintuitive prediction: providing visual feedback on fewer cycles, rather than every cycle, should enhance visuomotor adaptation, because the harmful effect of visual information from several previous cycles can be removed. We examined this prediction by ensuring that participants adapted rhythmic movements to a constant visual rotation with visual feedback for every cycle or with intermittent visual feedback (once every 2–5 cycles). As predicted, the visuomotor learning performance was enhanced in the intermittent visual feedback condition (once every fourth and fifth cycles), indicating that the effect of the error feedback goes beyond that of one cycle in visuomotor learning with rhythmic movements.
Materials and Methods
Participants.
A total of 77 neurologically normal volunteers (18 females and 59 males, aged 19–41 years) participated in one of three experiments. Only one participant took part in both Experiments 1 and 3, described below. All participants were right-handed, as assessed by laterality score (84.2 ± 22.6; mean ± SD) of the Edinburgh Handedness Inventory (Oldfield, 1971), except for one male. They were all naive to the purpose of the experiments, which were approved by the ethical committee of the Graduate School of Education, The University of Tokyo, and all participants signed an institutionally approved consent form.
Apparatus.
Participants sat on a straight-backed chair while grasping the handle of a robotic manipulandum with their right hand (Phantom Premium 1.5HF; SensAble Technologies) in a darkened room. A spring simulated by the device (1.0 N/mm) generated a virtual horizontal plane, on which the handle movement was restricted. A projector was used to display the position of the handle with a white cursor (diameter, 8 mm) on a horizontal screen (45 × 60 cm), placed ∼13 cm above the virtual plane and ∼15 cm below the shoulder. The screen board prevented the participants from directly seeing their arm. Participants controlled the cursor by the performance of rhythmic out-and-back movements (more details below) between a start position and a target (diameter, 8 mm) displayed on the screen. The start position was located ∼25 cm in front of the body in a midsagittal plane, and the target was 7 cm away from the starting position. The starting position and the cursor were always visible. The position and velocity of the handle were A/D converted by 500 Hz and were stored for later offline analysis.
Procedure.
Participants were instructed to move the cursor between the start position and the target by performing straight, fast, and uncorrected out-and-back movements with a sharp reversal at the target. Before each trial, they were required to place the cursor at the start point, following which the target appeared on the screen. The participants performed the out-and-back movements repeatedly, without any breaks between cycles, until the target disappeared (∼2.5 Hz), and were instructed not to make any deliberate correction within each cycle. Movement correction was allowed only at the turnaround point when the inward movement turned outward. Before the experimental session, participants practiced by synchronizing their movements with the beep of a metronome at 2.5 Hz (∼400 cycles). The metronome was turned off during the experimental session so that there was no need to adjust the movement with the sound by pausing or accelerating/decelerating the movements.
Experiment 1.
Experiment 1 was designed to examine the way in which visual error modifies subsequent rhythmic movements by a system identification technique (Ljung, 1999). Eight participants performed continuous, rhythmic, out-and-back movements for 60 s (i.e., ∼150 cycles). The cursor was displayed at the position obtained by rotating the handle position around the start position at a random angle selected from a Gaussian distribution (mean = 0°, SD = 15°) for every cycle. The random sequence of the rotation was the same for all participants. Using the sequences of the rotation and the movement errors, we identified the system for motor learning of rhythmic movements (for more details, see Identification of the motor learning system, below).
Experiments 2 and 3.
Motor adaptation to a constant visuomotor rotation was examined when visual feedback of the hand position, i.e., the cursor, was provided either continuously (CON; Experiment 2) or intermittently (Experiment 3). A total of 70 participants were asked to adapt to a 30° counterclockwise (CCW) rotation under one of the following five conditions: the visual feedback was provided during every cycle [Experiment 2: CON condition (n = 20)] or once every two, three, four, or five cycles [Experiment 3: INT2 (n = 10), INT3 (n = 10), INT4 (n = 10), and INT5 (n = 20) conditions, respectively]. The experiment comprised baseline (40 s, ∼60 cycles) and rotation (100 s, ∼250 cycles) sessions, with 3 min rests between sessions. In the baseline session, the cursor was displayed just above the handle position; in the rotation session, the cursor was displayed at the position obtained by rotating the handle position around the start position by 30° in the CCW direction.
Data processing.
The handle position and velocity data were low-pass filtered using the fourth-ordered Butterworth filter, with a cutoff frequency of 30 Hz. The position at the peak outward handle velocity was calculated for each cycle. The turnaround position for each cycle, in which the direction of the handle movement switched from inward to outward, was also detected. Ideally, this turnaround position would be at the start position, but there was often a slight positional shift, due to the difficulty in returning the handle to the start position in the absence of visual feedback. The movement direction was defined as the direction from the turnaround position to the position at the peak outward velocity of the handle, and the target direction was defined as the direction from the start position to the target position. Movement performance was evaluated by the directional error, defined as the angular difference between the movement and target directions.
Identification of the motor learning system.
We constructed a state space model for the motor learning system of rhythmic movements as follows:
These equations mean that the movement error e in the ith cycle is the sum of the movement direction of the hand (or the internal state) X and the imposed visual rotation R (Eq. 1), and that X in the next cycle is determined as a result of the modification of X according to e (Eq. 2). The constant α represents the spontaneous loss of motor learning and is ordinarily close to 1 (Smith et al., 2006; Nozaki and Scott, 2009), the constant kn represents the degree of the internal state update to the movement error for the nth previous cycle (hereafter referred to as error sensitivity), and M is the model order. Differently from previous models (Thoroughman and Shadmehr, 2000; Donchin et al., 2003; Cheng and Sabes, 2006), we assumed that the internal state referred to the movement errors up to the previous M cycles. Unaltered proprioception in the visuomotor learning task can reduce the kn for large movement errors (Wei and Körding, 2009), but the assumption that the kn is constant (i.e., a linear system) is valid when the imposed visual rotation is small (Cheng and Sabes, 2006), as in the present study.
Determination of model order and the confidence intervals of the parameters.
The values of kn were determined by fitting the model to the 100-cycle sequences (from 11th to 110th cycle in Experiment 1) of the rotation (R1, R2, · · ·) as an input, and averaging the movement errors among participants (e1, e2, · · ·) as an output for M = 1–9 using a least-squares method. The fitting was conducted by changing the value of α from 0.95 to 0.99 at intervals of 0.01. To identify the model, we examined (1) Akaike Information Criteria (AIC) (Akaike, 1974), (2) whiteness of the residual, and (3) appropriate step response. The AIC was calculated as AIC = Nlog(SSR/N) + 2M, where N = 100 (the number of data points) and SSR is the sum of squared residual between the model prediction and actual data. A smaller AIC means a better model. The whiteness (i.e., absence of time correlation) of the time series of the residual was evaluated using the Anderson test (Box et al., 1994). A lack of whiteness signifies that the identified model is not reliable. We also examined the step response to confirm the validity of the model.
The confidence interval for each kn was obtained as follows (Scheidt et al., 2012). The movement error sequence for each participant was fitted into the model, with the M fixed to the value identified for the average data. The 95% confidence intervals were obtained for each kn by the mean and SE among eight participants as mean ± 2.365 SE, where 2.365 is the critical value of the two-tailed t distribution (5%) of seven degrees of freedom.
Results
Kinematics features of rhythmic movement
The number of movement cycles performed during the rotation session was 158.5 ± 23.7 (Experiment 1; 60 s) and 271.5 ± 24.3 (Experiments 2 and 3; 100 s), indicating that the participants kept the predetermined tempo (2.5 Hz) without the metronome's sound. Movement was performed without a pause between cycles, because the duration during which the movement velocity was <5% of its peak value was <2% of the total time. The movement distance and velocity were also kept stable (Fig. 1); the coefficient variations of the peak velocity and movement distance were <9%. Furthermore, in Experiments 2 and 3, there were no significant differences in the movement distance and peak velocity among the conditions (CON, INT2-5; p > 0.05 by one-way ANOVA), indicating that the differences in motor learning performance, if any, could not be explained by the differences in the kinematics.
System identification of motor learning for rhythmic movements
In Experiment 1, participants tried to adapt rhythmic movements to a visual rotation, the magnitude of which varied randomly from cycle to cycle. By fitting the model (Eqs. 1, 2) to the sequence of imposed visual rotations and that of the movement errors averaged across the participants, we identified error sensitivities kn (n = 1–9) when the model order was changed from M = 1 to M = 9, and when α was changed from 0.95 to 0.99. The AIC reached a minimum when M = 6 (Fig. 2a) and the whiteness of the residual was statistically confirmed only when M ≥ 5 (p < 0.05). Figure 2b demonstrates that the prediction of the model was in accordance with the time series of the movement errors averaged across participants. The AIC indicates that M = 6 is the best choice, but the model predicted an unusual adaptation pattern when a constant visual rotation was applied (Fig. 2c–e). This unusual step response was caused by the fact that the sum of kn was a negative value: According to Equations 1 and 2, the final plateau level of the error ep can be represented as ep = (1 − α)R/(1 − α + ∑n = 1Mkn), and therefore the negative sum of kn leads to ep > R, which does not make sense. Conversely, the predictions using the M = 5 model fit the motor adaptation data well, especially for α = 0.98, when a constant visual CCW rotation of 30° rotation was imposed (Fig. 2d). On the basis of these considerations, we adopted a model with M = 5 and α = 0.98.
We refit the data with the model (M = 5 and α = 0.98) to calculate the error sensitivities kn for each participant. The 95% confidence intervals for k1, k2, and k5, which were calculated from their means and SE, did not contain zero (k1 = 0.185 ± 0.103, k2 = −0.063 ± 0.049, k5 = −0.037 ± 0.030; Fig. 2f), indicating that the motor learning system for rhythmic movements used the error information obtained from up to five previous cycles to modify the internal state of the current movement cycle. Significant negative error sensitivities were unexpectedly found in the second and fifth previous cycles, implying that the visual error feedback that ceaselessly flows into the motor learning system may contaminate the visuomotor adaptation of rhythmic movements. In particular, the negativity of k2 was robust and independent of the model order M, and every identified model with M = 2–5 exhibited negative values for k2. Compared with the model in which M = 1 exhibited a faster convergence, the negative sensitivity in k2 appeared to result in a slower convergence of the learning curve similar to that observed in the experimental data (Fig. 2c–e).
Effect of intermittent visual feedback on visuomotor learning
A prediction by the identified model is that intermittent visual feedback can enhance motor learning performance due to the suppression of the negative influence of the error feedback. To test this prediction, we investigated the effect of intermittent visual feedback on motor learning performance (Experiment 3). Each participant was assigned to one of the conditions in which they received visual feedback regarding their hand position from a cursor only once every two, three, four, or five cycles (INT2, INT3, INT4, and INT5, respectively).
The motor learning performance (an average of the values obtained from the 101st to 200th cycle) under the continuous feedback conditions in Experiment 2 (CON) was similar to the performance observed under the INT2 and INT3 conditions (vs CON, p > 0.05 by two-tailed t test; Fig. 3). However, the motor learning performance under the INT4 and INT5 conditions significantly improved (vs CON, p < 0.01 by two-tailed t test), even though the amount of visual feedback was only <25% of the CON conditions. These results clearly show that, as predicted, intermittent visual feedback can actually boost the visuomotor learning of rhythmic movements. It also suggests that the visual error information provided within ∼1.2 s (three cycles × 400 ms) after a cycle (i.e., CON, INT2, INT3) may have harmful effects on the correction of the motor command in subsequent cycles.
Discussion
Decades of research indicate that information regarding movement error is critical to learning and improving motor skills (Thorndike, 1931; Trowbridge and Cason, 1932; Adams, 1971; Schmidt and Lee, 2005). Previous studies generally assumed that obtaining error information is always preferable for learning/improving motor skills compared with not obtaining any feedback information (Shadmehr et al., 2010). However, considering that the correction of motor command based on error information is a physiological process that needs a certain amount of processing time, we predicted that the correction is adversely affected when the processing time is insufficient. Previous studies have reported that motor learning during a discrete reaching task deteriorated when the ITI was <1 s (Bock et al., 2005; Francis, 2005). Thus, we predicted that such impairment of learning with the shorter ITI should be the most prominent for rhythmic movements without any break between cycles. Indeed, we have shown that adaptation of out-and-back movements to a visual rotation was worse when it was performed continuously (i.e., rhythmic movement) than discretely; the convergence of movement error was slower, and the final plateau level of movement error was greater (Ikegami et al., 2010).
Thus, a problem is how the motor learning system processes the error information entering ceaselessly into the system during such rhythmic movements. To investigate this issue, participants adapted the rhythmic out-and-back movements to a visual rotation, the magnitude of which randomly varied from cycle to cycle, and the motor learning system (Eqs. 1, 2) was identified using a system identification technique (Ljung, 1999). In contrast to the conventional model for discrete movement with sufficient amount of ITI, the identified model indicates that movement of a cycle was corrected not only according to the error information of the immediately preceding cycle, but also according to that of more than one previous cycle (Fig. 2). This result suggests that the temporally appropriate association between the motor command and the resultant error is not guaranteed when the motor learning system ceaselessly receives the movement error information.
More surprisingly, the movement error information of more than one previous cycle can have a negative effect on the correction of the movement, as reflected by the negative value of the sensitivity coefficient (Fig. 2f). Interestingly, such negative effects will lead to a counterintuitive prediction: providing error information in only a fraction of cycles rather than every cycle improves the adaptation, because the negative influence of error information from more than one previous cycle can be suppressed. As predicted, we observed that providing visual feedback once every fourth or fifth cycle, rather than every cycle, significantly enhanced motor adaptation to a constant visual rotation in terms of the final plateau level of the movement error (Fig. 3). Longer training with CON condition would not decrease the error to this level, because it remained unchanged by the extensive training with >1000 movement cycles (Ikegami et al., 2010). Notably, previous psychological studies (Winstein and Schmidt, 1990; Wulf and Shea, 2002) have demonstrated the beneficial effects of a reduced frequency in error feedback on motor learning. These studies used a discrete movement task performed with a much longer ITI (e.g., 15–20 s). Furthermore, the enhancement by reducing frequency in error feedback was observed only for retention, while acquisition (e.g., the learning curve) was adversely affected. Thus, their findings are not directly related to the problem of temporal association of motor command with the error information, as reported in the current study.
Huang and Shadmehr (2007) proposed that the motor learning system for discrete reaching movement uses memory trace of error that decays with time to modify the motor command. According to this idea, when the ITI is not long enough, the motor learning system cannot fully use the memory trace information, resulting in the degradation of motor learning for shorter ITIs. Our model implies the possibility that such a motor trace may persist even across movements within a small time window (∼1.2 s; 0.4 s for one cycle × three cycles), preventing proper temporal association of motor commands with the resultant error and further contributing to motor learning degradation. However, one remaining puzzle is why such error feedback beyond cycles can have a negative effect on motor learning. Although the neural plasticity mechanisms underlying motor learning have been progressively elucidated in the cerebellum (Medina and Lisberger, 2008) and vestibular nuclei (McElvain et al., 2010), little is known regarding the time required for neural processing of an error signal, or how the motor learning system processes an error signal when it receives continuous neuronal inputs. Further physiological and computational studies are required to elucidate the mechanism underlying the error information beyond cycles and its negative effect on motor learning, as found in the present study.
Footnotes
This work was supported by the NEXT Program (LS034 to D.N., LS139 to R.O.), by KAKENHI (20670008 to D.N, 20588660 to T.I) and by SRBPS, MEXT (R.O.). We thank Issac Kurtzer, Ganesh Gowrishankar, Makoto Miyazaki, Gentaro Taga, Yoshiharu Yamamoto, and members of the Nozaki laboratory for their helpful comments.
References
- Adams JA. A closed-loop theory of motor learning. J Mot Behav. 1971;3:111–149. doi: 10.1080/00222895.1971.10734898. [DOI] [PubMed] [Google Scholar]
- Akaike H. A new look at the statistical model identification. IEEE Trans Automat Contr. 1974;19:716–723. [Google Scholar]
- Bock O, Thomas M, Grigorova V. The effect of rest breaks on human sensorimotor adaptation. Exp Brain Res. 2005;163:258–260. doi: 10.1007/s00221-005-2231-z. [DOI] [PubMed] [Google Scholar]
- Box GE, Jenkins GM, Reinsel GC. Time series analysis: forecasting and control. 3rd edition. Englewood Cliffs, NJ: Prentice-hall; 1994. [Google Scholar]
- Cheng S, Sabes PN. Modeling sensorimotor learning with linear dynamical systems. Neural Comput. 2006;18:760–793. doi: 10.1162/089976606775774651. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Donchin O, Francis JT, Shadmehr R. Quantifying generalization from trial-by-trial behavior of adaptive systems that learn with basis functions: theory and experiments in human motor control. J Neurosci. 2003;23:9032–9045. doi: 10.1523/JNEUROSCI.23-27-09032.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Francis JT. Influence of the inter-reach-interval on motor learning. Exp Brain Res. 2005;167:128–131. doi: 10.1007/s00221-005-0062-6. [DOI] [PubMed] [Google Scholar]
- Hogan N, Sternad D. On rhythmic and discrete movements: reflections, definitions and implications for motor control. Exp Brain Res. 2007;181:13–30. doi: 10.1007/s00221-007-0899-y. [DOI] [PubMed] [Google Scholar]
- Huang VS, Shadmehr R. Evolution of motor memory during the seconds after observation of motor error. J Neurophysiol. 2007;97:3976–3985. doi: 10.1152/jn.01281.2006. [DOI] [PubMed] [Google Scholar]
- Ikegami T, Hirashima M, Taga G, Nozaki D. Asymmetric transfer of visuomotor learning between discrete and rhythmic movements. J Neurosci. 2010;30:4515–4521. doi: 10.1523/JNEUROSCI.3066-09.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ljung L. System identification: theory for the user. 2nd edition. Upper Saddle River, NJ: Prentice Hall PTR; 1999. [Google Scholar]
- McElvain LE, Bagnall MW, Sakatos A, du Lac S. Bidirectional plasticity gated by hyperpolarization controls the gain of postsynaptic firing responses at central vestibular nerve synapses. Neuron. 2010;68:763–775. doi: 10.1016/j.neuron.2010.09.025. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Medina JF, Lisberger SG. Links from complex spikes to local plasticity and motor learning in the cerebellum of awake-behaving monkeys. Nat Neurosci. 2008;11:1185–1192. doi: 10.1038/nn.2197. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nozaki D, Scott SH. Multi-compartment model can explain partial transfer of learning within the same limb between unimanual and bimanual reaching. Exp Brain Res. 2009;194:451–463. doi: 10.1007/s00221-009-1720-x. [DOI] [PubMed] [Google Scholar]
- Oldfield RC. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia. 1971;9:97–113. doi: 10.1016/0028-3932(71)90067-4. [DOI] [PubMed] [Google Scholar]
- Scheidt RA, Dingwell JB, Mussa-Ivaldi FA. Learning to move amid uncertainty. J Neurophysiol. 2001;86:971–985. doi: 10.1152/jn.2001.86.2.971. [DOI] [PubMed] [Google Scholar]
- Scheidt RA, Zimbelman JL, Salowitz NM, Suminski AJ, Mosier KM, Houk J, Simo L. Remembering forward: neural correlates of memory and prediction in human motor adaptation. Neuroimage. 2012;59:582–600. doi: 10.1016/j.neuroimage.2011.07.072. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schmidt RA, Lee TD. Motor control and learning: a behavioral emphasis. 4th edition. Champaign, IL: Human Kinetics; 2005. [Google Scholar]
- Shadmehr R, Smith MA, Krakauer JW. Error correction, sensory prediction, and adaptation in motor control. Annu Rev Neurosci. 2010;33:89–108. doi: 10.1146/annurev-neuro-060909-153135. [DOI] [PubMed] [Google Scholar]
- Smith MA, Ghazizadeh A, Shadmehr R. Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biol. 2006;4:e179. doi: 10.1371/journal.pbio.0040179. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Thorndike EL. Essentials of learning. New York: Century; 1931. [Google Scholar]
- Thoroughman KA, Shadmehr R. Learning of action through adaptive combination of motor primitives. Nature. 2000;407:742–747. doi: 10.1038/35037588. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Trowbridge MH, Cason H. An experimental study of Thorndike's theory of learning. J Gen Psychol. 1932;7:245–258. [Google Scholar]
- Wei K, Körding K. Relevance of error: what drives motor adaptation? J Neurophysiol. 2009;101:655–664. doi: 10.1152/jn.90545.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Winstein CJ, Schmidt RA. Reduced frequency of knowledge of results enhances motor skill learning. J Exp Psychol Learn Mem Cogn. 1990;16:677–691. [Google Scholar]
- Wulf G, Shea CH. Principles derived from the study of simple skills do not generalize to complex skill learning. Psychon Bull Rev. 2002;9:185–211. doi: 10.3758/bf03196276. [DOI] [PubMed] [Google Scholar]