Randomly changing the width of the target (random schedule) has a deep impact on the optimality of the reaching movements made by human participants, which suggests the existence of a switching cost for motor control. In addition, a change in target width made participants unable to take into account the error observed on the previous trial to update their movement plan for the next trial. These results suggest new principles for movement planning.
Keywords: motor control, motor planning, optimal control, switching cost
Abstract
Movement planning consists of choosing the intended endpoint of the movement and selecting the motor program that will bring the effector on the endpoint. It is widely accepted that movement endpoint is updated on a trial-by-trial basis with respect to the observed errors and that the motor program for a given movement follows the rules of optimal feedback control. In this article, we show clear limitations of these theories. First, participants in the current study could not tune their motor program appropriately for each individual trial. This was true even when the participants selected the width of the target that they reached toward or when they had learned the appropriate motor program previously. These data are compatible with the existence of a switching cost for motor planning, which relates to the drop in performance due to an imposed switch of motor programs. This cost of switching shares many features of costs reported in cognitive task switching experiments and, when tested in the same participants, was correlated with it. Second, we found that randomly changing the width of a target over the course of a reaching experiment prevents the motor system from updating the endpoint of movements on the basis of the performance on the previous trial if the width of the target has changed. These results provide new insights into the process of motor planning and how it relates to optimal control theory and to an action selection based on the reward consequences of the motor program rather than that based on the observed error.
NEW & NOTEWORTHY
Randomly changing the width of the target (random schedule) has a deep impact on the optimality of the reaching movements made by human participants, which suggests the existence of a switching cost for motor control. In addition, a change in target width made participants unable to take into account the error observed on the previous trial to update their movement plan for the next trial. These results suggest new principles for movement planning.
movement planning, which includes all the processes occurring before the movement (detection and selection of target, selection of feedback control policy, etc.), is widely thought to follow two distinct rules. First, the control policy (how the brain will react with respect to incoming sensory input during the movement) is tuned according to optimal control laws (Diedrichsen et al. 2009; Pruszynski and Scott 2012; Shadmehr and Krakauer 2008; Todorov and Jordan 2002). Second, movement performance is adjusted on a trial-by-trial basis via learning from errors, even when learning or adaptation are not explicitly required (Dingwell et al. 2013; van Beers 2009; Verstynen and Sabes 2011; Wong and Shelhamer 2014). In this article, we demonstrate that these two rules are violated as soon as the target width changes randomly over the course of trials.
Such disruption of performance due to randomization is widespread in motor tasks (Afsanepurak et al. 2012; Bennett et al. 2010; Edin et al. 1992; Elliott and Allard 1985; Horak and Diener 1994; Khan et al. 2002; Pruszynski et al. 2008; Selen et al. 2009; but see White and Diedrichsen 2013). For instance, in a recent study (Orban de Xivry 2013), we asked participants to reach to either a narrow or wide target and found that participants reacted differently to a perturbation as a function of the target width (compatible with the minimum intervention principle of optimal control; Todorov and Jordan 2002) but unexpectedly also as a function of the schedule (random or blocked). That is, the modulation of the response to perturbations with target width was smaller in the random (target width randomly changes from trial to trial) than in the blocked schedule. The behavior in the blocked schedule was optimal because it minimized energy while maximizing reward.
A similar decrease in performance due to randomization is also observed in cognitive tasks where it falls in the category of switching costs under the name of mixing cost (i.e., the long-term cost of task switching). A switching cost relates to the inability of the participants to switch from one task to another without a penalty in behavior (Kiesel et al. 2010; Monsell 2003). Typically, in cognitive studies, this cost is reflected as an increase in reaction time (Allport et al. 1994; Rogers and Monsell 1995). For instance, when participants are required to report the color or the shape of a symbol on the basis of some abstract rules, the reaction time is shorter when the rule is presented in a blocked schedule (i.e., the rule remains unchanged for several trials in a row) than when the rule can change randomly from trial to trial. Such switching cost is also present when participants choose the rule that they want to apply on each trial (Arrington and Logan 2004, 2005).
Whereas in the previous article (Orban de Xivry 2013), we merely reported the observation that randomly changing the width of the target affected the optimality of motor behavior, we present in this article three experiments that provide an explanation for the absence of optimality in the random schedule and identify limits on the flexibility of motor planning. Importantly, in this study, as in many others (Ahmadi-Pajouh et al. 2012; Dimitriou et al. 2013; Nashed et al. 2012; Orban de Xivry 2013; White and Diedrichsen 2013), we consider that motor planning (i.e., before movement onset) consists of the derivation of a goal-directed feedback control law (Liu and Todorov 2007; Scott 2004; Todorov and Jordan 2002; Todorov 2004) and that responses to any perturbations during the movement are driven by this feedback law. Thus these perturbations at any time during movement allow us to probe the feedback control policy determined before movement (Gallivan et al. 2016; Liu and Todorov 2007; Nashed et al. 2012).
Each of these three experiments represents a critical test of the hypothesis that the absence of optimality in the random schedule is due to a switching cost and that this switching cost shares similarities with the one observed in cognitive tasks. In the first experiment, we test whether voluntary selection of target width can improve switching behavior, knowing that it does not in cognitive task switching experiments. Such failure to optimize the control policy under voluntary selection of target width would confirm that the difference between the random and blocked schedules was not related to the imposed target selection and to a too-short time for movement planning. The second experiment aims at testing whether the difference between the random and blocked schedules was due to the ignorance of the optimal motor behavior during the random schedule because it was always tested first. The third experiment investigates the limit in trial-to-trial switches to confirm the inability to fully modify the motor program from one trial to another. A failure of any of these experimental manipulations would cast doubt on the existence of a switching cost for motor control and would favor other alternative accounts for the observed differences between the random and blocked schedules. In contrast, confirmation of such a switching cost would provide new insights into the mechanisms of motor planning (Cisek and Kalaska 2010; Wong et al. 2015).
METHODS
Participants
Sixty-one healthy participants (21 for experiment 1 and 20 for experiments 2 and 3) were enrolled in the experiments after written informed consent. All participants had no history of neurological disorders, were right-handed, and were between 18 and 40 yr old. All procedures were approved by the Ethics Committee of the Université catholique de Louvain. We also analyzed some new aspects of the data from the 50 participants from our previous publication (Orban de Xivry 2013).
Setup
Participants were sitting in front of a robotic arm (Endpoint Kinarm; BKin Technologies, Kingston, ON, Canada) that controlled a cursor that appeared to be positioned at the same position in space as the hand through a mirror setting. Participants could not see their hand, and the displayed cursor was the only available visual feedback of their hand position. Hand position, velocity and acceleration, and the force exerted by the participants on the handle of the robot were recorded at 1,000 Hz and stored on a computer for offline analysis. The robot was also able to exert forces to perturb or direct the hand of the participants.
Protocol
Cognitive task switching experiment.
In this cognitive task switching experiment, participants had to indicate either the color or the shape (depending on a specific instruction) of a symbol by applying a force with their right or left hand (Fig. 1, cognitive task switching experiment). Depending on the schedule, the instruction varied randomly from trial to trial or remained unchanged for several trials in a row. More specifically, participants had to respond to the appearance of a visual stimulus [disk (radius: 1 cm; surface: 3.14 cm2) or square (1.75 × 1.75 cm; surface: 3.06 cm2)] that was either blue or orange (these colors were colorblind-friendly) on the screen of the Kinarm setup by applying an isometric force of 7 N on the left or right arm of the robot (bimanual task) following one of two possible rules. The direction of the force was irrelevant; only its magnitude was taken into account. For the shape rule, participants had to indicate whether the shape was a square or a circle. The association between shape and hand of the response was counterbalanced across participants. For the color rule, the participants had to indicate whether the symbol was orange or blue. The association between color and hand of response was counterbalanced across participants. On each trial, the relevant rule was indicated by a solid white line (20 cm wide) that was displayed 1.5 cm above the symbol for the shape rule and 1.5 cm below the symbol for the color rule. When a participant made a mistake, a red line (20 cm wide) appeared on top of the symbol. Intertrial time interval was 400 ms. Participants had a maximum of 5 s to provide their response.
Fig. 1.
Experimental protocols. A: cognitive task switching experiment. Participants had to indicate the shape of the symbol (shape task: square or circle) if a white line was displayed above the symbol on the screen (as shown) or its color (color task: orange or green) if the line was below it (as indicated by the dashed line). The response was indicated by exerting an isometric force on the left or right handle of the robot manipulandum. The situation depicted requires a left hand response. The hands were not visible to the participants. B: motor task switching experiment. In this task, participants were instructed to bring the cursor through the aperture of a circle. This aperture was always at the same location but could have 1 of 2 possible widths (0.8 or 8 cm). Each trial started with the appearance of the starting position (black square; return and cue phase). During this phase, a cue indicated the width of the aperture for the ensuing trial (gray symbol; return and cue phase). The target was then presented 15 cm away from the starting position in front of the participant (target phase). When the participants initiated their reach, the hand cursor was replaced by a circle whose radius increased with the distance traveled by the hand (movement phase). Veridical hand position was provided after movement end to provide feedback about movement accuracy (feedback phase). Additional information was provided via the imprint that the hand left on the circle when it crossed it. The imprint was blue if movement duration was too long, yellow if movement duration was too short, and green if movement duration was between 500 and 600 ms. One point could be earned for good movement speed, and another point could be earned for good accuracy. The total number of points collected so far during the block was displayed at the end of each trial (156 points in the illustration). For purposes of clarity, actual colors used in the experiments are not represented in B but are used in the text. C: an illustration of the repartition of trials with the narrow and wide target over the course of the 3 experiments (see text). The y-axis represents target width and the x-axis, trial number. In the CHOI schedule, participants selected the width of the target for each trial and were instructed to do so randomly. In the random (RND) schedule, target width was randomly varied from trial to trial. In the blocked (BLK) schedule, target width remained unchanged for either 45 trials (experiment 1) or 60 trials (experiments 2 and 3). The order of target width was counterbalanced across participants. In the MIX schedule, there was an imbalance in the percentage of trials for each target width (75% high-probability trials vs. 25% low-probability trials). Each participant received 60 trials with higher probability of one target width, followed by another series of 60 trials where the other target width was more frequent. The order was also counterbalanced across participants.
After one to three blocks of training to demonstrate the task (20 trials each), the participants performed three blocks of trials. In the random schedule (first and third blocks, 40 and 64 trials), the rule (indicate shape or indicate color) could change pseudorandomly from trial to trial. In this schedule, every mini-block of eight trials contained all eight possible combinations of shape, color, and rule. Within each mini-block, the order of the eight different trials was fully randomized. The first and last block contained, respectively, five and eight mini-blocks. In contrast, during the second block, the rule remained unchanged for the first half of the block (20 trials), after which it switched to the other rule until the end of the block (20 trials).
Motor task switching experiment.
Participants were instructed to shoot through a target whose width could vary from trial to trial. The target consisted of an aperture within a white circle (radius: 15 cm), which was always centered on the starting position (25-mm2 red square located 15 cm in front of the participant's chest). The width of the aperture could be either 0.8 or 8 cm (Fig. 1, target phase). The width of the target on the next trial was presented well in advance of target presentation (Fig. 1, cue phase). An orange 0.8-cm2 square or a 0.8 cm × 8 cm rectangle was provided as a cue to indicate the width of the aperture for the upcoming movement. This cue was presented 3 mm below the starting position and at the same time as the starting position (∼1 s before reach onset). This cue disappeared at reach onset. The circle that delimited the target appeared after a delay between 700 and 850 ms once the hand of the participant was stabilized inside the starting position and, in the CHOI schedule, after the participant had made his/her choice about the width of the target (see below). Once the target had appeared, participants had to reach through the target in a given time interval (500–600 ms; Fig. 1, movement phase) without any information about the direction of their movement (see below). As soon as the hand crossed the circle, it left an imprint to provide accurate feedback about the accuracy of the movement (Fig. 1, feedback phase). The color of this imprint provided feedback about the speed of the movement (blue = too slow, yellow = too fast, green = good speed). When movement duration was between 500 and 600 ms, participants earned one point. They also earned an additional point if the imprint was within the aperture.
Movement onset and offset were detected online with position and velocity criteria. Movement onset was flagged when the hand left the area of the starting position and hand velocity was higher than 0.02 m/s. Movement end was detected when the distance traveled was larger than 15 cm. A few hundreds of milliseconds after movement end, the hand was pushed back toward the starting position while the hand cursor was removed from the screen.
The experiment was divided into blocks of trials that were separated by a 1-min break. All participants started the experiment with a practice block. During this block, they performed tens of movements (40 trials in experiment 1, 60 trials in experiments 2 and 3) toward the 0.8-cm aperture with a small hand cursor (cursor was 9 mm2) that provided online visual feedback of hand position to the participants. This block aimed at training participants to reach to the target at the good speed. After the practice block and for the rest of the experiment, a modified hand cursor was used during the movements toward the target (Fig. 1, movement phase). This cursor was an expanding circle that indicated the distance traveled by the hand but not the direction taken by the participant. Outside the movement period, the hand cursor was a 9-mm2 square. When the hand exited the circle, the small cursor indicated again the actual position of the hand. An imprint that was located on the circle indicated exactly where the hand of the participants crossed the boundary of the circles to provide accurate endpoint performance.
For 20% of the trials of all experimental blocks (i.e., not during the practice blocks), the hand path was constrained by stiff virtual walls (perturbation trials). These walls (Scheidt et al. 2000) were created by applying a stiff unidimensional spring (spring stiffness: 2,500 N/m; viscosity: 25 N·s/m) and acted as a mechanical guide that directed the hand toward the center of the target or 2 cm to either the left or the right of it. The lateral deviation observed during these trials was less than 0.5 mm In these trials, the actual position of the hand was concealed to the participant by 1) using the circular feedback during the movements, 2) displaying the feedback imprint at the center of the target when the subjects crossed the target, and 3) showing the projection of the actual position of the hand on the midline rather than its actual position once the hand exited the circle. To make sure that our results were not influence by this fake feedback, we removed all the trials immediately following a perturbation trial from all analyses. These trials were randomly interspersed throughout the experimental blocks.
In experiment 1, we tested the ability of the participants to voluntarily modulate their behavior when they themselves select the target width. This experiment started with a block of at least 120 trials during which participants selected the width of the target on each trial (CHOI schedule). The actual number of trials in this first block varied from participant to participant, because the block was stopped only once the participant had chosen each of the two targets 60 times. Before the appearance of the starting position, participants indicated orally to the experimenter the width of the target that they would like to receive on the next trial. The operator then selected the correct target size for the upcoming movement. This first block was followed by 2 blocks of 90 trials each. In the first block (RND schedule), target width was varied pseudorandomly (the order of trials was prearranged before the experiment). In the second block (BLK schedule), half of the participants experienced the narrow target for 45 trials followed by another 45 trials with the wide target. The order of target width was counterbalanced for the other half of the participants. In summary, experiment 1 consisted of three different schedules: CHOI, RND, and BLK.
In experiment 2, we tested the ability of the participant to form a memory of the optimal behavior exhibited during the BLK schedule in a later random schedule. In this experiment, during the first and second experimental blocks (66 and 90 trials, random schedule, RND1), the target width was varied randomly (i.e., for block 2, the order of trials was prearranged before the experiment) from trial to trial. During the third block (blocked schedule, BLK), half of the participants experienced the narrow target for 60 trials followed by another 60 trials with the wide target. This order was counterbalanced for the other half of the participants. During the fourth block (RND2, 120 trials), the width of the target was again pseudorandomly presented from trial to trial. In summary, experiment 2 consisted of three schedules: RND1, BLK, and RND2.
In experiment 3, we tested the ability of the participants to update their motor program from one trial to the next. This experiment consisted of four different blocks. During the first and second experimental blocks (66 and 90 trials, random schedule, RND), the target width was varied randomly from trial to trial. During the third block (Fig. 1C, mixed schedule, MIX), half of the participants experienced 60 trials where the narrow target was presented in 75% of the trials (high-probability trials) and the other 25% (low-probability trials) were randomly interspersed trials with the wide target. These 60 trials were followed by another 60 trials with high-probability trials with the wide target (75% of the trials) and low-probability trials with the narrow target. The order of target width associated with high-probability trials was counterbalanced for the other half of the participants. In other words, all the participants experienced high- and low-probability trials with both the narrow and wide targets. Importantly, 8 of the 15 low-probability trials were perturbation trials. During the fourth experiment block (BLK), the width of the target was blocked. In summary, experiment 3 consisted of three schedules: RND, MIX, and BLK.
Data Analysis
In perturbation trials, we used the signed lateral force exerted by the participants against the wall of the channel as a proxy of their willingness to go toward the center of the target. Force measures were low-pass filtered (second-order Butterworth filter with cutoff at 75 Hz). The average lateral force from straight ahead perturbation trials was subtracted from the measures of lateral force during rightward or leftward perturbation trials separately for each schedule and target width. These corrected measures were sign reversed for the rightward perturbations such that a larger positive force represents a larger reaction to the perturbation. To quantify the reaction to the perturbation, we extracted the lateral force exerted by the participants when they were 2 cm away from the target. These 2 cm allowed us to avoid late correction of the movements or the period where the cursor was back to normal size. To assess the effect of time on this variable, the same measure was also taken at 7 cm from the starting position (∼240 ms into the movement).
In unperturbed trials, our proxy for movement performance was the lateral position of the hand when it was 2 cm away from the target. This measure was considered as movement endpoint. This measure of movement endpoint was highly correlated with the movement endpoint when the hand crossed the target (R > 0.8) but had the advantage of containing many more points. Indeed, on some trials, participants stopped their movement before reaching the target (percentage of such movement varies between participants and experiments but is smaller than 0.25% of the trials). In such cases, the value at 15 cm cannot be computed. Unperturbed trials were excluded from these analyses if they immediately followed a perturbed trial. To assess how the performance of one trial affected the performance on the next trial, we computed lag-1 autocorrelation of endpoint errors between pairs of trials. Note that pairs of consecutive trials that include a perturbation trial were also excluded from this analysis. The autocorrelation function was computed for each subject individually, and the values of the autocorrelation were then averaged across subjects.
In experiment 1, ANOVA was performed with schedule (CHOI, RND, BLK) and target width (narrow and wide) as within-subjects factors. In experiment 2, ANOVA was performed with schedule (RND1, BLK, RND2) and target width (narrow and wide) as within-subjects factors. In experiment 3, ANOVA was performed with schedule (RND, MIX-high probability, MIX-low probability, BLK) and target width (narrow and wide) as within-subjects factors. The factor MIX-high corresponded to the 75% of trials that had the same target width, whereas the factor MIX-low represented the remaining 25%.
In the cognitive task switching experiment, the reaction time was taken as the time between the appearance of the symbol on the screen and the force magnitude (in any direction) reached a threshold of 6 N (which required a strong response from the participant). We then computed the median reaction time of the correct trials for each epoch separately.
To assess how much the performance on one trial was taken into account in the next trial, we followed the method proposed by Van Beers and colleagues (van Beers 2009; van Beers et al. 2013a, 2013b). Van Beers showed that when errors are taken into account, the correlation between errors on consecutive trials is low because the trial-to-trial error correction mechanisms should reduce the relationship between the movement directions from one trial to the other. In contrast, when error correction mechanisms are absent, the relationship between the planned directions of two consecutive trials is only disrupted by noise because it looks like a random walk. Therefore, the correlation of endpoint error between pairs of consecutive trials is much higher when error correction mechanisms are absent than when they are present. Further justification for this analysis can be found in Van Beers et al. (2013a). To assess this correlation, we took into account all the pairs of consecutive trials that did not contain a perturbation trial. For each of these pairs, we considered the lateral position of the hand shortly before it reached the target (see above). We then computed the correlation coefficient between the lateral positions in these pairs of trials.
To take into account the different levels of force observed in the different experiments (e.g., due to reaching or shooting), the analysis of the switching cost across the experiments from this work and from the previous study (Orban de Xivry 2013) was carried on the normalized lateral force obtained by dividing the lateral force by the average force.
RESULTS
Self-Selection of Target Width Does Not Alleviate the Inability to Switch Between Two Different Control Policies
In the first experiment, participants were asked to randomly select the width of the target before each trial (CHOI schedule) to test whether self-selection of target width would allow the participants to optimize their reaching movements on a trial-to-trial basis during a random schedule. This block was then followed by two other blocks. In the first one, the target width was randomly changed from trial to trial (random schedule, RND), whereas in the second, the target width remained constant for 60 trials before it was switched to the other target width (block schedule, BLK). In each of these three schedules, we analyzed how participants responded to perturbations during their movements as a function of the target width (Fig. 2). By doing so, we made the assumption that participants would use the same control policy in perturbed and unperturbed trials because the perturbation trials were unpredictable and the response to the perturbation reflects the control policy that is used in unperturbed trials. Most of the participants did not actually realize that in some trials, their hand was perturbed away from its trajectory.
Fig. 2.
Self-selection of target width does not improve optimality of behavior compared with the random schedule. A: average force profiles across all participants recorded during the perturbation trials for each schedule (CHOI, self-selection of target width; RND, random schedule; BLK, blocked schedule) and each target width separately (red, narrow target; blue, wide target). In these plots, the forces are represented against the distance from the starting position to match the level of perturbation. Shaded area around each curve represents SE. B: average force (N) recorded at 13 cm in the force profiles as a function of target width and schedule. Error bars are SE. C: average distribution of the length of series of trials with the same target width (i.e., how many trials in a row had the same target width). Error bars are SE. C, inset: proportion of trials where the target width is different from the target width on the previous and next trials.
In both the CHOI and RND schedules, participants slightly modulated the force that they exerted against the perturbation with target width. However, the modulation of the force with target width was significantly smaller than what was observed in the BLK schedule (Fig. 2). This observation was confirmed by a repeated-measures ANOVA with schedule (CHOI, RND, and BLK) and target width (narrow or wide) as within-subject factors. The ANOVA yielded an interaction between target width and schedule [F(2,40) = 6.9, P = 0.0026]. Further inspection of this interaction revealed that the difference in force for the narrow and wide targets (ΔF) was larger in the BLK schedule than in the two other schedules [paired t-test: ΔFBLK vs. ΔFCHOI: t(20) = −3.12, P = 0.005; ΔFBLK vs. ΔFRND: t(20) = −3.04, P = 0.006]. There was no statistical difference in ΔF between the CHOI and RND schedules [t(20) = −0.19, P = 0.84]. The modulation of force with target width was nevertheless significant both in the CHOI and in the RND schedules [ΔF > 0: CHOI: t(20) = 5.3, P < 0.001; RND: t(20) = 4.2, P < 0.001]. Overall, these results suggest that despite participants being able to select the target width on each trial, there was only a limited modulation of their response to the perturbation with target width during the CHOI schedule compared with the BLK schedule.
In the CHOI schedule, participants were explicitly instructed to select the target width randomly before each trial. However, they were not able to do so. Most of the participants tended to select the narrow target for several trials in a row, whereas they more often selected the wide target for a single trial before switching back to the narrow target. Therefore, after 80 trials, most participants had performed more trials with the narrow target (on average ∼42 trials) than with the wide target (on average ∼38 trials). The proportion of single trials with the narrow and wide targets differed [narrow vs. wide: 18% vs. 25%, t(20) = −4.25, P = 0.0003]. There were less isolated trials (hence, more trials in a row) for the narrow target than for the wide target. If the participants wanted to minimize effort, they should have had a bias toward the wide target (i.e., they should have chosen the wide target more often than the narrow target).
Relationship Between Switching Cost for Motor and Cognitive Tasks
The subjects who participated in the first experiment also participated in a cognitive task switching experiment. In this task, participants had to indicate either the shape or the color of a symbol that appeared in the middle of the robot workspace (Fig. 1A). Subjects were able to learn the rules in a satisfactory way (RND: 85% accuracy; BLK: 90% accuracy). When the task instruction changed randomly from trial to trial, the reaction time of the participants was slightly above 1 s (1,085 ± 47 ms, mean ± SE). In contrast, the reaction time when the task instruction remained unchanged for several trials in a row was 400 ms shorter [640 ± 45 ms, t(20) = 13.66, P < 0.0001]. These 400 ms represent the switching cost between the RND and BLK schedules (Fig. 3A). Similarly, the switching cost in the motor task was around 0.3 N, which represents the difference in modulation of the force with target width between the RND and BLK schedules (Fig. 3C). We found that the cognitive and motor switching costs were correlated across participants [Fig. 3B; r = 0.57, t(20) = 3.06, P = 0.007]. That is, participants for whom the schedule had a small effect in the cognitive task (reaction time in RND not much worse than in BLK) were better able to modulate their force in the RND schedule compared with the BLK schedule. This relationship suggests that both motor and cognitive switching cost might stem from a general mechanism of the central nervous system.
Fig. 3.
Link between cognitive and motor switching cost. A: reaction time (ms) in the random (RND) and blocked (BLK) schedules during the cognitive task switching experiment. The difference between the 2 bars (RND minus BLK) represents the average cognitive switching cost (ms). B: correlation between the cognitive and motor switching costs. C: amount of modulation in the force exerted against the perturbation with target width (force for narrow target minus force for wide target) for the RND and BLK schedules. The difference between the 2 bars (BLK minus RND) represents the motor switching cost (N).
Knowledge of a Better Motor Program is Not Used During a Subsequent Random Schedule
The results from these experiments and previous studies on optimality of motor behaviors (de Rugy et al. 2012; Kistemaker et al. 2014) are actually confounded by the possibility that participants might not know what the optimal behavior is because they have never used it. Therefore, in the second experiment (Fig. 4), we tested whether the worst motor behavior observed during the random schedule was due to the inability to use a better motor behavior or to the ignorance of a better one. Indeed, so far in all the experiments, participants had always experienced the RND schedule before the BLK schedule. Again, we found in the first random schedule that participants applied more force against a perturbation when the target was narrow than when it was wide [paired t-test, t(19) = 2.37, P = 0.028; Fig. 4]. However, this difference between the force applied against the perturbation for the narrow and wide targets was much larger during the blocked schedule [interaction between schedule (RND1 and BLK) and target width: F(1,19) = 7.08, P = 0.015]. We then tested whether the participants were able to use the learned and better behavior in a second random schedule (RND2). We found that when participants experienced the random schedule again, the difference in force applied against the narrow and wide targets was again smaller than the modulation of force with target width observed during the blocked schedule [interaction between schedule (BLK and RND2) and target width: F(1,19) = 7.09, P = 0.015], and they did not differ significantly from the force in the first random schedule [interaction between schedule (RND1 and RND2) and target width: F(1,19) = 0.25, P = 0.62]. These data suggest that even though participants have used a better motor behavior in the preceding blocked schedule, they were unable to use it during the following random schedule. In other words, learning and using a better control policy in the blocked schedule does not lead to a better performance in the subsequent random schedule.
Fig. 4.
Knowing the optimal motor program does not improve behavior in a subsequent random schedule A: average force profiles across all participants recorded during the perturbation trials for each schedule (RND1 and RND2, random schedules; BLK, blocked schedule) and each target width separately (red, narrow target; blue, wide target). In these plots, the forces are represented against the distance from the starting position to match the level of perturbation. Shaded area around each curve represents SE. B: average force (N) recorded at 13 cm in the force profiles as a function of target width and schedule. Error bars are SE.
Inability to Instantaneously Switch Between Motor Programs
Whereas the previous experiment aimed at emphasizing the inability to use a better motor program in the random schedule while knowing one, we wanted next to identify the cost of switching when target width changed unexpectedly during a blocked schedule. To do so, 20 other participants experienced a new schedule (MIX schedule) where 75% of the trials had a given target width (MIX-high: high-probability trials, e.g., narrow target) and 25% (MIX-low: low-probability trials) had the other target width (e.g., wide target). Half of these low-probability trials were perturbation trials.
In this group, differences in force profiles were again more marked during the blocked schedule than during the random schedule (Fig. 5). During the MIX schedule, the force that the participants exerted against the perturbation was clearly modulated by target width for the most frequent target width, but not for the target width presented in the low-probability trials. These observations were confirmed by an interaction between target width and trial type [random, MIX-high, MIX-low, and BLK: F(3,57) = 2.9267, P = 0.041]. Post hoc Tukey's t-tests revealed that target width modulated force only for the high-probability trials in the MIX schedule and for the blocked schedule (P = 0.001 and P = 0.013), but not in the random schedule or in the low-probability trials (P = 0.31 and P = 0.99). A sizeable switch of control policy induced by a change in target width during the MIX blocks should result in a significant difference in force between low- and high-probability trials. Although there was a trend toward significance for comparison of trials with different target widths in the mix block with high probability of wide target (Tukey's post hoc test: wide vs. narrow, P = 0.051), there was no significant modulation of force with target width in the mix block with high probability of narrow targets (Tukey's post hoc test: narrow vs. wide, P = 0.66). In other words, the behavior in the low-probability trials was close to what was observed in the random schedule.
Fig. 5.
Change in motor program over one trial. A: average force profiles across all participants recorded during the perturbation trials for each schedule (RND, random schedule; MIX-high, majority of trials in the mixed schedule; MIX-low, low-probability trials in the mixed schedule; BLK, blocked schedule) and each target width separately (red, narrow target; blue, wide target). In these plots, the forces are represented against the distance from the starting position to match the level of perturbation. Shaded area around each curve represents SE. B; average force (N) recorded at 13 cm in the force profiles as a function of target width and schedule. Error bars are SE.
Additional Knowledge from Our Large Pool of Subjects
In the three experiments presented in this article and the four experiments presented in a previous report (n = 111 subjects), we found a very strong effect of schedules on the ability of participants to modulate the response to a perturbation [Fig. 6A; interaction between target size and schedule (RND and BLK): F(1,110) = 35.18, P < 0.00001, partial η2 = 0.24]. To take into account the different levels of force observed in the different experiments (e.g., due to reaching or shooting), this analysis was carried out on the normalized lateral force (obtained by dividing the lateral force by the average force).
Fig. 6.
Analysis of the switching cost for the experiments reported in this article (5: MEM; 6: Mix; 7: CHOI) and in our previous publication (1: Main; 2: Long; 3: Shoot; 4: Cue experiments). A: normalized lateral force for the RND and BLK schedules and target widths for the 111 subjects (a.u., arbitrary units). B: representation of the variability of the observed effect across the 7 experiments. Error bars represent SE.
This database allows us to have a look on the actual difference between RND and BLK. Indeed, there appears to be quite some variability across experiments (Fig. 6B). Whereas in some experiments (Fig. 6B, Expe 1), the increase in force for the narrow target from RND to BLK was pronounced, it was absent or small in other experiments (Fig. 6B, Expe 6 and 7). Similarly, for the wide targets, there was an important decrease in force from the RND to the BLK schedule is some experiments (Fig. 6B, Expe 7), but not in others (Fig. 6B, Expe 1). On average, on the basis of data from the 111 subjects, we can conclude that the schedule modulated the force both for the narrow target (RND vs. BLK: Tukey's post hoc test, P = 0.0002) and for the wide target (Tukey's post hoc test, P = 0.0001). This observation allows us to reject the possibility that subjects were lazy and pushing too hard for the wide target as a safety measure. In addition, we observe that the same control policy was not used for both narrow and wide target in the RND condition given that the response to the perturbation was modulated by target width (Tukey's post hoc test: narrow vs. wide target in RND condition, P = 0.0001). However, this modulation was smaller in the RND condition than in the BLK condition as indicated by the significant interaction.
Given the nature of the perturbation (position-dependent perturbation), it is difficult to judge how early the motor system responded differently to a narrow or wide target. Nonetheless, an ANOVA performed on force measures at a distance of 7cm (∼240 ms after movement onset) from the starting point revealed the same target width × schedule interaction [F(1,110) = 37.8, P < 00001].
Absence of History Effect in Response to the Perturbation
Costs in cognitive task switching experiments are highlighted not only by the difference between random and blocked schedules but also by a difference in performance between switch trials (previous trial had a different goal from the current one) and non-switch trials (previous trial was similar to the previous one).
In contrast to cognitive switching cost, we did not find differences in performance between switch (change in target width between the previous trial and the current one) and non-switch trials (same target width on the previous trial as on the current one). That is, we expected that the force to a perturbation would be larger on one trial when it was preceded by a trial with the narrow target and smaller when it was preceded by a trial with the wide target. In other words, we expected an interaction between target width and switching state (switch vs. non-switch trials). In our large pool of participants (n = 111; Fig. 7), we found that the response to the perturbation was very weakly influenced by the switching [F(1,109) = 2.06, P = 0.15, partial η2 = 0.02]. As always, the absence of significance could be due to an absence of power. For this reason, we reported the effect size of the effect: 0.02, which suggests that even if the effect exists, it should be considered as a small effect (Cohen 1988). In addition, such history-dependent effects, in other contexts, are typically observable with much a smaller sample size (10 in Franklin et al. 2008; 2 in Kowler et al. 1984; 7 in Tabata et al. 2008; 8 in Witney et al. 2001). This suggests that the smaller modulation of the response to perturbation with target width during the random schedule is not due to the inability of the participants to converge to a better solution and that it is not a gradual process where participants adjusted their movement planning trial after trial.
Fig. 7.
History effect: influence of the target size from the previous trial on the force (N) exerted on the current trial. Graph summarizes the data from 111 participants (61 from this study and 50 from Orban de Xivry 2013). Data are from the RND schedules only.
Trial-to-Trial Changes in the Motor Program
Updating the planned direction of the next movement as a function of the error experienced in the previous trial leads to low lag-1 autocorrelation between consecutive movement endpoint errors (van Beers et al. 2013a). In contrast, a high lag-1 autocorrelation is associated with little or no trial-to-trial changes in motor performance (similar to a random walk). In the three experiments presented in this article and the four presented in a previous publication (n = 111 participants), we found that during the BLK schedule, the lag-1 autocorrelation was almost zero for the narrow target but was much larger for the wide target [Fig. 8; n = 111, −0,034 ± 0.027 vs. 0.24 ± 0.029, t(110) = −8.12, P < 0.0001, partial η2 = 0.37]. We also looked at the lag-1 autocorrelation of endpoint error between pairs of trials with or without a change in target width. During the random schedule, pairs of consecutive trials with narrow targets or with wide targets exhibited the same behavior as in the BLK schedule [smaller for consecutive trials with the narrow target than for consecutive trials with the wide target: 0.11 ± 0.04 vs. 0.36 ± 0.033, t(110) = −5.8, P < 0.0001, partial η2 = 0.24]. This suggests that the small lag-1 autocorrelation was not due to a completely random behavior (pure noise would also result in a zero auto-correlation) but to error correction. In contrast, a change in target width appeared to reduce the ability to take the performance from the previous trial into account given that lag-1 autocorrelation between pairs of trials where the target width changed was high in both cases (narrow to wide: 0.2 ± 0.025; wide to narrow: 0.29 ± 0.025). Overall, we found an interaction between switch and target width variables [Fig. 8; F(1,110) = 47.6, P < 0.0001]. Post hoc tests suggest that the lag-1 autocorrelation was lower for the pair of trials with narrow targets than for the three other pairs (Dunnett's t-test: SS vs. SL, P < 0.001; SS vs. LS, P = 0.023; SS vs. LL, P < 0.001). This pattern appears similar for the two data sets [Orban de Xivry 2013 vs. present study; interaction between switch, target width, and data set: F(1,109) = 0.04468, P = 0.83].
Fig. 8.
Trial-to-trial changes in movement endpoint: lag-1 autocorrelation of movement endpoint error for different pairs of trials from the RND and BLK schedules. Error bars are SE. Graph summarizes the data from 111 participants (61 from this study and 50 from Orban de Xivry 2013).
In the BLK schedule, repetitiveness did not disengage the participants because they were more accurate and more often inside the target in the blocked condition (46% of the time) than in the random condition for the small target (40% of the time). Therefore, the success rate increased by 15% for the small target between the RND and BLK conditions [t(110) = 4.51, P = 0.00002]. Therefore, even though the RND schedule was potentially more engaging than the BLK schedule, participants were less successful during that schedule.
Finally, we did not notice any cost of switching in the median reaction time as confirmed by an ANOVA performed on the median reaction time with schedule and target width as within-subject factors [main effect of target width: F(1,110) = 1.44, P = 0.23, partial η2 = 0.012; interaction: F(1,110) = 1.79, P = 0.18].
DISCUSSION
This study makes two important points about movement planning. First, the three experiments confirmed that the inability of our participants to behave optimally in a changing environment was consistent with the presence of a switching cost. More specifically, in our experiments, participants exhibited different motor behaviors in the random and blocked schedules, even when the participants themselves selected the width of the target that they will reach toward or when the participants had previously demonstrated the knowledge of a better motor program for the task at hand. They were also unable to switch from one control policy to another over one trial. Each of these experiments represents a critical test of the switching cost hypothesis for motor planning. Note that the switching cost should not be considered as a part of a general cost function to optimize the control policy. Rather, it should be linked to the limited flexibility in the random schedule compared with the blocked schedule. Second, participants were unable to use the endpoint error on one trial to adjust their planning of the next movement if the target width changed. These results directly speak to recent theories of motor planning (Wong et al. 2015), which suggest that movement initiation results from a series of processes that can be separated into two categories. Processes from the first category determine the motor goal (the “what” pathway: target extraction and selection, attention, etc.) and are largely responsible for variation in reaction time, whereas the processes of the second category are responsible for action selection and for the specification of the movement kinematics (the “how” pathway). Given the absence of reaction time modulation in our task, we believe that our results speak directly to the limitations of the “how” pathway, which encompasses both action selection and movement specification.
For the rest of the discussion, we refer to the motor behaviors observed in the blocked scheduled as “optimal.” This claim of optimality can be viewed from two different perspectives. 1) Following optimal feedback control theory, the lateral force should be modulated with target width. Nobody can predict how much the lateral resistance should be modulated with target width to be considered optimal because the actual cost function is unknown, but the blocked schedule is the best possible context in which to observe the optimal behavior. 2) Among all the behaviors observed in our study, behavior in the blocked schedule was the best one because it minimized error and energy. Such an assertion is similarly used in several other studies (Liu and Todorov 2007; Nashed et al. 2012; Shadmehr et al., 2010).
Reduced Ability to Learn from Error in the Previous Trial to Adjust Endpoint Position
In simple reaching movements like those in this study, action selection refers to the selection of the next aiming direction, which varies from trial to trial. Such a trial-to-trial change in aiming direction has been considered as a learning from error process, where the aiming direction on a trial is equal to the aiming direction on the previous trial plus a fraction of the error (van Beers 2009; van Beers et al. 2013b). Such a process causes the correlation of endpoint errors between two conse cutive trials to be small if the errors are taken into account (e.g., for the narrow target) and a positive autocorrelation when it is not (e.g., for a wide target, where errors do not matter) (van Beers et al. 2013a). In the latter case, the planned movement endpoint follows a random walk (van Beers et al. 2013a) where, in the absence of error correction, movement endpoint on a given trial is more similar to movement endpoint on the previous trial than chance. However, this theory is inconsistent with the autocorrelations observed during the random schedule. When the target presented on one trial was narrow, participants updated the aiming direction for their next trial only if the target on the next trial was also narrow, but not if it was wide (Fig. 8). This suggests that the aiming direction cannot be updated before the target width of the next trial is known and that participants held the error in memory until they saw the target width of the next trial. This scheme appears very unlikely. If this scenario was possible, then the error observed in trials with wide target should be used if the next target was narrow. To the contrary, the success rate in these trials with narrow target was low compared with success rate in the BLK schedule. Alternatively, it is possible that the subjects were unable to estimate the size of the error with respect to the center of the target because its ends were too far away. We believe that this explanation does not hold because the wide target is 8 cm wide (∼9° of visual angle) and people can accurately estimate the middle of a 20-cm line in a line bisection task (error of 1% reported by Milner et al. 1992) or can accurately maintain their eye at the center of large targets (30° of visual angle) defined only by illusory contours (Wyatt et al. 1994).
The inability of our participants to adjust movement endpoint when target width changed suggests that an alternative framework for the update of the aiming direction is necessary. This framework could have the same characteristics as the Monte-Carlo Markov chain framework (MCMC) proposed by Haith and Krakauer (2014). In this framework, the update of the aiming direction does not depend on the error but on the rewarding consequences of a trial (Skinner 1981), which depends on target width, as is the case for optimal control (Nashed et al. 2012). This selection by consequences suggests that the reward received at the end of the movement (i.e., the consequences) affects the probability of selecting this motor program in the future. The previous aiming direction is therefore replaced by the current one with some probability that depends on the value of the previous and current aiming direction (Haith and Krakauer 2014). However, the update of the aiming direction can only take place when the cost function is unchanged between two consecutive trials, because one cannot compare values from different scales (like comparing apples and oranges). This would also explain why participants tended to repeat consecutive trials with a narrow target (Fig. 2C) when asked to select the target width on each trial. Choosing the narrow target repeatedly allowed them to update their aiming direction as a function of their past performance. To summarize, the MCMC framework provides a convincing explanation for the absence of updating of movement plans when target width changes and for the repetitive choice of the same target width when the target is narrow.
Inability to be Optimal
After action selection, the kinematics of the movement are specified through an independent process (Wong et al. 2015). Our data suggest that the optimality of the control policy was disrupted by the frequent changes in target width. Optimal control predicts that every movement should be optimal (Nashed et al. 2012; Todorov and Jordan 2002), independently of the schedule. However, we found that the modulation of the response to perturbations with target width was much reduced but well present in the random schedule compared with the blocked schedule. This contrasts with the absence of difference between the two schedules observed by Nashed et al. (2012). However, their inability to detect the difference between the two schedules might be due to the restricted number of participants per group (8 for RND and 8 for BLK) and to their between-subject design. Accordingly, based on their quantitative data for the standard deviation of reach endpoint, the effect seems more pronounced in the BLK (SD wide target: 2.5 cm; SD narrow target: 0.4 cm) than in the RND schedule (SD wide target: 1.6 cm; SD narrow target: 0.4 cm).
This inability to modulate the feedback control policy adequately in the random schedule led to a cost in accuracy and a cost in effort. Participants were unable to reach the same accuracy level for the narrow target in the random schedule as in the blocked schedule, and they spent too much energy controlling the hand in the random schedule when the target was wide compared with in the blocked schedule. This decrease in performance due to the randomization of condition is a general finding that extends far beyond our simple task (Afsanepurak et al. 2012; Edin et al. 1992; Elliott and Allard 1985; Horak and Diener 1994; Khan et al. 2002; Pruszynski et al. 2008;). In this article, we provide, for the first time, an account for this phenomenon.
First, this absence of optimality in the random schedule is not due to the long time required to reach the optimality. Indeed, we observed an abrupt change in force response after the (unique) change in target width in the blocked schedule (Orban de Xivry 2013). Second, optimization of the control policy did not lead to trial-to-trial changes in force response against the perturbation. Indeed, there was no effect of the target width experienced on the previous trial on the force applied on the next trial (Fig. 7). Therefore, this suggests that our effect is not due to the fact that it takes several trials to reach the optimal behavior. In contrast, in the blocked schedule, optimality was achieved in the first few trials (Orban de Xivry 2013). This also rejects the idea that performance in the random schedule is deteriorated due to a history-dependent effect. In contrast, such history-dependent effects are observed in predictive eye movements (Kowler et al. 1984; Tabata et al. 2008) for which the memory of target trajectory is updated after each trial (Orban de Xivry et al. 2013) and for grasping movements where an internal model of the object weight is built trial after trial (Johansson and Westling 1988; Loh et al. 2010). Finally, our data argue against the possibility that a single habitual control policy was used in the random schedule, because we observed a significant modulation of the behavior with target width in the random schedule, as well. The origin of this modulation in the random schedule remains unresolved. We can only speculate about how the brain achieves this feat. It might be that an intermediate control policy is used given the uncertainty about target width (Gallivan et al. 2016; Haith et al. 2015). Alternatively, it might be that the observed response is the sum of two independent processes (Kurtzer et al. 2014), only one of which is limited by the switching cost. Finally, it might be that two different control modes are used in these contexts, one optimal and one robust (Crevecoeur et al. 2014). In this context, the absence of difference between switch and non-switch trials (Fig. 7) can be explained by the fact that there is full flexibility in the RND schedule but within tighter limits allowed by the control policy.
This is obviously not the first study to demonstrate a lack of optimality in motor behaviors. However, the previous studies required the learning of a new control policy in a new and unknown environment or with a new tool (de Rugy et al. 2012; Hudson et al. 2010; Kistemaker et al. 2010, 2014; Zhang et al. 2012). That is, the participants from these studies never exhibited the optimal motor behavior, and it is unknown whether participants would have been able to execute the optimal motor behavior. In contrast, in the present study, we show that the participants knew the optimal control policy because they exhibited it in the blocked schedule. However, a switching cost prevented them from using the same policy in the random schedule. In other words, whereas previous studies demonstrated that we cannot always find the optimal motor behavior, the present study demonstrates that even if you have found it (e.g., in the blocked schedule), you might not be able to use it (e.g., in the random schedule). Indeed, in experiment 2, despite having used a better (more accuracy for the narrow target and less energy wasted for the wide target) feedback control policy in the blocked schedule, participants were unable to use these control policies in the subsequent random schedule. This observation supports our claim that movements in the random schedule were not optimal compared with those in the blocked schedule and confirms the existence of a cost associated with changing the feedback control policy, i.e., a switching cost. If no such cost existed, then participants could use their best (less costly) strategy to perform the task (the one used in the blocked schedule).
In the case of multiple targets, several motor programs are prepared together during the planning stage (Cisek and Kalaska 2005, 2010), and the nonselected one is later inhibited. The current study leads to the question, are there multiple programs prepared when a unique target location must be associated with two different control policies for the narrow and wide targets (Gallivan et al. 2016)? In this case, inhibition of the nonselected motor program could lead to subsequent switching cost as it became harder to retrieve the inhibited motor program (Duque et al. 2005; Duque and Ivry 2009; Mars et al. 2009) or the inhibited cognitive task (Mayr et al. 2000, 2006). Alternatively, the switching cost might arise from the fact that a new control policy must be selected each time the target width changes.
We do not question the fact that, following optimal feedback control, feedback gains are determined before movement to bring the hand on the target or to respond to potential perturbations during the movement. However, we want to highlight that this optimization of the feedback control law is not as flexible as suggested by many authors and that switching between control policies carries some costs.
Finally, the inability of our participants to learn from the errors observed on the previous trials when target width changed and to modulate the feedback gains appropriately in the random schedule reflect two independent limitations of the planning of movements. Indeed, the aiming direction is updated by the observed endpoint errors, but there is no particular reason to update the control policy in function of this error. In addition, a switch in target width had a major effect on the update of the aiming direction (Fig. 8) but had no effect on force modulation (Fig. 7). Accordingly, an update of the control policy following the MCMC framework would lead to a history effect for the force modulation, which was not observed (Fig. 7). As outlined by Wong et al. (2015), action selection (choosing the aiming direction) and movement specification (choosing the control policy) are two independent processes.
How Similar is this Phenomenon to Cognitive Task Switching?
The motor switching cost identified on the basis of force modulation with schedules bears many similarities to the switching cost for cognitive tasks. However, it also differs from it in a number of ways. First, whereas in this study we identified a switching cost for the “how” pathway (Wong et al. 2015), cognitive studies typically focus on a cost for the “what” pathway, which results in an effect on the reaction time (Rogers and Monsell 1995). It is unknown whether a switching cost for the “how” pathway exists in cognitive tasks, whereas we demonstrated it for motor tasks. The observation that the switching cost in our task did not affect the reaction time is consistent with the fact that movement preparation and initiation are two independent processes (Haith et al. 2016) and that several factors directly influence the feedback control policy but do not modulate the reaction time. For instance, the influence of target size on reaction time is very limited (Quinn et al. 1980). It is believed that the feedback control policy is only selected among several possibilities. Such process does not largely influence the reaction time.
Despite this difference, self-selection of target (experiment 1) or selection of cognitive task rule (Arrington and Logan 2004) does not abolish the switching cost. Second, although a switching cost can be identified on a single-trial basis in cognitive tasks (reaction time is different in switch and non-switch trials), we failed to identify such effect in our data (Fig. 7). However, we found that the switch prevented participants from taking the performance of the previous trial into account to update their movement plan for the next trial (Fig. 8).
Finally, we found that motor and cognitive switching costs were correlated across participants. This correlation might stem from the fact that updating the association between a stimulus and a motor program might be controlled by a mechanism that is independent of the rationale behind the update (i.e., change in task rule or change in target width) such as the one taking place in the premotor and motor areas for cognitive and motor tasks. Cells from the cingulate cortex are differentially modulated when the movement plan needs to be updated or when it needs to be repeated (Procyk et al. 2000; Shima and Tanji 1998). Lesions of the same area impair performance at a cognitive task switching experiment in monkeys (Rushworth et al. 2003). The presupplementary and supplementary motor areas (SMA) (Nachev et al. 2008) also appear to play an important role in the cognitive and motor switching processes. Cells of the presupplementary and supplementary motor areas only responded before switch trials, and not before non-switch trials, during a reaching task to two targets (Matsuzaka and Tanji 1996). Similarly, pre-SMA first inhibits the previous motor program and then boosts the desired saccade when participants are instructed to switch (Hikosaka and Isoda 2010; Isoda and Hikosaka 2007).
Conclusion
A simple manipulation of target width during a reach experiment reveals the limitations of the mechanisms of motor planning. These data reveal that action selection might proceed via a selection by consequence mechanism and that movement specification suffers from a switching cost for motor planning. This switching cost for motor tasks shares many similarities with the switching cost identified in cognitive tasks.
GRANTS
This work was supported by the Interuniversity Attraction Poles (IAP) program and Program for the Development of Scientific experiments (PRODEX) initiated by the Belgian Federal Science Policy Office, Actions de Recherche Concertée (French community, Belgium), and the European Space Agency (ESA) of the European Union. J.-J. Orban de Xivry was supported by the “Brains Back to Brussels” of the Brussels Region (Belgium).
DISCLOSURES
No conflicts of interest, financial or otherwise, are declared by the authors.
AUTHOR CONTRIBUTIONS
J.-J.O.d.X. conceived and designed research; J.-J.O.d.X. performed experiments; J.-J.O.d.X. analyzed data; J.-J.O.d.X. interpreted results of experiments; J.-J.O.d.X. prepared figures; J.-J.O.d.X. drafted manuscript; J.-J.O.d.X. and P.L. edited and revised manuscript; J.-J.O.d.X. and P.L. approved final version of manuscript.
Footnotes
Preprint available at http://biorxiv.org/content/early/2016/04/12/047621.
REFERENCES
- Afsanepurak SA, Karimiyani N, Moradi J, Safaei M. The effect of blocked, random, and systematically increasing practice on learning of different types of basketball passes. Eur J Exp Biol 2: 2397–2402, 2012. [Google Scholar]
- Ahmadi-Pajouh MA, Towhidkhah F, Shadmehr R. Preparing to reach: selecting an adaptive long-latency feedback controller. J Neurosci 32: 9537–9545, 2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Allport DA, Styles EA, Hsieh S. Shifting intentional set: exploring the dynamic control of tasks. In: Attention and Performance XV, edited by Umiltà C and Moscovitch M. Cambridge, MA: MIT Press, 1994, p. 421–452. [Google Scholar]
- Arrington CM, Logan GD. The cost of a voluntary task switch. Psychol Sci 15: 610–615, 2004. [DOI] [PubMed] [Google Scholar]
- Arrington CM, Logan GD. Voluntary task switching: chasing the elusive homunculus. J Exp Psychol Learn Mem Cogn 31: 683–702, 2005. [DOI] [PubMed] [Google Scholar]
- Bennett SJ, Orban de Xivry JJ, Lefèvre P, Barnes GR. Oculomotor prediction of accelerative target motion during occlusion: long-term and short-term effects. Exp Brain Res 204: 493–504, 2010. [DOI] [PubMed] [Google Scholar]
- Cisek P, Kalaska JF. Neural correlates of reaching decisions in dorsal premotor cortex: Specification of multiple direction choices and final selection of action. Neuron 45: 801–814, 2005. [DOI] [PubMed] [Google Scholar]
- Cisek P, Kalaska JF. Neural mechanisms for interacting with a world full of action choices. Annu Rev Neurosci 33: 269–298, 2010. [DOI] [PubMed] [Google Scholar]
- Cohen J. Statistical Power Analysis for the Behavioural Sciences (2nd ed) Hillsdale, NJ: Lawrence Erlbaum, 1988. [Google Scholar]
- Crevecoeur F, Cluff T, Scott SH. Computational approaches for goal-directed movement planning and execution. In: The Cognitive Neurosciences (5th ed.). Cambridge, MA: MIT Press, 2014, p. 461–475. [Google Scholar]
- de Rugy A, Loeb GE, Carroll TJ. Muscle coordination is habitual rather than optimal. J Neurosci 32: 7384–7391, 2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Diedrichsen J, Shadmehr R, Ivry RB. The coordination of movement: optimal feedback control and beyond. Trends Cogn Sci 14: 31–39, 2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dimitriou M, Wolpert DM, Franklin DW. The temporal evolution of feedback gains rapidly update to task demands. J Neurosci 33: 10898–10909, 2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dingwell JB, Smallwood RF, Cusumano JP. Trial-to-trial dynamics and learning in a generalized, redundant reaching task. J Neurophysiol 109: 225–237, 2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Duque J, Ivry RB. Role of corticospinal suppression during motor preparation. Cereb Cortex 19: 2013–2024, 2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Duque J, Mazzocchio R, Dambrosia J, Murase N, Olivier E, Cohen LG. Kinematically specific interhemispheric inhibition operating in the process of generation of a voluntary movement. Cereb Cortex 15: 588–593, 2005. [DOI] [PubMed] [Google Scholar]
- Edin BB, Westling G, Johansson RS. Independent control of human finger-tip forces at individual digits during precision lifting. J Physiol 450: 547–564, 1992. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Elliott D, Allard F. The utilization of visual feedback information during rapid pointing movements. Q J Exp Psychol A 37: 407–425, 1985. [DOI] [PubMed] [Google Scholar]
- Franklin DW, Burdet E, Tee KP, Osu R, Chew CM, Milner TE, Kawato M. CNS learns stable, accurate, and efficient movements using a simple algorithm. J Neurosci 28: 11165–11173, 2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gallivan JP, Logan L, Wolpert DM, Flanagan JR. Parallel specification of competing sensorimotor control policies for alternative action options. Nat Neurosci 19: 1–9, 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Haith AM, Huberdeau DM, Krakauer JW. Hedging your bets: intermediate movements as optimal behavior in the context of an incomplete decision. PLoS Comput Biol 11: e1004171, 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Haith AM, Krakauer JW. Motor learning by sequential sampling of actions. In: Translational and Computational Motor Control, edited by Smith MA and Krakauer JW. Washington, DC: American Society of Neurorehabilitation, 2014. [Google Scholar]
- Haith AM, Pakpoor J, Krakauer JW. Independence of movement preparation and movement initiation. J Neurosci 36: 3007–3015, 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hikosaka O, Isoda M. Switching from automatic to controlled behavior: cortico-basal ganglia mechanisms. Trends Cogn Sci 14: 154–161, 2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Horak FB, Diener HC. Cerebellar control of postural scaling and central set in stance. J Neurophysiol 72: 479–493, 1994. [DOI] [PubMed] [Google Scholar]
- Hudson TE, Tassinari H, Landy MS. Compensation for changing motor uncertainty. PLoS Comput Biol 6: e1000982, 2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Isoda M, Hikosaka O. Switching from automatic to controlled action by monkey medial frontal cortex. Nat Neurosci 10: 240–248, 2007. [DOI] [PubMed] [Google Scholar]
- Johansson RS, Westling G. Coordinated isometric muscle commands adequately and erroneously programmed for the weight during lifting task with precision grip. Exp Brain Res 71: 59–71, 1988. [DOI] [PubMed] [Google Scholar]
- Khan MA, Elliot D, Coull J, Chua R, Lyons J. Optimal control strategies under different feedback schedules: kinematic evidence. J Mot Behav 34: 45–57, 2002. [DOI] [PubMed] [Google Scholar]
- Kiesel A, Steinhauser M, Wendt M, Falkenstein M, Jost K, Philipp AM, Koch I. Control and interference in task switching–a review. Psychol Bull 136: 849–874, 2010. [DOI] [PubMed] [Google Scholar]
- Kistemaker DA, Wong JD, Gribble PL. The central nervous system does not minimize energy cost in arm movements. J Neurophysiol 104: 2985–2994, 2010. [DOI] [PubMed] [Google Scholar]
- Kistemaker DA, Wong JD, Gribble PL. The cost of moving optimally: kinematic path selection. J Neurophysiol 112: 1815–1824, 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kowler E, Martins A, Pavel M. The effect of expectations on slow oculomotor control-IV. Anticipatory smooth eye movements depend on prior target motions. Vision Res 24: 197–210, 1984. [DOI] [PubMed] [Google Scholar]
- Kurtzer I, Crevecoeur F, Scott SH. Fast feedback control involves two independent processes utilizing knowledge of limb dynamics. J Neurophysiol 111: 1631–1645, 2014. [DOI] [PubMed] [Google Scholar]
- Liu D, Todorov E. Evidence for the flexible sensorimotor strategies predicted by optimal feedback control. J Neurosci 27: 9354–9368, 2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Loh MN, Kirsch L, Rothwell JC, Lemon RN, Davare M. Information about the weight of grasped objects from vision and internal models interacts within the primary motor cortex. J Neurosci 30: 6984–6990, 2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mars RB, Klein MC, Neubert F, Olivier E, Buch ER, Boorman ED, Rushworth MFS. Short-latency influence of medial frontal cortex on primary motor cortex during action selection under conflict. J Neurosci 29: 6926–6931, 2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Matsuzaka Y, Tanji J. Changing directions of forthcoming arm movements: neuronal activity in the presupplementary and supplementary motor area of monkey cerebral cortex. J Neurophysiol 76: 2327–2342, 1996. [DOI] [PubMed] [Google Scholar]
- Mayr U, Diedrichsen J, Ivry RB, Keele SW. Dissociating task-set selection from task-set inhibition in the prefrontal cortex. J Cogn Neurosci 18: 14–21, 2006. [DOI] [PubMed] [Google Scholar]
- Mayr U, Keele SW. Changing internal constraints on action: the role of backward inhibition. J Exp Psychol Gen 129: 4–26, 2000. [DOI] [PubMed] [Google Scholar]
- Milner AD, Brechmann M, Pagliarini L. To halve and to halve not: an analysis of line bisection judgements in normal subjects. Neuropsychologia 30: 515–526, 1992. [DOI] [PubMed] [Google Scholar]
- Monsell S. Task switching. Trends Cogn Sci 7: 134–140, 2003. [DOI] [PubMed] [Google Scholar]
- Nachev P, Kennard C, Husain M. Functional role of the supplementary and pre-supplementary motor areas. Nat Rev Neurosci 9: 856–869, 2008. [DOI] [PubMed] [Google Scholar]
- Nashed JY, Crevecoeur F, Scott SH. Influence of the behavioral goal and environmental obstacles on rapid feedback responses. J Neurophysiol 108: 999–1009, 2012. [DOI] [PubMed] [Google Scholar]
- Orban de Xivry JJ. Trial-to-trial reoptimization of motor behavior due to changes in task demands is limited. PLoS One 8: e66013, 2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Orban de Xivry JJ, Coppe S, Blohm G, Lefèvre P. Kalman filtering naturally accounts for visually guided and predictive smooth pursuit dynamics. J Neurosci 33: 17301–17313, 2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Procyk E, Tanaka YL, Joseph JP. Anterior cingulate activity during routine and non-routine sequential behaviors in macaques. Nat Neurosci 3: 502–508, 2000. [DOI] [PubMed] [Google Scholar]
- Pruszynski JA, Kurtzer I, Scott SH. Rapid motor responses are appropriately tuned to the metrics of a visuospatial task. J Neurophysiol 100: 224–238, 2008. [DOI] [PubMed] [Google Scholar]
- Pruszynski JA, Scott SH. Optimal feedback control and the long-latency stretch response. Exp. Brain Res 218: 341–359, 2012. [DOI] [PubMed] [Google Scholar]
- Quinn JT, Schmidt RA, Zelaznik HN. Target-size influences on reaction time with movement time controlled. J Mot Behav 12: 239–261, 1980. [DOI] [PubMed] [Google Scholar]
- Rogers R, Monsell S. Costs of a predictible switch between simple cognitive tasks. J Exp Psychol Gen 124: 207–231, 1995. [Google Scholar]
- Rushworth MF, Hadland KA, Gaffan D, Passingham RE. The effect of cingulate cortex lesions on task switching and working memory. J Cogn Neurosci 15: 338–353, 2003. [DOI] [PubMed] [Google Scholar]
- Scheidt RA, Reinkensmeyer DJ, Conditt MA, Rymer WZ, Mussa-Ivaldi FA. Persistence of motor adaptation during constrained, multi-joint, arm movements. J Neurophysiol 84: 853–862, 2000. [DOI] [PubMed] [Google Scholar]
- Scott SH. Optimal feedback control and the neural basis of volitional motor control. Nat Rev Neurosci 5: 532–546, 2004. [DOI] [PubMed] [Google Scholar]
- Selen LP, Franklin DW, Wolpert DM. Impedance control reduces instability that arises from motor noise. J Neurosci 29: 12606–12616, 2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shadmehr R, Krakauer JW. A computational neuroanatomy for motor control. Exp Brain Res 185: 359–381, 2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shadmehr R, Orban de Xivry JJ, Xu-Wilson M, Shih TY. Temporal discounting of reward and the cost of time in motor control. J Neurosci 30: 10507–10516, 2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shima K, Tanji J. Role for cingulate motor area cells in voluntary movement selection based on reward. Science 282: 1335–1338, 1998. [DOI] [PubMed] [Google Scholar]
- Skinner B. Selection by consequences. Science 213: 501–504, 1981. [DOI] [PubMed] [Google Scholar]
- Tabata H, Miura K, Kawano K. Trial-by-trial updating of the gain in preparation for smooth pursuit eye movement based on past experience in humans. J Neurophysiol 99: 747–758, 2008. [DOI] [PubMed] [Google Scholar]
- Todorov E. Optimality principles in sensorimotor control. Nat Neurosci 7: 907–915, 2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Todorov E, Jordan MI. Optimal feedback control as a theory of motor coordination. Nat Neurosci 5: 1226–1235, 2002. [DOI] [PubMed] [Google Scholar]
- van Beers RJ. Motor learning is optimally tuned to the properties of motor noise. Neuron 63: 406–417, 2009. [DOI] [PubMed] [Google Scholar]
- van Beers RJ, Brenner E, Smeets JB. Random walk of motor planning in task-irrelevant dimensions. J Neurophysiol 109: 969–977, 2013a. [DOI] [PubMed] [Google Scholar]
- van Beers RJ, van der Meer Y, Veerman RM. What autocorrelation tells us about motor variability: insights from dart throwing. PLoS One 8: e64332, 2013b. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Verstynen T, Sabes PN. How each movement changes the next: an experimental and theoretical study of fast adaptive priors in reaching. J Neurosci 31: 10050–10059, 2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- White O, Diedrichsen J. Flexible switching of feedback control mechanisms allows for learning of different task dynamics. PLoS One 8: e54771, 2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Witney AG, Vetter P, Wolpert DM. The influence of previous experience on predictive motor control. Neuroreport 12: 649–653, 2001. [DOI] [PubMed] [Google Scholar]
- Wong AL, Haith AM, Krakauer JW. Motor planning. Neuroscientist 21: 385–398, 2015. [DOI] [PubMed] [Google Scholar]
- Wong AL, Shelhamer M. Similarities in error processing establish a link between saccade prediction at baseline and adaptation performance. J Neurophysiol 111: 2084–2093, 2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wyatt HJ, Pola J, Fortune B, Posner M. Smooth pursuit eye movements with imaginary targets defined by extrafoveal cues. Vision Res 34: 803–820, 1994. [DOI] [PubMed] [Google Scholar]
- Zhang H, Morvan C, Etezad-Heydari LA, Maloney LT. Very slow search and reach: failure to maximize expected gain in an eye-hand coordination task. PLoS Comput Biol 8: e1002718, 2012. [DOI] [PMC free article] [PubMed] [Google Scholar]