Seeing versus Believing: Conflicting Immediate and Predicted Feedback Lead to Suboptimal Motor Performance

Abstract

Reaching hand movements tend to follow straight paths. Previous work has suggested that when visual feedback is perturbed such that straight hand motions are seen as curved motions, the motor system adapts to restore straight visual motion. We show that under a nonlinear visuomotor transformation, one that maps straight hand motions to high-curvature motions of a visual cursor, reaching movements do not converge with practice toward a straight path of either the hand or the cursor. Instead, hand trajectories converged to a repeatable and characteristic curved shape. We propose a new computational model in which the adapted trajectories are obtained by minimizing a cost function composed of two terms. The first term enforces hand-movement smoothness. The second term penalizes average visual aiming error, which is the instantaneous discrepancy between the direction of the hand movement and the direction of the vector that points from the cursor to the target. Our results are consistent with the model's predictions and demonstrate a persistent effect of the predicted feedback of direction errors despite the possibility of producing smoother hand motions by ignoring it.

Introduction

Human performance is often considered to be guided by some optimization principle. Measuring performance under a variety of conditions may uncover the nature of the minimized cost function associated with the optimization principle. To this end, the study of reaching movements yielded a variety of cost-functions (Flash and Hogan, 1985; Uno et al., 1989; Harris and Wolpert, 1998; Todorov and Jordan, 2002) whose optimal solutions mostly agree with psychophysical data. In particular, the tendency of reaching movements to follow a straight path is well explained by optimization models since a straight path is an optimal (or near optimal) solution for many possible cost functions under most boundary conditions.

Under normal conditions, perceptual and motor criteria for movement optimization coincide. However, when vision is perturbed adapted trajectories can be used to uncover the influence of perceptual criteria on movement planning. Earlier studies of visuomotor perturbations of hand movements have shown that when visual feedback is artificially altered such that straight hand motions are mapped to curved motions of a cursor, subjects tend to restore rectilinear trajectories of the controlled visual cursor (Flanagan and Rao, 1995; Wolpert et al., 1995; Krakauer et al., 1999). In these studies, the cursor position was obtained from the hand position via a linear or a smooth nonlinear map that transforms straight lines in the hand space into low-curvature convex curves in the visual space.

In this study, we consider the effects of a nonlinear visuomotor transformation that both distorts the path of the controlled cursor and changes the initial and final position such that straight hand motions map to high-curvature concave motions. Under this transformation if the adapted behavior recovers a rectilinear motion of the cursor, one would conclude that the reorganization of movement is driven by optimization of smoothness of the visual motion. In contrast, if adaptation results in a rectilinear motion of the hand, one would conclude that smoothness of the actual, not the visual, hand motion guides the adaptive process. Our findings are not compatible with either conclusion. Subjects learned rapidly to correct the final error, but did not converge with practice toward a rectilinear path of either the hand or the cursor. Instead, the adapted trajectories can be explained by minimizing a cost function that contains both a hand-smoothness term (jerk) and a visual “aiming error,” which is the discrepancy between the direction of the cursor motion and the line joining the cursor to the target. Our results demonstrate that moving toward a visible target is a persistent goal of the motor system, even when it conflicts with the well-practiced final position of the movement.

Materials and Methods

Experimental setup

A total of 25 neurologically healthy subjects (11 male and 14 female) participated in the studies; all subjects provided written informed consent that was approved by the Northwestern University Institutional Review Board. Subjects were trained to perform 20 cm back and forth reaching movements between two circular targets (radius of 0.5 cm) in the horizontal plane (Fig. 1) using a robotic manipulandum (Dingwell et al., 2002). The subjects' arm was free to move in the horizontal plane and the motors on the manipulandum were shut off and supplied no forces to the arm.

Figure 1.

Experimental setup. a, The subject controls the position of a cursor (black circle) by moving the handle of a 2 link robotic manipulandum. The subject has to bring the cursor to the target (gray circle) within 600 ms after it appears. The cursor and target are projected on a white screen that conceals the subject's arm. b, On most trials, the location of the cursor is perturbed by a sigmoidal perturbation. The gray curve demonstrates the cursor path when the hand moves straight forward (black line).

Reaching tasks

The controlled cursor and the targets were projected in real-time on a horizontal white board that was located above the manipulandum. Subjects had no vision of their arm. They were instructed to bring the cursor to the target and stop there within 0.6 ± 0.1 s. When both accuracy and time requirements were satisfied, the target would change its color from yellow to red and an “explosion” sound would go off. When the cursor hit the target outside the required time limits (i.e., either too early or too late), the target would turn green but no sound would be heard. In addition, after each reaching movement subjects were given visual feedback as to whether their movement was on time, too slow, or too fast. To advance to the next trial subjects had to bring the cursor to the center of the target, which served as the starting point for the next reaching movement. No instructions were given concerning what trajectories subjects could follow: they were free to choose any trajectory they desired as long as they were able to stop in the target zone within the designated time. Data regarding the cursor and hand positions were recorded at 100 Hz and were analyzed off-line.

Each experiment was a sequence of unperturbed and perturbed movement trials. During unperturbed trials, the location of the projected cursor (c_x,c_y) was identical to the location of the hand (h_x,h_y) on the horizontal plane. During visually perturbed trials, the location of the cursor was perturbed by a sigmoidal transformation along the lateral (right/left) axis, i.e.:

with σ(h_y) = D(2/(1+e^{−(hy−Thr)/S)}−1). This perturbation displaced the cursor laterally depending upon the position of the hand along the frontal (proximal/distal) direction.

Experimental protocols

The basic reaching tasks described above (unperturbed and perturbed) were used in each of four experiments.

Experiment 1.

Seven subjects participated in the first experiment, which included 4 consecutive days with two training sessions per day. The first session in each day included 50 unperturbed trials

The second session included 250 perturbed trials. We used two values for the perturbation extent (D = 5 cm and 2.5 cm) and two values for the sigmoidal slope (S = 0.002 and 0.01) for a total of 4 different experimental conditions. The sigmoidal threshold (Thr) was set midway between the two targets. The perturbation parameters (D and S) were kept constant throughout each session. Across sessions, we randomized the perturbation parameters to avoid order effects.

Experiment 2.

Experiment 2 was aimed at testing how adaptation to the visual perturbation generalizes to unpracticed targets. Seven different subjects participated in this experiment. Here, only one set of perturbation parameters was used (D = 5 cm, S = 0.002). This experiment included a familiarization session of 80 veridical trials followed by a visually perturbed session that included 600 trials, of which 200 were “catch” trials. During catch trials the visual feedback of the cursor was suppressed and subjects were not given knowledge of their performance. Catch trials included reaches to the two practiced targets and to three additional new, unpracticed targets that were equally spaced between the practiced targets (5, 10 and 15 cm from the distal target) (see Fig. 3b). The unpracticed targets were introduced only during catch trials with no visual and performance feedback.

Figure 3.

a, Postadaptation average trajectories on catch trials without visual feedback (solid curves). Average trajectories for trials with vision are presented for comparison (dotted curves). Data shown for large and steep perturbation. b, Postadaptation average trajectories (solid curves) to unpracticed intermediate targets (red crosses on the right panel) without visual feedback. Dotted curves show average trajectories to practiced targets with vision.

Experiment 3.

Experiment 3 was aimed at testing predictions of our model. Seven different subjects were trained in four sessions on one day. The same set of perturbation parameters (D and S) were used in this experiment as in the second experiment. However, in this experiment there were two different sigmoidal thresholds, Thr. One threshold was placed between the midpoint and the proximal target, and the other between the midpoint and the distal target. Unperturbed trials were used in the first and third session. Perturbed trials were used in the second and fourth session. Each perturbed session used one of the two Thr values and the presentation order was randomized for each subject.

Experiment 4.

Experiment 4 was aimed at testing whether sensory uncertainty can explain the shape of the adapted trajectories observed in the previous experiments. In this experiment, four different subjects performed reaching movements from a proximal starting point to a distal target. The starting point was positioned 5 cm to the right of the proximal target of the previous experiments, and the location of the target was positioned 5 cm to the left of the distal target of the previous experiments. This was done to align the start and end hand positions of this experiment with those of previous experiments under the sigmoidal visual perturbation. At the end of each trial visual feedback was turned off and the robotic manipulandum actively transported the subject's hand back to the starting location. After 60 unperturbed practice trials with veridical visual feedback, subjects were trained in two sessions (420 trials each) during which the cursor became invisible once it left the starting point. Vision of the cursor resumed once it crossed an invisible horizontal line between the initial location and the target. When the cursor became visible, its location deviated horizontally from the location of the hand by a random amount drawn from a Gaussian distribution with zero mean and SD of 2.5 cm. The location of the “visibility line” between the target and the starting-point was fixed for each session and was set to 5 cm away from the starting point during one of the sessions and 5 cm away from the target during the other session. Catch trials (10 in the first session and 60 in the second and third sessions), for which no visual feedback was given, were occasionally presented. Data were analyzed for the catch trials only.

Data analysis

Average trajectories were calculated using the last 15% trials of each experimental session. Trajectory curvature was measured by the deviation from a straight line at peak velocity. Initial movement direction was calculated between the starting point and the location of the hand when the velocity first reached 40% of peak velocity. Average trajectories were fit using the model described below by means of grid search on the parameter space {a_1,…5; b_1,…5).

Results

Experimental results

Subjects performed reaching movements in the horizontal plane using a robotic manipulandum (Dingwell et al., 2002) without vision of their arm. The position of the controlled visual cursor (c_x,c_y) was calculated based on the position of the hand, (h_x,h_y) and projected along with the targets on a horizontal screen located above the manipulandum. On visually perturbed trials, the location of the cursor was perturbed using a sigmoidal transformation on the lateral axis (Eq. 1; Fig. 1) (Materials and Methods).

During the familiarization block when visual feedback was unperturbed, subjects tended to move between the targets in a straight movement with a bell shaped velocity profile. During the first, unexpected exposure to the perturbed trials, subjects moved the cursor in a straight line toward the target and then corrected sharply (Fig. 2a) a couple of hundred milliseconds after experiencing the perturbation. After a few dozen trials they were able to attain the target using a smooth curved hand motion (Fig. 2b), that was highly stereotyped and showed little trial-to-trial variation (Fig. 2c). The curvature of the hand trajectory varied slightly across subjects (Fig. 2d) and was higher for the large perturbation extent (0.02 ± 0.009 m) than for the small extent (Table 1). In fact, the hand trajectories under the two extents had a similar shape, with a 2:1 scaling of the horizontal dimension consistent with the ratio between the two extents. In addition, the perturbation extent had a significant effect on the endpoint error, both during initial exposure to the perturbation and for postadaptation trials (Table 1). The slope of the sigmoidal perturbation (parameter S in Eq. 1) had a significant but smaller effect on the trajectory curvature and on the endpoint error during initial-exposure trials (Table 1). However, the sigmoidal slope had no effect on the endpoint error during postadaptation trials.

Figure 2.

a, Two successive trials (away and toward the body, shown in black and blue, respectively) immediately after a subject was exposed to the visual perturbation. The subject corrects his movements a few hundred milliseconds after the perturbation occurs (left panel, hand trajectory; right panel, cursor trajectory). b, Postadaptation average trajectories for all subjects (small perturbation size, shallow perturbation slope). A straight hand trajectory (red dotted curve) would have produced a smoother hand and cursor paths than the paths subjects performed after adaptation. c, Postadaptation single trial trajectories for a typical subject show little trial-by-trial variation. d, Average trajectories of each subject are plotted with different colors. All share the same shape but with a slightly different curvature.

Table 1.

Effects of perturbation parameters on kinematics of average trajectories and endpoint error of individual trajectories (experiment 1)

	Perturbation extent		Perturbation slope
	2.5 cm	5 cm	Shallow (0.01)	Steep (0.002)
Curvature	1 ± 0.5 cm	2 ± 0.9 cma	1.4 ± 0.8 cm	1.6 ± 0.9 cma
Initial direction	92.16 ± 8.82	91.1 ± 13.14a	91.26 ± 9.9	91.98 ± 12.42
Endpoint error (initial exposure)	1.2 ± 1.1 cm	2.6 ± 3.6 cma	1.6 ± 2.2 cm	2.2 ± 3.2 cma
Endpoint error (post adaptation)	0.7 ± 0.67 cm	1.0 ± 1.7 cma	0.9 ± 1.57 cm	0.85 ± 1.0 cm

^a Statistically significant effect.

In contrast with the trajectory curvature, the initial movement direction was less influenced by the perturbation parameters. The perturbation slope had no significant effect on the initial trajectory direction while the perturbation extent had a small but significant effect on the initial direction (Table 1).

Increased training did not “straighten” hand trajectories. The last 100 trials (of 450 trials) showed no significant change in the curvature of hand trajectory. The adapted curved hand trajectory does not optimize the smoothness of the hand motion; nor does it optimize the smoothness of the cursor motion. A straight hand path with a bell shaped velocity profile would minimize the jerk of the hand motion and would also result in a smoother cursor motion than the adapted cursor trajectory (Fig. 2b, dotted red curve).

After adapting to the visual perturbation, hand trajectories during catch trials with no visual feedback of the moving cursor were curved similarly to the trajectories of trials with visual feedback (Fig. 3a). Additionally, when reaching to the not-practiced set of intermediate targets without visual feedback, subjects generalized the adapted trajectory by scaling it down in proportion to the distance to the target (Fig. 3b). We therefore conclude that the curved hand trajectory reflects a preplanned feedforward motion and that its curvature is not due to a visually triggered correction.

A model

An important feature of the adapted hand trajectory is that the initial direction of the movement is aimed almost straight toward the target location and not toward the “correct” final hand location, the position at which the hand needs to be after the sigmoidal perturbation (Eq. 1) has occurred to place the cursor in the target. This is remarkable especially given the magnitude of the directional deviation (∼14° and 26° for the small and large perturbation extent, respectively). Furthermore, after experiencing the perturbation for a few trials the displacement of the cursor becomes quite predictable. Yet subjects do not take advantage of the deterministic nature of the perturbation and instead continue beginning the movement with cursor and hand motions that will need some correction. This phenomenon cannot be explained by movement planning that does not anticipate the cursor perturbation. As shown in the previous section, adapted hand trajectories maintained a similar path even when visual feedback of the cursor was removed. Subjects also exhibit clear aftereffects, a hallmark of movement adaptation. We therefore hypothesize that moving straight toward the visual target is an objective of the movement plan. This may be interpreted as a reluctance of the planning system to generate anticipatory corrections that would cause a large visual feedback error.

We combined the maximization of smoothness and the minimization of visual feedback error into a single cost function that represents a tradeoff between the smoothness of the hand motion (measured using the hand jerk) and the deviation of the instantaneous hand direction from the visual direction established by the vector connecting the cursor and the target (Fig. 4). Formally, this cost function is defined as the following:

where (x_f,y_f) is the visual target location, t_f is the movement time and k is the jerk of the minimum-jerk trajectory for the given boundary conditions. k ensures that P is consistently dimensionless. Since obtaining an analytical solution for the minimization of the cost function is hard, we solve the minimization problem numerically. We first assume a parameterized solution for the hand trajectory and then minimize the cost function over this set of parameters. We chose the family of fifth order polynomials as they include the optimal solution for the cost function for α = 1. The trajectories are therefore defined as the following:

where τ = t/t_f, and P was minimized numerically over the set of a_1,2,…,5 and b_1,2,…,5. The only free parameter of the model (α) was fit to the subjects' average trajectory under the small extent and the steep slope condition (α = 0.96) (Fig. 5a, dashed red curve). We then used the same parameter value to successfully predict (Fig. 5a, solid black curves) the average trajectory for the rest of the experimental conditions (Fig. 5a, solid red curves and green curves). Note that using an α value of 0.96 yields a much more curved trajectory than using an α value of 1.0 (which results in a straight line). Average trajectories of each subject varied in curvature and thus in the fitted model-parameter α (Table 2). One possible concern over parameterizing the optimal solution using Equation 3 is that it may constrain the solution space. To address this concern, we use a less constrained parameterized solution to Equation 2 in which x_hand(t) and y_hand(t) are each spline-interpolated between N = 10 anchor points equally spaced in time. The optimal solution to Equation 2 using this 20 dimensional parameterization yield a trajectory which is very similar to the solution obtained using Equation 3, although it is less curved toward the end of the movement (Fig. 5a, dashed black curve).

Figure 4.

Aiming error represent the angular difference between the direction of the hand movement (solid arrow) and the direction of the vector (dashed arrow) that connects the visible cursor and the target.

Figure 5.

a, Postadaptation average trajectories for small (red curves) and large (green curves) perturbation size and shallow (solid) and steep (dashed) perturbation slope. Note that the dashed and solid curves mostly overlap. The model free parameter, α, was fit to the large steep condition and then used to predict the other three perturbation conditions. The model's optimal solutions were parameterized using a fifth order polynomials (solid black curves) and using an interpolated trajectory over 10 anchor points (dashed black curve). b, The model predicts (solid and dashed black curves) that when the perturbation midpoint is set closer to the target (dashed green line), the average postadaptation trajectory is much more curved than when the perturbation midpoint is set closer to the starting point (red dashed line). Experimental data (green and red curves) confirms the prediction. Note that the optimal solution that uses the fifth order plynomials fits the experimental data slightly better than the solution obtained using the interpolated trajectories.

Table 2.

Values of the model's free parameter (α) as fitted per experiment for the average trajectory of all subjects and for individual subjects (different subjects per experiment)

	Average trajectory	Individual subjects
Exp. 1	0.94	0.01	0.80	0.96	0.96	0.96	0.993	0.997
Exp. 2	0.98	0.56	0.96	0.96	0.98	0.989	0.989	0.997
Exp. 3	0.91	0.56	0.91	0.91	0.91	0.997	0.997	0.999

Exp., Experiment.

One of the model's predictions is that setting the position of the perturbation's midpoint (Thr) closer to the starting position will decrease the curvature of the optimal trajectory. This is because the earlier the perturbation occurs in the movement, the earlier the hand trajectory can head toward the correct final hand location while moving in the direction of the vector that connects the cursor and the target. Conversely, when the perturbation's midpoint is set closer to the target, straight or slightly curved hand trajectories will be heavily penalized during much of the movement for deviating from the visual cursor-target vector. This will increase the curvature of the optimal trajectory. Indeed, the results of experiment 3 (Fig. 5b) show that when the perturbation's midpoint was set closer to the target (dashed green line) the average trajectory of all subjects (solid green curve) was much more curved than when the perturbation's midpoint (dashed red line) was set closer to the starting point (dashed red curve) The model's optimal trajectories obtained using the fifth order polynomials (solid black curves) closely match these results. Similar trajectories are obtained using the interpolated anchor-points solution, although the latter match the experimental data less closely than the polynomial solutions. These results are consistent with the hypothesis that reaching movement in free space constitute a system of “kinematic primitives” that the adaptive controller uses as a basis for generating optimal solutions in more complex conditions, such as those explored here.

Since sensory uncertainty can affect motor planning (Todorov and Jordan, 2002; Chhabra and Knill, 2005; Chhabra and Jacobs, 2006), it is possible that the curved trajectories we observe in this study are shaped by the uncertainty regarding the lateral location of the controlled object. It may be that, despite the long exposure to the perturbation and despite the deterministic nature of the perturbation, the relatively high curvature of the cursor movement in the middle of the workspace increases the perceived uncertainty of the lateral location of the cursor. Optimal control that minimizes endpoint error under signal dependent noise may result in a curved planned trajectory when the uncertainty regarding the controlled endpoint is not equal in the two axes. To challenge this hypothesis, we conducted a forth experiment in which subjects had no vision of the cursor until the cursor crossed an invisible line between the starting point and the target. When the cursor became visible, its location deviated from the location of the hand by a random amount drawn from a Gaussian distribution (see Materials and Methods). Catch trials, during which no visual feedback was given, were occasionally presented. This procedure was repeated for two different “transition” lines between the starting point and the target, one closer to the target and the other closer to the beginning location. The average trajectories of both conditions are straight and starkly different from the trajectories subjects performed during catch trials when practicing under the sigmoidal visual perturbation (Fig. 6). Therefore, we conclude that uncertainty regarding the target location cannot account for the curved shape of the trajectories observed under the deterministic perturbation of Equation 1.

Figure 6.

Subjects were trained to perform reaching movements from a proximal starting point to a distal target. All curves are average trajectories for catch trials with no visual feedback. Blue curve, Average trajectory after training with veridical visual feedback. Green curve, Average trajectory after training with no vision at the beginning of the movement and deviated visual feedback that reappears close to the target. Red curve, After training with deviated visual feedback that reappears close to the starting point. All trajectories are straight and thus exclude the possibility that uncertainty regarding the target location accounts for the curved trajectories observed under the sigmoidal perturbation.

Discussion

It remains an open question whether motion planning optimizes perceptual or execution costs. Several studies and models of motor planning suggest that motor planning optimizes execution related costs, whether kinematic (Flash and Hogan, 1985), dynamic (Uno et al., 1989) or endpoint error (Harris and Wolpert, 1998; Todorov and Jordan, 2002). Others studies have demonstrated that motor planning is influenced by perceptual factors accumulated throughout the movement (Wolpert et al., 1994, 1995; Flanagan and Rao, 1995). Note that we consider endpoint error as an execution cost despite being based on perception. These studies show that under nonlinear visual perturbations, adapted trajectories tend to restore straight paths in the visual domain. Here, we show that under sigmoidal visual perturbation the motor system plans a trajectory that does not minimize the smoothness of the hand motion or the smoothness of the cursor. Both hand and cursor trajectories are curved considerably even after prolonged training.

Several studies have demonstrated the effect of execution and sensory uncertainty on motor planning (Todorov and Jordan, 2002; Chhabra and Knill, 2005; Chhabra and Jacobs, 2006). However, we argue that sensorimotor uncertainty does not contribute to the phenomena reported here. First, the visual perturbation used in our experiments is deterministic and therefore, after the adaptation period is complete, should not increase the uncertainty regarding the cursor location. Second, even if we accepted that visual perturbations increase sensory uncertainty we demonstrated experimentally that inducing uncertainty regarding the lateral location of the cursor does not significantly change subjects' trajectories. We therefore conclude that our results cannot be explained by an increased uncertainty regarding the lateral location of the cursor.

Another alternative interpretation to our results is that subjects adapt differently depending on when they observe the visual perturbation. Recent work by Ghez and Scheidt (Ghez et al., 2007; Scheidt and Ghez, 2007) has shown that under visual rotation conditions subjects maintain a straight motion path when they see the full cursor trajectory. However, if they only receive final position feedback, they adapt by generating a curved hand path. These results suggest that when subjects receive error feedback late in the movement, they will only adjust the latter part of the movement. In contrast, if they receive error feedback early in the movement, they will adapt their movement right from the beginning. It is thus possible that the curved hand trajectories observed in our experiments can be attributed to the fact that the visual perturbation is not noticeable at the beginning of the movement. Thus, the roughly straight hand motion seen at the beginning of the trajectory (Table 1) may not be a result of optimizing our cost function but rather just the default motion path. However, this explanation cannot account for the marked change in the initial direction of motion seen in experiment 3. When the perturbation midpoint (Thr) is set close to the target, the initial direction of the hand movement changes significantly from the default straight path (79 ± 18.8°; p < 0.001). Surprisingly, the initial movement direction is in the direction opposite to the expected final position of the hand when the target is obtained (Fig. 5b). This phenomenon was predicted by our model and was confirmed experimentally.

Instead, we suggest that the planned trajectory minimizes a cost function that consists of two terms. The first term is a performance related cost, which in the current implementation of the model is a standard smoothness term (minimum jerk) of the hand motion. The second cost term penalizes deviations of the hand movement direction that is not aligned with the direction of the vector that points from the cursor to the target. Both terms are integrated throughout the movement. We show that the model, which has only one free parameter, the relative weight of the two cost terms, fits the experimental data very well. The model also very accurately predicts the large effect the midpoint of the perturbation has on the adapted trajectory.

In a recent study (Liu and Todorov, 2007), the authors studied how subjects adapt to systematic target perturbations. They report that when subjects expect a target jump in a certain direction during the movement, they nevertheless initiate a movement toward the initial target position, which leads to curved trajectories. However, if the target becomes invisible immediately after movement onset and reappears at the new location after the movement has ended, subjects now make straight movements to the predicted final position. These results are consistent with our results and extend them to conditions in which the target (not the cursor) undergoes visual perturbation. When the subjects of Liu and Todorov (2007) expect the target and cursor to be visible throughout the movement they adapt to a trajectory that minimizes both smoothness (execution cost) and cursor-target direction error (visual cost). In the experimental condition where the target becomes invisible at movement onset, the visual cost becomes irrelevant, as the cursor-target direction error is integrated throughout the movement. Lack of visual feedback of the target or the cursor throughout the movement renders the visual part of the cost function irrelevant, resulting in an optimal straight trajectory.

The second cost-term in our proposed cost-function contains an estimate of the hand movement direction. What is the physiological source of this estimate? Several possibilities come to mind. First, it may be that what is really used in the cost function is not the direction of the hand motion but rather the direction of the cursor motion. Because of the nonlinearity of our visuomotor perturbation, the movement direction of the hand and the movement direction of the cursor are not identical throughout the movement. Therefore, replacing the hand motion direction with the cursor direction should change the optimal trajectory of the model. We tested this possibility by fitting the experimental data with a modified version of the cost function, one that contains the direction of the cursor movement instead of the hand movement. We quantified the quality of the fit by calculating the error area enclosed between the actual trajectory and the best fitted model trajectory: the smaller the area the better the fit. We found that the cost function that uses the hand direction fits the experimental data better (average error area = 0.03 ± 0.03 cm²) than the cost function that uses the cursor direction (average error area = 0.43 ± 0.41 cm²).

The second possibility for the estimate of the hand movement direction is proprioception. There are no exact estimates of how accurate is the proprioceptive perception of the motion direction of the hand. Therefore, although proprioception may not be accurate enough to contribute to the cost function proposed here we cannot rule out this possibility.

A third mechanism that can be used for estimating the hand movement direction is predictive, namely a forward model that uses the efferent copy of the motor command. This is an intriguing possibility as it suggests that the aiming error in our cost function represents a discrepancy between the forward model prediction of the perceived consequences of the motor command and the desired movement direction. Note that under this interpretation the vector (x_hand, y_hand) is not the actual hand velocity vector, but rather the predicted visual consequence (in our case, the predicted cursor velocity vector) of the motor command. This interpretation is especially appealing given recent evidence that motor adaptation is driven by sensory prediction error (Tseng et al., 2007). This interpretation also reconciles our results with previous studies that show that under linear and smooth nonlinear visuomotor perturbations subject restore straight visual trajectories. If the direction error is estimated using the predictions of the forward model rather than the proprioceptive feedback, than the minimization of the direction error is achieved by adjusting the forward model and restoring a preperturbation trajectory. A well adapted forward model successfully predicts the visual consequences of a hand movement, and a straight visual path will minimize both terms of the proposed cost function (Eq. 2). However, in our experimental setup, subjects experience a highly curved visuomotor perturbation that for most of the movement keeps the direction of the cursor identical to the direction of the hand. It is therefore likely that the forward model does not “learn” the perturbation. In this case, the minimization of sensory prediction error is achieved by changing the planned trajectory.

Our results and model suggest that reaching movements are planned to reduce the integrated instantaneous discrepancy between the expected movement direction and the straight line that connects the visual representation of the hand and the target. This happens despite that by ignoring the current location of the target (which is equivalent to setting α to 1 in Eq. 2) and planning the movement based on the well-practiced final location of the movement, one can achieve a more efficient straight hand motion.

Why does the motor system persistently align the movement direction with the visual direction to the target? One possibility is that this mechanism helps to produce straight movements under normal conditions, which are optimal for execution. Another reason may be purely perceptual. It may be that maintaining a direction of movement that is aligned with the visual direction to the target allows for better visual estimation of directional errors, which are used for online corrections and for longer term adaptation. Ultimately, our findings suggest that our brain has a bias against accepting statistical inference as the only basis for movement planning (Körding and Wolpert, 2006). We tend to base our movements on what we currently see, no matter how likely it is that this behavior would be outperformed by basing our movement on what we expect to see eventually.