A Real-Time State Predictor in Motor Control: Study of Saccadic Eye Movements during Unseen Reaching Movements

Abstract

Theoretical motor control predicts that because of delays in sensorimotor pathways, a neural system should exist in the brain that uses efferent copy of commands to the arm, sensory feedback, and an internal model of the dynamics of the arm to predict the future state of the hand (i.e., a forward model). We tested this theory under the hypothesis that saccadic eye movements, tracking an unseen reaching movement, would reflect the output of this state predictor. We found that in unperturbed reaching movements, saccade occurrence at any timet consistently provided an unbiased estimate of hand position at t + 196 msec. To investigate the behavior of this predictor during feedback error control, we applied 50 msec random-force perturbations to the moving hand. Saccades showed a sharp inhibition at 100 msec after perturbation. At ∼170 msec, there was a sharp increase in saccade probabilities. These postperturbation saccades were an unbiased estimator of hand position at saccade timet + 150 msec. The ability of the brain to guide saccades to the future position of the hand failed when a force field unexpectedly changed the dynamics of the hand immediately after perturbation. The behavior of the eyes suggested that during reaching movements, the brain computes an estimate of future hand position based on an internal model that relies on real-time proprioceptive feedback. When an error occurs in reaching movements, the estimate of future hand position is recomputed. The saccade inhibition period that follows the hand perturbation may indicate the length of time it takes for this computation to take place.

When a reaching movement starts, the neural commands to the arm are “feedforward” in the sense that they rely on an internal model that predicts forces necessary to initiate the movement (Shadmehr and Mussa-Ivaldi, 1994; Thoroughman and Shadmehr, 1999). However, as the movement proceeds, these signals are augmented by “feedback” components that take into account sensory information from the moving limb. If a perturbation displaces the hand, compensatory commands are produced via a “short-loop” feedback mechanism with delays of 30–50 msec (Ghez and Shinoda, 1978) and a “long-loop” mechanism with delays of ∼150 msec (Gielen et al., 1988; Petersen et al., 1998). In theory, these delays can destabilize the limb. The importance of proper function of the long-loop error feedback control system in humans is illustrated in Huntington's disease, in which damage to the basal ganglia accompanies abnormal long-latency motor responses to somatosensory stimuli (Noth et al., 1985; Thompson, 1988; Thilmann et al., 1991). In these patients, movements often begin normally, but slight errors result in motor responses that produce jerky movements (Smith et al., 2000). How does the normal brain perform error feedback control despite long delays in sensory feedback?

Theory suggests that the brain may cope with sensory feedback delays by relying on an internal, neural model that predicts the effects of motor commands on the arm (Miall et al., 1993; Wolpert and Ghahramani, 2000). This is a “forward model” because it translates motor commands into predictions of future system state, modeling the forward dynamics of the system (Jordan and Rumelhart, 1992). To make this prediction, the forward model requires sensory feedback from the limb and a copy of currently planned motor commands (efferent copy). The state predictor might generate its output via an integration of the inputs through a model of the physical dynamics of the limb (Bhushan and Shadmehr, 1999). If such a system is used, motor commands that respond to a sensed error can be based on an estimate of the future state of the limb, rather than the potentially destabilizing alternative of where the limb was when error was sensed.

A crucial component of this theory is the idea that the brain can use feedback and efferent copy to compute the future state of the limb. To test this idea, experiments have been designed that ask subjects to program motor commands to one arm as a function of their estimate of the state of the other arm (Blakemore et al., 1998; Witney et al., 1999). Here, we approached the problem differently by asking whether subjects can use their eyes to pinpoint in real time the position of their unseen, moving hand. We chose eye movements as a proxy for the hypothesized state estimator because it has been suggested that the oculomotor system has access to an efferent copy of arm motor commands during visual tracking (Vercher et al., 1997). By perturbing the unseen hand and recording the saccadic response, we hoped to quantify the estimate of hand positions by the brain during error feedback control.

MATERIALS AND METHODS

Subjects (n = 6) held the handle of a robotic arm (Shadmehr and Brashers-Krug, 1997) and made reaching movements in the horizontal plane to targets that appeared at a distance of 10 cm in random directions (Fig. 1). The handle of the robot housed a force transducer at its base and a high-intensity light-emitting diode (LED) at its tip. An opaque screen (12×10 inches) was suspended 0.5 cm above the plane of the tip of the handle. A liquid-crystal display (LCD) projector suspended from the ceiling painted this screen. A dark, heavy cloth was draped around the screen and prevented the subjects from viewing their arms and hands.

Fig. 1.

Experimental setup. A, Subjects used a bite bar and were instructed to try to track their unseen hand during reaching movement. They were prevented from viewing their hands and arms, because both arms were covered under a heavy cloth (data not shown). The right hand held the handle of a robotic arm and moved it in the horizontal plane. The handle housed a high-intensity LED at its center and a force transducer at its base. The hand was below an opaque screen. An LCD projector, held from the ceiling, painted this screen.B, The task began with the robot bringing the hand to a random starting position, where a start target (green square) was displayed. The subject fixated the handle LED. The LED was turned off, and a movement target was displayed (yellow square). The subject saccaded to the target. The movement target was turned off, the handle LED was turned on, and the movement target was redisplayed. The subject fixated the LED. A stationary random-dot pattern was displayed and the start target was removed, signaling the subject to start the reaching movement. As soon as movement was detected, the handle LED was turned off. At the completion of the reach, the handle LED was turned on, the random-dot pattern was removed, and the target square was repainted, providing feedback to the subject. C, Two example trials. The component of eye and hand position parallel to the direction of target is plotted with green and black lines, respectively. Saccade origin is marked with a red dot, and saccade end point is marked with a blue dot.disp., Displacement.

A trial began with the handle LED on. A start target was displayed in a randomly selected start position, and the robot moved the handle (and the subject's hand) to it. After the subject fixated the start location, the handle LED and start target were extinguished, and a final target appeared at a randomly selected direction. The subject was required to fixate the final target for 0.5 sec (while maintaining the handle in the start position), after which the target was extinguished and the start target and the handle LED reappeared. The subject returned fixation to the start target, after which a stationary random-dot pattern filled the entire screen (∼35 dots/in²). Each dot was 0.25 mm² in size, and its position was randomly selected for each trial. After subjects fixated the start target for 0.5 sec, the target was extinguished so that only the random-dot pattern and the LED were visible. This signaled to the subject to begin movement to the remembered location of the target. As soon as movement initiation was detected (encoder resolution of handle position was better than 0.05 mm, and movement initiation was detected using a fixed 0.02 m/sec threshold), the handle LED was extinguished. Therefore, during the movement, the handle LED was off and the target was not displayed. The only visible information was the pattern of random dots. After completion of movement, the random dots disappeared, the handle LED was turned on, and the target reappeared. The target color provided feedback regarding the timing of the movement, and the position of the LED with respect to the target provided feedback regarding the accuracy of the movement. If the hand reached the target in 0.9 ± 0.05 sec, the target exploded and made a pleasing sound. It turned red if the hand arrived at the target too soon and blue if it arrived too late.

After ∼150 unperturbed trials, on pseudorandomly selected trials (probability of 58%), a 50 msec, 25 N force pulse pushed the hand in one of two directions perpendicular to the target direction at either 200, 250, 300, or 350 msec after movement initiation. Force-pulse latency was determined pseudorandomly. As before, no auditory or visual cues were available in these trials.

We recorded eye position at 100 Hz using an infrared camera and light source (iView system; SMI Corp., Berlin, Germany) mounted on a helmet that was itself tracked using a Polhemus tracker (Polhemus Corp., Colchester, VT). To stabilize the recordings, subjects used a bite bar anchored to the floor. We performed a two-part calibration procedure. Each block of ∼40 trials began with saccades to 24 points painted on the plane immediately above the hand. For each trial within the block, we checked on the calibration of the eye system with the hand position (as reported by the robot) by using the fixation period when the LED at the handle was on and the fixation period when the target was painted. Trials were aborted if eyes were not within 7.5 mm of the presented target. False starts also resulted in the rejection of that trial.

Subjects received training before experiments to familiarize them with the setup. In the pre-experiment session, robot motors were always off, and subjects were asked to try to look at the perceived position of their hand during unseen reaching movements. A typical subject's performance began with a single saccade to the remembered location of the target as the reach took place. With additional practice, they produced multiple saccades during the reach. Subjects never received feedback regarding the accuracy of their saccades during the reach. We began the experiments once the subjects were able to consistently make at least two saccades per reaching movement. Once the experimental session began, however, we did not exclude any trials other than those with false starts or poor calibration.

RESULTS

We instructed the participants to look at their hand to the best of their ability without vision of their hand or the target as they reached to the remembered location of a target (Fig. 1).

We begin by presenting the results of the unperturbed trials. The reach took place in 0.78 ± 0.18 sec and was accompanied by a sequence of 3.50 ± 1.1 saccades (mean ± SD). The trajectory of the hand was usually straight, but it varied in speed from movement to movement. An example of two trials that differed in speed is shown in Figure 1C. The timing and placement of saccades appear to correspond to the trajectory of the hand in each trial. However, the timing and placement properties of saccades varied greatly across the movements, resulting in a broad distribution in the probability of saccades during the reach (Fig.2A) and a wide scattering of saccade origins and end points (Fig.2B).

Fig. 2.

Saccades during reaching movements in unperturbed trials. A, Probability (prob.) of saccade occurrence calculated in 10 msec time bins as a function of hand movement time. Each bin indicates the probability that in a single trial, a saccade would occur at that time bin. Reaches were to various target directions. B, Mean trajectory of the reach (± 1 SD; yellow line) is shown by the black line and is represented along directions parallel or perpendicular to the target direction. Saccades are represented by vectors of eye position change: this vector has an origin (red dots) and an end point (blue dots).C, Average timing, origins, and end points of the first three saccades (red and blue lines; ± 1 SD). The SD on timing of saccade end points is identical to the SD of saccade origin and is not shown for clarity. The green line is the average eye position across all trials and corresponds to the continuous representation of the discrete saccade data. Black lines show average hand position. A saccade end point is occasionally not equal to the origin of the subsequent saccade because of small amounts of smooth pursuit.Dir., Direction.

Because there were, on average, approximately three saccades during the reach, we began our analysis by considering the position of these saccades with respect to the hand. On average, saccades were initiated shortly after the hand passed the point of eye fixation and were made to a point that led the hand position (Fig. 2C). The end point of the first saccade was, on average, ahead of the hand toward the target (Fig. 2C). This eye position was maintained (although there were sometimes very small amounts of smooth pursuit, usually <4 mm) until the hand passed the point of fixation by ∼7 mm, at which point a second saccade was generated. Note that the hand is not visible during the movement. After hand position progressed beyond the current eye position, a third saccade was generated. The data in Figure 2C describe the temporal and spatial distribution of all first, second, and third saccades, regardless of the number of saccades that there might have been in a given reaching movement. However, there were many trials with only two saccades and other trials with four saccades (the probability that a trial would have a given number of saccades is shown in Fig.3A). It is not clear whether the second saccade in a two-saccade trial should be combined with the second saccade in a four-saccade trial. This variability in the number of saccades per trial results in the counterintuitive observation that in Figure 2C, the average origin of the third saccade is actually slightly less than the average end point of the second saccade. Indeed, the broad distribution of saccade timings (170–220 msec in Fig. 2C) suggests that they were not generated in a rigid temporal pattern. To investigate this further, we examined saccade generation probability as a function of time for trials with two, three, and four saccades separately. In each type of trial, we found a broadly distributed probability function peaking at ∼280 msec, with little evidence of periodicity (Fig. 3B), reinforcing the idea that saccade timing is highly variable and saccades are not generated with rigid temporal structure.

Fig. 3.

Saccades during reaching movements in unperturbed trials. A, Percentage of reaching movements that accompanied a given number of saccades. B, Probability (prob.) of saccade occurrence calculated in 10 msec time bins for reaching movements that had two, three, or four saccades.

Because our objective here is to compare the behavior of the eyes with that of the hand, we thought that rather than averaging all first, second, etc., saccades, a better approach might be to divide reaching movements into small temporal bins and analyze the behavior of saccades that began in each bin. For instance, we analyzed the end point, with respect to the hand, of saccades generated at different times into the movement.

We considered all saccades that took place in a given 10 msec time bin. For each of these saccades, we compared the eye positione(t) with the hand positionh(t) in the corresponding reaching movement. We represented these positions as displacements toward the target (i.e., scalar quantities). The result is an error measuree(t) − h(t) for each saccade. In principle, the measure could be positive or negative depending on values of e(t) andh(t) in a given trial. We averaged this error across all saccades in each 10 msec time bin. When all time bins were considered, the result was a measure of error between saccade end points and hand positions as a function of time into the movement. We extended this analysis by considering the possibility that the saccade was related not to the current position of the hand (current meaning the time bin in which the saccade occurred) but perhaps to where the hand was in the past or would be in the future. This measure is described by the formula e(t) −h(t + Δ). We considered Δ ranging from −300 to 500 msec. For example, when Δ = 0, the saccade end point was compared with hand position at the time of that saccade in the corresponding movement. When Δ = 200, the saccade end point was compared with hand position 200 msec after saccade time.

For a given 10 msec time bin, we found the value of Δ for which the average of the quantity e(t) −h(t + Δ) crossed zero (the average was over all the saccades that took place in that time bin). The result is shown in Figure 4A. On they-axis of this figure, we have the time bins in which saccades took place. On the x-axis, we have the time shift, Δ, that was imposed on the hand trajectory. We found that up to 600 msec into the reaching movement, the average error between eye and hand was consistently zero if the hand trajectory was shifted back by ∼200 msec (196 ± 31 msec). This means that on average, although saccade end point was a poor estimator of hand position at saccade time, it estimated hand position at a fairly consistent time in the future. This method of analysis suggested that saccades at any time (up to 600 msec into the movement) on average predicted hand position at time t + 196 msec.

Fig. 4.

Saccades during reaching movements in unperturbed trials. A, Eye position at saccade end point and hand position are represented as scalar quantitiese(t) andh(t), indicating position along the direction of the target. The figure illustrates the distribution of the time delays, Δ, for which the error measuree(t) −h(t + Δ) had an average value of zero across all saccades that took place in a 10 msec time bin. Each 10 msec time bin is considered independently, and the corresponding Δ is found. The Δ clusters at 196 ± 31 msec, indicating thate(t) was an unbiased estimator ofh(t + Δ) at Δ ≈196 msec.B, The SD of the error measuree(t) −h(t + Δ) at the Δ for which the measure had zero mean. Time of saccade refers to the time at which the saccade ended.

Although there were saccades that took place after 600 msec, there were no values of Δ for which the quantity e(t) − h(t + Δ) crossed zero; therefore, this resulted in no data points for these saccades in Figure4A. A closer look at these data showed that saccades that took place after 600 msec overestimated final hand position by an average of 11 mm, resulting in e(t) −h(t + Δ) to be positive for all Δ.

Figure 4A displays the time Δ for which saccades that took place at a given time t had an average errore(t) − h(t + Δ) that was zero. It is informative to ask for the distribution about this zero mean. To illustrate this, in Figure 4B we plotted the SD of e(t) −h(t + Δ) at the time Δ at which the average of this quantity was zero. Results showed thate(t) − h(t + Δ) had, on average, an SD of 11 mm about its mean of zero at optimal time Δ. Using the cumulative distribution function of the error quantity, this SD implies that on any given trial, a saccade at time tpredicted hand position at t + 196 msec (the average of the optimum Δs) to within 5 mm with a probability of 35.1%.

Another way to estimate Δ is to quantify howR² (coefficient of determination, equivalent to percentage variance explained by the regression) between e(t) andh(t + Δ) varies with change in Δ. This is illustrated in Figure 5. We considered a range of Δ between −200 and +500 msec and found thatR² reached a peak value of 0.88 at +150 msec.

Fig. 5.

Saccades during reaching movements in unperturbed trials. Eye position at saccade end point and hand position are represented as scalar quantitiese(t) andh(t), indicating position along the direction of the target. The figure illustratesR² between eye position at saccade end point e(t) and hand position at time of saccade t + Δ [(i.e.,h(t + Δ)] for all saccades. The maximum R² is at Δ = 150 msec. Time of saccade refers to the time at which the saccade ended.

The method of comparison of eye and hand data used to generate Figure 4weighs equally the Δs that were estimated across different times in finding an average Δ of 196 msec. This method is blind to the fact that the likelihood of saccade generation was not uniform across different saccade times (Fig. 2A). The method used to generate Figure 5, however, is more influenced by the relatively large number of saccades that occurred at ∼200 msec. Although the results of the two methods are similar, the former method of analysis is more consistent with the hypothesis that regardless of when a saccade is generated, it should predict hand position at a specific time in the future. Therefore, we used the average Δ of 196 msec to ask how much of the variability in hand trajectory from trial to trial was reflected in the variability in the saccades.

For each saccade that took place during the interval −200 to 600 msec, we compared eye position with hand position at saccade time + 196 msec (Fig. 6). We fitted a linear function to the hand and eye data and found a slope of 1 with bias of nearly zero for both x- and y-components of the data.R² in both cases was >0.85. The cluster of points around zero in the x- andy-components of this figure occurs because some of the movements were either horizontal or vertical, producing no change along the x- and y-directions, respectively.

Fig. 6.

Saccades during reaching movements in unperturbed trials. Eye position at saccade end point and hand position were represented as two-dimensional quantities indicating position in a Cartesian coordinate system with respect to origin of the reaching movement. The figure illustrates the position of the end point of a saccade and hand position in the corresponding reaching movement at saccade time + 196 msec for all saccades in all trials. Time of saccade refers to the time at which the saccade ended. There is a cluster of points at zero because a number of reaching movements were vertical or horizontal and had no x- or y-components.h, Hand position; e, eye position; m, slope of the line; b, crossing point.

The above data were all collected in conditions in which the hand freely performed a reaching movement. We subsequently quantified the behavior of the eye when the hand was suddenly perturbed. As before, subjects made reaching movements to targets, but now on randomly selected trials (probability of 58%), a 50 msec 25 N force pulse pushed the hand in one of two directions perpendicular to the intended movement direction at one of four possible times into the movement. Two perturbed trials are shown in Figure 7. In both cases, the first saccade leads the hand and has an end point that is along the direction of the target. However, the perturbation significantly alters the expected trajectory of the hand. Remarkably, the second saccade is not to a point where the hand was when it was perturbed, but rather to a point that the hand will visit in the future. The eyes fixate that location until the hand passes it; then a third saccade is generated to a location that again leads the hand.

Fig. 7.

Two representative trials during which the hand was perturbed with a force pulse. The gray andblack lines represent the trajectories of eye and hand, respectively. Gray dots locate the end point of each saccade. Black dots indicate hand position at the time of origination of that saccade. For example, e1 is the end point of the first saccade and h1 is the position of the hand at the start time of that saccade. pos., Position; sac, saccade.

The combination of movement directions and force pulses produced a wide range of hand displacements (the mean of the maximum hand displacement across movements was 40.2 mm). The probabilities of saccades during perturbed and unperturbed trials are shown in Figure8A. We found that a force pulse at 200 msec was followed by a sharp dip in saccade probability, reaching a minimum at 320 msec. When the hand was not perturbed, there was a peak in probabilities at ∼230 msec (Fig.2A). The time between the pulse onset and the dip was similar regardless of pulse timing. Pulses at 200, 250, 300, and 350 msec produced a probability minimum at latencies of 120, 90, 90, and 120 msec. A peak in saccade probability immediately followed the minimum. The pulses produced a peak in saccade probabilities at latencies of 180, 170, 160, and 180 msec after pulse onset. Therefore, it appeared that the perturbation to the hand caused a significant inhibition of saccades at 100 msec, followed by an increased probability of saccades at ∼170 msec after perturbation onset.

Fig. 8.

Trials in which movements were perturbed with a force pulse. A, Saccade probability as a function of movement time. The first gray bar indicates the period (50 msec) during which the reach was perturbed with a force pulse. In all subplots, the second bar is positioned at perturbation (pert.) onset + 150 msec and is 50 msec wide. This period consistently coincides with a peak in postperturbation saccade probabilities. B, Eye positions at the saccade end point for saccades that occurred 150–200 msec after onset of the pulse were represented as two-dimensional variables and were multiplied by a matrix [a −b; b a] to best estimate hand position at t + Δ with respect to the time, t, of the saccade in the same trial. Parameters a andb are shown (±95% confidence interval). Whena = 1 and b = 0, the saccade vector remains unscaled and unrotated. A rotation, arctan(b/a), and scaling, $\sqrt{(a^2+b^2)}$, of the saccade vector was needed to estimate hand position at all delays except at Δ = 150 msec. For this time lead, the actual saccades were an unbiased estimator of hand position. C, Force pulses were perpendicular to the direction of the target and displaced the hand by various amounts in each movement. The end points of saccades that occurred 150–200 msec after the pulse are plotted versus hand position in the same movement at saccade time + 150 msec. perp. dir, Perpendicular direction;m, slope of the line; b, crossing point.

We compared these postperturbation saccades (saccades that occurred at 150–200 msec after the pulse onset) with hand positions in the same movement. Unlike our analysis in Figures 4 and 5, in which eye and hand positions were viewed as one-dimensional quantities (displacement toward the target), because of the perturbation, here the eye and hand positions were represented as two-dimensional quantities (position parallel and perpendicular to the direction of target). For all saccades during the interval 150–200 msec after perturbation, we compared e(t) with the correspondingh(t + Δ). At a given Δ, we multiplied eye position by matrix [a − b; b a] to best estimate h(t + Δ) in the same trial. For all Δs other than 150 msec, we found that the relationship between eye and hand required a scaling $\left[\sqrt{(a^2+b^2)}\right.$] and rotation [arctan(b/a)] of the eye position vector (Fig. 8B). However, at Δ = 150 msec, the rotation angle was zero and the scale was 1.06. Therefore, saccades that occurred “in response” to the perturbation appeared to be an unbiased estimator of where the hand would be 150 msec in the future.

However, considerable movement-to-movement variations existed for error that was caused by the pulse in the trajectory of the hand. Depending on the direction of motion of the hand and its speed, force pulses, which were perpendicular to the direction of motion, produced a large range of perpendicular displacements (−70 to 70 mm). Did the saccades predict this variation from trial to trial? In Figure 8C, we compare the saccades that took place 150–200 msec after pulse onset with the error (perpendicular displacement) that was recorded in the trajectory of the hand in the same movement 150 msec after the saccade. The slope of the fit is 1.08, bias is −1.6 mm, andR² is 0.80. This suggests that when there were unpredictable pulses that displaced the hand from the nominal trajectory, a saccade that occurred in response to the pulse was, on average, an unbiased estimator of this displacement at saccade time + 150 msec and accounted for 80% of the trial-to-trial variance.

It seems reasonable that the brain programmed these postpulse saccades on the basis of feedback that it had received from the perturbed arm and an internal model that predicted where the hand would be in the future. If, indeed, the prediction was based on an internal model, then changing the dynamics of the arm unexpectedly after the perturbation should make the saccades inaccurate. For example, a resistive (or assistive) field introduced after the 50 msec pulse should cause a saccade that takes place 150–200 msec after pulse onset to overestimate (or underestimate) future hand position.

We performed two separate experiments in which we introduced a viscous force field that either assisted or resisted the subject's hand. As before, the pulse was present in only some of the trials (50%), and the field was engaged after the pulse in only a fraction of these pulsed trials (25%). We observed a postpulse minimum in saccade probability at 120 msec, followed by a peak at 150–200 msec (Fig.9A). For each postpulse saccade, we compared its end point with hand position at 150 msec after the saccade. In trials in which a field was present after the pulse, these saccades were inaccurate. A resistive field caused saccade overestimation of the effect of the pulse (Fig. 9B). An assistive field caused underestimation of the effect of the pulse.

Fig. 9.

In these trials, a viscous force field that either assisted or resisted the movement of the hand was introduced immediately after the offset of the force pulse to the hand.A, Saccade probabilities. The first gray bar indicates pulse period (50 msec). The second gray bar indicates a perturbation (pert.) period 150–200 msec after pulse onset. B, The end points of saccades that occurred 150–200 msec after the pulse are plotted versus hand position in the same movement at saccade time + 150 msec. Saccades overestimated hand position in resistive trials and underestimated hand position in assistive trials. perp. dir, Perpendicular direction.

DISCUSSION

Cortical motor commands that are generated in response to a perturbation reach the arm muscles 100–150 msec after perturbation onset (Petersen et al., 1998). For a moderate-speed movement that is completed in 500 msec, these delays are dangerously long, because they can destabilize the limb. How is stability maintained? One suggestion is that the brain may cope with sensory feedback delays by relying on an internal model that predicts the future state of the arm (Miall et al., 1993; Wolpert and Ghahramani, 2000). According to this theory, the brain programs motor commands on the basis of the prediction, rather than basing commands on where the arm was when it was perturbed (Bhushan and Shadmehr, 1999). We thought that if the brain can predict the future state of the arm, then perhaps eye movements can serve as a proxy for this prediction.

Our rationale for this conjecture was the wealth of data suggesting that smooth-pursuit eye movements are influenced by commands to the arm. For one-dimensional arm movements, it has been demonstrated that when the eyes track a visual target attached to the hand, the smooth-pursuit eye movements that result are more closely linked to the motion of the target (Steinbach, 1969; Gauthier and Hofferer, 1976;Koken and Erkelens, 1992) and can track a faster target (Gauthier et al., 1988; Vercher et al., 1993) than if the target were driven by an external source. This has suggested that the smooth-pursuit control system of the eyes uses an efferent copy of arm motor control signals to anticipate the direction and timing of arm movements (Vercher et al., 1996; Scarchilli and Vercher, 1999).

Given these data, we reasoned that subjects might be able to express an estimate of the state of their arm through programming of their eye movements. However, in smooth pursuit, visual information regarding the state of the limb is always present. We eliminated this information so that the behavior of the state estimator could be observed when the only feedback was through proprioception. Preventing viewing of the moving hand prevents generation of smooth pursuit and results in saccadic eye movements. If the target of the movement is visible, the eyes saccade to the target and maintain fixation until the end of the reaching movement (Neggers and Bekkering, 2001). Therefore, we eliminated target information during the reach and asked subjects to try to look at their unseen hand.

Some characteristics of reaching movements were stereotypical: the trajectory of the hand was straight, with a bell-shaped speed profile (Morasso, 1981). Subjects instructed to look at their unseen hand during reaching might have been expected to generate saccades stereotypically. We did not find this to be the case. The probability of generating a saccade was widely distributed during the movement. Although on average a 10 cm, 0.8 sec movement had approximately three saccades, the timing and position of these saccades varied greatly. On average, the end point of a saccade would lead the hand, and the eyes would remain there until the hand passed the fixation point, at which time another saccade would be generated (Fig. 2C). This finding is similar to the observation made by Johansson et al. (2001) in that the eye movement was apparently dependent on the occurrence of a kinematic event for the hand. However, whereas some reaching movements had two saccades, similar reaching movements in other cases had three or four. We therefore began by assuming that the timing of the saccade, rather than its serial order, was the relevant variable to analyze and tested the hypothesis that the end point of the saccade predicted hand location with a specific latency. For each saccade, we computed the distance between saccade end point and hand position (delayed by some latency, Δ). To allow for the possibility that the eye position related to hand position at some time in the future (or past), we computed the distancee(t) − h(t + Δ) for all Δ. For each t, we found the Δ that produced an average of zero distance for saccades that took place at that time. When all times were considered, the Δ clustered along a straight line centered on 196 msec. It appeared that saccades were an unbiased estimator of hand position at t + 196 msec.

Although the estimator was unbiased for this particular Δ, it was quite noisy. The saccades estimated hand position with an error of <5 mm with a probability of only ∼35%. The distribution of the error about its zero mean was broad throughout the movement period (Fig.4B). In preliminary experiments, we found that the projection of a stationary random-dot pattern with a modest density (35 dots/in²) on a plane immediately above the hand during the reaching movement was useful because it facilitated generation of saccades during the reach. It is possible that the noise in estimation of hand position was because the saccades moved to a dot that was closest to the estimate of hand position. However, the SD of the error was more than twice as high as would be expected if errors were solely a result of dot density. When we comparede(t) with h(t + 196 msec) for all saccades, we found a linear relationship with a slope of 1. Saccades predicted >85% of the variance in the hand data.

For the eyes to be able to estimate future hand position in real time as a reaching movement takes place, one theory suggests that the brain may use an internal model that relies on sensory feedback and efferent copy (Scarchilli and Vercher, 1999). This is a particular example of the forward-model theory (Jordan and Rumelhart, 1992; Miall et al., 1993), in which an internal model of the dynamics of the arm (mathematically represented by a differential equation that relates force input to acceleration output) is integrated from an initial condition specified by the delayed proprioceptive feedback (or visual feedback, if available) with a forcing function specified by the efferent copy (Bhushan and Shadmehr, 1999). The importance of the theory is that it provides the means by which a controller can effectively respond to perturbations despite the long delays in sensorimotor pathways. A strong prediction of the theory is that if a brief perturbation displaces the arm at unpredictable instances, the state estimator should still be able to predict the future state of the limb by combining the delayed sensory feedback with efferent copy.

In randomly selected trials and at random times during the selected trials, we applied a 50 msec force pulse to the hand. The direction of the pulse was always perpendicular to the direction of the target, but it could be either clockwise or counterclockwise (randomly selected). Because of the anisotropy of the inertia and stiffness of the arm, the pulses produced a very large range of hand displacements. The range of perpendicular displacements was ∼140% of the distance traveled in the unperturbed movement, and the distribution within this range was fairly uniform (Fig. 8C). Therefore, the perturbations produced a large variance in the trajectory of the hand.

In response to the perturbation, we observed saccadic inhibition at a latency of ∼100 msec. Choi and Guitton (2002) recently observed that when the head was perturbed during free eye–head movements toward remembered targets, ongoing saccades were interrupted and, after a pause period, a new saccade was generated. Fixation neurons in the rostral pole of the superior colliculus were activated in response to the head perturbation, terminating the saccade. In the current task, it appears that during the postperturbation period, a saccade gating mechanism sensitive to proprioceptive error from the arm caused previously planned saccades to be aborted.

At ∼150–200 msec after perturbation onset, the probability of saccades increased from its low by approximately eightfold (Fig.8A). The error caused the ongoing or planned saccades to be inhibited and a new saccade to be computed. The computation did not result in a saccade to the position where the hand was when it was perturbed. Rather, the resulting saccade was to where the hand would go in the near future. The analysis of the relationship between the postperturbation saccades and hand positions in the same movement found a remarkable correlation between saccade end point and hand position at saccade time + 150 msec. Thus, the postperturbation depression is interpreted as the point at which the proprioceptive information regarding the perturbation is integrated into the state estimator. The time between the perturbation onset and saccade generation ∼170 msec later is a period during which we speculate that the behavior of the eyes is most influenced by a forward model of the arm. Because the saccades that responded to the perturbation predicted hand state 150 msec after the saccade, the brain appears to compute the future state of the hand for a time interval that is nearly equal to the delays in sensorimotor pathways.

In summary, we found that saccades were unbiased, real-time estimators of the future position of the hand. Immediately after a brief perturbation, the brain appears to be able to predict where the limb will be in the near future. It seems plausible that the long-latency motor commands that are computed in response to an error take into account such an estimator. As subjects learn to control their arms in novel dynamics, these estimators are thought to adapt (Shadmehr and Brashers-Krug, 1997). It remains to be seen whether adaptation of the internal model for the arm results in changes in eye movements.