Reconstruction of Target Speed for the Guidance of Pursuit Eye Movements

Abstract

We studied how object speed is reconstructed from the responses of motion-selective cells for the generation of a behavior that is tightly linked to the speed of visual motion. In theory, the speed of an object could be estimated either from the speed tuning of the active population of motion-selective cells or from the rate of displacement of activation across the cortical map of visual space. We measured the pursuit eye movements evoked by stimuli containing two conflicting motion components: a local component designed to excite motion-selective cells with a particular speed tuning and a displacement component designed to excite cells with a sequence of spatial receptive fields. Pursuit eye movements were driven primarily by the local-motion component and were affected to only a small degree by the rate of target displacement across visual space. Extracellular single-unit recordings using the same stimuli revealed that the responses of cells in the middle temporal visual area (MT) depended primarily on the local-motion component but were influenced by the displacement component to the same degree as were pursuit eye movements. We conclude that the initiation of pursuit is consistent with a reconstruction of target speed based on the speed tuning of the active population of MT cells.

Visual motion is critical for the guidance of movement. For example, smooth pursuit eye movements respond to the motion of visual targets (Rashbass, 1961). Pursuit is guided by estimates of both the direction and speed of the object to be tracked. In monkeys, the middle temporal visual area (MT) is necessary for the normal initiation of smooth pursuit eye movements, and microstimulation of MT drives pursuit (Newsome et al., 1985; Komatsu and Wurtz, 1989; Born et al., 2000). Neurons in MT are excited by moving targets (Dubner and Zeki, 1971) and are tuned for both target direction and target speed (Maunsell and Van Essen, 1983). Thus, the question of how the nervous system estimates or “reconstructs” target motion from the responses of neurons in MT may be addressed by measuring the smooth eye movements guided by that reconstruction.

In principle, there are two different approaches that might be used by the nervous system to reconstruct target speed. The first approach would rely on the speed tuning of MT neurons. When a target moves at a constant speed, MT neurons with preferred speeds near the target speed will be the most active. Any of a variety of neural computations could be used to reconstruct target speed by estimating the preferred speed of the most active neurons. The second approach would measure the rate of displacement of a target across adjacent receptive fields of a sequence of MT neurons. This approach would not be sensitive to the speed tuning of the active neurons, but only to their receptive field locations. There is a precedent for displacement computations, as they must be used at some level of the nervous system to create direction-selective neurons. In primates, displacement computations create direction-selective neurons in the primary visual cortex (V1) from the non-direction-selective neurons in the lateral geniculate nucleus (LGN) (Saul and Humphrey, 1992). A displacement computation could solve the problem that the speed tuning of MT neurons is not constant, but varies as a function of target features such as contrast and spatial frequency (Movshon et al., 1985; Cassanello et al., 2000). The variance of speed tuning would adversely impact the accuracy of a reconstruction of target speed based on the speed tuning of MT cells but would not affect a reconstruction based on a displacement computation.

The goal of the present study was to test whether target speed is reconstructed from the firing of MT neurons by a displacement computation or a speed-tuning computation. We contrived stimuli that would cause different estimates of target speed, depending on which computation is actually used. Stimuli provided two components of motion. The first component was local, and was intended to excite MT neurons with preferred speeds near the speed of the local motion. The second component consisted of a displacement of the local motion across the visual field, at a different speed. We evaluated the neural estimate of target speed by measuring the initiation of smooth pursuit. Eye acceleration during pursuit initiation is closely related to target speed (Lisberger and Westbrook, 1985). By measuring eye acceleration, we were therefore able to assess the estimate of speed used by pursuit. Our data indicate that pursuit is driven by a reconstruction of target speed based on the speed tuning of the active neurons in MT and not by a displacement computation based on the spatial location of the activity.

MATERIALS AND METHODS

Pursuit experiments. Pursuit experiments were run on three male rhesus monkeys (Macaca mulatta) that had been trained to pursue spot targets. The experimental and training protocol has been described previously (Lisberger and Westbrook, 1985). Eye movements were measured with the scleral search coil method (Judge et al., 1980), using eye coils that had been implanted with a sterile procedure while the animal was anesthetized with isoflurane. In a separate surgery, stainless steel plates were secured to the skull and attached with dental acrylic to a cylindrical receptacle that could be used for head restraint. During experiments, the head was immobilized by attaching a post to both the receptacle and the ceiling of a specially designed primate chair. Eye velocity was obtained by analog differentiation of the eye position outputs from the search coil electronics (DC-25 Hz, −20 dB/decade). During experiments, animals were rewarded with juice or water for accurate tracking. Experiments were run daily and typically lasted 2 hr.

Single-unit recording experiments. Single-unit recordings were made in two anesthetized, paralyzed macaque monkeys (Macaca fascicularis). After the induction of anesthesia with ketamine (5–15 mg/kg) and midazolam (0.7 mg/kg), cannulae were inserted into the saphenous vein and the trachea. The head of the animal was then fixed in a stereotaxic frame and the surgery was continued under an anesthetic combination of isoflurane (2%) and oxygen. A small craniotomy was performed directly above the superior temporal sulcus (STS) and the underlying dura was reflected. The animal was maintained under anesthesia using an intravenous opiate, sufentanil citrate (8–16 μg/kg/hr) for the duration of the experiment. To minimize drift in eye position, paralysis was maintained with an infusion of vecuronium bromide (Norcuron, 0.1 mg/kg/hr) for the duration of the experiment and the animals were artificially ventilated with medical-grade air. Body temperature was kept at 37°C with a thermostatically controlled heating pad. The electrocardiogram, electroencephalogram, autonomic signs, and rectal temperature were continuously monitored to ensure the anesthetic and physiological state of the animal. The pupils were dilated using topical atropine and the corneas were protected with +2D gas-permeable hard contact lenses (Copper Vision, Inc., Scottsville, NY). Supplementary lenses were selected by direct ophthalmoscopy to make the lens conjugate with the display. The locations of the foveae were recorded using a reversible ophthalmoscope.

Tungsten-in-glass electrodes were introduced by a hydraulic microdrive into the anterior bank of the STS and were driven down through the cortex and across the lumen of the STS into area MT. Location of unit recordings in MT was confirmed by histological examination of the brain after the experiment, using methods described previously (Lisberger and Movshon, 1999). After the electrode was in place, agarose was placed over the craniotomy to protect the surface of the cortex and reduce pulsations. Single units were isolated and recorded for subsequent analysis. The responses included here are from five electrode penetrations at different sites in two monkeys.

All methods for both awake and anesthetized monkeys had received prior approval by the Institutional Animal Care and Use Committee at University of California San Francisco and were in compliance with the regulations of the Committee.

Stimulus presentation. Visual stimuli were presented on an analog oscilloscope (models 1304A and 1321B, P4 phosphor; Hewlett-Packard, Palo Alto, CA) using signals provided by digital-to-analog converter outputs from a PC-based digital signal-processing board (“Detroit” system; Spectrum Signal Processing, Vancouver, Canada). This method affords extremely high spatial and temporal resolution, with a frame refresh rate of 500 or 250 Hz and a spatial resolution of 64K × 64K pixels. The apparent motion created by our display is effectively smooth at these sampling rates (Mikami et al., 1986; Churchland and Lisberger, 2000). The display was positioned 30 cm from the animal and subtended 48.4° horizontally by 38.6° vertically. Experiments were performed in a dimly lit room. Because of the dark screen of the display, background luminance was beneath the threshold of the photometer (<1 mcd/m²). The same display technology was used for the pursuit and unit recording experiments.

Spot targets were round and were <0.25°. The spot targets were used both as fixation points and as tracking targets and had net luminances of 1.6 and 25 cd/m², respectively. Because spot targets were small, these luminances were bright but not dazzling. Motion of the target was achieved by flashing the spot in a new location every 2 or 4 msec. Each flash lasted ∼260 μsec.

Patch targets consisted of six dots randomly placed within a 3° × 3° virtual window that the monkey was required to follow. Each dot had a luminance of 1.6 cd/m². Patch targets were surrounded by a field of stationary random dots of the same density (1 dot per 1.5 deg²) and luminance as the patch target. The dots in the patch target and the borders of the virtual window always moved in the same direction, although sometimes at different speeds. As the patch target moved across the display, the dots in the background texture remained stationary but were displayed only when outside of the virtual window defined by the patch target. Thus, there was no luminance boundary to demarcate the patch target. Boundary conditions arose when a dot inside the patch moved beyond the limits of the window or when the limits of the window moved past a dot. When this occurred, a new dot was randomly placed within the bounds of the window. In addition to the constraint provided by the edges of the window, each single dot was allowed to move a maximum of 1° before it was extinguished and replaced with a new dot that was placed randomly in the patch window. At the beginning of a trial, each dot was randomly assigned an initial spatial lifetime between 0 and 1°, so that dots were replaced asynchronously. Because of boundary constraints and limits on the distance moved, a single dot was repositioned on average every 4 msec during these trials; the set of dots within a patch was cycled completely at least every 40 msec.

For the pursuit experiments, targets were presented in individual trials that began with the appearance of a fixation point. The monkey was required to fixate the point within 600 msec after its appearance and to maintain fixation within 2° for an additional 200–800 msec. The fixation spot was then extinguished and replaced with a tracking target that was either a spot or a patch, depending on the experiment. The tracking target appeared eccentric to fixation and immediately began to move toward the point of fixation (Rashbass, 1961). The duration of target motion varied from 270 to 1200 msec, depending on the speed of the target. Faster targets neared the edge of the monitor sooner and were extinguished earlier. For the very fast targets and short durations of motion used in some experiments, the target stopped and remained visible near the edge of the monitor, and the monkey was required to fixate the stationary target for 600 msec. This approach was designed to motivate the monkeys to track to the best of their abilities even for very brief target motions. If fixation requirements were met for the duration of the trial, a juice reward was delivered. Each pursuit experiment consisted of multiple repeats of a list of up to 50 types of trials; each trial type presented a different stimulus. The trials were sequenced by shuffling the list and requiring the monkey to complete each trial successfully once. Failed trials were placed at the end of the list and presented again after all the other trials had been completed. After all trials had been completed once, the list was shuffled and presented again.

For single-unit recording experiments, we initially mapped the receptive fields of the individual MT neurons by hand using bars on a tangent screen. The receptive fields of the cells included in this study were all within 10° of the fovea and were 4–10° of visual arc in diameter (mean, 6.1°; SD, 1.7°). After the receptive field location was determined, a mirror was positioned such that a random dot texture on the display oscilloscope fell within the receptive field of the cell. Textures were used to characterize the preferred direction and speed of the cell (Lisberger and Movshon, 1999). We subsequently studied each cell with a sequence of trials that provided motion of the same spot and patch targets that had been used to analyze pursuit. To render the stimuli identical with those used in the pursuit experiments, the spot trials began with the appearance and immediate motion of the target from its initial position. The patch trials began with the appearance of a stationary, uniform random dot texture that was visible for 256 msec before a patch target like those described above provided motion in either the preferred or null direction of the cell being recorded. Target movement continued for 256 msec or until the target reached the end of the display.

Data acquisition and analysis. Experiments were controlled by a computer program running on a Unix workstation. The workstation sent commands to a Pentium PC that both controlled the stimuli and acquired data. For the pursuit experiments, signals proportional to horizontal and vertical eye position and eye velocity were sampled at 1 kHz on each channel. For the unit recording experiments, a hardware discriminator was used to convert the extracellular action potentials to transistor–transistor logic pulses and the time of each pulse was recorded by the computer to the nearest 10 μsec. After each trial, data were sent via the local area network to the Unix workstation and saved for later analysis, along with a record of the commands given to generate the stimulus.

For pursuit, we aligned the responses to multiple repetitions of the same stimulus on the onset of target motion and computed the average eye velocity as a function of time, in 1 msec bins. We then estimated the time of the initiation of pursuit from the averages and defined our analysis interval to start at the initiation of pursuit and have a duration equal to one open-loop interval. The duration of the open-loop interval was estimated as the latency of the eye velocity response to a change in target velocity during sustained pursuit. Both the latency of pursuit and the open-loop interval varied slightly between monkeys and as a function of the form of the target, and the latency of pursuit also varied as a function of target direction. The latency of pursuit initiation was typically slightly longer (75–110 msec) than the duration of the open-loop interval (60–85 msec). For each trial type, we measured the change in average eye velocity during the analysis interval and computed average eye acceleration as the change in eye velocity divided by the duration of the open-loop interval. SEs were computed by measuring the eye acceleration on a trial-by-trial basis. We did not analyze the later, closed-loop, and maintenance periods of pursuit because the retinal stimulus driving pursuit differs from the presented target motion, making interpretation difficult. Trials with saccades during the open-loop interval after pursuit initiation were excluded from all analyses.

For the single-unit data, we aligned the responses to multiple repetitions of the same stimulus on the onset of target motion and computed the average firing rate as a function of time, in 16 msec bins. The number of repetitions of each trial ranged from 12 to 56 and averaged 18.8. We then measured firing rate in the interval from 80 to 176 msec after the onset of stimulus motion, an interval chosen because it approximates the period during which MT responses drive eye acceleration at the initiation of pursuit. The response latency to motion at 8°/sec was measured for all cells in the sample population of neurons and ranged from 58 to 102 msec (mean = 78 msec). To quantify the speed tuning of each MT neuron, we presented textures that were stationary for 256 msec before starting to move at constant speeds of 0.125, 0.25, 0.5, 1, 2, 4, 8, 16, 32, 64, and 128°/sec. We computed the average firing rate in the analysis interval for each speed, plotted average firing rate as a function of speed, and fit the data with the following function:

$$G(s)=R_{max}\left(e^{-{\left[\frac{log\left(\frac{s}{{\mu }_{\text{s}}}\right)}{{\varsigma }_{\text{s}}+\zeta log\left(\frac{s}{{\mu }_{\text{s}}}\right)}\right]}^2}-e^{-\left(\frac{1}{{\zeta }^2}\right)}\right),$$ Equation 1

where R_max is the maximal firing rate, μ_s is the optimal speed, s is the speed of the stimulus,ς_s is the tuning width, andζ is the skew of the cell, after the background firing rate has been subtracted. The quality of the fits was excellent. For the 20 MT neurons in our sample, the fitted parameters yielded a mean χ² of 4.98 + 4.06, where there were 6 degrees of freedom. To allow comparison across neurons, the response of each neuron to each stimulus was normalized by the value ofR_max from equation 1.

RESULTS

The basis for our experimental design is illustrated in Figure 1. In this figure, each graph places a population of MT neurons on two axes: the horizontal axis corresponds to the spatial locations of the receptive fields of the neurons and the vertical axis corresponds to their preferred speeds. Stimulus motion excites cells with appropriate preferred speeds and with receptive fields at the location of the target. In principle, displacement computations could estimate target speed according to how quickly the activity peak is displaced along the horizontal axis, represented by the filled arrows along the top of each graph. Speed-tuning computations based on the preferred speeds of the active population of neurons could estimate target speed by measuring the location of the peak of the activity along the vertical axis, represented by the open arrows along the right of each graph. If the stimulus is conceptualized in this way, then each target motion has two components: one related to local motion and one related to the rate of displacement of the motion. We will refer to the two stimulus components as “local-motion” and “displacement” components.

Fig. 1.

A schematic representation of the population of neurons in area MT, showing how speed could be reconstructed either from the speed tuning of the active neurons or from the rate of displacement of the active site across the map of visual space. In each panel, the activation of MT cells is indicated by the shading; the darkest cells have the greatest activity. The length of the filled arrow above each graph indicates the reconstruction of speed based on a displacement computation. The length of the open arrow on the right of each graph indicates the reconstruction of speed based on the speed tuning of the active population of cells. A, The local-motion and displacement signals are in agreement, yielding equivalent reconstructions from displacement and speed-tuning computations. B, The local-motion signal is fast while the target is displaced slowly across the visual field.C, The local-motion signal is slow while the target is displaced rapidly across the visual field.

Figure 1A presents the usual situation, in which computations based on either the local motion or the rate of displacement would yield the same estimate of target speed. In Figure1B, the stimulus contains fast local motion but is displaced slowly across visual space. In Figure 1C, the stimulus contains slow local motion but is displaced quickly across visual space. We created the latter two situations in the first three experiments, by contriving stimuli that contained conflicting displacement and local-motion components. As we will show below, the result of each experiment is consistent with the idea that pursuit is driven by the local-motion component of the stimuli. In the fourth experiment, we used the same target motions as visual stimuli while recording from cells in area MT. This allowed us to be sure that the speed tuning of the active population of MT neurons was determined primarily by the local-motion component of our stimuli.

Experiment 1: The gaps experiment

Gap targets achieved the dissociation between the speed of local motion and the rate of displacement by using alternate periods in which the target was visible and invisible. For example, the top trace in Figure 2A shows the velocity profile of a spot target that started with a visible period (solid trace) in which it moved at 10°/sec for 16 msec. Target motion was sampled at 4 msec intervals, so that each visible period delivered five flashes of the target. During the subsequent gap period (dashed part of the trace), the target was invisible for 16 msec. At the end of the gap period, the target reappeared at a new position as if it had moved at 20°/sec during the gap, a displacement of 0.32°. After three cycles of visible and gap periods, the target reappeared and moved uninterrupted at 15°/sec so that the monkey could establish accurate tracking of an unambiguous target motion. We refer to the target motion in Figure2A as the “10-visible condition.” Its companion, in which the first, visible motion was at 20°/sec and gap motion was at 10°/sec, is termed the “20-visible” condition (not illustrated). Both targets were displaced at a rate of 15°/sec, but during their visible periods should have excited populations of cells tuned for different speeds. The 10-visible condition is expected to preferentially excite cells with preferred speeds near 10°/sec, whereas the 20-visible condition is expected to excite cells with preferred speeds near 20°/sec. The bottom traces in Figure2A show averages of eye velocity from one experiment to illustrate the general finding that the 10-visible and 20-visible conditions evoked different initial pursuit responses, even though the target was displaced at the same average rate in both conditions.

Fig. 2.

Pursuit responses in the gaps experiment.A, The top trace shows the velocity of the spot target in the 10-visible condition. Solid lines indicate when the target was visible and moving; dashed lines indicate when the target was not visible but was moving. Average eye velocity is shown in the bottom traces. The thin traces and thick traces show the pursuit responses to the 10-visible and the 20-visible conditions, respectively. The upward arrow indicates the end of the open-loop interval. For this and all figures, upward deflections indicate rightward movement. The scale bar on the right refers to both target and eye velocity. B, Bar graphs showing the average open-loop eye acceleration measured during the initiation of pursuit for one experiment in each of two monkeys. Error bars give the SEMs.

Figure 2B shows that the mean eye acceleration during the open-loop interval was lower in the 10-visible condition than in the 20-visible condition for both monkeys we tested. Each group of four bars shows the average results for a single experiment. These results are the first piece of evidence we present to suggest that eye acceleration at the initiation of pursuit is sensitive to the speed tuning of the population of active MT cells. Note, however, that the rate of displacement of the target was held constant in this experiment, so it is not possible to know whether a displacement computation also contributes to pursuit initiation.

We performed two control experiments to ensure that our results were not related to other features of the stimulus that differed between the 10-visible and 20-visible conditions. First, to control for any effects of the order of the speeds within the first versus second interval, we used a “20-reversed” stimulus in which visible motion was at 20°/sec but the gap interval was first and the visible interval was second. Second, to control for the fact that the two stimuli provided targets that moved different distances during the visible period, we used a “20-short” stimulus in which visible motion was at 20°/sec but the visible periods were only 8 msec in duration: gap period duration was 16 msec and velocity was 10°/sec, as before. Figure2B shows that both of these stimuli elicited eye accelerations that were consistent with the visible component of the stimulus, which provided target motion at 20°/sec. Note that the rate of stimulus displacement across the visual field was reduced to 13.3°/sec for the 20-short stimulus. If a pure displacement computation were used to extract speed information, then the 20-short stimulus should yield lower eye accelerations than any of the other stimulus conditions. However, the data show that initial eye acceleration was similar to that evoked by the 20-visible condition and higher than that evoked by the 10-visible condition.

Experiment 2: The jumps experiment

Jumps targets dissociated the speed of local motion from the rate of displacement by interrupting motion at one speed with sudden steps of target displacement. The test target (illustrated by thedashed target trace in Fig.3A) moved at 8°/sec for successive 16 msec intervals, but underwent 2° jumps in the direction of target motion between intervals, producing a net displacement rate of 133°/sec. After five or six intervals separated by jumps, the target ceased jumping and moved at a constant speed of either 8°/sec (shown) or 133°/sec with equal probability. This “2° -jumps” target was designed to excite MT cells with speed tunings near 8°/sec but to traverse visual space at a much faster rate. A jump size of 2° was selected to exceed the maximum spatial integration distance of MT neurons (Mikami et al., 1986) and therefore not to excite cells with fast preferred speeds, despite the rapid displacement of the stimulus. We confirm in a later section that the stimulus design was successful in creating this effect. Two control targets moved at either 8°/sec or 133°/sec (illustrated, respectively, by the thin and thick target traces in Fig. 3A). All target motion was sampled at 2 msec intervals, so that the 2°-jumps target was flashed nine times during each 16 msec interval of smooth motion.

Fig. 3.

Pursuit responses in the jumps experiment.A, The top trace shows target position for three conditions. The thin solid trace and thick solid trace represent control 8°/sec and 133°/sec targets. Thedashed trace represents the 2°-jumps target, which moved at 8°/sec, but jumped 2° in the direction of target movement every 16 msec. The bottom traces indicate the average eye velocity for the three different conditions. B, Bar graphs showing the average open-loop pursuit acceleration for one experiment in each of two monkeys. Error bars indicate the SEM.

The average eye velocity traces in Figure 3A show that the initial pursuit response to the 2°-jumps target (dashed trace) is similar to that evoked by the 8°/sec target (fine solid trace), and much smaller than that evoked by the 133°/sec target (bold solid trace). The bar graphs in Figure 3B show that mean eye acceleration in the open-loop interval for the 2°-jumps target (indicated by thebars marked 2-deg-Jumps) was slightly larger than that for the 8°/sec target but much smaller than that for the 133°/sec target. The 2°-jumps target was designed to contain two components: local motion at 8°/sec and a net rate of displacement of 133°/sec. The response to the 2°-jumps target was close to that for the control 8°/sec target and therefore was dominated by the speed of the local motion. However, the faster displacement component did have an impact. The response to the 2°-jumps target was larger than that for the 8°/sec target, and the difference was statistically significant for both monkeys shown in Figure 3 (p < 0.05).

We quantified the relative contributions of the local-motion and displacement components of the 2°-jumps stimulus using the following equation:

$$R_{\text{local}/\text{displacement}}=l\times (R_{\text{local}})+(1-l)\times (R_{\text{displacement}}),$$ Equation 2

where l is the proportion of the response governed by local motion, R_{local/displacement} is the measured smooth eye acceleration for the target with conflicting local and displacement speeds (the 2°-jumps target for these experiments), R_local is the measured smooth eye acceleration to the control target whose speed was the same as the local-motion component of the conflicting stimulus (the 8°/sec target), and R_displacement is the measured smooth eye acceleration to a control target whose speed was the same as the rate of displacement of the conflicting stimulus (the 133°/sec target). If the response to the 2°-jumps target were the same as the response to the 8°/sec or 133°/sec targets, thenl would be equal to 1 or 0, respectively. Smooth eye acceleration was measured as the average acceleration during the open-loop interval, as described in Materials and Methods. For rightward pursuit in monkeys Ka and Mo, l was 0.83 and 0.81, respectively, indicating that the majority of the response to the 2°-jumps stimulus can be accounted for as a response to the local-motion component of target motion. To ensure that these results generalized, we ran 8 additional jumps experiments, for a total of 10 experiments on three monkeys, including tests of both horizontal and vertical pursuit. Although pursuit accelerations differed dramatically among the four directions tested, the l value did not depend on whether pursuit was along the vertical or horizontal axis (Table1).

Table 1.

Summary of the jumps experiments

	8°/sec (4°)	8°/sec (1°)	2°-jumps (4°)	133°/sec (4°)	l
Monkey Mo (right, 1st)1-a	118° /sec2	109° /sec2	165° /sec2	367° /sec2	0.81
Monkey Ka (right)1-a	106° /sec2	110° /sec2	123° /sec2	245° /sec2	0.87
Monkey Qu (right)	69° /sec2	112° /sec2 1-b	116° /sec2	392° /sec2	0.85
Monkey Qu (left)	55° /sec2	109° /sec2 1-b	97° /sec2	322° /sec2	0.84
Monkey Qu (up)	29° /sec2	46° /sec2	67° /sec2	125° /sec2	0.61
Monkey Qu (down)	43° /sec2	60° /sec2	51° /sec2	184° /sec2	0.94
Monkey Mo (right, 2nd)	78° /sec2	98° /sec2	112° /sec2	460° /sec2	0.91
Monkey Mo (left)	78° /sec2	104° /sec2 1-b	101° /sec2	490° /sec2	0.94
Monkey Mo (up)	54° /sec2 1-b	72° /sec2	59° /sec2	136° /sec2	0.94
Monkey Mo (down)	61° /sec2 1-b	87° /sec2	67° /sec2	324° /sec2	0.97
Physiology (peak response)	6.4°/sec		8.7° /sec	22.8° /sec	0.86

From left to right, the columns show data for: the control 8°/sec target starting at 4° eccentric from fixation, the control 8°/sec target starting at 1°, the 2°-jumps target starting at 4°, the control 133°/sec target starting at 4°, and the value of l computed from Equation 2. The top 10 rows report eye acceleration during the initiation of pursuit for target motion in different directions, using different monkeys or recorded on different days. For some experiments using vertical directions, initial pursuit was weak, and it was necessary to increase the starting eccentricity of the targets to reduce the prevalence of early saccades. For these experiments, the eccentricity of all targets was increased by 1 or 2°. Relative eccentricities remained unchanged. The bottom row gives the target speed reconstructed for each target type from the population response recorded in area MT.

^F1-a Data shown in Figure 3.

^F1-b No significant difference between the responses to the indicated control 8°/sec target and the 2°-jumps target (paired t test).

All measurements were based on averages of at least 50 trials.

We conducted two controls for the 2°-jumps experiment. The first control was run for the experiments illustrated in Figure4, to determine whether the jumps could influence the initiation of pursuit if they were smaller. We measured the response to a “0.2°-jumps” target that was identical to the 2°-jumps target except that each jump was only 0.2°. Smaller jumps are expected to fall within the spatial integration ability of MT neurons and to excite cells with preferred speeds near the net speed created by the jumps. The bars labeled 0.2-deg-Jumps in Figure 3C indicate that initial pursuit acceleration was consistently larger for the 0.2°-jumps target than for the 8°/sec target (p < 0.05 for both monkeys) and larger even than for the 2°-jumps target (p < 0.05 for monkey Mo, not significant for monkey Ka).

Fig. 4.

Pursuit responses in the patch experiment.A, Eye and target position are shown for conditions in which the displacement and local-motion signals are in conflict. Thesolid traces correspond to eye position and thedashed lines indicate the position of the virtual window defining the patch target. The top traces demonstrate the condition during which the dots moved at 10°/sec, but the window moved at 30°/sec. The bottom traces correspond to the converse condition: dots, 30°/sec; window, 10°/sec.B, Average eye velocity for the four combinations of dot and window velocity, for the open-loop interval only. The solid traces and dashed traces plot responses to conditions in which dot and window motion were at the same or different speeds, respectively. Thick traces and thin traces indicate dot motion at 10°/sec or 30°/sec, respectively. The downward arrow indicates the initiation of pursuit. C, Bar graphs show the average open-loop pursuit eye acceleration for one experiment on each of two monkeys. Error bars indicate the SEM.

For the second control, we asked whether the slightly larger eye acceleration for the 2°-jumps target versus the control 8°/sec target occurs because of the fact that the two targets had different average eccentricities (0.74° and 3.74°, respectively) during the first 64 msec of target motion (approximately the open-loop interval). Less eccentric targets typically evoke larger eye accelerations (Lisberger and Westbrook, 1985), potentially accounting for the larger acceleration evoked by the 2°-jumps target. To test this hypothesis, we recorded pursuit as a function of the initial eccentricity of the 8°/sec target. Starting eccentricities of 4°, 1.5°, and 1° yielded average eccentricities of 3.74°, 1.24°, and 0.74° in the first 64 msec of target motion and had small and variable effects on eye acceleration in the open-loop interval. For the rightward pursuit of monkey Mo, average open-loop acceleration was 118 ± 4°/sec, 102 ± 3°/sec, and 109 ± 3°/sec, respectively. For the rightward pursuit of monkey Ka, average open-loop acceleration was 105 ± 3°/sec, 100 ± 3°/sec, and 110 ± 3°/sec. In comparison, the 2°-jumps target evoked an average eye acceleration of 164 ± 7°/sec for monkey Mo and 123 ± 6°/sec for monkey Ka. We conclude that the increase in initial eye acceleration produced by the 2°-jumps target is not simply a product of the change in average target eccentricity.

This control was also performed for the subsequent eight experiments using the jumps stimuli (shown in Table 1). For some of these experiments, the change in acceleration as a function of eccentricity was large enough to potentially account for the increase in eye acceleration produced by the 2°-jumps target (relative to the 8°/sec target). For these experiments, we cannot be sure whether the displacement component of the 2°-jumps target influenced pursuit or whether the changes in eye acceleration occurred because of the difference in average eccentricity. However, for many of the experiments, the response to the 8°/sec target was, for both eccentricities, smaller than that to the 2°-jumps target. Therefore, it does appear that the displacement component makes a small contribution to the initial pursuit response, although the values ofl we report may slightly underestimate the dominance of the local component.

Experiment 3: The patch experiment

Patch targets dissociated the speed of local motion from the rate of displacement by painting dots within a 3° × 3° window surrounded by a static random dot field and then contriving to have the dots within the window and the borders of the window move at different speeds (see Materials and Methods for details). Window and dot speed could each be either 10°/sec or 30°/sec, yielding four combinations, two of which put the window and dot speeds in conflict. Motion was sampled every 2 msec.

Figure 4, A and B, shows typical eye position responses to stimuli in which the dots moved slower or faster than the boundaries of the window. When dot speed was 10°/sec and the window was displaced at 30°/sec (top traces in Fig.4A), the smooth component of eye velocity was slower than the window displacement and the monkey made a staircase of rightward saccades to keep eye position (solid trace) close to window position (dashed trace), which was the requirement to receive a reward. When dot speed was 30°/sec and the window traversed visual space at 10°/sec (bottom traces in Fig.4A), smooth eye movement started briskly so that eye position led target position and a backwards saccade was required to fulfill the reward requirements.

Averages of eye velocity in the open-loop interval for the four stimulus conditions show that the initiation of pursuit depended primarily on the speed of dot motion and not on the rate of window displacement (Fig. 4B). As long as dot motion was at 10°/sec, the pursuit response depended little on whether the window was displaced at 10°/sec (bold solid trace) or 30°/sec (bold dashed trace). Similarly, as long as dot motion was at 30°/sec, pursuit depended little on whether the window was displaced at 30°/sec (fine solid trace) or 10°/sec (fine dashed trace). The bar graphs in Figure4C show means and SEs of the initial eye acceleration for all four conditions, revealing a consistent dependence on dot speed but not on window movement.

We again used equation 2 to estimate the contribution of the local-motion signal provided by dot speed to the signals driving pursuit. For monkey Na, the value of l was 0.99 and 0.82 when the dots moved slower or faster than the window. For monkey Mo, the value of l was 0.98 and 0.78 when the dots moved slower or faster than the window. Thus, pursuit responses were determined primarily by the local motion of the dots, but were weakly influenced by the rate of window displacement, especially when fast-moving dots were paired with slow displacement of the window. As in the jumps experiment, the effect of the displacement component was particularly large near the end of the open-loop interval. A total of eight patch experiments were run on three monkeys, including tests of both horizontal and vertical pursuit. As summarized in Table2, the pursuit responses were consistently dominated by the local motion component of motion of the dots. Again, the l value did not depend on whether pursuit was along the horizontal axis or in the upward direction. The value ofl was lower for fast dots and slow window displacement that for the converse situation in all but one experiment. This unexpected asymmetry may result from a weak disruption of pursuit gain in the unfamiliar situation in which the dot and window speeds do not match. For the three monkeys tested, the patch targets evoked little downward pursuit, and it was not possible to conduct the experiment for this direction.

Table 2.

Summary of the patch experiments

	Dots 10/window 10	Dots 10/window 30	Dots 30/window 10	Dots 30/window 30	l10/30	l30/10
Monkey Mo (right, 1st)2-a	52.1° /sec2 2-b	53.2° /sec2 2-b	125.1° /sec2	146.2° /sec2	0.98	0.78
Monkey Na (right)2-a	31.8° /sec2 2-b	32.1° /sec2 2-b	55.9° /sec2	61.4° /sec2	0.99	0.82
Monkey Qu (right)	48.8° /sec2	53.1° /sec2	66.5° /sec2	75.3° /sec2	0.84	0.67
Monkey Qu (left)	34.7° /sec2 2-b	34.5° /sec2 2-b	44.5° /sec2	49.8° /sec2	1.01	0.65
Monkey Qu (up)	13.5° /sec2 2-b	13.1° /sec2 2-b	18.5° /sec2	20.2° /sec2	1.06	0.75
Monkey Mo (right, 2nd)	64.9° /sec2	72.3° /sec2	110.0° /sec2	112.9° /sec2	0.85	0.94
Monkey Mo (left)	44.7° /sec2 2-b	44.9° /sec2 2-b	68.8° /sec2	70.7° /sec2	0.99	0.93
Monkey Mo (up)	31.4° /sec2 2-b	33.9° /sec2 2-b	47.6° /sec2	53.4° /sec2	0.89	0.74
Physiology (peak response)	6.4° /sec	7.6° /sec	16.1° /sec	17.8° /sec	0.90	0.85

The top eight rows report eye accelerations during the initiation of pursuit for target motion in different directions, using different monkeys, or on different days. The bottom row gives the target speed reconstructed for each target from the population response recorded in area MT.

^F2-a Data shown in Figure 4.

^F2-b The conditions dots 10/window 10 and dots 10/window 30 did not evoke statistically significant differences in pursuit acceleration. Note that for the two l values >1, there was no significant difference between the dots 10/window 10 and dots 10/window 30 conditions.

All measurements were based on at least 28 trials.

Experiment 4: Single-unit responses in area MT

Experiment 4 was designed as to determine whether neurons in area MT responded solely to the local-motion component of our stimuli, as we had assumed when we designed experiments 1–3, or whether the displacement component of motion influenced their responses. Single MT cells were recorded in anesthetized monkeys. After the preferred direction and speed of a neuron were determined using moving random dot textures, we recorded responses to the target motions used in the patch and jumps experiments. For each cell, the stimulus was shown moving in both the preferred and null direction. The speed of the stimulus was not customized for each cell, as we wished to know how neurons with different preferred speeds responded to the stimuli we had used to measure pursuit.

Figure 5 shows the responses of two neurons when presented with the stimuli used in the patch experiment. For the neuron that responded to fast speeds (preferred speed = 33°/sec), a brisk response was elicited when dot speed was 30°/sec, regardless of whether the speed of window displacement was 10 or 30°/sec. For a neuron that responded to slower speeds (preferred speed = 10°/sec), a strong response was elicited when dot speed was 10°/sec, regardless of whether the speed of the window was 10 or 30°/sec. For both example neurons, the amplitude of the responses was determined primarily by the speed of dot motion. The time course of the response was shorter when the window moved at 30°/sec, presumably because the patch exited the spatial confines of the receptive field more quickly than when the window moved at 10°/sec. The variation in time course is expected to have minimal impact on our analysis, which considered the firing rate only in the interval from 80 to 172 msec after the beginning of stimulus movement, a period analogous to the open-loop interval in the pursuit experiments.

Fig. 5.

Representative single-unit responses to the four combinations of dot and window velocity in the patch experiment. Thetop row provides schematic drawings of the stimulus; thefilled dots and arrows indicate the speed of dot motion and the open arrows indicate the speed of window motion. The middle row and bottom row show the responses of two MT neurons to the four stimuli. The neuron in the middle row had a preferred speed of 33°/sec. The neuron in the bottom row had a preferred speed of 10°/sec. Each histogram shows the firing rate of the neuron in response to the stimulus shown above the histogram. Bin width was 16 msec. The bars underneath each histogram indicate the interval of stimulus motion.

To summarize these data, for each target we first computed the directional component of the firing rate of each neuron. The directional component of the firing rate is defined as the response to motion in the preferred direction minus the response to motion in the null direction. We then normalized the firing rate for each target by the maximal response of the same neuron in the speed-tuning experiments, grouped the neurons according to their preferred speed into bins that were 1 octave wide, and computed the mean and SD of the response, in each bin, for each target. The general trend in Figure6A shows that neurons with preferred speeds in the 4 and 8°/sec bins responded best when the dot speed was 10°/sec (yellow and red bars); neurons with preferred speeds in the 32 and 64°/sec bins responded best when the dot speed was 30°/sec (green and blue bars). Neurons with preferred speeds of 16°/sec gave the same response for all four stimuli. In general, neurons responded strongly only when the local motion provided by dot speed was near their preferred speed. In addition, the window speed did have a small effect. For example, for dot motion at 10°/sec, neurons with preferred speeds in the 4 and 8°/sec bins responded better when the window speed was 10°/sec (yellow bars) than when it was 30°/sec (red bars). Because responses to motion in the null direction were almost always small, the same basic trends appeared when the analysis was based solely on the response to the preferred direction (data not shown). These results validate our assumption that the preferred speeds of the active population are determined primarily by the local motion of the dots themselves.

Fig. 6.

The response of the population of MT neurons to the stimuli used to record pursuit eye movements. Responses to patch and jumps stimuli are summarized in the left andright columns. A, B, Cells were pooled into five groups based on their preferred speed. Each bar graph shows the average normalized response of MT neurons to each stimulus minus the response to the null direction, as a function of preferred speed.C, D, The population response plotted as a function of preferred speed. A, C, Patch targets. The color coding for both bars and curves is as follows:yellow, dots 10/windows 10;orange, dots 10/windows 30; green, dots 30/windows 10; blue, dots 30/windows 30. B, D, Jumps targets. The color coding for both barsand curves is as follows: yellow, control 8°/sec target motion; red, 2°-jumps;blue, control 133°/sec target motion.

The targets used in the jumps experiment also evoked MT responses that were driven primarily by the local-motion component (Fig.6B). Both the control 8°/sec target (yellow bars) and the 2°-jumps target (red bars) evoked responses that were larger for neurons with slower preferred speeds. The 133°/sec target (blue bars) evoked the largest response in neurons with preferred speeds in the 32 and 64°/sec bins. The same basic trends seen in Figure6B, which plots the difference between responses to the preferred and null directions, are seen in the responses to the preferred direction (data not shown). Thus, MT neurons respond mainly to the 8°/sec local motion in the 2°-jumps target, as we had assumed in interpreting the jumps experiment.

Quantitative comparison of population responses in MT and pursuit behavior

Although MT neurons responded primarily to the local-motion component of our stimuli, the displacement component also had an effect. We quantitatively compared the relative influences of the two components on the MT population response. This was done by reconstructing target speed from the responses of our sample population of neurons for each stimulus condition. We then computed thel value from these reconstructions of target speed to measure the relative effect of the local and displacement components of motion on the reconstruction of target speed.

We normalized the speed-tuning curve for each neuron (equation 1) to have a peak response of 1, weighted each normalized curve by the response of the neuron to the stimulus, summed these curves over all MT neurons in our sample, and normalized for the sum of the responses using the following equation:

$$P(s)=\frac{\sum _{\text{i}} R_{\text{i}}\times G_{\text{i}}}{\sum _{\text{i}} R_{\text{i}}},$$ Equation 3

where P(s) is the population response for stimulus speed s, R_i is the normalized directional response of the ith MT neuron to stimulus s, G_i is the speed-tuning curve of cell i, and the sum is taken over all 20 MT neurons we recorded. This approach uses the speed-tuning curve of each neuron as a filter to smooth the population code, compensating for our relatively sparse sampling of the population.

Figure 6C shows the population response obtained for each of the four target motions used in the patch experiment. Each curve plots the normalized activation of the population as a function of the preferred speed of the neurons. The curves form two pairs. The two curves with peaks at lower preferred speeds were obtained from responses to the “dots 10/window 10” target (yellow trace) and the “dots 10/window 30” target (red trace). The two curves with peaks at higher preferred speeds were obtained from responses to the “dots 30/window 10” target (green trace) and the “dots 30/window 30” target (blue trace). In addition, there is a small effect of window speed: the curves for a window speed of 30°/sec (red, blue) lie slightly to the right of those for a window speed of 10°/sec (yellow, green). Like pursuit, the MT population response is dominated by the local component but is influenced by the displacement component.

For the jumps experiment (Fig. 6D), the population responses for the 8°/sec target (yellow) and 2°-jumps target (red) are similar. They both peak at lower preferred speeds than the population response for the 133°/sec target (blue curve), and the curve for the 2°-jumps target (red) has a slightly higher peak than that for the 8°/sec target (yellow curve). Because of our incomplete sampling of MT neurons, including few neurons with preferred speeds of >30°/sec, the population response to the 133°/sec target peaks at a much lower preferred speed, just under 23°/sec. However, it is the relative locations of the peaks that are important. Faster target speeds lead to larger estimates of speed, even if the estimates are not exact. As with the patch targets, the population response to the jumps targets is influenced by the local and displacement components of motion in the same way as pursuit.

To reconstruct target speed from the population responses and compare it with the pursuit responses, we measured the preferred speed of the neurons at the peak of the population response. For the patch experiment (leftmost four bars in Fig.7), the primary determinant of the reconstructed target speed was the speed of the local motion provided by the dots, although the reconstruction was biased slightly toward the speed of the window. The effect of both the dot and window speed was statistically significant, as evidenced by a jackknife technique (Sokal and Rohlf, 1995) that was used to compute error bars for each stimulus condition and by pairwise t tests that were used to determine the significance of the differences between the reconstructions. Application of equation 1 to the reconstructions from the unit recordings revealed that the l values for the reconstruction of speed from MT neurons were 0.90 and 0.85 for the dots 10/window 30 and dots 30/window 10 targets, comparable with those for pursuit (mean values of 0.95 and 0.79, respectively).

Fig. 7.

Reconstructions of speed based on the population responses from cells in area MT. The left andright panels plot reconstructions for the patch and jumps experiment, respectively. Error bars indicate the SEM reconstructed speed.

For the jumps experiment (rightmost three bars in Fig. 7), the reconstructed target speed was slightly higher for the 2°-jumps target than for the 8°/sec target motion and was much higher for the 133°/sec target motion. All of the differences were statistically significant. For the 2°-jumps target, the l value for the reconstruction was 0.86, indicating that the reconstruction of speed from MT neurons was determined primarily by the speed of the local motion but was influenced slightly by the rate of the target displacement. For pursuit, the mean l value was similar: 0.87.

DISCUSSION

The goal of our experiments was to determine how the pursuit system reconstructs an estimate of target speed. We contrived targets that placed into conflict the speed of local motion and the overall rate of displacement of the stimulus. Our behavioral experiments show that initial pursuit eye acceleration is determined primarily by the speed of local motion and argue that the reconstruction of target speed is based primarily on the speed tuning of MT cells. The rate of displacement of the stimulus did have a small effect on the initiation of pursuit, suggesting that a displacement-based computation might also contribute to the reconstruction of target speed. However, the rate of displacement also had a small effect on the responses of MT neurons, so that the speed-tuning reconstruction was sufficient to account for the behavioral data. It is therefore unnecessary to suppose that a displacement-based computation contributes anything to the estimate of target speed used during pursuit initiation. We conclude that the estimate of target speed driving eye acceleration during the initiation of pursuit is derived purely from a speed-tuning-based estimate of target speed.

Our experiments raise four technical questions that we will consider now: (1) Why did the displacement component of motion in our stimuli affect the peak of the active population of MT neurons at all? For the jumps experiment, we chose to elevate the rate of target displacement from 8°/sec to 133°/sec by the addition of 2° jumps because such large jumps should not support direction-selective responses in the majority of MT cells (Mikami et al., 1986). Some cells, particularly those with a combination of selectivity for low spatial frequencies and high speeds, may have sufficient spatial integration to respond directionally to the 2° jumps. Alternately, the response to local motion may facilitate a response to the 2° target displacements. For the patch experiment, the displacement of the window fails to provide any moving luminance borders and is an example of “second-order motion.” Because second-order motion evokes a response from some MT neurons (Albright, 1992; O'Keefe and Movshon, 1998), it is not surprising that window displacement did have a small effect on both the response of MT neurons and the initiation of pursuit.

(2) Would our results have been different if we had used a different computational approach to reconstruct target speed? For simplicity, we took the speed at the peak of the population curve as our estimate of the target speed. This corresponds to a category of approaches that falls under the rubric of “winner-take-all.” An alternative approach involves estimating the center of mass of the population response, commonly termed “vector averaging.” Inspection of the population responses in Figure 6, C and D, makes it clear that population responses were unimodal and well behaved, and that we would have obtained the same results from almost any sensible method. Note that the simpler method of taking the average firing rate over all MT neurons would not have worked. It fails even on control target motions: the output of such a model will actually be lower for a target speed of 133°/sec than for 8°/sec. Finally, although we based our estimates of target speed on the directional component of MT neuron responses, calculated by taking the difference between firing rate for motion in the preferred and null directions, we obtained the same general results when we repeated the computations based on responses for motion in the preferred direction only.

(3) Were our results altered by smoothing the population responses using the speed-tuning curves as filters? In fact, results were very similar when we computed the population response via a vector average that weighted the normalized response of each neuron according to its preferred speed (data not shown). However, this approach would not have allowed the clean graphical presentation in Figure 6, C andD.

(4) Would our estimate of the value of l, the relative contribution of local-motion signals to the response of MT cells, differ if we had a larger sample of MT neurons? The distribution of preferred speeds that we sampled resembles that found by other researchers (Mikami et al., 1986). Therefore, we do not believe that a skewed sampling of preferred speeds has influenced our estimate of the value of l. Although the reconstructions of target speed from this population of MT neurons did not yield quantitatively accurate estimates of actual target speed (Fig. 7), the estimates did increase monotonically with the target speed. Because the lvalue is computed from the relative locations of the peaks, it would be influenced minimally by systematic inaccuracies in the absolute estimate of speed.

Our results provide a major constraint on how the responses of the population of MT neurons are pooled to drive smooth pursuit eye movements: the estimate of speed used by pursuit is extracted by a computation based on the speed tuning of the active neurons. A number of different neural computations could be used, all of which can be termed “labeled-line computations” because they rely on knowing both the firing rate and the preferred speed or speed tuning of a neuron (Salinas and Abbott, 1994). Labeled-line computations provide reliable estimates of stimulus parameters only if the tunings of a neuron for that parameter remain fixed independently of other stimulus parameters. Consistent labeled-line estimates could be made for orientation and direction of motion, because tuning may broaden or narrow, strengthen or weaken, but the location of the peak is invariant with stimulus form or contrast (Sclar and Freeman, 1982; Jones and Palmer, 1987; Albright, 1992). However, the preferred speed of most of the neurons in V1, MT, and V2 depends on the spatial frequency content of the stimulus (Movshon et al., 1985, 1988; Cassanello et al., 2000). If a labeled-line computation based on speed tuning is used, then the estimate of speed may vary as a function of spatial frequency.

It is unknown whether the initiation of pursuit varies as a function of the spatial frequency of the visual stimulus, although ocular following is known to do so (Miles et al., 1986). Psychophysical experiments have demonstrated that changes in both contrast and spatial frequency consistently affect the perception of speed (Diener et al., 1976;Campbell and Maffei, 1981; Thompson, 1983; McKee et al., 1986; Stone, 1992). However, other approaches imply that representations of speed that are invariant with spatial frequency do exist in the brain (Schrater and Simoncelli, 1998; Reisbeck and Gegenfurtner, 1999). It is unclear whether these representations are based on a subset of MT neurons that have invariant speed tunings or on responses in areas downstream of MT.

We stress that our results do not exclude the use of displacement-based algorithms earlier in the visual motion pathway. In the primary visual cortex, a displacement-based algorithm is used to convert the firing of LGN neurons into direction-selective responses (Saul and Humphrey, 1990, 1992). In addition, cells in area MT may use displacement-based algorithms as part of the mechanism that creates their responses from the activity of cells in V1. A computation that reads the displacement of activation across the cortical map of visual space in V1 would account for the observation that MT neurons retain directional responses even when apparent motion causes the majority of V1 neurons to lose direction selectivity (Mikami et al., 1986). Finally, displacement-based reconstructions of target speed from the firing of MT neurons may be used for some purposes, such as the detection of long-range apparent motion (Braddick, 1980), but do not drive eye acceleration at the initiation of smooth pursuit eye movements.