Abstract
Visual signals generated by self-motion are initially represented in retinal coordinates in the early parts of the visual system. Because this information can be used by an observer to navigate through the environment, it must be transformed into body or world coordinates at later stations of the visual-motor pathway. Neurons in the dorsal aspect of the medial superior temporal area (MSTd) are tuned to the focus of expansion (FOE) of the visual image. We performed experiments to determine whether focus tuning curves in area MSTd are represented in eye coordinates or in screen coordinates (which could be head, body, or world-centered in the head-fixed paradigm used). Because MSTd neurons adjust their FOE tuning curves during pursuit eye movements to compensate for changes in pursuit and translation speed that distort the visual image, the coordinate frame was determined while the eyes were stationary (fixed gaze or simulated pursuit conditions) and while the eyes were moving (real pursuit condition). We recorded extracellular responses from 80 MSTd neurons in two rhesus monkeys (Macaca mulatta). We found that the FOE tuning curves of the overwhelming majority of neurons were aligned in an eye-centered coordinate frame in each of the experimental conditions [fixed gaze: 77/80 (96%); real pursuit: 77/80 (96%); simulated pursuit 74/80 (93%); t-test, P < 0.05]. These results indicate that MSTd neurons represent heading in an eye-centered coordinate frame both when the eyes are stationary and when they are moving. We also found that area MSTd demonstrates significant eye position gain modulation of response fields much like its posterior parietal neighbors.
INTRODUCTION
Studies have found that MSTd neurons are sensitive to the location of the focus of expansion (FOE) in optic flow (Bradley et al. 1996; Duffy and Wurtz 1995). The FOE can be used to determine the direction of self-motion (Gibson 1950), suggesting that MSTd may play a role in navigation. Further support for a role of MSTd neurons in self-motion perception derives from their ability to shift their FOE tuning curves to compensate for visual perturbations caused by pursuit eye movements to represent the direction of heading as opposed to representing the retinal image (Bradley et al. 1996). In addition, MSTd neurons are able to compensate for different pursuit speeds (Shenoy et al. 2002), simulated translation speed during pursuit (Lee et al. 2007), and gaze rotation during vestibular ocular reflex cancellation (Shenoy et al. 1999).
To successfully navigate through the environment requires the retinal image signals derived from earlier visual areas to be converted into movement signals in higher level coordinate frames, such as head, body, world, and eventually motor coordinates. Neighboring areas within the posterior parietal cortex, such as the lateral intraparietal area (LIP) and the parietal reach region (PRR), have been shown to code space primarily in retinal coordinates (Andersen et al. 1993; Batista et al. 1999). Thus it is perhaps of little surprise that recent studies have indicated that MSTd also codes the FOE in retinal coordinates (Fetsch et al. 2004, 2005, 2007; Lee et al. 2005), although it is not known if this coding also applies during pursuit eye movements. Studies of the compensation of FOE tuning in MSTd during pursuit show that at least a portion of the pursuit compensation is derived from an efference copy of the eye movement command (Bradley et al. 1996; Lee et al. 2007; Shenoy et al. 1999, 2002). As a result, it is often assumed that the MSTd FOE tuning is transformed to head, body, or world reference frames during pursuit compensation (Fetsch et al. 2007; Royden et al. 1994; Zhang et al. 2004).
However, it has also been pointed out that pursuit compensation could occur in eye-centered coordinates, adjusting the focus tuning curves to indicate where the FOE would be on the retina if the eyes were stationary (Andersen and Buneo 2002). Such a finding would be consistent with neighboring cortical areas LIP and PRR in which the locations of saccade or reach targets compensate for intervening saccades in eye coordinates (Batista et al. 1999; Duhamel et al. 1992; Gnadt and Andersen 1988). In this study, we examine in what coordinate frame MSTd neurons code FOE locations during stationary fixation, pursuit, and simulated pursuit conditions.
METHODS
The methods described in this section have been previously reported (Lee et al. 2007; Shenoy et al. 1999, 2002). We will briefly review them here.
Animal preparation
All procedures and protocols were reviewed and approved by the Caltech Institutional Animal Care and Use Committee. Extracellular responses were recorded from 80 MSTd neurons from the right hemisphere in two rhesus monkeys (Macaca mulatta). A methylmethacrylate head cap with a titanium head post was built on the skull of each monkey. In monkey DON, eye position was monitored using a scleral search coil at 1,000 Hz (Judge et al. 1980), whereas in monkey ROY, an optical eye tracker monitored eye position at 240 Hz (ISCAN, Burlington, MA). Because we did not perform further analysis of the recorded eye movement other than for the behavioral task, using two different methods to monitor eye movements did not present any problems. After the initial surgery, behavioral training began. Once performance levels reached >90%, a second surgery was performed to open a craniotomy and implant a surface-normal chronic recording chamber at 5 mm posterior, 17 mm lateral, on the right hemisphere in both DON and ROY.
Recording techniques
A stainless steel guide tube was lowered through the dura. A glass-coated tungsten microelectrode with an impedance between 1.0 and 1.5 MΩ (Alpha-Omega, Nazareth, Israel) was then advanced into the cortex via an FHC hydraulic microdrive (FHC, Bowdoinham, ME). MSTd neurons were identified by their location in the recording chamber, MRI anatomical images, position relative to other cortical areas, and response properties such as optic flow tuning. Neural data were recorded at a 20 kHz sampling rate and analyzed with custom software.
Visual stimuli
The recording environment was completely dark except for the visual stimuli. The stimulus was shown on a flat screen CRT monitor placed 38.1 cm from the eyes of the monkey and set to a resolution of 800 × 600 and 120 Hz frame rate with a visual angle of 56 × 44°. Fixation points were 5 × 5 pixels in size while stimulus dots were 2.5 × 2.5 pixels. The stimulus consisted of 400 white dots (10 candela/m2) in a 20 × 20° stimulus window on a black background. Each dot had an initial age between 0 and 287 ms and remained visible until 300 ms elapsed or it crossed the border of the stimulus window, in which case it was reborn at a new random position. The velocity of the stimulus dots was proportional to the distance from the focus of expansion. The stimulus was viewed monocularly with the left eye. In the characterization tasks, this 20 × 20° stimulus window was centered on the (0°,0°) position of the monitor while in the fixed, real, and simulated coordinate frame tasks, the stimulus window was moved to coincide with one of five possible fixation points along the preferred axis of pursuit to maintain the same retinal image regardless of fixation target position.
Characterization tasks
Once a neuron that was responsive to an expansion optic flow stimulus was isolated, three sets of characterization tasks were run to determine the neuron's preferred spiral space pattern, preferred laminar motion, and preferred pursuit direction. Spiral space is constructed by rotating the motion vectors in an expansion stimulus by different counterclockwise angles spaced 45° apart: 0° for expansion, 45° for a counterclockwise-expanding spiral, 90° for counterclockwise rotation, 135° for a counterclockwise-contracting spiral, etc (Graziano et al. 1994). We next determined the laminar motion tuning of each neuron using a laminar motion stimulus consisting of random dots moving unidirectionally in one of eight directions, spaced at 45° apart. In the preferred pursuit task, the monkey pursued in eight directions spaced 45° apart that crossed the (0°,0°) position on the monitor.
Coordinate frame task
We ran experiments to determine if MSTd neurons were coding in eye coordinates versus screen coordinates. Screen coordinates refers to any downstream coordinate frame other than the one that is explicitly being tested. In this case, screen coordinates refers to any coordinate frame other than eye coordinates, i.e., head, body, or world. If the results indicated that neurons in MSTd were coding in eye coordinates, no additional tests would be necessary. However, if the results indicated screen coordinates, additional tasks would be required to determine the other coordinate frames.
Procedures
The first set of trials was performed to determine the coordinate frame with the eyes stationary, whereas the second set was performed to determine the coordinate frame during pursuit eye movements. A third set of trials was performed to test the coordinate frame of MSTd neurons during a simulated pursuit condition.
In all three conditions, fixed, real pursuit, and simulated pursuit, the focus tuning was determined at five fixation target positions from −12 to +12° in 6° increments along the axis of preferred pursuit. In other words the heading stimuli were displayed in a 20 × 20° window centered at (0°,0°) with respect to the eye. The stimulus was presented on the screen at one of the five fixation target positions so that the retinal image remained the same regardless of fixation target position. The stimuli moved with the fixation target position that was being tested and was not fixed to the center of the screen. The stimuli consisted of an expansion flow field with the foci located at 11 points in 6° steps (range ± 30°) along the previously determined preferred axis of pursuit. Only a small portion of the overall flow field was visible to the monkey inside the 20 × 20° window and the remainder of the field of view was dark. Although the FOE would sometimes be outside the stimulus window, the centrifugal dots from the expansion pattern were always visible inside this window (Fig. 1).
Fig. 1.
Coordinate frame task diagram. The monkey was required to fixate within a ± 2° window surrounding five possible fixation points (5 × 5 pixels) oriented along the preferred axis of pursuit separated by 6° steps (range: ±24°). In the fixed, real, and simulated coordinate frame tasks, this 20 × 20° stimulus window was moved to coincide with 1 of 5 possible fixation points along the preferred axis of pursuit to maintain the same retinal image regardless of fixation target position. The stimuli consisted of an expansion flow field with the foci located at 11 points in 6° steps (range: ±30°) along the previously determined preferred axis of pursuit. Although the focus of expansion (FOE) would sometimes be outside the stimulus window, the centrifugal dots from the expansion pattern were always visible inside this window (shown in figure). In the fixed gaze condition, the monkey fixated on 1 of the 5 fixation points. In the real pursuit condition, the monkey pursued a pursuit target across the stationary stimulus window. In the simulated pursuit condition, the retinal image was identical to the real pursuit condition by having the monkey fixate at a stationary fixation point and drifting the entire stimulus across the screen in the opposite direction and same speed as in the real pursuit condition. In both the real and simulated pursuit conditions, pursuit was performed along the horizontal plane and also along the diagonals; however, due to screen size limitations, pursuit was not performed on the vertical axis.
In the fixed gaze condition, the monkey fixated a stationary target. In the real pursuit condition, the pursuit target traveled in the direction of preferred pursuit across the stimulus window. In the simulated pursuit condition, the monkey fixated on a stationary target while the FOE and the window through which the flow pattern was seen drifted together at the same speed as the target in the real pursuit condition but in the opposite direction. In this manner, all aspects of the retinal image were identical in the real pursuit and simulated pursuit condition. Our previous studies showed both extra-retinal and visual cues contributed to compensation during pursuit. The last two tasks test for the coordinate frame when both cues are present during real pursuit and during simulated pursuit when only visual cues are present.
The first set of experiments examined the coordinate frame of MSTd expansion tuning curves when the eyes were stationary. The focus tuning was determined at five fixation target positions from −12 to +12° in 6° increments along the axis of preferred pursuit. The stimulus was presented in the same relative position to the eye for each fixation target position so that the retinal image remained the same across fixation target positions. We simulated the optic flow experienced by an observer who moves forward at a speed of 16 cm/s while viewing a random dot pattern distributed on a tangent screen at a distance of 38.1 cm. The monkey was required to obtain fixation on the target (± 2° window) within 800 ms and remain fixated for an additional 1,200 ms that the stimulus was presented. Only the last 1,000 ms of the trial was used for calculating the mean firing rate to avoid the phasic response caused by the onset of the stimuli. The total trial length was 2,000 ms. The tuning curves were plotted in screen and eye coordinates. If the neurons were coding in eye coordinates, then the focus tuning curves would best align in that coordinate frame. However, if the tuning curves aligned in screen coordinates, then the tuning curves were in a nonretinal coordinate frame such as head, body, or world coordinates.
The next set of experiments examined the coordinate frame of MSTd neurons during real and simulated eye pursuit. This experiment was similar to the one in the preceding text, but added pursuit and simulated pursuit conditions. In the real pursuit condition, the monkey pursued the pursuit target, which appeared at the beginning of each trial and moved across the visual stimuli at 8°/s along the preferred axis of pursuit for a total of 2,000 ms. Due to screen size constraints, the vertical pursuit directions (90 and 270°) were not used and the nearest 45° diagonal was substituted. The pursuit path was repositioned to produce the same retinal stimulus at the different eye positions. Again in this condition, the monkey was allowed 800 ms to acquire the target and was required to maintain pursuit for an additional 1,200 ms for a total trial time of 2,000 ms. At the midpoint of the 1,200 ms stimulus period, the monkey's eyes passed through the eye position selected for that trial. Therefore at the start of the trial at 0 ms, the pursuit target appeared and began to move at −11.2° with respect to the eye position; at 800 ms (end of acquisition period and beginning of the stimulus period) the eyes were at −4.8°; and at the end of the trial at 2,000 ms, the eyes were at +4.8°. Because we only used the last 1,000 ms of the trial period in our calculations, the data were analyzed when the eyes passed from −3.2 to +4.8° with respect to the selected eye position. The selected eye position on each trial corresponded to one of five eye positions from −12 to +12° in 6° increments along the axis of preferred pursuit (see preceding text).
Data analysis
The preferred spiral space pattern, preferred laminar motion direction, and preferred pursuit direction were calculated by computing the response weighted vector sum (Geesaman and Andersen 1996; Shenoy et al. 1999, 2002). Significance was determined using the Rayleigh test (Geesaman and Andersen 1996; Zar 1996).
Previous studies demonstrated that posterior parietal areas, including MSTd, do not vary their preferred directions with changes in eye position (Bremmer et al. 1997). This allowed us to compare the tuning curves at each fixation target position without having to worry about changes in preferred tuning, which could introduce bias into the results.
All analyses were performed with respect to the eye, which means the five fixation points were collapsed into a single point, and the focus tuning curves were compared with each other. If area MSTd was coding in eye coordinates, the focus tuning curves would move with the eye and when the fixation target positions were overlaid onto the same point, all the tuning curves would overlap each other closely and there would be little or no horizontal shift between them. However, if neurons in MSTd were coding in screen coordinates (or any other coordinate frame other than eye-centered), the focus tuning curves would stay at the same position on the screen regardless of fixation target position and, when the fixation target positions were overlaid onto the same point, all the tuning curves would be misaligned by multiples of 6°, which is the distance between the fixation target positions.
We calculated the shift between the focus tuning curves measured at fixation target position 1 (−12°) and fixation target position 5 (+12°) which are the most distant fixation target positions and span 24°. The fixation points were overlaid on top of each other and, if area MSTd is coding in eye coordinates, the measured shift between the two curves would be ∼0°. If MSTd neurons are coding in screen coordinates, the measured shift would be ∼24°.
We used cross-correlation to measure the amount of shift between the tuning curves. Cross-correlation works well for this type of well-sampled data because it indicates the horizontal shift between tuning curves and is insensitive to any vertical shifts or gain changes that may be present. To avoid calculating cross-correlations where there were not enough overlapping points, we constructed the analysis such that there were always six or more overlapping FOE locations. To detect small shifts, we first interpolated between the 6° FOE points with 1° spline interpolation (Shenoy et al. 1999, 2002). We then calculated the cross-correlation coefficient at each 1° step. The shift that produced the highest correlation coefficient was used to calculate the amount of shift (Bradley et al. 1996; Shenoy et al. 1999, 2002).
Shift index
To calculate the coordinate frame of the population, we created a shift index where 0 = eye coordinates and 1 = screen coordinates. We took the calculated shift between fixation target position 1 and fixation target position 5 and divided this by the actual distance between fixation target position 1 and fixation target position 5, which is 24°. This index gives an indication whether area MSTd is coding in eye coordinates, screen coordinates, or somewhere in between.
Singular value decomposition
To test whether there was a multiplicative interaction between eye position and focus of expansion tuning curves, we performed a singular value decomposition (SVD) on the matrix of responses where fixation target position and focus of expansion were varied. The SVD determines how well fixation target positions can be separated from focus of expansion tuning curves by multiplication (Pena and Konishi 2001; Pesaran et al. 2010). This was done by modeling the responses as a weighted sum of matrices each of which multiplies the fixation target position and the focus of expansion tuning curves. If the response was separable as a gain field, the response energy was primarily concentrated in the first singular value and a multiplicative model was a good fit. However, if the response was vectorial, a multiplicative model was not a good fit, and a gain field was not separable. In this scenario, the response energy would be strong in singular values other than just the first singular value.
The mean value of the entire response matrix, averaged over rows and columns, was subtracted from the response matrix before performing the SVD. Separability was defined by a significantly (P < 0.05) large first singular value when compared with the first singular value of randomized trial conditions (randomization test). In our analysis, we classified the responses as separable or inseparable according to the P = 0.05 threshold instead of by the strength of separability, which is defined by the magnitude of the first singular value compared with the others.
A multiplicative interaction would result if there was an eye position gain field modulation of responses. For responses that showed a multiplicative interaction, the SVD estimated the shape of the average tuning curve and gain field.
Gain analysis
The gain fields were fit by a linear model, and then linear regression techniques were used to partition the variability into the components of residual “pure error” and “lack-of-fit” (Draper and Smith 1998; Kleinbaum and Kupper 1978; Neter et al. 1983). The sum-of-squares lack-of-fit was the difference between the sum of squares of the model data and the sum of squares of the pure error. The F statistic was the ratio of mean-square lack-of-fit divided by the mean-square pure error. A P value was then calculated from the F statistic.
Our analysis separated the gain fields into four categories (Andersen et al. 1990). Neurons that had significant linear components (P < 0.05) and no significant lack-of-fit (P > 0.05) were classified as linear (L). This type of gain field was best fit by a linear model. Neurons that had a significant linear component (P < 0.05) but also had a significant lack-of-fit (P < 0.05) were classified as linear component (LC). A simple linear model fit part of the gain fields but could not account for all of the variance. No linear component (LO) gain fields had no significant linear component (P > 0.05) but did have a significant lack-of-fit (P < 0.05). This meant that there was no linear component to the gain field, but the activity did vary with fixation target position. Neurons that showed no response to fixation target position and thus had no gain field (NG), had no significant linear component (P > 0.05) and no significant lack-of-fit (P > 0.05).
RESULTS
The neurons were first tested in the characterization tasks for preferred spiral space pattern, preferred laminar motion, and preferred pursuit direction. Then the neurons were tested in the coordinate frame task. We analyzed data from 80 neurons in two monkeys, 51 from monkey DON and 29 from monkey ROY. Neurons were selected for having significant responses in the three conditions of fixed gaze, real pursuit, and simulated pursuit.
Characterization tasks
Seventy-four of 80 (93%) neurons recorded were significantly tuned to a single direction in spiral space (P < 0.05, Rayleigh test; Fig. 2A). Most of the neurons showed the greatest response to an optic flow pattern that contained an expansion component 69/80 (86%) of neurons. The preferred response across the population of neurons was −2 ± 15° (95% CI), i.e., expansion, and the distribution was not uniform (P < 0.05, Rayleigh test). However, it is important to note in our study that we selected neurons that showed the greatest response to an expansion pattern. Therefore our population of neurons may not be an accurate representation of the optic flow preferences of neurons in MSTd.
Fig. 2.
Population: spiral/laminar/pursuit. A: spiral space: 74/80 (93%) neurons recorded were significantly tuned to 1 direction in spiral space (P < 0.05, Rayleigh test). The preferred response across the population of neurons was −2 ± 15° (95% confidence interval) and the distribution was not uniform (P < 0.05, Rayleigh test). Most of the neurons showed the greatest response to an optic flow pattern that contained an expansion component: 69/80 (86%) of neurons. B: laminar: 67/80 (84%) medial superior temporal area (MSTd) neurons showed significant tuning to laminar motion stimuli (P < 0.05, Rayleigh test). The distribution of preferred directions was not uniform and was biased toward upward motion (P < 0.05, Rayleigh test). There was not an ipsilateral/contralateral bias: 40/80 (50%) neurons. C: pursuit: 49/80 (61%) MSTd neurons showed significant tuning to pursuit (P < 0.05, Rayleigh test). The distribution of preferred pursuit directions was not uniform and was biased toward downward motion (P < 0.05, Rayleigh test). There was also a slight contralateral bias: 42/80 (53%) neurons.
Sixty-seven of 80 (84%) MSTd neurons showed significant tuning to laminar motion stimuli (P < 0.05, Rayleigh test, Fig. 2B) The distribution of preferred directions was not uniform and was biased toward upward motion (P < 0.05, Rayleigh test). The preferred direction was not biased toward the ipsilateral or contralateral visual field: 40/80 (50%) neurons.
Across the population, 49/80 (61%) MSTd neurons showed significant tuning to pursuit (P < 0.05, Rayleigh test; Fig. 2) The distribution of preferred pursuit directions was not uniform and was biased toward downward motion (P < 0.05, Rayleigh test). There was also a slight contralateral bias: 42/80 (53%) neurons.
Coordinate frame testing
EXAMPLE NEURON: SHIFT.
To determine the coordinate frame of MSTd neurons when the eyes were stationary, we used cross-correlation to measure the shift of the focus tuning curves measured between fixation target position 1 (−12°) and fixation target position 5 (12°). These fixation target positions are the furthest apart so the focus tuning curves are most likely to show the largest shift if the response is not eye-centered. The example neuron in Fig. 3 has a shift of 0°. This means the focus tuning curve moves with the eye and the neuron is coding in eye coordinates.
Fig. 3.
PSTH for five fixation target positions in fixed gaze. Raw tuning curves made from several peristimulus time histograms (PSTHs) of a single neuron at 5 fixation target positions in the fixed gaze condition. Each row is made up of PSTHs that represent the activity of the neuron at each of the 11 focus positions (6° steps, range ± 30°) that make up the heading stimuli. Each row is a different fixation target position, which range from −12 to 12° in 6° steps. The tuning curves are plotted in eye coordinates and the relative shift from fixation target position 1 (−12°) to fixation target position 5 (12°) measured by cross-correlation is 0°. This indicates that this neuron represents focus tuning in eye coordinates.
The same neuron is plotted in Fig. 4 with the focus tuning curves overlaid on each other. The neuron is plotted in both eye coordinates and screen coordinates. The tuning curves will align on top of each other in the coordinate frame in which they are best represented. In the fixed gaze condition, the tuning curves line up exactly in eye coordinates.
Fig. 4.
Eye/screen coordinates plot. Focus tuning curves from 5 fixation target positions overlaid on each other in eye and screen coordinates for fixed gaze, real pursuit, and simulated pursuit. The y axis is the firing rate and the x axis is the focus position. The tuning curves align with each other in the coordinate frame in which they are represented. This neuron represents focus tuning curves in eye coordinates for all 3 conditions of fixed gaze, real pursuit, and simulated pursuit. A slight vertical gain effect can be seen; however, the horizontal alignment is 0° in all 3 task conditions as measured by cross-correlation.
Similarly, for the same neuron, when the monkey is pursuing a dot through different fixation target positions, there is also 0° shift between the focus tuning curves measured at fixation target position 1 and fixation target position 5. This means that MSTd neurons use the same eye-centered coordinate frame whether the eye is stationary or moving.
In the simulated pursuit condition, where the visual stimulus moved across the field of view while the monkey fixated a stationary target, there is also zero shift. This condition is behaviorally similar to the fixed gaze condition in that the eyes are not moving, but the visual stimulus on the retina is identical to that in the real pursuit condition. This result is not unexpected considering that both the fixed gaze and real pursuit conditions demonstrated that MSTd neurons represent heading in eye coordinates.
The cross-correlation analysis indicates that the neurons in MSTd encode the focus of expansion in an eye-centered coordinate frame. However, it is possible the best shift obtained from this analysis was not statistically significant. To test whether the shift at 0° was statistically significant we examined the R2 value generated from the cross-correlation calculation. If the R2 value is large, it indicates a 0° shift aligns the focus tuning curves significantly better than chance. We found the vast majority of neurons had a significant 0° shift and so encoded heading in eye coordinates for all three conditions of fixed gaze, real pursuit, and simulated pursuit [fixed gaze: 77/80 (96%), real pursuit: 77/80 (96%), simulated pursuit: 74/80 (93%)].
POPULATION: SHIFT INDEX.
To quantify the coordinate frame across the population of neurons, we defined a shift index where 0 = eye coordinates and 1 = screen coordinates. We took the calculated shift between fixation target position 1 and fixation target position 5 and divided this by the actual distance between fixation target position 1 and fixation target position 5. In Fig. 5, the fixed gaze condition, the mean shift index was 0.05 ± 0.04 (95% CI). This shows that the population is coding in eye coordinates. In the real pursuit condition, the mean shift index was −0.02 ± 0.04 (95% CI), and in the simulated pursuit condition, the mean shift index was 0.06 ± 0.04 (95% CI). In all three conditions of fixed gaze, real pursuit, and simulated pursuit, the population of MSTd neurons had a shift index that was not significantly different from 0 and so represented focus tuning in eye coordinates.
Fig. 5.
Population shift index. Shift index for fixed gaze, real pursuit, and simulated pursuit. The shift index is defined as the shift between fixation target position 1 and fixation target position 5 divided by the actual distance between fixation target position 1 and fixation target position 5 (24°). An index value of 0 indicates the neuron is coding in eye coordinates while a value of 1 indicates screen coordinates. In the fixed gaze condition, the population mean was 0.05 ± 0.04 [95% confidence interval (CI)], in real pursuit the mean was −0.02 ± 0.04 (95% CI), and in the simulated pursuit condition, the mean was 0.06 ± 0.04 (95% CI). In all 3 conditions, the population of MSTd neurons represent focus tuning curves in eye coordinates.
Because we found the majority of neurons in MSTd were encoding in eye coordinates, it was not necessary to test the other coordinates frames (head, body, or world coordinates). Since there are a number of different factors between fixation and pursuit conditions including extraretinal compensation (Bradley et al. 1996), adaptation of the extraretinal signal (Haarmeier et al. 2001), and changes in motion sensitivity (Schwartz and Lisberger 1994), we only compared the effect of changing eye position within the three conditions of fixation, pursuit, and simulated pursuit. The simulated pursuit condition is used to determine the degree of compensation due to retinal and extraretinal cues (Bradley et al. 1996). In the current study, it provides an additional control to show that regardless of the source of compensation the spatial representation remains eye-centered.
Gain analysis
GAIN TYPES.
For the neurons that demonstrated a multiplicative interaction as accessed by SVD analysis (see methods) we used linear regression techniques to separate the gain fields into four categories: linear, linear component, no linear component, and no gain (Andersen et al. 1990). This allowed us to identify the shape of the fixation target position gain fields. Figure 6A is an example of a neuron with a linear gain field. Figure 6B demonstrates a linear component gain field. Figure 6C is a no linear component gain field. The neuron in Fig. 6D has no gain field.
Fig. 6.
Example gain fields. Example tuning curves and their respective gain fields. Left: the example tuning curves at 5 fixation target positions overlaid on one another; right: the corresponding mean gain field. Error bars are 95% confidence intervals.
The first column in Table 1 shows the distribution of gain types for our population of neurons. The second column in Table 1 shows the percentages. Neurons that showed no gain effects ranged from 44 to 60% of the neurons depending on the task condition. The third column in Table 1 shows the percentages after removing those neurons that have no fixation target position gain effects. The distribution of linear or linear component neurons versus no linear component neurons is similar in all three task conditions. Approximately 2/3 of the neurons have significant linear or linear component gain fields (∼69%). The other one-third (∼31%) demonstrate no linear component gain fields. These results are similar to those found in other parietal areas (Andersen et al. 1990).
Table 1:
Fixation target position gain field types
| Neurons | Percent, Includes No Gain | Percent, Excludes No Gain | |
|---|---|---|---|
| Fixed | |||
| Linear | 25/79 | 32 | 57 | 
| Linear Component | 5/79 | 6 | 11 | 
| No Linear Component | 14/79 | 18 | 32 | 
| No Gain Field | 35/79 | 44 | |
| Real Pursuit | |||
| Linear | 12/77 | 16 | 39 | 
| Linear Component | 9/77 | 12 | 29 | 
| No Linear Component | 10/77 | 13 | 32 | 
| No Gain Field | 46/77 | 60 | |
| Simulated Pursuit | |||
| Linear | 19/74 | 26 | 44 | 
| Linear Component | 12/74 | 16 | 28 | 
| No Linear Component | 12/74 | 16 | 28 | 
| No Gain Field | 31/74 | 42 | 
Linear regression techniques were used to separate the gain fields into four categories: linear, linear component, no linear component, and no gain. Neurons that had significant planar components (P < 0.05) and no significant lack-of-fit (P > 0.05) were classified as linear. Neurons that had a significant linear component (P < 0.05) but also had a significant lack-of-fit (P < 0.05) were classified as linear component (PC). No linear component (NO) gain fields had no significant linear component (P > 0.05) but did have a significant lack-of-fit (P < 0.05). Neurons that showed no response to fixation target position and thus had no gain field (NG), had no significant linear component (P > 0.05) and no significant lack-of-fit (P > 0.05). Percentages were first calculated including neurons that had no gain fields and then again excluding the neurons with no gain fields.
GAIN MODULATION.
To quantify the amount of change in the gain fields due to fixation target position, we calculated the percent modulation as 100 * (1 − min/max), which gives us the percent change of the maximum over the fixation target position range of 24° (Andersen et al. 1990). For example, a result of 50% means that the activity changed 50% of the maximum activity spanning 24°.
The gain modulation (Table 2) varied from 26 to 50%. These results are similar to those found by Andersen et al. (1990) in other parietal areas. Gain modulation across all three task conditions was approximately similar. Gain modulation by gain type was slightly higher for linear component. Overall gain modulation for the population was 33%.
Table 2.
Gain modulation
| Gain Modulation | L, % | LC, % | NO, % | Total, % | |
|---|---|---|---|---|---|
| Fixed Gaze | 29 | 33 | 37 | 32 | |
| Real Pursuit | 30 | 50 | 26 | 34 | |
| Simulated Pursuit | 28 | 39 | 35 | 33 | |
| Total | 29 | 42 | 33 | 33 | Overall | 
The gain percent modulation was calculated as 100 * (1 − min/max). This gave us the percent change of the maximum over the fixation target position range of 24°. The gain modulation across all three task conditions was approximately similar. Gain modulation by gain type was slightly higher for linear component. Overall gain modulation for the population was 33%. L, linear; LC, linear component, NO, no linear component.
DISCUSSION
The primary goal of the experiments presented here was to determine the coordinate frame in which MSTd neurons represent heading direction during gaze rotation. When aligned in eye coordinates, the focus of expansion tuning curves showed zero shift relative to each other both when the eyes are fixating or pursuing. Thus area MSTd neurons represent the FOE in eye centered coordinates during both of these behaviours.
Eye coordinate representations in posterior parietal cortex
Based on lesion studies in monkeys and humans, it is thought that the posterior parietal cortex plays an important role in spatial perception and visually guided self-motion (Andersen 1987, 1989; Lynch 1980; Stein 1992). Areas in the posterior parietal cortex have specialized functions such as LIP for saccades and related cognitive processes, such as attention, PRR for reaches, and the anterior intraparietal area for grasping. MSTd neurons have been implicated in processing spatial information for the purpose of visually guided locomotion. Their response properties are in line with an area that is involved with computing heading direction from optic flow (Bradley et al. 1996; Duffy and Wurtz 1991a,b, 1995; Lappe and Rauschecker 1994; Shenoy et al. 2002), object motion detection (Tanaka et al. 1993), and the control of smooth pursuit eye movements (Komatsu and Wurtz 1988).
Each of these posterior parietal areas is located in a position that takes in visual inputs and then transforms and sends the information to downstream motor command areas. Because each planning region has a different action, it has been suggested that the motor plan for each region is represented in the appropriate coordinate frame most relevant to the movement (Colby 1998). With all the evidence for the involvement of area MSTd in self-motion computation and heading perception, there was a strong possibility that neurons in MSTd already represented focus tuning curves in body or world coordinates. However, we have shown that area MSTd actually codes in an eye-centered coordinate frame, similar to areas LIP and PRR of the posterior parietal cortex.
Because vision is a primary source of information to guide locomotion, there must be a transformation from the image on the retina in eye coordinates, to muscle commands that are in body coordinates (Crawford et al. 2004). As an area that is involved in self-motion perception, the finding that MSTd neurons represent focus tuning curves in eye coordinates, like many other posterior parietal cortical areas, is not surprising from an efficiency point of view. Areas LIP and PRR compensate for intervening saccades to maintain target location in eye coordinates for saccades (LIP) (Duhamel et al. 1992; Gnadt and Andersen 1988) and reaches (PRR) (Batista et al. 1999).
There are several possible reasons why neurons in MSTd are coding in eye coordinates. The first reason might simply be that MSTd represents an early stage where the information is closer to the input than it is the output. While this may be partially true, the fact that neurons in MSTd can compensate for eye movements that perturb the visual scene, adjust for pursuit speed, and adjust for forward translation speed suggests that MSTd is already prominently involved in heading computation. The visual scene is constantly changing as monkeys move throughout the world. Because MSTd neurons are involved in detection of object motion (Tanaka et al. 1993) and have smooth pursuit activity and tuning (Komatsu and Wurtz 1988), it would be most economical to represent the visual scene in eye coordinates and make the necessary corrections at this early stage rather than later. Thus the transformation from retinal to other coordinate frames can be performed by the same neural mechanism, and it does not need to differentiate between whether the eyes are fixating or pursuing. By coding in eye coordinates, similar to LIP and PRR, this common coordinate frame allows the areas that control different actions such as pursuit, saccades and reaches to coordinate these movements efficiently.
Comparison to other studies
It was found that both pursuit and simulated pursuit conditions code in eye coordinates. The visual image in both cases is the same, but in the pursuit case, the eyes are moving. In our previous experiments, we found that comparing these two experimental conditions led to the conclusion that a portion of the pursuit compensation is derived from retinal cues and still present in the simulated pursuit condition. An additional and considerable portion of compensation is due to extraretinal signals and is only present during pursuit (Bradley et al. 1996; Lee et al. 2007; Shenoy et al. 1999, 2002).
The most similar study to ours is by Fetsch et al. (2007), who examined the reference frame of visual and vestibular heading signals in MSTd with the eyes stationary. They found, consistent with our finding, that the visual heading signals were largely in eye coordinates. Our current results indicate that this finding of eye-centered coding extends to conditions where the eyes are moving and during simulated pursuit.
Fetsch et al. (2007) found that vestibular signals for inertial motion were coded in coordinates in a head-centered frame or intermediate between head and eye-centered. Previously we found that compensation of FOE tuning for gaze rotation during VOR cancellation has an extraretinal component that may also be vestibular in origin. It is possible that the vestibular signals seen in MSTd, which are considerably weaker than the visual signals (Fetsch et al. 2007), are for the purpose of compensating the visual FOE tuning curves in eye coordinates. In other words, the main output of MSTd related to heading perception may be primarily visual in nature, and in eye coordinates, even though some of the compensation signals within MSTd are of extraretinal origin and coded in nonretinal coordinates.
Population coding
In our experiment, we found the response fields were gain modulated by fixation position. This leads to the possibility that the combined population response can be useful for decoding the position of the eyes in the orbits and thus the location of the stimulus with respect to the head. This type of population coding has been pointed out in other studies such as Takemura et al. (2001) that suggested that while individual neurons encoded only a small part of a binocular disparity stimuli, when summed together, they encoded the entire vergence velocity response. Similarly, Pouget and Sejnowski (1994) found in their model that none of their gain-modulated units were specifically tuned to distance; however, as a whole, egocentric distance could be determined from the distributed network representation.
Gain fields
We found that firing rates of MSTd neurons are modulated by fixation target position. Many MSTd neurons have a multiplicative interaction between the focus of expansion tuning curves and fixation target position gain fields. This result fits with other posterior parietal areas, which also show fixation target position gain effects. It is likely that gain modulation is a general mechanism that is present throughout the posterior parietal cortex. Eye position gain modulation has been found in area V3A (Galletti and Battaglini 1989), V6 (Galletti et al. 1995), area 7a (Andersen and Mountcastle 1983; Andersen et al. 1985), and LIP (Andersen et al. 1990).
A number of computational studies give support to the idea that eye position gain modulation is important in coordinate transformations (Andersen and Zipser 1988; Goodman and Andersen 1989, 1990; Van Den Berg and Beintema 1997; Zipser and Andersen 1988). For instance, the hidden units of a neural network trained to represent positions in head-centered coordinates given eye positions and retinal inputs had similar properties to recorded LIP and 7A neurons (Zipser and Andersen 1988). Interestingly, if the cells do not compensate for changes in the FOE during gaze rotation, then gain modulated coordinate transformations would have to use different representations for pursuit and fixation conditions. Thus representing the FOE in eye coordinates under both behavioral conditions reduces the complexity of the transformation process.
Summary
Area MSTd has characteristics similar to its cortical neighbors, namely coding in eye-centered coordinates and fixation target position gain modulation. These common properties suggest that there is a shared general mechanism for visual-motor transformations in several cortical areas within the posterior parietal cortex involving an eye-centered representation.
GRANTS
This research was supported by the National Eye Institute, J.G. Boswell Professorship, a Career Award in the Biomedical Sciences from the Burroughs Wellcome Fund, and a Howard Hughes Medical Institute Pre-Doctoral Fellowship.
DISCLOSURES
No conflicts of interest, financial or otherwise, are declared by the author(s).
ACKNOWLEDGMENTS
The authors thank K. Pejsa, N. Sammons, and L. Martel for animal care and surgical assistance, T. Yao for administrative assistance, V. Shcherbatyuk for technical assistance, J. A. Crowell for optic flow stimulus assistance, and M. Brozovic and B. Breznen for scientific discussions on the manuscript.
REFERENCES
- Andersen RA. The role of the inferior parietal lobule in spatial perception and visual-motor integration. In: Handbook of Physiology. The Nervous System. Higher Functions of the Brain. Edition Bethesda, MD: Am. Physiol. Soc., 1987, sect. 1, vol. V, p. 483–518, 1993 [Google Scholar]
- Andersen RA. Visual and eye movement functions of the posterior parietal cortex. Annu Rev Neurosci 12: 377–403, 1989 [DOI] [PubMed] [Google Scholar]
- Andersen RA. Multimodal integration for the representation of space in the posterior parietal cortex. Phil Trans Royal Soc B Biol Sci: 1421–1428, 1997 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Andersen RA, Bracewell RM, Barash S, Gnadt JW, Fogassi L. Eye position effects on visual, memory, and saccade-related activity in areas LIP and 7a of macaque. J Neurosci 10: 1176–1196, 1990 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Andersen RA, Buneo CA. Intentional maps in posterior parietal cortex. Annu Rev Neurosci 25: 189–220, 2002 [DOI] [PubMed] [Google Scholar]
- Andersen RA, Essick GK, Siegel RM. Encoding of spatial location by posterior parietal neurons. Science 25: 456–458, 1985 [DOI] [PubMed] [Google Scholar]
- Andersen RA, Mountcastle VB. The influence of the angle of gaze upon the excitability of the light- sensitive neurons of the posterior parietal cortex. J Neurosci 3: 532–548, 1983 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Andersen RA, Snyder LH, Bradley DC, Xing J. Multimodal representation of space in the posterior parietal cortex and its use in planning movements. Annu Rev Neurosci 20: 303–330, 1997 [DOI] [PubMed] [Google Scholar]
- Andersen RA, Snyder LH, Li C-S, Stricanne B. Coordinate transformation in the representation of spatial information. Current Opin Neurobiol 3: 171–176, 1993 [DOI] [PubMed] [Google Scholar]
- Andersen RA, Zipser D. The role of the posterior parietal cortex in coordinate transformations for visual-motor integration. Can J Physiol Pharmacol 66: 488–501, 1988 [DOI] [PubMed] [Google Scholar]
- Batista AP, Buneo CA, Snyder LH, Andersen RA. Reach plans in eye-centered coordinates. Science 285: 257–260, 1999 [DOI] [PubMed] [Google Scholar]
- Bradley DC, Maxwell M, Andersen RA, Banks MS, Shenoy KV. Mechanisms of heading perception in primate visual cortex. Science 273: 1544–1547, 1996 [DOI] [PubMed] [Google Scholar]
- Bremmer F, Ilg UJ, Thiele A, Distler C, Hoffmann KP. Eye position effects in monkey cortex. I. Visual and pursuit-related activity in extrastriate areas MT and MST. J Neurophysiol 77: 944–961, 1997 [DOI] [PubMed] [Google Scholar]
- Cohen YE, Andersen RA. A common reference frame for movement plans in the posterior parietal cortex. Nat Rev Neurosci 3: 553–562, 2002 [DOI] [PubMed] [Google Scholar]
- Colby CL. Action-oriented spatial reference frames in cortex. Neuron 20: 15–24, 1998 [DOI] [PubMed] [Google Scholar]
- Crawford JD, Medendorp WP, Marotta JJ. Spatial transformations for eye-hand coordination. J Neurophysiol 92: 10–9, 2004 [DOI] [PubMed] [Google Scholar]
- Draper NR, Smith H. Applied Regression Analysis (3rd ed.). New York: Wiley, 1998 [Google Scholar]
- Duffy CJ, Wurtz RH. Sensitivity of MST neurons to optic flow stimuli. II. Mechanisms of response selectivity revealed by small-field stimuli. J Neurophysiol 65: 1346–1359, 1991a [DOI] [PubMed] [Google Scholar]
- Duffy CJ, Wurtz RH. Sensitivity of MST neurons to optic flow stimuli. I. A continuum of response selectivity to large-field stimuli. J Neurophysiol 65: 1329–1345, 1991b [DOI] [PubMed] [Google Scholar]
- Duffy CJ, Wurtz RH. Response of monkey MSTd neurons to optic flow stimuli with shifted centers of motion. J Neurosci 15: 5192–5208, 1995 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Duhamel JR, Bremmer F, BenHamed S, Graf W. Spatial invariance of visual receptive fields in parietal cortex neurons. Nature. 389: 845–848, 1997 [DOI] [PubMed] [Google Scholar]
- Duhamel JR, Colby CL, Goldberg ME. The updating of the representation of visual space in pareital cortex by intended eye movements. Science 255: 90–92, 1992 [DOI] [PubMed] [Google Scholar]
- Fetsch CR, Gu Y, DeAngelis GC, Angelaki DE. Visual and vestibular heading signals in area MSTd do not share a common spatial reference frame. Soc Neurosci Abstr 390.6, 2005 [Google Scholar]
- Fetsch CR, Wang S, Gu Y, DeAngelis GC, Angelaki DE. Spatial reference frames of visual, vestibular, and multimodal heading signals in the dorsal subdivision of the medial superior temporal area. J Neurosci 17: 700–712, 2007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fetsch CR, Watkins VP, Gu Y, DeAngelis GC, Angelaki DE. Coordinate frames for representing heading direction in area MSTd. Soc Neurosci Abstr 368.5, 2004 [Google Scholar]
- Galletti C, Battaglini PP. Gaze-dependent visual neurons in area V3A of monkey prestriate cortex. J Neurosci 9: 1112–1125, 1989 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Galletti C, Battaglini PP, Fattori P. Eye position influence on the parieto-occipital area PO (V6) of the macaque monkey. Eur J Neurosci 7: 2486–2501, 1995 [DOI] [PubMed] [Google Scholar]
- Geesaman BJ, Andersen RA. The analysis of complex motion patterns by form/cue invariant MSTd neurons. J Neurosci 16: 4716–4732, 1996 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gibson JJ. The Perception of the Visual World. Boston: Houghton Mifflin, 1950 [Google Scholar]
- Gnadt JW, Andersen RA. Memory related motor planning activity in posterior parietal cortex of macaque. Exp Brain Res 70: 216–220, 1988 [DOI] [PubMed] [Google Scholar]
- Goodman SJ, Andersen RA. Microstimulation of a neural-network model for visually guided saccades. J Cognit Neurosci 1: 317–326, 1989 [DOI] [PubMed] [Google Scholar]
- Goodman SJ, Andersen RA. Algorithm programmed by neural network model for coordinate transformations. In: International Joint Conference on Neural Networks. San Diego, CA: IEEE Neural Network Council, 1990, p. 381–386 [Google Scholar]
- Graziano M, Andersen RA, Snowden R. Tuning of MST neurons to spiral motions. J Neurosci 14: 54–67, 1994 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Harmeier T, Bunjes F, Lindner A, Berret E, Their P. Optimizing viusal motion perception during eye movements. Neuron 32: 527–536, 2001 [DOI] [PubMed] [Google Scholar]
- Judge SJ, Richmond BJ, Chu FC. Implantation of magnetic search coils for measurement of eye position: an improved method. Vision Res 20: 535–538, 1980 [DOI] [PubMed] [Google Scholar]
- Kleinbaum DG, Kupper LL. Applied Regression Analysis and Other Multivariable Methods. North Scituate, MA: Duxbury, 1978 [Google Scholar]
- Komatsu H, Wurtz RH. Relation of cortical areas MT and MST to pursuit eye movements, I. Localization and visual properties of neurons. J Neurophysiol 60: 580–603, 1988 [DOI] [PubMed] [Google Scholar]
- Lappe M, Rauschecker JP. Heading detection from optic flow. Nature 369: 712–713, 1994 [DOI] [PubMed] [Google Scholar]
- Lee B, Pesaran B, Andersen RA. Translation speed compensation in the dorsal aspect of the medial superior temporal area. J Neurosci 27: 2582–2591, 2007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lee B, Pesaran B, Andersen A. MSTd represents heading in an eye-centered coordinate frame. Soc Neurosci Abstr 390.8, 2005 [Google Scholar]
- Lynch JC. The functional organization of posterior parietal association cortex. Behav Brain Sci 3: 485–534, 1980 [Google Scholar]
- Neter J, Wasserman W, Kutner MH. Applied Linear Regression Models. Homewood, IL: Irwin, 1983 [Google Scholar]
- Pena JL, Konishi M. Auditory spatial receptive fields created by multiplication. Science 292: 249–252, 2001 [DOI] [PubMed] [Google Scholar]
- Pesaran B, Nelson MJ, Anderson RA. A relative position code for saccades in dorsal premotor cortex. J Neurosci 30: 6527–6537, 2010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pouget A, Sejnowski TJ. A neural model of the cortical representation of egocentric distance. Cereb Cortex 4: 314–329, 1994 [DOI] [PubMed] [Google Scholar]
- Royden CS, Crowell JA, Banks MS. Estimating heading during eye movements. Vision Res. 34: 3197–3214, 1994 [DOI] [PubMed] [Google Scholar]
- Schwartz JD, Lisberger SG. Initial tracking conditions modulate the gain of visuo-motor transmission for smooth pursuit eye movements in monkeys. Visual Neurosci 11: 411–424, 1994 [DOI] [PubMed] [Google Scholar]
- Shenoy KV, Bradley DC, Andersen RA. Influence of gaze rotation on the visual response of primate MSTd neurons. J Neurophysiol 81: 2764–2786, 1999 [DOI] [PubMed] [Google Scholar]
- Shenoy KV, Crowell JA, Andersen RA. Pursuit speed compensation in cortical area MSTd. J Neurophysiol 88: 2630–2647, 2002 [DOI] [PubMed] [Google Scholar]
- Stein JF. The representation of egocentric space in the posterior parietal cortex. Behav Brain Sci 15: 691–700, 1992 [DOI] [PubMed] [Google Scholar]
- Takemura A, Inoue Y, Kawano K, Quaia C, Miles FA. Single-unit activity in cortical area MST associated with disparity-vergence eye movements: evidence for population coding. J Neurophysiol 85: 2245–2266, 2001 [DOI] [PubMed] [Google Scholar]
- Tanaka K, Sugita Y, Moriya M, Saito HA. Analysis of object motion in the ventral part of the medial superior temporal area of the macaque visual-cortex. J Neurophysiol 69: 128–142, 1993 [DOI] [PubMed] [Google Scholar]
- Van Den Berg AV, Beintema JA. Motion templates with eye velocitygain fields for transformation of retinal to head centric flow. Neuroreport 8: 835–840, 1997 [DOI] [PubMed] [Google Scholar]
- Xing J, Andersen RA. Memory activity of LIP neurons for sequential eye movements simulated with neural networks. J Neurophysiol 84: 651–665, 2000 [DOI] [PubMed] [Google Scholar]
- Zar JH. Biostatistical Analysis (4th ed.). Englewood Cliffs, NJ: Prentice Hall, 1996 [Google Scholar]
- Zhang T, Heuer HW, Britten KH. Parietal area VIP neuronal responses to heading stimuli are encoded in head-centered coordinates. Neuron: 42: 993–1001, 2004 [DOI] [PubMed] [Google Scholar]
- Zipser D, Andersen RA. A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons. Nature 331: 679–684, 1988 [DOI] [PubMed] [Google Scholar]






