SUMMARY
Background
Many visual areas of the primate brain contain signals related to the current position of the eyes in the orbit. These cortical eye-position signals are thought to underlie the transformation of retinal input – which changes with every eye movement – into a stable representation of visual space. For this coding scheme to work, such signals would need to be updated fast enough to keep up with the eye during normal exploratory behavior. We examined the dynamics of cortical eye-position signals in four dorsal visual areas of the macaque brain: the lateral and ventral intraparietal areas (LIP; VIP), the middle temporal area (MT), and the medial-superior temporal area (MST). We recorded extracellular activity of single neurons while the animal performed sequences of fixations and saccades in darkness.
Results
The data show that eye-position signals are updated predictively, such that the representation shifts in the direction of a saccade prior to (<100ms) the actual eye movement. Despite this early start, eye-position signals remain inaccurate until shortly (10–150ms) after the eye movement. Using simulated behavioral experiments, we show that this brief misrepresentation of eye position provides a neural explanation for the psychophysical phenomenon of ‘perisaccadic mislocalization’, in which observers misperceive the positions of visual targets flashed around the time of saccadic eye movements.
Conclusions
Together, these results suggest that eye-position signals in the dorsal visual system are updated rapidly across eye movements and play a direct role in perceptual localization, even when they are erroneous.
INTRODUCTION
To localize a visual object, an observer must take into account not only its position on the retina, but also the positions of the eyes in the orbit, the angle of the head relative to the body, and many other postural variables. A candidate mechanism for this type of integration is embodied in cortical neurons that modulate their response to stimuli as a function of gaze direction [1–5]. Such neurons represent simultaneously the contents of a visual scene and the current positions of the eyes in the orbit, and thus contain the crucial ingredients for the construction of a stable spatial code in the context of ongoing eye movements. Indeed, since their discovery in parietal cortex, a wealth of theoretical work has linked these eye-position signals to many fundamental spatial abilities of the primate nervous system, including navigation, multisensory integration, sensorimotor transformations, and perceptual localization [6–8].
Despite the maturity of the theoretical framework, it remains controversial whether eye-position signals in visual areas of the brain are actually used for online spatial coding [see Ref. 9 for a recent discussion]. Further, many characteristics of these signals that are fundamental to the theory have not been investigated experimentally. In particular, it is unknown whether they are updated fast enough to keep up with the eyes during exploratory behavior – an obvious pre-requisite for their purported roles.
In contrast, the dynamics of eye-position representation have been investigated extensively in the human psychophysical literature [10–13]. There, inferences about eye-position signals have been made on the basis of perceptual performance around the time of saccadic eye movements. Such experiments have shown that targets flashed just before saccades are mislocalized in the direction of eye movement, while targets flashed just after saccades are mislocalized in the opposite direction [see Ref. 14 for a review]. This pattern of perceptual error has been attributed to a damped (i.e. temporally smoothed) internal representation of eye position [10, 12, 13, 15]. If cortical eye-position signals contribute to visuospatial perception, as suggested by theoretical work, one would predict a match between their dynamics and the eye-position signal inferred from psychophysical data.
To examine the dynamics of cortical eye-position signals, we recorded extracellular spiking activity of single neurons in two macaque monkeys (four hemispheres) as they performed a combination of fixations and saccadic eye movements in near-darkness. Recordings were performed in four visual cortical areas: the lateral intraparietal area (LIP), the ventral intraparietal area (VIP), the middle temporal area (MT), and the medial superior temporal area (MST). These areas contain neurons that exhibit systematic changes in firing rate across eye positions, even in darkness [2, 4, 5, 16].
Our experiments show that the updating of cortical eye-position signals begins before the onset of eye movement but remains incomplete until shortly after saccade offset. We show that these dynamics provide a striking match to those predicted from psychophysical studies of perisaccadic mislocalization. This correspondence between neural data and a well-established perceptual illusion provides, to our knowledge, the first physiological evidence for a direct role of cortical eye-position signals in perception.
RESULTS
We recorded from 290 neurons; 75 from area LIP, 115 from area VIP, and a total of 100 from areas MT and MST. Each trial of the task required in pseudo randomized order either a rightward or downward 10° saccade from one of five initial positions (Figure 1A). This design allowed measurements of spiking activity while the eyes were stationary at thirteen unique positions (‘fixation’ epochs), as well as during eye movements (‘perisaccadic’ epochs). We report our findings in two sections. First, we present physiological data that reveal the dynamics of cortical eye-position signals. Second, we use a simulation approach to determine the patterns of perisaccadic localization that would be expected if these signals were used to localize visual objects.
Figure 1.
Experimental design and data analysis. (A) Left panel: Schematic of the behavioral paradigm in Experiment 1. The animal performed either a rightward or downward saccade from one of five initial fixation positions (filled circles) to a small target stimulus (open circles) on each trial. Right panel: Trial epochs used for the analysis of neural firing rates. Pre- and post-saccadic fixation epochs were defined as the intervals from −700ms to −300ms and +300ms to +700ms relative to the onset of the saccade. (B) The eye-position field of an example neuron from area LIP. Filled circles show the mean firing rate during fixation epochs for each of the thirteen unique eye positions. Error bars = ± 1 standard error. The eye-position field was approximated by a two-dimensional, second-order polynomial function, shown here as a mesh surface. This regression analysis showed that there was a significant overall effect of eye position (F[3,9]=60.93, p < .001). In this example, the neuron spiked more frequently when the monkey fixated the right side of the screen than when the monkey fixated the left side of the screen.
Modulation of firing rates by eye position during fixation
Figure 1B shows mean firing rates during fixation at each of the unique eye positions for an example LIP neuron. The rate varied systematically with the position of the eye. We refer to this relationship between firing rate and eye position as an (‘eye-position field’. To quantify these effects, we fitted the firing rates observed during fixation for each neuron with a two-dimensional polynomial (using stepwise regression). The surface in Figure 1B Figure 1 shows the fitted function for the example neuron. On average, eye-position fields consisted of a near-doubling of firing rate across the oculomotor region we examined. The modulations associated with the change in eye position for a given saccade condition were smaller (approximately 20% on average; Figure S1A). These results were comparable across all cortical regions we examined. Significant regression surfaces were observed in approximately half (VIP, MT/MST) to three-quarters (LIP) of the recorded neurons (see Figure S1B for results and statistical analyses). These results are consistent with those reported previously [4, 5, 16].
Dynamics of cortical eye-position signals
Our aim was to determine the time-course of the transition from the firing rate during fixation before a saccade to the new firing rate after the saccade. Individual neurons, however, do not provide information about a change in eye position for all saccade directions. For instance, the neuron in Figure 1B would be constant for a vertical saccade and should not be included when studying the population dynamics around vertical saccades. For this reason, we included a neuron in the analysis of a particular saccade condition only if there was a statistically significant step in activity from the pre- to the post-saccadic fixation (see Methods for details and Experiment 2 for an alternative approach). For a given saccade condition, this step could be either positive or negative, depending on the local gradient of the eye position field between the two fixation positions. We thus divided neurons into positive gradient (POS) and negative gradient (NEG) groups, separately for each saccadic condition.
Figure 2A shows population time-courses for POS- and NEG-groups, averaged over the individual saccade conditions of the task and pooled across all brain areas. This pooled response provides a useful summary of the aggregate representation of eye position across visual cortex. Surprisingly, the saccade-induced shift in firing rates began before the onset of eye movement for both classes. The NEG-group exhibited a step-like decrease in activity approximately 50ms before the onset of the eye movement and stabilized shortly after (<50ms) the offset of the saccade. The POS-group, in contrast, showed a transient increase in activity prior to (~150ms) the eye movement, followed by stabilization after (~150ms) the eye landed. These changes in firing rate occurred earlier than could be accounted for by the temporal averaging associated with the calculation of firing rates (up to 25ms of shift). These dynamics were comparable across the ten saccade conditions, as shown implicitly by the small standard errors in Figure 2A (shaded regions).
Figure 2.
Dynamics of eye-position signals in dorsal visual cortex. (A) Population activity across saccades, plotted separately for neurons that shift from a low-to-high firing rate (POS) and from a high-to-low firing rate (NEG) across successive fixations. Data are pooled across brain regions and aligned to the onset of the saccade. Each time-course represents the mean population curve across the ten saccade conditions shown in Figure 1A. Firing rates are expressed as a percentage change relative to baseline activity (defined as the mean activity across the two fixation intervals; the same results were observed without this normalization). The shaded column indicates the time and average duration of the saccade. (B) The derived eye-position signal, obtained by subtracting POS and NEG curves for each condition and averaging the time-courses. The dotted line indicates the timing and metrics of the saccade, scaled to match the step in cortical firing rates. Eye tracking data for each condition is presented in Figure S2. Shaded regions in (A) and (B) represent ± 1 standard error across ten saccade conditions. This analysis shows that an anticipatory eye-position signal is available in a distributed population code across posterior parietal cortex.
The dynamics observed for POS and NEG groups suggest that they provide incongruent information about eye position during saccades. For object localization, however, the brain must compute a singular estimate of gaze direction. One simple way the brain might extract such a signal is to take the difference in activity between the neurons in the two groups, as shown in Figure 2B. For comparison, the monkey’s actual saccade dynamics are also plotted (dotted line, scaled to match the overall change in firing rate). The derived eye-position signal resembled a damped (i.e. low-pass filtered) version of the saccade.
Figure 3 shows the same analyses when neurons from each cortical region were considered separately. Although there was variation in the specific patterns for POS and NEG groups across areas, such as drifts during fixation for VIP and MT/MST, the time-courses shared many common characteristics (Figure 3A). As can be seen in the derived eye-position signals (Figure 3B), the primary difference across areas was the strength of modulation rather than the dynamics. To confirm the predictive nature of these signals, as well as to compare their magnitudes across areas, we compared their values during the initial fixation epoch with those just before the onset of the saccade (t=−25ms). The data were analyzed using a mixed-design ANOVA with factors of time [2 levels] and area [3 levels], and the ten independent observations across task conditions as the dependent variable. There was a significant main effect of time (F(1,27) = 5.97, p < .05), confirming the predictive effect, and no significant interaction (F(2,27) = 0.122, p = .89), suggesting that predictiveness did not vary across areas.
Figure 3.
A comparison of eye-position signals across different cortical regions (LIP, VIP, and MT/MST). The graphical conventions are identical to those of Figure 2. The data show that a nimble representation of eye-position is available in each of these areas, albeit with different signal-to-noise ratios.
To this point, we have considered only the dynamics of the average population activity. To determine whether individual neurons also carried predictive information about future eye positions we compared two quantities for each neuron in each condition. The first was the overall change in mean firing rate across successive fixations. By definition, positive and negative values on this measure correspond to POS- and NEG-neurons, respectively. The second was the difference in activity between the baseline rate during the initial fixation epoch and that observed just before the saccade (−75ms to −25ms). A positive correlation between these two measures across the population would indicate that neurons anticipate the impending change in eye position. For NEG neurons, this was indeed the case for all of the cortical regions we examined (Figure S3). That is, neurons that showed larger decreases in activity across successive fixations also tended to show larger drops in activity just prior to the saccade. POS neurons, in contrast, showed no such effect in any area, suggesting that their predictive behavior is limited to a pre-saccadic bump in the population response.
Optimized saccade directions
In Experiment 2, we first performed a preliminary assessment of the eye-position field to assess the axis along which the neuron modulated its firing rate maximally, and then performed an experiment with large (40° amplitude) saccades back and forth along this axis (Figure 4A). This tailored design increased the magnitude of eye-position-dependent modulations across successive fixations, and also ensured that we recorded the same neuron during both POS- and NEG-direction saccades over the same region of oculomotor space.
Figure 4.
Dynamics of eye-position signals for large (40°), optimized saccades. (A) Saccades were performed back and forth along a sloping axis of the eye-position field under study. The schematic shows the arrangement of stimuli for a hypothetical eye-position field that is oriented upwards and toward the right (indicated by the contrast gradient). Eye tracking data for the task is presented in Figure S4. (B) Upper panel: Population time-courses for saccades performed in the POS- and NEG-directions, pooled across brain regions. Each curve represents the median firing rate across all neurons that had a significant change in firing rate across the successive fixations (paired t-tests). The arrow for each curve indicates the time at which the firing rate stabilizes after the saccade (100ms and 10ms for POS and NEG curves, respectively). Lower panel: The derived eye position signal. The error shading in both plots represents the standard error of the median (i.e. the standard deviation of medians across 1000 bootstrap samples of the neurons). These results replicate the damped cortical representation of saccade dynamics observed in Experiment 1 and confirm that updating occurs more slowly for POS saccades than for NEG saccades.
Most neurons (62 out of 78) showed significant and large changes in mean firing rate across the two fixation positions (paired t-tests), as expected for this tailored design. Figure 4 Figure 5B shows the population dynamics for saccades in the POS- and NEG-directions, pooled across cortical areas. The data provide a striking replication of the key findings of in the first experiment (cf. Figure 2A). In particular, it is clear that the transition from a low-to-high firing rate across the saccade was delayed relative to that of a high-to-low transition. Moreover, the derived eye position signal (POS–NEG) again resembled a damped version of the actual saccade. Figure S5 shows the time-courses for example neurons from areas LIP, VIP, and MST.
Figure 5.
Dynamics of eye-position signals for large (40°), optimized saccades, plotted separately for each cortical region. The format is identical to Figure 4. Areas VIP and MT/MST showed the familiar damped eye position signal. Moreover, as in Experiment 1, the pre-saccadic changes in firing rate and the trans-saccadic changes in firing rate were correlated significantly for saccades in the NEG-direction (VIP: r = 0.33, p < .05, df = 25; MT/MST: r = 0.55, p < .05, df = 14; data not shown) but not for those in the POS-direction (both p > .25, df ≥ 12). For area LIP, however, the updating of eye position signals was delayed until after the saccade and no significant correlations were observed across the sample (both p > .41, df ≥ 13). The cause of this anomalous result is not clear. One possibility is that saccade-related responses – which are common in area LIP and antagonistic for NEG neurons – were more prevalent in this sample of LIP neurons than those in Figure 3. This could have arisen from natural sampling variability for heterogeneous populations (the sample size was small), or perhaps because of the larger saccade amplitude.
For completeness, we also report the population dynamics separately for each cortical region (Figure 5), though these results should be interpreted with caution because of relatively small sample sizes. The results largely replicated the findings of Experiment 1. One exception was the dynamics for NEG saccades in area LIP, which did not show the otherwise ubiquitous decline in activity just before and during the saccade. Instead, a pre-saccadic increase in firing rate was observed, which resulted in a delay of the derived eye position signal. This suggests that saccade-related responses – known to be prevalent in LIP – can mask the predictive eye position signal when they predominate in a given sample of neurons.
Perceptual localization and the psychophysical “eye-position signal”
We next considered the pattern of perceptual localization that would be expected if the cortical eye-position signals reported here were used for online spatial coding. We simulated an experiment in which a visual target was flashed at a fixed position in space at different times relative to the onset of the saccade (Figure S6). The perceived position of the target was computed as the sum of the position of the target on the retina (which changes depending on the actual eye position at the time of the flash) and the position of the eye encoded by the neural eye-position signal (from Figure 2B). A latency parameter, τ, was included to take into account the delay between the actual time of the stimulus event and the time at which the relevant visual signals are combined with eye-position information. The specific pattern of mislocalization depends on the assumed value of τ (a free parameter).
The results of these simulations show that the neural eye-position signal can account for both the spatial and temporal aspects of perisaccadic mislocalization in humans [10, 12, 13]. Specifically, simulated visual targets were localized accurately when flashed well before or well after the saccades, but mislocalized in the direction of the eye movement when flashed just before saccade onset. This effect peaked at saccade onset – at which time the error was approximately half the saccade amplitude– and declined rapidly thereafter. After the offset of the saccade, visual targets were mislocalized in the direction opposite to that of the saccade.
DISCUSSION
Our experiments reveal that the dorsal visual processing stream of the macaque brain is furnished with a surprisingly nimble representation of eye position. We found that eye-position signals were updated predictively, such that just before the onset of a saccade, neurons behaved as if the eyes had already begun their journey toward the new fixation position. The updating occurred more slowly than the saccade itself, but was completed well within the duration of a typical fixation (≈300ms; [17]). Thus, although imperfect, eye-position signals in cortex are updated sufficiently fast to keep stead with the eyes during normal exploratory behavior. Using simulations, we showed that the imperfections of this neural signal predict a pattern of spatial (mis)localization that matches that observed in humans [10, 12, 13]. To our knowledge, these findings provide the first physiological evidence that cortical eye-position signals contribute to the localization of visual objects.
Before discussing the implications of these findings, we first consider whether our results could be explained by incidental visual stimulation from the visual target or ambient illumination. Such effects are unlikely to account for our findings. First, we minimized visibility by performing experiments in near-darkness and by preventing dark adaptation. Second, visual responses cannot explain the pre-saccadic reduction of activity for saccades in the NEG direction. Finally, pre-saccadic increases in activity for saccades in the POS direction peaked just before the saccade (<75ms; see Figure 2A), later than the expected onset for visually-evoked responses (50–100ms after target onset; mean saccadic latency was 215ms]; [18]). Therefore, the dynamics we observed are probably related to intrinsic signals around the time of an eye movement.
The data suggest a role for eye position signals in the computations that underlie visual stability across saccades. However, the nature of this involvement remains unclear. In particular, it is unknown whether they participate directly in an implicit representation of visual space [7], or indirectly through their involvement in the construction of explicit head- or world-centered receptive fields downstream [8]. Head-centered receptive fields have been reported in multiple visual areas, including VIP [19] and MT [20–22, but see Refs. 23, 24–26], but are generally considered to be rare in cortex. Consistent with this observation, our data show that explicit representations are not necessary to explain visual stability and its imperfections, and thus lend support to theories that incorporate distributed spatial codes.
One caveat is that these conclusions rely on a comparison of physiological data with previously published psychophysical data and not with simultaneous behavioral measurements from our animals. A second caveat is that localization is more complex than the combination of a perfect representation of retinal position with a possibly erroneous eye-position signal. This is supported by data showing a dependence of perisaccadic mislocalization on such features as visual references [27] and stimulus contrast [28], as well as physiological data showing remapping or distortions of retinal codes during eye movements [29, 30]. Additionally, under certain experimental conditions, measurements of perisaccadic localization also reveal a compression component in which targets are misperceived toward the saccade target [31]. This latter effect could reflect differences in eye-position signal dynamics for neurons whose receptive fields cover different parts of the retina [32, 33].
We next consider the kinds of signals that could give rise to the dynamics we observed. The predictive behavior is inconsistent with proprioceptive input from the stretch receptors of the extraocular muscles, because such signals necessarily lag behind the eye [9]. Moreover, the timing does not correspond well to the output of the neural integrator of the brain stem oculomotor plant [34], which pre-empts eye movement by less than 20ms [35]. In contrast, the dynamics provide a good match to those reported in studies of eye-position signals in the central thalamus [36, 37]. There, eye-position signals are updated as early as 100ms before saccade onset and as late as 200ms after an eye movement, consistent with our results. Moreover, such neurons exhibit a comparable hysteresis for saccades in the POS and NEG directions [for specific examples, compare our data with Figures 5b and 8 of Ref. 37]).
Although these subcortical effects bolster our findings, they nevertheless leave unanswered the question of how such eye-position signals arise. We speculate that they are computed in cortex from corollary signals that predict the onset and metrics of an impending saccadic eye movement. There are many such signals, including those related to saccade planning [38, 39], shifts of attention [40], spatial remapping [29, 41], and explicit corollary discharge [42]. LIP, for example, carries both a representation of current eye position (via eye-position fields [2]), and a representation of the impending saccade vector (via eye-centered “motor” fields [39]), and thus has all the ingredients needed to compute future eye positions.
An interesting possibility is that eye position signals are linked to mechanisms of saccadic suppression. Saccadic suppression is characterized by a reduction of visually-evoked and spontaneous activity in cortical neurons around the time of a saccade [43] For a neuron whose eye-position field imposes a lower firing rate after the saccade (i.e., a NEG-neuron), this suppression would also act as a predictive eye-position signal. In contrast, suppression is antagonistic for a POS-neuron and would cause the eye-position signal to be sluggish. These effects are consistent with the current data as well as the time-course of saccadic suppression in these cortical regions [18]. Moreover, this interpretation could explain why behavioral effects of saccadic suppression and perisaccadic mislocalization exhibit similar time-courses [43].
In sum, our experiments demonstrate that eye-position signals in the dorsal visual system are updated rapidly – albeit imperfectly – around the time of an eye movement. The imperfections in these signals mirror the cracks in visual stability observed experimentally in humans. These results suggest that cortical eye-position signals play a direct role in spatial vision in the context of ongoing eye movements.
EXPERIMENTAL PROCEDURES
Electrophysiology
All animal procedures have been described in detail previously [18]. Briefly, two male macaque monkeys (‘M1’ and ‘M2’) were implanted with recording chambers (M1: left hemisphere = LIP and VIP; right-hemisphere = MT and MST; M2: opposite configuration) and scleral search coils. The animal sat in a primate chair with the head restrained and received liquid reward for each completed trial. All procedures were in accordance with published guidelines on the use of animals in research (European Council Directive 86/609/EEC and the National Institute of Health Guide for the Care and use of Laboratory Animals) and approved by local ethics committees.
Stimuli
Target stimuli were small light-emitting diodes (0.5° diameter, 0.4 cd/cm2) back-projected onto a translucent screen subtending 60° × 60°. There were no other sources of ambient illumination and room lighting was used between recordings to prevent dark-adaptation.
Behavioral task
Experiment 1
At the beginning of each trial, the monkey maintained gaze (within 1° of error) on a target at one of five positions ([x, y] = [0°, 0°], [−10°, 10°], [−10°, −10°], [10°, −10°], [10°, 10°]). After 1000ms, the target stepped 10° either rightward or downward. The animal performed a saccade to the new position within 500ms and maintained fixation for another 1000ms. All trial types were interleaved in a pseudorandom order. The mean number of trials per condition across neurons was 15 (SD = 5).
Experiment 2
Experiment 2 required larger saccades back and forth along an inclined region of the neuron's eye-position field (M1 only; LIP: N=24; VIP: N=29; MT/MST: N=25). The task was identical to that of Experiment 1 except that the target stimuli were located at diametrically opposite positions on an imaginary circle around the center of the display (i.e. one of 0°/180°, 45°/225°, 90°/270°, or 135°/315°; radius = 20°). To choose this saccade axis, the neuron’s eye position field was quantified online by fitting a regression plane to data from preliminary trials in which the animal performed the task of Experiment 1. One of the two saccade directions was thus in a “POS” direction for the neuron and the other was in a “NEG” direction. These conditions alternated across trials. The mean number of trials for each saccade direction was 41 (SD = 13).
Data analysis
Primary saccades were detected offline using eye-velocity-based criteria and trials in which the monkey failed to perform the task correctly were discarded.
Dynamics of eye position signals
A firing rate time-course was constructed for each neuron by counting spikes that occurred within a 50ms window stepped in 25ms increments across each trial. These data were then aligned to the onset of the saccade and averaged across trials to provide separate rate curves for each of the saccade conditions. Finally, to facilitate the combination of data across neurons, these curves were normalized by expressing the rates as a percentage of the mean firing rate across the two fixation intervals in each condition (Figure 1A).
Population responses were computed separately for neurons that on average increased (POS) and decreased (NEG) their activity across a given saccade condition. For Experiment 1, in which the saccade vectors in the task were held constant across recordings, the categorization of neurons into POS and NEG groups was achieved post-hoc by identifying neurons that had significantly different firing rates during two fixation epochs for each condition (paired t-tests). To ensure that the results were not unduly influenced by our operational definition of "fixation", a family of population curves was computed for each condition, each incorporating different temporal windows for the POS/NEG categorization. These included all factorial combinations of five different window positions before (−700 to −300ms, 100ms increments, 100ms window width) and after the saccade (+300 to +700ms, 100ms increments, 100ms window width). For each configuration, a population curve was computed by taking the median firing rate across neurons at each point in time. Finally, to combine the data across window configurations, and remove potential selection effects, we erased from each curve the epochs used for the categorization (taking into account temporal smoothing) and averaged over the remaining portions of the datasets.
For Experiment 2, in which the task parameters were optimized at the time of data collection, there was less chance of false categorizations and thus a single population time-course was obtained for each of the two saccade directions (using the standard fixation epochs shown in Figure 1; identical results were found using the more robust method of Experiment 1).
HIGHLIGHTS.
-
→
Eye position is encoded in the activity of neurons in the dorsal visual system.
-
→
Eye-position signals are updated in advance of an impending eye movement.
-
→
The updating process occurs more slowly than the actual change in eye position.
-
→
The misrepresentation of eye position in parietal cortex explains a well-known illusion.
Supplementary Material
Figure 6.
A comparison of the predicted patterns of perisaccadic mislocalization (black curves) and perceptual data from classical human studies by Dassonville, Schlag, and Schlag-Rey [13], and Honda [10, 12] (red curves; shading = standard deviation across subjects; replotted with permission). The abscissa represents the time of a target flash relative to saccade onset. The ordinate represents the predicted or actual perceptual error, expressed as a percentage of saccade amplitude. Predicted errors were calculated as the mismatch between the actual eye position and that represented by neurons in dorsal visual cortex (i.e. Figure 2B), taking into account visual latency (Figure S6). The best-fitting latency values for the Dassonville et al. and Honda datasets were 66ms and −6ms, respectively. The first of these values matches the latency of visual responses in these cortical areas ([18]), but the second (negative) latency is unexpected (assuming that the brain does not compensate for visual latency during perceptual localization). However, good accounts of the psychophysical data were produced across a wide range of latencies (see Figure S6 for further examples). The inclusion of visual latency as a free parameter was inspired by theoretical work that has linked the distinct patterns of mislocalization in these two psychophysical studies to differences in visual processing times [32].
Acknowledgments
We thank Jason Mattingley, Till Hartmann, and Frederic Römschied for critical discussions and comments on the manuscript. A.P.M. was funded by the National Health and Medical Research Council of Australia. This work was supported by grants from the National Institute of Health (R01EY017605), the Pew Charitable Trusts (BK), the Deutsche Forschungsgemeinschaft (FOR 560), the Human Frontiers Science Program (RG0149/1999-B), and the European Union (MEMORY).
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Additional details of experimental procedures are provided in the Supplemental Information.
References
- 1.Galletti C, Battaglini PP. Gaze-dependent visual neurons in area V3A of monkey prestriate cortex. J. Neurosci. 1989;9:1112–1125. doi: 10.1523/JNEUROSCI.09-04-01112.1989. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Andersen RA, Bracewell RM, Barash S, Gnadt JW, Fogassi L. Eye position effects on visual, memory, and saccade-related activity in areas LIP and 7a of macaque. J Neurosci. 1990;10:1176–1196. doi: 10.1523/JNEUROSCI.10-04-01176.1990. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Andersen RA, Mountcastle VB. The influence of the angle of gaze upon the excitability of the light-sensitive neurons of the posterior parietal cortex. J. Neurosci. 1983;3:532–548. doi: 10.1523/JNEUROSCI.03-03-00532.1983. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Bremmer F, Distler C, Hoffmann KP. Eye position effects in monkey cortex. II. Pursuit- and fixation- related activity in posterior parietal areas LIP and 7A. J Neurophysiol. 1997;77:962–977. doi: 10.1152/jn.1997.77.2.962. [DOI] [PubMed] [Google Scholar]
- 5.Bremmer F, Ilg UJ, Thiele A, Distler C, Hoffmann KP. Eye position effects in monkey cortex. I. Visual and pursuit-related activity in extrastriate areas MT and MST. J Neurophysiol. 1997;77:944–961. doi: 10.1152/jn.1997.77.2.944. [DOI] [PubMed] [Google Scholar]
- 6.Deneve S, Latham PE, Pouget A. Efficient computation and cue integration with noisy population codes. Nat Neurosci. 2001;4:826–831. doi: 10.1038/90541. [DOI] [PubMed] [Google Scholar]
- 7.Pouget A, Sejnowski TJ. Spatial Transformations in the Parietal Cortex Using Basis Functions. J Cogn Neurosci. 1997;9:222–237. doi: 10.1162/jocn.1997.9.2.222. [DOI] [PubMed] [Google Scholar]
- 8.Zipser D, Andersen RA. A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons. Nature. 1988;331:679–684. doi: 10.1038/331679a0. [DOI] [PubMed] [Google Scholar]
- 9.Wang X, Zhang M, Cohen IS, Goldberg ME. The proprioceptive representation of eye position in monkey primary somatosensory cortex. Nat Neurosci. 2007;10:640–646. doi: 10.1038/nn1878. [DOI] [PubMed] [Google Scholar]
- 10.Honda H. Eye movements to a visual stimulus flashed before, during, or after a saccade. In: Jeannerod M, editor. Attention and Performance. Volume 13. Hillsdale: LEA; 1990. pp. 567–582. [Google Scholar]
- 11.Matin L, Pearce DG. Visual Perception of Direction for Stimuli Flashed During Voluntary Saccadic Eye Movements. Science. 1965;148:1485–1488. doi: 10.1126/science.148.3676.1485. [DOI] [PubMed] [Google Scholar]
- 12.Honda H. The time courses of visual mislocalization and of extraretinal eye position signals at the time of vertical saccades. Vision Res. 1991;31:1915–1921. doi: 10.1016/0042-6989(91)90186-9. [DOI] [PubMed] [Google Scholar]
- 13.Dassonville P, Schlag J, Schlag-Rey M. Oculomotor localization relies on a damped representation of saccadic eye displacement in human and nonhuman primates. Vis Neurosci. 1992;9:261–269. doi: 10.1017/s0952523800010671. [DOI] [PubMed] [Google Scholar]
- 14.Schlag J, Schlag-Rey M. Through the Eye, Slowly: Delays and Localization Errors in the Visual System. Nat Rev Neurosci. 2002;3:191–191. doi: 10.1038/nrn750. [DOI] [PubMed] [Google Scholar]
- 15.Matin L. Saccades and extraretinal signal for visual direction. In: Monty RA, Senders JW, editors. Eye movements and psychological processes. New York: Lawrence Erlbaum; 1976. pp. 203–204. [Google Scholar]
- 16.Bremmer F, Graf M, Ben Hamed S, Duhamel J-R. Eye Position Encoding in the macaque ventral intraparietal area (VIP) Neuroreport. 1999;10:873–878. doi: 10.1097/00001756-199903170-00037. [DOI] [PubMed] [Google Scholar]
- 17.Ballard DH, Hayhoe MM, Salgian G, Shinoda H. Spatio-temporal organization of behavior. Spatial Vision. 2000;13:321–333. doi: 10.1163/156856800741144. [DOI] [PubMed] [Google Scholar]
- 18.Bremmer F, Kubischik M, Hoffmann KP, Krekelberg B. Neural Dynamics of Saccadic Suppression. J Neurosci. 2009;29:12374–12383. doi: 10.1523/JNEUROSCI.2908-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Duhamel JR, Bremmer F, Ben Hamed S, Graf W. Spatial invariance of visual receptive fields in parietal cortex neurons. Nature. 1997;389:845–848. doi: 10.1038/39865. [DOI] [PubMed] [Google Scholar]
- 20.d'Avossa G, Tosetti M, Crespi S, Biagi L, Burr DC, Morrone MC. Spatiotopic selectivity of BOLD responses to visual motion in human area MT. Nat Neurosci. 2007;10:249–255. doi: 10.1038/nn1824. [DOI] [PubMed] [Google Scholar]
- 21.Melcher D, Morrone MC. Spatiotopic temporal integration of visual motion across saccadic eye movements. Nat Neurosci. 2003;6:877–881. doi: 10.1038/nn1098. [DOI] [PubMed] [Google Scholar]
- 22.Crespi S, Biagi L, d'Avossa G, Burr DC, Tosetti M, Morrone MC. Spatiotopic Coding of BOLD Signal in Human Visual Cortex Depends on Spatial Attention. PLoS ONE. 2011;6:e21661. doi: 10.1371/journal.pone.0021661. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Gardner JL, Merriam EP, Movshon JA, Heeger DJ. Maps of Visual Space in Human Occipital Cortex Are Retinotopic, Not Spatiotopic. Journal of Neuroscience. 2008;28:3988–3999. doi: 10.1523/JNEUROSCI.5476-07.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Hartmann TS, Bremmer F, Albright TD, Krekelberg B. Receptive Field Positions in Area MT during Slow Eye Movements. J Neurosci. 2011;31:10437–10444. doi: 10.1523/JNEUROSCI.5590-10.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Morris AP, Liu CC, Cropper SJ, Forte JD, Krekelberg B, Mattingley JB. Summation of visual motion across eye movements reflects a nonspatial decision mechanism. J Neurosci. 2010;30:9821–9830. doi: 10.1523/JNEUROSCI.1705-10.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Ong WS, Bisley JW. A lack of anticipatory remapping of retinotopic receptive fields in the middle temporal area. J Neurosci. 2011;31:10432–10436. doi: 10.1523/JNEUROSCI.5589-10.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Lappe M, Awater H, Krekelberg B. Postsaccadic visual references generate presaccadic compression of space. Nature. 2000;403:892–895. doi: 10.1038/35002588. [DOI] [PubMed] [Google Scholar]
- 28.Michels L, Lappe M. Contrast dependency of saccadic compression and suppression. Vision Research. 2004;44:2327–2336. doi: 10.1016/j.visres.2004.05.008. [DOI] [PubMed] [Google Scholar]
- 29.Duhamel JR, Colby CL, Goldberg ME. The updating of the representation of visual space in parietal cortex by intended eye movements. Science. 1992;255:90–92. doi: 10.1126/science.1553535. [DOI] [PubMed] [Google Scholar]
- 30.Krekelberg B, Kubischik M, Hoffmann KP, Bremmer F. Neural correlates of visual localization and perisaccadic mislocalization. Neuron. 2003;37:537–545. doi: 10.1016/s0896-6273(03)00003-5. [DOI] [PubMed] [Google Scholar]
- 31.Ross J, Morrone MC, Burr DC. Compression of visual space before saccades. Nature. 1997;386:598–601. doi: 10.1038/386598a0. [DOI] [PubMed] [Google Scholar]
- 32.Pola J. Models of the mechanism underlying perceived location of a perisaccadic flash. Vision Research. 2004;44:2799–2813. doi: 10.1016/j.visres.2004.06.008. [DOI] [PubMed] [Google Scholar]
- 33.Bischof N, Kramer E. Untersuchungen und Überlegungen zur Richtungswahrnehmung bei willkürlichen sakkadischen Augenbewegungen. Psychol Res. 1968;32:185–218. doi: 10.1007/BF00418660. [DOI] [PubMed] [Google Scholar]
- 34.Robinson DA. The use of control systems analysis in the neurophysiology of eye movements. Annu Rev Neurosci. 1981;4:463–503. doi: 10.1146/annurev.ne.04.030181.002335. [DOI] [PubMed] [Google Scholar]
- 35.Sylvestre PA, Choi JT, Cullen KE. Discharge dynamics of oculomotor neural integrator neurons during conjugate and disjunctive saccades and fixation. J Neurophysiol. 2003;90:739–754. doi: 10.1152/jn.00123.2003. [DOI] [PubMed] [Google Scholar]
- 36.Schlag-Rey M, Schlag J. Visuomotor functions of central thalamus in monkey. I. Unit activity related to spontaneous eye movements. J Neurophysiol. 1984;51:1149–1174. doi: 10.1152/jn.1984.51.6.1149. [DOI] [PubMed] [Google Scholar]
- 37.Tanaka M. Spatiotemporal Properties of Eye Position Signals in the Primate Central Thalamus. Cereb Cortex. 2006;17:1504–1515. doi: 10.1093/cercor/bhl061. [DOI] [PubMed] [Google Scholar]
- 38.Ipata AE, Gee AL, Goldberg ME, Bisley JW. Activity in the lateral intraparietal area predicts the goal and latency of saccades in a free-viewing visual search task. J Neurosci. 2006;26:3656–3661. doi: 10.1523/JNEUROSCI.5074-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Snyder LH, Batista AP, Andersen RA. Coding of intention in the posterior parietal cortex. Nature. 1997;386:167–170. doi: 10.1038/386167a0. [DOI] [PubMed] [Google Scholar]
- 40.Bisley JW, Goldberg ME. Neuronal activity in the lateral intraparietal area and spatial attention. Science. 2003;299:81–86. doi: 10.1126/science.1077395. [DOI] [PubMed] [Google Scholar]
- 41.Morris AP, Chambers CD, Mattingley JB. Parietal stimulation destabilizes spatial updating across saccadic eye movements. Proc Natl Acad Sci U S A. 2007;104:9069–9074. doi: 10.1073/pnas.0610508104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Sommer MA, Wurtz RH. A pathway in primate brain for internal monitoring of movements. Science. 2002;296:1480–1482. doi: 10.1126/science.1069590. [DOI] [PubMed] [Google Scholar]
- 43.Ibbotson M, Krekelberg B. Visual perception and saccadic eye movements. Curr Opin Neurobiol. 2011 doi: 10.1016/j.conb.2011.05.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.