The Native Coordinate System of Spatial Attention Is Retinotopic

Abstract

Visual processing can be facilitated by covert attention at behaviorally relevant locations. If the eyes move while a location in the visual field is facilitated, what happens to the internal representation of the attended location? With each eye movement, the retinotopic (eye-centered) coordinates of the attended location change while the spatiotopic (world-centered) coordinates remain stable. To investigate whether the neural substrates of spatial attention reside in retinotopically and/or spatiotopically organized maps, we used a novel gaze-contingent behavioral paradigm that probed spatial attention at various times after eye movements. When task demands required maintaining a spatiotopic representation after the eye movement, we found facilitation at the retinotopic location of the spatial cue for 100–200 ms after the saccade, although this location had no behavioral significance. This task-irrelevant retinotopic representation dominated immediately after the saccade, whereas at later delays, the task-relevant spatiotopic representation prevailed. However, when task demands required maintaining the cue in retinotopic coordinates, a strong retinotopic benefit persisted long after the saccade, and there was no evidence of spatiotopic facilitation. These data suggest that the cortical and subcortical substrates of spatial attention primarily reside in retinotopically organized maps that must be dynamically updated to compensate for eye movements when behavioral demands require a spatiotopic representation of attention. Our conclusion is that the visual system's native or low-level representation of endogenously maintained spatial attention is retinotopic, and remapping of attention to spatiotopic coordinates occurs slowly and only when behaviorally necessary.

Introduction

A remarkable feature of human vision is our ability to maintain stable percepts of the world despite retinal inputs that change dramatically with each eye movement. Neurons that can rapidly update or “remap” transient visual inputs to compensate for saccadic eye movements can facilitate this perceptual stability (Duhamel et al., 1992). Most remapping studies have examined the neural substrates and temporal dynamics of this effect when visual information is updated in the absence of sustained attention, with particular focus on presaccadic and perisaccadic time periods (Duhamel et al., 1992; Nakamura and Colby, 2002; Kusunoki and Goldberg, 2003; Merriam et al., 2003; Sommer and Wurtz, 2006). A fundamentally different question involves the consequences of an eye movement on the representation of a spatial location facilitated by sustained, top-down attention, an ecologically relevant issue given the obvious advantages of being able to maintain a stable attentional representation across eye movements. However, the question of how attentional topography and attentional salience maps (i.e., sustained loci of endogenous attention) reorganize after eye movements remains largely uninvestigated.

When spatial attention is directed to a location in external space (spatiotopic or world-centered location) and the eyes move elsewhere while that location remains behaviorally relevant, the attended, spatiotopic location will fall on a different part of the retina; that is, it will occupy a new retinotopic, or eye-centered, location. Thus, retinotopically organized salience maps must be updated in response to each eye movement to reflect the new retinotopic coordinates of the attended spatiotopic location. If the neural mechanisms involved in sustaining a locus of attention across eye movements resemble those involved in remapping transient visual stimuli, then attention should be rapidly updated, possibly even in advance of the eye movement, to maintain attention at task-relevant spatiotopic coordinates. However, if salience maps are maintained or reinforced through endogenous activity, e.g., via increased spontaneous firing (Luck et al., 1997) and/or synchronization with neighboring neurons (Fries et al., 2001), shifting of the attentional locus could take significantly longer. When the eyes move, even if the oculomotor plan triggers immediate remapping of the attended spatiotopic location, attentional facilitation at the original retinotopic location may take some time to decay, particularly if neurons are organized into highly recurrent, reverberatory circuits (Wang, 2001). Thus, facilitated responses at the retinotopic location of a cue may persist after a saccade, even when this location no longer carries behavioral significance.

To investigate attentional topography after saccades, we took advantage of the fact that spatial locations can be attentionally modulated in the absence of visual stimulation through working memory tasks (Awh et al., 1998; Kastner and Ungerleider, 2000). We used a novel behavioral paradigm in which subjects executed a series of guided eye movements while simultaneously maintaining a separate, cued location in working memory (Fig. 1). Attentional facilitation was assessed by analyzing reaction times (RTs) or discrimination accuracy to probe stimuli presented during the memory delay.

Figure 1.

Tasks and conditions, experiment 1. A, Example trial for saccade task. While subjects maintained fixation on the white fixation dot, a memory cue appeared briefly at another location. Subjects were instructed to hold this cued location in memory throughout the trial. The fixation dot then moved, and after completion of a saccade to the new fixation location, a probe stimulus (oriented bar) appeared after a variable delay in either the spatiotopic (top), retinotopic (middle), or control (bottom) location of the cue. Subjects made a button press response to indicate probe orientation. A memory test stimulus then appeared, and subjects indicated whether it occupied the same spatiotopic location as the memory cue. B, In the no-saccade task, subjects remained fixated on the original location for an equivalent amount of time, so that probe delays relative to the cue were consistent across tasks. Probes appeared in the spatiotopic/retinotopic (top) or control (bottom) locations. C, In the return-saccade task, subjects held fixation at the second location for 750 ms before making a return saccade back to the first fixation location. Probes appeared at the spatiotopic/retinotopic (top), updated retinotopic (middle; reported in supplemental material, available at www.jneurosci.org as supplemental material), or control (bottom) locations; probe delays were the same as in the saccade task relative to onset of the final fixation. Gray arrows indicating saccades did not actually appear on screen. Stimulus configurations illustrated here represent only one of 108 possible cue–saccade configurations for each task (for additional examples, see supplemental Fig. 1, available at www.jneurosci.org as supplemental material).

Materials and Methods

Subjects

Sixteen subjects were tested in each experiment, with the exception of experiment 3 (12 subjects). Some subjects participated in multiple experiments; a total of 38 subjects (23 women; mean age, 20.9 years; range, 18–32 years) with normal or corrected-to-normal vision completed at least one of the experiments described below. Additional subjects were recruited but were unable to complete an experiment for the following reasons: five subjects had difficulty maintaining fixation with the eye-tracker and/or making the directed saccades without fixation errors, four subjects asked not to continue after the first or second session because of time constraints and/or lack of interest, and seven subjects did not meet the “no-saccade” prescreening criteria (described below). Informed consent was obtained from all subjects, and the study protocol was approved by the Human Investigation Committee of the School of Medicine and the Human Subjects Committee of the Faculty of Arts and Sciences at Yale University. Subjects were compensated with a monetary sum for their time.

Experimental setup

Stimuli were generated on a Macintosh G4 computer (Apple Computers) using the Psychtoolbox extension (Brainard, 1997) for Matlab (Mathworks) and were presented on a 19 inch flat-screen cathode-ray tube monitor. Subjects were seated at a chinrest positioned 78 cm from the monitor, so that the entire display subtended 23.9 × 17.9° of visual angle. The experiments were performed in a dark room, with ambient illumination from the monitor and eye-tracking equipment. Eye position was monitored using an ISCAN eye-tracking system recording pupil and corneal reflection (CR) position at 60 Hz; if gaze angle (pupil-CR) at any point during fixation deviated >2° from the center of the fixation dot, subjects received an error signal, and the trial was aborted and repeated later in the run.

Stimuli

White fixation dots (0.12° in diameter) appeared at one of four possible fixation locations, arranged as the corners of a 6.5 × 6.5° square centered on the monitor. Saccades could thus be directed to a location 6.5° distanced in the vertical direction, 6.5° distanced in the horizontal direction, or 9.2° distanced along the diagonal direction (see supplemental Fig. 1, available at www.jneurosci.org as supplemental material). Selection of fixation and saccade targets from among the four possible locations was randomized. Memory cue (0.35 × 0.35° black square frame), probe (0.49 × 0.05° black bar oriented 45° right or left of vertical), and memory test (0.35 × 0.35° black square containing a question mark) stimuli appeared in one of nine possible stimulation regions. The regions in which the cue, probe, and test stimuli appeared were distinct from the fixation locations, and each region contained four possible stimulation positions (see Fig. 6A); the separation distance between the four positions in a region was adjusted after each run to maintain a constant level of difficulty, as described later. Selection of cue, probe, and test locations for each trial was dependent on task and condition.

Figure 6.

Memory test performance, experiment 1. A, Arrangement of possible fixation locations (white dots) and cue/probe locations (black squares). Each of the nine cue/probe regions contained four possible positions; the memory test stimulus always appeared in the same region as the cue. Separation distance between test positions was adjusted after every run to keep performance ∼75% correct. Separation distance was increased after blocks of poorer performance and decreased after blocks of better performance. Inset illustrates relationship of test positions within a region: S, same position as the cue; ND, near different position in the adjacent horizontal or vertical direction; and FD, far different position along the diagonal. B, Task difficulty is plotted for each of the three tasks, given as the separation distance in degrees of visual angle required to correctly perform the memory task 75% of the time. Larger values indicate increased difficulty. C, Accuracy on the memory task when equated for difficulty. Percentage correct is given when the test stimulus appeared in the same, near different, and far different locations for each of the three tasks. Near different positions should be more difficult to correctly reject because of their close proximity to the cued location. Error bars indicate SEM after normalization to remove between-subject variability (Loftus and Masson, 1994); n = 16.

Experiment 1

“Saccade” task.

Figure 1A illustrates an example of the trial progression for the saccade condition of experiment 1. Subjects initiated each trial by fixating on the fixation dot. After 500 ms, the memory cue appeared. Subjects were instructed to covertly attend to the location of the cue and to hold its spatiotopic coordinates (“exact screen location”) in memory throughout the trial. After 200 ms, the memory cue was removed, and subjects continued fixating for 500 ms. The fixation dot then moved to a new fixation location, and subjects had 350 ms to execute a single accurate saccade and fixate this new location (while still attending/remembering the spatiotopic location of the memory cue). Until the fixation dot jumped to the new location, subjects had no advance knowledge of the direction of the upcoming saccade; neither the first fixation nor the memory cue location was predictive of saccade direction, and the entire task design was fully counterbalanced.

Once fixation was acquired at the new location, a probe stimulus appeared after a delay of 75, 150, 250, 400, or 600 ms. (An additional experiment with shorter delays was also conducted with probes appearing at 0, 25, 50, or 75 ms.) Subjects were instructed to report the orientation of the probe (left or right) as quickly as possible by pressing one of two keypad buttons. The probe could appear in the same spatiotopic location as the memory cue, the same retinotopic location (shifted in world- or head-centered coordinates), or a control location located at the same retinal eccentricity as the original cue, relative to the current fixation (further details on the selection of control locations are available at www.jneurosci.org as supplemental material). Half of the trials probed the memory task-relevant “spatiotopic” condition, and the other half were evenly divided between “retinotopic” and “control” conditions. Thus, subjects were implicitly encouraged to maintain the cue in spatiotopic coordinates in addition to the explicit instructions received at the start of testing. Probes appeared at eccentricities of ∼5° (±1° jitter) or 11° (±1° jitter) from fixation; eccentricities were balanced across conditions. Spatiotopic, retinotopic, and control locations were always separated by at least 6.5°. The probe was presented for 200 ms before being extinguished.

After subjects reported the probe orientation, there was a 750 ms blank interval before the fixation dot disappeared (freeing subjects to move their eyes) and a separate memory test stimulus appeared. Subjects were instructed to indicate by button press whether or not the memory test stimulus occupied the same exact spatiotopic position as the memory cue. The test stimulus always appeared in the same region as the cued location; 50% of the time it appeared in the same exact position (“same”), and 50% of the time it appeared in one of the three neighboring positions (“different”). Subjects were instructed to take their time and focus on accuracy when responding to the memory test. The test stimulus remained on the screen until a response was made. Auditory feedback for the memory task was given after each trial. To ensure that the memory task was sufficiently challenging for each subject and thereby ensure that subjects would actively maintain the spatiotopic location in working memory, the distance separating the four neighboring positions within a region was adjusted after each run to keep performance in the range of 65–85% correct. The initial distance separating the memory test positions was set at 1.2°, and distances on subsequent runs were increased or decreased in steps of 0.12°. If memory performance on a given run was between 85 and 90% correct, the separation distance was decreased by one step for the next run to make the task more challenging. If performance exceeded 90%, the distance was decreased by two steps. Likewise, performance between 60 and 65% correct or <60% correct increased the distance by one step or two steps, respectively, to make the task easier. The separation distances ranged from 0.58° to a maximum distance of 1.99°. A blank interval of 2000 ms preceded the beginning of the next trial.

“No-saccade” task.

As a baseline control task to assess attentional facilitation in the absence of eye movements, subjects in experiment 1 also performed no-saccade trials (Fig. 1B). These trials were similar to the saccade trials of experiment 1, except that the fixation dot remained in its original location for a period of time equal to planning and executing a saccade (∼250 ms). The probe then appeared after the same variable-length (75–600 ms) probe delay period; probes thus appeared at the same delays relative to the cue onset in both saccade and no-saccade trials. The probe appeared in the same spatiotopic location as the memory cue (which was also the same retinotopic location, because the eyes never moved) on 50% of trials; on the remaining trials it appeared in a control location of equal eccentricity.

“Return-saccade” task.

Subjects were also run on a return-saccade task (Fig. 1C) as part of experiment 1. While covertly attending to the cued location, subjects executed a guided saccade exactly as in the saccade task, followed by a second guided saccade returning their eyes to the original fixation location before the probe appeared. In these trials, timing was the same as in the saccade task, with the addition of the second guided saccade 750 ms after the initial saccade was completed. Once the return saccade was completed, the probe appeared after the same variable delays. The probe locations were chosen based on the final saccade. In 50% of trials the probe appeared in the same spatiotopic location as the memory cue, which was also its original retinotopic location; in 25% of trials it appeared at the same retinotopic location as the cue relative to the second fixation (“updated retinotopic” location; reported in a supplemental analysis, available at www.jneurosci.org as supplemental material), and in the remaining 25% it appeared in a control location of equal eccentricity from the final fixation.

Experiment 2: Accuracy version

A modified version of the saccade task was run using probe accuracy as a measure of attentional facilitation instead of RT. In this experiment, probes were presented at delays of either 75 ms or 400 ms. To make the probe discrimination more difficult, the probe was replaced by a masking stimulus after 20–200 ms. The stimulus onset asynchrony (SOA) between the probe and mask was adjusted after each run using an adaptive procedure to keep performance ∼75% correct. SOA adjustments were made separately for the small and large retinal eccentricity probes within each run. The mask was the same size as the memory cue (0.35 × 0.35°) and completely covered the probe. It was composed of randomly oriented and positioned lines and provided no information about probe orientation. Subjects were instructed to respond as accurately as possible to the probe by pushing one of two buttons indicating its orientation.

Experiment 3: Retinotopic emphasis

In the third experiment, subjects were run on a version of the saccade task described in experiment 1, except now the retinotopic location of the memory cue was task-relevant, instead of the spatiotopic location. Here subjects were instructed to remember the cue's “location relative to the eyes”, instead of the “exact screen location”, and the memory test stimuli appeared in the retinotopic, instead of spatiotopic, region of the cue. In half of the trials the memory test appeared at the exact retinotopic location of the cue (“same”); the remainder appeared at one of the three neighboring positions (“different”).

Experiment 3 used the same probe delays (75, 150, 250, 400, 600 ms) and trial timing as experiment 1. Probes were presented unmasked for 200 ms, and subjects were instructed to respond as quickly as possible to indicate the orientation of the probe. The retinotopic location was probed 50% of the time, compared with 25% for spatiotopic and control. Thus, the cue was again 50% valid for the task-relevant location.

Task design

Trials for each task and experiment were blocked and run in separate sessions occurring on separate days; within each session, probe delays and locations were intermixed. Subjects participated in multiple sessions for each task, ensuring that each delay × condition was probed in a minimum of 30 trials for each subject. Trial order was randomized, and the fixation dot and memory cue positions were chosen at random from a list of possible configurations for the specified condition. If a trial was aborted because of a fixation break, it was replaced later in the same run with new positions chosen at random for the same delay and condition.

Practice and prescreening

At the beginning of each session, subjects performed a calibration task for the eye-tracker, followed by a brief fixation practice task, which was used to verify the accuracy of the calibration. This procedure was repeated if the calibration was inaccurate or became inaccurate at any point during the session. On the first session of each task, subjects completed two short practice sessions (10 trials each) before beginning the actual task. The first practice session was a simplified version of the task without the probe and corresponding orientation discrimination. The purpose was to familiarize the subject with the basic fixations and/or saccades, and to stress the importance of the memory task. The second practice session was the full version of the task.

Because we were primarily interested in how representations of sustained spatial attention change after saccades, we wanted to ensure that attention was deployed to and maintained at the cued location before the saccade. Therefore, each subject first performed one session of the no- saccade task as a prescreening session. If subjects did not demonstrate clear facilitation at the location being held in memory, they were run in a second prescreening session. Subjects who still did not show facilitation were not brought back for saccade or return-saccade sessions.

Analysis of attentional facilitation

Behavioral measures of attentional facilitation were calculated separately for each subject and submitted to random effects analyses. For experiments 1 and 3, we calculated the subjects' average RTs for each condition and delay. RTs >1500 ms were excluded (<1% of all trials). Trials in which the subject responded incorrectly on the probe orientation task (∼2% of trials) were also excluded. For experiment 2, no trials were excluded, and task accuracy was analyzed instead of RT.

To assess attentional facilitation, we took the differences in RT/accuracy when the probe appeared in the spatiotopic or retinotopic locations compared with the control location baseline. The control baseline was calculated independently for each subject and delay, and the spatiotopic and retinotopic values were each compared with the same baseline, because all three conditions were equated for visual eccentricity. Difference values were calculated so that positive RT/accuracy differences reflected attentional facilitation. Because probe conditions and delays were different across the three tasks of experiment 1, and different sets of subjects were used for the early delays and experiments 2 and 3, we conducted all analyses separately for each task and experiment. The values representing each subject's average RT/accuracy differences were submitted to repeated measures ANOVA with condition and delay as within-participant variables. Delay did not meet the assumption of sphericity, so all tests involving this factor were adjusted according to the Greenhouse-Geisser correction. If there were no significant main effects of or interactions with delay, we collapsed across delays and ran t tests to assess whether spatiotopic or retinotopic locations were significantly facilitated compared with the control baseline (0 ms RT difference). If there were effects of delay, we conducted post hoc t tests separately for each delay. All t tests were paired and two-tailed.

For the saccade task of experiment 1, retinotopic and spatiotopic benefits were also fit separately with exponential functions (RT = Ae^τ*^Delay + A₀) to determine the degree to which benefits were influenced by delay. To assess whether the time courses were well fit by the exponential function, we compared the sum of squared error (SSE) for the actual fits to the SSEs obtained for fits of 1000 permutations of the raw data in which delays were randomly reassigned (Manly, 2006). Significance values reported in the main text indicate the fraction of randomized permutations with SSEs as small or smaller than the actual data; in other words, the probability of getting a better fit by chance.

Analysis of memory test performance

To ensure that subjects were maintaining spatiotopic representations in experiment 1, we analyzed performance on the working memory task. Because overall accuracy on the memory test was fixed at ∼75% by the staircasing procedure, subjects should have been pushed to their individual thresholds of spatial working memory precision, and their pattern of errors should provide insight into how well they were maintaining the spatiotopic representations. Memory test accuracy was compared for trials in which the test stimulus appeared in the same position and for two types of different positions: when it appeared immediately adjacent to the same position in either the vertical or horizontal direction (“near different”), and when it appeared across both directions at the farther diagonal position (“far different”). If subjects were simply guessing randomly on error trials, there should be no difference in accuracy for these two different conditions; however, if subjects were reliably maintaining spatiotopic representations with near-threshold precision, they should make more mistakes in the near different condition than in the far different condition, because the near different location is closer to and thus harder to discriminate from the same location.

We also analyzed memory test “difficulty” for each of the three tasks in experiment 1, as measured by the distance separating same and different test positions on the last run of the task. A larger separation distance indicates that greater memory probe displacement was required before subjects could reliably detect the difference; thus, separation is proportional to task difficulty.

Measuring fixation onsets

Fixation onset was defined as the time when the eye first crossed within 2° of the fixation point, although the eye could continue moving before reaching stable fixation. However, because there could be up to a 32 ms lag between actual and measured times due to latencies in the eye-tracker hardware, the delay values reported here reflect the best estimate of when stable fixation occurred. Subsequent off-line analysis of individual eye traces (data not shown) confirmed the accuracy of these fixation onset estimates.

Results

Attentional facilitation

Experiment 1 examined attentional facilitation when a spatiotopic location was to be endogenously sustained. To ensure that attention was successfully deployed to and sustained at the cued location before the eye movement, subjects were first run in the no-saccade task. Attentional facilitation (difference in RT compared with control location) is shown in Figure 2A for each delay; RT differences > 0 reflect attentional facilitation (raw RTs for all conditions are given in supplemental Table 1, available at www.jneurosci.org as supplemental material). There was no effect of probe delay (F < 1), and RTs were significantly faster when the probe appeared at the cued location compared with the control location (t₁₅ = 5.627, p < 0.001), confirming robust attentional facilitation for remembered locations in the absence of eye movements.

Figure 2.

Attentional facilitation: task-relevant spatiotopic location, experiment 1. Attentional facilitation is plotted as the difference in RT for probes appearing in the spatiotopic and retinotopic locations compared with the control location baseline (zero). Positive values indicate attentional facilitation (RTs faster than at control locations). Data are plotted as a function of probe delay. The right column illustrates sample probe locations for each experimental condition colored according to the plot legends on the left, with white indicating the control location. White and gray dots indicate final and previous fixation locations, respectively; squares indicate cued locations, and arrows indicate saccade patterns. A, No-saccade task. B, Return- saccade task. C, Saccade task. Error bars indicate SEM after normalization to remove between-subject variability (Loftus and Masson, 1994); n = 16.

To examine whether endogenous attention can be sustained at a spatial location while eye movements are executed elsewhere, we next ran the return-saccade task (Fig. 2B). Because the eyes returned to the original fixation location, both the spatiotopic and retinotopic coordinates of the cued location were preserved after the return saccade. As in the no-saccade task, there was no effect of delay (F < 1), and RTs were significantly faster at the cued location compared with the control location (t₁₅ = 4.475, p < 0.001), indicating that subjects were able to sustain attention at the cued location throughout the trial, despite a delay of several seconds that included the planning and executing of two intervening saccades.

These results demonstrate that spatial attention can be maintained across intervening saccades, which raises the question: In what coordinate frame of reference is the attentional facilitation maintained? To address this novel question, we examined facilitation in the saccade task (Fig. 2C), in which the probe stimulus could appear at the same spatiotopic location as the cue (task-relevant), the same retinotopic location as the cue (task-irrelevant), or a control location matched for retinal eccentricity relative to the final fixation location (i.e., the saccade endpoint). Both the spatiotopic and retinotopic locations were significantly facilitated over control (t₁₅ = 2.697, p = 0.017 and t₁₅ = 3.662, p = 0.002, respectively). Spatiotopic facilitation was expected, given that the spatiotopic location was the task-relevant location subjects were to be holding in memory. However, there was no behavioral advantage associated with facilitating the retinotopic location of the cue after the saccade.

There was no effect of condition (retinotopic vs spatiotopic; F < 1) or delay (F < 1), but there was a significant condition × delay interaction (F_4,60 = 3.379, p = 0.030), indicating a difference in the time course of facilitation between the retinotopic and spatiotopic conditions. At the shortest delay tested (75 ms), subjects were significantly faster to respond to a probe at the retinotopic location compared with the control (t₁₅ = 3.718, p = 0.002; 13 of 16 subjects showed this effect). In fact, this residual retinotopic benefit was so strong that it exceeded the task-relevant spatiotopic facilitation at this delay (t₁₅ = 2.361, p = 0.032). Retinotopic benefits relative to control locations were reduced, although still significant 150 ms after the saccade (t₁₅ = 2.320, p = 0.035), and were no longer significant by 250 ms post-saccade (all t values <0.800, p values >0.400). The time course of retinotopic facilitation was well fit by an exponential decay function (τ = 139.78 ms, p = 0.030). Spatiotopic facilitation did not systematically vary with delay (τ = 8.24 ms, p = 0.698).

An additional set of subjects was run in the same saccade task using shorter probe delays (0–75 ms) (Fig. 3). At these early time points, we found no significant main effect of delay (F_3,45 = 1.556, p = 0.218) and no delay × condition interaction (F_3,45 = 1.458, p = 0.248), suggesting that attentional topography was relatively stable over the first 75 ms after a saccade. Both retinotopic and spatiotopic locations were significantly facilitated over control (t₁₅ = 4.399, p = 0.001 and t₁₅ = 3.083, p = 0.008, respectively), and retinotopic facilitation was significantly stronger than spatiotopic facilitation during this early time frame (F_1,15 = 16.274, p = 0.001).

Figure 3.

Attentional facilitation: task-relevant spatiotopic location, early delays. Attentional facilitation is plotted as the difference in RT for probes appearing in the spatiotopic and retinotopic locations compared with the control location baseline (zero). Data are plotted as a function of probe delay. The right panel illustrates sample probe locations for the saccade task, colored according to the plot legends on the left, with white indicating the control location. Gray dot indicates initial fixation location, white dot is current fixation, and arrow indicates saccade pattern. Error bars indicate SEM after normalization to remove between-subject variability (Loftus and Masson, 1994); n = 16.

The results from experiment 1 suggest that a change in attentional topography takes place after a saccade when spatial attention must be updated to sustain a stable spatiotopic representation. Immediately after a saccade, attention is primarily maintained at the previously relevant retinotopic coordinates of the cue; after 100–200 ms, however, the retinotopic facilitation decays and the task-relevant spatiotopic coordinates dominate. To test this idea explicitly, we ran an additional set of subjects in experiment 2 using only probes presented at an early (75 ms) or late (400 ms) delay (Fig. 4A). Additionally, because many previous studies have characterized the effects of spatial attention using accuracy measures (Kowler et al., 1995; Deubel and Schneider, 1996; Hunt and Kingstone, 2003), in this experiment we used a measure of probe accuracy instead of RT.

Figure 4.

Attentional facilitation: task-relevant spatiotopic location, accuracy task. A, Experiment 2. Attentional facilitation is plotted as the difference in accuracy for masked probes appearing in the spatiotopic and retinotopic locations compared with the control location baseline (zero). Positive values indicate facilitation (accuracy better than at control locations). B, Experiment 1. RT data from the 75 ms and 400 ms probe delays of the saccade task replotted for comparison. Error bars indicate SEM after normalization to remove between-subject variability (Loftus and Masson, 1994); n = 16.

Consistent with the results of experiment 1, both the spatiotopic and retinotopic locations were facilitated over control (t₁₅ = 2.809, p = 0.013, and t₁₅ = 2.102, p = 0.053, respectively), there was a significant condition × delay interaction (F_1,15 = 21.722, p < 0.001), and there was no main effect of either condition (F < 1) or delay (F < 1). At the 75 ms delay, subjects were significantly more accurate when the probe appeared in the retinotopic location compared with both the control location (t₁₅ = 4.299, p = 0.001) and the spatiotopic location (t₁₅ = 2.762, p = 0.015). The accuracy benefit for the spatiotopic location over control was not significant at this early delay (t₁₅ = 1.012, p = 0.328). At the 400 ms delay, a strikingly different pattern was evident. The residual retinotopic benefit was no longer significant (t₁₅ = 1.198, p = 0.249), and the task-relevant spatiotopic location was significantly facilitated over both the control location (t₁₅ = 4.906, p < 0.001) and the retinotopic location (t₁₅ = 2.653, p = 0.018). This pattern of early retinotopic and late spatiotopic facilitation was thus reproducible across independent subject groups and detectable using both accuracy and RT measures (Fig. 4A,B).

Experiments 1 and 2 demonstrate a surprising new finding: immediately after a saccade, we found a robust and significant attentional benefit at the retinotopic location of a spatial cue, despite efforts to bias observers against using a retinotopic strategy. Instructions to subjects for the memory task were explicitly spatiotopic; in fact, spatiotopic representations were necessary to accurately perform the memory task, and probe stimuli were twice as likely to appear in the spatiotopic location as in either the retinotopic or control locations. Thus, subjects were both explicitly and implicitly biased toward using a spatiotopic strategy. Nonetheless, we observed consistent retinotopic facilitation immediately after the eye movement.

These findings suggest that the retinotopic coordinate system is the native frame of reference for spatial attention. To explore this idea further, in experiment 3, we modified the task to make the retinotopic location the task-relevant location and the spatiotopic location task-irrelevant (Fig. 5). In this experiment, subjects were explicitly and implicitly encouraged to use a retinotopic strategy. If the retinotopic system is indeed the native system and spatiotopic representations are computationally synthesized when needed, then when the retinotopic location is task-relevant, we would expect that (1) the residual retinotopic benefit should now persist across the entire delay period, and (2) there should be no residual spatiotopic benefit, even at early delays.

Figure 5.

Attentional facilitation: task-relevant retinotopic location, experiment 3. Attentional facilitation is plotted as the difference in RT for probes appearing in the spatiotopic and retinotopic locations compared with the control location baseline (zero). Data are plotted as a function of probe delay. The right panel illustrates probe locations for a sample configuration, colored according to the plot legend, with white indicating the control location. Gray dot indicates initial fixation locations, white dot is current fixation, and arrow indicates saccade pattern. The solid gray square represents the initial cued location; the dashed gray square represents the task-relevant retinotopic location to be held in memory. Error bars indicate SEM after normalization to remove between-subject variability (Loftus and Masson, 1994); n = 12.

Indeed, this is exactly what we found. There was no effect of delay over the 75–600 ms period probed (F < 1), nor was there a delay × condition interaction (F_4,44 = 1.156, p = 0.338). The retinotopic location was significantly facilitated over both control (t₁₁ = 3.327, p = 0.007) and spatiotopic (F_1,11 = 13.913, p = 0.003) locations across the entire delay period. Importantly, there was no spatiotopic facilitation when the spatiotopic location was not task-relevant (t₁₁ = 1.091, p = 0.299). Thus, spatiotopic facilitation can be abolished with a simple change of instructions, but the residual retinotopic facilitation is present regardless of behavioral relevance. Furthermore, when the retinotopic location was task-relevant, attentional facilitation here was so strong its magnitude approached that of attentional facilitation seen in the absence of an eye movement (34.9 vs 38.4 ms). This is in stark contrast to the task-relevant facilitation seen in experiment 1 at the spatiotopic location, which was much noisier and averaged only 13.7 ms. This suggests that the impaired spatial attention after eye movements reported in previous studies (Hoffman and Subramaniam, 1995; Kowler et al., 1995; Deubel and Schneider, 1996) may have less to do with oculomotor planning per se, but rather the specific coordinate transformations required to translate the sustained locus of attention in response to changes in eye position. In other words, the eye movement itself may not interfere with the ability to sustain a locus of attention, but the cost may instead arise from having to redirect attention to maintain spatiotopic representations.

Memory test performance

To determine the impact of saccade planning and execution on the maintenance of a spatial location in memory, we compared performance in the unspeeded working memory tests in the no- saccade, saccade, and return-saccade tasks of experiment 1 (Fig. 6). The memory discrimination was significantly easier in the no- saccade task than in both the saccade (t₁₅ = 6.964, p < 0.001) and return-saccade (t₁₅ = 5.938, p < 0.001) tasks. Interestingly, the return-saccade task was significantly easier than the saccade task (t₁₅ = 2.772, p = 0.014), although locations were maintained across more eye movements and longer intervals. Order effects could not account for this result (see supplemental Table 5 and Analyses, available at www.jneurosci.org as supplemental material).

It is not surprising that the memory task was easier on the simpler no-saccade task. However, if the act of planning a saccade interfered with the ability to attend/remember a location (Pearson and Sahraie, 2003; Lawrence et al., 2004), then we would expect the poorest performance on the return-saccade task, in which two saccades were executed during the memory delay; the fact that performance in the return-saccade task exceeded that in the saccade task provides further evidence that updating spatiotopic representations is more taxing to the attentional system than oculomotor planning itself.

Finally, to ensure that subjects were successfully holding the spatiotopic location in memory across multiple saccades and long delays, we analyzed accuracy as a function of test stimulus location (Fig. 6C). We conducted a 3 × 2, task × condition (task: no saccade, saccade, return saccade; condition: near different, far different) repeated measures ANOVA on the data shown in Figure 6C. The main effect of condition was significant (F_(1,15) = 313.411, p < 0.001), confirming that subjects' performance was worse in the near different condition than in the far different condition, regardless of task. The main effect of task was marginally significant (F_(2,30) = 3.700, p = 0.052; Greenhouse-Geisser corrected), suggesting that the tasks were not perfectly equated for difficulty. However, there was no significant condition × task interaction (F < 1), demonstrating that subjects were holding the location in memory similarly in all three tasks. This consistent pattern of errors illustrates that subjects truly were attempting and succeeding at maintaining these locations in memory, with and without intervening saccades. Furthermore, it demonstrates that subjects were indeed using spatiotopic representations for the memory task in experiment 1.

Discussion

The results from these three experiments reveal a novel property of spatial attention: that its native coordinate system is retinotopic. We report that an independent locus of sustained spatial attention is maintained primarily in retinotopic coordinates immediately after a saccade, even when task demands might dictate otherwise. This retinotopic facilitation may occur in addition to task-relevant facilitation at another location. Indeed, when the spatiotopic representation was enforced by behavioral means, a weak but sustained spatiotopic component of attentional facilitation coexisted alongside the transient retinotopic facilitation. In contrast, when the retinotopic location was task-relevant, no facilitation was detectable at the irrelevant spatiotopic location. In other words, we found evidence for a sustained component of attentional facilitation in whichever reference frame was task-relevant, and a transient component of facilitation in the retinotopic reference frame only.

At fixation, one cannot differentiate between the spatiotopic and retinotopic representations of a cue; only after an eye movement do these become dissociable, allowing comparison of attentional facilitation in these two coordinate systems. Numerous experimental studies have established a link between attention and eye movements: visual processing is enhanced at the target of an upcoming saccade (Hoffman and Subramaniam, 1995; Irwin and Gordon, 1998; Bisley and Goldberg, 2003; Gersch et al., 2004; Van der Stigchel and Theeuwes, 2005), and eye movements and covert shifts of attention evoke similar patterns of functional activity in the brain (Corbetta et al., 1998; Nobre et al., 2000; Schall, 2004; Eimer et al., 2007). It has even been suggested by the premotor theory that the central role of spatial attention is to plan upcoming saccades (Rizzolatti et al., 1987). Whereas some studies have supported this argument with demonstrations that it is not possible to simultaneously plan and execute eye movements to one location while successfully attending to a different location (Hoffman and Subramaniam, 1995; Kowler et al., 1995; Deubel and Schneider, 1996), others have demonstrated clear dissociations between oculomotor and attentional systems (Hunt and Kingstone, 2003; Juan et al., 2004; Awh et al., 2006). The present results demonstrate that it is possible to maintain a separate locus of spatial attention while saccading elsewhere, although this does not imply that attention has no role in oculomotor planning. Our results are compatible with demonstrations of facilitated processing at saccade target locations, given that attentional facilitation can coexist in multiple locations (Irwin and Gordon, 1998; Awh and Pashler, 2000), and that multiple saccades can also be programmed concurrently (McPeek et al., 2000).

Our results demonstrate that saccades do not necessarily destroy facilitation of visual processing at attended or remembered spatial locations, but it may be more difficult to maintain spatiotopic (compared with retinotopic) representations across saccades, and thus, attentional facilitation may be weaker or less reliable at spatiotopic locations after eye movements. Because past studies of attention and eye movements have largely focused on probing spatiotopic locations, this could account for some of the discrepancies in the literature. Additionally, previous studies of attention and eye movements have typically relied on execution of highly predictable saccade patterns in which the location of each eye movement is known well in advance. Under these conditions, subjects might anticipate the upcoming eye movement and attentional maps may shift well in advance of the saccade. The current task takes a more naturalistic approach, using unpredictable saccade targets and requiring saccades to be executed immediately. As a result, there is less time to deploy attention toward the saccade target, which could increase attentional resources available at the cued location and result in a stronger attentional representation of the cue after the eye movement.

In summary, we report the presence of an independent locus of sustained spatial attention that is maintained primarily in retinotopic coordinates immediately after saccades. Importantly, the observed retinotopic facilitation occurs even when no behavioral advantage is accrued for enhanced processing at the retinotopic location of the cue. These results cannot be explained by preferential processing at, or perisaccadic mislocalization toward, the saccade target location (Ross et al., 1997; Currie et al., 2000; Lappe et al., 2000), because the retinotopic, spatiotopic, and control locations all appeared at equal distance from the saccade target.

Consistent with evidence of both retinotopic and spatiotopic integration of masked stimuli (Irwin et al., 1988; Watanabe and Suzuki, 1993), motion (Melcher and Morrone, 2003; Burr et al., 2007), visual form (Hayhoe et al., 1991), and facilitation/inhibition of return (Shimojo et al., 1996; Abrams and Pratt, 2000) across saccades, our results demonstrate that the two coordinate frames of reference may be prioritized differently according to task demands. The attentional system functions well across saccades when the native retinotopic representations are to be reinforced, but when spatiotopic representations must be actively maintained, endogenous attention cannot update as effectively. This finding is reminiscent of a bias toward retinotopic representations during infancy that precedes the acquisition of higher-order spatiotopic, body-centered representations over the course of development (Kaufman et al., 2006). Our results reveal that facilitation of a previously attended retinotopic location takes measurable time to decay once a saccade plan redirects attention to a new retinotopic location. Thus, retinotopic benefits persist even when the facilitated location is no longer task-relevant.

These data suggest a simple model of neural facilitation and persistence that operates primarily in a retinotopic frame of reference. Spatiotopic signals from other brain regions may help redirect attention to new retinotopic locations when behaviorally relevant, but our data suggest that the salience maps themselves are retinotopically organized. Topographic maps of visual space have been discovered across various regions of human cortex (Kastner et al., 2007; Saygin and Sereno, 2008), and many of these maps, particularly those in the occipital cortex, have been demonstrated to operate in a retinotopic reference frame (Gardner et al., 2008). A number of these human regions may also update their representations of transient visual input after a saccade (Medendorp et al., 2003; Merriam et al., 2003, 2007), presumably based on feedback from efferent oculomotor signals (Sommer and Wurtz, 2006) and/or spatiotopically organized parietal areas (Duhamel et al., 1997; Sereno and Huang, 2006). In the absence of sustained endogenous activity, spatiotopic representations can update very rapidly, even before a saccade is completed (Duhamel et al., 1992; Nakamura and Colby, 2002; Kusunoki and Goldberg, 2003; Merriam et al., 2003). Our finding of persistent retinotopic facilitation after saccade completion suggests that different neural mechanisms are involved in sustaining a locus of attention (where recurrent, reverberatory activity that has built up before the saccade may not decay until well after the saccade), as opposed to responding to transient visual stimuli (where the information may be briefly represented and rapidly updated across saccades).

Physiological studies of the temporal dynamics of switching attention have demonstrated that when the eyes remain fixated while attention is shifted to a new spatial location, attentionally modulated neurons in early visual cortex facilitate the new location before disengaging from the previously attended location (Khayat et al., 2006). Our results suggest that when attention is to be maintained across eye movements, an attentional switch only occurs when spatiotopic, not retinotopic, representations must be maintained. In other words, to maintain a spatiotopic location, a new population of neurons must become facilitated after the eye movement, and the previously facilitated neurons take time (100–200 ms) to return to an unfacilitated state. However, when behavioral demands require maintaining a location in the native retinotopic coordinates, attention need not be shifted within the map, and there is no evidence of facilitation at the spatiotopic location. Based on these observations, we propose that the neural substrates of spatial attention reside primarily in retinotopically organized maps, and thus, spatial attention is encoded and maintained in retinotopic coordinates, regardless of task demands.