Attention doesn’t slide: spatiotopic updating after eye movements instantiates a new, discrete attentional locus

Abstract

During natural vision, eye movements can drastically alter the retinotopic (eye-centered) coordinates of locations and objects, yet the spatiotopic (world-centered) percept remains stable. Maintaining visuospatial attention in spatiotopic coordinates requires updating of attentional representations following each eye movement. However, this updating is not instantaneous; attentional facilitation temporarily lingers at the previous retinotopic location after a saccade, a phenomenon known as the retinotopic attentional trace. At various times after a saccade, we probed attention at an intermediate location between the retinotopic and spatiotopic locations to determine whether a single locus of attentional facilitation slides progressively from the previous retinotopic location to the appropriate spatiotopic location, or whether retinotopic facilitation decays while a new, independent spatiotopic locus concurrently becomes active. Facilitation at the intermediate location was not significant at any time, suggesting that top-down attention can result in enhancement of discrete retinotopic and spatiotopic locations without passing through intermediate locations.

Introduction

One of the most effective strategies for coping with complex visual environments is deploying covert visuospatial attention to salient objects and locations. When attention is drawn to a location via either bottom-up (Posner, 1980) or top-down (Awh, Jonides, & Reuter-Lorenz, 1998) signals, visual processing is facilitated for stimuli appearing at the attended location. In the everyday world, frequent saccadic eye movements dramatically change retinal stimulation, seriously challenging our ability to sustain attention on a task-relevant location in the world. In fact, some studies have concluded that it is not possible to maintain attention on one spatial location while saccading to another (Deubel & Schneider, 1996; Hoffman & Subramaniam, 1995; Kowler, Anderson, Dosher, & Blaser, 1995). This deficiency is often cited as evidence supporting the premotor theory (Rizzolatti, Riggio, Dascola, & Umiltà, 1987), which hypothesizes that attention is merely a by-product of oculomotor planning and the very act of planning an eye movement precludes the ability to attend anywhere except the saccade target. However, more recent studies have demonstrated that attention and eye movements are dissociable (Awh, Armstrong, & Moore, 2006; Hunt & Kingstone, 2003; Juan, Shorter-Jacobi, & Schall, 2004), including a recent article by Golomb, Chun, and Mazer (2008) suggesting that the difficulty in sustaining attention at an external location across saccades derives from challenges inherent in updating spatiotopic (world-centered) representations, not oculomotor planning itself.

The early stages of visual processing are retinotopically organized; consequently, to maintain a spatiotopic locus of attention, either retinotopic attentional representations must be updated with each saccade, or attentional salience maps must reside in higher, spatiotopically organized maps. Golomb et al. (2008) demonstrated that visuospatial attention is natively maintained in retinotopic coordinates that can be dynamically updated with spatiotopic information with each saccade. However, this updating is neither automatic nor instantaneous. As a result, when attentional facilitation has been sustained at a particular location before a saccade, it lingers at that same retinotopic location for some period of time after the saccade, even though the saccade has rendered that location behaviorally irrelevant; this phenomenon is termed the retinotopic attentional trace (Golomb, Pulido, Albrecht, Chun, & Mazer, 2010).

Several characteristic features of the retinotopic attentional trace have been described: It is maximal during the first 100 ms following a saccade, after which it decays (Golomb et al., 2008); it can be revealed using reaction time (RT) and behavioral accuracy techniques (Golomb et al., 2008), as well as fMRI and ERP neuroimaging techniques (Golomb, Nguyen-Phuc, Mazer, McCarthy, & Chun, 2010); it is robust across several different types of attentional tasks (Golomb, Nguyen-Phuc, et al., 2010; Golomb, Pulido, et al., 2010); and it is specific to the retinotopic coordinate system—that is, there is no analogous spatiotopic attentional trace (Golomb et al., 2008). Interestingly, the decaying retinotopic facilitation can temporarily coexist with emerging facilitation at the task-relevant spatiotopic location (Golomb et al., 2008; Golomb, Nguyen-Phuc, et al., 2010; Golomb, Pulido, et al., 2010). However, the spatiotemporal dynamics of this transition have not been explored: Does this phenomenon reflect attention directed simultaneously toward two different loci or a single locus of attention relocating position?

Figure 1 illustrates several models that could account for the decaying retinotopic attentional trace and concurrent updating to task-relevant spatiotopic coordinates. These models are based on the idea that the native coordinate system for spatial attention is retinotopic, and when attention is sustained at a particular location in space, neurons representing the relevant retinotopic location become facilitated, increasing spontaneous firing rates (Luck, Chelazzi, Hillyard, & Desimone, 1997), synchronization with neighboring neurons (Fries, Reynolds, Rorie, & Desimone, 2001), and recurrent activity (Wang, 2001). When attention is shifted to a new retinotopic location, the recurrent architecture of the visual cortex prevents activity at the previously relevant retinotopic location from instantaneously returning to baseline firing levels, resulting in a temporary retinotopic attentional trace. The relevant question for this report is how that attentional locus then transitions to the correct, task-relevant spatiotopic location. One possibility is that the locus of attention “slides” continuously across the map to the new location (Fig. 1a). Another model involves a transient change in shape of the locus, such that it temporarily enlarges to contain both locations and then shrinks back to the new relevant location (Fig. 1b). Both models involve a single locus of attention that transitions from one location to another and results in transient facilitation along the path between the two locations. In contrast, a third type of model involves two discrete attentional loci that independently emerge and decay (Fig. 1c). Such a mechanism would be consistent with demonstrations that in the absence of saccades, attention can be split between two locations without facilitating the area in between (Awh, Anllo-Vento, & Hillyard, 2000; Irwin & Gordon, 1998).

Fig. 1.

Possible models of attentional updating. Black square indicates cued location; white dots and arrows indicate fixation and saccade positions. In the no-saccade case, an attentional locus (white circle) is deployed to the spatiotopic/retinotopic (SR) location of the cue. After the eye movement, attention is initially directed at the retinotopic (R) location and updates over time to occupy the correct spatiotopic (S) location. The three models depict different possible spatiotemporal dynamics of this transition. C, control location; I, intermediate location; C_i, intermediate-control location. a A single locus of attention slides progressively from retinotopic to spatiotopic locations. b A single locus of attention transiently expands to include both locations, then contracts back to the spatiotopic location. c The old locus of attention decays at the retinotopic location while a new, discrete locus of attention arises at the spatiotopic location

The present article is primarily concerned with differentiating the latter mechanism from the two former mechanisms by probing attention at an intermediate location positioned between the spatiotopic and retinotopic locations. As in the previous studies (e.g., Golomb et al., 2008), subjects were instructed to remember and sustain attention at a task-relevant spatiotopic location while saccading to a different location; we then probed attentional facilitation at several delays after the saccade at the spatiotopic and retinotopic locations, as well as at the intermediate location. Since previous work has demonstrated that attention will update to spatiotopic coordinates only when behaviorally relevant (Golomb et al., 2008), our task was heavily biased toward the spatiotopic location. Thus the present question is not whether attention will update in order to maintain a spatiotopic representation, but how. If behavioral facilitation emerges at the intermediate location at some point after the saccade, that would suggest a single locus of attention transitioning between the retinotopic and spatiotopic locations. On the other hand, observing facilitation simultaneously at both the spatiotopic and retinotopic locations, but not at the intermediate location, would support the idea of two discrete loci of attention.

A number of studies have explored how covert attentional transitions from one location to another when the eyes remain fixated and attention is shifted to a new spatial location. Although a few studies have shown evidence that facilitation passes through an intermediate location (Shulman, Remington, & McLean, 1979) or takes longer to “slide” across longer distances (Tsal, 1983), most reports have been more consistent with a “jumping” model where attention arises at a new location and diminishes at the old location (reviewed in Cave & Bichot, 1999; Chastain, 1992a, 1992b; Sperling & Weichselgartner, 1995; Yantis, 1988). In the context of saccadic updating or “remapping” in the absence of sustained attention (Duhamel, Colby, & Goldberg, 1992), a few studies have also investigated visual or neuronal sensitivity at intermediate locations, with mixed results. A recent psychophysical study showed evidence for transient effects at intermediate spatial locations, suggesting a spreading or shifting of activity (Melcher, 2007). However, neurophysiological evidence suggests otherwise; in certain brain areas, visual responses are found in the original receptive field and the remapped receptive field, but not along the path in between (Sommer & Wurtz, 2006). Because sustained and transient attention may engage different neural mechanisms (Corbetta & Shulman, 2002), and attentional updating seems to operate on a different timescale than does traditional remapping (Golomb et al., 2008; cf. Kusunoki & Goldberg, 2003), it is important to investigate how these updating dynamics evolve in the case of sustained visuospatial attention.

Method

Subjects

Twenty subjects participated in the main saccade task, and 18 subjects in the no-saccade control. Additional details about subjects and participation criteria are reported in the online Supplement.

Experimental setup

Stimuli were generated on a Macintosh G4 computer using the Psychtoolbox extension (Brainard, 1997) for MATLAB (The Mathworks, Natick, MA) and were presented on a 22-in. flat-screen CRT monitor. Subjects were seated at a chinrest 75 cm from the monitor. Eye position was monitored using an ISCAN eye-tracking system (ISCAN, Burlington, MA) recording pupil and corneal reflection position at 60 Hz.

Task and stimuli

The task (Fig. 2) was modified from Golomb et al (2008). Subjects initiated the trial by fixating on a dot that appeared at one of four possible fixation locations (arranged as the corners of an 8.3° × 8.3° square). After 500 ms, a square memory cue appeared. Subjects were instructed to covertly attend to the location of the memory cue and to remember its location in spatiotopic coordinates (absolute location) throughout the trial. After 200 ms, the memory cue was removed, and subjects continued fixating for 500 ms. The fixation dot then moved to a different fixation location, and subjects had 350 ms to execute a single accurate saccade and fixate this new location (while still attending/remembering the spatiotopic location of the memory cue).

Fig. 2.

Task: Example trial. Subjects maintained fixation on the white fixation dot, while a memory cue appeared briefly at another location. Subjects were instructed to hold this cued location in memory throughout the trial. The fixation dot then moved, and after completion of a saccade to the new fixation location, a probe stimulus (oriented bar) appeared after a variable delay (shown here in the control location; a schematic of all five locations is shown in the inset at bottom). Subjects made a buttonpress response to indicate probe orientation. A memory test stimulus then appeared, and subjects indicated whether it occupied the same spatiotopic location as the memory cue. Gray arrow indicating a saccade did not actually appear on screen. The stimulus configuration illustrated here represents one example cue–saccade configuration; configurations were randomly intermixed from the four possible fixation locations and nine possible stimulus locations

Once fixation was acquired at the new location, a probe stimulus appeared after a delay of 50, 250, or 400 ms. The probe was presented for 200 ms before being extinguished. The probe was a thin bar angled 45° to the left or right of vertical; subjects were instructed to report the orientation of the tilt (left or right) as quickly as possible by pressing one of two keypad buttons. The probe could appear at the task-relevant spatiotopic location of the cue, the retinotopic location of the cue, or a control location matched for visual eccentricity and spatial uncertainty. Probes could also appear at a location intermediate to the spatiotopic and retinotopic locations. Because this intermediate location was necessarily located at a shorter eccentricity from fixation, an additional intermediate-control probe location was included to match the eccentricity of the intermediate location.

After subjects reported the probe orientation, a memory test stimulus appeared, and subjects indicated whether it occupied exactly the same spatiotopic position as the memory cue. The difficulty of the memory task was adjusted using a staircase procedure to maintain performance around 75% correct (Golomb et al., 2008); this ensured that the memory task was challenging enough that subjects would actively maintain the spatiotopic location in working memory.

Each block of trials included 6 spatiotopic trials and 2 each of retinotopic, control, intermediate, and intermediate-control trials, for each of the three delays (total of 42 trials per block). As in previous studies (Golomb et al., 2008), this greater likelihood of spatiotopic probes served to further encourage subjects to maintain attention in spatiotopic coordinates. Probe delays, probe locations, and saccade directions (horizontal, vertical, and diagonal) were randomly intermixed and unpredictable. For each trial, fixation and cue positions were chosen at random from a list of possible configurations for the specified condition. If a trial was aborted due to a fixation break (greater than 2° deviation), subjects received an error signal, and the trial was repeated later in the same block with new positions chosen at random for the same delay and location. Subjects completed at least 18 blocks of the task, spread across two to three sessions on multiple days, for a total of at least 36 trials per condition (delay × position).

In separate experimental sessions, a no-saccade version of this task was also run in order to assess the degree of facilitation at the intermediate location in the absence of a saccade. This task was identical to the saccade task, except that the fixation dot never moved from its original location. Probes were presented at the same delays as in the saccade task, relative to the average saccade completion time. Probes could appear at the spatiotopic location (which was also the retinotopic location, since the eyes never moved), as well as at locations corresponding to the control, intermediate, and intermediate-control locations.

Analysis of attentional facilitation

RT for the probe orientation report was averaged separately for each subject, location, and delay and was submitted to random effects analyses. Trials on which the subject responded incorrectly (4.2% of the trials) or RT was greater than 1.5 s (1.8% of the trials) were excluded. Spatiotopic facilitation and retinotopic facilitation were calculated as the difference in RT when the probe appeared at the spatiotopic or retinotopic location, respectively, as compared with the control location; intermediate facilitation was calculated relative to the intermediate-control location. Bonferroni-corrected one-sample, two-tailed ttests were conducted at each delay to test whether spatiotopic, retinotopic, and intermediate facilitation were significantly greater than zero, indicating significantly shorter RTs than on control trials.

Results

Attentional facilitation at the spatiotopic, retinotopic, and intermediate locations is plotted across delays in Fig. 3a. Facilitation at the previously relevant, but currently irrelevant, retinotopic location of the cue was significant at the early delay, t(19) = 4.14, p = .001, but not at the middle, t(19) = 1.93, p = .069, or later, t(19) = 1.45, p = .163, delays. (Significance of multiple comparison tests was Bonferroni corrected, α = .05/9 = .006.) The decay of the task-irrelevant retinotopic attentional trace was accompanied by an increase in task-relevant spatiotopic facilitation. Spatiotopic facilitation was significant at the middle, t(19) = 3.08, p = .006, and later, t(19) = 4.15, p = .001, delays, but not at the early delay, t(19) = 1.32, p = .204. This pattern of task-irrelevant retinotopic facilitation dominating immediately after a saccade and task-relevant spatiotopic facilitation dominating later closely replicates the original Golomb et al (2008; e.g., Fig. 4b) findings.

Fig. 3.

Attentional facilitation results. Attentional facilitation is plotted as the difference in reaction time (RT) for probes appearing in the spatiotopic, retinotopic, and intermediate locations, as compared with the control and intermediate-control location baselines (zero). Positive values indicate attentional facilitation (RTs shorter than those at the control locations). a Attentional facilitation after saccade (n = 20); data are plotted as a function of probe delay. b Attentional facilitation in no-saccade task (n = 18). Insets illustrate sample probe locations colored according to the plot legends, with white indicating the control location. Intermediate and intermediate-control locations are shown as hashed lines. White and gray dots indicate final and previous fixation locations, respectively; a square indicates the cued location, and an arrow indicates the saccade. Error bars are standard errors of the means (SEMs); asterisks indicate facilitation significantly greater than zero (Bonferroni-corrected). The dashed gray line indicates baseline facilitation at the intermediate location in the no-saccade task; intermediate facilitation after the saccade never exceeded this baseline

The critical question for the present study was whether facilitation was present at the intermediate location during this transition. Facilitation was not significant at the intermediate location at any of the timepoints tested (early, t(19) = 1.89, p = .075; middle, t(19) = 0.75, p = .465; late, t(19) = 1.97, p = .064). Although none of these timepoints were significant, particularly after Bonferroni correction, it is possible that some low level of facilitation existed at this location, perhaps due to residual spatial spread from the focus of attention (Downing & Pinker, 1985). The results from the no-saccade control task were consistent with this idea: The intermediate location exhibited a small benefit, as compared with the intermediate-control location (Fig. 3b), although this did not reach significance, t(17) = 1.95, p = .07. Importantly, whatever small numerical benefit existed for the intermediate location after a saccade was steady over time and never exceeded that for the intermediate location in the absence of any eye movements.

Discussion

The present results replicate earlier descriptions of the retinotopic attentional trace: Even when the retinotopic location is not task relevant, attentional facilitation is strongest at the retinotopic location of the cue immediately after the saccade and then decays, leaving only task-relevant spatiotopic facilitation at later delays (Golomb et al., 2008; Golomb, Nguyen-Phuc, et al., 2010). The present study probed an intermediate location between the spatiotopic and retinotopic locations to determine whether a single locus of attentional facilitation transitions from retinotopic to spatiotopic locations (either by transiently enlarging to encompass both or by sliding between the two) or whether retinotopic facilitation decays concurrently with activation of a new spatiotopic locus.

Facilitation at the intermediate location was not significant at any of the delays tested. Moreover, the small magnitude of intermediate facilitation was similar in the absence of the saccade and across all saccade delays, suggesting that if intermediate facilitation was present, it was likely not the result of any sort of dynamic updating related to the transition from retinotopic to spatiotopic locations after the saccade. Indeed, at the 250-ms delay, when spatiotopic facilitation had already emerged and retinotopic facilitation was still decaying, it is clear that the locus of attention had neither enlarged nor shifted to encompass the intermediate location. Rather, the results are most consistent with a model based on two discrete attentional loci, one at the retinotopic location, which is in the process of decaying, and another that has updated to the spatiotopic location to maintain task relevance.

In the present experiment, subjects were encouraged to maintain attention at a spatiotopic location using both explicit (spatiotopic memory task) and implicit (greater spatiotopic probe likelihood) cues. Previous work has demonstrated that without these strong task relevance cues, attention does not update to the spatiotopic location and, instead, remains in native retinotopic coordinates (Golomb et al., 2008). Fortunately, in the real world, analogous expectations and biases serve to similarly reinforce spatiotopic representations, enabling us to efficiently navigate our worlds. The present study was not designed to revisit the question of whether and when we can maintain spatiotopic representations—a debate that has been addressed more thoroughly in previous studies (Golomb et al., 2008; Golomb, Nguyen-Phuc, et al., 2010; Golomb, Pulido, et al., 2010). Rather, the question here is more focused on real-world functionality: When spatiotopic representations are task relevant, how are they maintained across eye movements, given that attention must update from the former retinotopic location? The finding of discrete facilitation at the previously relevant (retinotopic) and currently relevant (spatiotopic) locations, but not at locations intermediate between the two, argues strongly against a single sliding attentional locus.

The lack of intermediate location effects is also consistent with reports that attention moves discretely between locations in the absence of saccades (Cave & Bichot, 1999; Chastain, 1992a, 1992b; Sperling & Weichselgartner, 1995; Yantis, 1988), despite earlier work suggesting that attention slides through intermediate locations (Shulman et al., 1979; Tsal, 1983). These findings also help reconcile conflicting reports on the spatiotemporal dynamics of saccadic remapping, supporting physiological evidence for discrete transitions (Sommer & Wurtz, 2006), but not a report describing perceptual effects of remapping for tilt aftereffects at intermediate locations (Melcher, 2007). However, it should be noted that a more recent study failed to replicate the finding of spatiotopic tilt aftereffects (Knapen, Rolfs, Wexler, & Cavanagh, 2010), making the purported intermediate effects in the previous report difficult to interpret. Finally, the absence of intermediate facilitation is supported by previous demonstrations that attention can be split between two locations without facilitating the intervening region (Awh et al., 2000; Irwin & Gordon, 1998; cf. Jans, Peters, & De Weerd, 2010).

Although facilitation at intermediate locations never reached significance, there may have been a weak benefit at this location, perhaps reflecting the fact that this location was near enough to both the retinotopic and spatiotopic loci to receive some fringe attentional benefits as the loci tapered off in space (Downing & Pinker, 1985). In other words, although Fig. 1c depicts narrow attentional spotlights around attended locations, in reality the attentional spotlight may have a larger spatial extent, such that the intermediate location falls within its bounds. The intermediate location was positioned at least 4.15° (center-to-center distance) from both the spatiotopic and retinotopic locations, which was the maximum achievable separation in our experimental setup; however, one could imagine that with larger separation, the intermediate facilitation would further weaken. In addition to this theoretical possibility, facilitation at the intermediate location may have actually been overestimated, due to a nuance of the intermediate-control location. Both the intermediate-control and the regular control locations were matched perfectly for eccentricity from fixation, which we believed to be the most important confound to avoid (Carrasco, Evert, Chang, & Katz, 1995). The regular control location also controlled for distance from the cue (Golomb et al., 2008), but this was not possible for the intermediate-control without sacrificing other key aspects of our design (e.g., not having the intermediate-control appear directly in the path of the saccade). Thus, if anything, we are overestimating attentional benefits at the intermediate location.

The present results provide further evidence demonstrating that updating to spatiotopic coordinates depends on two discrete processes: activation at the newly relevant location and deactivation or decay at the previously relevant location. This idea has been proposed previously, both in the context of remapping transient visual representations (Kusunoki & Goldberg, 2003) and in the context of updating attentional representations (Golomb, Nguyen-Phuc, et al., 2010; Golomb, Pulido, et al., 2010), as well as for shifting attentional loci at fixation (Cave & Bichot, 1999; Chastain, 1992b; Sperling & Weichselgartner, 1995). In the absence of sustained attention, both of these processes can occur before the saccade is completed (Kusunoki & Goldberg, 2003); however, when attention is maintained at a location before a saccade, facilitation at the previous location—the retinotopic attentional trace—does not decay until 100–200 ms after completion of the saccade (Golomb et al., 2008). This timing is consistent with the temporal dynamics of shifting attention in the absence of saccades, where attentional effects arise at the new location before decaying at the old location (Khayat, Spekreijse, & Roelfsema, 2006). The existence of two distinct mechanisms is also supported by findings on inhibition of return (IOR), which can occur at both retinotopic and spatiotopic locations after a saccade (Posner & Cohen, 1984; Sapir, Hayes, Henik, Danziger, & Rafal, 2004). Interestingly, following parietal lobe impairment, IOR does not update to the spatiotopic location and remains only at the retinotopic location (Sapir et al., 2004; van Koningsbruggen, Gabay, Sapir, Henik, & Rafal, 2010).

The present results demonstrate that following a saccade, attentional representations concurrently update to the task-relevant spatiotopic location and decay at the previous retinotopic location, without spreading or transitioning to intermediate locations.Our lack of facilitation at the intermediate location, combined with the existence of the retinotopic attentional trace, suggests that visuospatial attention is represented in retinotopically organized salience maps that must dynamically update with each saccade when the spatiotopic location is task relevant. Instead of a single locus of attention that moves to the new location, facilitation at the old retinotopic location decays while an updated representation emerges at the spatiotopic location.