Retinotopic memory is more precise than spatiotopic memory

Abstract

Successful visually guided behavior requires information about spatiotopic (i.e., world-centered) locations, but how accurately is this information actually derived from initial retinotopic (i.e., eye-centered) visual input? We conducted a spatial working memory task in which subjects remembered a cued location in spatiotopic or retinotopic coordinates while making guided eye movements during the memory delay. Surprisingly, after a saccade, subjects were significantly more accurate and precise at reporting retinotopic locations than spatiotopic locations. This difference grew with each eye movement, such that spatiotopic memory continued to deteriorate, whereas retinotopic memory did not accumulate error. The loss in spatiotopic fidelity is therefore not a generic consequence of eye movements, but a direct result of converting visual information from native retinotopic coordinates. Thus, despite our conscious experience of an effortlessly stable spatiotopic world and our lifetime of practice with spatiotopic tasks, memory is actually more reliable in raw retinotopic coordinates than in ecologically relevant spatiotopic coordinates.

To catch a ball, reach for a cup of coffee, or find a friend in a crowd, we need to first determine the object's location. However, a fundamental challenge complicates this seemingly effortless task: visual input arrives at the eyes in retinotopic (i.e., eye-centered) coordinates, but visually guided behavior requires information about spatiotopic (i.e., world-centered) locations. How do we adapt retinotopic input to support spatiotopic behavior?

One possibility is that we simply “act in the moment” and recreate the visual world anew with each fixation (1, 2). However, this option is feasible only in cases in which visual information is constantly present; it would clearly fail in cases in which something must be remembered, attended, or compared across an eye movement. When we do need to maintain spatial information across an eye movement, it is an object's location in the world, not its location on our retinae, that is generally relevant for behavior. To keep track of real-world object locations across eye movements, we can imagine two types of solutions. The first possibility is that, in addition to early retinotopic maps, there also exists somewhere in the brain a “hard-wired spatiotopic” map.^* This idea is appealing because the initial retinotopic location of an object could be immediately translated into spatiotopic coordinates (using eye position information) and stored as a spatiotopic position. Thus, spatiotopic position would need to be computed only once,^† after which it would remain stable regardless of subsequent changes in eye position. Indeed, spatiotopic effects have been reported behaviorally (3–7) and physiologically (8). However, these effects do not necessarily require an explicit hard-wired spatiotopic map, and evidence for such large-scale, explicit spatiotopic neural organization has remained controversial (9–14). An alternative “retinotopic plus updating” solution is to only maintain information in retinotopic coordinates, but to update this retinotopic information with each eye movement based on information about eye position (15), corollary discharge from the eye movement (16, 17), and/or stable visual cues in the environment (18, 19). Of course, it is possible that retinotopic-plus-updating processes in some brain areas might coexist with hard-wired spatiotopic maps in others, but it is also possible that stability could be achieved on the basis of retinotopic-plus-updating alone.

Most current theories of visual stability favor some form of retinotopic-plus-updating, but they vary widely in how quickly, automatically, or successfully this updating might occur. For example, several groups have demonstrated that locations can be rapidly “remapped,” sometimes even in anticipation of an eye movement (20–22), but a recent set of studies has argued that updating requires not only remapping to the new location, but also extinguishing the representation at the previous location, and this latter process may occur on a slower time scale (23, 24). Moreover, it is an open question just how accurate we are at spatiotopic perception. If we do use retinotopic-plus-updating, does this process actually solve the stability problem and produce behavior as optimal as we would expect from a hard-wired spatiotopic system? We are not consciously aware of the world shifting with each eye movement, and intuitively we feel like our percept is based on a world-centered representation. However, might our percept of spatiotopic stability actually be somewhat of an illusion and the process of updating spatiotopic information not as efficient as it feels?

In the present experiment, we directly tested this counterintuitive prediction: that, despite our conscious experience of an effortlessly stable spatiotopic world and our lifetime of practice with spatiotopic tasks, people might actually be better at remembering raw retinotopic locations than more ecologically relevant spatiotopic locations. Subjects performed two sessions of a transsaccadic spatial memory task (Fig. 1): one in which they were asked to remember the spatiotopic (i.e., absolute) location of a cue, and another in which they were asked to remember the retinotopic location of the cue (i.e., relative to the eyes). Crucially, we asked not simply whether subjects could remember spatiotopic and retinotopic locations, but how precise this memory is. By measuring the accuracy of location memory after zero, one, or two eye movements, we tested whether spatial memory in either coordinate system was impaired by eye movements and, if so, whether memory performance continued to degrade with each subsequent shift in eye position. To tease apart saccade-related memory decrements from generic memory decay over time, we included two key comparisons that were matched for retention interval and number of saccades. First, we compared retinotopic and spatiotopic performance for each saccade condition, and second, we included two conditions involving multiple saccades: a standard two-saccade condition in which subjects saccaded to two different locations, and a return-saccade condition in which subjects returned to the original fixation location after the first saccade. Generic memory decay should manifest equally across all these conditions, whereas a selective impairment in performance in the retinotopic or spatiotopic task would provide insight into the mechanisms underlying transsaccadic behavior.

Fig. 1.

Experimental design. (A and B) Example trial progression: one-saccade condition, spatiotopic task. A peripheral memory cue (black square) was briefly presented while subjects fixated on the white fixation dot. During the memory delay, the fixation dot jumped to a new location, and subjects were required to saccade to the new fixation location. At 850 ms after completion of the saccade, the mouse cursor appeared over the fixation dot, cueing subjects to move the mouse to the remembered location. After subjects clicked the mouse to report the remembered location, a green square appeared where they clicked, and a black square appeared at the correct location for feedback. (A) Experiment 1. (B) Experiments 2 and 3. Here, when the fixation dot moved to a new location, the outline at the original fixation location remained visible. In experiment 3, a mask was presented after the memory cue. (C) Table illustrating example retinotopic and spatiotopic locations for all saccade conditions. For each row, a random cue location (black square) and initial fixation location (white dot) is shown in the cue figure. The corresponding correct spatiotopic and retinotopic locations are shown in green. In the no-saccade condition, the fixation dot does not move during the delay, and spatiotopic and retinotopic locations are identical. In the one-saccade condition, a single horizontal or vertical saccade (white arrow) is made, and in the two-saccade condition, a second saccade is made before reporting the remembered location. In the return-saccade condition, the second saccade is made back to the original location, realigning spatiotopic and retinotopic locations.

Intuitively, it is the spatiotopic locations that we would want to maintain better across eye movements, and in principle, a rapid and seamless retinotopic-plus-updating mechanism could result in performance indistinguishable from that of a hard-wired spatiotopic mechanism. In either of these cases, we would expect performance in the spatiotopic task to be accurate and relatively stable; i.e., it should not depend on the number of intervening eye movements, at least not beyond that expected from generic memory decay. However, although our world-centered percept feels stable, if spatial representations are supported by a retinotopic-plus-updating mechanism, there may be “noise” or costs involved with each transformation. If this process is not as seamless as our percept implies, then we predict (i) counterintuitively better precision for retinotopic than spatiotopic locations, and (ii) an accumulation of error for spatiotopic, but not retinotopic, locations with each saccade.

Results

Experiment 1.

Mean error is plotted for spatiotopic and retinotopic tasks as a function of saccade condition in Fig. 2A. On no-saccade trials, memory was equally accurate for spatiotopic and retinotopic tasks, as expected given that the tasks are identical in the absence of an eye movement. In both tasks, memory becomes less accurate following eye movements, but differentially more so for the spatiotopic task.

Fig. 2.

Accuracy and precision. (A–D) Experiment 1. (A) Memory accuracy (mean error magnitude in degrees of visual angle) for each condition and task. (B) Memory precision (SD of memory error) for each condition. (C) Accuracy for each condition, plotted only for the first 20 trials of the task (∼4 trials per condition per subject). (D) Accuracy shown separately for horizontal and vertical saccades (one-saccade condition, experiment 1). (E) Experiment 2: memory accuracy. (F) Experiment 3: memory accuracy. Smaller error magnitudes indicate greater memory fidelity. Note that, although only accuracy results are shown for C–F, precision measures revealed nearly identical plots in all cases. All error bars indicate SEM (n = 8).

A two-by-three [task (spatiotopic or retinotopic) by number of saccades (zero, one, or two)] repeated-measures ANOVA revealed a significant main effect of task [F(1,7) = 13.09; P = 0.009], indicating that the accuracy of memory for spatiotopic locations was significantly worse than for retinotopic locations. Overall, memory accuracy deteriorated with an increasing number of saccades [F(2,14) = 52.61; P < 0.001], and this deterioration was significantly steeper for spatiotopic locations than for retinotopic locations, as evidenced by the task–number of saccades interaction [F(2,14) = 17.08; P = 0.002]. Post-hoc t tests confirmed that retinotopic memory was significantly more accurate than spatiotopic memory after one saccade [t(7) = 2.69; P = 0.031] and two saccades [t(7) = 4.22; P = 0.004]. However, when spatiotopic and retinotopic tasks converged on the same location—i.e., when the fixation location was unchanged and the two coordinate systems could not be dissociated—performance was equivalent in the two tasks [no-saccade, t(7) = 0.18 and P = 0.863; return-saccade, t(7) = 1.74 and P = 0.126].

These same effects of memory accuracy (i.e., mean error) were also found for memory precision (i.e., SD of reported locations; Fig. 2B). Memory precision was significantly better for retinotopic than spatiotopic locations [main effect of task, F(1,7) = 10.48 and P = 0.014]. Memory precision also deteriorated with an increasing number of saccades [F(2,14) = 41.85; P < 0.001], and this deterioration was significantly steeper for spatiotopic locations than for retinotopic locations [task × number of saccades interaction, F(2,14) = 20.46 and P < 0.001].

The benefit for retinotopic memory over spatiotopic memory was independent of saccade direction (Fig. 2D). Although spatial memory after vertical saccades was slightly but significantly greater than after horizontal saccades [main effect of saccade direction, F(1,7) = 10.22 and P = 0.015], there was no task–saccade direction interaction (F < 1), and the significant main effect of task [F(1,7) = 7.22; P = 0.031] confirmed that retinotopic memory was more accurate than spatiotopic memory across both horizontal and vertical saccades.

Finally, we asked whether the retinotopic benefit might be acquired as the task became more familiar, or whether it was a more fundamental benefit. In other words, did subjects initially perform better at the more natural spatiotopic task, but improve over time on the retinotopic task as subjects were trained with repeated exposure and feedback? Our results suggest that retinotopic memory is more precise than spatiotopic memory after saccades, even before much training. Fig. 2C illustrates memory accuracy for the first 20 trials (i.e., first half of the first block) of each task. Even on this small, early subset of the data, there was again a significant task–number of saccades interaction [F(2,14) = 9.28; P = 0.004], and retinotopic memory was significantly more accurate than spatiotopic memory after one saccade [t(7) = 2.92; P = 0.022] and two saccades [t(7) = 3.43; P = 0.011]. Even more impressively, this benefit occurred despite the fact that the retinotopic task was initially harder in the no-saccade condition [t(7) = −1.64; P = 0.144], a trend that probably reflects the subjects’ initial unease with the retinotopic task. Thus, even during the period in which the retinotopic task was arguably the most unnatural and unfamiliar to subjects, their accuracy in reporting retinotopic locations after saccades still consistently exceeded their accuracy in reporting spatiotopic locations. Moreover, during this initial period, the spatiotopic error more than doubled (increasing by >1°) as the number of intervening saccades increased from zero to two, whereas retinotopic error remained strikingly flat, varying by less than 2% (0.02°).

Experiment 2.

It is possible that the fixation dot in experiment 1 could have served as a visual landmark in retinotopic coordinates that could partially explain the higher performance in the retinotopic task. To test this hypothesis, in experiment 2, we left the outline of the original fixation dot visible on the screen after the saccade to provide an analogous spatiotopic landmark. Fig. 2E illustrates mean error for retinotopic and spatiotopic tasks for the no-saccade and one-saccade conditions presented in this experiment; the results replicate those of experiment 1. Again, we found significant main effects of task [F(1,7) = 5.34; P = 0.054] and number of saccades [F(1,7) = 46.6; P < 0.001], and a task–number of saccades interaction [F(1,7) = 8.54; P = 0.022]. Post-hoc t tests revealed that retinotopic and spatiotopic memory did not significantly differ in the absence of a saccade [t(7) = −0.24; P = 0.814], but memory was significantly more precise in retinotopic than spatiotopic coordinates after a saccade [t(7) = 2.79; P = 0.027], even when a spatiotopic landmark was constantly present on the screen.

Experiment 3.

To verify that our pattern of results could not be explained by the presence of retinotopic afterimages, in experiment 3, a full-screen mask was presented after the initial cue. We also left the spatiotopic landmark visible, as in experiment 2. In general, the data were noisier in this experiment, and the presence of the full-screen mask made memorizing the location and maintaining fixation more difficult. However, the results are consistent with those of the initial experiment: spatiotopic error continued to increase with additional eye movements, whereas retinotopic error did not increase at all after a second saccade (Fig. 2F). The task–number of saccades interaction was significant, both when comparing across zero, one, and two saccades [F(2,14) = 4.29; P = 0.038] or just one and two saccades [F(1,7) = 9.70; P = 0.017]. In this experiment, there was not as clear of an overall retinotopic advantage, although this may be because one of the eight subjects had unusual difficulty performing the retinotopic task (responding in an entirely wrong region of the screen on 10% of the trials; Methods). Indeed, when this subject was excluded, the data were even more consistent with the previous versions: spatiotopic memory was significantly worse than retinotopic memory [F(1,6) = 6.19; P = 0.047], with a significant task–number of saccades interaction [F(2,12) = 6.68; P = 0.012].

Eccentricity Bias.

The analyses reported earlier focused on the magnitude of subjects’ memory errors in the two tasks. However, do these errors reflect random spread or a more systematic bias? Fig. 3A illustrates the average correct and reported target positions for each task and condition. For spatiotopic and retinotopic tasks, the reported location seemed systematically biased toward the initial fixation location (i.e., in the direction opposite the saccade), although this effect was much more pronounced in the spatiotopic task. It is often reported that eccentricity is underestimated as a result of a “foveal bias,” and the foveal bias is thought to originate at the time of encoding, such that underestimation occurs relative to the initial fixation location (25). To quantify these effects, we calculated foveal bias relative to the initial fixation location as [(measured eccentricity − true eccentricity) / true eccentricity]. We also explored alternative eccentricity biases (relative to the final fixation or screen center; SI Text and Fig. S1), although Fig. 3A suggests that the primary effect was driven by the initial fixation location. Analogous to the accuracy and precision results, foveal bias was greater in the spatiotopic task and amplified with increasing number of saccades, whereas, in the retinotopic task, the bias was smaller and relatively constant across saccade conditions (Fig. 3 B and C). A two-by-three [task (spatiotopic or retinotopic) by number of saccades (zero, one, or two)] ANOVA on the bias scores revealed significant main effects of task [F(1,7) = 30.2; P = 0.001] and number of saccades [F(2,14) = 14.15; P = 0.004], and a task–number of saccades interaction [F(2,14) = 5.36; P = 0.036], revealing that error not only accumulates in the spatiotopic condition, but accumulates in a systematic way.

Fig. 3.

Eccentricity bias. (A) Average correct and reported target locations are illustrated for each task and condition of experiment 1. Correct spatiotopic and retinotopic locations are indicated by dark blue and red crosses, respectively. Reported spatiotopic and retinotopic locations are indicated by cyan and orange crosses, respectively. Crosses indicate the average location across all trials, with the specific saccade direction for each trial aligned to the example saccade before averaging. These plots can reveal systematic biases (e.g., a foveal bias relative to the initial fixation location), but do not necessarily reflect overall differences in error magnitude (e.g., randomly distributed error would average zero). (B and C) Foveal bias (relative to initial fixation location) plotted for each condition and task in experiment 1 (B) and experiment 3 (C). Error bars indicate SEM (n = 8).

Discussion

These results reveal that memory for locations is significantly more accurate and precise in retinotopic than spatiotopic coordinates, and that spatiotopic – but not retinotopic – error accumulates with each eye movement. This finding is counterintuitive and surprising because successful human behavior requires information about objects’ world-centered spatiotopic positions, not the eye-centered positions that shift on the retina with each eye movement. It does not seem particularly adaptive to have a visual system that favors retinotopic over spatiotopic positions.

However, the results are consistent with a mechanism whereby locations are maintained in retinotopic coordinates and updated with each eye movement. If representations were instead maintained in hard-wired spatiotopic coordinates, we would have expected a pattern in which (i) precision was higher in the spatiotopic than retinotopic task, and (ii) spatiotopic precision did not continue to deteriorate with additional eye movements. Although a hard-wired spatiotopic mechanism would not necessarily require that retinotopic precision be worse than spatiotopic precision—representations could be preserved in both coordinates—it is the second point that is particularly important in ruling out the hard-wired spatiotopic hypothesis. If locations were maintained in spatiotopic coordinates, the retinotopic input would need to be transformed only once; after this initial transformation, information would already be in world-centered coordinates, and subsequent eye movements would be irrelevant. Thus, spatiotopic memory should be at least as stable across eye movements as retinotopic memory. Of course, some loss in precision is expected as a result of memory fading over time, but such nonspecific deterioration should affect retinotopic and spatiotopic memory equally, not just spatiotopic memory. Furthermore, the return-saccade condition (in which the second saccade brought the eyes back to the initial position instead of moving on to a new location) involved the same number of eye movements and length of delay as the two-saccade condition, but spatiotopic memory was not degraded in this case. Presumably, performance recovered here because spatiotopic and retinotopic coordinates reconverged, and/or subjects could revert back to their original representation instead of having to make an additional transformation.

It is also clear from these data that spatial information is in fact being maintained and updated across eye movements—i.e., it is not the case that everything is wiped clean and recreated anew with each eye movement. A number of studies have established that we are capable of transsaccadic memory (3, 26), at least for attended items (27, 28), and the capacity is approximately the same as visual short-term memory in the absence of eye movements (29), although transsaccadic memory for spatial locations may be worse than that for letters or other items (2, 30). Here, subjects were clearly capable of remembering retinotopic and spatiotopic locations across multiple eye movements, but the crucial difference was in how accurately and precisely they were able to maintain those representations. Most studies of transsaccadic memory have focused on coarser measures such as whether items are remembered at all, or how many items can be remembered across a saccade. As far as we know, the present study is the first to explore whether differences might instead be manifested in the precision of human spatial memory. Baker and colleagues (31) asked a related question in monkeys, showing that saccades to gaze-fixed targets were more precise than to world-fixed targets following slow displacements of gaze (smooth pursuit or whole-body rotations), although the results were less clear when the displacement was caused by a saccade, and the authors did not manipulate the number of movements during the memory delay. The present study demonstrates that retinotopic memory is significantly more accurate and precise than spatiotopic memory across saccades, and that this difference increases with each subsequent eye movement, despite the fact that nearly all subjects shared the intuition that the spatiotopic task would be more natural.

The idea that spatial representations are encoded and maintained in retinotopic coordinates and must be updated with each eye movement to preserve spatiotopic stability is consistent with previous behavioral (17, 23, 32), functional MRI (33–35), physiological (20), and computational (15) findings. The related transformation from egocentric (i.e., self-referenced) to allocentric (i.e., externally referenced) representations^‡ also requires a process of constant updating and reconstruction with each movement (36, 37). Moreover, transcranial magnetic stimulation to parietal cortex interferes with spatiotopic transsaccadic memory, with the strongest interference occurring when stimulation is delivered around the time of saccade, further supporting the idea that spatiotopic representations are updated at the time of the saccade, as opposed to items being placed in spatiotopic coordinates at the time of encoding (38).

Critically, although the present report is not the first to support a retinotopic-plus-updating process, previous studies have generally assumed that the purpose of remapping is to maintain faithful representations across saccades, and that—at least given sufficient motivation and time to update—we should be able to accurately remap locations across eye movements. Because recent reports have suggested that attention updates to spatiotopic coordinates only when behaviorally relevant, and that a “retinotopic attentional trace” lingers at the previously relevant retinotopic location for a brief period after the saccade (23, 33), in the present task, we provided ample motivation and time to complete the updating process. Even in these optimal circumstances, spatiotopic behavior was inferior, as if something were “lost in translation.” Apparently, not only is remapping computationally intensive and time-consuming, but concrete behavioral costs accompany this process. Updating to maintain a spatiotopic representation results in a less precise spatial representation, a representation that continues to degrade with each successive eye movement and transformation, just as clarity is lost when photocopying a page that has already been photocopied.

Furthermore, the present results demonstrate that this loss in fidelity is not a mere consequence of the act of preparing or executing an eye movement. A number of studies have concluded that saccades can interfere with the ability to perceive (39), attend (40, 41), or remember (30) locations. These effects are often attributed to the act of planning or executing the saccade—in other words, that it is something about the eye movement itself that causes these changes (42). Similarly, a previous study that looked at memory precision after two or five eye movements concluded that, because memory was worse after five eye movements even in the presence of a spatiotopic landmark, the memory decline was attributable to a nonspecific interference of eye movements with spatial memory (43). However, the present results suggest a different interpretation, that the difficulty in maintaining location representations across eye movements may be more attributable to the retinotopic updating process. In other words, the loss of spatial memory fidelity is not an inevitable cost of eye movements, because successive saccades degraded memory fidelity only in the spatiotopic and not the retinotopic task. It would be intriguing to test whether this principle extends to other reported consequences of eye movements, such as saccadic compression and mislocalization.

Why would our visual system have developed to use a system so apparently ill-fitted to our ecological needs? One possibility is that the spatiotopic stability problem is a major challenge for the visual system, and perhaps an imperfect retinotopic-plus-updating mechanism is simply the best we can do. Additionally, there may actually be benefits to engineering the visual system this way, in that it might allow for more flexible and efficient neural representations. Different tasks require different coordinate systems (e.g., world-centered, head-centered, object-centered), and it may be simplest to maintain a native retinotopic representation and dynamically update it to whichever coordinate system is relevant for the task at hand. Finally, it is possible that the costs of not having hard-wired spatiotopic representations may be minimized in the real world by taking advantage of compensatory cues in the environment, such as visual landmarks (19, 44, 45).

In the present study, we attempted to minimize potential landmark cues to allow us to test retinotopic and spatiotopic abilities in their more pure forms. In experiments 2 and 3, we left a small spatiotopic landmark visible to compensate for the possible role of the fixation point as a retinotopic landmark, but the addition of the spatiotopic landmark did not change the pattern of results. The outline of the monitor was also visible throughout and could have served as an additional spatiotopic landmark. Presumably, given sufficiently rich spatiotopic landmarks—e.g., if the task were performed superimposed on a grid or rich visual scene—we would expect to see an increase in spatiotopic memory fidelity, although this benefit could arise from numerous other location coding schemes (e.g., “on top of the lamp”), rather than an improvement of spatiotopic memory per se. Indeed, the presence of more compelling spatiotopic landmarks may explain why, in a recent report, spatiotopic memory appeared more preserved than retinotopic memory across a saccade (46).

Thus, although landmarks may aid spatiotopic behavior and compensate for the difficulties in maintaining spatiotopic representations, spatiotopic behavior is inherently less precise than retinotopic behavior. When it is relevant for the task at hand, we are capable of remembering spatiotopic locations, but the memory trace must be updated with each eye movement, a process that accumulates error with each transformation and belies our seamless conscious percept.

Methods

Subjects.

Eight subjects participated in each experiment (experiment 1, six female; mean age, 22.1 y; range, 18–29 y; experiment 2, five female; mean age, 24.9 y; range, 19–32 y; experiment 3, seven female; mean age, 25.1 y; range, 20–32 y). One subject participated in all three experiments; five participated in two experiments. Two additional subjects were recruited for experiment 3 but could not complete the task. All subjects were in neurologically intact condition with normal or corrected-to-normal vision. Informed consent was obtained for all subjects, and study protocols were approved by the Massachusetts Institute of Technology Committee on the Use of Humans as Experimental Subjects.

Experimental Setup.

Stimuli were generated using the Psychtoolbox extension (Brainard, 1997) for Matlab (Mathworks) and presented on a 22-inch flat-screen CRT monitor. Subjects were seated at a chinrest 64 cm from the monitor. Eye position was monitored by using an eye-tracking system (ISCAN) recording pupil and corneal reflection position at 240 Hz.

Experiment 1.

Subjects were instructed to remember a cued spatial location and report it following a short delay. During the delay, subjects executed zero, one, or two eye movements. The experiment took place in two sessions, conducted on separate days. Half the subjects were instructed to remember the locations in spatiotopic coordinates (i.e., “remember the absolute screen location”) during the first session and retinotopic coordinates (i.e., “remember the location relative to your eye position”) during the second session. For the other half of the subjects, the order was reversed.

Each trial began with a single fixation dot presented on the screen (Fig. 1A). Once subjects were accurately fixating (as determined by real-time eye-tracking), the memory cue (a black 0.8° × 0.8° square) was presented for 200 ms. There were four possible fixation locations, arranged as the corners of an 11° × 11° invisible square centered on the screen. The memory cue could appear anywhere within a smaller 3.5° × 3.5° region centered on the screen; the exact location was selected randomly for each trial. (This process ensured that target location varied from trial to trial, ranging in eccentricity from 5.4° to 10.2°, but across the experiment, average eccentricity was equated for spatiotopic and retinotopic locations.) When the cue disappeared, subjects remained fixated for 500 ms. On 20% of the trials (i.e., no-saccade trials), the fixation dot never moved. On 40% of the trials (i.e., one-saccade trials), the fixation dot jumped to a new location (i.e., the horizontal or vertical neighboring fixation location), and subjects saccaded to, and then held fixation at, the new location for 850 ms. On the final 40% of the trials, the fixation dot moved a second time, either to another new fixation location (i.e., two-saccade trials) or back to the original fixation location (i.e., return-saccade trials), and subjects again held fixation at the new location for 850 ms.

At the conclusion of the memory delay, the mouse cursor (crosshairs) appeared on top of the fixation dot, signaling subjects to report the remembered location. Subjects moved the mouse to the remembered location and clicked once to record their response. Subjects were given feedback on each trial to provide ample motivation and training to perform each task: a green square appeared at the location the subject clicked, and a black square appeared at the correct retinotopic or spatiotopic location (Fig. 1C). Subjects were instructed to get their green square as close to the center of the black square as possible.

Each run consisted of 40 trials, which included every possible combination of fixation location and saccade direction. The trials were presented in random order. At any point in the trial, if the subject's eye position deviated more than 2° from the correct fixation location, or if saccadic latency was greater than 600 ms, the trial was immediately aborted and was repeated later in the run. Subjects had to successfully complete all 40 trials before going on to the next run. Subjects completed four to six runs of each task (always the same number of runs for the retinotopic and spatiotopic tasks for a given subject). The eye tracker was calibrated at the beginning of the experiment and recalibrated as necessary between runs. Subjects were given 20 practice trials on the relevant task before each session.

Experiment 2.

Experiment 2 included the presence of a constantly visible spatiotopic landmark on the screen, to control for the possibility that the fixation dot could have served as a retinotopic landmark. The white fixation dot was now framed with a black outline. When the fixation dot jumped to a new location on saccade trials, the entire fixation dot reappeared at the new fixation location, but the black outline remained visible at the original fixation location (Fig. 1B).

Ideally, we would have removed all possible landmark cues in both reference frames. However, because it was critical that subjects remained fixated in the appropriate locations throughout the task, we could not eliminate the fixation dot. Because transsaccadic memory is thought to improve with the number and proximity of available landmarks (45), we included a single equidistant spatiotopic reference point to match the single retinotopic reference point. We used the original fixation location because it was equidistant from the cue (on average across trials), and because it was an object of interest in the task and more likely to be attended.

Experiment 2 tested only the no-saccade and one-saccade conditions. Each run consisted of 24 trials (n = 8 no-saccade, n = 8 horizontal saccade, n = 8 vertical saccade), with all possible saccade directions presented in random order. Subjects completed six runs of each task. Otherwise, the design was identical to that of experiment 1.

Experiment 3.

Experiment 3 included a visual mask after the memory cue to eliminate any afterimages. The mask was composed of 2,000 squares (of the same size and color as the cue) randomly positioned across the extent of the screen. After the memory cue was presented for 200 ms, there was a 200-ms blank fixation period, followed by a 500-ms masking period. The fixation dot was still visible over the mask. The rest of the trial proceeded as before.

The outline of the initial fixation location remained visible after saccades, as in experiment 2, and we tested all four conditions (no-saccade, one-saccade, two-saccade, and return-saccade), as in experiment 1. Additionally, cue and mask stimuli were colored white and presented on a black background, with minimal external room illumination.

Analyses.

For each trial, error magnitude was calculated as the distance between the subject's report and the correct location. Values were averaged separately for each subject, task, and saccade condition and submitted to random-effects analyses. Trials in which the subject responded in the wrong region of the screen (error >5.5°) were considered incorrect and excluded. These occurred on less than 0.5% of trials, with the exception of one subject in experiment 3, who had an error rate of more than 10% in the retinotopic task.