Mislocalization of Flashed and Stationary Visual Stimuli after Adaptation of Reactive and Scanning Saccades

Abstract

When we look around and register the location of visual objects, our oculomotor system continuously prepares targets for saccadic eye movements. The preparation of saccade targets may be directly involved in the perception of object location because modification of saccade amplitude by saccade adaptation leads to a distortion of the visual localization of briefly flashed spatial probes. Here, we investigated effects of adaptation on the localization of continuously visible objects. We compared adaptation-induced mislocalization of probes that were present for 20 ms during the saccade preparation period and of probes that were present for >1 s before saccade initiation. We studied the mislocalization of these probes for two different saccade types, reactive saccades to a suddenly appearing target and scanning saccades in the self-paced viewing of a stationary scene. Adaptation of reactive saccades induced mislocalization of flashed probes. Adaptation of scanning saccades induced in addition also mislocalization of stationary objects. The mislocalization occurred in the absence of visual landmarks and must therefore originate from the change in saccade motor parameters. After adaptation of one type of saccade, the saccade amplitude change and the mislocalization transferred only weakly to the other saccade type. Mislocalization of flashed and stationary probes thus followed the selectivity of saccade adaptation. Since the generation and adaptation of reactive and scanning saccades are known to involve partially different brain mechanisms, our results suggest that visual localization of objects in space is linked to saccade targeting at multiple sites in the brain.

Introduction

The interplay between vision and saccades is a prime example of an action–perception coupling: saccades are made to acquire new visual information, and vision, in return, is used to localize the next saccade target. Recent research has shown a reciprocal effect between visual localization and saccade targeting: modification of motor parameters by saccade adaptation leads to a distortion of visual localization (Bahcall and Kowler, 1999; Awater et al., 2005; Bruno and Morrone, 2007; Collins et al., 2007). This suggests that visual localization and saccade targeting share a common coordinate system, and common pathways in the brain, such that the signal that codes the required motor parameters to reach an object with a saccade also codes the perceived spatial position of that object.

Influences of saccadic adaptation on space perception have so far been studied mostly with flashed visual localization probes. Flashed probes are special because they attract transient attention. Saccades that are induced by flashed or by suddenly appearing targets are called reactive saccades, and are believed to receive target localization signals from parietal pathways to the superior colliculus and the brainstem saccade generator (Pierrot-Deseilligny et al., 1991; Gaymard et al., 2003; Müri and Nyffeler, 2008).

Such reactive saccades are often used in laboratory settings, but they occur rarely in normal viewing of a natural scene because few objects suddenly appear in a normal visual scene. Instead, during normal scanning of a stationary visual scene target selection is driven by task demands and by the voluntary selection between multiple targets. Such saccades have been called scanning (Deubel, 1995; Cotti et al., 2007), internally triggered (Erkelens and Hulleman, 1993; Fujita et al., 2002), or voluntary (Collins and Doré-Mazars, 2006; Walker and McSorley, 2006; Alahyane et al., 2007) saccades. Targeting of these saccades is believed to involve pathways from frontal cortex to superior colliculus and brainstem (Rivaud et al., 1994; Müri and Nyffeler, 2008).

In the present study, we asked whether adaptation of scanning saccade induces mislocalization, and in particular whether it can induce mislocalization of stationary targets that are visible throughout the scanning period and attract continuous attention. We expect that scanning saccades differ from reactive saccades in their ability to induce perceptual effects because of the specificity of their adaptation: whereas selective adaptation of reactive saccades induces little adaptation of voluntary saccades, selective adaptation of scanning saccades transfers substantially to reactive saccades (Erkelens and Hulleman, 1993; Deubel, 1995; Fujita et al., 2002; Hopp and Fuchs, 2004; Collins and Doré-Mazars, 2006; Alahyane et al., 2007; Cotti et al., 2007). Therefore, reactive saccade adaptation must occur mostly in the reactive pathway, whereas scanning saccade adaptation may involve both the scanning and the reactive pathways. If visual localization and saccade targeting rely on common pathways in the brain, we expect that adaptation of scanning saccades induces mislocalization of flashed and of stationary targets.

Materials and Methods

Adaptation of reactive saccades.

Adaptation of reactive saccades follows the procedure of McLaughlin (1967). In this procedure, a saccade target suddenly appears while the subject is looking at a fixation point. The subject makes a saccade to the target. While the saccade is in flight, the saccade target is displaced by a small amount. This displacement induces a visual error after the saccade, which is corrected by a short subsequent saccade. Over the course of successive trials with consistent displacement, the amplitude of the primary saccade is gradually changed to immediately reach the displaced target location. The adaptation involves gain changes in cerebellar and other subcortical structures (Desmurget et al., 1998; Robinson and Fuchs, 2001; Catz et al., 2008; Golla et al., 2008). The adaptation-induced mislocalization suggests effects of adaptation on the cortical level, or at least feedback from cerebellar or subcortical structures onto cortical localization mechanisms (Gaymard et al., 2001; Awater et al., 2005).

In the experiment, the subject was seated 57 cm in front of a 22 inch computer monitor (Eizo FlexScan F930) with the head stabilized by a chin rest. The visible screen diagonal was 20 inches, resulting in a visual field of 40 × 30°. Stimuli were presented on the monitor with a vertical frequency of 120 Hz at a resolution of 800 × 600 pixels. The room was completely dark. To avoid visibility of the screen borders, the display monitor was covered with a transparent foil that reduced the luminance by ∼2 log units. Eye movements were monitored by the Eyelink 1000 system (SR Research), which samples gaze positions with a frequency of 1000 Hz. Viewing was binocular, but only the dominant eye was recorded. The system detected start and end of a saccade when eye velocity exceeded or fell below 22°/s and acceleration was above or below 4000°/s².

Figure 1, A and B, depicts the procedure for adaptation of reactive saccades. A fixation point (1 × 1°; luminance, 0.06 cd/m²; red color), illustrated by the square in Figure 1A, was first placed 5° to the right of the left screen border. The subject had to establish and maintain fixation at this point. The circle in Figure 1A indicates the gaze of the subject. After 1 s, the fixation point was extinguished and a saccade target (red; 1 × 1°; luminance, 0.06 cd/m²) suddenly appeared 30° to the right of the fixation point. The subject was instructed to make a saccade to the target as quickly as possible. Eye position was monitored on-line. As soon as the eye crossed, an invisible border at 2.5° to the right of the fixation point the saccade target was stepped back by 6°. In the initial trials, this back step caused a visual error at the end of the saccade. With increased number of trials, this error is reduced such that the eye lands closer to the back-stepped target location (Fig. 1B). After 70 adaptation trials, when the subject already had begun to adapt, the back step was increased to 9° to increase the final amount of adaptation.

Figure 1.

A, Experimental procedure for reactive saccade adaptation. At the beginning of the trial (top panel), a fixation point (square) is presented near the left screen border. The subject's gaze (circle) is directed to the fixation point. After 1000 ms (middle panel), the fixation point disappears and a saccade target appears 30° to the right of the fixation point. The subject initiates the saccade to the target. When the saccade onset is detected (bottom panel), the saccade target is displaced, inducing a visual error after the saccade. B, After several such adaptation trials, the saccade amplitude becomes shorter. The saccade ends on the displaced target and the visual error after the saccade is reduced. C, Experimental procedure for scanning saccade adaptation. At trial onset (top panel), four saccade targets (squares) are presented. The subject fixates the bottom right target (circle). At a voluntary pace, the subject scans the targets in a counterclockwise manner. As the subject executes each saccade, the previously inspected target is extinguished. Adaptation takes place during the saccade from the bottom left to the bottom right target (bottom panel). When the onset of the saccade is detected, the bottom right target is displaced to the left, inducing a visual error after the saccade. D, After several adaptation trials, the saccade amplitude is adapted to the displaced target location.

To ensure that the subject really reacted to the sudden appearance of the target and did not preplan the saccade, some trials were randomly interspersed (probability, 0.33) in which the saccade target appeared 10° above or below the fixation point. These trials were checked for compliance with the instruction, but were not used for adaptation, and did not enter into the data analysis. They did not interfere with adaptation because adaptation is direction specific (Frens and van Opstal, 1994; Albano, 1996). Moreover, to counteract dark adaptation of the subject, these trials were followed by a 1 s period in which the screen turned white (luminance, 0.6 cd/m²) while the subjects had to maintain fixation at the target location.

Adaptation of scanning saccades.

The procedure for scanning saccades followed the paradigm introduced by Deubel (1995). In this procedure, saccade targets are continuously visible and the subjects look at each with a self-paced sequence of voluntary saccades. Four saccade targets (1 × 1°; luminance, 0.06 cd/m²; red color) were presented at trial onset (Fig. 1). The saccade targets were arranged in a rectangle with a horizontal distance of 30° and a vertical distance of 20°. The left edge of the rectangle was 5° to the right of the left screen border. Subjects began by fixating their gaze (circle) at the bottom right target. They then had to scan the other saccade targets in a counterclockwise manner at a voluntary pace. To ensure that subjects truly scrutinized each target, the saccade targets contained small discrimination dots, either one or two, that could be seen only by foveal inspection of the target. The subject had to count how often a pair of two discrimination dots was present in a trial.

While the subject made saccades from one target to the next, the previously inspected targets were extinguished. The top right target was turned off during the saccade from the top right target to the top left target. The top left target was turned off during the saccade to the bottom left target. The bottom left target was turned off during the final saccade from the bottom left to the bottom right target. Each target was extinguished when the eye had traveled a distance of 2.5° along the path of the respective saccade. When the subject performed the final saccade (i.e., the 30° rightward saccade from the bottom left target to the bottom right target), only the final target (bottom right) remained on the screen. This saccade was adapted. The bottom right target was shifted 6° to the left as soon as the eye crossed the invisible border at 2.5° to the right of the bottom left target. After 70 consecutive adaptation trials, the displacement was increased to 9°. The scanning adaptation procedure therefore differed from the reactive adaptation procedure in the way in which the saccades were initiated, but it was similar in terms of the metric of the adapted saccade, the stimuli visible at the time of adaptation, and the timing and size of the target back step.

Reactive and scanning saccades are known to differ strongly in latency (Deubel, 1995; Cotti et al., 2007). We therefore used latency differences in the two conditions as a first test of whether we were successful in eliciting different saccade types. Latency in the reactive case was measured from the onset of the target. Since for scanning saccades there is no target onset, latency for scanning saccades was calculated from the onset of the preceding fixation. This measure includes the fixation duration during which the data for the discrimination task must be gathered. It is thus not directly equivalent to the latency in the reactive case, but it is commonly used as a check for differences between saccade types (Deubel, 1995; Cotti et al., 2007) and will serve for this purpose here as well. Saccade latencies differed between the reactive and the voluntary saccade trials as expected. The mean reactive saccade latency over all reactive saccade adaptation sessions and all subjects was 210 ± 56 ms. The mean scanning saccade latency was 515 ± 113 ms. We also checked latencies in transfer trials (described later in detail) in which reactive saccades were performed after scanning saccade adaptation, and vice versa. The mean latency of reactive saccades performed in these transfer trials was 224 ± 44 ms. The mean latency of scanning saccades performed in the transfer trials was 484 ± 123 ms. A two-way repeated-measures ANOVA revealed a significant difference between saccades types in both regular and transfer trials (F = 75.41; p < 0.01). The difference latency suggests that we were successful in eliciting different saccades types in the different conditions.

Localization.

Localization was tested before and after adaptation while subjects performed normal or adapted saccades, respectively. Two types of localization trials were run. One used a flashed localization probe, like the targets used for reactive saccades. The other used a stationary localization probe like the targets used for scanning saccades. These stimuli were designed to imitate the temporal properties of the saccade targets that trigger reactive and scanning saccades.

In other respects, the probe stimuli were visibly distinct from the saccade target to avoid confusion in the localization task. The flashed localization probe was a small bar (0.3 × 4°; luminance, 0.2 cd/m²). The probe was presented for 20 ms at a randomly chosen horizontal position in a range of 2° around the saccade target (i.e., between 28 and 32°).

The vertical position of the bar was the same as that of the saccade target. In reactive saccade trials, the bar was flashed 50 ms after the appearance of the saccade target [i.e., ∼150 ms (depending on saccade latency) before the reactive saccade]. In scanning saccade trials, the bar was flashed when the eye tracker detected that the eye position was on the bottom left saccade target (i.e., before the saccade that was adapted). In both cases, trials in which the bar was flashed <100 ms before saccade onset were omitted from analysis because we did not want any interference from perisaccadic mislocalizations (Georg and Lappe, 2009). Furthermore, occasional trials in which subjects failed to notice the bar were also omitted from analysis. Subjects indicated when they did not see the bar by clicking with the mouse pointer in the lower right corner of the screen. Based on these two criteria, 6% of the data had to be omitted from analysis. If for any subject this resulted in <10 trials in either the target-off, the target-on, or the transfer trials, that subject repeated the recording session and we collapsed the data from both sessions.

The stationary localization probe was identical with the flashed probe but was presented from trial start until the occurrence of the saccade (i.e., when the eye tracker detected that the eye had traveled 2.5° along the path of the saccade). Thus, in the reactive saccade trials, the bar was continuously visible throughout the 1 s fixation period and during the latency of the saccade. In the scanning saccade trials, the bar was continuously visible throughout the time that the subject took to look at all but the final target.

The task of the subject in the localization trials was to indicate the location of the bar with a mouse pointer. The pointer appeared 1000 ms after the saccade near the bottom of the screen at a randomly assigned horizontal position between 35 and 40°. The localization error was calculated as the deviation of the mouse click position from the position at which the bar was presented.

Normally, because the localization was performed in conjunction with the execution of normal or adapted saccades, the saccade target either remained stationary or jumped back in the respective cases. Thus, it was visible after the saccade and during the reporting with the mouse. Therefore, the saccade target might serve as a visual reference for the localization task. To test for the influence of the postsaccadic target reference, we included also trials in which the target was turned off during the saccade. In these target-off trials, no visual references were available after the eye landed.

Sequence of events during a single adaptation and localization session.

A single session consisted of one type of saccade adaptation (reactive or scanning) with one type of localization task (flashed or stationary). Therefore, each subject had to complete four sessions: reactive saccade adaptation with flashed localization probes, reactive saccade adaptation with stationary localization probes, scanning saccade adaptation with flashed localization probes, and scanning saccade adaptation with stationary localization probes. These four sessions were run on different days and in counterbalanced order across subjects.

The basic structure of trial blocks was the same for each session (Table 1). The session started with a block of 40 preadaptation trials of the respective saccade type. These trials allowed calculation of the saccade amplitudes as a baseline before adaptation. Moreover, all preadaptation trials included the localization task to record a baseline for localization error. Trials in which the saccade target was turned off during execution of the saccade (target-off trials) were randomly interspersed (probability, 0.33) with trials in which the saccade target remained illuminated (target-on trials).

Table 1.

Trial structure for single saccade adaptation sessions

In the preadaptation phase, saccades of the to-be-adapted type and of the transfer type are performed, together with the localization task. In the adaptation phase, only adaptation saccades are performed without localization. About halfway during the adaptation phase, the target back step is increased from 6 to 9° to increase the final amount of adaptation. In the postadaptation phase, adapted saccades and transfer saccades are performed together with the localization task. A small number of deadaptation saccades end the session.

Next came a block of 30 preadaptation trials of the opposite saccade type (i.e., scanning saccades for reactive saccade adaptation sessions and reactive saccades for scanning saccade adaptation sessions). These trials served as a baseline for the transfer test between saccade types. The localization task was included in all of these trials. Target-off trials in which the saccade target was turned off during execution of the saccade were randomly interspersed (probability, 0.33) with target-on trials in which the saccade target remained illuminated.

The third block consisted of 150 adaptation trials. Saccade adaptation was induced stepwise to avoid subjects' noticing the saccade target back step. In the first 80 of the adaptation trials, the target stepped back 6° to the left of the initial saccade target position. In the remaining 70 trials, the back step was increased to 9°. These trials did not contain a localization task. They only served to establish adaptation.

The fourth block (postadapatation, 40 trials) continued with additional adaptation trials but also included the localization task. Target-on and target-off conditions were randomly intermixed. The saccade amplitude data from the target-off trials were used to measure the amount of adaptation.

Then, a block of 20 transfer test trials was performed, in which saccades of the opposite type had to be performed (i.e., scanning saccades after reactive saccade adaptation and reactive saccades after scanning saccade adaptation). These trials served to measure the amount of adaptation transfer from the adapted saccade type to the other saccade type. The localization task was also included to measure the amount of mislocalization transfer. In all trials, the saccade target was turned off during execution of the saccade to avoid deadaptation.

Thereafter, the opposite saccade type was tested again in 20 retest trials. The aim of the retest trials was to check for any deadaptation of saccade amplitude size after the transfer test trials. Again, to prevent deadaptation, the saccade target was turned off during execution of the saccade.

Finally, in 20 deadaptation trials, the saccade target remained in its initial position to help the subject to deadapt before leaving the experiment.

Participants.

Nine subjects, five males and four females (one author, eight naive subjects; mean age, 23 years), participated in all of the experiments. All subjects were students from the Psychology Department and had normal or corrected-to-normal vision. Subjects gave informed consent. All subjects underwent all experimental conditions. The experiments were performed along the principles laid down in the Declaration of Helsinki.

Results

We performed adaptation experiments with reactive and voluntary saccades. After adaptation, we measured the adaptation-induced mislocalization of probe stimuli. We used two different sets of probes, one flashed and one stationary, to study whether the mislocalization is specific to the visual properties of the associated saccade targeting pathway. We will first report measurements of the saccade amplitude adaptation and thereafter the results of the localization task.

Saccade adaptation and transfer

Figure 2A shows saccade amplitudes over a single session of reactive saccade adaptation. Trials in which reactive saccades were performed are shown in red. Pretest and transfer test trials in which scanning saccades were performed are shown in blue. The first 40 trials were preadapation reactive saccade trials in which the subject performed normal reactive saccades to a suddenly appearing target at 30°. The target remained at its position and did not jump during the saccade. These saccades were hypometric with a median at 26.43 ± 1.15° (black horizontal line), which is normal for saccades of this size.

Figure 2.

A, Example adaptation curve for reactive saccades. Reactive saccades are plotted in red, and intermixed scanning saccades are in blue. B, Example adaptation curve for scanning saccades. Scanning saccades are plotted in blue, and intermixed reactive saccades are in red. The example for reactive adaptation contained flashed localization trials. The example for scanning adaptation contained stationary localization trials.

The next 30 saccades (from trial 41 to trial 70) were preadapation scanning saccades, which were performed as part of a scanning sequence across four targets as described in Materials and Methods. The saccade shown is the last of those four saccades. It is directed from a target on the left to a stationary target 30° to the right and matches the reactive saccade in terms of target direction and amplitude. The only difference from the reactive saccade is that this saccade is conducted to a target that was present on the screen during the entire scanning series, whereas the reactive target suddenly appeared and triggered saccade execution. Like the reactive preadaptation saccades (first 40 trials), the scanning preadaptation saccades (trials 41–70) were somewhat hypometric in this subject. The median saccade amplitude (black line) was 28.25 ± 1.13°.

Trials 71–220 were reactive saccade adaptation trials in which the target was displaced to the left during the saccade. The displacement was initially 6° and was increased to 9° from trial 150 onward. The saccade amplitude decreases gradually over the adaptation period toward a value close to the displaced target location at 21°.

The amount of adaptation was measured in the postadaptation trials (221–260). The median saccade amplitude in the postadaptation trials in this session was 21.39 ± 1.24° (Fig. 2A, black line in trials 221–260).

The postadaptation trials were followed by transfer test trials (trials 261–280). In these trials, scanning saccades were performed in the identical procedure with that in the preadaptation scanning trials (41–70). The amplitude of these scanning saccades showed little indication of adaptation. The median saccade amplitude (black line) was 26.31 ± 1.43°.

After the transfer test trials, which often showed less adaptation for the scanning than for the reactive saccades, we checked that reactive saccades were still adapted. This was done in retest trials (numbers 281–300) that were identical with the target-off trials of the postadaptation reactive block (trials 261–280). The median saccade amplitude in these retest trials was 22.61 ± 0.85°. Thus, a large amount of adaptation for reactive saccades was retained across the block of scanning saccades that had shown little adaptation. Last, a few deadapation saccades (301–320) were performed to start extinguishing the adaptation. In these trials, the target did not jump during the saccade but stayed at the initial position. These trials were not used for data analysis.

Comparison of the saccade amplitude data from the different phases of the session clearly shows that adaptation occurs during the reactive adaptation trials and is retained through the post and retest phases, whereas scanning saccades in the transfer trials showed little modification of saccade amplitude. To quantify the amount of adaptation of the reactive saccades, we subtracted the average of the median saccade amplitudes in the postadaptation (21.39 ± 1.24°) and the retest (22.61 ± 1.06°) trials from the median saccade amplitude in the reactive preadaptation trials (26.43 ± 1.15°). For the data of Figure 2A, this gave an adaptation of 4.4°. The amount of adaptation to scanning saccades in the transfer condition was calculated from the difference between the median saccade amplitude in the scanning preadaptation trials (28.25 ± 1.13°) and the transfer test trials (26.31 ± 1.43°). This gave a transfer adaptation of 2°.

Saccade amplitudes over a single session of scanning saccade adaptation are shown in Figure 2B. Scanning saccade trials are shown in blue. Pretest and transfer test trials in which reactive saccades were performed are shown in red. The first 40 trials were preadaptation scanning saccade trials. As in the scanning preadaptation trials of the reactive saccade adaptation session (Fig. 2A, blue dots), the saccades before adaptation are slightly hypometric with a median saccade amplitude of 28.51 ± 1.36°. The following 30 trials (41–70) were preadaptation reactive saccades. They were elicited in the same way as in the reactive saccade adaptation sessions and differed from the scanning saccade trials only in that the saccade target was suddenly appearing. The median saccade amplitude was 28.7 ± 0.83°.

Adaptation of scanning saccades began with trial 71. An initial 6° jump displaced the saccade target from 30 to 24° for the next 80 trials (71–150). From trial 151 to 220, the size of the saccade target jump was increased to 9° displacing the target to 21°. After adaptation, the median scanning saccade amplitude in the postadaptation trials (trials 221–260) was 24.91 ± 1.91°, indicating an adaptation of 3.6°.

Reactive saccade amplitudes in the transfer test trials (red dots; trials 261–280) were partially affected by the adaptation of scanning saccades. The median saccade amplitude in the transfer test trials was 26.21 ± 0.93°, indicating an adaptation of 2.49°. Scanning saccades performed afterward in the retest trials (trials 281–300) remained mostly adapted. The median saccade amplitude was 24.48 ± 2.81°, close to the median saccade amplitude of the posttest trials (24.91 ± 1.91°). Finally, trials 301–320 were deadaptation trials in which the saccade target remained in its initial position at 30°.

The median saccade amplitudes in the reactive and the voluntary preadaptation trials in this subject differed slightly across sessions. In Figure 2A, reactive saccades are more hypometric than scanning saccades, whereas this is not the case in Figure 2B. Such differences occurred in some subjects but were not consistent. We calculated the median saccade amplitudes of the reactive and voluntary saccade amplitudes over all preadaptation trials for each subject. A paired t test revealed no significant difference between the reactive and the voluntary saccade preadaptation amplitudes.

The example results from Figure 2 show that adaptation occurred for both saccade types and that transfer between the saccade types was limited. Figure 3 shows adaptation and transfer amounts for reactive and scanning sessions averaged across all subjects. After reactive saccade adaptation (Fig. 3A), saccadic amplitudes to reactive targets were decreased on average by 4.9 ± 0.27°. Amplitudes of scanning saccades in that situation (transfer test) were decreased by only 1.6 ± 0.13°. After scanning saccade adaptation (Fig. 3B), saccadic amplitudes to scanning targets were decreased on average by 4.4 ± 0.27°, and transfer saccades to reactive targets were decreased on average by 1.6 ± 0.24°. A two-way repeated-measures ANOVA confirmed a significant reduction in the transfer condition but no difference between saccade types (F = 100.12; p < 0.01).

Figure 3.

A, Average amplitude reduction after reactive saccade adaptation for reactive saccades (gray) and for scanning saccades (black). B, Average amplitude reduction after scanning saccade adaptation for reactive saccades (gray) and for scanning saccades (black). In both cases, there is strong adaptation and small transfer to the other saccade type. Error bars are SEs.

We also tested whether the amount of transfer of adaptation from one saccade type to the other was different between reactive and scanning saccade adaptation sessions. We therefore calculated the percentage of transfer for each subject (amplitude decrease in transfer trials/amplitude decrease in adaptation trials * 100). Average transfer across all subjects was 36% from the adaptation of reactive saccades to the amplitude of scanning saccades and 43% from the adaptation of scanning saccades to the amplitude of reactive saccades. The transfer from scanning to reactive was, therefore, somewhat higher than from reactive to scanning, but the difference failed to reach significance (p = 0.06, paired t test).

The above analysis of saccade amplitude reduction shows that we adapted reactive and scanning saccades individually and that adaptation of one saccade type led to only partial adaptation of the other. This is consistent with previous reports of limited transfer between reactive and voluntary saccades. Reactive saccade adaptation has been found to transfer little (between 6 and 56%) to scanning (Deubel, 1995; Alahyane et al., 2007; Cotti et al., 2007) and other types of voluntary saccades (Erkelens and Hulleman, 1993; Deubel, 1995; Fujita et al., 2002; Collins and Doré-Mazars, 2006). Our findings are fully consistent with this. Adaptation of scanning saccades, however, also transfers only partially to reactive saccades, but the reported transfer rates are usually higher (between 24 and 74%) (Deubel, 1995; Alahyane et al., 2007; Cotti et al., 2007). Therefore, the transfer between reactive and scanning saccades has been called asymmetric: small from reactive to scanning and larger from scanning to reactive. The transfer from scanning to reactive saccades in our data is near the lower end of the range reported in the literature. However, it is still larger than the transfer from reactive to scanning saccades and, thus, consistent with an asymmetric transfer. Most importantly, however, the limited transfer in either direction is indicative of adaptation of different saccade targeting pathways, which is a prerequisite for the study of differences in mislocalization that we report next.

Localization results

To test influences of saccade adaptation on visual localization, a localization task was included in the trials before and after adaptation of each saccade type. In every adaptation session, localization was tested both in trials in which the adapted saccade type was performed and in trials in which the opposite saccade type was performed. The subject had to indicate the perceived bar position with the mouse pointer after execution of the saccade. Localization error was calculated as the difference between the horizontal position of the mouse click and the position at which the bar was presented on the screen. Negative values indicate that the subject reported the perceived bar position to the left of the veridical bar position. This corresponds to a shift in the direction of adaptation.

Figure 4A, left panel, shows localization errors for flashed bars in a reactive saccade adaptation session of the subject of Figure 2. Each dot is the measurement from a single trial. The dots on the left present preadaptation measurements from the preadaptation target-off trials. The localization errors are small with a median at 0.4 ± 0.58°, illustrating that localization was nearly correct before adaptation. The dots on the right are localization errors measured after adaptation in the postadaptation target-off trials. These localization errors are shifted into the direction of adaptation with a median at −1.9 ± 0.8°. A t test revealed a significant difference between preadaptation and postadaptation localization (t test, p < 0.01).

Figure 4.

Examples of localization results from the subject of Figure 2. The localization error plotted on the ordinate is the difference between true and perceived location of a probe stimulus. Each point is data from a single trial. Trials are arranged in the order in which they were conducted. In each subfigure, preadaptation data are plotted on the left, and postadaptation data are on the right. The black line connects the medians of both data sets. A, Localization results for reactive saccade adaptation with flashed (left) and stationary (right) probes. B, Results for scanning saccade adaptation. Note that the data from this subject were recorded in two adaptation sessions and that the combined data from both sessions are shown.

Figure 4A, right panel, shows localization errors of the same subject in the reactive saccade adaptation session with stationary bars. These bars were continuously visible from trial start onward and were turned off only when the eye tracker detected the onset of the saccade. Evidently, the localization of stationary bars was little affected by the adaptation of reactive saccades in this subject. The median localization error before adaptation was −0.5 ± 1.36°, and a median localization error of 0.43 ± 0.72° was found after adaptation. Reactive saccade adaptation in this subject, therefore, only influenced the localization of flashed bars (Fig. 4A, left panel), which were mislocalized in direction of adaptation, but not of stationary bars (Fig. 4A, right panel).

Figure 4B presents localization data from the scanning saccade adaptation sessions. In these sessions, the subject had to scan across four continuously visible targets at a voluntary pace, and the last saccade of that scan path was adapted. Figure 4B, left panel, presents data obtained with flashed targets. In the preadaptation trials, localization errors were near 0°, with a median at −0.3 ± 0.82°. In the postadaptation trials, localization errors for flashed bars shifted significantly in the direction of adaptation (t test, p < 0.01). Median localization error was at −1.6 ± 1.34°. Figure 4B, right panel, shows localization errors for stationary bars. Median localization error before adaptation was 0.5 ± 0.81°. In the postadaptation trials, localization error for stationary bars was significantly (t test, p < 0.01) shifted into the direction of adaptation with a median at −0.4 ± 0.71°. Thus, scanning saccade adaptation influenced both the localization of flashed and the localization of stationary bars.

To quantify the adaptation-induced mislocalization in each condition, we took the difference between the median localization errors before and after adaptation in the direction of the adaptation. For the data of Figure 4, the mislocalization values were 2.3° for reactive saccade adaptation and flashed bars, 0° for reactive saccade adaptation and stationary bars, 2.5° for scanning saccade adaptation and flashed bars, and 1.6° for scanning saccade adaptation and stationary bars.

Figure 5 shows the adaptation-induced mislocalization averaged across all subjects. After adaptation of reactive saccades (Fig. 5A), flashed bars were mislocalized with a mean across subjects of 1.8 ± 0.45° in the direction of saccade adaptation. There was no mislocalization for stationary bars (mean across subjects of 0.08 ± 0.23°).

Figure 5.

Average mislocalization after saccade adaptation. Plotted on the ordinate is the difference between the localization before and after adaptation. A, Mislocalization of flashed (left) and stationary (right) probes after reactive saccade adaptation. B, Mislocalization of flashed (left) and stationary (right) probes after scanning saccade adaptation. All data are from trials in which the saccade target was turned off during execution of the saccade (target-off trials). Error bars are SEs.

After scanning saccades were adapted (Fig. 5B), flashed bars were mislocalized on average by 1.8 ± 0.49° in the direction of adaptation. Stationary bars were mislocalized on average by 1.4 ± 0.36° in the direction of adaptation. A two-way repeated-measures ANOVA revealed a significant effect of the probe type (flashed or stationary; F = 11.13; p < 0.05), and a significant interaction between probe type and saccade type (F = 7; p < 0.05). We conclude that reactive saccade adaptation induces mislocalization for flashed but not for stationary probes, whereas scanning saccade adaptation induces mislocalization for both flashed and stationary probes.

It seems possible that saccade amplitudes are also influenced by the probes, and flashed and stationary probes could have differential influences on amplitudes, which then might have differential effects on mislocalization. We have therefore analyzed how the appearance of the bar influenced the saccade amplitudes: for every subject, we calculated the difference between the median amplitudes of the last 10 adaptation trials before a locatization phase and the median amplitudes in the localization phase. This quantifies how much the appearance of the probes changed the amplitude of the saccades. Averaged over all subjects the appearance of the bars increased saccade amplitudes by 1 ± 0.56°. However, this influence of bar appearance on saccade amplitudes was equal across conditions and there were no significant differences between session types. An influence of the probes on the saccade amplitude can therefore not explain the different mislocalization effects.

The adaptation-induced mislocalization for flashed targets after reactive saccade adaptation is consistent with several previous studies that found similar effects (Awater et al., 2005; Bruno and Morrone, 2007; Georg and Lappe, 2009). Mislocalization of flashed probes after scanning saccades has not been tested previously, but adaptation of saccades in an overlap paradigm, which is usually considered to induce voluntary saccades, also induced mislocalization of flashed targets (Collins et al., 2007). The adaptation-induced mislocalization of stationary targets is a novel finding. Our observation that mislocalization of stationary targets occurs only after adaptation of scanning saccades, and not after adaptation of reactive saccades, suggests that the origin of this mislocalization is confined to the scanning saccade pathway. Because mislocalization of flashed targets occurs for both saccade types, it may originate from mechanisms that are shared between both pathways.

The flashed targets that we used as probe stimuli were intended to mimic the temporal properties of the typical targets of reactive saccades. In a reactive saccade trial, the saccade target suddenly appeared, like the flashed probes, but unlike the flashed probes the saccade target thereafter stayed on for the entire saccadic reaction time. To check whether the results obtained with flashed probes are also pertinent to appearing probes, we ran a control condition with reactive saccade adaptation and probes that suddenly appeared and, like the reactive saccade targets, stayed on thereafter. In this condition, the bars appeared 50 ms before saccade target onset. They disappeared when the saccade onset was detected. Hence they stayed visible through the saccadic reaction time, like the saccade target. The average duration of bar presentation over all trials from both subjects was 328 ms. All other procedures were the same as in the flashed bar condition. We performed this control condition with two subjects, one of them the subject of Figure 4. The mislocalization results of the two subjects are shown in Figure 6. Both subjects show a consistent adaptation-induced mislocalization. The difference between the median localization in the preadaptation trials and the postadaptation trials was 4.24° for subject S5 and 4.75° for subject S8, and thus at least as large as their localization differences for flashed probes in the regular reactive adaptation sessions (2.25° for S5 and 4.35° for S8). We are therefore confident that our flashed target condition captures the essential properties of saccade targets that trigger reactive saccades.

Figure 6.

Localization results from reactive saccade adaptation with appearing probes. Two subjects were tested, one of them the subject of Figure 4. The localization error plotted on the ordinate is the difference between true and perceived location of a probe stimulus. Each point is data from a single trial. In each subfigure, trials are arranged in the order in which they were conducted. Preadaptation data are plotted on the left, and postadaptation data are on the right. The black line connects the medians of both data sets.

The selectivity of the mislocalization for target types (flashed vs stationary) is reminiscent of the asymmetry often observed in the transfer of adaptation between saccade types. Adaptation transfers little from reactive saccades elicited by flashed targets to scanning saccades directed to stationary targets (Deubel, 1995; Fujita et al., 2002; Collins and Doré-Mazars, 2006; Alahyane et al., 2007; Cotti et al., 2007). Similarly, mislocalization after reactive saccade adaptation occurs for flashed targets but not for stationary targets. Adaptation of voluntary saccades to stationary targets has been reported to transfer well to reactive saccades elicited by flashed targets (Deubel, 1995; Fujita et al., 2002; Collins and Doré-Mazars, 2006; Alahyane et al., 2007; Cotti et al., 2007). Similarly, mislocalization after scanning saccade adaptation occurs for stationary targets and also for flashed targets. Reactive saccade adaptation thus influences saccades to and localization of flashed targets. Scanning saccade adaptation influences saccades to and localization of flashed and stationary targets.

Localization results in the target-on and transfer conditions

For the above analysis, we have used only data from target-off trials to avoid any interference of visual reference information from the view of the postsaccadic target. A similar analysis of the target-on trials gave localization results very similar to those of the target-off trials (Fig. 7). After reactive saccade adaptation, flashed bars were mislocalized by on average 2.3 ± 0.37°, and stationary bars were mislocalized by on average 0.7 ± 0.24°. After scanning saccade adaptation, flashed bars were mislocalized by 1.9 ± 0.33°, and stationary bars were mislocalized by 1.6 ± 0.32°. A two-way repeated-measures ANOVA revealed a significant main effect of probe type (F = 10.24; p < 0.05) and a significant interaction between the probe type and saccade type (F = 16.55; p < 0.01).

Figure 7.

Average mislocalization after saccade adaptation in trials in which the saccade target was visible after saccade offset (target-on trials). The conventions are the same as in Figure 5..

Overall, mislocalization in the target-on condition was slightly higher than in the target-off condition. The differences were small (∼0.3°) and did not reach significance (t test, p = 0.06). We conclude that postsaccadic visual references from the target location contribute only little to the mislocalization effect. This is consistent with previous observations with reactive saccades and flashed probes in which target-on and target-off trials gave similar mislocalization (Awater et al., 2005). Collins et al. (2007), however, observed differences in mislocalization of flashed targets between target-on and target-off conditions with overlap saccades. However, these differences were most pronounced for probe locations farther away from the saccade target and were only small in the vicinity of the saccade target where the measurements in our study were taken.

We also measured mislocalization in the transfer trials. In these trials, one type of saccade was adapted, but the other type of saccade was performed. Because adaptation transfer was only partial (Fig. 3), the amplitudes of saccades in the transfer trials were typically less adapted than when the same saccades were performed after genuine adaptation. Figure 8A shows mislocalization when reactive saccades were performed after scanning saccades had been adapted. Mean mislocalization across subjects was 0.7 ± 0.24° for flashed bars and 0.2 ± 0.16° for stationary bars. Mean mislocalization across subjects when scanning saccades were performed after reactive saccades had been adapted was 0.6 ± 0.28° for flashed bars and 0.13 ± 0.14° for stationary bars (Fig. 8B). There was no significant difference between the conditions, but the average mislocalization in the transfer trials was significantly different from zero (t test, p < 0.01).

Figure 8.

Average mislocalization in the transfer test trials. A, Mislocalization when reactive saccades were performed after adaptation of scanning saccades. B, Mislocalization when scanning saccades were performed after adaptation of reactive saccades. In all cases, the saccade target was turned off during execution of the saccade. Error bars are SEs.

The comparison with Figure 5 reveals that the amount of mislocalization is overall lower in the transfer trials. To compare results from the transfer trials with results from the adaptation trials, we ran a three-way ANOVA with the factors experimental condition (target-off/transfer), saccade type that was adapted (reactive/scanning), and probe type (flashed/stationary). A significant main effect in the factor experimental condition (F = 9.263; p < 0.016) confirmed that mislocalization in the transfer trials was lower than in the target-off trials. A significant main effect in the factor probe type (F = 8.425; p < 0.02) showed that mislocalization for flashed and stationary probes differed. Saccade type had no main effect but a significant interaction occurred between saccade type and probe type (F = 7.758; p < 0.02), confirming that mislocalization depends on the properties of the probe and the saccade that is adapted. No other interaction was significant. The smaller mislocalization in the transfer condition than in the target-off condition is consistent with the amplitudes of the transfer saccades being only weakly adapted. It shows that not only the type of saccade that is adapted influences the mislocalization but also the type of saccade that is prepared.

In summary, we conclude that adaptation both of reactive and of scanning saccades influences the localization of visual stimuli and that this influence depends on whether the stimulus is flashed or stationary. After adaptation of reactive saccades, localization of flashed bars was shifted into the direction of adaptation as observed in previous studies (Awater et al., 2005; Georg and Lappe, 2009). The localization of stationary bars, however, was unaffected by reactive saccade adaptation. After adaptation of scanning saccades, in contrast, both flashed and stationary bars were mislocalized into the direction of saccade adaptation. The magnitude of mislocalization depends on the amount of adaptation, since in the transfer trials, in which the amount of adaptation was reduced, mislocalization was also smaller.

Discussion

To explain the different influences of reactive and scanning saccade adaptation on the localization of flashed and stationary targets, we must discuss the possibilities by which the visual system may estimate the location of the targets. The bars in our study were presented before the execution of the saccade. Thus, the position of the bar had to be encoded before the saccade, then retained in transsaccadic memory, and later retrieved after the saccade ended. For such transsaccadic localization, the visual system might encode objects with respect to visual landmarks, such as the saccade target, and retrieve them after the saccade from visual information about the postsaccadic location of the saccade target (Deubel et al., 1996, 2002; McConkie and Currie, 1996; Awater and Lappe, 2006). However, in the target-off trials, on which we based our main analysis, the target was not visible after the saccade and could not have served as a landmark for retrieval. Thus, visual reference information cannot explain the mislocalization.

In the absence of the saccade target, the visual system may instead use the current gaze direction as reference. Because the postsaccadic gaze direction after an adapted saccade is different from that after a normal saccade, the reported location should be shifted in the direction of adaptation. However, in this scenario, the amount of mislocalization should be the same as the amount of adaptation, which is not the case.

An additional possibility is the remapping of spatial location based on an efference copy signal. An efference copy of the saccade motor command may be used to predict the postsaccadic location of the object based on its presaccadic location and the amplitude of the saccade. Mislocalization may then arise if the efference copy signal does not match the amplitude of the saccade. This occurs if the efference copy reflects the size of the unadapted saccade (e.g., if adaptation takes place in a neural structure that is downstream from the structure that generates the efference copy). If this were the case, then the presaccadic location would be remapped to a postsaccadic location as if the saccade were unadapted. Since the saccade is actually shorter, a mislocation in the direction of the saccade would be the consequence (Bahcall and Kowler, 1999; Hernandez et al., 2008; Collins et al., 2009). However, the unadapted efference copy explanation would predict the same amount of mislocalization for all saccade and stimulus types. Our results show that this is not the case. Moreover, the efference copy explanation is also inconsistent with the spatial pattern of mislocalization reported by Collins et al. (2007). Their data indicated that mislocalization for objects farther away from the saccade target is not correlated with the performed saccade (i.e., the saccade for which the efference copy signal is generated) but with the adaptation state for the saccade that would be required to reach the object, even when this saccade is not performed.

According to the above considerations, neither postsaccadic reference signals nor efference copy or eye position signals can explain the dependence of the mislocalization on the target properties. We must therefore consider differences between the processing of the flashed and the stationary targets in the presaccadic encoding or memory stages. One possibility is that the mechanism of adaptation includes a modification of target location at an early stage of the sensorimotor transformation and that this modified target location is used for the transsaccadic memory. If this were the case, visual localization and saccade targeting would be equivalent in the sense that the perceived location of an object is derived from the target metrics of the saccade that would be needed to acquire the object (Collins et al., 2007). Thus, saccade metrics would be used for saccade targeting and visual localization alike. If saccade adaptation involves a change to the saccade target metrics, then the perceived location of the object at the target location must change in a similar manner. In this view, if the pathways that generate the saccade differ for different target conditions (flashed vs stationary), then the localization should also differ and depend on the target properties.

This proposal predicts that part of the adaptation of the saccade amplitude stems from the remapping of target location rather than from the adjustment of motor execution. Some evidence for an involvement of target remapping in saccade adaptation is reported in a few recent studies. Ethier et al. (2008) has analyzed the temporal velocity profile of adapted saccades and compared it with predictions of a model that can adjust saccade amplitude either by adjusting the parameters of the forward model of the saccade generator or by adjusting the target signal (Chen-Harris et al., 2008). The comparison showed evidence for adjustment of both motor and target parameters, although the target parameter adjustment was necessary only for gain increasing saccades. Other evidence that saccade adaptation may, in some conditions, include changes in target localization stages in addition to changes in motor execution comes from recent studies of transfer of adaptation between saccades and antisaccades (Collins et al., 2008; Panouillères et al., 2008; Cotti et al., 2009). Adaptation in target localization stages is also supported by the finding that hand pointing movements to a continuously presented target were misdirected after voluntary saccade adaptation but not after reactive saccade adaptation (Cotti et al., 2007). This is consistent with our data since we also found mislocalization for stationary bars only after adaptation of voluntary saccades. However, we also found that reactive saccade adaptation affects the localization of flashed bars. Evidence that reactive saccade adaptation affects localization via hand-pointing movements for flashed bars comes from the study by Bruno and Morrone (2007).

However, if mislocalizations were the result of a simple modification of the early stage of the sensorimotor transformation, then this modification should be revealed whatever the type of saccade being prepared. In the transfer test trials, however, localization is also a function of the type of saccade that is prepared, not only the type of saccade that is adapted. In addition, one might expect that a modification of the early stage of the sensorimotor transformation should also lead to a mislocalization when the saccade is not performed. Such a mislocalization has been reported by Moidell and Bedell (1988), but it seems small and difficult to measure reliably (Awater et al., 2005; Collins et al., 2007). However, if the mislocalization results from modifications in the presaccadic encoding or memory stages of transsaccadic memory, then it must not necessarily appear also during fixation, since in this situation transsaccadic memory is not involved. To reconcile such a transsaccadic memory explanation with the results from the transfer test trials, one would have to assume that transsaccadic memory draws on sensorimotor representations that are specific to the saccade that is currently prepared, mainly process particular types of stimuli (i.e., flashed or stationary) and may be modified by saccade adaptation. This explanation is quite speculative and should be treated cautiously, but in essence it predicts that transsaccadic memory is formed not as a visual buffer but as a buffer constructed from the activities in brain areas that are already involved in saccade planning and preparation.

The difference between the mislocalization of flashed and stationary targets may also relate to different coordinate frames in which saccades are planned. Niemeier et al. (2003) proposed that reactive saccade are coded in eye-centered coordinates, whereas voluntary saccades are coded in head-centered coordinates. In a computational model by Gancarz and Grossberg (1999), adaptation of scanning saccades occurs via target remapping in head-centric coordinates, whereas adaptation of reactive saccades takes places via cerebellar gain learning. Thus, our data might be explained by assuming that stationary targets are coded in head-centered coordinates and thus mislocalized for adapted scanning saccades only, whereas flashed targets might be coded in different coordinates and are adapted by a different mechanism. If we assume that head-centric encoding takes some time to complete, then flashed targets might not be present long enough to enter the head-centric stage. Likewise, reactive saccades that have a much shorter latency than scanning saccades may be prepared in response to suddenly appearing targets before these can be transformed to a head-centric representation.

In conclusion, object localization in space is strongly connected to object targeting for motor events instead of being just a readout of sensory input. Object localization is not the result of a static internal representation in the visual system but is tightly linked to the ability to move. Changes in motor targeting parameters thus result in parallel changes in visual object localization. This finding reflects the aim of object localization, which is mostly to guide additional movements. A connection between visual and motor targeting does not only save computational resources, it also guarantees that vision and action are aligned.