Attention modulates trans-saccadic integration

Abstract

With every saccade, humans must reconcile the low resolution peripheral information available before a saccade, with the high resolution foveal information acquired after the saccade. While research has shown that we are able to integrate peripheral and foveal vision in a near-optimal manner, it is still unclear which mechanisms may underpin this important perceptual process. One potential mechanism that may moderate this integration process is visual attention. Pre-saccadic attention is a well documented phenomenon, whereby visual attention shifts to the location of an upcoming saccade before the saccade is executed. While it plays an important role in other peri-saccadic processes such as predictive remapping, the role of attention in the integration process is as yet unknown. This study aimed to determine whether the presentation of an attentional distractor during a saccade impaired trans-saccadic integration, and to measure the time-course of this impairment. Results showed that presenting an attentional distractor impaired integration performance both before saccade onset, and during the saccade, in selected subjects who showed integration in the absence of a distractor. This suggests that visual attention may be a mechanism that facilitates trans-saccadic integration.

1. Introduction

Every second, we make multiple eye movements, shifting the high-resolution sampling of the fovea across the world to survey our surroundings in greater detail than our low-resolution peripheral vision can provide. In order to achieve a stable percept of the world, the brain needs to reconcile the differences in this pre- and post-saccadic information by integrating the peripheral information gathered before the saccade, and the foveal information obtained once the saccade has reached the target. While there is ample evidence that integration of this information does indeed occur (Cicchini, Binda, Burr, & Morrone, 2013; Demeyer, De Graef, Wagemans, & Verfaillie, 2010; Ganmor, Landy, & Simoncelli, 2015; Melcher, 2005, 2007; Niemeier, Crawford, & Tweed, 2003; Oostwoud Wijdenes, Marshall, & Bays, 2015; Wittenberg, Bremmer, & Wachtler, 2008; Wolf & Schütz, 2015), it is unclear as to the potential mechanisms that may underpin this integration process. Visual attention starts to build up at the upcoming location of a saccade around 150 ms before the saccade is executed (Deubel, 2008; Deubel & Schneider, 1996; Kowler, Anderson, Dosher, & Blaser, 1995; Rolfs & Carrasco, 2012), and it has been suggested that this pre-saccadic attention acts as a guide for other peri-saccadic stabilising processes such as predictive remapping (Cavanagh, Hunt, Afraz, & Rolfs, 2010; Mathot & Theeuwes, 2011). It may also therefore be the case that attention additionally guides or supports the integration of peripheral and foveal vision across a saccade. We aimed to investigate the role of attention in trans-saccadic integration by presenting an attentional distractor at multiple timepoints during a saccade, to determine first whether this disruption to pre-saccadic attention affected integration performance, and secondly to determine at which point around the saccade this had the greatest impact on integration.

1.1. Trans-saccadic integration

The retina is a non-homogenous structure, with greater photo-receptor density in the fovea than in the periphery. This results in a decline in visual sensitivity in the periphery (Rovamo, Virsu, & Näsänen, 1978), as well as a decline in the ability to process certain stimulus attributes such as orientation (Mäkelä, Whitaker, & Rovamo, 1993). However, humans do not actively perceive these differences in acuity across the visual field (as discussed in Herwig and Schneider (2014)), even though in everyday life people make constant eye movements to bring relevant areas of the world into greater focus (for reviews see Schutz, Braun, & Gegenfurtner, 2011; Tatler, Hayhoe, Land, & Ballard, 2011). This raises the question of how this pre-saccadic peripheral information and post-saccadic foveal information may be integrated to achieve such perceptual stability. While early studies argue against the existence of trans-saccadic fusion of information (for example: Irwin, Yantis, & Jonides, 1983; O'Regan & Lévy-Schoen, 1983; Rayner & Pollatsek, 1983), evidence is mounting to suggest that people are actually very good at combining information presented before and after a saccade. It has been shown that certain stimulus attributes, such as orientation and form are integrated across fixations (Demeyer et al., 2010; Paeye, Collins, & Cavanagh, 2017), as well as colour (Oostwoud Wijdenes et al., 2015; Wittenberg et al., 2008). Additionally, location information can be integrated, for example information about the position of line segments can also be retained and fused across a saccade (Prime, Niemeier, & Crawford, 2005), and the positions of lines flashed during the saccade can be integrated (Cicchini et al., 2013). Evidence further suggests that the outcome of integration is reliant on the reliability of both peripheral and foveal information (Demeyer, De Graef, Wagemans, & Verfaillie, 2009; Oostwoud Wijdenes et al., 2015).

While this provides evidence that information can be transferred across saccades, a number of studies have investigated the degree to which this integration occurs. One method of measuring whether integration occurs in an optimal manner is to consider the peripheral and foveal information as two separate sources of sensory information. Two studies (Ganmor et al., 2015; Wolf & Schütz, 2015) investigated the degree to which individual peripheral and foveal percepts are integrated using the maximum likelihood estimation model (MLE). MLE provides a model for estimating the integration of different sources of sensory information, by summing the reliabilities of two independent sources (Ernst & Bülthoff, 2004) – in this case peripheral and foveal information. This model thus provides a predicted value for integrated performance if peripheral and foveal information are perfectly integrated. Both studies (Ganmor et al., 2015; Wolf & Schütz, 2015) compared this predicted integration performance from the MLE model, and the observed experimental performance on integration tasks to show that the visual system integrates peripheral and foveal information in a statistically nearly optimal way across saccades.

While there is evidence suggesting that this integration of peripheral and foveal information does indeed occur, it is unclear what mechanisms may underlie this integration process. One such candidate is visual attention – a process that is not only inextricably linked with the planning and execution of saccades, but which is also implicated in numerous peri-saccadic processes that have been shown to play a key role in maintaining a homogenous view of the world across saccades.

1.2. What underlies trans-saccadic integration?

It is well-established that when a saccade is being planned, attention will shift to the location of the upcoming saccade before the saccade is initiated (Deubel & Schneider, 1996; Kowler et al., 1995). This pre-saccadic attentional shift builds up from around 150 ms to 100 ms before the onset of the saccade, with attentional performance plateauing for the duration of the saccade’s journey to the target (Deubel, 2008). Pre-saccadic attention results in perceptual benefits such as improved performance in letter discrimination tasks (Deubel & Schneider, 1996; Kowler et al., 1995), luminance discrimination (White, Rolfs, & Carrasco, 2013), and orientation discrimination tasks (Rolfs & Carrasco, 2012), and it has been shown that this attentional shift leads to an increase in both sensitivity and perceived contrast at the location of an upcoming saccade (Rolfs & Carrasco, 2012). This increased acuity at the pre-saccadic target location could act as a predictive mechanism to enhance the peripheral information which then has to be integrated with the post-saccadic foveal information at that location.

The pre-saccadic attentional shift may also be implicated in a number of processes that are attributed to visual stability across eye movements, and this may also provide evidence for a potential link between attention and trans-saccadic integration. Indeed, Hamker, Zirnsak, and Lappe (2008) suggest that peri-saccadic processes such as the pre-saccadic attentional shift, predictive remapping, receptive field shifts and peri-saccadic compression, are all linked via a single neural mechanism, and that attention may act as a bridge between these phenomena that allow us a stable perception of the world. Processes such as remapping may act as an important mechanism that facilitate the transfer of information from pre- to post- saccadic locations in the visual field, with potential consequences for how information is integrated across saccades (Cavanagh et al., 2010). Predictive remapping suggests that before a saccade is made, the receptive field will shift to the site of the impending movement (Duhamel, Colby, & Goldberg, 1992), and is a crucial part of maintaining a stable view of the world across eye movements. Pre-saccadic attention has been linked to the remapping process (Rolfs, 2015), and it has been suggested that attention creates a retinotopically organised map of both target locations and features at the upcoming saccade location, which is then used to determine which locations are remapped (for review see Cavanagh et al., 2010; Mathot & Theeuwes, 2011). There is evidence that receptive fields from locations that are attended before a saccade are then remapped, from both neurophysiological studies (Gottlieb, Kusunoki, & Goldberg, 1998), and behavioural studies (Melcher, 2009). Additionally, the locus of attentional facilitation can be remapped across saccades: studies have shown that attention can be allocated to both the original locus of attention before a saccade, and the retinotopic equivalent of this cued location after the saccade (Golomb, Chun, & Mazer, 2008; Mathôt & Theeuwes, 2009), and that attention can be predictively allocated to the future retinotopic location of a saccade target (Rolfs, Jonikaitis, Deubel, & Cavanagh, 2011). A disruption to pre-saccadic attention could affect the saccadic remapping process: Jonikaitis, Szinte, Rolfs, and Cavanagh (2013) found that attentional capture by a transiently presented, salient distractor could also influence the location of predictive remapping such that it coincided with the remapped distractor location. This suggests a strong link between attention and one of the fundamental processes underlying trans-saccadic stability. Indeed, direct evidence of the role of remapping in trans-saccadic integration comes from a recent study by Szinte, Jonikaitis, Rolfs, Cavanagh, and Deubel (2016), who showed that motion integration occurred for pre-saccadic stimuli, between an attended location and its remapped location prior to the saccade: this suggests that these two processes are closely linked.

So, attention seems to play a crucial role in the guidance of trans-saccadic process such as remapping, which are implicated in the maintenance of perceptual stability. The question then is whether attention also plays a role in the integration of trans-saccadic visual information. This study utilises an attentional distractor to disrupt attention during the critical integration point during the saccade: salient distractors have been demonstrated to capture attention (Muller & Rabbitt, 1989; Yantis & Jonides, 1990), and the onset of a salient, coloured distractor has been used by numerous studies to manipulate attention (Jonikaitis et al., 2013; Peterson, Kramer, & Irwin, 2004; Puntiroli, Kerzel, & Born, 2015; Schreij, Theeuwes, & Olivers, 2010). If attention is needed to integrate the pre- and post-saccadic information, disrupting attention during the saccade should impair integration performance. We measured orientation discrimination performance on stimuli presented in peripheral and foveal vision alone, and stimuli presented for the duration of the saccade (integration trials). We then compared observed integration performance across the time-course of the saccade with the predicted optimal performance obtained using MLE, to determine firstly whether the presentation of an attentional distractor impaired integration performance, and secondly at which time-point during the saccade the attentional distractor had the most effect.

2. Method

2.1. Equipment

Stimuli were presented using a back projection setup with a 91× 51 cm screen from Stewart Filmscreen, and PROPixx projector from VPixx Technologies with a resolution of 1920 × 1080, and refresh rate of 120 Hz. The screen was calibrated to ensure a linear gamma correction and background luminance was 92 cd/m² at the screen centre. To minimise hot spots, we were using a screen material with a low gain and a large viewing angle. As a result, the background luminance was reduced by about 10% at an eccentricity of 15°, where the saccade target and all discrimination stimuli were presented. Viewing distance was 106 cm. Eye movements were recorded with an Eyelink 1000 (SR Research Ltd., Ontario, Canada) with a sampling rate of 1000 Hz. The experiment was presented in Matlab with custom written software using the Psychophysics Toolbox (Brainard, 1997; Pelli, 1997). Participants responded using a standard keyboard and mouse.

2.2. Participants

A total of 12 participants participated in the study. All participants were naïve as to the purposes of the study, and received payment or course credit for participation. All participants provided informed consent and had normal or corrected-to-normal vision. The experiments were carried out in accordance with the Declaration of Helsinki (1964), and ethics approval was obtained from the local ethics commission of the Department of Psychology of Marburg University (proposal number 2015-35k). Data was collected for six subjects initially, however after analysis showed that two of them failed to integrate in the control condition, and one of them could not complete the control condition, data was collected from an additional six participants. All participants completed the study only once, and were not selected from a specific participant pool. Out of these additional six, two were excluded due to lack of sufficient data across the entirety of the measured time-course (there was insufficient data before −116 ms in one case, and −79 ms in the other). To ensure that the selection and exclusion of participants did not bias the overall results, we conducted the same major analysis reported in the results section, using a mixed model to compare the difference between predicted and integration performance with and without distractor. There was no effect of participant group (first vs second collection) on the difference between control and no-control conditions: F(1,5) = 1.48, p = .2784, and no interaction between distractor condition and group: F(1,5) = 0.186, p = .6845. Additionally, when the same analysis was conducted on 9 participants who showed integration within the control condition (including the two excluded for insufficient data), the mixed model still revealed a significant effect of distractor condition: F(1,8) = 7.8, p = .023. For the final results, the data from 7 subjects was used, with ages ranging from 22 to 29 (4 male, 3 female).

2.3. Stimuli

The initial fixation target was a combination of bulls eye and cross hair shape (Thaler, Schutz, Goodale, & Gegenfurtner, 2013), presented in a random colour generated in DKL colour space (Derrington, Krauskopf, & Lennie, 1984) with a set Cartesian value of 0.4 in the L + M axis, 0.6 on the L − M axis, and 0 on the S axis. The polarity of these values was randomized across trials to avoid the build-up of afterimages. The saccade target was a black dot with a diameter of 0.18 degrees and a luminance of 3.36 cd/m². Orientation discrimination stimuli were Gabors with a standard deviation of 3.2 degrees and a spatial frequency of 2c/°. The base gabor stimuli were summed with band pass filtered noise with a central frequency of 2c/° and a Gaussian standard deviation of 1°. The stimulus orientation was determined randomly in each trial, and could be at any angle from 0 to 180°. In integration trials, orientation was the same for the peripheral and foveal stimulus. Distractors were circles of 1.3 degrees diameter. On each trial, the distractor colour was a randomly selected colour generated in DKL colour-space (Derrington et al., 1984) with a set Cartesian value of 0.5 in the L + M axis, 1 on the L − M axis, and 0.5 on the S axis, resulting in 8 possible colours. The polarity of these values was randomized across trials, to make the distractor less predictable.

2.4. Preliminary task to estimate contrast thresholds

Contrast for foveal and peripheral stimuli were set individually for each observer in a preliminary task, using a QUEST procedure (Watson & Pelli, 1983) set at 82% threshold level: participants indicated whether a Gabor identical to the one in the main experimental task (presented at an orientation of either 8° or −8° relative to vertical) was tilted clockwise or counterclockwise, and the staircase was adjusted according to right or wrong answers. Measurements were taken for foveal and peripheral stimuli separately without a saccade. Stimuli were presented for 200 ms. Participants completed 3 blocks of 40 trials each for both peripheral and foveal stimuli, and the threshold was taken to be the average of these 3 blocks.

2.5. Procedure

Participants completed 15 practice trials prior to the experiment to familiarise themselves with the task. To start a trial, participants fixated at the central fixation cross and pressed the space bar. A trial only proceeded if the fixation was accepted by the Eyelink drift correction algorithm. After a random delay of 750–1500 ms, the saccade target/discrimination stimulus appeared to either the left or right of fixation, at an eccentricity of 15 degrees. The fixation cross disappeared 200 ms after the appearance of this first stimulus. In peripheral trials, the peripheral discrimination stimulus was shown only until saccade onset (calculated online as the point at which the eyes had moved a horizontal distance of 2 degrees from fixation). After saccade onset, only the saccade target stayed at that location for the remainder of the trial. On foveal trials, initially only the saccade target appeared, and the foveal discrimination stimulus only appeared once the sacade was initiated. In integration trials, the peripheral stimulus was shown at the same time as the saccade target, and the peripheral stimulus was switched to the foveal stimulus once the saccade had been initiated. Presentation time for peripheral and foveal stimuli was equated on each trial such that each stimulus (either discrimination stimulus or saccade target) was shown for the same duration. Peripheral, foveal and integration trials were all randomly interleaved. Peripheral stimuli were presented until the saccade was detected, and foveal stimuli were presented for the saccade latency for any given trial: thus in integration trials, the peripheral and foveal stimuli were both presented for the duration of the saccade latency for each trial.

The attentional distractor was displayed for 100 ms (per Goldberg, Bisley, Powell, Gottlieb, & Kusunoki, 2002) at one of 6 SOAs relative to saccade onset: −100 ms, −50 ms, 0 ms, +50 ms, +100 ms, +150 ms. The presentation time was calculated using a running average of median saccade latency over the preceding 15 trials. Distractor location was at half the eccentricity of the target (7.5 degrees), and either 4 degrees above or below the horizontal meridian: this distance was chosen to have minimise the number of saccades made to the distractor rather than the target (Walker, Deubel, Schneider, & Findlay, 1997). After the discrimination stimulus had disappeared, an oriented line appeared at the stimulus location. By moving the mouse to the left or right, participants could freely orient the line to match the orientation of the stimulus. Aural feedback was given if a saccade latency was below 120 ms (high tone) or above 320 ms (low tone). Saccade latency was measured after each trial using the Eyelink saccade detection algorithm.

2.6. Control condition

Participants also completed a control condition, in which peripheral, foveal and integration trials were completed without an attentional distractor. All methods were identical to the main experimental task, except for the absence of the distractor. Three participants completed the control condition after the distractor condition, and four completed it before the distractor condition.

2.7. Exclusions

For each participant, saccades that were not within ± 2SD of the median saccade latency were excluded, and saccades with a latency less than 50 ms were excluded to avoid anticipatory saccades, and saccades longer than 500 ms were excluded. Saccade latency was measured as the time from stimulus onset to saccade onset, as calculated by the Eyelink algorithm. Additionally, trials in which performance on the perceptual judgment task was outside 2SD of the mean perceptual error were excluded, and trials in which saccade endpoint was more than 2SD from mean saccade endpoint for each participant were excluded. For analysis of perceptual performance, trials in which participants made a saccade to the distractor were also excluded. All trials were retained for the purposes of calculating saccade planning error, as exclusions such as latency and motor error were the measures of interest for those analyses. 3.8% of trials were excluded for technical reasons. In total 86% of trials were included for perceptual judgement analyses, constituting 9276 trials (main experimental task only) or 13,386 (main experimental task, threshold and control tasks) across all participants. Individual trial numbers ranged between 604 and 1808 trials per participant (main experimental task only) or between 1150 and 2441 trials (main experimental task, threshold and control task). The number of trials differ as the distribution of data across the measured time-course differed across participants (for the final data however, each time-point had at least 20 trials for each participant). Each participant completed between 5 and 8 h of testing, in 2-hour blocks.

2.8. Perceptual performance

Perceptual performance was measured as the smallest angular distance between the actual orientation of the stimulus and the reported orientation of the stimulus. For each condition (foveal, peripheral, integration), the time-course of attentional distraction on perceptual performance was calculated using a moving window to find the performance at each SOA. SOA was calculated as the time between saccade onset and distractor onset and was measured from −200 to 200 ms. Performance for each millisecond SOA included data from a 75 ms window around that point. To quantify performance, a cumulative Gaussian curve was fitted to the angular judgment errors for stimuli presented within each time window, and the just noticeable difference (JND) was calculated as the standard deviation of that curve (Fig. 1D). An SOA was only included for analysis if that SOA contained data from at least 20 trials for every participant. This exclusion was conducted after the sliding window analysis. If an SOA did not have at least 20 trials for every participant, that time-bin was left empty. As the time-bins with the least number of trials were at the beginning and end of the measured time-course, this reduced the length of the measured time, but did not affect measurements throughout the remainder of the time-course. For the peripheral condition, 83 bins were dropped, resulting in measurements from −151 to 165 ms relative to saccade onset. For the foveal condition, 82 bins were dropped, resulting in a time-course of −161 to 156 ms. For the integration condition, 84 bins were dropped, resulting in a time-course of −151 to 164 ms. There were large differences in perceptual performance between participants. Individual JNDs in the control condition were: peripheral performance: 16.52, 12.39, 8.10, 14.12, 12.86, 12.46, 11.8; foveal performance: 10.43, 10.30, 7.4, 11.5, 10.18, 10.4, 11; integration performance: 9.69, 8.15, 7.30, 9.8, 8.92, 9.64, 9.3. To compensate for these individual differences, individual data were normalised using a z-transformation for each subject, including all data from each condition and the control condition. Unless stated otherwise, statistical analyses were conducted on normalised data.

Fig. 1.

Timeline of events. (A) To make a response, participants used the mouse to rotate a bar to match the perceived orientation of the discrimination stimulus. (B) Events in each trial. Participants fixated in the centre of the screen, and pressed a key to begin the trial. After a random delay, the saccade target appeared at 15 degrees to the left or right of fixation and stayed there for the remainder of the trial. The fixation cross disappeared 200 ms after the onset of the saccade target. Depending on the type of trial, the pre-saccadic discrimination stimulus was shown at the same location as the saccade target. When a saccade was detected (when the eye moved more than 2 degrees horizontally from the fixation cross), the pre-saccadic stimulus switched to the post-saccadic stimulus. At a variable SOA after key press, a distractor appeared between fixation and target for 100 ms. The post-saccadic stimulus was displayed for the same duration as the pre-saccadic stimulus (both pre- and post- saccadic stimuli were presented for the saccade latency of that trial), after which time the participants made their response using the mouse. (C) Potential discrimination stimulus and distractor locations. On each trial the discrimination stimulus appeared on either the left or right of the screen, and the distractor appeared either 4 degrees above or below the horizontal centre of the screen, halfway between the fixation cross and the discrimination stimulus. (D) Example cumulative density function (red) fitted to the perceptual error judgements from one observer (blue). JNDs were measured as the standard deviation of this curve. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

2.9. Predicted performance

The maximum likelihood estimation model (MLE) was used to determine predicted performance if peripheral and foveal information was integrated across the saccade (as used by Wolf and Schütz (2015)). The reliabilities of the individual peripheral, foveal and integrated information were calculated using the equation:

$$rel=\frac{1}{JN{D}^2}$$ (1)

If integration of these individual percepts occurs, integrated performance can be predicted as the sum of the foveal and peripheral reliabilities (Ernst & Bülthoff, 2004) as in the following equation:

$$re{l}_{\mathit{\text{int}}}=re{l}_{\mathit{\text{per}}}+re{l}_{\mathit{\text{fov}}}$$ (2)

The JND for the predicted performance can then be calculated as:

$$JN{D}_{\mathit{\text{int}}}=\sqrt{\frac{1}{re{l}_{\mathit{\text{int}}}}}$$ (3)

Predicted integration performance was then compared to observed integrated performance to determine whether the presentation of an attentional distractor impaired the integration of the foveal and peripheral information.

3. Results

3.1. Saccade dynamics

The median saccade latency across participants was 167 ms with a standard deviation of 43 ms. Individual median saccade latencies and standard deviations were: 150(41), 185(60), 185(41), 146(25), 167(33), 195(47), 168(28). Individual saccade latency distributions are shown in Fig. 2.

Fig. 2.

Saccade latencies for each participant represented as density plots.

3.2. Performance relative to saccade onset

Performance on the perceptual task was calculated relative to saccade onset for each condition. Fig. 3 shows JNDs at each SOA for foveal, peripheral and integration conditions, both when a distractor was present, and in the control condition without a distractor.

Fig. 3.

Mean performance across observers, binned relative to saccade onset, for individual conditions: foveal, peripheral and integration. Performance is represented as normalised JNDs, as described in the text. The distribution of target onset and offset for each condition are represented as histograms. Target onset/offset refers to the saccade target. In peripheral trials, the onset of the saccade target and the discrimination stimulus coincide. In foveal trials, the offset of the saccade target and the discrimination stimulus coincide. In integration trials, the onset of the saccade target and the peripheral discrimination stimulus and the offset of the saccade target and the foveal discrimination stimulus coincide. Error bars are 95% confidence intervals. Performance on the control condition is represented as a single point with 95% confidence intervals.

3.3. Performance in control condition

All subjects were selected on the basis that they showed an integration benefit in the control condition: to confirm that integration had occurred without a distractor present, we compared performance on the integration condition with performance on the best single condition (foveal or peripheral). Best single performance had a normalised mean of 0.34 and std of 0.9; integration performance had a mean and std of 0.31 (0.64). A t-test revealed a significant difference between integration and best single performance: t(6) = −4.07, p = .0066, indicating integration in the control. As integration occurred in all participants without the distractor present, any detriment to integration in the distractor condition can be attributed to the presence of the distractor.

3.4. Comparison of control and distractor tasks

To explore the effect of the distractor on integration performance, we used a linear mixed model to compare the difference between integrated and predicted performance in the distractor and control conditions. The model contained fixed effects of distractor condition (control or distractor) and time, with random effect of participant. The mixed model included a correction for auto-correlation between time-points. Mixed models were conducted on non-normalised data as this analysis takes into account individual performance levels. There was a significant effect of distractor condition: F(1,6) = 7.07, p = .038; time: F(1,4226) = 20.17, p < .0001, and the interaction between distractor condition and time: F(1,4226) = 20.17, p < .0001. This indicates that the difference between integrated and predicted performance was significantly larger with the presence of a distractor, and that this difference varied across time. To further examine how the effect of the distractor differed across time, we conducted Bonferroni-corrected posthoc pairwise comparisons as above, of the difference between observed and predicted performance between the control and distractor condition at discrete time-points in 50 ms increments. There was a significant difference between control and distractor condition at −150 ms: t(6) = −3.3, p = .021; at −100 ms: t(6) = −3.07, p = .027; at −50 ms: t(6) = −2.87, p = .035; at 0 ms: t(6) = −2.66, p = .046; but not at +50 ms: t(6) = −2.44, p = .061; +100 ms: t(6) = −2.23, p = .082; or +150 ms: t(6) = −2, p = .11. This shows that the distractor was effective up until saccade onset, but if it appeared after saccade onset, performance did not differ significantly from the control condition. To further determine whether integration occurred in the distractor task, we compared the best single performance for each time point with the integration performance for each time-point, using a mixed model as described above, here with fixed effects of condition (best single performance or integration) and time. There was no significant effect of condition: F(1,6) = 1.28, p = .3; or time: F (1,4296) = 1.82, p = .18; but there was a significant interaction between condition and time: F(1,4296) = 34.7, p < .0001. This suggests that there was no effect of integration overall, but this was dependent on time. To determine at which time-points integration did/did not occur, we used post-hoc pairwise comparisons (with Bonferroni correction for multiple comparisons), of the difference between the conditions at discrete time-points in 50 ms increments. There was no significant difference between integration and best single performance at −150 ms: t(6) = −0.82, p = .59; −100 ms: t(6) = −0.21, p = .96; −50 ms: t(6) = 0.44, p = .83; 0 ms: t(6) = 1.1, p = .43; +50 ms: t(6) = 1.75, p = .19; +100 ms: t(6) = 2.36, p = .09; but there was a significant difference at +150 ms: t(6) = 2.92, p = .04. This suggests that the only time at which integration occurred was when the distractor appeared 150 ms after saccade onset.

3.5. Temporal effects of the distractor

To investigate the effects of distractor onset time on performance in individual conditions in the distractor condition, we used linear mixed models with fixed effects of SOA and condition (peripheral, foveal and integration), random effect of participant, and correction for auto-correlation of successive time-points. There was a significant effect of condition: F(1,13) = 19.63, p = .0007, and a significant interaction between condition and time: F(1,6445) = 11.81, p = .0006, indicating that the distractor affected performance at different times in different conditions. This time-course of the integration condition shows a particularly striking effect when compared to the predicted performance (Fig. 4), with performance impaired from about 100 ms before saccade onset with a gradual decline until saccade offset. This pattern indicates that the presentation of an attentional distractor impaired integration performance in the lead-up to saccade onset.

Fig. 4.

Observed vs predicted integration performance. (A) Shows this observed integration performance compared to the predicted integrated performance calculated using the peripheral and foveal weights alone, as described above. If foveal and peripheral information are being integrated in an optimal manner, the observed and predicted performance should be equal. If the presentation of the attentional distractor disrupts integration, observed integration performance should be worse than the predicted performance, so JNDS should be higher. Fitted models for growth curve analysis are shown in blue. (B) Shows the difference between observed and predicted performance for the distractor condition and control condition. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Growth curve analysis (Mirman, 2016) was used to quantify the effect of the attentional distractor on integration across the saccade. The time course of perceptual performance was fitted with a fourth-order orthogonal polynomial with fixed effects of condition (observed or predicted performance), and random effect of participant. The predicted performance was treated as the baseline condition, as we wished to compare how observed performance differed from this condition, that is, how much the distractor affected integration. A normal approximation was used to determine significance. There was a significant effect of condition on the intercept: Estimate = 0.94, SE = 0.009, p < .0001. This indicates overall worse performance in the observed than predicted condition. There was a significant negative effect of condition on the linear term of the fit, suggesting that the difference between observed and predicted performance decreased across the time course of the saccade: Estimate = −1.48, SE = 0.15, p < .0001. This suggests that the effect of the attentional distractor decreases throughout the course of the saccade. There was also a significant effect on the quadratic term of the fit, indicating that there was a difference in central curvature between the conditions: Estimate = −1.46, SE = 0.15, p < .0001. The predicted condition had a greater effect in the quadratic fit, indicating a greater central in performance than the observed condition. The observed condition however showed a significantly greater effect on the cubic term of the fit: Estimate = 1.42, SE = 0.15, p < .0001, indicating that there were more changes in direction of performance, indicating a greater amount of curvature over time. This shows that the presentation of the distractor had an effect on integration, and while the predicted performance was relatively flat, showing only a small amount of time-contingent effects of the distractor on performance, the observed performance showed a large amount of modulation depending on when the distractor was presented. This means that attention affected the integration of peripheral and foveal information in a different manner than the performance on the individual conditions alone would predict.

To summarize, presenting an attentional distractor impairs performance on the integration task relative to the predictions calculated from the foveal and peripheral information alone, and this happens across the time-course of the saccade, though the distractor is most effective before the saccade.

3.6. Perceptual performance vs motor performance

Presenting an attentional distractor has been shown to affect various aspects of saccade planning such as saccade curvature and saccade latencies (Van der Stigchel & Theeuwes, 2005, 2006; Walker et al., 1997). We conducted an analysis on measures of motor error to verify the efficacy of the distractor. For saccade planning error, the maximum curvature of the saccade was used, measured as the largest absolute perpendicular deviation from the sample points to a straight path between the fixation cross and target (Van der Stigchel, Meeter, & Theeuwes, 2006). For each saccade, the distance between sample point and the straight path was measured for each sample (every 1 ms), and the peak deviation for the saccade was calculated as the sample with the maximum distance (Fig. 5A). We also measured saccade latency (Fig. 5B). Both saccade deviation and saccade latency were averaged across the same 75 ms time-window as the perceptual measures, so the data represented at each time-point are consistent across measures. Time-points were only compared before the onset of the saccade, as after this point any differences would not be due to the effects of attentional distraction on motor planning.

Fig. 5.

Comparison of saccade motor planning error and integration performance before saccade onset. (A) Saccade deviation versus perceptual performance for each SOA before saccade onset. Saccade deviation is calculated as maximum saccade curvature. Saccade deviation in the control condition is represented as a dotted line. Perceptual performance is plotted as the difference between predicted and observed integration performance. (B) Saccade latency versus perceptual performance for each SOA before saccade onset and perceptual performance. Saccade latency in the control condition is represented as a dotted line. Error bars are 95% confidence intervals.

To compare the difference between motor error in the control task and the distractor task at different points during the saccade, paired samples t-tests with a Bonferroni correction were used to compare control and distractor task at −150 ms and 0 ms. For saccade deviation, there was a significant difference between the conditions at −150 ms: t (6) = −7.8 p = .03; but not at 0 ms: t(6) = −0.25, p = .95. For saccade latency, there was a significant difference at −150 ms: t (6) = −7.8, p < .0001; but not at 0 ms: t(6) = −0.24, p = .89. This indicates that the distractor affected motor performance when it appeared around −150 ms before saccade onset, but this effect diminished as the offset between distractor and saccade onset lessened.

This analysis demonstrates that while measures of motor error show the expected deficit due to the presence of the attentional distractor, this can be dissociated from the perceptual error incurred from the distractor. Fig. 5 shows that the pattern of perceptual deficit can be dissociated from that of the motor deficit, indicating an independent deficit to integration, separate from any effects on oculomotor control. This indicates that pre-saccadic attention may be important in both motor planning aspects of a saccade, and in trans-saccadic integration. However, while the distractor effects in the motor variables monotonically decline towards saccade onset, the distractor effects in the integration performance increase up until saccade onset.

4. Discussion

This study measured the effect of an attentional distractor on trans-saccadic integration. The main result of this study showed that presenting a distractor during the saccade impaired observed integration performance at differing time-points during the saccade, compared to the predicted integration performance based on foveal and peripheral information alone. The time-course shows a large deficit in integration performance due to the distractor, compared to a control condition, in selected subjects who all showed integration without a distractor. This detriment is particularly prominent in the lead-up to saccade onset.

4.1. Attention disrupts saccadic integration

The results of this study show that when an attentional distractor is presented during a saccade, the observed integration performance is worse and shows more fluctuation over time than the predicted integration performance. By measuring the peripheral and foveal performance independently, we could then use maximum likelihood estimation to predict performance if integration occurred in an optimal manner. Previous evidence suggests that humans are able to integrate these two sources of information in a near-optimal manner (Ganmor et al., 2015; Hübner & Schütz, 2017; Wolf & Schütz, 2015), so if attention disrupted the integration process, observed integration performance with an attentional distractor should be worse than the predictions would suggest. This is indeed what this study showed – observed performance was worse than predicted performance to a differing extent across the saccade, showing that presentation of an attentional distractor causes a deficit in the integration of pre- and post-saccadic information at specific time-points throughout the saccade. This is evidence that attention may be a mechanism underlying trans-saccadic integration. Attention shifts to the location of an impending saccade before saccade onset (Deubel & Schneider, 1996; Kowler et al., 1995), and produces perceptual benefits such as enhanced contrast sensitivity and perceived contrast at that location (Rolfs & Carrasco, 2012). If, as suggested by the results of this study, attention plays a role in the integration process, this increase in sensitivity and contrast at the pre-saccadic location could be either a result of the need to highlight the information of this new location as important for maintenance across the saccade (Cavanagh et al., 2010; Mathot & Theeuwes, 2011), or the increase in sensitivity at the pre-saccadic location could be a predictive mechanism (Herwig & Schneider, 2014; Valsecchi & Gegenfurtner, 2016) aiming to equate the poor resolution pre-saccadic target information with the higher resolution post-saccadic target information. While the exact causal link between attention and trans-saccadic integration is beyond the scope of this study, it does provide evidence that an interruption to attention at the saccade target impairs integration, suggesting that integration may be guided by pre-saccadic attention, and that the perceptual benefits incurred by an attentional shift may play a role in perceptual continuity across saccades.

What are the potential mechanisms driving this disruption to integration? One potential explanation involves the role of LIP in the process of attentional selection, remapping and potentially, ultimately, integration. When an attentional distractor is presented, LIP neurons shift their locus of response from the saccade target to the distractor location (Goldberg, Bisley, Powell, & Gottlieb, 2006) – it is possible that, if attention acts as a guide for locations to be remapped and thus integrated (Cavanagh et al., 2010; Mathot & Theeuwes, 2011), this shift in attentional response causes this integration process to occur at the location of the distractor, rather than the saccade target. Indeed, multiple studies have shown LIP response is linked to remapped locations (Barash, Bracewell, Fogassi, Gnadt, & Andersen, 1991; Duhamel et al., 1992), and LIP response seems to shift from pre- to post-saccadic locations before the saccade (Kusunoki & Goldberg, 2003). There also seems to be a strong link between LIP response and attended stimuli – even briefly presented, task-irrelevant stimuli can elicit an “attended” LIP response (Gottlieb et al., 1998). In behavioural terms, there seems to be a strong link between the locus of attention and remapping (Cavanagh et al., 2010; Jonikaitis et al., 2013; Rolfs et al., 2011), and indeed between attention, remapping and integration (Szinte et al., 2016). If then attention, and subsequently LIP response is shifted away from the saccade target location, this could shift the “attentional pointer” (Cavanagh et al., 2010) needed to guide the remapping coordinates, impairing the remapping process at that target, thus impairing the extent to which pre- and post-saccadic information is integrated.

An alternative explanation is to consider the role of attention in integration as facilitating the binding of the two different features before and after the saccade (Treisman & Gelade, 1980; Treisman, 2006). This feature binding process seems to be reliant on attentional selection to facilitate the selection of objects for binding (Treisman & Schmidt, 1982), and some studies exploring integration of pre- and post-saccadic features have proposed that attention acts to join the stimulus features at a given location across the saccade (Irwin & Andrews, 1996; Irwin & Gordon, 1998). If this is true, then a disruption to attention may impair either the encoding of the stimulus features into short term memory, or may impair the system’s ability to spatially locate the features that must be bound, thus impairing integration.

A secondary finding of this study was that the attentional distractor also affected measures of motor control (saccade latency and curvature), and confirms that the distractor was effective in capturing attention. To maximise its efficacy, our distractor was randomly coloured, the distractor location was not visible before onset (i.e. there was no placeholder), and the onset time was variable, so there could be no suppression due to expectation, or discounting of the distractor location as irrelevant (Puntiroli et al., 2015). We also show in this study that the oculomotor plan was disrupted by the onset of the distractor, which indicates that the distractor was affecting motor error, and thus arguably covert attention (Peterson et al., 2004; Puntiroli et al., 2015; Van der Stigchel, 2010). This analysis demonstrates that the presentation of an attentional distractor disrupts both integration and motor planning, the latter of which is already known to be affected by attentional distraction. Pre-saccadic attention has been found to be strongly linked to both saccade planning (Deubel, 2008; Kowler et al., 1995; White et al., 2013) and reach planning (Jonikaitis & Deubel, 2011; Rolfs, Lawrence, & Carrasco, 2013; Stewart & Ma-Wyatt, 2015), and it has been suggested that this pre-saccadic attentional shift is used as a guidance system for the planning of these movements (Kowler et al., 1995; Stewart & Ma-Wyatt, 2015, 2017), and indeed multiple studies have shown that the presentation of a distractor during a saccade affects both temporal and spatial properties of the saccade (Findlay, 1982; Lévy-Schoen, 1969; Walker et al., 1997). This suggests that drawing the locus of attention away from the planned saccade target impairs motor performance, and also impairs integration, though at a later point in the saccade. These findings indicate that while theories that suggest that pre-saccadic attention is used for attentional motor control (Schneider, 1995) hold true, attention may play a greater role than just selecting objects for better recognition. Attention may additionally play a crucial role in trans-saccadic integration and the maintenance of perceptual stability across eye movements.

We analysed the motor consequences of the distractor in terms of saccade trajectory deviation and average saccade latency. It is currently debated whether distractor effects on average saccade latency are caused by saccadic inhibition – a specific modulation of saccade latency distributions (Buonocore & McIntosh, 2008; McIntosh & Buonocore, 2014; Reingold & Stampe, 2002) – or not (Walker & Benson, 2013, 2015). Unfortunately, there is not enough data in each time-bin to conduct an appropriate analysis of saccade latency distributions.

4.2. When does integration occur?

The results of this study clearly show that distracting attention has the most prominent effect on integration performance in the lead-up to saccade onset. This could suggest that presenting an attentional distractor impairs integration in a number of ways: first, the large detriment to performance before the saccade could suggest that the distractor disrupts the transfer of feature information obtained before the saccade, which is in line with findings that that feature information is remapped to a future retinotopic location immediately prior to a saccade (Harrison, Retell, Remington, & Mattingley, 2013). This account is supported by the fact that the decrease in performance begins to build around 100 ms before the saccade. Second, the sustained pattern of impaired integration just after saccade onset may also provide support for a late integration process, and a post-saccadic recalibration of sensory information (Deubel, Bridgeman, & Schneider, 1998). Third, the peak impairment around saccade onset could suggest a disruption to the transfer of pre-saccadic information during the saccade, so there can be no integration of pre-saccadic input with the post-saccadic percept. What do these accounts mean for the role of attention in integration? If integration is an early process, this supports both the idea that attention acts as a location marker for the information that needs to be integrated in the future (Cavanagh et al., 2010; Mathot & Theeuwes, 2011), or that attention acts in a predictive manner to equate peripheral and foveal information. If on the other hand integration is a late process, this could indicate that attention plays more of a role in the maintenance of pre- and post-saccadic information, and may suggest a more facilitatory role in the transference of this information into working memory, which is then used to compare and integrate the two sources of information (Prime, Tsotsos, Keith, & Crawford, 2007). This study does not aim to provide a definitive answer to whether integration is an early or late process, and it could be the case that attention plays a role in a number of these processes – the offset between peak motor disruption and peak integration disruption suggests that attention may act as a guidance mechanism for a number of modalities across the saccade. In the case of integration however, the peak impairment leading up to saccade onset lends support to attention as an early mechanism that facilitates the transference of information across the saccade.

4.3. Comparison between observed and predicted integration performance

Across the measured time-course of performance when a distractor was present, observed performance in the integration trials was worse than predicted performance based on optimal integration of peripheral and foveal performance. This is different from two previous studies, in which observed integration performance was much closer to optimal (Ganmor et al., 2015; Wolf & Schütz, 2015). One explanation for this overall sub-optimal integration is that integration might be impaired by the expectancy of a distractor. Since there was a distractor in every trial in the distractor condition, observers might have anticipated its appearance. Avoiding the distractor and integrating information across saccades could be considered as a dual-task condition, which limits resources that can be allocated to the integration task. There are however a number of reasons why the observed effects cannot be attributed to any dual-task interference. First, the appearance of a distractor does not constitute a “task” as such, as participants did not have to make any response to the presence of the distractor. Second, as discussed above, there was a marked effect of the distractor on oculomotor error, indicating that the distractor was effective in manipulating attention. Additionally, the performance for integration and individual conditions fluctuates across the measured time-course – if this were a dual-task effect, we would expect to see a constant detriment, irrespective of the timing of the distractor. Finally, this study is essentially a between-conditions analysis: we are not looking at the effect of the distractor on peripheral or foveal performance alone, but are comparing these conditions to the trans-saccadic integration benefit. Thus, if there were any simple dual-tasks effects, they would be equal in all conditions and thus negated by this comparison.

5. Conclusion

This study demonstrated that the presentation of an attentional distractor during a saccade impaired integration of pre- and post-saccadic information, as well as impairing saccade motor performance. This suggests that pre-saccadic attention may play a role not only in motor guidance aspects of eye movements, but also in the integration of peripheral and foveal information across saccades, and therefore, ultimately, in perceptual stability across saccades.