Rapid eye movements entail retinal motion streaks that provide spatiotemporal continuity of objects and guide gaze correction.
Abstract
Rapid eye movements (saccades) incessantly shift objects across the retina. To establish object correspondence, the visual system is thought to match surface features of objects across saccades. Here, we show that an object’s intrasaccadic retinal trace—a signal previously considered unavailable to visual processing—facilitates this match making. Human observers made saccades to a cued target in a circular stimulus array. Using high-speed visual projection, we swiftly rotated this array during the eyes’ flight, displaying continuous intrasaccadic target motion. Observers’ saccades landed between the target and a distractor, prompting secondary saccades. Independently of the availability of object features, which we controlled tightly, target motion increased the rate and reduced the latency of gaze-correcting saccades to the initial presaccadic target, in particular when the target’s stimulus features incidentally gave rise to efficient motion streaks. These results suggest that intrasaccadic visual information informs the establishment of object correspondence and jump-starts gaze correction.
INTRODUCTION
Saccadic eye movements are the fastest and most frequent movements of the human body. They provide high-acuity vision by placing the fovea, a small central pit on the retinal surface that features the highest density of cone photoreceptor cells, at new parts of the visual scene. At the same time, saccades result in retinal translations that constantly shift the projections of objects onto the retina and impose considerable motion smear on the retinal image. These consequences, however, do not impair our visual experience, a phenomenon widely known as visual stability (1–4). A core component of visual stability is the establishment of object correspondence across saccades: How does the visual system determine whether any object located in the periphery before a saccade is the same as the object close to the fovea right after that saccade has landed?
There is good evidence that visual short-term memory (VSTM) enables the matching of objects across saccades (5). For instance, the study in Hollingworth et al. (6) showed that surface features of visual objects encoded in VSTM, such as color or object identity, could be used for gaze correction when targets were displaced during saccades. To some extent, this result contradicted object-file theory (7), which supports the notion that objects are referenced via spatiotemporal continuity, not surface features (8). Later studies then suggested that both surface features and spatiotemporal continuity could contribute to object correspondence, across brief occlusions while fixating (9) and across saccades (10).
One ubiquitous source of information for object correspondence has been neglected by all studies up to this point: Intrasaccadic object motion across the retina may provide spatiotemporal continuity and access to surface features throughout the saccade. As illustrated in Fig. 1, because of the temporal integration in the visual system, objects moving across the retina at the high velocities of saccades [mean velocities saturate at 200 to 400°/s and peak velocities at 400 to 700°/s; (11)] routinely produce smeared traces, so-called motion streaks (12–17). Intrasaccadic motion streaks result from spatially circumscribed visual stimuli (such as objects in scenes) and can be regarded as a subclass of the full-field intrasaccadic smear that is induced when the entire visual scene is shifted across the retina (18). During natural vision, intrasaccadic smear is largely omitted from conscious visual perception, likely because of masking by pre- and postsaccadic retinal images (19–22), as well as extraretinal mechanisms such as saccadic suppression (23–27), although this notion has recently been challenged again (28). Most experiments on object correspondence were thus built on the premise that “vision is suppressed, creating a gap in perceptual input” [(10), p. 66] and that “people are virtually blind” [(6), p. 163] during saccades. In contrast to this premise, we have recently shown that observers can use intrasaccadic motion streaks to tell presaccadic objects from identical distractors upon saccade landing (29). The crucial question—whether intrasaccadic streaks could be used to establish object correspondence across saccades—remains unanswered, however. To test this idea, implicit behavioral measures rather than explicit perceptual reports must be used, as perceptual reports may draw observers’ attention to a source of information that they might have otherwise ignored. Here, using a high-speed projection system, we adapted a classic gaze correction paradigm (6) to investigate whether continuous object motion—exclusively present during saccades—may serve object continuity and, hence, facilitate gaze correction. In two experiments, we experimentally induced an intrasaccadic displacement of a target stimulus and hypothesized that secondary saccades to that target would be more accurate, and potentially faster, if the target had a continuous retinal trace throughout the saccade (similar to that induced naturally during each rapid eye movement; Fig. 1) than if it did not.
Fig. 1. Illustration of intrasaccadic motion streaks.
When making a saccade toward the bird on the right, its retinal projection rapidly travels from a peripheral location (fixation 1) to a foveal location (fixation 2), producing a motion streak along its retinal trajectory. This streak literally connects an object’s pre- and postsaccadic locations on the retina, possibly providing spatiotemporal continuity that may help establish object correspondence. Photo credit: Richard Schweitzer.
RESULTS
Postsaccadic surface features and intrasaccadic motion drive gaze correction
In a first experiment, our central research question was whether continuous motion would lead to more accurate gaze correction, that is, a higher proportion of secondary saccades to the initial presaccadic target stimulus. This analysis of proportions of secondary saccades to the target was conducted as planned during preregistration. Crucially, intrasaccadic displacements—whether continuous or apparent—had to occur exclusively during saccades, as extending intrasaccadic stimulus manipulations beyond saccade offset has been shown to drastically alter detection performance and subjective appearance of stimuli (12, 14, 17, 30, 31), despite the finding that the window of saccadic suppression often exceeds the saccade duration (22). Having excluded those displacements that were not strictly intrasaccadic (see the “Preprocessing” section), presentations finished, on average, 10.7 ms (SD = 3.1) before saccade offset (Fig. 2B), and there was no difference between motion-absent and motion-present conditions [Mabsent = 10.68 ms, SDabsent = 3.08, Mpresent = 10.75 ms, SDpresent = 3.08; paired t test: t(9) = 1.24, P = 0.243].
Fig. 2. Probing the role of postsaccadic surface features and intrasaccadic motion in gaze correction (experiment 1).
(A) Observers made a primary saccade to an exogenously cued target noise patch stimulus (one of two types). Strictly during the saccade, the target rapidly shifted positions, consistent with a 30° clockwise (CW) or counterclockwise (CCW) rotation of the entire stimulus array, so that primary saccades landed between the initially cued stimulus (the target) and the other-type stimulus (the distractor). The intrasaccadic stimulus motion was either continuous throughout 14.6 ms of rotation (i.e., 21 equidistant steps along its circular trajectory) or absent (blank screen for 14.6 ms between first and final stimulus positions). After the stimulus’s motion, pixel masks were displayed with a varying delay (surface-feature durations: 0, 25, 50, 100, 200, or 600 ms), thus occluding the identity of postsaccadic objects and limiting the observers’ ability to establish trans-saccadic correspondence using object features. (B) Stimulus motion was presented strictly during saccades, finishing, on average, 10.7 ms before saccade offset. (C) Probability of observers making a secondary saccade toward the initial presaccadic target was a function of surface-feature duration, as well as the presence of intrasaccadic motion (purple versus green points, respectively; error bars indicate ±SEM). The beige area illustrates the temporal interval in which intrasaccadic motion took place. Solid lines show predictions of the mixed-effects exponential growth model describing the increase of proportions with increasing surface-feature duration. Average parameter estimates are shown in the table below the model formula. (D) Mean differences between motion conditions for each surface-feature duration with corresponding 95% confidence intervals (CIs; gray-shaded area).
We first investigated the time course of stimulus feature processing after stimulus displacements. If object correspondence was established by comparing postsaccadic object features with presaccadic object features represented in VSTM (6, 10), then we would expect that observers orient their gaze more frequently toward the original presaccadic target when postsaccadic object features are available for longer periods of time before being occluded by masks (Fig. 2B). As shown in Fig. 2C, a clear main effect of surface-feature duration was found [F(5,45) = 22.89, η2 = 0.31, P < 0.001, PGG < 0.001]. Average proportions of secondary saccades made toward the target stimulus increased rapidly for surface-feature durations of 0 to 50 ms (0 ms: M = 0.53, SD = 0.03; 25 ms: M = 0.63, SD = 0.10; 50 ms: M = 0.74, SD = 0.17), reaching an asymptote 100 ms after displacements occurred (100 ms: M = 0.79, SD = 0.18; 200 ms: M = 0.79, SD = 0.19; 600 ms: M = 0.80, SD = 0.19). Note that average proportions asymptoted at a relatively low level, which was caused by three observers who selected the initial presaccadic target on a proportion of trials that was barely above chance (i.e., 55.2, 51.2, and 56.9% at the maximum surface-feature duration of 600 ms). There was, however, no reason or preregistered criterion for their exclusion.
When intrasaccadic motion was absent and surface features were masked right after displacement, the proportion of secondary saccades to the target was no different from chance level [M = 0.50, SD = 0.02, t(9) = 0.79, P = 0.447], as no information was available to perform gaze correction. Crucially, when continuous intrasaccadic motion was present, although postmotion surface features were unavailable, secondary saccades were made to the target in 56.1% of trials (SD = 6.4), a proportion significantly higher than the corresponding motion-absent condition [paired t test: t(9) = 2.70, P = 0.024]. Moreover, we found a significant increase in gaze correction accuracy across surface-feature durations [F(1,9) = 18.29, η2 = 0.01, P = 0.002]. Although this effect decreased slightly with increasing surface-feature duration (Fig. 2D), no significant interaction between the two factors was found [F(5,45) = 1.36, η2 < 0.01, P = 0.256, PGG = 0.274]. To further scrutinize this result, we fitted two linear mixed-effects models to the differences between motion conditions shown in Fig. 2D, one intercept-only model (β0 = 0.027, t = 4.51, 95% confidence interval (CI) [0.014, 0.040], P = 0.001) and one involving the factor surface-feature duration in addition. Hierarchical model comparisons further corroborated the previous results: Adding the factor surface-feature duration to the intercept-only model did not significantly improve the fit [BF10 = 0.001, ΔLL = +3.5, χ2(5) = 7.05, P = 0.217], suggesting that the effect of intrasaccadic motion was largely additive to the effect of surface-feature duration.
Intrasaccadic motion results in early onset of information accumulation for gaze correction
What is the nature of the effect of intrasaccadic object motion in the gaze correction paradigm? To find out, we performed an exploratory analysis: We fitted an exponential model (see the “Analysis” section) to the probability of making a secondary saccade to the target (Fig. 2C). Following this procedure, we estimated three parameters of the time course, i.e., asymptote (λ), slope (β), and onset (δ), for motion-absent versus motion-present conditions. We adopted a mixed-effects approach that allowed the three parameters to vary independently for each observer (32), so that paired hypothesis tests could be performed. Mean estimates and visualizations of the effects of parameters are shown in the inset embedded in Fig. 2C.
On the basis of this model, several hypotheses about the benefit of intrasaccadic motion could be distinguished. First, intrasaccadic motion may result in a gain increase for postsaccadic object features, such that additional information would be accumulated. In this case, we would expect that performance in the motion-present condition has the same time of onset and the same slope but then reaches a higher asymptote. Estimated λ were slightly larger in the motion-present condition (λpresent = 0.304, SEpresent = 0.058) than in the motion-absent condition (λabsent = 0.291, SEabsent = 0.061), but this difference did not reach significance [t(9) = 1.34, P = 0.214]. Second, intrasaccadic motion may lead to an increase of the rate with which postsaccadic information is accumulated, which would predict a steeper slope of the exponential model. Estimates of β, however, did also not differ between conditions [βpresent = 0.027, SEpresent = 0.003, βabsent = 0.029, SEabsent = 0.003, t(9) = −0.49, P = 0.634], providing no evidence for such rate increase. Third, despite the fact that all object displacements were finished strictly while the eye was still in flight, continuous intrasaccadic object motion may have revealed the postsaccadic location of the target at an earlier stage than if it was absent, thus allowing the onset of information accumulation to occur already during the ongoing motion. This would result in a shift of the exponential function to the left. Estimates of the onset parameter δ revealed a significant difference between the two conditions [δpresent = −6.969, SEpresent = 2.076, δabsent = 0.446, SEabsent = 0.053, t(9) = −3.45, P = 0.007]. The results of this analysis suggest that the observed benefit is mainly caused by an earlier availability of object location, which is revealed during intrasaccadic object motion.
Postsaccadic surface features and intrasaccadic motion reduce the latency of gaze correction
Given that the presence of intrasaccadic motion increased the likelihood of secondary saccades to the presaccadic target in a way consistent with an earlier onset of postsaccadic target localization, we next performed the planned analysis of secondary saccade latency (Fig. 3). We expected a facilitation of secondary saccade latencies when directed toward the target but not when directed toward the distractor. We first added the destination of the secondary saccade (distractor or target) as a within-subject factor to an analysis of variance (ANOVA). We found that, on average, secondary saccades to target stimuli were initiated slightly faster after primary saccade offset (M = 252.3 ms, SD = 37.4) than secondary saccades to distractor stimuli (M = 259.9 ms, SD = 47.6), but this difference did not reach significance [F(1,9) = 0.44, η2 = 0.01, P = 0.524]. Average secondary saccade latencies (Fig. 4C) were well consistent with those found in previous studies using similar paradigms [cf. (6)]. They varied significantly across surface-feature durations [F(5,45) = 3.94, η2 = 0.04, P = 0.005, PGG = 0.032] following a nonlinear time course (Fig. 3) and with only marginal differences between target-bound and distractor-bound secondary saccades [F(5,45) = 2.81, η2 = 0.01, P = 0.027, PGG = 0.055]. Specifically, secondary saccade latencies were lowest at 600 ms (M = 237.4 ms, SD = 33.0), when mask onsets almost exclusively occurred after the secondary saccade was already made. Contrary to our initial hypothesis, the ANOVA did not provide evidence for a main effect of intrasaccadic motion [F(1,9) = 0.54, η2 < 0.01, P = 0.482] or an interaction of the latter with the destination of the secondary saccade [F(1,9) = 1.38, η2 < 0.01, P = 0.270]. This approach, however, suffers from a lack of data in some conditions: As the destination of the secondary saccade is not a systematically varied experimental factor, but rather a dependent variable, there is large variation in the number of trials that enter aggregated cell means, a tendency that intensifies with increasing surface-feature durations. Some observers hardly made any secondary saccades to the distractor at long surface-durations, so that these means can become extremely unreliable. As a consequence, repeated-measures ANOVAs on secondary saccade latencies were also run separately for target-bound and distractor-bound secondary saccades. This was crucial to investigate the expected effect of target-congruent motion on target-bound secondary saccades, for which individual mean saccade latencies were reliably estimated.
Fig. 3. Intrasaccadic motion and surface-feature duration affect the latency of gaze correction.
Secondary saccade latency across observers when making secondary saccades to either the initial presaccadic target (thick lines, circles) or the distractor (thin lines, triangles), depending on surface-feature duration and presence of intrasaccadic motion (purple versus green points and lines; error bars indicate ±SEM). The beige area indicates the temporal interval of target motion, and the vertical dashed line shows the average time of saccade offset after motion offset. Solid lines are predictions of two mixed-effects generalized additive models (GAMs) that describe the time course of observers’ secondary saccade latencies as a function of increasing surface-feature duration. Parametric coefficients of the models indicated an overall significant reduction of secondary saccade latency in the motion-present condition when saccades were directed to the target (estimate = −5.99, t = −2.21, P = 0.028) but not when they were directed to the distractor (estimate = 2.38, t = 0.49, P = 0.624). The models’ difference smooth terms further suggested a time course modulation due to intrasaccadic motion for target-bound secondary saccades [estimated degrees of freedom (edf) = 9.91, F = 2.99, P = 0.001] but again not for distractor-bound secondary saccades (edf = 1.01, F = 0.04, P = 0.836).
Fig. 4. Primary saccade landing positions influence gaze correction.
(A) ddiff is the difference between two distances, from primary saccade landing to the target and to the distractor, respectively. Positive values denote that saccades landed closer to the target than to the distractor. (B) Logistic fits modeling the relationship between ddiff and the proportion of making a secondary saccade to the target for motion-absent (purple) and motion-present (green) conditions. Panels show results for each surface-feature duration separately. Points indicate group means per 0.5-dva bin. Shaded error bars indicate 95% CIs determined by parametric bootstrapping. (C) Distributions of secondary saccade latencies for each observer. Upper and lower densities represent the motion-present and motion-absent condition, respectively. (D) Linear fits predicting inverse secondary saccade latencies to the target stimulus (transformed back to raw secondary saccade latencies) based on ddiff, surface-feature duration, and presence of intrasaccadic motion.
For secondary saccades made to the target stimulus (Fig. 3), we found small, but significant, overall latency reductions in the motion-present condition (Mpresent = 249.9 ms, SD = 35.6) relative to the motion-absent condition [Mabsent = 254.7 ms, SD = 39.4; F(1,9) = 5.25, η2 = 0.002, P = 0.047] that was only marginally modulated by surface-feature duration [F(5,45) = 2.06, η2 < 0.01, P = 0.089, PGG = 0.133]. For secondary saccades made to the distractor, we observed neither a main effect of intrasaccadic motion [Mabsent = 258.3 ms, SD = 42.9; Mpresent = 261.5 ms, SD = 52.8; F(1,9) = 0.17, η2 < 0.01, P = 0.686] nor an interaction of motion and surface-feature duration [F(5,45) = 0.61, η2 < 0.01, P = 0.691, PGG = 0.586]. Notably, in the absence of surface features, the presence of intrasaccadic motion significantly reduced the secondary saccade latency to the target [Mabsent-present = 13.8 ms; paired t test: t(9) = 3.26, P = 0.001], a reduction remarkably similar to the duration of intrasaccadic motion, i.e., 14.6 ms. This was not the case for the distractor-bound secondary saccades [Mabsent-present = −5.4 ms; paired t test: t(9) = −1.12, P = 0.293]. To scrutinize the presence of an interaction between intrasaccadic motion and surface-feature duration for the target-bound secondary saccades, we first fitted an intercept-only mixed-effects model to the differences in inverse-transformed saccade latencies between motion-absent and motion-present conditions. The significant model intercept (β0 = 0.07, t = 2.76, 95% CI [0.015, 0.125], P = 0.020) confirmed that the secondary saccades were initiated faster in the motion-present condition. Adding the factor surface-feature duration significantly improved goodness of fit, although the intercept-only model was more parsimonious [BF10 = 0.06, ΔLL = +7.6, χ2(5) = 14.93, P = 0.011], providing evidence that latency differences between motion conditions did vary with surface-feature durations. For instance, while the effect of intrasaccadic motion was quite conspicuous at 0 ms (β = 0.12, t = 2.75, 95% CI [0.04, 0.21], P = 0.001), it was nearly reversed at 50 ms (β = −0.11, t = −2.46, 95% CI [−0.19, −0.02], P = 0.017). Applying the model comparison scheme to distractor-bound secondary saccades revealed neither a significant model intercept (β0 = 0.01, t = 0.230, 95% CI [−0.07, 0.10], P = 0.819) nor an increase in log-likelihood when adding the factor surface-feature duration [BF10 < 0.001, ΔLL = +1.1, χ2(5) = 2.30, P = 0.806].
Together, our results provide evidence that the presence of intrasaccadic stimulus motion not only increased the proportion of secondary saccades to the initial presaccadic target, but also reduced their latency. Although the main effect of intrasaccadic motion was significant, there was considerable variability across surface-feature durations. This may be related to an interference of the visual transient introduced by the mask onset with the preparation of the secondary saccade [e.g., (33)], especially as the time course was similar for both target-bound and distractor-bound saccades.
Primary saccade landing positions influence gaze correction
If more than one candidate object for postsaccadic gaze correction is available, then a secondary saccade often goes to the closer one (6). To investigate a potential interaction of this effect with our observed influence of surface-feature duration and intrasaccadic object motion, we conducted the following exploratory analysis on the data collected in experiment 1. For each trial, we computed the Euclidean distance from the landing position of the primary saccade to the center of the target and to the center of the distractor (Fig. 4A). Positive values of the difference between these distances (ddiff) denote landing positions closer to the target than to the distractor. Subsequently, we used ddiff in mixed-effects regressions to predict saccades to the target as opposed to the distractor (logistic regression; Fig. 4B) and inverse secondary saccade latency (linear regression; Fig. 4D) on a single-trial level.
In predicting secondary saccades to the target, ddiff drastically improved the model fit (compared to a model assuming only surface-feature duration) as an additive predictor [BF10 > 1050, ΔLL = +576.7, χ2(1) = 1153.26, P < 0.001], but only marginally in its interaction with surface-feature duration [BF10 < 0.001, ΔLL = +5.2, χ2(5) = 10.55, P = 0.061]. That is, landing 1 degree of visual angle (dva) closer to the target increased the probability of making a secondary saccade to the target across all conditions by a factor of 1.6 (β = 0.47, z = 31.45, 95% CI [0.47, 0.65], P < 0.001). As shown in Fig. 4B, this slope decreased slightly the longer surface features were available upon landing. For instance, the landing position effect was significantly larger than average at a surface-feature duration of 25 ms (β = 0.06, z = 2.01, 95% CI [0.001, 0.123], P = 0.045) but significantly smaller at 200 ms (β = −0.08, z = −2.21, 95% CI [−0.15, −0.01], P = 0.027). The presence of intrasaccadic target motion significantly increased the probability of secondary saccades to the target across surface-feature durations (β = 0.17, z = 4.41, 95% CI [0.09, 0.24], P < 0.001), an effect that was even enhanced at a surface-feature duration of 0 ms (β = 0.17, z = 2.11, 95% CI [0.01, 0.33], P = 0.034). Hierarchical model comparisons suggested that the effects of the two factors were largely additive: Including the presence of intrasaccadic motion improved the fit [BF10 = 849.46, ΔLL = +12.3, χ2(1) = 23.29, P < 0.001], while its interaction with ddiff or surface-feature duration did not [BF10 < 0.001, ΔLL = +7.3, χ2(11) = 16.75, P = 0.116].
The same analyses were conducted for inverse secondary saccade latency, provided that these saccades were made to the target. A distribution of untransformed secondary saccade latencies (Mabsent = 254.6 ms, SDabsent = 39.5, Mpresent = 250.4 ms, SDpresent = 36.4), stacked across observers, is shown in Fig. 4C. Again, ddiff predicted saccade latency very well, but more so as an additive predictor [BF10 > 1050, ΔLL = +191.0, χ2(1) = 381.67, P < 0.001] than combined with its interaction with surface-feature duration [BF10 < 0.001, ΔLL = +7.1, χ2(5) = 15.39, P = 0.008]. Across conditions, landing 1 dva closer to the target reduced secondary saccade latency by 6.9 ms (β = 0.14, t = 19.63, 95% CI [0.12, 0.15], P < 0.001). This effect was significantly modulated at some surface-feature durations (100 ms: β = −0.03, t = −2.15, 95% CI [−0.06, −0.01], P = 0.031; 200 ms: β = 0.04, t = 2.99, 95% CI [0.02, 0.07], P = 0.003). When intrasaccadic motion was present, overall secondary saccade latency to the target was significantly reduced by 3.7 ms (β = 0.07, t = 3.46, 95% CI [0.03, 0.11], P = 0.001). As shown in Fig. 4D, this effect did not vary significantly across surface-feature durations (0 ms: β = 0.10, t = 1.88, 95% CI [−0.01, 0.20], P = 0.060; 25 ms: β = −0.04, t = −0.89, 95% CI [−0.14, 0.05], P = 0.372; 100 ms: β = 0.03, t = 0.64, 95% CI [−0.06, 0.11], P = 0.524; 200 ms: β = 0.02, t = 0.44, 95% CI [−0.07, 0.10], P = 0.661), except at 50 ms (β = −0.11, t = 2.59, 95% CI [−0.19, −0.02], P = 0.010). Moreover, there was neither an interaction between intrasaccadic motion and ddiff (β = 0.001, t = 0.07, 95% CI [−0.03, 0.03], P = 0.944) nor any higher-level interaction [for full results, see Open Methods at Open Science Framework (OSF)]. Model comparisons revealed that a model including the presence of intrasaccadic motion as an additive factor should be preferred to a model including only ddiff and surface-feature duration [BF10= 11.6, ΔLL = +7.0, χ2(1) = 14.37, P < 0.001]. The full model (including all interactions) slightly improved the fit of the model over the additive model, whereas Bayes factor strongly penalized its considerably larger complexity [BF10< 0.001, ΔLL = +11.0, χ2(11) = 20.41, P = 0.040].
Efficient motion streaks facilitate gaze correction
Last, to establish which stimulus features drive secondary saccades to the target stimulus, we performed a planned, large-scale reverse regression analysis (see Materials and Methods). Both contrast sensitivity for moving stimuli and motion perception—especially of high–spatial frequency (SF) stimuli—are known to dissipate at saccadic velocities (34–38). We hypothesized, therefore, that the rapid movement of the target across the retina produced intrasaccadic motion streaks (12–15, 17). Geisler (15) proposed that motion-streak detectors are composed of direction-selective cells with orientation tuning perpendicular to motion direction (i.e., classic motion detectors) and non–direction-selective cells with orientation tuning parallel to motion direction (i.e., sensitive to motion streaks). There is evidence that such “parallel motion direction selectivity” exists in the visual cortex (16, 39, 40). If fast motion can be coded as an orientation, then the orientation of a stimulus parallel to its motion trajectory on the retina should produce especially distinctive motion streaks (29). In this case, and if gaze correction is indeed informed by intrasaccadic motion streaks, one would expect that secondary saccades to the target should be facilitated—in their accuracy and speed—by orientations (incidentally) close to the direction of the target object’s retinal motion trajectory.
As the noise patches used in this task could potentially contain all possible orientations, as well as SFs from 0.25 to 1 cycle per dva (cpd), it was possible to describe each noise patch—both target and distractor—in terms of energy per SF-orientation component (see Materials and Methods for details). In brief, for each trial of experiment 1 (regardless of whether intrasaccadic target motion was absent or present), we obtained a filter response map for the target stimulus by convolving the noise patch with a bank of Gabor filters (Fig. 5A). Next, we extracted the angle of the target’s trajectory across the retina, which was determined by the target trajectory presented on the screen and the gaze trajectory during presentation (for an illustration, see Fig. 5B). We then normalized stimulus orientations using this retinal angle, resulting in a measure of relative orientation. As a consequence, stimulus orientations parallel to the retinal angle would result in relative orientations of 0°, whereas stimulus orientations orthogonal to the retinal angle would result in relative orientations of 90°. Last, we first ran logistic mixed-effects regressions to predict secondary saccades to the target (as opposed to the distractor) from the filter responses present in all available target stimuli. Second, a linear version of the analysis was performed to predict fast saccadic reactions from the same filter responses, provided that secondary saccades were made to the target. Note that a positive relationship between filter responses in a given SF-orientation component and the respective dependent variable implies that this component is either beneficial for gaze correction to the target (Fig. 5C) or drives secondary saccades to the target at low latencies (Fig. 5D).
Fig. 5. Efficient motion streaks facilitate gaze correction.
(A) Example of a filter energy map computed by convolving the noise patch stimulus with a bank of Gabor filters. (B) The retinal trajectory of the target stimulus is the vector sum of the target’s trajectory presented on the screen and the eye position vector during presentation. We computed relative orientation by normalizing the stimulus’s orientation components using the angle of the retinal trajectory. As illustrated by motion filtering applied to the noise patch, orientations parallel to the stimulus’s motion trajectory on the retina should lead to distinctive motion streaks. (C) Results from the logistic reverse regression analysis, fitted by the multivariate GAM, averaged across all surface-feature durations. High z scores (orange) imply that filter responses in a given SF-orientation component predict the occurrence of a secondary saccade to the target when intrasaccadic motion was present (middle) and absent (left). Dotted lines demarcate the transition from negative to positive z scores estimated by the linear model corresponding to the GAM. Upper marginal means show the effect of relative orientation averaged across all SF components. The surface difference (right) clearly indicates that secondary saccades to the target (and not to the distractor) were driven by stimulus orientations parallel to the stimulus’s retinal trajectory, suggesting a role of temporal integration of fast-moving stimuli, i.e., motion streaks. (D) Results from the linear reverse regression analysis, using the inverse latency of secondary saccades made to the target as a dependent variable. High t-scores (orange) mark those SF-orientation components facilitating short saccadic reaction times. The pattern suggests that the same parallel orientations that drove secondary saccades to the target also reduced their latencies.
When intrasaccadic motion was absent (Fig. 5C, left), low SFs predicted secondary saccades to the stimulus better than high SFs [generalized additive model (GAM): estimated degrees of freedom (edf) = 4.02, F = 15.38, P < 0.001; linear model (LM): β = −2.48, t = −5.13, 95% CI [−3.43, −1.53], P < 0.001], whereas relative orientation did not have any impact on gaze correction (GAM: edf = 1.00, F = 0.16, P = 0.689; LM: β = 0.19, t = 1.02, 95% CI [−0.17, 0.54], P = 0.306). By contrast, when intrasaccadic motion was present, saccades to target stimuli were driven by smaller relative orientations (GAM: edf = 1.00, F = 15.43, P < 0.001; LM: β = −1.53, t = −5.97, 95% CI [−2.04, −1.03], P < 0.001), peaking at relative orientations close to zero (i.e., orientations parallel to the retinal trajectory; Fig. 5C, middle). Moreover, although low SFs were still most relevant, the difference between low and high SFs was reduced (GAM: edf = 2.14, F = 5.01, P = 0.005; LM: β = 1.43, t = 2.09, 95% CI [0.08, 2.77], P = 0.036), suggesting that high SFs played a larger role when intrasaccadic motion was available. Crucially, a significant interaction between SFs and relative orientation in the motion-present condition suggested that high SFs were not simply globally more influential but gained relevance at relative orientations close to zero (GAM: edf = 1.01, F = 21.97, P < 0.001; LM: β = −3.33, t = −4.63, 95% CI [−4.74, −1.91], P < 0.001), that is, when (high-SF) stimulus orientations were parallel to the stimulus’s retinal trajectory. This interaction was not present in the movement-absent condition (GAM: edf = 1.0, F = 2.24, P = 0.134; LM: β = 0.75, t = 1.47, 95% CI [−0.25, 1.74], P = 0.143). The right panel of Fig. 5C shows the difference surface of GAM fits for the two experimental conditions. Both conditions were similar with respect to the high predictive value for low-SF components, suggesting that mainly low SFs served as cues to initiate secondary saccades to target and distractor stimuli. This result seems plausible, not only because postsaccadic stimulus locations were in the visual periphery but also because filter responses to low SFs were more dissimilar between distractor and target than to high SFs (because of the way luminance was added or subtracted to make the noise patches dissimilar; see Materials and Methods) and therefore allowed for better discrimination between the two stimuli. For instance, filter responses to the target and filter responses to the distractor were positively correlated at a SF of 1 cpd [Pearson correlation coefficient, r(19176) = 0.235, 95% CI [0.222, 0.249], P < 0.001] but strongly negatively correlated at a SF of 0.25 cpd [r(19176) = −0.655, 95% CI [−0.663, −0.647], P < 0.001]. However, low SFs were beneficial in both motion conditions. Close inspection of the difference surface suggests that mid- and high-SF information drove saccades to the target only when intrasaccadic motion was presented and orientations were close to parallel to the target’s retinal trajectory.
Last, the analysis of the linear relationships between filter responses and inverse secondary saccade latency revealed that a similar principle applied to the generation of low-latency secondary saccades to the target. In the absence of intrasaccadic motion (Fig. 5D, left), secondary saccade latency was influenced by neither SF (GAM: edf = 1.00, F = 0.26, P = 0.611; LM: β = 0.23, t = 0.55, 95% CI [−0.59, 1.06], P = 0.580) nor relative orientation (GAM: edf = 1.81, F = 0.99, P = 0.369; LM: β = −0.08, t = −0.53, 95% CI [−0.39, 0.23], P = 0.598), or their interaction (GAM: edf = 1.39, F = 0.39, P = 0.710; LM: β = −0.14, t = −0.33, 95% CI [−1.02, 0.73], P = 0.745). In contrast, when motion was present (Fig. 5D, middle), we observed effects of both SF (GAM: edf = 2.43, F = 3.65, P = 0.012; LM: β = 2.67, t = 4.47, 95% CI [1.50, 3.84], P < 0.001) and relative orientation (GAM: edf = 1.00, F = 14.55, P < 0.001; LM: β = −1.21, t = −5.42, 95% CI [−1.66, −0.77], P < 0.001), as well as a significant interaction between these two predictors (GAM: edf = 1.06, F = 14.91, P < 0.001; LM: β = −2.52, t = −4.02, 95% CI [−3.75, −1.22], P < 0.001). This finding suggests that secondary saccade latencies decreased when continuously moving targets had higher energy around SFs of 1 cpd and relative orientations of zero, in other words, when (relatively) high-SF orientations parallel to the target’s retinal trajectory were present. These components were thus able to not only drive secondary saccades to the target but also increased the speed of their initiation.
Effects of spatiotemporal and surface-feature congruency on gaze correction
While results of experiment 1 suggested that intrasaccadic motion of the target facilitated secondary saccades in both accuracy and latency, the source of this facilitation remains unclear. That is, object correspondence could be established via continuity of surface features throughout the duration of a saccade (thus facilitating feature matching) or via a spatial connection between pre- and postsaccadic locations of the target object (e.g., facilitating attentive tracking). While in natural vision both components are always present, we intended to disambiguate these influences in a second experiment. Experiment 2 deviated from the first in only two key aspects. First, in experiment 1, it was only the target object that was present during the brief motion interval. This was intended to reduce the complexity of the intrasaccadic motion signal, and it was necessary for the previously reported reverse regression analysis, as we wanted to attribute changes of the dependent variables specifically to the features and retinal trajectories of the target. In experiment 2, we introduced the continuous rotation of the entire stimulus array (Fig. 6A), which resulted in a much more diverse retinal pattern of motion streaks, as each object’s individual motion direction significantly altered its respective retinal trajectory (Fig. 6B). Some objects could even become briefly stabilized on the retina when experimentally induced object motion incidentally counteracted saccade-induced retinal motion. While this manipulation adds complexity, it might rule out that a singular motion event at the target provided a localized attentional cue that captured secondary saccades. Second, experiment 2 also manipulated the extent to which each object’s intrasaccadic motion was congruent with the overall trans-saccadic rotation of the stimulus array. In experiment 1, continuous intrasaccadic motion of the target stimulus was always congruent with the target’s trans-saccadic displacement both in terms of its motion direction and the surface-feature that it carried. In experiment 2, in contrast, we orthogonally varied both motion-direction congruency and surface-feature congruency (see the “Procedure and task” section and Fig. 6C), in addition to a neutral (i.e., motion-absent) condition. This experiment will shed light on the respective contributions of these factors to establishing object correspondence across saccades as indicated by gaze correction upon saccade landing.
Fig. 6. Manipulating the congruency of intrasaccadic object motion (experiment 2).
(A) The entire stimulus array, not only the target, rotated continuously for 14.6 ms, once the onset of the primary saccade (gaze positions illustrated in blue dashed lines) was detected. Presaccadic object locations are shown as dashed circles and continuous intrasaccadic motion as arrows in the color corresponding to the polarity of the moving object. The panel thus illustrates a trial with congruent intrasaccadic rotation (in CCW direction) and congruent surface features. (B) Illustration of retinal object trajectories in a single experimental trial corresponding to the condition illustrated in (A). Eccentricities are plotted in degrees of visual angle. (C) Illustration of the five motion conditions used in experiment 2. The neutral and congruent/congruent conditions correspond to the motion-absent and motion-present conditions in experiment 1. (D) Proportions of secondary saccades made to the original presaccadic target (top) and secondary saccade latencies (bottom) averaged across both target- and distractor-bound saccades. Transparent points indicate individual means, and all error bars indicate within-subject ± SEM.
As in experiment 1, the intrasaccadic motion of the entire stimulus array was presented strictly intrasaccadically with motion offsets occurring 11.5 ms (SD = 3.1) before saccade offset, an average that was comparable in all five motion-congruency conditions [incongr./incongr.: M = 11.4 ms, SD = 3.0; incongr./congr.: M = 11.6 ms, SD = 3.3; neutral: M = 11.5 ms, SD = 3.0; congr./incongr.: M = 11.7 ms, SD = 3.1; congr./congr.: M = 11.6 ms, SD = 3.2; F(4,36) = 1.17, η2 = 0.001, P = 0.342, PGG = 0.339].
As planned during preregistration of experiment 2, we first analyzed proportions of secondary saccades made to the initial target (Fig. 6D, top). Hierarchical model comparisons revealed that the model involving all five levels of motion congruency explained the data better than the intercept-only model [BF10= 2.57, ΔLL = +8.8, χ2(4) = 17.53, P = 0.002], suggesting that some conditions significantly deviated from the grand mean of 75.3% target-bound secondary saccades. Specifically, this was the case when both motion direction and surface features were either congruent (β = 0.04, t = 3.30, 95% CI [0.02, 0.07], P = 0.002) or incongruent (β = −0.04, t = −3.34, 95% CI [−0.07, −0.02], P = 0.002), but not when directions were congruent, but features incongruent (β = 0.024, t = 1.84, 95% CI [−0.01, 0.05], P = 0.287), nor when directions were incongruent, but features congruent (β = −0.018, t = −1.38, 95% CI [−0.05, 0.01], P = 0.175). To further scrutinize this result, we normalized each observer’s performance by subtracting their respective performance in the neutral condition and subsequently ran a 2×2 repeated-measures ANOVA, which confirmed the presence of main effects of motion direction [Mcongr. = +3.9%, Mincongr. = −2.5%, F(1,9) = 7.87, η2 = 0.21, P = 0.021] and surface features [Mcongr. = +1.8%, Mincongr. = −0.4%, F(1,9) = 5.77, η2 = 0.03, P = 0.039], without an interaction [F(1,9) = 0.08, η2 < 0.01, P = 0.782]. The neutral condition was not different from the grand mean [F(1,9) = 0.18, η2 = 0.01, P = 0.677]. This suggests that the facilitation observed in experiment 1 was likely driven by an additive combination of congruent motion and congruent object features. We replicated the significant difference between neutral and direction-congruent/feature-congruent conditions—equivalent to motion-absent and motion-present conditions in experiment 1 (β = 0.049, t = 2.23, 95% CI [0.01, 0.09], P = 0.032).
As a second planned analysis, we investigated secondary saccade latencies (Fig. 6D, bottom), which amounted to an overall average of 257.3 ms (SD = 45.9). Secondary saccades to the target were initiated faster than those to the distractor by 9.7 ms (β = 0.17, t = 3.39, 95% CI [0.07, 0.26], P = 0.001). Hierarchical model comparisons, however, revealed no improvement of log-likelihood when introducing the factor motion congruency (five levels) as an additional predictor [BF10< 0.001, ΔLL = +1.9, χ2(4) = 3.55, P = 0.470] nor when allowing an interaction with the saccade destination (distractor versus target) and motion congruency [BF10< 0.001, ΔLL = +1.6, χ2(4) = 3.27, P = 0.513]. Note that this result is consistent with experiment 1, in which no difference between the motion-absent and present condition was evident for surface-feature durations of 50 ms. Yet, after normalizing for each observer’s secondary saccade latency by subtracting the neutral condition, we observed a significant main effect of motion-direction congruency [Mcongr. = −4.1 ms, Mincongr. = 3.3 ms, F(1,9) = 5.76, η2 = 0.04, P = 0.040] but none for surface-feature congruency [Mcongr. = −1.7 ms, Mincongr. = 0.9 ms, F(1,9) = 0.56, η2 < 0.01, P = 0.473]. This suggests that (when collapsing across target- and distractor-bound secondary saccades) secondary saccade latencies were shorter in the direction-congruent than in the direction-incongruent condition. There was no evidence for an interaction between the two factors [F(1,9) = 2.23, η2 = 0.03, P = 0.169], and no interaction between secondary saccade destination (distractor versus target) and motion-direction congruency [F(1,9) = 0.05, η2 < 0.01, P = 0.824] or surface-feature congruency [F(1,9) = 2.09, η2 = 0.02, P = 0.181], respectively.
Together, results of experiment 2 clearly replicated those of experiment 1: Intrasaccadic motion—now induced by the entire stimulus array—drove secondary saccades to either the target or distractor stimulus, depending on whether motion was either congruent or incongruent with the postsaccadic object locations. Again, we also found evidence that secondary saccade latencies decreased with congruent and increased with incongruent motion direction. Although they were only briefly manipulated for 14.6 ms during strictly intrasaccadic motion, the features of moving objects mattered for gaze correction, as the effect of direction-congruency was only significant if intrasaccadic and postsaccadic object features were also congruent.
DISCUSSION
With each saccade we make, visual objects move rapidly across our retinae, transiently producing motion-streak trajectories that lawfully relate to the ongoing movement. In this study, we emulated these trajectories using a projection system capable of displaying continuous object motion (as opposed to apparent motion from a simple displacement) strictly during saccades with high spatiotemporal fidelity. This technique allowed us to investigate the hypothesis that intrasaccadic information about the changing position of saccade targets facilitates postsaccadic gaze correction to these targets. We tightly controlled the postsaccadic availability of surface features that have been shown to play a crucial role in gaze correction tasks (6, 10), by presenting pixel masks at varying delays. In a first experiment, this manipulation permitted the assessment of the impact of intrasaccadic motion on the proportion and latency of secondary saccades to the target, in addition to the time course of the processing of object features. In a second experiment, we further elucidated the respective effects of intrasaccadic motion direction and object features on gaze correction.
Even when little or no postsaccadic object information was available, the presence of intrasaccadic target motion increased the rate of secondary saccades to the original presaccadic target and reduced their initiation latency. These results are central to our hypothesis, as they suggest that intrasaccadic information was not suppressed or otherwise omitted—as widely assumed [for a review, see (20)]—but clearly available for timely gaze correction. The magnitudes of these effects may seem small at first, but they were consistent with what was to be expected from a 14.6-ms intrasaccadic motion duration: Information about postdisplacement object features was accumulated in an exponential fashion right upon motion onset. A comparison of the parameters of these exponential functions suggests that facilitation caused by intrasaccadic motion was not due to an increase of gain or acculumation rate when processing object features and locations but due to the earlier availability of these, starting with the onset of intrasaccadic target motion. Notably, even when continuous object motion was absent, the models predicted that, at saccade offset (on average 10 ms after object motion offset), secondary saccade rates to the target would already be above chance, suggesting that visual processing starts before saccade landing. The idea that visual information picked up during the brief intrasaccadic interval may have implications for postsaccadic coarse-to-fine processing strategies that have been proposed before (41, 42). Consistent with this view, the estimated secondary saccade latency reduction in experiment 1 was found (and of remarkably similar magnitude as the motion duration) not only when surface features were unavailable but also when they were available for the entire 600 ms. Note that, in this task, the motion duration was barely a third of the mean saccade duration and that, in natural vision, any visual object could produce motion streaks across the entire duration of the saccade, possibly supporting short-latency corrective saccades upon saccade landing.
We considered alternative explanations of the facilitation of gaze correction observed in experiment 1. In this version of the paradigm, all irrelevant objects were removed during the 14.6-ms target motion interval. Although this was intended to correctly perform reverse regression analyses on the target object’s features and retinal trajectory, it could be that the salient, consistently valid motion cue caused attentional facilitation and directed the secondary saccade to the correct postsaccadic object location. In experiment 2, however, we clearly replicated the initial effect with equal or even larger magnitude, although the strictly intrasaccadic rotation of the entire stimulus array caused a highly complex pattern of retinal trajectories. These retinal trajectories provided various, equally salient directional cues that would not benefit localization of the postsaccadic target. Nevertheless, the facilitation of secondary saccades to the target persisted. The fact that we ruled out a simple attentional cueing effect does not preclude the involvement of attention in gaze correction based on intrasaccadic motion streaks. Attentional allocation to objects could be guided or facilitated by spatiotemporal continuity (43, 44), which would be well in line with our findings. That is, the spatiotemporal continuity of objects on the retina throughout the saccade duration may guide attention to their postsaccadic location. If this were true, then other types of manipulations (e.g., flashed exogenous cues at the future target location) might similarly facilitate gaze correction, which could be investigated in future studies. The results of experiment 2 show that a transient, intrasaccadic change in the feature that distinguishes the target from a neighboring distractor (here, luminance polarity) did not completely eliminate the impact of a motion streak on gaze correction. Clearly, however, motion had its largest effect if the target’s feature remained unchanged throughout the saccade (experiment 1 and direction-congruent/feature-congruent condition in experiment 2). Postsaccadic blanking intervals and changes in contrast polarity have been shown to impair object recognition (45), an effect attributed to breaking object continuity.
Furthermore, we not only showed that the effect of intrasaccadic object motion is orthogonal to the effect of primary saccade landing positions [cf. (6)] but also provided evidence for the benefit of effective temporal integration when stimulus orientations were aligned with their retinal motion trajectories—a typical signature of motion streaks (29). In other words, the more effectively the combined movement of eye and target in a given trial generated a motion streak, the more often did a secondary saccade go to the target. We found the same effect for the latency of these saccades, providing evidence that efficient motion streaks increased not only the accuracy but also the speed of gaze correction. Although it has been shown that motion perception during saccades is well possible (31, 35), contrast sensitivity to gratings orthogonally oriented to their motion trajectories is drastically reduced at saccadic velocities (34, 46). In contrast, motion streaks often remain well resolved even at saccadic speeds (12–14, 17), because motion-streak detectors could code fast motion (for which visual persistence plays a larger role) as orientations parallel to motion direction (15, 16). Thus, motion streaks are a prime candidate to link objects across saccades via spatiotemporal continuity. Our results show that, even when objects were displaced while the eyes were in mid-flight, a continuous presence of the target throughout the saccade—as opposed to a very brief disruption of this continuity—facilitated gaze correction, regardless of how long feature information was available after the displacement. The magnitude of this facilitation was largest shortly after motion offset, and it appeared to decrease with increasing surface-feature durations, possibly because of near-ceiling gaze-correction accuracy for long surface-feature durations in some observers [cf. (6)]. To provide a strong case for the absence of an interaction between surface features and intrasaccadic motion, future studies could threshold gaze-correction performance of individual observers to a medium range, for instance, by adjusting stimulus contrast. Interactions between postmotion surface-feature durations and presence of intrasaccadic motion were rarely significant, and statistical modeling favored additive effects of surface features and intrasaccadic motion. These consistent results suggest that postsaccadic object features and spatiotemporal continuity—established by intrasaccadic continuous object motion—contributed rather independently to gaze-correction performance. This conclusion is well in line not only with the predictions of the object-file theory (7, 8), which suggests that objects are bound to spatial indexes, but also with the view that surface features are functional for the establishment of object correspondence (6, 9, 10): Intrasaccadic motion streaks may not only be indicators of amplitude and direction of continuous shifts of objects across saccades but, to some extent, also maintain the object’s surface features, such as color, which has been shown to be largely unaltered by saccadic suppression (47–49), throughout the saccade. Although the high refresh rate of the projection system precluded the use of color, direct evidence in favor of this view was provided by the finding of experiment 2 that inverting the feature polarity of objects during their intrasaccadic 14.6-ms motion interval significantly decreased the likelihood of making secondary saccades to the target. More specifically, our results suggested two additive congruency effects—congruency of motion direction and of surface features—and that significant increases or decreases of secondary saccades rates to the target were only found if both motion direction and surface features were either congruent or incongruent to the postsaccadic object configuration. Given the fact that, in natural conditions, intrasaccadic motion streaks should carry the inducing objects’ surface features across saccades, these results might suggest that the facilitation observed in both experiments depended on object features, such as luminance or color, as well as SF and orientation content.
To conclude, our results support the idea that saccades do not cause gaps in visual processing, as even motion streaks induced by high-velocity, brief, unpredictable, and strictly intrasaccadic object motion were taken into account when performing gaze correction. Of course, it has not yet been investigated whether intrasaccadic object motion in natural visual scenes can produce similar benefits. On the basis of the following considerations, however, we consider it likely. Clutter in natural scenes is known to influence search efficiency [e.g., (50)], and it may well impair the matching of surface features for gaze correction. At the same time, the presence of multiple irrelevant objects upon saccade landing reduces the likelihood of detecting intrasaccadic motion streaks (29, 30), even when these distractors were presented at large eccentricities (14), a condition similar to the experiments reported here. It has also been shown that sensitivity to intrasaccadic stimulation is reduced if an object is presented against or succeeded by a structured background (51–54), but these studies almost exclusively used near-threshold stimuli. Stimuli with contrasts well above threshold, like the ones used in this study, are common in natural scenes (55, 56), suggesting that motion streaks induced by salient objects remain resolved at saccadic speeds, even in complex spatial configurations [e.g., (57)], and are a routine part of the natural visual input. These motion streaks are most likely harder to detect consciously—presumably because of processes related to masking (18–21)—but could still undergo visual processing in the absence of conscious awareness. Experimental procedures like the ones presented here are necessary to investigate whether they are used by visuomotor processes despite the observers’ inability to report them. While we did not systematically assess conscious detection of intrasaccadic signals in this study [see, e.g., (29)], most of our observers volunteered during their debriefing that they remained unaware of the intrasaccadic manipulations that we imposed. This is especially notable in the face of the extensive retinal translations induced by the rotation of the entire stimulus array in experiment 2. Thus, even while intrasaccadic motion is omitted from conscious awareness (as it usually occurs during natural scene viewing), our results suggest that they can still be used for gaze correction. Future investigations should follow up on this intriguing hypothesis, investigating the impact of motion streaks in natural scenes. Depending on the efficiency of intrasaccadic vision in real-world visual environments, the visual consequences induced by our very own saccades may constitute an unexpected contribution to achieving object continuity and, through it, visual stability.
MATERIALS AND METHODS
Apparatus
Stimuli were projected onto a 16:9 (250.2 × 141.0 cm) video-projection screen (Stewart Silver 5D Deluxe, Stewart Filmscreen, Torrance, CA), mounted on a wall 340 cm in front of the participant, using a PROPixx DLP projector (Vpixx Technologies, Saint-Bruno, QC, Canada) running at a 1440-Hz refresh rate and a resolution of 960 × 540 pixels. The experimental code was implemented in MATLAB 2016b (Mathworks, Natick, MA, USA) on Ubuntu 18.04, using Psychtoolbox (58, 59) and was run on a Dell Precision T7810 Workstation supplied with a Nvidia GTX 1070 graphics card. Eye movements of both eyes were recorded via a TRACKPixx3 tabletop system (Vpixx Technologies, Saint-Bruno, QC, Canada) at a sampling rate of 2000 Hz, running firmware version 11 in experiment 1 and version 16 in experiment 2. Participants rested their head on a chin rest. A custom wrapper function toolbox was used to control the eye tracker, which is made publicly available on Github: https://github.com/richardschweitzer/TrackPixxToolbox.
Participants
In each experiment, 10 observers gave written informed consent before inclusion in the study. Both studies were conducted in agreement with the latest version of the Declaration of Helsinki (2013), approved by the Ethics board of the Department of Psychology at Humboldt-Universität zu Berlin, and preregistered at the OSF.
Experiment 1
Ten observers completed three sessions each (duration of approximately 1 hour, each on separate days) and received 26 euros as remuneration (plus 2 euros for every 15 min of overtime). All observers (five male, five female; mean age, 28; age range, 20 to 37) had normal or corrected-to-normal vision (20/20 ft. acuity in the Snellen test; four observers wore glasses and one observer wore contact lenses) and were right-handed. Seven of the 10 observers had right ocular dominance (established by a variant of the Porta test). In accordance with preregistered exclusion criteria, four invited participants had to be replaced because they did not complete all three sessions. Preregistration, data, and analysis scripts can be found at https://osf.io/aqkzh/.
Experiment 2
Ten observers completed two sessions each (duration of approximately 45 min) and received 13 euros plus 2 euros for every 15 min of overtime. All 10 observers (6 male, 4 female; mean age, 30; age range, 22 to 43) had normal or corrected-to-normal vision (1 observer wore glasses and 1 observer wore contact lenses) and were right-handed. Five of the 10 observers had right ocular dominance. Preregistration, data, and analysis scripts are made available at https://osf.io/uqdkf/.
Procedure and task
Experiment 1
A six-stimulus circular array at an eccentricity of 10 dva was displayed while observers fixated an area with a 1.5-dva radius around a central fixation dot for 400 ms. The stimulus array contained two types of dissimilar noise patches (see the “Stimuli” section), in alternating order (Fig. 2A, top row). Specific stimulus positions were at 0°, 60°, 120°, 180°, 240°, and 300° (as shown in Fig. 2A) or alternatively at 30°, 90°, 150°, 210°, 270°, and 330° relative to the central fixation dot (0°: below the fixation point). After successful fixation, an exogenous cue was presented to indicate the saccade target: The target stimulus—one of the six presented stimuli and one of the two types of noise patches—was enlarged linearly up to twice its initial size for 25 ms and then decreased for 25 ms until the initial size was restored. Saccades were detected online using the algorithm described in (38) with parameters k = 2, λ = 10, and ϑ = 40, on both eyes. In one-third of all trials, the target remained in its presaccadic location. In the other two-thirds of all trials, as soon as the saccade was detected, the cued stimulus moved 30° in a clockwise (CW) or counterclockwise (CCW) direction for 14.6 ms—amounting to a distance traveled of 5.2 dva at a velocity of approximately 360 dva/s. This 14.6-ms motion was either continuous (motion-present condition), i.e., presenting 21 frames of equally spaced stimulus positions along the circular trajectory (0.25 dva per frame), or apparent (motion-absent condition), i.e., presenting a blank screen between the first and final positions of the stimulus. In both motion conditions, all other noise patches were removed during this short and rapid stimulus motion. As soon as the moving stimulus reached its final position, all stimuli were displayed at their postmotion locations consistent with a 30° CW or CCW rotation of the stimulus array. Observers’ saccades thus landed between two dissimilar noise patches: One was always the target stimulus that had been cued before saccade initiation, and the other one was an uncued and therefore irrelevant distractor. As a consequence, a secondary saccade was made to the target (or erroneously to the distractor) to correct for the intrasaccadic displacement. Crucially, a pixel noise mask (Fig. 2A, bottom row) occluded the identity of all stimuli presented on the screen with varying delay relative to stimulus motion offset (0, 25, 50, 100, 200, and 600 ms), thus limiting observers’ time to use stimulus surface features to guide their secondary saccades. This postmotion mask onset delay will henceforth be referred to as of surface-feature duration. Each trial ended 650 ms after stimulus motion offset.
Similar to (6), observers were instructed to make a saccade to the target stimulus upon cue presentation and fixate it. They were informed that the stimulus array could rotate in some trials, in which case they could make a secondary saccade to follow the initial target. If observers’ initial central fixation was unsuccessful, or if their primary saccade did not end within a circular region of 2 dva around the presaccadic target location, or if more than one saccade was made to reach the presaccadic target location, then appropriate feedback was provided verbally on the screen, and the trial was repeated at the end of the session. No feedback related to observers’ secondary saccades was given. To elucidate the trial procedure, a 60-frames/s video (slowed down by a factor of 24 and using the mouse cursor as a representative of gaze position) can be found at OSF: https://osf.io/f48rm/.
Experiment 2
With respect to task, procedure, and instructions, experiment 2 was identical to experiment 1, except for three relevant changes. First, whereas in experiment 1, all other objects were removed during 14.6-ms target motion interval, the entire stimulus array was rotated in experiment 2 (Fig. 6A), thus causing all six objects—not only the target—to move continuously. Second, to limit the number of overall trials, experiment 2 included only one surface feature. We chose to use the 50-ms condition as it exhibited intermediate gaze correction performance in experiment 1, thus avoiding floor or ceiling effects in our main dependent variable. Third, and most importantly, while in experiment 1 continuous intrasaccadic motion was either absent or present (in which case only the direction and feature of the target object was used), experiment 2 now featured four conditions of motion streak congruency in addition to a neutral condition, which was identical to the motion-absent condition in experiment 1 (Fig. 6C). Specifically, we manipulated the congruency of intrasaccadic continuous motion (of the entire array) with the trans-saccadic rotation of the stimulus array in terms of motion direction and object features. In trials with congruent intrasaccadic motion direction, all objects moved toward their correct postsaccadic locations, whereas they moved in the opposite direction in direction-incongruent trials. Whereas in experiment 1 the direction of intrasaccadic motion was always compatible with the rotation of the array, intrasaccadic motion could now also provide incompatible cues in experiment 2. In trials with congruent surface features, object features during the continuous motion period remained unchanged, whereas they reversed their respective feature polarity relative to their pre- and postsaccadic presentations in feature-incongruent trials. In other words, the feature-incongruent condition caused all objects to briefly—and strictly during the induced intrasaccadic motion—change their identity, as defined by their surface features. The combination of the two orthogonal factors of intrasaccadic motion direction and surface features resulted in the four conditions illustrated in Fig. 6C.
Stimuli
Stimuli in both experiments 1 and 2 were achromatic, random Gaussian noise patches (SD = 1) band-pass–filtered to SFs from 0.25 to 1 cpd and displayed on a uniform, gray background. One initial band-pass–filtered noise matrix was generated on each trial. To maximize the dissimilarity between the two types of noise patches, 75% of a noise SD was added to or subtracted from the initial noise matrix, thus increasing or decreasing its luminance (for one example, see Fig. 2A). This procedure inevitably led to some differences in SF and orientation for the two types of content in the pairs of noise patches. This effect was intended, as it allowed for both easier discrimination of the two types during trials and reverse regression analyses involving stimulus features at a later stage (29, 60). All noise patches were at full Michelson contrast to maximize their intrasaccadic visibility. Noise patches were enveloped in a Gaussian aperture with an SD of 0.5 dva. Masks displayed at postmotion locations had the same dimensions but consisted of random black-white pixel noise. Noise masks were identical copies for all six stimuli of the array.
The central fixation dot at the beginning of each trial consisted of a white circle of 0.3-dva radius. To indicate that the dot was fixated by the observer, the area within the circle was be filled by another white circle of 0.1-dva radius.
Preprocessing
In experiment 1, observers completed at least 3456 trials, i.e., at least 1152 trials per session. This number of trials resulted from the fully counterbalanced experimental factors: cued location (six levels: one to six stimuli), initial position of the stimulus array (two levels: 0° and 30°), motion direction (three levels: CW, CCW, and static), presence of continuous intrasaccadic motion (two levels: absent and present), and delay between the displacement/continuous motion and the masks, i.e., the surface-feature duration (six levels: 0, 25, 50, 100, 200, and 600 ms), thus resulting in a total of eight trials per experimental cell. Trials were repeated at the end of each session if fixation control was not passed, primary saccades did not reach the presaccadic target position, or multiple saccades were made to reach it. On average, observers completed 3705 (SD = 209) trials across all experimental sessions (including repeated and later excluded trials).
In experiment 2, 1620 trials were completed, thus at least 810 trials per session. The same counterbalanced experimental factors were used as above, with the exception that the congruency of intrasaccadic motion was systematically varied (five levels: incongruent direction with incongruent feature, incongruent direction with congruent feature, congruent direction with incongruent feature, congruent direction with congruent feature, and neutral or motion absent; see also Fig. 6C) and only one surface-feature duration was used (50 ms), resulting in nine trials per experimental cell. Applying the same criteria for trial repetition during experiment 2, observers completed on average 1877 (SD = 254) trials across both sessions.
Preprocessing involved three major steps. First, 0.5% (SD = 0.4%) of trials in experiment 1 and 0.3% (SD = 0.2%) of trials in experiment 2 were excluded because of unsuccessful fixations (within a central circular boundary of 1.5-dva radius) and dropped frames.
Second, saccades (i.e., primary, secondary, and tertiary saccades in each trial) were detected using the Engbert-Kliegl algorithm (61, 62) with a velocity factor of 10 and a minimal duration of 15 ms. Before saccade detection, eye movement data were downsampled to 1000 Hz using band-limited interpolation. Each trial’s data were padded with its first and last samples and shifted before downsampling to compensate for the edge effects and delays introduced by low-pass filtering. Sections of missing data due to blinks or tracking problems were expanded by 40 samples on each side and linearly interpolated, but only if those samples were not collected during the relevant trial interval, i.e., from the onset of the saccade cue until 450 ms after the offset of the stimulus motion. Saccade detection was performed on both eyes, but only data collected from the observer’s dominant eye were analyzed, unless the latter was not available because of missing samples, which occurred in 2.4% (SD = 2.1%) of all trials in experiment 1 and 2.3% (SD = 3.2%) of all trials in experiment 2. To achieve a conservative criterion for saccade offset (to remove trials in which stimulus motion was not strictly intrasaccadic), we did not consider above-threshold postsaccadic oscillations (if detected within a window of 50 ms after the first below-threshold sample) to be part of the primary saccade.
Third, on average, 10.1% (SD = 7.4) of the remaining trials in experiment 1 and 5.0% (SD = 3.8) of the remaining trials in experiment 2 were excluded because they failed to satisfy the following criteria: (i) no missing data within the relevant trial interval (see above), (ii) detection of one single primary saccade that reached the 2-dva area around the presaccadic target location (see the “Procedure and task” section), (iii) primary saccade metrics compatible with the instructed 10-dva saccade (i.e., amplitude 6 to 15 dva, peak velocity below 600 dva/s, duration below 75 ms), and (iv) strictly intrasaccadic stimulus motion (i.e., motion onset after saccade onset and motion offset before saccade offset, regardless of whether continuous motion was present or not), taking into account a deterministic 8.3-ms video delay of the PROPixx projection system [see (38)].
Ultimately, an average of 3397 (SD = 310) trials per observer in experiment 1 and 1698 (SD = 220) trials per observer in experiment 2 entered further analyses. Across observers, stimulus motion was physically displayed 17.8 ms (SD = 0.5) after saccade onset (19.2 ms in experiment 2, SD = 0.5) and ended 10.7 ms (SD = 3.1) before saccade offset (11.5 ms in experiment 2, SD = 3.0), as computed on the basis of saccades detected offline and including all system latencies; see also Fig. 2B. Mean primary saccade amplitudes amounted to 9.1 dva (SD = 0.3) and 9.2 dva (SD = 0.4), mean primary saccade durations to 43.8 ms (SD = 3.0) and 46.5 ms (SD = 3.3), and mean primary saccade peak velocities to 327.1 dva/s (SD = 34.1) and 328.1 dva/s (SD = 24.7) in experiments 1 and 2, respectively. While intrasaccadic motion was presented, the eyes had a mean average velocity of 279.8 dva/s (SD = 25.5) and 258.1 dva/s (SD = 38.8) and a mean maximum velocity of 317.3 dva/s (SD = 34.9) and 308.7 dva/s (SD = 34.1) in experiments 1 and 2, respectively. The average retinal velocities of the target object (i.e., the combination of stimulus- and saccade-induced motion on the retina) thus amounted to 498.3 dva/s (SD = 15.2) in experiment 1 and 469.5 dva/s (SD = 23.7) in experiment 2.
Analysis
Secondary saccades
In experiment 1, secondary saccades were made in 88.2% (SD = 14.1, Mdn = 95.1) of CCW trials, in 88.6% (SD = 14.2, Mdn = 94.8) of CW trials, and in 32.0% (SD = 26.9, Mdn = 26.8) of static trials. In experiment 2, these numbers were similar, as secondary saccades were made in 91.0% (SD = 11.3, Mdn = 94.6), 91.4% (SD = 9.4, Mdn = 94.9), and 27.4% (SD = 16.8, Mdn = 25.6) of trials in CCW, CW, and static conditions, respectively. In experiment 1, mean secondary saccade rates were slightly reduced by two observers who rarely made secondary saccades despite intrasaccadic displacements, i.e., in 54.5 and 73.3% of trials. Note that overall secondary saccade probability was largely constant across surface-feature duration (0 ms: M = 86.7%, SD = 13.0; 25 ms: M = 88.3%, SD = 13.6; 50 ms: M = 89.0%, SD = 14; 100 ms: M = 88.8%, SD = 14.8; 200 ms: M = 88.24%, SD = 14.9; 600 ms: M = 89.0%, SD = 14.3) and motion conditions (absent: M = 88.4%, SD = 14.3; present: M = 88.4%, SD = 13.8). This was also the case in experiment 2 (direction-incongr./feature-incongr.: M = 91.2%, SD = 9.7; direction-incongr./feature-congr.: M = 92.0%, SD = 9.8; neutral: M = 90.6%, SD = 11.0; direction-congr./feature-incongr.: M = 91.0%, SD = 10.8; direction-congr./feature-congr.: M = 91.2%, SD = 10.3). To determine whether secondary saccades were made to target or distractor stimuli, we determined whether the offset of the secondary saccade landed within a 3-dva window around the center of either stimulus. In CCW and CW trials, 94.5% (SD = 5.3) and 94.7% (SD = 5.1) of secondary saccades landed within these regions in experiment 1, while 92.7% (SD = 9.8) and 92.5% (SD = 9.5) did in experiment 2. In static trials, which did not enter further analyses, 55.6% (SD = 30.4) and 57.7% (SD = 20.9) of secondary saccades in experiments 1 and 2, respectively, were refixations in the region around the target stimulus. In experiment 1, tertiary saccades, i.e., saccades following secondary saccades, were made in only 8.3% (SD = 4.4) of CCW trials, 8.3% (SD = 4.5) of CW trials, and 1.8% (SD = 2.4) of static trials. In experiment 2, tertiary saccades occurred in 10.8% (SD = 6.8) of CCW trials, 12.1% (SD = 7.8) of CW trials, and 2.4% (SD = 3.0) of static trials. Because of the low number of trials, these data were not further analyzed.
We applied repeated-measures ANOVAs to proportions of secondary saccades to the presaccadic target and inverse-transformed secondary saccade latencies (defined as the time passed between the offset of the primary saccade and the onset of the secondary saccade) to assess main effects and interactions between experimental factors intrasaccadic motion (experiment 1: absent, present; experiment 2: direction-incongr./feature-incongr., direction-incongr./feature-congr., neutral, direction-congr./feature-incongr., direction-congr./feature-congr.) and surface-feature duration (experiment 1: 0, 25, 50, 100, 200, and 600 ms; experiment 2: 50 ms only, thus not included). In the case of multiple factor levels, a PGG value is reported, adjusted using Greenhouse-Geisser sphericity correction. Furthermore, we used logistic and linear mixed-effects regression analyses (63) for proportion and inverse-transformed latency data, respectively, specifying observers as intercept-only random effects. Experimental conditions were effects-coded ([−0.5, 0.5] for two-level factors and [−1, 1] multiple-level factors, respectively), so that the grand mean constituted the intercept of each model. Inverse transformations for latency data were applied to deal with the inherent skewness of secondary saccade latency distributions (e.g., Fig. 4C), which can be crucial, especially when performing analyses on the single-trial level (64, 65). CIs for slopes were determined via parametric bootstrapping with 2000 repetitions each. Along with CIs, P values were computed via Satterthwaite’s degrees of freedom method. To test the relevance of experimental manipulations, hierarchical model comparisons were performed using the likelihood ratio test, and Bayes factors were computed from two models’ respective Bayesian information criteria (66). While we consistently report the results of model comparisons (pointing out the best model), all reported estimates stem from the full model, not the best model.
To describe the time course of secondary saccade rate to the target, we fitted an exponential growth model with the formula p(t) = 0.5 + λ(1 − e−β(t − δ)). This model, previously used to describe speed-accuracy tradeoffs [e.g., (67)], was now used to approximate the proportion of secondary saccades to the target p(t)—increasing from a chance level of 0.5—at any given surface-feature duration t. We estimated the three parameters of the model (illustrated in insets in Fig. 2C)—asymptote (λ), slope (β), and onset (δ)—in a mixed-effects approach using the stochastic approximation expectation maximization algorithm (starting parameters: λ = 1, β = 1, δ = 4), implemented in the saemix R package (32). This approach allowed each of the parameters to be estimated independently for each observer, separately for absent and present intrasaccadic object motion. Subsequently, paired t tests were used to test whether estimated parameters differed between motion conditions. As we conducted independent hypothesis tests on three parameters, significance levels were Bonferroni-corrected, resulting in α = 0.016. All analyses were implemented in R (68) and can be found in a markdown document on OSF: https://osf.io/uafsk/ (experiment 1), https://osf.io/6dtfm/ (Exp. 2). Furthermore, to describe the time course of secondary saccade latencies, mixed-effect GAMs were fitted using the mgcv package in R (69). These models, fitted separately for secondary saccades to the target and to the distractor, allowed to capture the nonlinear dynamics of secondary saccades latencies over surface-feature durations, both for experimental conditions of intrasaccadic motion (treatment-coded as ordered factor; reference smooth: absent, difference smooth: present) and for each observer. Thin-plate regression splines (70) were used as smooth functions. Figure 3 shows the model predictions averaged across observers.
Reverse regression
As a first step, target noise patches were convolved with Gabor filters (in sine and cosine phases) of varying orientations (from to , in steps of rad) and SFs (0.25, 0.29, 0.34, 0.39, 0.46, 0.54, 0.63, 0.73, 0.86, and 1 cpd), resulting in one energy map per noise patch, that contained the filter responses for each orientation-SF component [an example is shown in Fig. 5A (see also (29, 60)]. Second, we estimated the angle of the stimulus’s motion trajectory on the retina (for an illustration, see Fig. 5B). To compute this retinal trajectory, we subtracted the gaze positions during stimulus presentation (spline-interpolated to match the projector refresh rate of 1440 Hz) from the stimulus locations over time. From the retinal positions, retinal angles were computed, whose median was subsequently used to normalize each stimulus’s orientation components for its respective retinal trajectory. Relative orientation is the angular difference between the retinal angle and the orientations contained in a given noise patch (29). To achieve the equal-sized steps of relative orientations (in the face of retinal angles that naturally varied between trials), the filter responses for the defined orientation and SF levels were interpolated on the basis of a full tensor product smooth (using cubic splines) of each stimulus’s energy map. Relative orientation could take any value between 0 (orientation parallel to motion direction) and (orientation orthogonal to motion direction).
Last, we fitted mixed-effects logistic and linear regressions [random intercepts and slopes for observers; (63)] to predict secondary saccades to the target stimulus and inverse latencies of target-bound saccades, respectively, from standardized filter responses in each combination of relative orientation and SF of the target’s energy map. This was done separately for both motion-present and motion-absent conditions (Fig. 5, C and D), as well as for surface-feature durations. A significant positive slope for filter responses in a particular relative orientation-SF component indicated that this component drove secondary saccades to the target stimulus or secondary saccades with reduced latency, respectively. Instead of reporting the weights of the model, we reported the corresponding z- and t-statistics, i.e., the ratio of the estimated weights and their SEs, as they allowed for a more straightforward and comparable evaluation of significance. We further analyzed these outcomes with GAMs. Smooth terms for relative orientation (continuous: 0 .. ) and SF (continuous, log10-transformed: −0.6 .. 0), as well as their interactions, were again based on thin-plate regression splines (70) and could include by-variables coding the experimental condition of intrasaccadic motion (treatment-coded as ordered factor; reference smooth: absent, difference smooth: present). Surface-feature duration (reference smooth: 0 ms, difference smooths: 25, 50, 100, 200, 600 ms) was also added in a full model, which can be found in the Open Methods (OSF link: https://osf.io/uafsk/). Results shown in Fig. 5 (C and D) are averages across all surface-feature durations, thus equally taking into account the effect of object features in both movement-present and movement-absent conditions. For each coefficient of the GAM, a complexity of the smooth term (i.e., edf) and the significance of the term were estimated. As these estimates cannot be interpreted directly, we complemented the GAM with a simple multiple regression (LM) with the same variable coding to report the linear trends within the data.
Acknowledgments
We thank E. M. Rehse for indispensable work in data collection for experiment 1; L. Kröll, G. Häberle, F. Geweke, and B. Yahn for support in the initial stages of the project; and P. Cavanagh and members of the Active Perception and Cognition laboratory for helpful discussions and feedback. Funding: R.S. was supported by the Studienstiftung des deutschen Volkes and the Berlin School of Mind and Brain. M.R. was supported by the Deutsche Forschungsgemeinschaft (DFG; grants RO3579/8-1 and RO3579/10-1), as well as the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement no. 865715). Furthermore, R.S. and M.R. were supported by Germany’s Excellence Strategy–EXC 2002/1 “Science of Intelligence”–project no. 390523135. Author contributions: R.S. and M.R. conceived, designed, and preregistered the study. R.S. implemented and conducted experiments. R.S. analyzed data under M.R.’s supervision. R.S. drafted the manuscript, and M.R. provided critical revisions. Competing interests: The authors declare that they have no competing interests. Data and materials availability: All data needed to evaluate the conclusions in the paper are present in the paper and are publicly available at OSF: https://osf.io/aqkzh/ (experiment 1) and https://osf.io/uqdkf/ (experiment 2). Additional data related to this paper may be requested from the authors.
REFERENCES AND NOTES
- 1.Bridgeman B., Van der Heijden A. H. C., Velichkovsky B. M., A theory of visual stability across saccadic eye movements. Behav. Brain Sci. 17, 247–258 (1994). [Google Scholar]
- 2.D. M. MacKay, Visual stability and voluntary eye movements, in Central Processing of Visual Information A: Integrative Functions and Comparative Data, H. Autrum, R. Jung, W. R. Loewenstein, D. M. MacKay, H. L. Teuber, Eds. (Springer, 1973), pp. 307–331. [Google Scholar]
- 3.McConkie G. W., Currie C. B., Visual stability across saccades while viewing complex pictures. J. Exp. Psychol. Hum. Percept. Perform. 22, 563–581 (1996). [DOI] [PubMed] [Google Scholar]
- 4.Wurtz R. H., Neuronal mechanisms of visual stability. Vision Res. 48, 2070–2089 (2008). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.D. Aagten-Murphy, P. M. Bays, Functions of memory across saccadic eye movements, in Processes of Visuospatial Attention and Working Memory, T. Hodgson, Ed. (Springer International Publishing, 2019), pp. 155–183. [DOI] [PubMed] [Google Scholar]
- 6.Hollingworth A., Richard A. M., Luck S. J., Understanding the function of visual short-term memory: Transsaccadic memory, object correspondence, and gaze correction. J. Exp. Psychol. Gen. 137, 163–181 (2008). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Kahneman D., Treisman A., Gibbs B., The reviewing of object files: Object-specific integration of information. Cogn. Psychol. 265, 175–219 (1992). [DOI] [PubMed] [Google Scholar]
- 8.Mitroff S. R., Alvarez G. A., Space and time, not surface features, guide object persistence. Psychon. Bull. Rev. 14, 1199–1204 (2007). [DOI] [PubMed] [Google Scholar]
- 9.Hollingworth A., Franconeri S. L., Object correspondence across brief occlusion is established on the basis of both spatiotemporal and surface feature cues. Cognition 113, 150–166 (2009). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Richard A. M., Luck S. J., Hollingworth A., Establishing object correspondence across eye movements: Flexible use of spatiotemporal and surface feature information. Cognition 109, 66–88 (2008). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Becker W., The neurobiology of saccadic eye movements. metrics. Rev. Oculomot. Res. 3, 13–67 (1989). [PubMed] [Google Scholar]
- 12.Bedell H. E., Yang J., The attenuation of perceived image smear during saccades. Vision Res. 41, 521–528 (2001). [DOI] [PubMed] [Google Scholar]
- 13.Brooks B. A., Yates J., Coleman R., Perception of images moving at saccadic velocities during saccades and during fixation. Exp. Brain Res. 40, 71–78 (1980). [DOI] [PubMed] [Google Scholar]
- 14.Duyck M., Collins T., Wexler M., Masking the saccadic smear. J. Vis. 16, 1 (2016). [DOI] [PubMed] [Google Scholar]
- 15.Geisler W. S., Motion streaks provide a spatial code for motion direction. Nature 400, 65–69 (1999). [DOI] [PubMed] [Google Scholar]
- 16.Geisler W. S., Albrecht D. G., Crane A. M., Stern L., Motion direction signals in the primary visual cortex of cat and monkey. Vis. Neurosci. 18, 501–516 (2001). [DOI] [PubMed] [Google Scholar]
- 17.Matin E., Clymer A. B., Matin L., Metacontrast and saccadic suppression. Science 178, 179–182 (1972). [DOI] [PubMed] [Google Scholar]
- 18.Campbell F. W., Wurtz R. H., Saccadic omission: Why we do not see a grey-out during a saccadic eye movement. Vision Res. 18, 1297–1303 (1978). [DOI] [PubMed] [Google Scholar]
- 19.Breitmeyer B. G., Ganz L., Implications of sustained and transient channels for theories of visual pattern masking, saccadic suppression, and information processing. Psychol. Rev. 83, 1–36 (1976). [PubMed] [Google Scholar]
- 20.E. Castet, Perception of intra-saccadic motion, in Dynamics of Visual Motion Processing, G. S. Masson, U. J. Ilg, Eds. (Springer, 2010), chap. 10, pp. 213–238. [Google Scholar]
- 21.Matin E., Saccadic suppression: A review and an analysis. Psychol. Bull. 81, 899–917 (1974). [DOI] [PubMed] [Google Scholar]
- 22.Volkmann F. C., Human visual suppression. Vision Res. 26, 1401–1416 (1986). [DOI] [PubMed] [Google Scholar]
- 23.Ibbotson M. R., Crowder N. A., Cloherty S. L., Price N. S. C., Mustari M. J., Saccadic modulation of neural responses: Possible roles in saccadic suppression, enhancement, and time compression. J. Neurosci. 28, 10952–10960 (2008). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Ross J., Morrone M. C., Goldberg M. E., Burr D. C., Changes in visual perception at the time of saccades. Trends Neurosci. 24, 113–121 (2001). [DOI] [PubMed] [Google Scholar]
- 25.Sylvester R., Haynes J. D., Rees G., Saccades differentially modulate human lgn and v1 responses in the presence and absence of visual stimulation. Curr. Biol. 15, 37–41 (2005). [DOI] [PubMed] [Google Scholar]
- 26.Thiele A., Henning P., Kubischik M., Hoffmann K.-P., Neural mechanisms of saccadic suppression. Science 295, 2460–2462 (2002). [DOI] [PubMed] [Google Scholar]
- 27.Thilo K. V., Santoro L., Walsh V., Blakemore C., The site of saccadic suppression. Nat. Neurosci. 7, 13–14 (2004). [DOI] [PubMed] [Google Scholar]
- 28.Idrees S., Baumann M. P., Franke F., Münch T. A., Hafed Z. M., Perceptual saccadic suppression starts in the retina. Nat. Commun. 11, 1977 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Schweitzer R., Rolfs M., Intrasaccadic motion streaks as cues to linking object locations across saccades. J. Vis. 20, 17 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Balsdon T., Schweitzer R., Watson T. L., Rolfs M., All is not lost: Post-saccadic contributions to the perceptual omission of intrasaccadic streaks. Conscious. Cogn. 64, 19–31 (2018). [DOI] [PubMed] [Google Scholar]
- 31.Castet E., Jeanjean S., Masson G., Motion perception of saccade-induced retinal translation. Proc. Natl. Acad. Sci. 99, 15159–15163 (2002). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Comets E., Lavenu A., Lavielle M., Parameter estimation in nonlinear mixed effect models using saemix, an R implementation of the SAEM algorithm. J. Stat. Softw. 80, i03 (2017). [Google Scholar]
- 33.Reingold E. M., Stampe D. M., Saccadic inhibition in voluntary and reflexive saccades. J. Cogn. Neurosci. 14, 371–388 (2002). [DOI] [PubMed] [Google Scholar]
- 34.Burr D. C., Ross J., Contrast sensitivity at high velocities. Vision Res. 22, 479–484 (1982). [DOI] [PubMed] [Google Scholar]
- 35.Castet E., Masson G. S., Motion perception during saccadic eye movements. Nat. Neurosci. 2, 177–183 (2000). [DOI] [PubMed] [Google Scholar]
- 36.García-Pérez M. A., Peli E., Intrasaccadic perception. J. Neurosci. 21, 7313–7322 (2001). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Kelly D. H., Motion and vision. ii. stabilized spatio-temporal threshold surface. J. Opt. Soc. Am. 69, 1340–1349 (1979). [DOI] [PubMed] [Google Scholar]
- 38.Schweitzer R., Rolfs M., An adaptive algorithm for fast and reliable online saccade detection. Behav. Res. Methods 52, 1122–1139 (2020). [DOI] [PubMed] [Google Scholar]
- 39.Apthorp D., Schwarzkopf D. S., Kaul C., Bahrami B., Alais D., Rees G., Direct evidence for encoding of motion streaks in human visual cortex. Proc. Biol. Sci. 280, 20122339 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Krekelberg B., Vatakis A., Kourtzi Z., Implied motion from form in the human visual cortex. J. Neurophysiol. 94, 4373–4386 (2005). [DOI] [PubMed] [Google Scholar]
- 41.Boi M., Poletti M., Victor J. D., Rucci M., Consequences of the oculomotor cycle for the dynamics of perception. Curr. Biol. 27, 1268–1277 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Rucci M., Ahissar E., Burr D., Temporal coding of visual space. Trends Cogn. Sci. 22, 883–895 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.J. I. Flombaum, B. J. Scholl, L. R. Santos, Spatiotemporal priority as a fundamental principle of object persistence, in The Origins of Object Knowledge, B. M. Hood, L. R. Santos, Ed. (Oxford Scholarship Online, 2009), pp. 135–164. [Google Scholar]
- 44.B. J. Scholl, What have we learned about attention from multiple object tracking (and vice versa), in Computation, Cognition, and Pylyshyn, D. Dedrick, L. Trick, Eds. (MIT Press, 2009), pp. 49–78. [Google Scholar]
- 45.Poth C. H., Herwig A., Schneider W. X., Breaking object correspondence across saccadic eye movements deteriorates object recognition. Front. Syst. Neurosci. 9, 176 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Mitrani L., Yakimoff N., Smearing of the retinal image during voluntary saccadic eye movements. Vision Res. 10, 405–409 (1970). [DOI] [PubMed] [Google Scholar]
- 47.Burr D. C., Morrone M. C., Ross J., Selective suppression of the magnocellular visual pathway during saccadic eye movements. Nature 371, 511–513 (1994). [DOI] [PubMed] [Google Scholar]
- 48.Bridgeman B., Macknik S. L., Saccadic suppression relies on luminance information. Psychol. Res. 58, 163–168 (1995). [DOI] [PubMed] [Google Scholar]
- 49.Knöll J., Binda P., Morrone M. C., Bremmer F., Spatiotemporal profile of peri-saccadic contrast sensitivity. J. Vis. 11, 15 (2011). [DOI] [PubMed] [Google Scholar]
- 50.Henderson J. M., Chanceaux M., Smith T. J., The influence of clutter on real-world scene search: Evidence from search efficiency and eye movements. J. Vis. 9, 32.1–32.8 (2009). [DOI] [PubMed] [Google Scholar]
- 51.Brooks B. A., Fuchs A. F., Influence of stimulus parameters on visual sensitivity during saccadic eye movement. Vision Res. 15, 1389–1398 (1975). [DOI] [PubMed] [Google Scholar]
- 52.Chekaluk E., Llewellyn K. R., Visual stimulus input, saccadic suppression, and detection of information from the post-saccade scene. Percept. Psychophys. 48, 135–142 (1990). [DOI] [PubMed] [Google Scholar]
- 53.Mitrani L., Mateeff S., Yakimoff N., Is saccadic suppression really saccadic? Vision Res. 11, 1157–1161 (1971). [DOI] [PubMed] [Google Scholar]
- 54.Mitrani L., Yakimoff N., Mateeff S., Saccadic suppression in the presence of structured background. Vision Res. 13, 517–521 (1973). [DOI] [PubMed] [Google Scholar]
- 55.Mante V., Frazor R. A., Bonin V., Geisler W. S., Carandini M., Independence of luminance and contrast in natural scenes and in the early visual system. Nat. Neurosci. 8, 1690–1697 (2005). [DOI] [PubMed] [Google Scholar]
- 56.Tadmor Y., Tolhurst D. J., Calculating the contrasts that retinal ganglion cells and lgn neurones encounter in natural scenes. Vision Res. 40, 3145–3157 (2000). [DOI] [PubMed] [Google Scholar]
- 57.Schweitzer R., Watson T., Watson J., Rolfs M., The joy of retinal painting: A build-it-yourself device for intrasaccadic presentations. Perception 48, 1020–1025 (2019). [DOI] [PubMed] [Google Scholar]
- 58.Kleiner M., Brainard D., Pelli D., Ingling A., Murray R., Broussard C., What is new in psychtoolbox-3. Perception 36, 1–16 (2007). [Google Scholar]
- 59.Pelli D. G., The videotoolbox software for visual psychophysics: Transforming numbers into movies. Spat. Vis. 10, 437–442 (1997). [PubMed] [Google Scholar]
- 60.Wyart V., Nobre A. C., Summerfield C., Dissociable prior influences of signal probability and relevance on visual contrast sensitivity. Proc. Natl. Acad. Sci. 109, 3593–3598 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Engbert R., Kliegl R., Microsaccades uncover the orientation of covert attention. Vision Res. 43, 1035–1045 (2003). [DOI] [PubMed] [Google Scholar]
- 62.Engbert R., Mergenthaler K., Microsaccades are triggered by low retinal image slip. Proc. Natl. Acad. Sci. 103, 7192–7197 (2006). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Bates D., Mächler M., Bolker B., Walker S., Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67, 1–48 (2015). [Google Scholar]
- 64.Harald Baayen R., Milin P., Analyzing reaction times. Int. J. Psychol. Res. 3, 12–28 (2010). [Google Scholar]
- 65.Lo S., Andrews S., To transform or not to transform: Using generalized linear mixed models to analyse reaction time data. Front. Psychol. 6, 1171 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Jarosz A. F., Wiley J., What are the odds? A practical guide to computing and reporting bayes factors. J. Probl. Solving 7, 2 (2014). [Google Scholar]
- 67.Carrasco M., McElree B., Covert attention accelerates the rate of visual information processing. Proc. Natl. Acad. Sci. 98, 5363–5367 (2001). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.R Core Team, R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, 2015); www.R-project.org/.
- 69.S. N. Wood, Generalized Additive Models: An Introduction with R (Chapman and Hall/CRC, 2017). [Google Scholar]
- 70.Wood S. N., Thin plate regression splines. J. R. Stat. Soc. Ser. B 65, 95–114 (2003). [Google Scholar]