Computational Modeling of Collicular Integration of Perceptual Responses and Attention in Microsaccades

Abstract

During visual fixation on a target object, our eyes are not motionless but generate slow fixational eye movements and microsaccades. Effects of visual attention have been observed in both microsaccade rates and spatial directions. Experimental results, however, range from early (<200 ms) to late (>600 ms) effects combined with cue-congruent as well as cue-incongruent microsaccade directions. On the basis of well characterized neural circuitry in superior colliculus, we construct a dynamical model of neural activation that is modulated by perceptual input and visual attention. Our results show that additive integration of low-level perceptual responses and visual attention can explain microsaccade rate and direction effects across a range of visual cueing tasks. These findings suggest that the patterns of microsaccade direction observed in experiments are compatible with a single dynamical mechanism. The basic principles of the model are highly relevant to the general problem of integration of low-level perception and top-down selective attention.

Introduction

Microsaccades are rapid, small-amplitude eye movements that are generated involuntarily during visual fixation (Barlow, 1952; Zuber et al., 1965; Steinman et al., 1973). The function of microsaccades for vision has been investigated extensively (Martinez-Conde et al., 2006; Rucci et al., 2007; Hafed, 2011; for review, see Martinez-Conde et al., 2004, 2009). For example, recent behavioral findings suggest that microsaccades provide a mechanism for precisely relocating gaze in high-acuity tasks (Ko et al., 2010). The primate superior colliculus (SC), the top-level brainstem structure involved in the control of saccadic eye movements (Carello and Krauzlis, 2004; McPeek and Keller, 2004), has been identified as the key neural structure for microsaccade generation (Hafed et al., 2009; Hafed and Krauzlis, 2012).

Since statistical properties of microsaccades are sensitive to changes of perceptual input as well as to modulations of cognitive states (Hafed and Clark, 2002; Engbert and Kliegl, 2003), the analysis of fixational eye movements and microsaccades might provide new insights into the integration of low-level perceptual signals and top-down selective attention (Posner, 1980). In a typical attentional cueing task, effects of covert attention on microsaccades did not depend on the subsequent response type (e.g., saccade or key press; Engbert and Kliegl, 2003). As a first effect of visual or auditory stimulus onsets, a microsaccade rate modulation is observed experimentally (Fig. 1A). A decrease in microsaccade rate is found during a time window from ∼100 to 200 ms after stimulus onset. After this microsaccadic inhibition effect, an epoch of microsaccadic enhancement starts ∼200–250 ms after the display change and extends up to 500 ms, depending on the details of the experimental setup (see also Pastukhov and Braun, 2010; Hafed et al., 2011; for review, see Engbert, 2006). This pattern of rate modulations is qualitatively similar to saccadic inhibition (Reingold and Stampe, 2002), i.e., the decrease in saccade rate following display changes.

Figure 1.

Microsaccade in attentional cueing tasks. A, In attentional cueing experiments, a participant is required to keep the eyes on a fixation stimulus. After a variable interval, a display change occurs. In attentional cueing, exogenous (flash-type), endogenous (symbolic), our auditory cues indicate the direction of a later target stimulus (data not shown). In all condition, microsaccade rate is modulated as an immediate response to the display change. B, Microsaccadic direction effects can be cue-congruent (+) or cue-incongruent (−) with respect to the cue direction. Direction effects depend on the time window considered and on type of cueing.

In addition to microsaccadic rate modulations, experiments with covert spatial attention shifts indicated that the distribution of microsaccade directions can be influenced by attentional cues. The first studies on fixational eye movements in the context of attentional cueing tasks (Fig. 1A) indicated that microsaccades produce a cue-congruent bias (Hafed and Clark, 2002; Engbert and Kliegl, 2003); however, a series of later experiments demonstrated that both cue-congruent and cue-incongruent microsaccadic responses can be found (Rolfs et al., 2004, 2005; Laubrock et al., 2005) (Fig. 1B). Thus, theoretical models of microsaccadic rate and direction effects need to explain a complicated temporal pattern of responses.

Materials and Methods

Baseline model.

We model the generation of fixational eye movements by neural activity in superior colliculus (Hafed et al., 2009) on a finite square lattice of L × L of continuous activations with periodic boundary conditions. We implement a self-avoiding walk (Freund and Grassberger, 1992) as an explicit model for the generation of neural noise (Engbert et al., 2011). In each time step, activation is added to the current position (i, j) of the walker,

while all other lattice sites (k, l) ≠ (i, j) relax slowly according to the equation,

To keep the walker at an intended fixation position, we simulate the self-avoiding walk in a 2D quadratic potential u(i,j) of the form,

where the parameter λ is used to adjust the steepness of the potential and (i₀, j₀) represents the rostral pole of the SC related to movements with vanishing movement amplitude. For the baseline model, the time-varying functions a(t) and b(t) are fixed at a constant value, i.e., a(t) = b(t) = 1. The potential is a parsimonious concept to implement a mechanism to keep the eyes at an intended target position. The walker moves to the lattice position among the four neighboring sites with the smallest value of the sum of self-generated activation and potential. We interpret the numerical value of h_ij as the neural activation of a local population of SC neurons that drive fixational eye movements (Engbert et al., 2011).

A microsaccade is triggered when the walker reaches a lattice site with activation higher than a predefined critical value, i.e., h_ij>h_c. The target location for a microsaccade is given by the lattice site which represents the global minimum of activation. In humans, a preference for horizontal and vertical microsaccade directions is observed (Engbert and Kliegl, 2003). Such a preference could result from the dissociation into horizontal and vertical saccade centers (Sparks, 2002) with independent activity patterns for fixational eye movements. In the model (Engbert et al., 2011), we introduced an additional oculomotor potential to account for the preferences in microsaccade directions. All model parameters were taken from the original publication of the model (Engbert et al., 2011), i.e., L = 51, i₀ = j₀ = 25, ε = 10⁻³, λ = 1, and h_c = 7.9.

Effects of display changes and covert attention shifts.

The modeling strategy in the current work is based on the hypothesis that peripheral stimulus onsets introduce a transient asymmetrical change of the potential. Neurons in the SC representing the peripheral part of the visual field change their activity during covert attention (Fecteau et al., 2004; Ignashchenkova et al., 2004). The transient change of the potential in our model is used as a mathematical tool to introduce a possible coupling mechanism of peripheral SC activity to the activity in the rostral pole that modulates fixational eye movements. A similar approach was chosen by Hafed et al. (2009) in simulations of the activity distribution in SC. Note that the potential studied here is more fundamental than the activity distribution, since the potential drives the activity in our model. More specifically, temporal variations of microsaccade rate and direction were modeled by transient changes of the potential (Fig. 2B) with time-varying functions a(t) and b(t) and a time-dependent threshold h_c(t). First, we model the impact of perceptual input on the slope of the potential by functions:

Second, the attentional variation of the potential is specified as:

where the two free parameters κ and τ_A account for possible changes in waveform (parameter κ gives the order of a polynomial for short time scales while the tail of the function D(t) is an exponential decay that is independent of the value of κ) and temporal delay τ_A with respect to the perceptual change. Third, the attentional change a_A(t) and a delayed version of perceptual change f(t) = a_P(t − τ_P) are additively integrated to result in a function determining the threshold variation for microsaccade triggering, i.e.,

The model parameters were set to τ_P = 150 ms, λ₁ = 0.2, λ₂ = 0.7, ρ₁ = 0.0002, ρ₂ = 0.02, and β = 0.3.

Figure 2.

Self-avoiding walk in a potential. A, A random walk on a discrete square lattice builds up a self-generated activation at each lattice point. The sum of self-generated activation and an additional movement potential determine movement direction. Visiting a lattice site with overcritical activation induces a microsaccade (straight line) to the global minimum of the sum of activation and potential. B, Temporal variations of the movement potential occur as a result of changes in perceptual input and attentional effects. The changes are transient and are different for low-level perceptual effects (dashed line) and directional attention effects (green line).

In Figure 3A, κ = 1.0 and τ_A = 30 ms. In Figure 4A, parameters κ and τ_A were varied systematically. The model's spatial distribution of activation {h_ij} is self-generated; earlier simulations indicated that it takes several thousand iterations for a stationary distribution to evolve (Engbert et al., 2011). Therefore, the first 5000 iterations were discarded from each model run to remove transients from the statistical analyses of the model's behavior.

Figure 3.

Modulation of microsaccade rate and direction. A, Transient variations of the potential due to perceptual input change (black line) and attention (green) and resulting threshold variation (red dashed line). B, Microsaccadic directions computed from 2000 runs of the model (raster plot). C, Corresponding rate modulation (blue line) computed from raster plot (where each line represents a single model run). D, Rate difference between cue-congruent and cue-incongruent microsaccades (red line) and bootstrap-based confidence region around the zero difference.

Figure 4.

Qualitative analysis of the model's dynamics. A, Variations of attentional delay τ_A and waveform exponent κ_A indicates that three regions with endogenous cue-congruent (blue), endogenous cue-incongruent (orange), exogenous (red) pattern of microsaccadic direction effects are reproduced by the model. B–D, Patterns of microsaccadic direction effects as a function of time for three different combinations of model parameters highlighted in the 2D plot in A.

Results

In any successful computational model, microsaccades should be generated as a consequence of neural noise (Otero-Millan et al., 2011), which is supported by exponentially distributed inter-microsaccade intervals, compatible with a Poisson process of temporally independent events (Engbert, 2006; Mergenthaler and Engbert, 2010). However, recent experiments indicated that microsaccades might also be part of the process of gaze-control in high-acuity tasks (Ko et al., 2010). The kinematic similarity between saccades and microsaccades lends support to a subthreshold mechanism for the generation of microsaccades within the same neural circuitry as for voluntary saccades (Zuber et al., 1965; Otero-Millan et al., 2008; Hafed and Krauzlis, 2012). Furthermore, models should be evaluated based on a number of criteria (Otero-Millan et al., 2011): The distributions of saccadic intervals, the relation between microsaccades and slow movements (drift), and the timing and waveforms of microsaccadic inhibition.

We used a recently proposed integrated model of fixational eye movements and microsaccades (Engbert et al., 2011) to simulate microsaccades under the influence of display changes and selective attention. The model reproduced a range of behavioral findings based on an emergent dynamical coupling of microsaccades and slow movements (Engbert and Kliegl, 2004; Engbert and Mergenthaler, 2006) and might provide an adequate theory for the statistical properties of slow fixational eye movements and microsaccades. The dynamic principle underlying the model is a self-avoiding random walk (Freund and Grassberger, 1992) in a potential as an integrative framework for the generation of both slow movements (physiological drift) and microsaccades (Engbert et al., 2011), i.e., drift (slow movements) and microsaccades were produced by the same laws of motion.

In the model, activation is increased at the walker's current position while all other lattice sites relax (Fig. 2A). A movement potential, which is added to the self-generated activation field, keeps the walker at an intended fixation position (Fig. 2B). The walker tends to a local minimum of the sum of self-generated activation and potential (slow movements). A microsaccade to the global minimum across the lattice is generated, when the walker visits a lattice site with overcritical activation (see Materials and Methods).

Since the model's movement potential is the dynamical mechanism underlying the intention to fixate a particular location, a temporal variation of the slope of the potential can be used to implement effects of display changes and top-down selective attention (Hafed and Clark, 2002; Engbert and Kliegl, 2003; Rolfs et al., 2004, 2005; Laubrock et al., 2005). We assume that a display change induces a transient global reduction of the slope of the potential (Fig. 2B, dashed line), while an attentional shift to the periphery introduces an asymmetrical deformation of the potential toward the locus of attention (Fig. 2B, green line). This assumption is compatible with earlier simulations of changes of neural activity in microsaccade generation (Hafed et al., 2009).

Our model simulations are based on three critical assumptions (see Materials and Methods). First, we assume that the transient reduction of the potential (Fig. 3A, black line) by changes in perceptual input is followed by a delayed signal that is forwarded to the trigger for microsaccades (Fig. 3A, blue line). Second, we postulate that attention causes a similar, but asymmetrical change of the potential that follows the display change with a variable temporal delay and a specific waveform determined by the properties of the attentional cue (Fig. 3A, green line). Third, both signals are integrated additively and result in a threshold variation that is determined by both perceptual input and attentional response (Fig. 3A, dotted red line). Such an integration mechanism is compatible with the neuroanatomical structure of the SC with well characterized sensory and motor layers and a functional connection (Lee et al., 1997).

The immediate response of the model's intrinsic dynamics to the transient change of the potential is a reduction of the microsaccade rate (Fig. 3C), while a later change of the threshold generates the overshoot in microsaccade rate observed in experiments. It is important to note that this change of the potential does not introduce a direction bias, which is induced only later by effects of covert attention (Fig. 3B,D). When rate differences of cue-congruent and cue-incongruent microsaccades are plotted (Fig. 3D), an early and strong attentional variation of the potential generates a pattern of immediate cue-congruent, an intermediate cue-incongruent, and a late cue-congruent microsaccadic direction bias. For statistical evaluation of the obtained direction patterns, we applied a rate-estimation procedure with a causal window developed for spike train analysis (Dayan and Abbott, 2001; see also Engbert, 2006). Randomization of microsaccade directions was used to compute confidence regions around the baseline of a zero difference in microsaccade directions (Fig. 3D, blue area). The resulting statistically significant patterns of directions are typically observed in experiments with exogenous flash cues (Laubrock et al., 2005).

Based on systematic numerical simulations of the model, we identified the two critical parameters of the attentional cues that had a strong impact on the interaction of both rate and directions of microsaccades. First, the delay of the earliest effect of selective attention relative to the immediate perceptual effect on the movement potential varies between exogenous flash-type visual cues or auditory cues (short delay) and endogenous symbolic cues that need top-down processing before an attentional response can be generated across the oculomotor pathway (long delay). Second, the waveform of the attentional response can in principle range from a steep and short (κ small) to a slow and more sustained (κ large) transient reduction of the potential. A systematic variation of both parameters indicates that the model predicts the typical effects of attention on microsaccades found in experiments (Fig. 4A). Auditory cues produce intermediate cue-incongruent effects (Rolfs et al., 2005) that are captured by model simulations with a short-delay, but slow potential change (Fig. 4B). Simulation of endogenous cues which produce a cue-congruent microsaccade bias (Hafed and Clark, 2002; Engbert and Kliegl, 2003) are obtained for longer delays (Fig. 4C). Finally, the model generated the more complicated pattern with cue-congruent as well as cue-incongruent microsaccade effects for exogenous cues (Laubrock et al., 2005; Rolfs et al., 2005) (Fig. 4D).

Discussion

Microsaccades recorded during visual fixation before a motor response to a peripheral stimulus provide an important tool for the analysis of vision, attention, and eye movements. The present study proposed a mathematical framework to simulate influences of peripheral stimulus onsets on microsaccades during visual fixation. We identified two critical parameters of experimental manipulations of attention: (1) the temporal delays of the attention effects relative to stimulus onset and (2) the waveform (or steepness) of the attentional onset. Variations of these two parameters could explain complicated patterns of attentional effects on microsaccades observed in experiments. The simulation of the model suggest that properties of the attentional manipulations interact with the intrinsic dynamics of fixational eye movements (i.e., slow movements, microsaccades, and their interaction) to reproduce the observed patterns of microsaccade rate and direction effects.

In our model, low-level perceptual responses and top-down attentional modulation are integrated to produce the highly specific responses of microsaccades across a number of attentional cueing tasks. While the low-level responses are strongly expressed in microsaccade rates, the effects of selective attention are observed in microsaccade directions. The model proposed and analyzed in this study is based on the assumption of an additive integration of both processes as a single mechanism to explain a range of experimentally observed effects. These results extend earlier simulations of neural activation underlying the control of microsaccades in superior colliculus (Hafed et al., 2009; Hafed and Krauzlis, 2012).

Our model might be extended to the more general problem of saccadic inhibition (Reingold and Stampe, 2002; Buonocore and McIntosh, 2008), where a transient change of a scene display during a visual task induces a decrease in saccade rate. Solutions to the problem of integration of low-level perceptual signals (Martinez-Conde et al., 2000; Rucci and Casile, 2004; Hafed and Krauzlis, 2010) and top-down attention (Posner, 1980; Hafed et al., 2011) studied here in the context of microsaccades might contribute to other aspects of this long-standing issue in active vision (Findlay and Walker, 1999) and scene perception (Itti and Koch, 2001).