Visual Perception and Saccadic Eye Movements

Abstract

We use saccades several times per second to move the fovea between points of interest and build an understanding of our visual environment. Recent behavioral experiments show evidence for the integration of pre- and postsaccadic information (even subliminally), the modulation of visual sensitivity, and the rapid reallocation of attention. The recent physiological literature has identified a characteristic modulation of neural responsiveness - perisaccadic reduction followed by a postsaccadic increase that is found in many visual areas, but whose source is as yet unknown. This modulation seems optimal for reducing sensitivity during and boosting sensitivity between saccades, but no study has yet established a direct causal link between neural and behavioral changes.

Introduction

No understanding of vision can ignore the basic limitations of the retina and the fact that our eyes change gaze direction many times per second. In primates only the central few degrees of the visual field have the photoreceptor density to offer sufficient fidelity for high-resolution vision. Primates therefore make frequent saccades to capture detailed snapshots with the fovea and integrate those into a coherent understanding of the visual environment.

To accumulate this information across multiple snapshots, the visual system must overcome several challenges with each eye movement [1]. First, it must link the representation of objects before and after a saccade. This requires either information on the position of the eye (i.e. an egocentric reference), or an explicit identification of salient landmarks as being the same before and after the saccade (i.e. an allocentric, or world-fixed reference). Second, because the retinal motion caused by eye-movements is a potent visual stimulus that should not be confused with real object motion, it must be marked as special, or hidden from awareness. Third, because attentional resources are limited and affect neurons with eye-centered receptive fields, the brain must re-allocate these resources with each eye movement. While these challenges are conceptually independent, implementations of their solutions could overlap significantly. This overlap may lead to some of the perceptual disturbances that occur around the time of saccades and can provide interesting clues about visual processing [2,3].

It is essential, however, to keep note of the main game - that the goal of perisaccadic processing is to retrieve information from the visual environment while maintaining perceptual stability. Our goal here is to review insights into perisaccadic processing based on the recent literature.

Saccadic Suppression

Every saccade generates some retinal motion that is within range for motion detectors. Introspection, however, clearly shows that these motion signals never reach awareness. In the laboratory, this behavioral phenomenon is called saccadic suppression and is usually investigated by presenting stimuli before, during or after saccades and quantifying whether subjects perceive some aspect of the stimulus.

Vision is impaired from approximately 100ms before until 100ms after saccade-onset [1]. Experiments on perisaccadic perception in the last two years have shown that cardinal orientation discrimination is impaired while oblique orientation discrimination is slightly improved [4], even high spatial frequency letters are more difficult to identify [5], and three-dimensional depth, in the form of binocular disparity, is more difficult to discern [6]. Saccades also influence perception beyond low level vision, as evidenced by perisaccadic overestimation of numerosity [7] and mislocalization of auditory stimuli [8]. It seems unlikely that all these phenomena are intended consequences of perisaccadic processing; we believe they are examples of interference between the multiple perisaccadic tasks that the visual system is trying to accomplish.

Given the pervasive influence of saccades on so many different perceptual processes, an attractive hypothesis states that saccadic suppression is implemented at the earliest level of the visual system; the lateral geniculate nucleus (LGN) of the thalamus [9] and inherited by later stages. Early and absolute suppression of the visual input would make sense if the goal was to completely remove the visual input. Several recent perceptual experiments, however, speak against this hypothesis, and suggest that while the LGN likely plays a role in saccadic suppression, not all saccadic suppression is inherited from the LGN.

First, the strength of suppression is determined by the combined properties of the visual input to both eyes [10]; such binocular interactions require cortical involvement. Second, a number of studies have shown that intrasaccadic displacements of a stimulus can become visible when the stimulus is switched off and on again after the saccade [11], or even when it changes shape after the saccade[12]. Similarly, intrasaccadic displacements that are not perceived can be the target of correct hand-pointing movements [13]. A third recent study used a visual form illusion in which the brief presentation of a bar makes a subsequently presented circle appear elliptical. When a bar was presented during presaccadic suppression, and the subjects reported that they saw no bar (i.e. perceptually the bar was omitted), a circle presented after the saccade still appeared elliptical [14]. Together these studies show that perisaccadic visual input is not removed from the visual system; the information is available, and retrievable [12].

Electrophysiological studies in behaving monkeys have identified a number of perisaccadic changes in neural response properties that could underlie these behavioral phenomena. While there is variability at the single neuron level [15], a common theme at the population level is that response amplitudes are reduced just before and during saccades, and increased afterwards (Figure 1). This pattern of response modulation is reliably found in the lateral geniculate nucleus (LGN) [16–18], the primary visual cortex (V1) [19,20], and in areas of the dorsal stream: middle temporal (MT [21,22]), medial superior temporal (MST[21–23]) and ventral intraparietal (VIP[22]). Responses in the superior colliculus (SC) [24], the pulvinar [25], and the lateral intraparietal area (LIP) [22] are generally reduced before, during, and after the saccade, with little evidence for postsaccadic increases. While earlier studies suggested that response reduction was weak in LGN and V1 and only gathered full strength in the parietal cortex, recent data show that even LGN cells can exhibit strong perisaccadic response reduction [18]. It is important to note that the attenuation is not absolute some signal remains and this is critical when trying to understand the perisaccadic perceptual changes outlined above. It is also important to consider that differences in experimental design and methods of data analysis between studies can influence the amplitude of perisaccadic response reduction and postsaccadic response increases (see legend to Figure 1). A fair comparison requires the use of identical experimental paradigms and analyses across areas. Finally, the evidence that these response changes are true neural correlates of the behavioral phenomenon of saccadic suppression is still circumstantial; box 1 discusses some of the challenges one faces when trying to identify the neural mechanisms of saccadic suppression.

Figure 1. The pattern of perisaccadic modulation.

The horizontal axis shows the time relative to saccade onset at which stimuli were presented. The vertical axis shows the scaled modulation of the average firing rate for different areas in the primate brain as well as the scaled average detection performance of two human observers [57]. Zero on the vertical axis represents the average firing rate/performance during steady fixation.

In SC experiments, monkeys made small fixational saccades (<30’), in LGN experiments monkeys made voluntary saccades (1.2°–12°) between targets [16] or made small (20’–40’) involuntary saccades during fixation [18]. In V1 experiments [19], voluntary and fixational saccades ranged in size up to 2.4°. In experiments on parietal areas the monkeys made voluntary saccades of 10° [23] or 20°[22]. In the human experiments subjects made 12° saccades[57]. The stimuli were moving or flashed bars (SC, V1, MST, and VIP), large-field textures (MST), gratings (humans), whole-field luminance modulations (LGN), or pseudorandom white noise (LGN). Two studies [18,19] used gaze-contingent stimulus presentation to keep the stimulus within the receptive field.

In our view these experimental differences likely contribute to considerable differences in the absolute size of the perisaccadic response modulations. Therefore, to avoid unwarranted comparisons of absolute modulation strength across studies, we scaled the curves per study to set the range (difference between the highest/best and lowest/worst time point) to 1. As a result, the vertical scale is somewhat arbitrary, but this scaling maintains the magnitude relationship between troughs and peaks within a study, as well as the time course across studies. In other words, the trough of the yellow curve indicates that nearly all (90%) behavioral modulation was a reduction of performance relative to fixation.

This representation highlights the consistent finding that a firing rate reduction close to saccade-onset is often followed by a postsaccadic firing rate increase.

Box 1. Identifying Neural Mechanisms.

The link between changes in neural responsiveness and the behavioral phenomenon of saccadic suppression remains indirect. Importantly, monkeys do show behavioral saccadic suppression: saccadic reaction times are slower [24] and contrast detection thresholds are elevated [20] when targets are presented during micro-saccades. Moreover, the time course of neural response changes in many brain areas match the time course of behavioral saccadic suppression in humans (Fig. 1), and the stimulus selective nature of behavioral suppression [9] finds its correlate in single cell recordings [26] and functional imaging [27] in V4.

However, there is no conclusive evidence demonstrating a direct causal link between changes in neural activity and perceptual suppression. A major complicating factor is uncertainty about which aspect of neural firing would best predict behavioral performance. Theoretically, the behavioral effect could be caused by a reduction in firing [9], an increase in variability [15], an increase in firing (saturation) [57], population effects such as changes in synchrony [58,59], an increase in temporal or spatial uncertainty due to changes in temporal or spatial integration [7], or any combination thereof. Most of these proposals have at least some indirect empirical support. Unpacking these complexities will require the simultaneous measurement of behavioral and neural responses, as well as formal, quantitative models [3,60]that can guide and constrain the search for neural mechanisms.

In the ventral cortical stream, neurons in V4 show significantly reduced response amplitudes for presaccadic stimuli defined by luminance, but much smaller and more variable effects for chromatic stimuli [26]. This fits well with functional imaging results in human V4 [27], and matches the behavioral finding that saccadic suppression affects achromatic stimuli more than chromatic stimuli [28]; this adds to the confidence that these response changes in V4 are neural correlates of saccadic suppression. Interestingly, this stimulus specificity is not found in V1, where cells preferring chromatic or achromatic stimuli are similarly modulated around saccades [20]. This study adds further evidence that not all perisaccadic changes in firing rate are inherited from the LGN and leads us to the broader point that even though the time course of perisaccadic firing rate changes is similar across areas (Figure 1), there are important differences at a finer level of detail. These differences may be related to the idiosyncratic roles each area plays in visual perception. For instance, an area involved in motion processing such as MT may have quite different demands on the perisaccadic responses than an area more typically associated with the allocation of attention (LIP)[22]. This suggests that instead of a narrow search for a neural correlate of saccadic suppression, a broader investigation of perisaccadic changes in neural processing and how they are related to the functional specialization of an area, may be a fruitful approach for future studies.

Transsaccadic Transfer

When the eyes move to a new position, some information about the presaccadic scene must be maintained to explain the incremental build-up of an understanding of the visual environment [29]. At what level of abstraction is presaccadic information stored? The observation that presaccadic information can be masked by low-level visual stimuli presented after the saccade [30] argues for storage that is not entirely abstract. Moreover, some reports claim that low level features of the visual scene are transferred across saccades (motion [31,32], orientation [33,34], color [35]). However, the claims about motion features [36–38] and orientation [39] are strongly contested.

The idea that information on low-level features is transferred is based on the physiological finding of remapping; some neurons respond to stimuli that will be brought into their receptive field by an impending saccade. This property is common in areas such as LIP, frontal eye fields (FEF) and SC, but found with decreasing prevalence in early visual areas (V1, V2, V3) [1]. The fact that remapping is common in areas without strong feature selectivity and relatively rare in the visual areas that carry low-level feature information suggests that remapping may not be feature specific, and may be limited to salience, or the allocation of attention[40,41].

Saccadic Enhancement

We usually make saccades to inspect salient locations, so we would expect attentional resources to be re-allocated around saccades. Consistent with this, improved discrimination performance at the target location can be observed before a saccade [42]. Few studies have specifically investigated behavioral performance immediately after saccades, however, available data suggest that performance is improved at that time (e.g. ocular following [21,43], reaction times [44] ). We refer to these behavioral phenomena as saccadic enhancement.

Tentative neural correlates of these phenomena have recently been identified. For instance, before each saccade, the visual responses of neurons in cortical area V4 whose receptive fields overlap with saccade targets sharpen their orientation tuning and reduce their trial-to-trial variability [45]. This could underlie the saccadic enhancement of orientation discrimination.

After each saccade, many visual areas show a response increase that could underlie postsaccadic behavioral enhancements (Fig. 1). The onset of the postsaccadic increase coincides with saccade-end, persists for 200–400 ms, and is a strong effect throughout the geniculo-cortical pathway [19,22,23,46]. The response increases are accompanied by a reduction in response latency [18,21–23,47]. Thus, visual responses are not only larger they also arrive in the brain earlier.

The strength of the post-saccadic increase varies between reports, even from the same cortical areas. This is not unexpected as any effect that increases the neural response is limited by a given cell’s dynamic properties. Thus, the increase is expected to be stronger for weakly driving stimuli [23] than for bright flashes [22]. In macaque V1 and MST, postsaccadic activity increases are also found in the absence of visual stimulation [15,19,21], and in V1 they are the consequence of neither intrasaccadic motion, nor adaptation [46]. These data suggest an extraretinal origin for these response increases. Moreover, the increases appear particularly strong and long-lasting under ecologically relevant conditions when stimuli embedded in natural scenes are brought into the receptive field by a saccade [46]. Finally, in human V1 image displacements caused by voluntary saccades evoke larger BOLD responses than those same image displacements during fixation [48]. Even though the time course of this BOLD effect is not yet known, it is consistent with a postsaccadic response increase.

Primates, including humans, make about three saccades per second. Thus, the duration of the postsaccadic response increase matches the typical inter-saccade interval, suggesting that the effect could be dubbed an “inter-saccadic increase”. Functionally, the increased responsiveness could be a signature of the allocation of attentional resources to the new fixation location, although this conclusion is premature as too many aspects of postsaccadic increases remain unknown. For instance, is the increase maximal at specific locations in the visual field or is it spatially widespread? Which properties of the visual stimulus, saccade or cortical area determine its strength, onset, and duration? In Box 2 we discuss how these postsaccadic increases may also help to achieve complete behavioral saccadic suppression.

Box 2. A role for postsaccadic response increases in saccadic suppression?

It has long been argued that the arrival of the eye in a new target location provides a far more robust stimulus than the rapidly moving scene during the saccade. Because the presentation of a salient pattern between 0 and 150 ms after a stimulus can remove that stimulus from awareness entirely, this suggests that the robust image at re-fixation could overpower awareness of the retinal motion that occurs during the saccade, even without an active saccadic-suppression mechanism [61,62]. However, this theory cannot explain all visual situations where, for example, it is perfectly possible to make saccades between low contrast stimuli but move the eyes across a high contrast feature. The biphasic pattern of response modulation seen at the time of saccades (Fig. 1) could be a very effective mechanism to achieve saccadic suppression [63]. In this theory the response to visual stimulation during the saccade is weakened by perisaccadic response reduction, but some signal remains. By increasing visual responses after the saccade, the response to even weak postsaccadic stimulation is boosted sufficiently to mask what came before. This theory unifies what has been called active suppression with backward masking. Note, however, that backward masking in this theory is not the passive consequence of the mere presence of a post-saccadic stimulus, but the result of actively boosting the response to the post-saccadic stimulus.

Circuitry

One pathway that actively contributes to perceptual stability connects the SC to the FEF via the medio-dorsal thalamus. Without this pathway, monkeys are impaired in their ability to transfer spatial information across saccades (for review, see [49]). The pervasive influence of the frontal eye fields on visual processing has been demonstrated recently using a combination of functional imaging and microstimulation [50,51]. We speculate that this circuitry may also underlie both perisaccadic response reductions and postsaccadic response increases.

In humans similar circuits are likely involved in perceptual stability: Transcranial Magnetic Stimulation (TMS) of the human frontal eye fields impairs perceptual stability (Ostendorf F et al. J. of Vision 10/7/518 (2010)), as does a lesion in the (central) thalamus [52]. The parietal cortex is the fourth player in the circuit for stability; TMS over parietal areas impairs stability [53], and parietal patients [54] show stability deficits. It is not yet known, however, whether these parietal areas, like the frontal eye fields, can modulate activity in early visual areas.

Mapping the cortical and subcortical pathways involved in modulating visual processing around saccades is a major task that will benefit greatly from the whole-brain view afforded by functional imaging, especially in combination with microstimulation [51,55]. One particular challenge will be to determine whether each of the three main candidate areas (SC, FEF, LIP) plays a unique role, or whether instead they partially serve as each other’s backup [56].

Conclusion

The view that visual processing faces a difficult task in the presence of saccades has been around for well over a century. Recent experiments have started to study perisaccadic perception and the underlying neural mechanisms quantitatively. Across the visual system, neural response amplitudes are modulated at the time of saccades, showing perisaccadic reduction and postsaccadic increases in activity. The next challenge is to find the direct link between these neural changes, the perceptual changes, and the tasks that the visual system needs to complete with every eye movement.

Highlights

• Responses in many visual areas are reduced during but also just before saccades.

• Immediately after each saccade responses are often increased for 200–400 ms.

• Perisaccadic visual stimulation that is not perceived is not lost; it can be retrieved.

• As yet, the link between neural and behavioral perisaccadic changes is indirect.

• Little information on low level features is transferred across saccades.

• Global brain circuits for perceptual stability are being unraveled.