A physiological perspective on fixational eye movements

Abstract

For a behavioral neuroscientist, fixational eye movements are a double-edged sword. On one edge, they make control of visual stimuli difficult, but on the other edge they provide insight into the ways the visual system acquires information from the environment. We have studied macaque monkeys as models for human visual systems. Fixational eye movements of monkeys are similar to those of humans but they are more often vertically biased and spatially more dispersed. Eye movements scatter stimuli from their intended retinal locations, increase variability of neuronal responses, inflate estimates of receptive field size, and decrease measures of response amplitude. They also bias against successful stimulation of extremely selective cells. Compensating for eye movements reduced these errors and revealed a fine-grained motion pathway from V1 feeding the cortical ventral stream. Compensation is a useful tool for the experimenter, but rather than compensating for eye movements, the brain utilizes them as part of its input. The saccades and drifts that occur during fixation selectively activate different types of V1 neurons. Cells that prefer slower speeds respond during the drift periods with maintained discharges and tend to have smaller receptive fields that are selective for sign of contrast. They are well suited to code small details of the image and to enable our fine detailed vision. Cells that prefer higher speeds fire transient bursts of spikes when the receptive field leaves, crosses, or lands on a stimulus, but only the most transient ones (about one-third of our sample) failed to respond during drifts. Voluntary and fixational saccades had very similar effects, including the presence of a biphasic extraretinal modulation that interacted with stimulus-driven responses. Saccades evoke synchronous bursts that can enhance visibility but these bursts may also participate in the visual masking that contributes to saccadic suppression. Study of the small eye movements of fixation may illuminate some of the big problems in vision.

1. Introduction

Fixational eye movements are the movements that occur when subjects are trying to control their gaze within a restricted location. They occur in a variety of situations and they serve diverse purposes. When scanning a visual scene, fixational eye movements occur for short pauses between the large saccadic eye movements that move the eye from place to place. These fixational pauses consist of unintentional slow drifts and involuntary small saccades while the subject acquires information before deliberately saccading to a new location of interest. Longer periods of fixation occur when performing fine sensorimotor tasks, such as threading a needle. Under these circumstances, tiny fixational saccades may move the gaze between two nearby objects to accomplish a demanding task (e.g. Ko, Poletti, and Rucci, 2010; Poletti, Listorti, and Rucci, 2013). A different kind of task involves a subject waiting for something unpredictable to happen at a particular place--for example, when a predator is waiting for a small prey to emerge from a hiding place. Then, fixational drifts and saccades occur while keeping the fovea on target. We call this maintained fixation, and it is the main task that has been employed for physiological studies of fixational eye movements.

This is a focused review of work done in my laboratory and closely related work from other laboratories beginning in the 1970’s and continuing to the present day. I apologize in advance to colleagues whose equally valuable contributions may not be discussed adequately in this framework. The emphasis of this review is on understanding how fixational eye movements affect the acquisition of sensory information and how they relate to neural coding in the visual pathway from the retina to the early stages of the visual cortex. Many (perhaps most) behavioral neuroscientists regard fixational eye movements as a nuisance, because they are not under the control of the experimenter. However, they are an integral part of the visual process and we gain important insights by understanding their effects.

Our story begins with behavioral studies comparing monkey and human fixational eye movements. Next, I describe how fixational eye movements influence descriptions of neuronal response properties, the mapping of visual receptive fields in physiological studies and sampling biases for neuronal cell types. Then, I consider how fixational eye movements contribute to neural coding of specific types of sensory information,. Finally, I discuss how extraretinal influences linked to saccades—both voluntary and involuntary—modulate neuronal activity and interact with stimulus-driven responses to determine the input to the rest of the brain and the sensory process. All work was carried out in accordance with the code of Ethics of the World Medical Association (Declaration of Helsinki), including informed consent of human subjects.

2. Macaque monkeys as models for the study of human fixational eye movements

“Macaque” monkeys are members of the Asian genus Macaca that has been shown to have color and spatial vision nearly identical to humans (DeValois et al., 1974; DeValois, Morgan, and Snodderly, 1974). Most investigations of eye movements have used Macaca mulatta, the rhesus monkey, or Macaca fascicularis, also known as the cynomolgous monkey. Although there are measureable differences in some retinal features between these species (Snodderly and Sandstrom, 2008), no important differences in their eye movements have been documented to date.

Skavenski et al. (1975) conducted the first quantitative study of monkey fixational eye movements, using rhesus monkeys with implanted scleral search coils (Robinson, 1963). They showed that the monkeys could learn to control eye position within a long (15s) fixation trial with a precision similar to humans, but only after extensive training with extremely stringent criteria. However, these conditions are not compatible with physiological investitgations and they are not representative of natural vision. To compare performance under less extreme conditions, we measured eye position of humans and M. fascicularis monkeys during fixation tasks lasting 1–4 s (Snodderly and Kurtz, 1985). Fixation targets were presented in the dark, and eye position was measured with a dual-Purkinje image eyetracker (Crane and Steele, 1978). The fixation target either dimmed or changed orientation at an unpredictable time. One of the human subjects was trained nonverbally to be sure that performance was controlled by the task, not by instructions.

Monkeys had much greater trial-to-trial dispersion of fixation position on the vertical axis than humans did (See also Motter and Poggio, 1984). This dispersion resulted from less precise control of saccades in the dark environment. The poorer control caused an “upshift” of fixational eye positions in the dark that was seen in the monkeys, but not in humans (Snodderly, 1987). The upshift in the dark was confirmed in two other laboratories (Barash et al., 1998, for M. fascicularis and Goffart et al, 2006, for M. mulatta) who showed that the upshift occurred with large voluntary saccades and with memory-guided saccades in the dark as well. The fact that the lighted environment and stimulation of the extrafoveal retina was sufficient to eliminate the upshift with minimal effect on horizontal eye position indicates a different influence of the parafoveal retina on the vertical and the horizontal eye movement systems of the monkeys. The separation of the vertical and horizontal oculomotor control systems in the brainstem (Krauzlis, 2008) may predispose these subsystems to receive somewhat different sensory inputs.

Both the monkeys and one of the human subjects in our initial study made smaller, but more frequent saccades in the light. In general, monkey fixational eye positions and eye movements became much more similar to humans when tested in a lighted environment (Snodderly, 1987). However, saccadic displacements (sizes, see below) of the monkeys were still 2–4 times those of humans, and between-trial standard deviations of mean eye position were 2–7 times as large. During a maintained fixation task, about half the monkeys studied in my lab (Snodderly, 1987; Kagan, Gur and Snodderly, 2008) and in the Horwitz lab (Horwitz and Albright, 2003; Hass and Horwitz, 2011) showed a pattern of upward drift counteracted by downward saccades. This pattern of movements appears to reflect a general tendency for the eyes to drift upward whenever visual stimulation is minimal, such as in total darkness or with a small, isolated fixation target. Apparently, many monkeys cannot completely eliminate the upward drift; consequently, they must make corrective downward saccades to maintain a stable mean eye position. The vertical bias of the monkey eye movements differs from the behavior of most human subjects, who are more likely to exhibit a distribution of drifts and corrective saccades with a horizontal bias or a radial symmetry (e.g., Cherici et al., 2012).

Saccade control during maintained fixation seems to be a difficult challenge for some individuals, both monkeys and humans. Fig 1 illustrates a phenomenon that I called saccade clusters, in which a fixational saccade (FX) away from the fixation locus is followed with no delay by one or more saccades that counteract it (left column). Upper panels show data from a human subject and lower panels display data from a monkey. This behavior suggests that the first saccade is unwanted, and its effect is cancelled immediately by some low-level monitoring network that does not require time for conscious intervention. The saccade cluster executes a looping movement that returns the eye to the vicinity of the mean fixation locus (middle column; see also Fig. 7, and observations by Horwitz and Albright, 2003). Voluntary saccades (VL) elicited by stepping the fixation point do not show such complex waveforms (left column), and they carry the eye in a simple, nearly linear trajectory (right column). Although most researchers refer to all small saccades as “microsaccades”, the looping saccades have motivated me to distinguish fixational saccades (when the subject is trying to maintain a steady gaze) from voluntary ones (when the subject is instructed to shift gaze to another point). A voluntary saccade and a fixational saccade cluster may have very different net displacement even though they cause comparable maximum displacement. The complex waveforms of the fixational saccade clusters are not an artifact of the eyetracker nor are they limited to subjects whose heads are fixed. Similar waveforms can be seen in records from subject RS with a magnetic search coil and the head free (fig. 5 of Skavenski et al., 1979; reproduced more clearly as Fig. 2 of Steinman et al., 1982).

Figure 1.

Comparison of fixational and voluntary saccades. Upper panels left, position vs time plots of a fixational saccade cluster, FX, of human subject H93, along with a voluntary saccade, VL, elicited by abruptly stepping the fixation target to the right. Upper panels right. Shorter segments of the same fixation trial plotted two dimensionally to show the spatial trajectories of the saccades. For FX the trial segment from 0 to 0.7 sec was plotted and for VL, the time period was 0.7–1.1 sec. The lower panels give a similar display of fixational and voluntary saccades for monkey M11. For both FX and VL a trial segment of 0.5 sec including the saccade cluster is plotted two-dimensionally in the lower right panels. The zero position is the mean of eye position in the last half-second in all trials of the sessions in which these trials were recorded. Recordings made with a dual Purkinje image eyetracker. From Snodderly (1987).

Figure 7.

Characteristics of a mixed cell. Conventions are the same as in Figs. 4–6. (A) Activation of the cell by an optimally oriented green incremental stationary bar 5 min wide, placed at the mean receptive-field position (file 20). Four separate time segments (I–IV) from different trials are shown. Arrowheads under the time axis indicate the times of saccade occurrences. (B) Saccade-triggered histograms of average spike frequency for saccades that were followed by increased activity (n=23), saccades followed by decreased activity (n=11), and all saccades combined (both, n=34). Saccades occurred at t=0. Also shown is the average response histogram of the cell to flashing the stimulus bar (on 150 ms, off 250 ms for each repetition). (C) Gray-scale map of CRF in eye position coordinates, along with elliptical outline used in subsequent panels. Stimulus bar used to generate the map is drawn to the same spatial scale. (D) Spatial interactions between stimulus and CRF during the trial segments in panel A. CRF moves onto stimulus. II—CRF moves off the stimulus. Eye position axis has a different scale in this panel to accommodate large saccade. III—Small looping saccade pair moves CRF so that stimulus remains within the CRF. IV—Successive small saccades move CRF, keeping stimulus within the CRF. (From Snodderly et al., 2001).

For larger voluntary saccades, waveforms recorded with the dual Purkinje image eyetracker display an “overshoot” that is caused by inertial lag and oscillation in the movement of the lens as the eye moves abruptly to a new position (Deubel and Bridgeman, 1995a; Tabernero and Artal, 2014). However, for smaller voluntary saccades in the size range of fixational saccades (< 1°), Fig. 1 shows that the “overshoots” are minimal, consistent with the principle that inertial forces on the lens should be reduced for small saccades. It seems likely that some of the data for small saccades in Fig. 5 of Deubel and Bridgeman (1995a) that imply overshoots as large as the saccade, may instead represent looping back-to-back saccades of the type illustrated here. Given that inertial motions of the lens are accompanied by perceptual disturbances (Deubel and Bridgeman, 1995b) it is functionally advantageous that the lens motions are negligible during small saccades that may occur during demanding visual tasks.

Publications from my lab have either reported both net and maximum displacements (Snodderly, 1987), or only maximum displacements (Snodderly et al., 2001; Kagan et al., 2008). Other laboratories have often reported the net displacement only (e.g. Horwitz and Albright, 2003; Chen and Hafed, 2013), so readers will need to make comparisons with care. The examples here of differences between net and maximum displacement are relatively extreme, and they are idiosyncratically dependent on the subject, but even for less extreme cases the measures of saccade metrics need to be clearly specified.

In summary, fixational eye movement patterns of monkeys generally show higher dispersion of eye position than humans. Monkeys’ eye position distributions often are vertically biased, but like humans, their distributions of eye position and saccade directions are very idiosyncratic; they can include fixational saccade trajectories unlike those of voluntary saccades. Furthermore, small changes in the fixation task can modify the distributions of eye position and saccades (Snodderly and Kurtz, 1985). Consequently, for precise studies of physiological mechanisms, accurate measures of eye position and eye movement have been an integral part of our experiments. At various times we have used either a dual Purkinje Image eyetracker (Crane and Steele, 1978) or an implanted scleral search coil (Robinson, 1963; Remmel, 1984). Both of these instruments measure eye position with a spatial resolution of 1–3minarc, and temporal band pass of 300 Hz or more.

3. Effects of fixational eye movements on measurements of neuronal properties

Neurons in the early visual pathway have receptive fields that correspond to discrete retinal regions where visual stimuli influence their firing. The receptive fields of the neurons are topographically organized in maps that preserve the spatial relationships in the retinal image. At least to the level of visual cortical areas V1 and V2, we showed that receptive fields maintain fixed locations on the retina, and they move in space with the movements of the eye (Gur and Snodderly, 1987, 1997). Our conclusions were initially challenged (Motter, 1990), but later confirmed (Meirovithz et al., 2011) by cortical imaging experiments showing that neuronal responses in V1 move within the retinotopic map to track the location of the image on the retina, not the location of a stimulus in space. Thus the activity of V1 neurons carries ambiguous information about locations of objects in the environment (Bridgeman, 1999; Meirovithz et al., 2011) and it cannot account for space constancy and a stable environment. It is not until information reaches V4 (e.g. Tolias et al., 2001), or the parietal and frontal cortices, that receptive field locations on the retina shift with eye position (reviewed by Wurtz, 2008). In these later cortical regions, changes in receptive fields related to eye movements are thought to enhance processing of behaviorally relevant stimuli and to contribute to the perception of a stable environment.

Even though the receptive fields of V1 neurons are fixed on the retina, their responses are still affected by the position of the eye in the orbit. Eye position modulates the response gain of V1 neurons so that they give maximal responses when their receptive fields are in the straight-ahead position (Durand, Trotter, and Celebrini, 2010; Przybyszewski, Kagan, and Snodderly, 2014; see also Strappini et al., 2014). This modulation could be helpful in the navigation of cluttered environments, including detection of dark features like shadows or holes that might be hazardous. However, the magnitude of the modulation within the limited range of fixational eye movements is small, and it would have the most impact when a subject fixates eccentrically. Since most studies of fixational eye movements have been done while the subject fixates straight ahead, I will ignore the effects of eye position in the orbit for the rest of this paper.

3.1 Effects of fixational eye movements on measurements of response variability

Because receptive fields are fixed on the retina, repeated stimuli presented at the same location in the visual field are scattered about the retina (Snodderly et al, 1978) and the receptive field by the fixational eye movements. Receptive fields have spatial sensitivity profiles that are often modeled as Gaussian functions, so the response that is elicited is a function of where the stimulus lands on that profile. Typically, stimuli are presented multiple times and averaged to estimate the amplitude of the response and the geometry of the receptive field. By unpredictably displacing the receptive field relative to the stimulus, fixational eye movements increase the variability of the responses and introduce biases into measures of neuronal properties.

To reduce the uncontrolled effects of fixational eye movements, we monitored eye position during physiological recordings and added the eye position signal to the stimulus control signal to compensate for movements of the eye (Gur and Snodderly, 1987). Note that the position compensation that we have used is sometimes called “image stabilization”, but we use that terminology sparingly because readers often misinterpret it to mean that we are inducing image fading, which is not our purpose. The stimulus bar was repeatedly swept across the receptive field and response histograms were accumulated so that we could measure the maximum response and the receptive field profile. In these experiments on reliability, responses were only included if there was no saccade (of any size) in the period beginning 100 ms before the response and continuing for the duration of the response. This criterion was applied because the position compensation was too slow to correct for saccadic eye movements (time lags of 10–28 ms). This meant that all data were collected during drift periods of durations of ~ 133–350 ms.

As expected, responses of both LGN and V1 neurons were more reliable when the perturbations of fixational eye movements were minimized (Gur, Beylin, and Snodderly, 1997). A common measure of variability, the ratio of the variance to the mean spike count (the Fano Factor, FF) was among the lowest reported in the literature (Kara et al., 2000). We found that cells in the output as well as the input layers of the cortex responded with high reliability (Gur and Snodderly, 2006), indicating that the cortex did not degrade incoming signals as previously thought (Movshon, 2000). One factor that contributed to the high reliability was the brief, transient input provided by the sweeping bar moving at speeds within the range of eye movements. Other studies have shown that stimulus transients evoke more reliable responses than later parts of a prolonged stimulus (Muller, et al., 2001) and stimulus onsets reduce neural variability widely across the cortex (Churchland et al., 2010).

An important insight was that the variance of the responses did not increase proportionally with the mean in our experiments, as would be expected from a Poisson process. We found that the high variability that is often ascribed to cortical neurons (FF ≥ 1) was representative of responses near threshold, but when strong stimuli were presented, responses were very reliable (median FF~ 0.3). These results suggest that in natural vision, suprathreshold perception could draw upon reliable responses from cell groups that are being strongly activated by suprathreshold contrasts and dynamic stimulation provided in part by eye movements.

3.2 Influences of fixational eye movements on measurements of receptive field geometry and response magnitude

It is intuitively clear that the spatial scatter of fixational eye movements is incorporated into the stimulus positions on the retina. By comparing measurements done while compensating for the eye movements with those done without compensation, the effect of eye movements on experimental measurements can be estimated (Tang et al. 2007). The receptive field width was estimated by fitting a Gaussian curve to the histogram created by sweeping a narrow bar across the receptive field, with or without compensation. The width of the field was measured as the width of the Gaussian curve at 5% of the peak value. A non-parametric measurement from the raw histogram produced similar results. Fixational eye movements inflate the estimates of both LGN and V1 by about the same amount for receptive fields of all sizes (Fig. 2). With well-trained monkeys maintaining fixation within ± 60 min of a fixation point, the magnitude of the effect was between 3 and 7 minarc for receptive fields between 3 and 10° eccentricity. This seemingly minor inflation, in fact is large compared to behavioral measures of spatial resolution of humans and macaques (DeValois, Morgan, and Snodderly, 1974) and if uncorrected, it obscures the spatial dimensions of the small receptive fields that underlie our high spatial acuity.

Figure 2.

Population comparisons of receptive field width with and without compensation for fixational eye movements. Left panel shows lateral geniculate data and right shows cortical data, (distinguishing the two monkeys by color). Black lines are lines of equality. A, B: Uncompensated receptive field width is plotted against compensated width. Each point represents data from a single cell. Solid markers show estimates from a relatively non-parametric method, and open circles show widths derived from fitting Gaussians to histograms. (From Tang et al., 2007).

The same fixational movements that inflate measures of receptive field size cause an underestimation of the response amplitude. On repeated trials the stimulus crosses the receptive field at different times, and when the average is computed, the smeared profile of the receptive field is not only wider, but the peak response (amplitude) is reduced (Fig. 3). Under the conditions described in the preceding paragraph, we found about a 20% lower amplitude for the uncompensated responses. This reduction in the estimate of the response magnitude is a contributor to inflated measures of neuronal variability as described earlier.

Figure 3.

A, B: Population comparisons for the maximum response amplitude in uncompensated vs. compensated conditions. Solid line is the line of equality. C, D: Uncompensated vs. compensated maximum slopes of the receptive field profiles. Other conventions same as Fig. 2. (From Tang et al., 2007).

We also measured the slope of the receptive field profile as a parameter related to the sensitivity of the cell to small changes in stimulus position. The region of maximum slope was 30–60% steeper when compensation was used. This means that the sensitivity of neurons to small changes in stimulus position is underestimated when only uncompensated values are used. Of course, our values are also underestimates. For these experiments, saccades were not excluded and there were small compensation errors due to the time lags in the system in addition to 2–3 minarc position errors. For a full appreciation of the most refined receptive field parameters, even more accurate methods of eye position measurement and compensation will be needed.

Finally, we recognize that in natural situations, neurons in the early visual pathway will not compensate for eye movements and there will be head and body movements to cope with as well. However, the brain has access to all neurons that are being stimulated and to relationships between neurons. Position compensation is a tool to help experimenters determine the true capabilities of visual neurons, so that we understand the raw materials the brain has to work with while it is interacting with the environment.

4. Parallel motion pathways revealed by compensating for fixational eye movements

Electrophysiological recordings from the visual pathway are subject to multiple sampling biases. For example, it is easier to record large spikes fired by large cells, but often harder to identify the best stimulus to excite the cell (Gur, Beylin, and Snodderly, 1999). These are just two of the factors that can interact with each other when collecting the sample of cells included in an analysis. An unappreciated source of bias is the unpredictable motions and positions of the retinal stimulus imparted by the eye movements. Fixational eye movements can impart different directions of motion at various times and place the stimulus in excitatory zones at some times and inhibitory zones at other times. A particularly challenging case is a V1 direction-selective cell with a small, tightly oriented receptive field and a powerful inhibitory surround. For the experimenter, unpredictable eye movements introduce confusing directions of motion and location that can make it impossible to collect reliable data from such a cell, thus establishing a sampling bias. However, by compensating for fixational eye movements we have characterized these demanding direction-selective cells and found them to have the smallest receptive fields in V1 (Gur, Kagan, and Snodderly, 2005; Gur and Snodderly, 2008). Furthermore, they comprise about half the cells in our sample from layer 3, which provides major inputs to the ventral cortical stream for object recognition and discrimination (Gur and Snodderly, 2007). These results and related ones have enabled us to identify 3 parallel pathways carrying motion information at different spatial scales in the dorsal and ventral cortical streams (Fig 4). The pathway into the ventral stream is at the finest spatial scale and is especially suited to sensing small motions, including those caused by fixational eye movements.

Figure 4.

Motion-selective pathways in V1 and their relationships to the dorsal and ventral cortical streams of information processing. The dorsal and ventral streams are indicated on a drawing of a lateral view of a monkey brain. Rectangular colored boxes represent median values of physiological characteristics of direction selective cells in individual V1 layers. Cells were considered direction selective if they fired twice as many spikes in the preferred as in the nonpreferred direction. Box width is proportional to measured receptive field width. Box height is proportional to the percentage of cells responsive to a bar longer than 60 min. The symbol for included angle is proportional to the measured half bandwidth of the orientation tuning curve. Layer 4Cα feeds layer 4B and layer 6 in the pathway leading to the dorsal stream (magenta). Layers 4B and 6 send outputs from V1 through area MT to the parietal areas (magenta arrow) responsible for sensing object motion and location, including relations in depth. Layer 4Cα also feeds layers 4Cm and layer 3 in a separate direct pathway to V2, which projects to the ventral stream (cyan) responsible for object recognition, including faces. The small dimensions of layer 3 receptive fields (narrow widths as well as preferences for very short stimuli) are well suited to sense subtle motions within objects such as changes in facial expressions. (From Gur and Snodderly, 2007).

The realization that motion processing is an integral part of the ventral stream is important because it reduces the scope of what is known as the “binding problem” in vision (Ibbotson, 2007). Instead of assigning motion processing to a dorsal processing stream, to be combined later with shape processing in the ventral stream (e.g. Oram & Perret, 1996), our evidence indicates that motion processing is already built into the ventral stream. Motion is such a powerful visual cue, that the ventral stream can exploit it for multiple object-related tasks. For a primate, these tasks can be as diverse as the detection of motion of small camouflaged insect prey, or elevation of the eyebrows within a face. How fixational eye movements contribute to the neural coding of motion and of fine visual detail that feed the ventral stream is the subject of the next section.

5. Neuronal activation by fixational eye movements

If fixational eye movements were merely a hindrance to studying visual neurons, their physiological effects would not be very interesting. However, the saccades and drifts that occur during fixation selectively activate visual neurons that have quite different spatiotemporal characteristics. When the head is stabilized, the interactions between the intrinsic properties of neurons in the early visual pathway and the motions and displacements of the retinal image caused by eye movements determine the inputs that the rest of the visual system has to work with. Of course, this is a simplification of a natural situation, where additional motions such as head and body movements occur, but it allows us to precisely control and describe the retinal input, and it is a good place to start.

Each neuron is activated by stimulation of a restricted retinal region that has come to be called the “classical receptive field” or CRF, which can be composed of multiple activating regions (ARs). The receptive field also often has a suppressive surround that extends beyond the activating regions and is considered the “nonclassical receptive field”. We have only studied the effects of eye movements as they relate to the CRF.

When a stationary object is present in the visual field, eye movements produce a variety of interactions between the retinal image of the object and the CRF of a visual neuron. The image can move onto the CRF, off of the CRF, within the CRF, or across the CRF without ever being on the CRF. We refer to the saccades that cause these motions respectively as “landing”, “leaving”, “within”, or “crossing” saccades. For area V1, we have studied the effects of these motions while monkeys viewed a stationary bar of optimal orientation, color, and width, placed at the mean position of the CRF during maintained fixation. The CRF was mapped in eye position coordinates by a reverse correlation method so that we knew its relationship to the stimulus in real time (Snodderly, Kagan, and Gur, 2001). For a better appreciation of the dynamics of these interactions, I recommend that the reader view the movies available as supplemental material with the paper by Kagan, Gur, and Snodderly (2008).

5.1 Position/drift-activated cells

We identified three types of activation by fixational eye movements, which will be illustrated in the next three figures. The separation of the three types was based on a cluster analysis comparing the mean firing rate after a landing saccade in the period 40–250 ms immediately after the saccade to the activity later in the drift period 250–500 ms after the saccade (Snodderly, Kagan, and Gur, 2001). One neuronal type we called position/drift activated (position/drift cells for short). These cells discharge for the entire time that the CRF remains on the stimulus and stop responding as soon as the CRF leaves the stimulus (Fig. 5). We call them by a combined name because we cannot separate the effects of being positioned on the CRF from the effects of small drifts within the CRF. In principle the effects could be separated if we could compensate completely for the drifts, but it would require extreme precision to be sure that residual motion from small errors in compensation was eliminated. Regardless, it is clear that this type of cell will discharge primarily in the drift periods between saccades, but only when the CRF is on the stimulus. Position/drift cells also give prolonged responses to flashed stimuli that are positioned on the CRF. Flashed stimuli were presented in intersaccadic periods while compensating for drift movements within the limits of our system (~2–3 minarc).

Figure 5.

Characteristics of a position/drift cell. (A) Activation of the cell by a green incremental stationary bar 6 min wide of optimal orientation, placed at the mean receptive field position. Four separate time segments (I–IV) from different trials are shown. (B) Saccade-triggered histograms of average spike frequency. Separate histograms are presented for saccades followed by increased activity (increase, n=18), saccades followed by decreased activity (decrease, n=12), and all saccades combined (both, n=30). Saccades occurred at t=0. Also shown is the average response histogram of the cell to flashing the bar (n=25, stimulus on 100 ms, off 250 ms for each repetition; temporal profile indicated under time axis). (C) Gray-scale map of CRF in eye position coordinates, along with elliptical outline used in subsequent panels. Stimulus bar used to generate the map is drawn to the same spatial scale. Minor axis of ellipse is width of cell’s CRF measured with a stabilized bar sweeping across its receptive field. (D) Spatial interactions between stimulus and CRF during the trial segments in panel A. The eye position data shown as a function of time in panel A are plotted here in space, along with the stimulus bar that evoked the activity. I—Saccade brings the CRF onto the stimulus. II—Saccade takes CRF off the stimulus. III—CRF moves onto the stimulus (first saccade), stays on the stimulus briefly, and then moves off it (second saccade). IV—Pair of saccades sweeping CRF rapidly (50 deg/s) across the stimulus and back. (From Snodderly et al., 2001).

5.2 Saccade-activated cells

A striking contrast is provided by the group of cells that responds to saccadic displacements but does not continue to discharge during the drift periods, even when the CRF remains on the stimulus (Fig. 6). These cells respond to every type of saccadic movement, whether it moves the stimulus onto, off, across, or within the CRF. Under all conditions they respond in a very transient manner and they give similarly transient responses to stationary flashed stimuli. Many of them are direction selective, and they respond when saccades move the image in their preferred direction of motion (cf. Bair and O’Keefe, 1998).

Figure 6.

Characteristics of a saccade cell with a transient discharge. Conventions are the same as in Fig. 5. (A) Activation of the cell by an optimally oriented, red incremental stationary bar 4 min wide, placed at the mean receptive-field position. Four separate time segments (I–IV) from different trials are shown. (B) Saccade-triggered averaged spike histograms for all saccades followed by increased activity (n = 110) compared with responses of the cell to a stationary flashing bar (n = 75; stimulus on 100 ms, off 200 ms for each repetition). (C) Gray-scale map of CRF in eye position coordinates, along with elliptical outline used in subsequent panels. Stimulus bar used to generate the map is drawn to the same spatial scale. Minor axis of ellipse is width of cell’s CRF measured with a stabilized bar sweeping across its receptive field. (D) Spatial interactions between stimulus and CRF during the trial segments in panel A. The eye position data shown as a function of time in panel A are plotted here in space, along with the stimulus bar that evoked the activity. I—Saccade takes CRF off the stimulus. II—Effect of directional selectivity for saccade-induced motion: Saccade c–d takes CRF off the stimulus and saccade d–e puts CRF back onto the stimulus. Preferred direction for external stimulus motion is shown. Saccade d–e generates image motion in the null direction and elicits no activity. III—Saccade putting CRF onto stimulus with preferred direction of image motion (opposite to eye motion) produces sharp burst of spikes. IV—Small amplitude (about 10 minarc) saccade moves stimulus within CRF to generate discharges. Positions of stationary bar in segments I and II differ from that of panels III and IV. (From Snodderly et al., 2001).

5.3 Mixed cells

Finally, many cells fall between the two extremes and they respond to all saccades, but they also discharge above the baseline rate during the drift periods as long as the CRF remains on the stimulus (Fig. 7). We call these mixed cells because they have a mixture of properties. Their response to a stationary flashed stimulus is quite similar to the response to a landing saccade.

5.4. Comparisons among activation types

To help summarize this diverse set of properties, we constructed saccade-triggered average response histograms for a set of 28 cells that were studied in the detailed manner described above (Fig. 8). All saccade-triggered averages were aligned to saccade onset. From these histograms, important differences between the cells can be readily appreciated. First, the position/drift cells give an unambiguous signal—their CRF is on the stimulus during the drift period. Furthermore, they are often sensitive to the sign of contrast and to small stimuli, suggesting that they could signal details of the stimulus. Second, the majority of the cells (2 of 3 classes) have a sustained discharge in the drift period between saccades. Finally, the saccade-related activity is profoundly ambiguous. The cells give very similar saccade-related discharges whether the CRF lands on the stimulus, leaves it, or crosses it. It is difficult to imagine that details of the image could be reconstructed from such ambiguous inputs. The saccade-related discharges must play other roles.

Figure 8.

Saccade-triggered average spike histograms for position/drift activated cells (pos, n =10), mixed cells (mix, n=11), and saccade cells (sac, n=7) when saccades caused the CRF to land on the stimulus (first column), leave the stimulus (second column), or cross the stimulus without remaining on it (third column). Saccade onset occurred at t=0. For each cell, data were based on five saccades causing the CRF to land on the stimulus and five saccades causing the CRF to leave the stimulus. Because the monkeys maintained gaze within a compact region, saccades sweeping the CRF across the stimulus without remaining on it were less frequent. Four or five crossing saccades were analyzed for 15 cells, three saccades for four cells, and two saccades for nine cells. (From Snodderly et al., 2001).

6. Relative strength of drift-related activity. (For clarity, section 6 is adapted from Kagan et al., 2008)

The interpretation of activations by eye movements is most straightforward when the saccadic displacements cause the CRF unambiguously to land on the stimulus, leave it, or cross it (Snodderly et al. 2001). However, when studying fixational eye movements, their unpredictability, the idiosyncratic differences among animals, and the range of CRF sizes, make it difficult to obtain a complete set of cleanly separated landing, leaving, and crossing interactions for each cell. For example, with larger CRF sizes, few of the small fixational saccades cause the CRF to cross cleanly over the stimulus without either landing on it or leaving it. To overcome these difficulties, we have employed two approaches (Kagan, Gur, and Snodderly, 2008). First, we analyzed data from all fixational saccades that caused an increase in firing in the 250 ms following a saccade. Based on analyses of the 28 cells described above, we found that including all such “increasing” saccades yielded results that differed only slightly from results based on precisely mapped landing saccades. This allowed us to expand our analysis of fixational eye movements to 118 cells. Second, we utilized visually-guided voluntary saccades to generate distinct landing and leaving as well as crossing trajectories (44 cells). In 33 of 44 cells data were collected for both fixational and voluntary saccades.

To sort the cells into groups, we calculated an index designed to quantify the relative strength of post-saccadic and inter-saccadic firing. A normalized saccade-drift difference (SDD) was calculated using saccades separated from the nearest saccade by at least 250 ms. For fixational eye movements, the index was calculated using data from “increasing” saccades as defined above (62% of all fixational saccades); for voluntary saccades the index was calculated using landing saccades. Increasing fixational saccades comprise most saccades in saccade-activated and mixed classes; they are primarily landing saccades and some within saccades in the position/drift class. The saccade-drift difference compared the mean firing rate in the periods 0–150 ms immediately after the saccade (FR_sac) to the firing rate (FR_drift) during drift periods from 250 ms after the saccade to the next saccade - therefore drift periods had variable duration (829±570 ms fixational, 691±255 ms voluntary). Firing rates were corrected for the ongoing firing rate (FR_base), which was measured with a lighted blank screen of 1 or 5 cd/m².

$$\mathit{SDD}=\frac{({FR}_{\mathit{sac}}-{FR}_{\mathit{base}})-({FR}_{\mathit{drift}}-{FR}_{\mathit{base}})}{({FR}_{\mathit{sac}}-{FR}_{\mathit{base}})+({FR}_{\mathit{drift}}-{FR}_{\mathit{base}})}=\frac{{FR}_{\mathit{sac}}-{FR}_{\mathit{drift}}}{{FR}_{\mathit{sac}}+{FR}_{\mathit{drift}}-2{FR}_{\mathit{base}}}$$

Saccade-activated cells, with strong post-saccadic bursts and little or no discharge in the drift periods, had high values of SDD (>0.7, 25% of our sample); position/drift-activated cells, with comparable burst firing rates and drift firing rates had low values of SDD (<0.3, 38%); and mixed cells (37%) had intermediate values. These boundaries are consistent with those derived from the cells with detailed mapping, but some transitional cases occur. Nevertheless, differences in the SDD distinguish cells ranging over a wide continuum, and cells at the extremes have very different properties. For example, 95% of position/drift cells defined in this way did not respond to crossing saccades, whereas all saccade cells did respond to clear crossing saccades.

The average firing patterns evoked by increasing fixational saccades and voluntary landing saccades are very similar for the three types of cells (Fig. 9A). Although they have usually been ignored, the drift responses of most V1 neurons are substantial. To illustrate this point, we calculated the ratio of the mean drift firing rate to the mean post-saccadic firing rate (using same time periods as in SDD computation) and expressed it as a percentage. Fig. 9B shows the cumulative distribution of cells with drift firing rates achieving specific percentages of the post-saccade firing rate, both for fixational saccades that caused an increase in firing, and for voluntary landing saccades. About ¾ of the cells had at least 25% as high a mean firing rate in the drift period as in the post-saccade period, and nearly half the cells had mean firing rates at least 50% as high in the drift period as in the post-saccade period. Comparing different activation types defined by the SDD values, saccade cells fired less than 17% as fast in the drift period as in the post-saccade period (mean±s.d. 4±11%), mixed cells fired up to 50% as fast in the drift period (38±18%), and position/drift cells fired more than 50% as many spikes in the drift period as in the post-saccade period (80±24%). Importantly, the mean post-saccadic firing rate of position/drift cells (44±30 spikes/s) was as high as that of saccade cells (43±28 spikes/s), so the similarity between post-saccadic and inter-saccadic firing of the position/drift-activated population was not due to low post-saccadic responses.

Figure 9.

(A) Population perisaccadic averages triggered by fixational increasing saccades for 118 cells (30 sac, 43 mix, 45 pos) and voluntary landing saccades for 44 cells (10 sac, 17 mix, 17 pos). Cells were assigned to eye movement activation classes according to the saccade-drift difference SDD as described in the text. Dashed vertical lines denote saccade onset. (B) Cumulative distribution of V1 cells as a function of the firing rate in the drift period relative to the firing rate in the post-saccadic burst period for fixational increasing saccades (open circles) and voluntary landing saccades (filled squares). Vertical dotted lines mark the borders between cell classes based on the SDD index. (From Kagan et al., 2008).

7. Comparisons of effects of eye movements and effects of externally imposed transients and motions

7.1 Flashes vs saccades

When a saccade moves the CRF onto a steady stimulus, there is an abrupt change in flux in the CRF. For comparison with this situation, we recorded responses to a stabilized flashed stimulus positioned on the CRF during a drift period. The flash was 150–250 ms in duration, and it began at least 200 ms after any fixational saccade. A measure of transiency was highly correlated between the two cases (r=0.85, p <0.00001), indicating that the abrupt spatiotemporal transients imparted by saccades affect neuronal activity in a manner very similar to the abrupt temporal transients of flashed stimuli. A slower saccade-related temporal modulation of extraretinal origin is considered in a later section of this paper.

Peak firing rates of responses to a flashed stimulus were very similar to the firing rates evoked by a steadily illuminated stimulus that was moved abruptly onto or across the receptive field by saccades. Fig. 10A illustrates peak firing rates for fixational increasing saccades, voluntary landing saccades and voluntary crossing saccades of different amplitudes, normalized by the firing rate for flashes for each cell before averaging. For the most similar conditions --flashes and voluntary landing saccades—peak firing rates were remarkably comparable for all three eye movement classes (gray bars). Activation by fixational saccades was nearly as effective as flashes, ranging from 60% to more than 90% of the flash response (purple bars).

Figure 10.

Comparison of peak firing rates in response to flashes and to saccades. Peak firing rate values are based on the mean of the three highest 10-ms bins in the 0–120 ms period of the initial transient of the response. (A) Peak firing rates for fixational increasing saccades and voluntary landing and crossing saccades were normalized by the flash response (horizontal dotted line). Voluntary crossing saccades were grouped as small (s, amplitude ≤1.5°), medium (m, 1.5° to 3°), and large (l, from 3° to 6°). Error bars denote standard error. (B) Fixational saccade- and flash-triggered population averages of spike probability for two monkeys. Spike probability was estimated in 1 ms time bin and averaged across 10 ms bin. Shaded regions represent ±s.e.m. of each time-course. Note that the apparent difference in “flash-to-saccade” ratios between (A) and (B) is explained by larger variations in the latency of the post-saccadic peak as compared to flash responses. This results in “smearing” the post-saccadic peaks in (B) but not in (A), where the post-saccadic firing rate was estimated for each cell individually using the three highest 10 ms bins rather than averaging with a fixed latency before normalizing and averaging across cells. (From Kagan et al., 2008).

We performed a separate analysis to compare our results to those in an earlier study by Martinez-Conde et al. (2002). They reported that activation of V1 neurons by fixational saccades was rather weak, being only about one-seventh as effective as a stimulus flash. They computed saccade-triggered or flash onset-triggered average spike probability for all saccades of 6 cells. Their analysis ignored the diversity of activation patterns that are possible. Fig. 10B shows our results from two monkeys when we adopted the same approach, combining data from all fixational saccades and all eye movement activation types. Activation by fixational saccades, after subtracting ongoing rates, was 39–42% the maximum response to flashes (thin black curves) and spike probability returned to baseline within 250 ms. When data were included only for saccades that caused increases in firing (thick black curve), there was an increase in the magnitude of the post-saccadic burst to 43–53% of the flash response, and spike probability stayed above the pre-saccadic baseline. These results illustrate how the maintained response was diluted when all saccades and all cell types were included.

Perhaps realizing that poor activation by small saccades was inconsistent with advocating for the importance of these eye movements, Martinez-Conde et al. (2009) later re-analyzed both their original data and our data. In an effort to show that their data were equivalent to ours, they calculated responses based on the ratio of the firing rate before the saccade (or the flash) and the firing rate after the saccade (or the flash). Because many cortical neurons have very low ongoing firing rates (Gur, Kagan and Snodderly, 2005), such a ratio can magnify even a tiny activation. This is not an accepted way to characterize neuronal responses.

Another concern is that the procedures of Martinez-Conde et al. did not clearly separate saccadic and flash activation. We compared the responses to a flash in the absence of saccades with the responses to saccades with a steadily illuminated stimulus that did not flash. They compared the responses to a flash with activity evoked by saccades that occurred during the flash. I predict that if the authors identify saccades that place the CRF on a steadily illuminated stimulus and compare that saccadic activation with the response to a flashed stimulus carefully placed on the CRF at a time that does not collide with saccadic influences, they will arrive at a more favorable comparison between saccadic and flash activation.

7.2 Smoothly moving stimuli and speed selectivity

For comparison with activation by eye movements, we studied the responses of V1 neurons to smoothly moving stimuli at different speeds to produce a range of retinal image speeds. The activation by fast saccades and slower drifts closely parallels the responses to abrupt flashes and smoothly moving external stimuli as follows: Saccade cells have transient, short-latency responses to flashes and they prefer relatively high speeds. Position/drift cells have a sustained, longer-latency flash response and they prefer slow speeds. Mixed cells have intermediate flash response properties and speed preferences. In general, cells that preferred faster movement had more transient discharges and were also more likely to be selective for direction of movement. For numerical data establishing these relationships, see Kagan et al, (2008).

One indicator of the effects of retinal image velocity is the activation of neurons when saccades of different sizes sweep the CRF across a stimulus. A robust outcome of our experiments was that all cell types responded more vigorously to small than to large crossing saccades (dark blue bars in Fig. 10A). This result is consistent with early experiments by Judge et al. (1980), who showed that V1 neurons respond only weakly to the fast retinal image speeds produced by large (20°) saccades. For the smaller saccades that we have used, saccade-activated cells responded fairly well at amplitudes up to 5° (the largest size tested), mixed cells’ response was considerably weaker, and most position/drift cells did not respond at all to crossing saccades of any size. In fact, only 2/17 position/drift cells responded weakly to crossing saccades. In summary, for saccade and mixed cells, small saccades provide potent inputs to the visual system by the retinal image speeds that they introduce. For the position/drift cells, the intersaccadic drifts provide the preferred retinal image speeds.

8. Spatial selectivity of eye movement types

With the smoothly moving stimuli, we were also able to measure size and contrast selectivity of the spatial CRF of the neurons (Kagan et al., 2002). For this analysis, neurons in the parafovea were studied (eccentricity <7°), and saccade and mixed cells were combined into another group. We found a smaller CRF and greater specificity for sign of contrast for the position/drift cells (Kagan et al., 2008). Specifically, position/drift cells had mean activating regions with a width of 24±17′, vs. 31±23′ for the others (p<0.05), and a higher frequency 35% of very small (<15′) CRFs, compared to 18% for the others (p<0.05, Fisher’s exact test). Similarly, the relative numbers of position/drift cells responding to only one sign of contrast at each location in the CRF, i.e. simple and monocontrast neurons, was 40%, but only 18% for the others (Fisher’s exact test, p<0.02). These data suggest that position/drift cells, with their finer spatial resolution and greater specificity for sign of contrast, are especially well suited to encode precise stimulus position as well as localized contrast to convey fine spatial detail.

9. Extraretinal modulation associated with saccades

Until now, I have emphasized the influence of fixational eye movements on visual inputs while viewing patterned stimuli. However, fixational eye movements are motor acts that are accompanied by an extraretinal modulation of neuronal activity even without deliberate stimulation of the CRF. To study these effects we have recorded the ongoing activity of V1 neurons with extrafoveal CRFs while monkeys fixated a small spot of light in darkness or on a uniform gray background of 1 or 5 cd/m². For a subset of cells, voluntary saccades were elicited by stepping the fixation point abruptly to a new location.

The perisaccadic modulation of the ongoing firing typically had a biphasic time course – initial weak suppression followed by stronger enhancement peaking 100 to 200 ms after saccade onset. For each neuron, we considered the modulation to be significant if it exceeded ± 2.5 SD of the baseline. Approximately one-third of our sample of cells had significant extraretinal modulation. Fig. 11 plots the average time course of the extraretinal modulation by fixational and voluntary saccades in the light and in the dark for all cells in two monkeys for which there was a significant effect.

Figure 11.

Extraretinal modulation of neuronal firing associated with saccades. Population saccade-triggered averages of extraretinal modulation by fixational and voluntary saccades for cells with a statistically significant modulation. Averages for fixational eye movements in the dark are based on 26 cells, fixational eye movements in the light on 86 cells and voluntary eye movements on 17 cells. The ongoing firing rate has been subtracted. For fixational saccades, only saccades of ≤100 minarc amplitude were included. To avoid overlap of effects from adjacent saccades, we only analyzed saccades that were not preceded by another saccade in a period at least 300 ms before saccade onset and were not followed by another saccade for at least 400 ms after the saccade onset. There was a statistically significant difference between the latency of the post-saccadic enhancement in the dark (170 ± 31 ms, median 162.5 ms) and in the light (142 ± 34 ms, median 137.5 ms) for M45 (p<0.05, one-way ANOVA for multiple comparisons), but no difference between those values and latency in the light for M46 (153 ± 50 ms, 137.5 ms). (From Kagan et al., 2008).

The extraretinal modulation was remarkably uniform across types of cells and types of saccades. There was no significant relationship between strength of extraretinal modulation and receptive field class (simple, complex or monocontrast; Kagan et al. 2002), the type of eye movement activation (saccade, position/drift, mixed), or the size of the saccade.

9.1 Differences between extraretinal modulations and stimulus-driven responses

Figure 12 illustrates how the slow biphasic time course of the extraretinal influence differs from the responses evoked by near-optimal visual stimuli. Fig. 12A compares an averaged perisaccadic histogram for fixational saccades that caused increased firing in the presence of a steadily illuminated stimulus with averaged responses from the same cells with no stimulus present. Fig. 12B presents the corresponding comparison for voluntary landing saccades. The stimulus-evoked response in both situations is larger and faster than the extraretinal modulation. However, the stimuli were chosen to be near-optimal for the cells, and in a natural environment, many cells will encounter sub-optimal stimuli that will not be so effective. Moreover, the strength of the extraretinal modulation is activity-dependent; its strength depends on the current level of firing, being stronger when the ongoing firing rate is higher (Kagan, Gur, and Snodderly, 2008). Consequently, in a natural situation the relative strengths of the stimulus-driven activity and the extraretinal modulation may covary in intricate ways that will be very interesting to study.

Figure 12.

(A–B) Comparison of stimulus-driven and extraretinal influences. Population saccade-triggered averages for fixational increasing (A) and voluntary landing (B) saccades with and without a stimulus. The ongoing rate has been subtracted. For this figure, extraretinal influences measured in the light and in the dark have been combined. Bin width 10 ms, convolved with Gaussian of σ 15 ms. Shaded regions represent ±SEM. (A) n=27 significantly modulated cells out of 73 tested with and without stimulus; 25 in light, 10 in dark. (B) n=15/22 cells, 14 light, 3 dark. (C–D) Comparison of time-courses of responses to prolonged flashes of an optimal stationary bar (flash duration at least 250 ms) and responses to voluntary landing saccades. (C) Mean post-saccadic and post-flash activity for a subset of 13 transient (sac and mix) neurons with TI_flash >0.6. (D) Mean differential (post-saccadic – post-flash) activity for the same 13 neurons. The differential activity was calculated separately for each neuron and then averaged across neurons. Note the similarity of the differential time-course to the time-course of ongoing activity modulation (cf. Figure 11). (From Kagan et al., 2008).

We tested whether extraretinal influences play a role in shaping the saccade-evoked activity even when a strong visual stimulus is present. We looked specifically for differences in the time-course of transient responses to voluntary landing saccades and to flashes. To characterize the immediate time-course of post-saccadic modulation, we used a transiency index, (TI) that is similar to the SDD index, but computed for firing rates in shorter periods of 0–120 ms after the saccade (response peak), and 120–250 ms after the saccade (tail of the post-saccadic response):

$$TI=\frac{{FR}_{\mathit{peak}}-{FR}_{\mathit{tail}}}{{FR}_{\mathit{peak}}+{FR}_{\mathit{tail}}-2{FR}_{\mathit{base}}}$$

The peak was calculated as the mean of the 3 highest 10-ms bins in the 0–120 ms period, to accommodate variations in response latency. These peak estimates were also used for analysis of peak firing rates in Fig. 10.

Fig. 12C shows that in transient cells with TI_flash >0.6 the response to the landing eye movement lasts longer than the response to a flash (TI_flash 0.91±0.13, TI_land 0.75±0.18; n=13, p <0.05), which suggests a post-saccadic enhancement by the extraretinal modulation. Also, the difference between time-courses of flash and saccadic responses, plotted in Fig. 12D, was similar to the form of the biphasic extraretinal modulation of ongoing activity (cf. Fig. 11). These modulations in neuronal activity associated with saccadic eye movements are candidate mechanisms for contributing to changes in stimulus visibility associated with saccades.

9.2 Sources of extraretinal modulation

There are probably multiple sources of the extraretinal modulation in V1. At least part of the modulation is already present in the LGN inputs. Fig. 13A compares the time course of extraretinal modulation in V1 with extraretinal modulation measured in LGN by 3 groups. Unlike other groups, Martinez-Conde et al. (2002) found no extraretinal effect in either LGN or V1. Extraretinal modulations are biphasic, with suppression followed by enhancement. The time course of the enhancement in V1 – both in the light and in the dark – is much slower than the time course in LGN studies that used full-field flash stimulation (Ramcharan et al. 2001; Reppas et al. 2002), and cannot be explained by latency differences in visual responses (~25 ms LGN, ~50 ms V1). Another LGN study using a dark background reported slower enhancement, similar to our V1 data (Royal et al. 2006). The difference between LGN visual stimulation regimes (bright full-field flash vs. dark background) may have contributed to the differences between these reports. Some of the long-latency activation in V1 may be contributed by descending inputs from higher-order cortical areas (cf. Toyama et al. 1984).

Figure 13.

Comparison of extraretinal perisaccadic modulation at different levels of visual processing. Saccade onset-triggered averages with a uniform lighted background in each case. (A) Modulation of neuronal firing by saccades in the presence of a blank field. Gray curve, V1 cells, our data, differential firing rate associated with fixational saccades, mean values from 2 monkeys. Black curve, differential firing rate associated with voluntary saccades. Dashed orange line, monkey LGN normalized firing rate (data from Reppas et al. 2002). Solid yellow line, monkey LGN normalized firing rate (data extracted from Figure 9A of Royal et al. 2006 using WinDig software). All neuronal data are for cells showing significant modulation. (B) Comparison of our monkey V1 data with extraretinal modulation of human perceptual thresholds. Black curve, V1 extraretinal modulation associated with voluntary saccades copied from A. Dashed blue line, human psychophysical data: differences between log contrast sensitivity at times relative to a real saccade and relative to simulated saccadic motion for a flashed grating (2 human subjects averaged, data from Diamond et al. 2000). The human data have been shifted rightward by 50 ms to account for visual processing delay. (From Kagan et al., 2008).

9.3 Suppression vs enhancement

Analogous modulations of human perceptual sensitivity associated with saccades have been reviewed by Ross et al. (2001), and the most pertinent example is reproduced in Fig. 13B. These data show human contrast sensitivity for a flashed grating presented at various times relative to the onset of a large voluntary saccade compared to fluctuations in sensitivity evoked by externally imposed motion that simulated the retinal image motion produced by the saccade (Diamond et al. 2000). The difference between those two situations should represent an extraretinal influence and it is plotted here (dotted blue curve). However, since our neurophysiological data represent a modulation of sensitivity at the cortex, and the psychophysical data represent a modulation of sensitivity relative to the time that a stimulus falls on the retina, an approximate retina-to-cortex processing time of 50 ms has been added to the psychophysical data. A qualitative correspondence between the two types of data is apparent, including some enhancement. One cannot conclude from this comparison whether the extraretinal modulations in V1 are major determinants of the perceptual effect, but they probably play a role, and they must affect the activity of downstream cortical areas that participate in perceptual decisions. Ibbotson and Krekelberg (2011) summarize perisaccadic response functions from a large number of cortical and subcortical sites that show a similar biphasic shape and time course.

As I noted earlier, extraretinal modulation is activity-dependent and its magnitude varies with the ongoing rate of the cell when the saccade occurs. An obvious example is that in our experiments with extracellular recording, the early suppressive phase is truncated because the firing rate cannot go below zero. For comparison, Leopold and Logothetis (1998) used continuously viewed grating stimuli that produced high sustained firing of V1 cells, and analyzed the firing patterns after very small saccades (10′ median) that caused relatively little change in light flux within the receptive field. Under these conditions, the responses show a stronger suppressive phase with a time-course very similar to our extraretinal effect. Their experiment could be interpreted to mean that microsaccades have a predominantly inhibitory effect in V1, but it seems more appropriate to interpret it as revealing an extraretinal influence in a situation where the stimulus is elevating activity but not modulating it. Under more typical conditions with lower ongoing activity, the biphasic modulation seen in Figs. 11 and 13 is more likely. This interpretation is consistent with microsaccadic modulations in V1 recorded by Hass and Horwitz (2011) during white noise stimulation. Their Fig. 8B shows an early suppression and later enhancement very similar to our data with a lighted background in Fig. 11. The effect of the level of neural activity may have particular importance for understanding activity patterns during viewing of natural scenes, where neurons are expected to encounter non-optimal stimuli and exhibit low response rates (e.g. Vinje and Gallant, 2000). These different scenarios illustrate that--for a better understanding of active vision--we need to account for interactions between stimulus-driven and extraretinal effects, and not just consider them separately.

10. Effects of fixational eye movements on activity in cortical areas downstream from V1

10.1. Effects of drifts

There have been several studies of the effects of fixational eye movements on downstream cortical areas in the dorsal stream. Only one of them has examined the effects of fixational drifts. Recognizing the implications of the strong drift responses that we found in V1 neurons, Hohl and Lisberger (2011) studied responses of MT neurons to random dot stimuli during drift periods. They concluded that MT neurons were sensitive to fixational drifts and they signaled the direction of motion, but not the speed of the motion. This raises the question: Why are these motions not visible to us? They argued that the response was sufficiently variable that the stimulus would be interpreted as stationary, given a Bayesian framework with a prior of zero velocity. How such a Bayesian operation might be implemented by the brain remains an open question.

10.2 Effects of fixational saccades

Other studies of dorsal stream responses have examined the influence of fixational saccades (termed microsaccades) on neurons in multiple cortical areas. The results nicely illustrate the principle that the neuronal activity is an interaction between the stimulus-driven and the extraretinal effects. Bair and O’Keefe (1998) reported, as we found in V1, that MT neurons enhanced activity when microsaccades moved the retinal image in the preferred direction of the cell, and suppressed stimulus-evoked activity when the image flow was in the nonpreferred direction. However, saccade-related suppression was stronger when the stimulus-evoked firing rate was high and saccade-related enhancement was stronger when the stimulus-evoked firing rate was low. This relationship may help to explain the results of a later study by Herrington et al. (2009). They trained monkeys to detect transient or sustained changes in speed or strength of motion of random dot stimuli while they recorded from areas MT, LIP, and VIP. They chose test stimuli to match the recorded neuron’s preferred direction of motion so that the test stimulus was expected to evoke an increase in the neuron’s response. They found that microsaccades suppressed neural activity and inhibited behavioral detection of the motion stimulus by as much as 20%. It would be very interesting to know what would happen with a discrimination based on a decrease of activity—would microsaccades improve behavioral performance or introduce variability that would degrade performance?

In the ventral stream downstream from V1, there has been less study of the effects of fixational eye movements. Leopold and Logothetis reported that areas V2 and V4 both showed strongly enhanced activity following saccades, but there is still little understanding of the possible ways that stimulus-driven and saccade-related influences might interact.

11. Discussion and Conclusions

11.1. Fixational vs voluntary saccades

As the recognition of the ubiquitous influences of microsaccades grows, it is useful to ask how their effects compare to those of larger voluntary saccades. Our research shows, somewhat surprisingly, that their effects are nearly equivalent in most respects (Kagan et al, 2008). Voluntary and fixational landing saccades evoke responses of about the same magnitude and time course, and extraretinal effects are very similar. One of our findings was that extraretinal modulation of V1 activity does not vary with fixational saccade size, an outcome that was replicated by Hass and Horowitz (2011). A similar invariance of extraretinal modulation with larger (voluntary) saccade size has been reported for LGN cells (Reppas et al., 2002). The only clear difference in the effects of saccades of different sizes is the weak or absent activation by the larger saccades as they sweep the retinal image across receptive fields. The lack of response to larger saccades as they cross receptive fields appears to be a straightforward consequence of speed tuning of V1 cells, which prefer lower speeds (Judge et al., 1980; Kagan et al., 2008). All in all, these results show that the small saccades of fixation create all the visual inputs that large saccades do, and then some. The challenge has been to learn their perceptual effects.

11.2. Saccadic suppression

There is a rich legacy of perceptual research on the modulation of visual sensitivity and on the visual distortions occurring in the perisaccadic period (Volkmann, 1986; Matin, 1974; Wurtz, 2008; Ibbotson and Krekelberg, 2011). Although early experiments on microsaccades produced mixed results (e.g. Beeler, 1967; Krauskopf, et al., 1966), current evidence indicates that perturbations of visual inputs occur around microsaccades in much the same way that they do for larger voluntary saccades. Detection thresholds for both contrast and motion are elevated at the time of microsaccades (Hass and Horowitz, 2011; Herrington et al., 2009), and small perceptual distortions of space occur prior to microsaccades (Hafed, 2013).

When we make a saccadic eye movement, we are not aware of the blurred scene that passes over the retina between the beginning and the end of the saccade. This phenomenon is known as saccadic suppression (Matin, 1974). Part of the loss of visibility is due to the high velocity of the saccade as described above. However two other mechanisms are known to be involved. One is the extraretinal modulation that accompanies saccades, (Volkmann, 1986; Wurtz, 2008; Diamond et al. 2000). The other is suppression of the input during the saccade by visual masking. The masking can be forward masking by the pre-saccadic activity and/or backward masking by the post-saccadic activity. Both forward and backward masking have been demonstrated in V1 (Macknik and Livingstone, 1998). Wurtz (2008) has made a strong case that forward masking should have a major impact on saccade-related activity.

I propose that the sources of peri-saccadic masking are likely to be the “landing” and “leaving” transients of the saccade and mixed cells (Fig. 8). In a natural environment, all receptive fields are continuously leaving image elements and landing on new ones, generating two transients linked to every saccade. It is not generally appreciated how prominent the transients are when a receptive field leaves a stimulus, but this is precisely the activity that is needed to provide forward masking. Conversely the post-saccade “landing” transient would be appropriate to provide backward masking (Matin, 1974).

There are also losses of sensitivity for a variety of visual parameters in the peri-saccadic interval that might be considered examples of saccadic suppression. For example, in MT, VIP, and LIP, microsaccades inhibited neural activity and elevated behavioral thresholds for motion detection (Herrington et al. 2009). The suppression in these areas has the appearance of a simple inhibitory effect, perhaps driven by extraretinal inputs. On the other hand, in experiments where microsaccades elevated behavioral thresholds for contrast detection, extraretinal inputs in V1 cause a bimodal modulation of neural activity (Kagan et al., 2008; Hass and Horwitz, 2011). This bimodal modulation has weaker suppression than enhancement. When considering such detection experiments, it may be useful to avoid the term “suppression” because the underlying responses may be more variable around microsaccades rather than simply being suppressed. Herrington et al. (2009) argue against increased variability as an explanation for their experiments on cortical areas in the dorsal stream, but it is worth reconsideration in cortical areas of the ventral stream, where extraretinal effects may have a stronger enhancement component (Leopold and Logothetis, 1998).

11.3. Interactions upon landing at a new location

The inputs to the visual system consist of a temporal stream of events punctuated by saccades leaving one locus and jumping to another. Input from each locus presumably acts to mask input from the other (Matin, 1974; Wurtz, 2008; Macknik and Livingstone, 1998). The bimodal shape of the extraretinal modulation in V1 (suppression followed by enhancement) may be one mechanism for giving preference to information from the new locus. Our evidence suggests that the extraretinal modulation should enhance the strength of the neural activity at the new locus (recall Fig. 12).

Immediately after microsaccades there is a period of about 50 ms when the eye drifts opposite to the direction of the saccade at higher velocity than the drifts later in the post-saccadic period (Chen and Hafed, 2013). This period of enhanced drift is within the suppressive phase of the extraretinal modulation (recall Fig. 13), implying that there can be an interaction of these influences along with the others I have discussed. An increasingly intricate picture emerges as new information is added.

11.4. Enhancement of vision by fixational eye movements

11.4.1. The importance of drifts

An important positive effect of fixational eye movements is the enhanced perception of fine spatial detail. This enhancement is experienced by observers who make virtually no fixational saccades, so it is clearly due to fixational drifts (Rucci et al., 2007). Our work is unique in identifying the robust activity of V1 neurons during the drift periods of fixational eye movements (Snodderly et al., 2001; Kagan et al., 2008). Approximately two-thirds of our sample, the mixed cells and the position/drift cells, gave sustained discharges that continued as long as the stimulus remained on the CRF. We also showed that position/drift cells that are active only in the drift periods are more selective for small spatial features and for sign of contrast,. Their activity must be essential to the fine spatial vision of primates (DeValois et al., 1974).

One might ask why other labs have not observed high rates of activity in the drift periods. A critical requirement for eliciting the maximal sustained response is to place the stimulus accurately on the CRF. We satisfied this requirement by obtaining a precise measure of the CRF size and retinal location while compensating for fixational eye movements. Then we turned off the compensation and recorded the effects of the eye movements themselves. This sounds straightforward, but the accuracy and speed requirements for precise, gaze-contingent, stimulus control are more stringent than the requirements of many experiments and they are not easily met by commercially available hardware or software. Consequently most experimental setups do not have the necessary capabilities.

As we have emphasized (Kagan et al., 2008), for analysis it is important to separate the drift periods into ones following saccades that cause an increase in firing from those that do not. When no increase occurs, it means one of three things:

1. The saccade has moved the CRF off the stimulus entirely; or

2. The saccade has moved the CRF so that the stimulus falls on a less sensitive part of the field; or

3. The CRF has not encountered the stimulus at all.

If these conditions are included in the overall average, the true drift response is diluted (Fig. 10B) and it may be missed altogether.

Given the challenges of characterizing the activity during drift periods, Martinez-Conde et al., (2002) have suggested that the responses might be caused by undetected saccades. However, there is no technical reason to believe that saccades would be undetected (See discussion in Kagan et al., 2008, including Supplemental Methods). Furthermore the suggestion that undetected saccades might cause activity mistakenly assigned to drifts is incompatible with the fact that sustained drift responses are selectively associated with cells that prefer slower retinal image speeds.

11.4.2 Contributions of fixational saccades

How about fixational saccades? What benefits do they confer? In addition to compensating for drifts (Cherici et al., 2012) and positioning the fovea for best performance (Poletti et al., 2013) they have strong dynamic effects. First, they generate bursts of neuronal activity in the cortex whenever a contrast border moves on the retina. In the absence of head and body movements, they can counteract fading of low-contrast images in the periphery (Martinez-Conde et al., 2006). Recent evidence also indicates that abrupt, saccade-like motions of a border are perceptually more important and physiologically more effective than static lateral interactions in the absence of eye movements (Ennis et al., 2014). However, the possible contributions of slower, drift-like motions were not tested. More broadly, fixational saccades set up synchronous bursts that could contribute to detection of extended contours, binding the features of an object, and selectively activating neurons further along the visual pathway (Maldonado et al., 2008).

Highlights.

• Fixational saccades and drifts selectively activate different types of V1 neurons.

• Cells that prefer slower speeds respond during drifts and may code fine details.

• Compensating for eye movements revealed a motion pathway to the ventral stream.

• Fixational and voluntary saccades are accompanied by extraretinal modulations.