Eye Movements and the Neural Basis of Context Effects on Visual Sensitivity

Abstract

The effects of context on visual sensitivity are well established (e.g., sensitivity to luminance flicker is substantially higher on mean-gray surrounds than on white or black surrounds). The neural mechanisms generating context effects, however, remain unresolved. In the absence of direct tests, some theories invoke enhancement of edges by lateral inhibition, whereas others rely on transients caused by miniature eye movements that maintain fixation. We first replicated the luminance results on human observers and found unexpectedly that sensitivity to red-green flicker is also affected by surround color, being substantially higher on mean-gray surrounds than on red or green surrounds. To identify the neural bases of both context effects, we used in vivo electrophysiological recordings of primate magnocellular and parvocellular ganglion cell responses to luminance and red-green modulations, respectively. To test neuronal sensitivity to stationary edge contrast, neuronal responses were measured at various distances from the modulation edge against various surrounds. We found no evidence of enhanced responses to stationary edges on any surrounds, ruling out lateral inhibition-type explanations. To simulate the effects of eye movements, target patches were abruptly displaced while measuring responses. Abruptly displaced edges evoked vigorous transient responses that were selective for modulation-phase on mean-gray surrounds, but were phase-invariant on other surrounds. Eye movements could thus enhance detection of flicker on mean-gray surrounds, and neurometric analyses supported a primary role for eye movements in enhancing sensitivity. In addition, the transformation of spatial edges to transient neuronal responses by eye movements provides the signals for detecting luminance and color edges in natural scenes.

Introduction

The ability to detect changes in lights over time and space is fundamental to visual function. Temporal sensitivity is generally characterized by the contrast required to detect sinusoidal flicker for each frequency. The plot of sensitivity (1/threshold) versus frequency is called the temporal contrast sensitivity function (TCSF). This function has proven to be invaluable in quantifying the speed of information processing and in developing models of higher visual functions, including motion perception. Classic measurements of the TCSF, done with flickering targets set in a dark surround, showed that the luminance TCSF is bandpass with a peak ∼8 Hz (de Lange, 1958; Kelly, 1961). However, if the surround luminance is changed from dark to the mean-gray of the target modulation, sensitivity to low frequencies increases many-fold, making the TCSF low-pass (Kelly, 1961, 1971).

The cause of the improvement in performance with a mean-gray surround has remained unresolved. Kelly (1969, 1971, 1972, 1979) proposed that fixational eye movements introduce high-frequency transients into cellular responses, resulting in lower thresholds on mean-gray surrounds, but not on dark surrounds. The eye movement hypothesis was challenged by experiments with stabilized images that claimed no differences between stabilized and regular viewing conditions, and a lateral inhibitory model was proposed (Keesey, 1970). Using a variety of steady and modulated surrounds, Spehar and Zaidi (1997) showed that the surround effects were largely contrast-related. They suggested that a system sensitive to high-spatial, but low-temporal, frequencies detects the modulation but is saturated by a contrast-pedestal at the target edge. This edge-contrast based explanation has been used to account for a variety of perceptual results (Kremers et al., 2004; Shapiro et al., 2005), but the psychophysics results, obtained under fixation, do not rule out the influence of fixational eye movements. Even more fundamentally, whether the center-surround structure of retinal neurons enhances responses to stationary sharp edges has not been tested directly.

Miniature eye movements are an inherent part of maintaining fixation on a target. Recent impetus to reexamine the role of eye movements on spatial context effects comes from Ahissar and Arieli (2001) and Rucci and Casile (2005) who have suggested, on the basis of psychophysical and computational results, that spatial differentiation in perception is primarily derived from the temporal transients caused by eye movements. Cortical recordings have incorporated eye movements (Leopold and Logothetis, 1998; Martinez-Conde et al., 2000, 2004; Snodderly et al., 2001; Kagan et al., 2008), but the physiological substrates for these suggestions have not been directly identified.

We combined human behavioral measurements and electrophysiological recordings from primate retinal ganglion cells (RGCs) to resolve the neural basis of surround effects on flicker perception. We show that the transient neuronal responses that are caused by miniature eye movements dragging receptive fields (RFs) across stimulus edges provide information about flicker, but only on surrounds on which humans show enhanced sensitivity. We also show that the RF structure of ganglion cells does not enhance responses to stationary edges on any surround, so eye movement-evoked transient responses also provide the neuronal substrate for detecting chromatic and luminance edges in natural scenes.

Materials and Methods

We used a combination of human psychophysics and primate electrophysiology to test whether retinal lateral inhibition, reflected in the center-surround RF structure of ganglion cells, can in any way provide a plausible explanation for surround effects on the shape of the TCSF or whether a role for eye movements is essential.

We first replicated the psychophysical results in the luminance domain and then tested for surround effects in the chromatic domain. Whereas the temporal responses of magnocellular (MC) cells are bandpass, the sensitivity of parvocellular (PC) retinal ganglion cells for red-green chromatic modulation is low-pass (Lee et al., 1990, 2007). Hence, an interesting variation on context effects could be provided by detection of equiluminant red-green chromatic flicker. When measured on neutral backgrounds, the chromatic TCSF is low-pass in shape (i.e., there is no decrease in sensitivity at low frequencies) (van der Horst, 1969; Swanson et al., 1987). However, the equivalent comparison for the chromatic TCSF would be between a mean-gray surround and equiluminant red or green surrounds, and that has not been measured. The presence of chromatic context effects would provide a new perspective for the lateral inhibition versus eye movement debate.

Because central mechanisms are constrained by the signals received from the retina, we recorded in vivo responses of primate retinal ganglion cells. To test neuronal sensitivity to edge contrast, we recorded cell responses at various distances from the modulation edge. To simulate the effects of fixational eye movements (Ratliff and Riggs, 1950; Ditchburn, 1973, Martinez-Conde and Macknik, 2008), we recorded cell responses to abruptly displaced target patches. To compare luminance and chromatic cases, we recorded responses of MC retinal ganglion cells to modulation of achromatic luminance, and responses of PC cells to modulation of equiluminant red-green. Effects of edge contrast on ganglion cell responses were found to be minimal with stationary targets but substantial with transiently displaced targets, suggesting a major role for eye movements in edge detection.

Psychophysics

Temporal contrast sensitivity measurements.

At the beginning of each session, observers adapted to the mean levels of the surround and the target for 2 min (Fig. 1A). Between trials, observers readapted to the surround and mean level of the target for 2 s. During each trial, the observer viewed an interval in which the target modulation was present and one in which it was absent, and was asked to identify the target interval (2AFC; two alternative forced choice). Threshold values of 75% correct were determined using two randomly interleaved 3-down 1-up staircases. The staircases were started above and below a preliminary estimate for the threshold, based on 8 halving trials starting at maximum contrast. After the two staircases crossed, each staircase was continued for eight reversals, and the average of the last six reversals of both staircases was taken as the threshold for the condition. Contrast at threshold was expressed in terms of modulation amplitude (0%–100%) for the appropriate color axis, and sensitivity was calculated as its reciprocal. Surround conditions were presented in random order. We tested observers in blocks consisting of one surround condition combined with the three target frequencies. Surround conditions were run in random order.

Figure 1.

Spatiotemporal depiction of stimulus and psychophysical results for a 2AFC (two alternative forced choice) detection threshold experiment. A, Spatial configuration and time course of the psychophysical experiments (the classical dark surround condition is illustrated). B, C, Temporal sequence of target and surround modulation depicted as light to dark gray or red to green (i.e., a horizontal slice of the spatiotemporal modulation is depicted over time). S, Surround; T, target. The psychophysical results are plotted as sensitivity = 1/(threshold amplitude) versus temporal frequency (Hz) for three temporal frequencies (TFs; 0.24, 2.4, and 24 Hz), where the mean threshold amplitude ± 1 SEM (average of 4 observers) is expressed in 0%–100% contrast. B, Steady mean-gray, light, and dark surrounds for luminance (LD)-modulated targets (3 TFs × 3 surrounds = 6 conditions). C, Steady mean-gray, red, and green surrounds for chromatic (RG)-modulated targets (3 TFs × 3 surrounds = 6 conditions).

Apparatus and calibration.

Stimuli were presented on a SONY GDM-F520 CRT display, driven by a Cambridge Research Systems Visage graphics system. The monitor's frame-rate was 120 Hz, resolution 1024 × 768 pixels, maximum luminance 208.19 cd/m², and minimum luminance 10.79 cd/m². The monitor was calibrated for linearity, and excursions along the LD and red-green (RG) axes were generated according to the method described in Zaidi and Halevy (1993). Mean luminance of the screen was set at 52 cd/m² and mean chromaticity at a neutral white W (L, M, S = 0.640, 0.332, 0.028). On the CRT monitor, the (L, M, S) coordinates of the ends of the axes were D = 0.0663, 0.0344, 0.0029, L = 1.28, 0.664, 0.056, r = 0.687, 0.285, 0.028, and G = 0.593, 0.379, 0.028. The observer viewed the stimuli through a fixed head and chin rest at a distance of 1 m from the screen. The experiment was run in a dark room. Stimulus generation and data collection were programmed in MATLAB (MathWorks) using the CRS Toolbox.

Observers

All four observers (two male and two female; R.E. was one of the male observers) had normal or corrected-to-normal vision, signed institutional review board-approved consent forms and were compensated for their time. They were informed about the purpose of the experiment after data collection was completed.

Electrophysiology

Recording.

Experimental procedures have been described in detail previously (Sun et al., 2004, 2007). Adult Macaca fascicularus (3) and M. radiata (2) monkeys were initially sedated with intramuscular ketamine (∼10 mg/kg) and then anesthetized with thiopental (10 mg/kg) for initial surgery. Subsequently, anesthesia was maintained by 1%–2% isoflurane in an inspired 70%–30% mixture of NO₂:O₂. The electroencephalogram and electrocardiogram were monitored to ensure adequate depth of anesthesia. A venous infusion of 5 mg/(kg × h) of gallamine triethiodide in 6 ml/kg/h of dextrose Ringer solution was used to maintain muscular paralysis. All procedures were approved by the State University of New York Optometry Animal Care and Use Committee and conformed to the Society for Neuroscience Policy on the Use of Animals in Neuroscience Research.

Stimuli were presented on a SONY Trinitron CRT display calibrated to produce a similar mean-gray (32.7 cd/m²) as the monitor used in the psychophysical experiments. A Cambridge Research Systems VSG 3 drove the monitor at a frame rate of 120 Hz at a resolution of 800 × 600. The VSG was controlled by CRS Stimulus Description Language programs governed by C wrapper programs.

The animals provided data for a number of experiments. For this study, we obtained extensive datasets from 16 cells (11 MC cells: 5 ON Center, 6 OFF Center; and 5 PC cells: 3 +L−M, 2 +M−L), and partial data from other neurons. Cells were classified according to their chromatic or luminance selectivity and their transient or sustained properties. RFs were plotted on a tangent screen located 226 cm from the pupil. All RFs were located in the parafovea, at eccentricities between 5° and 10°. Stimuli were precisely centered on the RF centers by adjusting a counterphase-modulated bipartite field until cell responses were nulled.

Procedure

Square targets patches (2° × 2°) in a square surround (14° × 14°) were used. Modulation was along the (L+M+S) axis (luminance axis) for MC cells and along the (L−M) axis (red-green axis) for PC cells. PC cells gave very weak responses to 0.2 Hz luminance modulation at 40% contrast, with a mean response (n = 5) <2 imp/s. Three surround conditions were used. In the first set of experiments, a stationary target (40% contrast) was modulated at a number of positions along the horizontal axis of the screen relative to the RF, to test for the dependence of responses on the cell's RF location relative to the edge of the target. Similar frequencies were used as in the psychophysical experiments. For MC cells, the surround was dark, mean-gray, or light (0.48, 32.7, and 65.4 cd/m² respectively; CIE x,y; 0.38 and 0.46), whereas it was red, mean-gray, or green for PC cells (32.7 cd/m²; CIE x,y; 0.62, 0.34; 0.38, 0.46; and 0.28, 0.61, respectively). In a second set of experiments, during the modulation of the target, it was displaced abruptly back and forth between two positions along the horizontal axis of the screen, such that its edge traversed the RF. In a third set of experiments, the modulation depth was varied in contrast, either with stationary or displaced targets.

Physiology data collection and analysis

Data were recorded and stored on a Apple Quadra 950 computer as averaged histograms of spike trains accumulated with 64 bins per cycle of modulation; times of spike occurrence were also recorded to a temporal accuracy of 0.1 ms. The number of averaged cycles depended on the modulation frequency and experimental condition. For sinusoidal modulation, response amplitude and phase were extracted from the histograms, by taking the first harmonic of the Fourier transform at the modulation frequency. Response phase was taken into account for the stationary stimuli by referring responses to response phase with the target centered on the RF; we had anticipated that with the RF center just outside the target (but the target still covering the surround), a surround (phase inverted) response might be evoked, but such effects were weak and variable. For each cell, response magnitude was referred to the phase of the response to the center of the patch. This procedure was aimed at detecting any phase shift should the surround response be dominant and to minimize noise when response amplitude was very weak at low frequencies.

Neurometric functions were calculated for a detector that took a sample of the response in a 100 ms window for luminance (300 ms window for red-green) and counted the number of spikes that occurred in each cycle of the histogram (Lee et al., 1993, 2007). The detector was used on responses to the displacement stimuli and to stationary, sinusoidally modulating stimuli with the same contrast series. The beginning of the sampling window was always placed so that either the vigorous responses generated by the displacement or the peak responses generated by the stationary, sinusoidal modulation were centered within the window. The total count for each cycle was tallied, and the tallies were used to construct ROC curves. Neurometric thresholds were then calculated.

Results

Psychophysics

Surround effects on flicker detection

Our first aim was to replicate the surround effects for luminance flicker (Spehar and Zaidi, 1997) and to test on the same observers, whether the surround effect generalizes to chromatic flicker or not. Consequently, we measured temporal sensitivity to luminance (LD) modulation along the (L+M+S) axis (L, M, S refer to long-, middle-, and short-wavelength sensitive cone photoreceptors), and to RG modulation along the (L−M) axis (Krauskopf et al., 1982; Derrington et al., 1984). A 1° target disk was embedded inside an annulus with an outer diameter of 8° (Fig. 1). Either the luminance or the color of the target was modulated sinusoidally at 0.24, 2.4, or 24 Hz, to sample the low-frequency, near-peak, and high-frequency segments of the TCSF. The surround was set to either the mean-gray (W), lightest white (L), or darkest black (D) for LD targets (Fig. 1B) and to mean-gray (W), extreme red (R), or extreme green (G) for RG targets (Fig. 1C). Thresholds were measured using a 2IFC procedure in a double-random staircase, and sensitivity was expressed as the reciprocal of the modulation amplitude (0%–100%) at threshold.

Results for luminance flicker (Fig. 1B) are consistent with earlier publications (Spehar and Zaidi, 1997). The average TCSF on the dark or light surround has a bandpass shape but has a low-pass shape on the mean-gray surround because of substantial elevation of sensitivity at low frequencies. At 0.24 Hz, sensitivity on the mean-gray surround is higher than on the dark and light surrounds by over a factor of 8. The ratio is 2.3 at 2.4 Hz and <1 at 24 Hz.

The contrast between target and surround oscillates around the value of 0 for the steady mean-gray surround, but not for the steady dark or light surrounds. These results led Spehar and Zaidi (1997) to propose that a change in luminance contrast is detected by a temporal low-pass mechanism with a spatial structure that responds at contrast edges. However, if contrast modulation is presented on top of a large contrast pedestal, this mechanism's response is saturated by the pedestal, and a temporally bandpass mechanism may have a lower threshold for detecting the luminance change within the target. The notion of enhanced contrast sensitivity at the edge is also predicted by the lateral-inhibition proposal of Keesey (1970). However, the psychophysical results do not rule out the explanation of Kelly (1979) in terms of fixational eye movements. In addition, the notions of both enhanced edge sensitivity and a temporally low-pass contrast mechanism need to be tested with physiology experiments.

Results for chromatic RG modulation (Fig. 1D) are qualitatively similar to those for luminance modulation. At low frequencies, psychophysical sensitivity with a mean-gray surround is elevated compared with a red or green surround. At 0.24 Hz, sensitivity on the mean-gray surround is higher than on the red or green surround by a factor of 6. The ratio is ∼2 at 2.4 Hz, and ∼1 at 24 Hz.

The psychophysical data with chromatic targets show that the presence of a large chromatic step at the target edge shifts the TCSF from low pass to bandpass. The resemblance to the luminance results is surprising, even though the drop in low-frequency sensitivity was less for chromatic than luminance conditions. RG modulation detection is mediated by PC cells, which show sustained responses and lack an antagonistic center-surround structure for color that would respond preferentially to chromatic edges (Valberg et al., 1992). Therefore, if a common cause underlies luminance and chromatic results, it is more likely to be eye movements than RF structure.

Electrophysiology

Effects of RF structure: physiological measurements with stationary targets

To examine whether ganglion cell RF structures promote enhanced responses to stationary edges, we first tested whether the position of the ganglion cell RF relative to the modulation edge affected cell responsivity with different surrounds. At 0.2, 2, and 20.0 Hz, we measured responses of MC cells to luminance flicker and of PC cells to RG chromatic flicker because these are their preferred color axes (Sun et al., 2006).

The luminance experiments used square targets (120 arc min/side) set in gray, dark, and light surrounds (Fig. 2A). MC cell centers in the parafovea have been estimated to be ∼15–20 arc min in diameter (Derrington and Lennie, 1984; Lee et al., 2010), and Figure 2B sketches positions at which we set the RF center relative to the target, approximately to scale. Square targets were used so that any error in up-down centering on the modulation edge would not play a role.

Figure 2.

Spatial configuration of physiological stimulus and responses of RGCs to different contrasts and temporal frequencies at various spatial locations, including abrupt displacements of the stimulus. A, Conditions of the physiology paradigm. Square (2°) targets were set in gray (same mean luminance as target), dark, or light surrounds. B, The pattern (target + surround) was positioned at various loci relative to the RGC RF center (circles). The size of the circles approximately represents the diameter of the parafoveal MC cell center. C, Mean responses ± 1 SD of MC cells (n = 12, On- and Off-center cells combined, 0.2 Hz, 40% contrast, mean of 8 cycles, first harmonic amplitude) as a function of center location relative to target (hatched bar). Responses were weak; and to improve signal-to-noise ratio and look for surround effects, responses were referenced to the response with the target centered on the RF, using a cosine correction. Responses show no indication of edge effects near the edge of the target (i.e., no enhancement of responsivity deriving from lateral inhibitory effects). The response with a gray surround is similar in amplitude to the response with a light surround, and both are 30%–40% larger than with a dark surround, to which responses extended spatially beyond the edge of the target, possibly because of stray light effects. The responses at locations where the RF center is on the steady background while the RF surround is on the modulated target (shown as negative impulses/sec) were very weak and in opposite phase to the center response. These weak responses are compatible with the absence of edge enhancement by MC cells. D, Mean responses ± 1 SD of PC cells to chromatic modulation (n = 5, +L−M and +M−L cells combined, 0.2 Hz, 100% contrast, mean of 8 cycles, first harmonic amplitude) as a function of center location relative to target (hatched bar). Responses were referenced to the response with the target centered on the RF, using a cosine correction. Responses show no indication of edge effects near the edge of the target and are similar under the different conditions. E, Mean responses ± 1 SD of MC cells as a function of contrast at different temporal frequencies of target modulation (n = 5, ∼6–10 s of data acquisition per condition, target centered on RF). Data have been fitted with a Naka–Rushton function (Naka and Rushton, 1967). Open symbols show peak response amplitudes to displacement of a target (at appropriate contrast and phase, 100 ms window) for a cell sample (n = 5). F, Mean responses ± 1 SD of PC cells as a function of contrast at different temporal frequencies of target modulation (n = 4, ∼6–10 s of data acquisition per condition, target centered on RF). Data have again been fitted with a Naka–Rushton function. Open symbols represent peak response amplitudes to displacement of a target (at appropriate contrast and phase, 300 ms window) for a representative +L−M cell.

Figure 2C shows mean responses of 12 MC cells to 0.2 Hz modulation (40% contrast) for the three surrounds. The x-axis represents distance of the RF from the center of the target. The hatched portion of the axis to 60 min indicates the target extent. On- and Off-center cells did not show systematic differences in their responses, and their data have been combined. MC cell responses were weak at this frequency, as expected from their transient properties. The most important feature of the data is the absence of any response enhancement near the stimulus edge for any of the surrounds. It should be noted that, for MC cells responding to luminance targets and surrounds, center-surround structure is often assumed to cause response enhancement at edges, but our results refute that assumption. Whereas the responses with mid-gray and light surrounds are very similar, responses with dark surrounds were ∼40% smaller than in the other two conditions (p < 0.01), unlike the psychophysical results (Fig. 1), and extended 10 min beyond the target edge, probably because of stray light during target modulation on the dark surround. The weaker responses with a dark surround may be related to incompletely understood remote effects on MC cell responsivity (Solomon et al., 2006) but cannot explain the much larger psychophysical effects (Fig. 1). We also measured response as a function of RF position relative to the target at the higher frequencies, 2 and 20 Hz. Evidence of surround or edge effects was not found under these conditions either.

A similar set of experiments with PC cells was performed using RG chromatic modulation against mean-gray, equiluminant red, or equiluminant green surrounds. PC cell RF center sizes in the parafovea have been estimated to be 15–20 arc min, similar to those of MC cells (Derrington and Lennie, 1984; Lee et al., 2010). Results are shown in Figure 2D (n = 5), in the same format as Figure 2C. Responses of +L−M and +M−L cells were similar, and data have been combined. The data show no indications of any edge effects on responses, and there is no indication that the surrounds of different colors affected the response. A similar spatial pattern of response was found at 2 Hz. It should be noted that PC cells have cone-opponent centers and surrounds that sum the signals from chromatic targets and backgrounds (Valberg et al., 1992); hence, center-surround structure would not be expected to cause any response enhancement at edges; rather, it would smear out the response.

To show that the lack of edge enhancement by MC cells is not the result of response saturation of an edge-sensitive neuron, we measured contrast/response relations at the three frequencies tested (Fig. 2E) for a subsample of the cells shown in Figure 2C (n = 5). Each of the datasets has been fitted with the Naka and Rushton (1967) saturation function. The data show the usual rapid saturation at 20 Hz, but, as expected, there is little saturation at low frequency. PC response amplitudes to different modulation contrasts at 0.2, 2, and 20 Hz were all approximately linear, with only a small fall-off of response at low frequencies (Fig. 2F). This is expected as PC cells give vigorous, sustained responses to chromatic modulation.

The results in Figure 2 demonstrate that lateral inhibitory effects around a stationary edge are insignificant at the retinal level and cannot form a substrate for the psychophysical surround effects. Also, the response at 0.2 Hz is weak and unaffected by surround conditions. It is difficult to see how the enhancement of psychophysical sensitivity by mid-level surrounds could occur at central sites without a large retinal signal. This led us to reconsider how eye movements might enhance psychophysical sensitivity.

Effects of eye movements: physiological measurements with abruptly displaced targets

To mimic the effects of fixational eye movements and microsaccades, we introduced small abrupt displacements of the target patch during the modulation. We chose to use abrupt displacements rather than slow drifts to simplify timing of the displacement relative to the target modulation. However, slow eye drifts are also likely to evoke significant MC cell responses, as taken up in Discussion.

The stimulus configuration and the distinctive response patterns evoked are shown in Figure 3A for an MC On-center cell initially located just outside or just inside the modulation edge. The sinusoid shows the modulation of the target. Abrupt displacements of 20 arc min in one frame were applied at four phases through the modulation cycle, as indicated in the sketches above the sinusoid. The plots of response histograms show the effects of a rightward displacement that occurs mid-histogram (vertical dotted lines). As shown in the histograms, if a displacement occurs when the phase of the modulation is such that there is no contrast across the edge (first and third panels; 0°, 180°), no change in firing occurs. For the On-center cell with the RF initially on the surround, when the target is higher in luminance (90° phase), a vigorous response is evoked as the positive target contrast moves over the RF. For comparison, the peak response of a cell at the center of the stationary target, to the same modulation, is shown as the horizontal dotted line and is weak. When the target is low in luminance (270° phase), there is a contrast decrement, and a suppression of firing can be seen on close examination of the histogram. For the On-center cell with the RF initially inside the modulated target, the phase of modulation causing a vigorous response is now reversed, as shown in the lower set of histograms. If the surround is of higher luminance, there is a vigorous response to the contrast increment when the surround is displaced over the RF. For Off-center cells, the modulation phases evoking excitatory responses were reversed from those of the On-center cells. The first row of histograms also corresponds to the effect of an eye movement that moves an On-center RF from the surround to the modulated target, and the second row of histograms corresponds to the effect of an eye movement that moves the RF from the target to the surround.

Figure 3.

Responses of an MC cell to abrupt displacements of a modulating target on a mean gray and black background. A, Responses of a typical MC On-center cell initially located just outside or just inside the modulation edge to displacement (20 arc min) of a target border over the RF center. The sinusoid shows the modulation phase of the target. Abrupt displacements of 20 arc min in one frame were applied at four phases through the modulation cycle, as indicated in the sketches above the sinusoid (target modulation 0.2 Hz, 40% contrast, mean of 20 responses, bin width 50 ms). Plots of response histograms represent the effects of a rightward displacement that occurs mid-histogram (vertical dotted lines). Top row of histograms, The displacement moved the target over the RF located just outside the modulation edge. Bottom row of histograms, The displacement moved the surround over the RF located just inside the modulation edge, causing a change in the polarity of the response. The horizontal dashed lines indicate the estimated depth of modulation to the stationary target. The phase of the stimulus modulation (0°, 90°, 180°, and 270°) determines the response to the displacement. B, Responses with a dark surround in a similar format. Response amplitude was little affected by the phase of the target modulation or whether modulation was present.

The data show that small eye movements can evoke a vigorous response. With intermediate phases (and thus intermediate contrasts), we assume that the cell responds, but less vigorously, provided the displacement is in the appropriate direction. Also, the response will depend on the angle of the eye movement relative to the target edge; this issue is also taken up in Discussion. The responses of On- and Off-center MC cell arrays close to the edge are activated by the displacement and provide information about the presence of the edge, depending on the current point of fixation (on the target or on the surround) and the timing of the displacement (or eye movement) relative to the target's modulation.

With the surround set to dark or light levels (Fig. 3B), vigorous responses also occurred, but they were similar for all phases of modulation. The vigorous excitatory responses of the On-center cell are seen when the displacement causes a movement of the On-center cell's RF from the dark surround to the lighter target patch. This occurred regardless of the modulation phase, because of the high contrast between surround and target. It would be similar even if the modulation were absent. When the jumps were from the target patch to the surround, firing was completely suppressed. A similar result was seen for Off-center cells and dark surrounds, except response polarity was reversed. With light surrounds, On- and Off-center cells showed the reverse pattern. Thus, with dark or light surrounds, eye movements are unlikely to be able to aid detection of the modulation.

The peak response in Figure 3 was ∼70–80 imp/s. This is about a factor of 10 larger than MC cell responses to the low temporal frequency modulation in Figure 2, with the same contrast. This difference approaches the difference in psychophysical sensitivity. To further quantify the magnitude of the response to displacements relative to modulation, we measured responses at different modulation contrasts of the displacement targets set in a mean-gray surround. Peak impulse rates in a 100 ms sample window (mean of n = 5 cells) are plotted in Figure 2D, along with the contrast response relations for sinusoidal modulation, and have again been fitted with a Naka–Rushton function. The initial slope of this function is the contrast gain, which for stationary modulation was 0.25, 1.35, and 5.83 imp/s/% for 0.2, 2, and 20 Hz, respectively. The contrast gain for the displacement data was 3.50 imp/s/%. This implies that the displacement improves MC cell responsivity by almost a factor of 14 for 0.2 Hz and a factor of 2.6 for 2 Hz, with no enhancement at 20 Hz. This is in approximate accord with the psychophysical data.

Finally, we used a neurometric analysis similar to that in Sun et al. (2008) to estimate detectability of the target in these experiments. In that analysis, we assumed that a peak detector compared activity of On- and Off-center cells. To implement this for sinusoidal modulation, the number of impulses in a 100 ms sample window, centered on the response peak, was compared with an equivalent period in the response trough. For the displacement responses, the number of impulses in a 100 ms window encompassing the peak response was compared with the inhibitory response. Mean contrast for 75% detection by a detector based on a single cell (mean of 4 cells, 2 M-ON, 2 M-OFF) of the stationary, 0.2 Hz target was 25%, and of the displaced target 3.2%. Thus, displacement by eye movements could contribute to the enhancement of psychophysical sensitivity by almost an order of magnitude. The contrast values are a factor of 2–4 higher than psychophysical thresholds, as in the earlier study (Sun et al., 2008), suggesting pooling across cells in the psychophysical decision.

The response of PC cells to the 0.2 Hz chromatic modulation (Fig. 4) was larger than of MC cells to 0.2 Hz luminance modulation, which made changes in response due to the displacements less obvious. PC cells did not exhibit the large response peaks of MC cells but showed a smaller peak superimposed on the relatively sustained firing rate modulation. An example is shown in Figure 4 for a +L−M cell; the dotted horizontal line depicts the peak response to the target modulation alone, which can be seen to be vigorous. For instance, in the first histogram in the top, representing a gray target within the gray surround, there is no response to the displacement but a subsequent slow response to the slow target modulation, which is over the RF after the displacement. In the second histogram, there is a small response peak (arrow) to the displacement, which merges into the ongoing response to the modulation. In the fourth histogram, displacement of a green target over the RF inhibits firing. In the histogram below this, when displacement of the gray surround over the RF replaces the green target, a response peak is evoked. In Figure 4B, displacement responses with a green surround are shown. They were found to be similar at different phases of the target modulation and independent of whether modulation was present. Similar results were obtained for four other PC cells and for red surrounds.

Figure 4.

Responses of a PC cell to abrupt displacements of a modulating target on a mean gray and green background. A, Responses of a +L−M cell to displacement (20 arc min) of a target border over the RF center, with a similar format as in Figure 3, except that chromatic modulation was used. Top row of histograms, The displacement moved the target over the RF located just outside the modulation edge, at the instant indicated by the vertical dashed lines. Bottom row of histograms, The displacement moved the surround over the RF located just inside the modulation edge, causing a change in the polarity of the response. Displacement responses are superimposed on the ongoing response to the target, depending on the location of the RF and the phase of the displacement. Again, the horizontal dashed lines indicate the estimated depth of modulation to the stationary target, which was greater for PC cells and chromatic modulation than for MC cells and luminance modulation. The phase of the stimulus modulation (0°, 90°, 180°, and 270°) determines the response to the displacement. B, Responses with a green surround in a similar format. Response amplitude was little affected by the phase of the target modulation or whether modulation was present.

In the example in Figure 4, the firing peak in the second histogram was ∼50 imp/s, compared with a response of ∼30 imp/s to the target modulation alone. Figure 2F (fourth curve) shows peak response amplitude for this cell as a function of the contrast of the target. The firing rate was averaged over a 300 ms window, the maintained firing just before the displacement was subtracted, and the data were again fitted with the Naka–Rushton (1967) function. The contrast gains for these curves for simple modulation were 0.22, 0.35, and 0.37 imp/s/% for 0.2, 2, and 20 Hz, respectively. The contrast gain for the displacement data was 0.63 imp/s/%. Thus, there is an approximately threefold increase in firing for the displacement condition superimposed on 0.2 Hz modulation. We also applied a neurometric analysis to a small sample of PC cells (n = 4). A 300 ms window was considered, like in Sun et al. (2008), because of the longer critical duration for chromatic stimuli. There we considered that a detection mechanism would compare the output of +L−M and +M−L cells. Here, we compared impulses set in windows at the peak and trough of firing. Neurometric thresholds were 8.2% on average for a displaced target versus 32.1% on average for stationary sinusoidal modulation, a factor of ∼4. Thus, displacement of a target across a chromatic border by means of an eye movement might be expected to elevate psychophysical sensitivity, in approximately the same range as the psychophysical results.

These analyses suggest that signals evoked in ganglion cells by simulated eye movements could account for enhanced psychophysical contrast sensitivity to low temporal frequencies when targets are set in mean-gray surrounds, compared with thresholds on polarized backgrounds (dark and light for MC, and red and green for PC). Our neurometric estimates of the enhancement in sensitivity that are possible with such movements are of approximately comparable magnitude to the psychophysical effects. However, the amplitude, direction, and velocity of fixational eye movements will affect response enhancement, and some of these issues are taken up in Discussion.

Discussion

Psychophysical sensitivity to both luminance and chromatic flicker at low temporal frequencies was enhanced by mean-gray surrounds, compared with flicker on surrounds that differed from the mean. To test whether the surround effects are the result of transients created by eye movements (Kelly, 1979), lateral inhibition at the target edge (Keesey, 1970), or saturation of an edge-sensitive neural mechanism (Spehar and Zaidi, 1997), we recorded from RGCs with and without simulated eye movements. To a first approximation, the physiological results provide a neural account for the surround effects. In the absence of eye movements, responses to luminance or chromatic modulation are not enhanced at the edge of the target, neither for MC nor PC cells. Hence, the idea of a retinal mechanism sensitive to steady edge contrast is not supported. With mean-gray surrounds, small eye movements can generate substantial responses in cells whose RFs are crossed by the edge of the target, depending on modulation phase at the moment of displacement. With a surround that is different from mean-gray, the contrast step between target and surround evokes large phase-invariant responses when eye movements move the edge of the target across RFs; psychophysical sensitivity may then be set by responses of cells anywhere within the target, and the TCSF reflects the temporal properties of RGCs. We now consider the physiological implications in more detail.

For achromatic modulation, the signals deriving from the MC-cell array do not support models that explain the psychophysical results by enhancement of response near a contrast edge, such as those due to lateral inhibitory effects. The strength of the inhibitory surround in MC cells has not been extensively described in the literature. In recordings from this laboratory, a range of surround strengths have been encountered, as evidenced by spatial frequency tuning curves, although they are seldom strong enough to diminish the low-frequency response to less than half its peak value. In particular, with the stimulus configuration used here, little “edge enhancement” was present. Also, the effect of surround luminance level was small (dark surround) or absent (light surround). It is difficult to see how lateral interactions at a cortical site could change this conclusion because responses at 0.2 Hz were very weak, providing little firing substrate to be sculpted by central mechanisms.

This prompted us to reexamine a possible role for eye movements. Even when fixating, people make miniature eye movements (Ratliff and Riggs, 1950; Ditchburn, 1973; Steinman et al., 1973) that can evoke neuronal responses by sweeping image features across RFs (Leopold and Logothetis, 1998; Martinez-Conde et al., 2000; Snodderly et al., 2001; Kagan et al., 2008). It is now recognized that, in early work with stabilized images, methods of eye stabilization may not have been adequate (Martinez-Conde et al., 2004; Santini et al., 2007; Collewijn and Kowler, 2008). In our results, the imposition of brief displacements during target modulation was capable of evoking vigorous responses from MC cells, which were adequate to largely account for the difference in psychophysical thresholds. In our physiological measurements, the amplitude of the displacements (20 min of arc) was large enough to encompass the center of MC cell RFs in the parafovea (Derrington and Lennie, 1984; Lee et al., 2010). In the fovea, the Gaussian radius of MC cell centers is ∼2–3 arc min (Derrington and Lennie, 1984; Lee et al., 2010). During fixation of a marker, fixational microsaccades have a mean of ∼20 min of arc in amplitude (Cherici et al., 2012). Thus, microsaccades could evoke vigorous responses from MC cells for foveally fixated psychophysical targets.

How might different detection thresholds for mean-gray, as opposed to other, surrounds be mediated? With a mean-gray surround, for a given point on the target border, an eye movement can yield a vigorous response from On- and Off-center cells, if the phase of the target modulation and the direction of eye movement is right. Just the presence of a response transient as in Figure 3 would provide a signal for detection (and for extraction of the target's form). In a blank trial, no such transients are present. Because there will be correlated transients in ganglion cell RFs along the displaced target border, the eye movement will be particularly effective in driving cortical cells (Dan et al., 1998; Bruno and Sakmann, 2006; Stanley et al., 2012). With a dark (or light) surround, transient responses always occur on both modulated and blank trials, as in Figure 3B. To distinguish between the two trials, response amplitudes derived from modulations in the center of the target would have to be stored and compared, and Sachtler and Zaidi (1992) showed that discrimination thresholds for gray levels increase many-fold as soon as the two levels are not contiguous in space or time. From observer reports, with a mean-gray surround, detection was associated with the contrast border, which waxes and wanes with modulation, whereas detection in a dark surround was not obviously associated with the target's edge.

We chose to use abrupt displacements of an image to mimic saccades because this simplified the timing of the movement relative to the modulation of the target. In addition to saccades, drifts occur during fixation, with amplitudes of a few arc min and speeds of ∼30 arc min/s (Cherici et al., 2012). Thus, drifts would traverse the center of a foveal MC cell in a fraction of a second, substantially faster than the slowest target modulation. This would lead to a transient MC cell response. However, the angle of fixational eye movements relative to a contrast edge will affect the amplitude of responses evoked. It has long been speculated that gaze drift is under some control (Steinman et al., 1973), and there is some evidence that fixational microsaccades may be under tight control depending on the task (Ko et al., 2010, Poletti et al., 2013), so the pattern of movements as observers performed the psychophysical tasks in this paper would be of interest.

The amplitude of the surround effect found psychophysically with luminance modulation approximately matched the enhancement in response of MC cells caused by small displacements. The psychophysical effects with red-green chromatic modulation were smaller, possibly because discrimination thresholds for chromatic differences survive temporal gaps better than gray-level differences (Sachtler and Zaidi, 1992), and PC cells did show response peaks to displacements that were of an amplitude adequate to largely account for the psychophysical results. Hansen and Gegenfurtner (2009) showed that isoluminant edges are as common as luminance edges in natural scenes. The transformation of spatiochromatic edges to temporal neuronal responses by fixational and other eye movements provides a mechanism for detecting these edges, even with spatially low-pass chromatically tuned neurons.

When viewing a scene, humans are always moving their eyes. Although classical work on the role of eye movements in detection and perception (Averill and Weymouth, 1925; Marshall and Talbot, 1942) predates the work of Kelly (1979), there has been a recent resurgence of interest on the role of eye movements in detection and perception driven by instrumental and conceptual advances. For example, Ahissar and Arieli (2001) and Rucci and Casile (2005) have suggested that spatial coding in perception is primarily derived from the temporal signals associated with eye movements. A case has been made for the role of temporal transients caused by eye movements in decorrelating the signals across natural images (Kuang et al., 2012) and enhancing sensitivity to fine detail (Greschner et al., 2002; Donner and Hemilä, 2007; Rucci et al., 2007; Rucci, 2008). As pointed out by Collewijn and Kowler (2008), under natural viewing conditions, extensive temporal signals are generated not only by eye movements, but also by the head and body movements associated with normal behavior. In experiments on ganglion cells with simulated free viewing of natural images, van Hateren et al. (2002) found that the spatial context of a natural image contained much less information than information content derived from temporal changes (Victor, 1999; Butts et al., 2007). When viewing the targets used in the current experiments with mean-gray surrounds, observers reported that the target edge was subjectively very apparent. The experiments described here support a major role for eye movements in detection of borders and are in line with suggestions that the importance of eye movements in pattern processing has been underrated.

To complement recent psychophysical and theoretical advances on the critical perceptual role of eye movements, this paper provides the first electrophysiological recordings from the early visual pathway linking contextual effects on contrast sensitivity to the responses of retinal ganglion cells evoked by small displacements of the stimulus. Because the critical neuronal responses consist of spike bursts, they may not be well captured by linear systems methods, so our results will provide a better basis for the theoretical modeling that has until now been using estimates of temporal and spatial modulation transfer functions.