Modulation of shifting receptive field activity in frontal eye field by visual salience

Abstract

In the monkey frontal eye field (FEF), the sensitivity of some neurons to visual stimulation changes just before a saccade. Sensitivity shifts from the spatial location of its current receptive field (RF) to the location of that field after the saccade is completed (the future field, FF). These shifting RFs are thought to contribute to the stability of visual perception across saccades, and in this study we investigated whether the salience of the FF stimulus alters the magnitude of FF activity. We reduced the salience of the usually single flashed stimulus by adding other visual stimuli. We isolated 171 neurons in the FEF of 2 monkeys and did experiments on 50 that had FF activity. In 30% of these, that activity was higher before salience was reduced by adding stimuli. The mean magnitude reduction was 16%. We then determined whether the shifting RFs were more frequent in the central visual field, which would be expected if vision across saccades were only stabilized for the visual field near the fovea. We found no evidence of any skewing of the frequency of shifting receptive fields (or the effects of salience) toward the central visual field. We conclude that the salience of the FF stimulus makes a substantial contribution to the magnitude of FF activity in FEF. In so far as FF activity contributes to visual stability, the salience of the stimulus is probably more important than the region of the visual field in which it falls for determining which objects remain perceptually stable across saccades.

our visual perception is stable despite frequent and abrupt shifts of the retinal image by saccades. One hypothesis is that the failure to perceive the shifts depends on knowledge of the impending saccade that is provided by an internal copy of the motor command, a corollary discharge or efference copy (Sperry 1950; von Holst and Mittelstaedt 1950). Neurons that respond to visual stimuli would anticipate the occurrence of an upcoming saccade as a result of this corollary discharge input. Duhamel et al. (1992) found such a potential neuronal mechanism in the parietal cortex. In anticipation of an upcoming saccade, parietal neurons became sensitive to visual stimuli at the spatial location that the receptive field (RF) would occupy after the saccade. They proposed that this anticipatory activity at the site of the RF after each saccade indicated a remapping that underlies visual stability. These shifting RFs subsequently have been found in the frontal eye field (FEF; Sommer and Wurtz 2006; Umeno and Goldberg 1997) and in several other cortical and subcortical areas (for a summary, see Sommer and Wurtz 2008).

The procedure most frequently used to identify shifting RFs is illustrated schematically in Fig. 1A. The RF of a neuron is first mapped while the monkey fixates (Fig. 1A, red fixation cross and dotted circle). Just before a saccade to a visual target, a stimulus is flashed at the location the RF will occupy after the saccade, which we refer to as the future field of the neuron (FF; Fig. 1A, blue fixation cross and blue dotted circle). In a substantial fraction of neurons in FEF and lateral intraparietal area (LIP), there is an increase in activity following the flash of the FF stimulus, which is always presented before the saccade begins. From Fig. 1A, it is clear that the stimuli used in most experiments to study shifting RF activity in LIP and FEF are flashed spots of light against a uniform background, which creates what must be a salient visual stimulus. The salience of the stimulus is particularly relevant in light of the emerging view that both LIP and FEF can be regarded as having salience maps for visual stimuli (Kusunoki et al. 2000; Thompson and Bichot 2005). Such salience results from the bottom-up effect of the stimulus characteristics (Koch and Ullman 1985) but with modulation by top-down influences (Folk et al. 1992). Neurons activated by stimuli with the highest salience represent the regions of the visual field that are likely to attract attention, those that are likely to be selected during visual search, and those likely be selected as the target for a future saccade in FEF (Schall et al. 1995; Thompson et al. 1996) and in LIP (Gottlieb et al. 1998; Kusunoki et al. 2000). (Fig. 1B emphasizes that top-down or goal-directed attention is directed toward the target of the saccade being made.)

Fig. 1.

Potential onset and goal-directed attention actions on shifting receptive field (RF) activity. A: while the monkey looks at a fixation point (red cross), a RF can be mapped (red dotted circle). As the monkey prepares to make a saccade from the fixation point to a peripheral target (blue cross), there is an anticipatory shift of the RF of the neuron to a future spatial location. This future field (FF) location (blue dotted circle) is at the same location with respect to the final eye position after the saccade (blue dotted circle to blue cross) as the RF is to the current fixation point (red dotted circle to red cross). B: 2 types of attention act at the time of the RF shift. First, as the monkey makes the saccade to the target, there is a shift in goal-directed, or endogenous, attention to the target. Second, the single stimulus flashed against a dark background at the FF location produces an onset, or exogenous, attention effect.

One stimulus characteristic that produces salience is its abrupt onset. In our normal vision, such stimulus onset plays a major role in directing attention to those stimuli. Perhaps the most dramatic demonstration of this is our failure to recognize even large changes made in a visual scene when the onset of the change is masked (Rensink et al. 1997). In these change blindness experiments, either a saccade or a brief blanking of the entire scene sharply reduces the perception of even large changes in the visual scene. In normal vision, the attention directed to a stimulus that has just appeared is referred to as exogenous attention or simply stimulus onset attention (Egeth and Yantis 1997).

The next question is how much this onset salience contributes to the FF activity in the shifting RF experiments. In LIP, the effect of stimulus salience on shifting RF activity was observed as an ancillary finding in experiments that established the importance of stimulus salience (Gottlieb et al. 1998). These experiments compared the response of LIP neurons to a stimulus turned on in the RF to the response when the RF was brought onto the stimulus by a saccade, without any abrupt onset. Gottlieb et al. (1998) found that with the stimulus onset, the visual response was substantially larger. They also pointed out that for an LIP neuron illustrating this difference (their Fig. 2A), the neuron's activity occurred with a shorter than expected latency, which they took as an indication of what we refer to as FF activity. The FF activity was larger when the visual stimulus had an onset than when it did not, that is, when the stimulus had greater salience.

Fig. 2.

Task for studying exogenous (onset) attention on the shifting RF activity. A: locations of the RF, FF, and added stimuli during the shifting RF saccade task. The task has 2 conditions: a shift without added stimuli (top row) and one with added stimuli (bottom row). In both conditions, after the monkey fixated a central point (red cross), a RF stimulus (red spot in red dotted circle) was flashed, and then after a variable delay, the fixation point was extinguished as the target (blue cross) came on. Before the saccade began (dashed arrow), a 50-ms flash probed the sensitivity at the FF location (blue spot in blue dotted circle). Two points to note: on every trial there was a 50-ms flash in the RF and then, just before the saccade, a 50-ms flash in the FF; the FF flash was over before the saccade began. In trials with added stimuli, 8 added visual stimuli came on at locations that did not evoke a visual response. The same spatial configuration of the added stimuli was presented for the RF and FF, but shifted by the vector between the initial and future fixation points (the saccade vector). B: timing of the task events. The lengths of the colored bars represent the duration of the respective event. See methods for further details.

In the present experiments we tested the possible contribution of visual salience to the FF activity of FEF neurons. The goal was to see whether the magnitude of the FF activity in previous experiments (Sommer and Wurtz 2006; Umeno and Goldberg 1997) actually was enhanced by the salience resulting from stimulus onset as suggested by the observation in LIP. In our experiments we first studied the FF activity as had been done previously; we flashed the stimulus in the FF of the neuron just before the saccade to identify the subset of neurons that showed shifting RFs. In those that did, we then added the onset of other stimuli to see whether these added stimuli reduced the FF activity. The goal was to reduce the salience of the FF stimulus without changing the stimulus itself, and judging from behavioral experiments, having additional stimuli in the visual field seemed a reasonable way to do that. In visual search tasks, the addition of stimuli in the visual field (usually referred to as nontargets or distractors) reduces the salience of a stimulus as indicated by increased reaction time (Duncan and Humphreys 1989; Kim and Cave 1999). In addition, the effect of stimulus onset has been shown to be reduced if distractors are added to a stimulus (Wright and Richard 2003). Our goal was to place the added stimuli outside of the estimated area of the RF and FF to minimize a direct visual effect of the added stimuli. If the FF activity is ordinarily facilitated by exogenous attention, we would expect to see reduced sensitivity to the FF stimulus when multiple stimuli are added, and we frequently did.

METHODS

In two adult male monkeys (Macaca mulatta) weighing from 8 to 11 kg, we implanted scleral search coils for measuring eye position, recording cylinders for accessing FEF, and a post for immobilizing the head during experiments as described previously (Sommer and Wurtz 2000). All procedures were approved by the Institute Animal Care and Use Committee and complied with Public Health Service Policy on the humane care and use of laboratory animals.

Experimental Setup

The monkey sat in a primate chair with its eyes 57 cm in front of a tangent screen. The chair was in the center of magnetic field coils in a dark room that was sound attenuated. Computers running REX (Hays et al. 1982) and associated programs controlled stimulus presentation, administration of reward, the recording of eye movements and single neuron activity, and the online display of results. Visual stimuli appeared on a gray background, back-projected by a DPI projector.

RF Mapping

The first experimental step to determine the consequences of added stimuli on the magnitude of the shifting RF activity was to obtain detailed knowledge of their conventional RFs. While the monkey fixated a central red cross, we determined the location and extent of the RF and the optimal stimulus size to elicit the strongest visual response. This enabled us to place the added stimuli at locations that should minimally invade the RF of the neuron and thus minimize visual interactions. We determined three key points about the RF of each neuron.

RF center.

We first estimated the location of the RF by creating a coarse spatial map of the visual activity. While the monkey fixated a central red cross, we probed the RF of the neuron by systematically presenting a visual stimulus (with a diameter of 1–5° depending on the eccentricity) at one of nine locations on a 3 × 3 grid. The grid spacing was adjusted to cover as much of the estimated RF as possible. The RF center was taken to lie at the mean of the nine locations weighted by the magnitude of the visual response at each location.

RF center size.

On successive fixation trials, we presented filled circles of different sizes (diameter between 1° and 70°) at the estimated RF center. The optimum stimulus size and estimate of the RF center was taken as the point where the curve relating visual response to spot size reached a peak. To determine the peak, we fit the plot of visual response to each spot size to a curve using the difference of two Gaussian functions, one representing a narrow excitatory center of the RF and the other the wider suppressive or inhibitory surround (Rodieck 1965). This experiment and analysis provided a two-dimensional model of the RF structure.

RF suppressive surround.

Qualitative examination indicated that there was a suppressive surround in all the FEF neurons studied (the area around the excitatory center that when stimulated suppressed the neuron's response). We estimated the extent of the suppressive surround by presenting a spot of light in the center of the RF (the size and location determined from the 2 tasks described above) and then varying the size of an annulus surrounding the spot. The annulus had a constant outer diameter of 70° but a variable inner diameter so that as we increased this inner diameter, the annulus became narrower, and less light from it fell on the RF. We took the outer boundary of the suppressive surround to be the smallest inner diameter at which there was no significant difference from the response to a center stimulus presented alone and with the annulus.

In all three tests, the monkey received a liquid reward for maintaining fixation (within a 1.5° square) for the duration of the trial. There were 8 presentations of a visual stimulus per trial, and each stimulus presentation was 50 ms in duration with an interstimulus interval of at least 500 ms.

Shifting RF Measures

Following these preliminary RF estimation experiments, we performed the main experiment to test FEF neurons for a shifting RF and determine the effect of added stimuli on the magnitude of this activity.

Shifting RF saccade task.

The monkey was trained to make a saccade to a target that appeared at the same time as the initial fixation cross was turned off (Fig. 2). The saccade target was always placed in the ipsilateral field to diminish the saccade-related responses that were stronger to the contralateral field. (For testing whether the neuron was a visuomotor one, contralateral saccades were made in a separate test). During this task both the RF and FF were sequentially examined. On each trial, after 500 ms of initial fixation of the central red cross, a visual stimulus the size of the RF excitatory center (filled red circle in Fig. 2A) was flashed in the center of the RF (red dashed circle) for 50 ms. After a variable delay (500–700 ms), the fixation point was turned off and a saccade target (blue cross) was simultaneously presented in the periphery. [This delay ensured an interval of at least 500 ms between the responses to the RF stimulus and the FF stimulus. There was no observed interaction between the two responses, and the delay provided enough time to distinguish the FF activity from the background activity (see Data Analysis).] Before the monkey made a saccade to the new fixation point (dashed arrow), the same stimulus used to probe the RF was flashed in the FF (blue dashed circle) for 50 ms. That is, the same stimulus was presented again, but at a location displaced from the center of the RF by the saccade vector. For example, if the center of the RF is at x = 15°, y = 10°, and the monkey is required to make a saccade from the fixation point at 0°, 0° to −30°, 0°, the FF center is −15°, 10°. Importantly, the FF stimulus was extinguished before the saccade was made (solid arrow). Note that on every trial both the RF and the FF stimuli were presented.

There were two conditions in this task: with and without added stimuli (Fig. 2A, bottom and top rows, respectively). The description above is the without added stimuli condition. During the with added stimuli condition, eight added visual stimuli were also presented with the RF or FF stimulus in the same trial. These added stimuli were the same size (determined from the RF center size task) and were presented for the same duration as the visual stimulus in the without added stimuli condition. Added stimuli were positioned at locations beyond the suppressive surround, that is, where a peripheral stimulus did not alter the response of a spot in the center of the RF. (This was typically 5° beyond the RF extent as determined with the RF surround test). As depicted in Fig. 2A, we frequently placed the additional stimuli in the opposite hemifield of the RF. This had two advantages: 1) there was less chance of a visual interaction because the RF field rarely extended into the opposite hemifield, and 2) this placement allowed us to maintain all stimuli presented with the RF or FF stimulus on the screen despite the relatively large saccades. Within a trial, the same configuration of the added eight stimuli was presented for the RF and FF, but shifted by the saccade vector. This configuration of the added stimuli was randomized from trial to trial. That is, the added stimuli were always positioned outside the suppressive surround, but the exact positions varied from one trial to the next. In each condition, the monkey received a liquid reward for making a saccade to the new fixation cross (within a 5.0° square) within 500 ms after the appearance of the saccade target.

Control Experiments

We performed several control experiments on every neuron to ensure that the observed FF activity was dependent on the combination of the FF stimulus and that the generation of the saccade and was not solely a saccade-related or visual response. To determine that the saccade alone was not producing the FF activity, we performed experiments in which the monkey made saccades to the target in the shifting RF saccade task but in the absence of the FF stimulus. In this case, the saccadic eye movement was made in the absence of the FF stimulus. These control trials were pseudorandomized and embedded in the shifting RF saccade task. To ensure that the FF activity was not a visual response, we presented the FF stimulus while the monkey fixated the central red cross. In this case, the FF stimulus was presented in the absence of the saccade. This control was done before the shifting RF saccade task to ensure that the FF was beyond the RF of the neuron.

Neuron Recording

We placed one neuron recording cylinder over the FEF approximately normal to the skull. After initial estimation using MR images, we located FEF within the cylinder electrophysiologically. We recorded single-neuron responses and microstimulated in FEF with tungsten microelectrodes advanced by a stepper microdrive. Electrodes passed through guide tubes in a 1-mm-resolution grid in the recording cylinder (Crist et al. 1988). Neuronal responses were discriminated from background activity using a software-based waveform discriminator. We characterized visual and movement fields by monitoring neuronal activity while the monkey made saccades to targets throughout the contralateral visual field. We targeted neurons in the anterior bank of the arcuate sulcus, and we verified that they were in FEF using two criteria: saccade-related activity was found in many neurons, and saccades were evoked with currents of ≤50 μA (Bruce and Goldberg 1985). Neurons were excluded from further analysis if they did not demonstrate shifting RF activity or if we were unable to acquire sufficient data.

Data Analysis

RF visual responses were measured in a time window starting 40 ms after stimulus onset and ending when activity fell below 2 SD of a background activity epoch. This background epoch was from 60 ms before to 40 ms after stimulus onset. Because FF activity is synchronized to saccade onset (see Sommer and Wurtz 2008), neuronal activity was measured in a period beginning 200 ms before to 300 ms after saccade onset. Within this period, the FF activity began when neuronal activity was ≥2 SD of the background activity epoch and ended when activity dropped below that level. This background epoch was neuronal activity 300–200 ms before saccade onset. Stimulus-dependent modulation of the RF response and FF activity were determined using a two-tailed t-test (α = 0.05). In these tests, RF responses without added stimuli were compared with RF responses with added stimuli, and FF activity was similarly compared just with FF activity. If there was a difference in the time of the start and termination of the response between the two conditions, we determined the smallest window to perform the statistical test. That is, the firing rates between the two conditions were compared between the latest start and earliest termination time of the responses. Repeating the analysis with a full window (beginning with the earliest start and ending with the latest termination time of the responses) yielded similar results. Saccade initiation was identified as the time that eye velocity and acceleration exceeded both 100°/s and 5,000°/s², respectively.

RESULTS

We recorded FEF neurons that showed shifting RFs in three hemispheres of two monkeys. We studied 171 neurons of which 52 (30%) had significant FF activity. We were able to assess the saccade related activity in addition to the visual response in 47 of these 52 neurons and found that 41 of the 47 were visuomotor neurons. We were able to compare the magnitude of the RF response and FF visual activity with and without added stimuli in 50 neurons (24 from monkey Ar and 26 from monkey Fl). We saw no significant difference between the monkeys and have combined their results.

Effect of Added Stimuli on RF and FF Activity

We compared the response to the RF and FF stimuli presented alone to that with added stimuli. Figure 3 illustrates the results for an example neuron with the test of the RF response shown at left and that for the FF activity shown at right. First, a stimulus of the optimal size (in this case, 6°) was flashed in the RF (Fig. 3A, red dashed circle) long before the saccade to the target to serve as a baseline for the magnitude of the visual response. The same stimulus was then flashed in the FF (Fig. 3B, blue dashed circle) just before the saccade to see if there was FF activity and to determine its magnitude. In the next series of trials, we then interleaved these trials with RF and FF stimuli alone with trials with eight added stimuli flashed at the same time and for the same duration as the RF and FF stimuli (Fig. 3, C and D). Figure 3, E and F, shows the effect of these added stimuli on the RF response and the FF activity. Note that the RF response has a sharp onset typical of a visual response, whereas the FF response is best described as “activity” because of its synchrony with the saccade rather than the FF stimulus (see Sommer and Wurtz 2008; Umeno and Goldberg 1997). For this example neuron, there was little effect of these added stimuli on the RF response (Fig. 3E). The average firing rate was statistically indistinguishable with and without the added stimuli (P = 0.93, 2-tailed t-test). In contrast, the FF activity showed a 44% decrease in the response with the added stimuli (Fig. 3F, P < 0.001, 2-tailed t-test). The latency of the activity was not affected, a finding that was consistent across our sample (P = 0.83, 2-tailed t-test).

Fig. 3.

Reduction of the FF activity by added stimuli for an example frontal eye field (FEF) neuron. A, C, and E depict the stimulus configuration and results for the RF response, and B, D, and F depict the equivalent results for the FF activity. In trials with no added stimuli, a single visual stimulus (red and blue spots) was flashed in the RF (A) and FF (B). The RF, including excitatory center and suppressive surround, is represented by the red dashed circle, and the possible FF by the blue circle. The stimuli were at the same location with respect to the initial fixation point (red cross at 0°, 0°, with the RF stimulus at 27°, 19°) and future fixation point (blue cross at −30°, 0°, with the FF stimulus at −3°, 19°). In the trials with added stimuli, the RF and FF stimulus was accompanied by 8 added visual spots (C and D, respectively), all 6° in diameter. The stimuli added to the RF spot did not significantly alter the visual response (E; P = 0.93, 2-tailed t-test), but the addition of the stimuli reduced the FF activity significantly (F; 44% reduction, P < 0.001, 2-tailed t-test). sp/s, Spikes per second.

We attempted to keep the added stimuli out of the RF (and by inference, out of the FF) of the neuron by placing them at least 5° outside of the RF, including both the excitatory central area and the suppressive area (see methods). For the example neuron in Fig. 3, the lateral extent of the RF was 25° from the RF center and the added stimuli were placed at locations outside a 30° radius from the RF center. The lack of any significant difference in the visual activity with and without the added stimuli (Fig. 3E) is consistent with their placement outside the RF.

Figure 4 shows the effect of the added stimuli for the 50 FEF neurons studied. The RF response and FF activity are again shown at left and right, respectively. In Fig. 4, A and B, the responses with added stimuli are plotted against the responses without them. The circles represent neurons that had a significant change in activity with added stimuli (P < 0.05, 2-tailed t-test), and of these, the filled circles indicate where the change was a decrease. There was a significant decrease in the RF response in 14 (28%) of the 50 neurons and a significant increase in the response in 3 (6%). There was a significant decrease in the FF activity in 15 (30%) of the 50 neurons and a significant increase in the activity in 2 (4%).

Fig. 4.

Change in the RF response and FF activity with added stimuli for the sample of neurons. The RF response (A) and FF activity (B) with added stimuli (vertical axis) is plotted against the respective responses without them (horizontal axis). Filled circles represent neurons that had a significant decrease in the visual activity with added stimuli (P < 0.05, 2-tailed t-test). The red and blue circles represent the example neuron from Fig. 3, and the dashed line is the unity line. C and D display histograms of the percent change of the RF response and FF activity, respectively, with added stimuli. Black markers indicate the average percent change for the sample; red (RF) and blue (FF) markers indicate the percent change values for the example cell. There were significant decreases with added stimuli: a small 4% decrease for the RF response and a larger 16% decrease for the FF activity. E and F display histograms for only those neurons showing a significant change (P < 0.05, 2-tailed t-test) in the FF activity, which show a nonsignificant decrease for the RF response (E; 6%) and a larger significant decrease for the FF activity (F; 34%).

The histograms in Fig. 4, C and D, show the percent change in the magnitude of activity with added stimuli for the RF response and FF activity. The black triangles mark the average percent change for the 50 neurons, and the red and blue markers indicate the percent change values for RF and FF of the example neuron in Fig. 3. On average, the added stimuli led to a small (4%) but significant decrease in the RF response (P = 0.03, significantly different from 0, 2-tailed t-test). Added stimuli had a greater influence on FF activity, causing an average 16% decrease that was also significantly different from 0 (P < 0.001). The magnitude of the change in the RF response and the FF activity for individual neurons was not correlated (R = 0.087, P = 0.548).

The slight decrease in the RF response in some neurons could be either the result of the effect of added stimuli on salience or an indication that they invaded the suppressive surround of the neuron. If we make the worst case assumption on the placement of the added stimuli and assume they invade the RF of these neurons, they likely act on FF as well (making the additional assumption that the RF and FF are similar). Therefore, the most conservative estimate of the effect of the added stimuli on the FF activity would be the difference in the average percent change of the RF response and FF activity. This difference of the percent change with added stimuli (a reduction in the RF response by 4% and in FF activity by 16%) is 12% for the sample of neurons. The average percent reduction of the FF activity with added stimuli is significantly greater than the average percent reduction of the RF activity (P = 0.003, 2-tailed t-test). This indicates that the decrease in the FF activity was greater than the amount that could be attributed to any visual interaction.

The histograms in Fig. 4, E and F, are data for the subset of 17 neurons that demonstrated a significant change (decrease or increase, P < 0.05, 2-tailed t-test) in the FF activity with added stimuli. The RF response for this subset decreased by an average of 6%, and the FF activity decreased by 34%. The average decrease in the RF response was not significantly different from 0 (P = 0.08, 2-tailed t-test). Similar to results for the entire sample, the average decrease in FF activity was significantly different from 0 (P < 0.001, 2-tailed t-test). In addition, the average reduction in the FF activity was significantly greater than that in RF (P < 0.001, 2-tailed t-test).

In summary, we saw a clear reduction of the FF activity with the addition of stimuli in 30% of the FEF neurons that have FF activity. Across the population and for the subset of cells that demonstrated a significant change, the average FF activity reduction with the additional stimuli was significantly greater than the average reduction in the RF responses.

Distribution of Shifting RFs in the Visual Field

If the shifting RFs are related to stable visual perception during saccades, one possibility is that these shifts are concentrated in the fovea or the central visual field rather than distributed equally throughout the visual field. If that were the case, we might expect that the frequency of shifting receptive fields and the magnitude of the shift activity would be higher in the central visual field. We therefore attempted to study FEF neurons with RFs over a range of eccentricities.

Figure 5 shows the frequency of the RF shifts from both monkeys expressed as a function of RF eccentricity. The graph shows cumulative plots in which each point on the cumulative curve represents the percentage of the total number of neurons reached at the eccentricity shown on the horizontal axis. The black cumulative curve shows the proportion of FEF neurons at each eccentricity whose RFs were determined (n = 171).

Fig. 5.

Cumulative distribution for the sample of neurons as a function of RF eccentricity. The cumulative distribution of the sample of neurons tested (black circles, n = 171) and the subsample of these cells that demonstrated a shifting RF (blue circles, n = 52) are plotted as a function of RF eccentricity. The orange circles represent the shifting RF neurons that demonstrated a significant decrease (P < 0.05, 2-tailed t-test) in the FF activity in the presence of added stimuli (n = 15).

This curve shows a reasonably linear progression out to the maximum eccentricity studied (39°), indicating that our sampling was reasonably distributed across the visual field. The blue cumulative curve plots the subsample of these cells with shifting RFs (n = 52), about 30% of the total sample. The cumulative curve for neurons with shifting RFs shows a similar linearity except for the anomaly between 20° and 30°, where no neurons with shifting RFs were sampled and the subsequent series of points appear piled up at 30°. This implies that the number of neurons with shifting RFs occurs with equal frequency across the visual field; there was no significant difference between the two distributions (P = 0.37, 2-sample Kolmogorov-Smirnov test). In addition, there is no evidence that the shifts are particularly related to neurons with RFs near the center of the visual field. We also examined the magnitude of FF activity across RF eccentricity. Figure 6A displays the distribution (bin width 5°) of the magnitude of the FF activity as a function of RF eccentricity (n = 52). There was no significant difference in the mean magnitude of the FF activity across eccentricity (P = 0.73, 1-factor ANOVA). Thus both the frequency and the magnitude of the shift activity in FEF neurons remained roughly constant across the central 30° of the visual field.

Fig. 6.

Magnitude of FF activity across RF eccentricity. A: distribution (bin width 5°) of the magnitude of FF activity as a function of RF eccentricity (n = 52). The height of each bar represents the average FF activity for the cells that fell within that bin. B: distribution (bin width 5°) of the FF activity with (orange bars) and without added stimuli (blue bars) as a function of RF eccentricity. The black trace represents the difference in mean FF activity with and without added stimuli. Only cells with a significant decrease (P < 0.05, 2-tailed t-test) in FF activity with added stimuli are displayed (n = 15). The height of each bar represents the average activity for the cells that fell within that bin. The gaps at the 3 eccentricity ranges are due to a lack of cells that demonstrated a significant decrease in the FF activity with added stimuli.

Another factor that might vary with eccentricity is the frequency with which added stimuli reduced the FF response. In Fig. 5, the orange points on the graph show the fraction of shifting RF neurons whose FF activity was significantly reduced by added stimuli (n = 15, 30% of the subsample). We found no such neurons within the central 10°, and there was a significant difference between this distribution and the cumulative curve for neurons with shifting RFs (P = 0.002, 2-sample Kolmogorov-Smirnov test). Within our small sample, neurons showing decreased responses with added stimuli may be more frequent in the peripheral visual field. Figure 6B displays the distribution (bin width 5°) of the mean magnitude of FF activity with (orange bins) and without added stimuli (blue bins) as a function of RF eccentricity. Only cells that demonstrated a significant decrease in the FF activity with added stimuli are shown (n = 15). There were variations in the magnitude of FF activity with and without added stimuli at different eccentricities, but the difference between them (black triangles) did not systematically vary with eccentricity (P = 0.24, 1-factor ANOVA). (The gaps at 3 eccentricity ranges are due to a lack of cells that demonstrated a significant decrease in the FF activity with added stimuli.)

In summary, we found that both the frequency of neurons with shifting RFs and the magnitude of any shift activity were relatively constant within the central 35° of the visual field, with no special emphasis on the central visual field.

Tests for Extraneous Visual and Saccadic Factors

The goal of our experiments was to test the effect of reducing stimulus salience due to stimulus onset on the shifting RF activity by using the added stimuli. Since we had both the RF and the FF stimuli flashed on each trial and had the additional stimuli on many trials, we included several control experiments and analyses to address the following questions.

Are there visual interactions between RF, FF, and added stimuli?

From the information we had about the RF size of the neurons studied, we placed our FF stimulus and added stimuli outside the measured RF. This was, however, only a best estimate, and we therefore adopted an empirical approach to test whether the FF stimulus and the added stimuli invaded the RF by examining the timing of the FF visual activity. If the activity after the FF stimulus and FF added stimuli were the result of a shifting RF, the latency of that activity would be related to the onset of the saccade. However, if the activity was a visual response because the FF stimulus, added stimuli, or both impinged on the RF of the neuron, the latency would be fixed to the onset of the FF and added stimuli.

Figure 7 shows that for the same example neuron presented in Fig. 3, the increased neuronal activity occurred long after the visual latency of the neuron and was not aligned to the stimulus onset but was aligned on the saccade onset as expected for shifting RF activity. Figure 7A shows neuronal activity for the example neuron aligned with onset of the FF stimulus and added stimuli (green vertical line). Each row of dots represents spikes on one trial with the trials plotted in ascending order of saccade latency, with shortest latency at the bottom. The dashed red line parallel to the green FF stimulus line indicates the time at which the activity should increase if it were a visual response. The activity increases much later than the 47-ms visual latency of this neuron. Furthermore, the increase in neuronal activity is not parallel to the vertical FF line but instead more closely parallels the tilted saccade onset vertical line. This is supported in Fig. 7B, which aligns the same neuronal activity as in Fig. 7A on saccade onset (blue vertical line). The neuronal discharge on the raster occurs long after the visual latency of the neuron and parallels the now vertical line of saccade onset. The relationship between the onset of the FF activity and the saccade is confirmed in Fig. 7C. The onset of the FF activity is plotted against the onset of the saccade for the data presented in Fig. 7A. In this case the onset times are plotted with respect to the onset of the FF stimulus and added stimuli. There was a clear relationship between the onset of the FF activity and the onset of the saccade (R = 0.44, P < 0.001). (The onset of the FF activity was determined by finding the maximum instantaneous firing rate within the interval plotted in Fig. 7A.) For all the neurons studied, we found the FF stimulus and added stimuli activity to have a much longer latency than the visual latency of the neuron (across the population, the mean visual latency and FF activity latency were 52 ± 14 and 158 ± 55 ms, respectively) and the same parallel relation of the FF activity and the saccade onset. We were only able to perform the test displayed in Fig. 7C on four neurons in our sample due to the need for a substantial number of trials, for variability of the interval between onset of the saccade and FF stimulus, and for a sufficient number of spikes per trial to determine the instantaneous rate. However, for the four neurons there was a significant linear relationship between the onset of the FF activity and the onset of the saccade (R > 0.25, P < 0.001). In addition, there was no significant correlation between the FF onset and the stimulus onset (R < 0.1, P > 0.4). We conclude that activity related to the FF stimulus does not have the characteristics of a visual response and that the FF field activity is not due to the onset of stimuli within the RF of the neuron.

Fig. 7.

FF activity is better aligned to saccade onset than FF stimulus onset. A: each row on the raster plot represents spikes on 1 trial for the example cell; the trials are plotted in ascending order of saccade latency with the shortest latency at the bottom. The neuronal activity is aligned to the onset of the FF stimulus and added stimuli (green vertical line). The increased activity occurred long after the visual latency for this neuron (47 ms, indicated by dashed vertical red line) and followed the saccade onset (blue line). B: the same neuronal activity but aligned to the saccade onset. Note the difference in time scales between A and B. C: the onset of the FF activity plotted against the onset of the saccade for the data presented in A. Onset times are with respect to the onset of the FF stimulus and added stimuli. There was a significant linear relationship between the onset of the FF activity and the onset of the saccade (R = 0.44, P < 0.001).

Does any change in the variability of saccade amplitude or latency account for the decrease in the FF activity?

A factor that could contribute to the general decrease in the FF response with added stimuli is a change in the amplitude or latency of the saccade, and we investigated the extent of the changes in each.

A difference in the average saccadic endpoint and its variability would be important if the saccade went to a substantially different location in the two experimental conditions, resulting in a significantly different saccade vector and therefore a potentially different corollary discharge signal. Figure 8A shows the mean eye-movement position and endpoint variability (95% confidence intervals) of saccades made to the same target with and without added stimuli (black and blue traces and ellipses, respectively) during a session for the example neuron in Fig. 3. There was no significant difference in mean saccade endpoints in the presence of added stimuli. However, there was an increase in the saccade endpoint variability with the added stimuli. This increase could result in different trial-to-trial stimulation of the FF by the stimulus and lead to a decrease in FF activity. We therefore determined whether the increased variability was large enough to account for the decrease in FF activity with the added stimuli. Because for each neuron we selected a stimulus size that when placed in the center of the FF would optimally activate the neuron, any displacement of the stimulus as a result of changed saccade amplitude would lead to a reduction of the visual activity. We can estimate magnitude of the reduction from saccade displacement by seeing how such displacement of the stimulus from the center of the RF would reduce the response. Figure 8B shows the results that such a displacement would produce on the two-dimensional map of the RF (see methods). The vertical axis shows the percent change in FF activity with added stimuli that we observed across our sample of neurons (Fig. 4D). The horizontal axis shows the change in activity that would result if we moved the stimulus spot from the center (as would occur with different saccade amplitudes). As shown in Fig. 8B, there was minimal change in the activity that can be attributed to the variability of the saccade compared with the change with the added stimuli. There was no significant correlation (R = 0.2, P = 0.21). Therefore, such a small increase in saccade endpoint variability was unlikely to account for the decrease in FF activity. This analysis, however, does assume that the FF has the same organization as the RF, a point that needs to be tested in future investigations.

Fig. 8.

Changes in saccade endpoint scatter and latency with added stimuli do not account for the FF activity decrease. A: for the example neuron presented in Fig. 3, the mean position of the saccade endpoint with and without added stimuli did not increase, but the scatter did increase (black and blue traces and ellipses, respectively). The ellipses represent 95% confidence intervals around the mean endpoint. B: percent change in FF activity with added stimuli (vertical axis) plotted against the percent change in FF activity due to saccade endpoint variability (horizontal axis). There was no significant relationship between the percent change in FF activity with added stimuli and the percent change due to saccade endpoint variability (R = 0.2, P = 0.21). (Note that positive percent changes on the horizontal axis represent cases where the saccade endpoint variability for the added stimuli condition was less than the variability in the without added stimuli condition.) C: distribution of the intervals between the FF stimulus onset and saccade onset with (black trace) and without added stimuli (blue trace). The 0 on the horizontal axis represents saccade onset, and the red dashed lines represent the points where the interval distributions overlap (between −167 and −107 ms). D: FF activity with (black trace) and without added stimuli (blue trace) for the trials within the red dashed lines in C. The presence of the added stimuli reduced the FF activity significantly (41% reduction, P < 0.001, 2-tailed t-test) even when the intervals were matched.

Changes in saccade latency with the addition of stimuli are of greater concern because the FF stimulus was flashed just before the saccade and occurred with a delay after the offset of the fixation point (which was the cue for the saccade). Therefore, if the latency of the saccade changed, the time at which the FF stimulus flash occurred would change with respect to saccade onset. Because the FF activity in the FEF is dependent on the proximity to the FF stimulus to the saccade onset (see Sommer and Wurtz 2008), this latency change itself could alter the FF activity. For the neuron shown in Fig. 8A, the added stimuli did alter the saccade latency from 203 ± 22 ms without added stimuli to 225 ± 18 ms with the added stimuli, a mean difference of 22 ms. This 22-ms increase in mean saccade latency could reduce the FF activity and could be the source of the reduced FF activity we see with added stimuli. We therefore performed an additional analysis on the neural activity to determine if this was the case. Figure 8C displays the distribution of the intervals between FF stimulus onset and saccade onset for the example neuron. The two distributions overlap, but note that there was more variability in the interval distribution with added stimuli (black trace) than without them (blue trace). We therefore reanalyzed a subset of trials where this interval was the same in both conditions (within the vertical dashed red lines). For this subset of trials, the presence of the added stimuli still significantly reduced the FF activity by 9.8 spikes/s on average (Fig. 8D, 41% reduction, P < 0.001, 2-tailed t-test). These results were consistent for our neurons in our sample that had sufficient variability in the stimulus-saccade interval for us to perform the above analysis (P < 0.001, 2-tailed t-test, same 4 neurons as above). The reduction was comparable to that seen for all trials as shown in Fig. 3. Importantly, the neuronal activity on the nonoverlapping trials (those outside the dashed red lines) was not significantly different from the neural activity on the overlapping trials (P = 0.64, 2-tailed t-test). We conclude that the change in saccade latency did not produce the reduction in FF activity with added stimuli.

Is the FF activity dependent on both the FF stimulus and the saccade?

The above analysis shows that the FF activity in FEF requires the presence of the FF stimulus but that its latency is fixed to the onset of the saccade. However, the analysis does not demonstrate that the FF activity is not simply a saccade-related response; we show it occurs in association with saccades. We verified that the FF activity was not just a saccade-related response by using control trials embedded in every experiment. In this control, when the monkey made saccades to the target but in the absence of the stimulus, there was no FF activity. Figure 9A shows the example neuron in which the absence of the FF stimulus eliminated the visual activity. This was true for all cells in our sample of neurons that had shifting RFs, as has been demonstrated previously for FEF neurons (Sommer and Wurtz 2008; Umeno and Goldberg 1997). The saccade-only activity within the same window that the FF activity was determined (see methods) was significantly less than the FF activity for the sample of neurons (P < 0.05, 1-tailed t-test). Therefore, the saccade alone was not producing the FF activity. The FF stimulus without the saccade also produced no response. In Fig. 9B for the same neuron, presenting the FF stimulus in the absence of the saccade produced no response, whereas the same stimulus when presented in the RF did produce a visual response. This was also tested on all neurons, with the same result; in no case did the activity when the FF stimulus was presented in the absence of the saccade meet the criteria for a response (see methods). Therefore, the FF activity was dependent on the combination of the FF stimulus and the generation of the saccade.

Fig. 9.

FF activity is not saccade-related activity or a RF response. A: for the example neuron presented in Fig. 3, when the monkey made a saccade without a FF stimulus, the neuron did not respond (black trace), in contrast to the case with the FF stimulus present (blue trace). B: for the same neuron, a FF stimulus flashed while the monkey fixated without making a saccade did not activate the neuron (black trace). This activity is different from that elicited by a visual stimulus placed in the RF (red trace).

DISCUSSION

Visual Salience Effect on Shifting RF Activity

The major explanation of visual stability despite displacement of the visual image with each saccade is that advanced knowledge of the impending saccade is available from an internal copy of the motor command to move the eye (a corollary discharge or efference copy). This advanced knowledge makes it possible to recognize that the displacements are self-generated. First in parietal area LIP (Duhamel et al. 1992) and then in the FEF region of frontal cortex (Umeno and Goldberg 1997), neurons have been shown to become sensitive to visual stimuli at the spatial location that their RF would occupy after the saccade, the FF of the neuron. Most of the experiments studying FF activity have used a highly salient stimulus, an isolated flash against a uniform background, to determine whether a neuron had FF activity. In the present experiments we explored the effect of reducing this salience by adding the onset of other stimuli in the visual field at the same time as the FF stimulus appeared. In a sample of 50 neurons with shifting RFs, we found that 30% of the neurons had reduced FF activity, with a reduction averaging 16%. This is consistent with a reduction in the FF activity observed in an LIP neuron in the experiments of Gottlieb et al. (1998) as described in the Introduction. Thus, in the usual shifting RF experiment, the activity resulting from the onset of the FF stimulus can be regarded as resulting from both the anticipatory nature of the FF stimulus and the relative salience of that stimulus.

Our experiments have a striking similarity to two human psychophysical experiments that studied the effect of attention drawn by an abrupt onset of a stimulus flashed near the time of a saccade (Golomb et al. 2008; Mathot et al. 2010; Mathot and Theeuwes 2011). In one set of these experiments (experiment 3 in Mathot et al. 2010), the subject made a saccade from one point to another and a cue stimulus was flashed just before the saccade, identical to our paradigm. The subject was instructed to remember the cue location to make a discrimination based on a stimulus appearing briefly at that point after the saccade. The discrimination was better when the cue was located at what we refer to as the FF location rather than other areas of the visual field, indicating that attention had shifted to the FF even before the saccade was made. This behavioral measure of a shift of attention to the FF before a saccade parallels our interpretation of the flash in the FF of a neuron benefiting from the salience of the stimulus. At this point, the demonstration of the neuronal changes in the monkey and the attention discrimination benefit in humans provide at least an indication that attention is likely to be involved in both cases.

As in the previous experiments studying the effects of exogenous attention on the response of neurons, our current experiments did not measure attention. We did not have any behavioral measure of the monkey's attention to gauge the magnitude of the added stimuli effect; we inferred the reduction of attention with the addition of stimuli from related behavioral experiments. We think our observations are highly likely to result from the effects of visual attention drawn to the salient stimulus onset because of three related observations. First, psychophysical experiments have shown a decrease in performance with the addition of visual stimuli in the visual field during search (Duncan and Humphreys 1989; Kim and Cave 1999) and on cued attention tasks (such as Kahneman et al. 1983; Wright and Richard 2003). Of course, these search and attention experiments on humans do not directly relate to shifting RFs in monkeys; they simply provide the only guidance available on the consequence of adding visual stimuli. Second, visual search tasks have shown the reduction of neuronal responses in both FEF (Cohen et al. 2009) and LIP (Balan and Gottlieb 2006; Balan et al. 2008) with the addition of visual stimuli in the visual field. Third, our experiments are remarkably parallel to those described above (Golomb et al. 2008; Mathot et al. 2010; Mathot and Theeuwes 2011), which did measure and find exogenous attention effects. None of these experiments, however, can substitute for the needed direct measurement simultaneously of the FF response and the effect of salience on exogenous attention.

Visual Factors Affecting the Magnitude of Salience on FF Activity

The strength of stimulus salience leading to exogenous or onset attention effects in our experiments is almost certainly reduced by two factors. First, the FF stimuli were presented repeatedly in the same region of the visual field, although at multiple locations within this region on successive trials. It is reasonable to expect that the onset effect we saw had habituated at least somewhat over the training and experimental periods preceding the particular neuronal experiment. Added stimuli presented for the first time might produce an even greater reduction in the FF activity with added stimuli. Second, we always placed the added stimuli at a substantial distance from the FF stimulus to minimize direct visual stimulation generated by the stimuli falling in the presumed FF of the neuron. The consequence of this was that the added stimuli were pushed off to the side of the monkey's visual field. This might also change the magnitude of the effect of the added stimuli (Hagenaar and van der Heijden 1986).

Probably the most important questions on the addition of visual stimuli in the shifting field experiments are related to the organization of the FF, particularly the extent of its visual surround and whether the added stimuli fell in a FF suppressive surround. At the start of the experiment, we had established the presence of a suppressive surround and had estimated its extent using the annulus test for the RF (see methods). We then placed the added stimuli about 5° beyond the outer edge of the surround to minimize the direct visual activation of the neuron. The more important question, however, is the size and organization of the FF. In each experiment, it would have been challenging to both map the FF in detail and do the added stimuli experiment, so we relied on information about the organization of the FF from other ongoing experiments. In experiments on FEF (Joiner WM, Cavanaugh J, and Wurtz RH, personal communication), we found that the beginning of the surround for the RF and FF were within a few degrees of each other. In LIP neurons (Phillips and Goldberg 2010), a comparison of the RF and FF showed that the FFs were somewhat more narrowly tuned than RFs. Both observations imply that there may be differences in the RF and FF, but the differences are small compared with our placement of the added stimuli well beyond the estimated surround. We therefore interpret the reduction of the FF response with the added stimuli as a reduction of salience rather than an effect of a suppressive surround.

Our observations emphasize the visual modulation of FF activity. This is the second component of the FF activity, the other being the temporal proximity to the saccade and its accompanying corollary discharge (Kusunoki and Goldberg 2003; Sommer and Wurtz 2008). Thus the FF activity results from the conjunction of the visual stimulus in the right part of the visual field and the corollary discharge associated with the right saccade directed to that part of the visual field. It is not a FF visual response, but FF activity. The present experiments emphasize that the magnitude of the FF activity is dependent on the characteristics of the stimulus, particularly the salience of the stimulus, just as it is dependent on the temporal proximity to the saccade. The difference in the composition of the RF visual response and the FF activity may account for the absolute differences in the size of the effect of added stimuli on the RF responses and FF activity.

Salience and Its Relation to Visual Stability

Change blindness experiments emphasize the key role that attention plays in determining what we see in the visual world. This attention has also been shown to be relevant for our stable visual perception; attended objects are critical for maintenance of visual stability (Mathot and Theeuwes 2011). The implication of this for visual stability is that stability might not be maintained for the entire visual scene but just for attended parts.

One possibility is that stability is maintained just for those regions in and around the fovea to which attention is directed during each visual fixation (see discussion in Wurtz et al. 2011). Such a concentration of stabilization near the target of the saccade has been demonstrated for the suppression of target motion during saccades (Deubel et al. 1996, 2002). If the shifting RFs are related to visual stability, then they too might be expected to have a higher frequency near the center of the visual field. Our population of 171 FEF neurons sampled across varying eccentricities within the central 35° of the visual field seemed adequate for addressing this question. We found no evidence of a difference in frequency of shifting RFs with eccentricity. The proportion of neurons with shifting RFs tracked the proportion of neurons with visual RFs with remarkable precision (Fig. 5). We also found no systematic difference in the magnitude of the FF activity with increasing eccentricity (Fig. 6A). The limitation to these observations is that none were made within the fovea so that if the frequency of FF activity in the fovea soars, we would have missed it. Within the visual field studied, our results in FEF are consistent with the finding in LIP that there was no difference in the strength of FF activity in neurons with central, intermediate, and peripheral RFs (Heiser and Colby 2006). In both FEF and LIP, we have no evidence that the shifting receptive fields are concentrated in the central visual field.

Another possibility is that stability across saccades is maintained for attended stimuli regardless of the region of the visual field in which they fall. Stimulus salience would draw attention and be included in what remains stable during a saccade. Our finding of a larger magnitude of FF activity in FEF when the stimulus is a salient one is consistent with that possibility. We did not have enough data to determine whether the salience effect on the FF or its magnitude was related to eccentricity in the visual field (Figs. 5 and 6B).

In summary, so far as FF activity contributes to visual stability, our evidence indicates that the salience of an object is probably more important than its location in the visual field for determining whether the object is included in what is perceptually stable across saccades.

GRANTS

This work was supported by the National Eye Institute Intramural Research Program at the National Institutes of Health.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the author(s).