Saccades during Object Viewing Modulate Oscillatory Phase in the Superior Temporal Sulcus

Abstract

Saccadic eye movements (SEMs) are the primary means of gating visual information in primates and strongly influence visual perception. The active exploration of the visual environment (“active vision”) via SEMs produces suppression during saccades and enhancement afterward (i.e., during fixation) in occipital visual areas. In lateral temporal lobe visual areas, the influence, if any, of eye movements is less well understood, despite the necessity of these areas for forming coherent percepts of objects. The upper bank of the superior temporal sulcus (uSTS) is one such area whose sensitivity to SEMs is unknown. We therefore examined how saccades modulate local field potentials (LFPs) in the uSTS of macaque monkeys while they viewed face and nonface object stimuli. LFP phase concentration increased following fixation onset in the alpha (8–14 Hz), beta (14–30 Hz), and gamma (30–60 Hz) bands and was distinct from the image-evoked response. Furthermore, near-coincident onsets of fixation and image presentation—like those occurring in active vision—led to enhanced responses through greater phase concentration in the same frequency bands. Finally, single-unit activity was modulated by the phase of alpha, beta, and gamma oscillations, suggesting that the observed phase-locking influences spike timing in uSTS. Previous research implicates phase concentration in these frequency bands as a correlate of perceptual performance (Womelsdorf et al., 2006; Bosman et al., 2009). Together, these results demonstrate sensitivity to eye movements in an object-processing region of the brain and represent a plausible neural basis for the enhancement of object processing during active vision.

Introduction

During everyday behavior, primates obtain visual information by directing gaze toward objects in the environment, primarily via saccadic eye movements (SEMs). The foveation of salient objects comes at the cost of adding noisy motion to the retinal input. The effect of a moving image on the retina is mitigated through saccadic suppression (i.e., the reduction of visual sensitivity during saccades) (Latour, 1962; Burr et al., 1994). The neural basis of saccadic suppression has been explored in the LGN (Reppas et al., 2002) and motion-sensitive cortex (Thiele et al., 2002), and some of these studies also report that visual responses following saccades can be enhanced (Ibbotson et al., 2007, 2008; Cloherty et al., 2010). For example, in primary visual cortex, fixation onset in darkness leads to a resetting of oscillations to a phase associated with high firing rates (Rajkai et al., 2008), and during visual stimulation, fixation onset leads to increased firing rates (Gallant et al., 1998; MacEvoy et al., 2008), spike synchronization (Maldonado et al., 2008; Ito et al., 2011), and sparseness (Vinje and Gallant, 2000). Whereas both saccadic suppression and enhancement following fixation are seen in many visual areas (for review, see Ibbotson and Krekelberg, 2011), they are poorly understood in object-selective visual areas. For example, it remains unclear whether visual and visuomotor signals interact to facilitate object processing in the temporal lobe.

Temporal lobe regions such as the upper-bank superior temporal sulcus (uSTS) and inferotemporal cortex (IT) are critical for face and object recognition (Bruce et al., 1981; Horel et al., 1987; Perrett et al., 1992; Horel, 1993; Afraz et al., 2006). The uSTS is well connected with oculomotor networks, unlike the connectivity seen from IT (Seltzer and Pandya, 1989, 1994; Yeterian and Pandya, 1989). Furthermore, deficits in visually guided saccades were seen following lesions to uSTS but not IT (Scalaidhe et al., 1995, 1997). Previous research has shown that saccades modulate activity in IT and hippocampal regions under conditions of darkness (Ringo et al., 1994), electrical stimulation (Sobotka et al., 2002), and viewing of synthetic stimuli (Sobotka et al., 1997; Purpura et al., 2003), although modulation in uSTS has not been measured. Moreover, it is unknown whether modulation in the temporal lobe would be useful during active viewing of naturalistic images, by showing enhanced image-evoked responses following fixation, or through phase-locking of activity to fixation onset.

Here, we ask whether saccades modulate uSTS activity during naturalistic object viewing. In accord with reports from early visual areas, we will measure whether the phase of oscillations becomes aligned following saccades, and whether this depends on the timing of image onset relative to fixations. Finally, for oscillations showing phase-locking to fixation, we will determine whether these same oscillations modulate single-unit spiking (Rajkai et al., 2008).

Materials and Methods

Surgical procedures.

All procedures including surgery, electrode localization, task, and recordings have been described in detail (Hoffman et al., 2007). Briefly, subjects were two adult male rhesus macaque monkeys (Macaca mulatta). A recording chamber of 19 mm inner diameter was surgically implanted, centered directly above the auditory cortex in the left hemisphere in both subjects (monkey P: AP +8.0, ML +18.0; monkey K: AP +7.5, ML +22.5). Preoperative localization of auditory cortex was determined by magnetic resonance imaging. Location was determined postoperatively by functional criteria within auditory cortex and subsequently through gray/white matter transitions for the uSTS. Experiments were conducted with the approval of local authorities (Regierungspraesidium) and in accordance with the guidelines of the European Community (EU VD 86/609/EEC) for the care and use of laboratory animals. Surgery was performed and data were collected at the Max Planck Institute for Biological Cybernetics in Tübingen, Germany.

Task design and behavioral analysis.

Monkeys viewed images of conspecific faces or nonface objects presented centrally on a CRT monitor. A trial began with a centrally located dot, which the monkey had up to 2 s to fixate. If the central eye position was maintained within 2° of the dot for 500 ms, the dot was replaced by one of the images, which remained on the screen for another 500 ms before the image disappeared and juice was delivered. This paradigm allowed for monkeys to make small saccades within the boundary of the fixation window. At least 10 repetitions of the 36 images were presented, for a minimum of 360 trials per experimental session.

Saccade detection procedure.

Eye movement data were sampled at 200 Hz using a scleral search coil system (CNC Engineering). We detected saccades using a modification of the method described by Krauzlis and Miles (1996). We took the difference in neighboring time points in the x- and y-position time series as an estimate of instantaneous velocity. The radial velocity is then defined as the absolute value of this velocity vector. The time series is first coarsely segmented into epochs of fixations and saccades, using a radial velocity criterion (35°/s). The onset of saccades and fixations (saccade offsets) are then iteratively refined by checking if time points neighboring the onset and offsets of saccades exceed a radial acceleration criterion (1000°/s) in the appropriate direction (positive for saccade onset; negative for saccade offset). We imposed additional spatial and temporal criteria to remove false detections: saccades lasting <10 ms or with amplitudes of <3 arc min were discarded. This resulted in the rejection of 5.5% of putative saccades, on average, per session (SD, 3%), with a maximum of 12.5% putative saccades rejected in one session.

Neural recordings.

We recorded simultaneously from up to eight electrodes within the STS, auditory cortex, or both sites concurrently using a custom-made staggered 4 × 2 electrode drive. Only data from the STS are presented in this paper. This array covered 12 mm in the anterior–posterior axis and 1.5 mm in the medial–lateral axis, holding eight glass-coated tungsten electrodes, with impedances between 1 and 3 MΩ measured at 1 kHz (Alpha Omega). The cranial implant chamber was used as the electrical reference for each electrode. Signals were amplified and bandpass filtered between 1 Hz and 5 kHz (Alpha Omega), and continuously recorded at a sampling rate of 20.8 kHz (National Instruments; BNC-2090).

Signal processing.

Unless otherwise specified, all off-line behavioral and neuronal data analysis was performed in MATLAB (MathWorks). Continuous recordings were bandpassed filtered off-line to obtain local field potentials (LFPs) (1–300 Hz) using a second-order Butterworth filter with zero-phase adjustment. For single-unit activity (SUA), the continuously recorded signals were loaded into an off-line spike sorting program and isolated based on spike peak and valley amplitude, energy, and the first three principal components of the wave shape (Plexon). SUA spike times were smoothed with a Gaussian kernel (σ = 2.5 ms) on a single-trial basis before averaging.

Visual responses were evaluated by averaging activity (LFP or SUA) relative to image onset, separately for each image category. We considered a recording site visually responsive if it showed mean absolute voltage levels >3 z-scores for >10 ms following image onset for at least one category, relative to the 100 ms baseline period preceding image onset. By this criterion, all 100 uSTS LFP recordings and 18 of 25 of SUA recordings were visually responsive.

The phase-concentration spectrogram was estimated using custom scripts based on the Chronux signal processing toolbox for MATLAB (http://chronux.org) (Bokil et al., 2010). We used a sliding window of 256 ms in 1 ms steps. We opted to use a single Hann taper instead of multiple Slepian tapers, as the goal of this exploratory step was to identify frequency bands for subsequent analysis and a single Slepian has more spectral leakage than a Hann taper (Bosman et al., 2009).

To obtain frequency-specific estimates of instantaneous phase of the LFP without loss of temporal resolution, we bandpass filtered our LFP signal into several constituent frequency bands using a bidirectional fourth-order Butterworth filter. Based on the peaks of activity seen in the spectrogram, our nominal frequency band decomposition is as follows: alpha (8–14 Hz), beta (14–30 Hz), and gamma (30–60 Hz).

Mean subtraction of image-evoked activity.

To extract neural activity that was locked to visual fixations after accounting for image onset evoked responses, we performed a mean subtraction. The average evoked response to image onset was calculated separately for each electrode and stimulus example combination, to account for differences in preferred stimuli (see Fig. 1). Average activity was then subtracted from all single trials in the time domain before aligning to fixation onset. Only fixations that occurred 150–245 ms from image onset were analyzed, and only sessions that had >10 fixations in this epoch were included in analyses locked to fixation onset. This resulted in the exclusion of four LFP recordings from one session in monkey KZ, resulting in a population of 96 visually responsive LFP and 18 visually responsive SUA recordings available for fixation-related analysis. For the comparison of LFP latency-to-peak in the residual response, the same trials from which we took fixation-aligned activity were used for comparison in the image onset reference frame. Only electrode sites that had a peak phase concentration after event onset in both fixation and image onset reference frames were included in this latency analysis. This resulted in 63, 87, and 71 of 96 potential electrodes in the alpha, beta, and gamma bands, respectively.

Figure 1.

Fixation timing and neural category selectivity. A, Distribution of fixation times relative to image onset, for both monkeys, across all sessions. Bin width is 5 ms. The y-axis is bin count, in number of fixations. For all figures, the vertical solid black line at time 0 indicates image onset, and the x-axis is time relative to image onset, in milliseconds. B, Category-selective LFP responses. The y-axis represents average voltage, in microvolts. The vertical black bar denotes 50 μV. The red traces are mean responses to faces, and the blue traces are mean responses to objects. The top and middle trace pairs are example trial-averaged responses from two electrode sites from the same recording session. The bottom trace is the population average, across both monkeys (N = 100 electrodes). Positive deflections indicate positive voltage polarity. The shaded regions indicate 95% confidence intervals, estimated by bootstrapping (10,000 samples). C, Category-selective SUA activity. The red traces are mean responses to faces, and the blue traces are mean responses to objects. The top and middle trace pairs are example trial-averaged responses from different sessions. For the top two pairs of traces, the y-axis depicts firing rate, in hertz. The top vertical black bar denotes 40 spikes per second. The bottom trace is the population average, across both monkeys (N = 18 visually responsive neurons). For the bottom population trace, the y-axis depicts mean Z-scored firing rates (−100 to 0 ms relative to image onset). The bottom vertical black bar denotes 2 SDs. The shaded regions indicate 95% confidence intervals, estimated by bootstrapping (10,000 samples).

Statistical assessment of residual activity.

Hypothesis testing on whether residual activity was different from zero was accomplished by a nonparametric permutation technique that robustly accounts for multiple comparisons across time (Womelsdorf et al., 2006; Maris and Oostenveld, 2007; Maris et al., 2007). Under the null hypothesis that the difference in mean-expected and observed activity was random, null distributions of the maximal and minimal expected value of residual activity were created for each electrode site, by randomly reassigning the observed and mean-expected responses on a trial-by trial basis over 10⁴ permutations. This is equivalent to flipping a fair coin on each permutation of every trial and multiplying the residual by −1 when the coin shows “heads.” Taking the maximal and minimal values across the entire time series from each permutation controls the global false-alarm rate for multiple comparisons across time. The global error rate was set to p = 0.05, meaning the 2.5th and 97.5th percentiles were taken from the minimal and maximal null distributions, respectively, and used as the multiple-test-corrected 95% confidence interval for the residual activity. All time-local error bars shown are 95% confidence intervals, strictly for visualization purposes, and based on 10⁴ permutations. This method was used for both LFP and SUA residual activity.

Phase concentration.

All circular statistical analyses were performed with the circular statistics toolbox for MATLAB (Berens, 2009). We define phase concentration as the mean vector length derived from all the phases across events for a given electrode site, at a given time point (Rajkai et al., 2008). This is equivalent to setting all complex Fourier coefficients to unit length before averaging across trials, also referred to as intertrial coherence (Bosman et al., 2009).

LFP phase estimation bias correction for spike-phase coupling.

As with previous reports from forebrain recordings in mammals, we found that waveform asymmetry led to biased estimates of phase from the Hilbert transform (Siapas et al., 2005; Sirota et al., 2008). This bias in phase estimation manifests as a nonuniform prior distribution on phase over the entire data set, which violates the assumptions of parametric circular statistics and confounds all analyses based on these paradigms. We therefore implemented a method to effectively match the nonuniform phase histograms to produce a flat, uniform prior (Siapas et al., 2005) for our spike-phase coupling analysis. This is accomplished by calculating the empirical cumulative probability function of the original phases; for an original phase at a given cumulative probability level x, the bias-corrected phase is equal to 2πx − π. We performed this correction on an electrode-by-electrode basis, for all frequency bands.

Spike-LFP phase locking.

After correcting for prior phase bias in the LFP, we used Rayleigh's test for nonuniformity of a circular distribution, equivalent to the mean vector length, to determine whether there was a preferred phase of firing of single units across different frequency bands of the LFP. This was first calculated using the LFP recorded from the same electrode as the neuron in question, and then on an adjacent electrode to control for possible spike contamination of the LFP. Highly similar results were obtained for both same and adjacent reference electrode analyses, and the pattern of results and main findings were unchanged. The results depicted in Figure 5 are from the adjacent-electrode analysis. The significance criterion was set to 10⁻⁴ to account for multiple comparisons across frequency bands.

Figure 5.

Preferred phases of firing among uSTS neurons. A, Alpha (8–14 Hz) preferred phases. Each dot represents results from a single neuron. Data are presented in polar form. The angle, counterclockwise from phase = 0 (far right), represents the preferred phase of firing of a neuron. Eccentricity from the center represents the magnitude of phase locking (Rayleigh's r statistic). The red symbols are visually responsive neurons, and the gray dots are visually unresponsive neurons. The dots represent neurons with a significant preferred phase of firing (p < 10⁻⁴, Rayleigh's test for circular nonuniformity), and x's are nonsignificantly phase-locked neurons. The dashed red line depicts the mean preferred phase of firing for all significantly phase-locked visually responsive neurons. The thin lines are for not significant mean preferred phases of firing, and the thick lines are for significant mean preferred phases of firing (p < 10⁻⁴, second-order Rayleigh's test). B, As A, but for the beta (14–30 Hz) band. C, As A, but for the gamma (30–60 Hz) band.

Interaction between stimulus onset and fixation onset reference frames.

Individual trials of residual activity (voltage for broadband 1–300 Hz LFP, phase values from Hilbert transform for band-limited LFP) were sorted into 5 ms bins according to the relative timing between fixation onset and image onset (see Fig. 4, y-axis). This bin width was the lower bound set by the 200 Hz temporal resolution of our eye tracker. We then smoothed our results with a boxcar kernel that was 1 ms wide along the x-axis (“time from image onset”) and 20 ms long along the y-axis (“fixation time from image onset”).

Figure 4.

Interaction of image-evoked and fixation-evoked activity. A, Residual evoked response, in millivolts, as a function of absolute time from image onset and the relative timing of image onset and visual fixations. The horizontal axis depicts the absolute time from image onset, in milliseconds. The vertical axis depicts the relative timing between fixation onset and image onset, in milliseconds. Negative y-axis values indicate fixation preceding image onset. The vertical white solid line indicates the time of image onset. The oblique white solid line indicates the time of fixation onset. The gray dashed vertical lines enclose the time interval used to calculate the image-evoked modulation functions in B. The color bar represents mean residual voltage. The cooler colors are lower residual voltage values, and the warmer colors are high residual voltage values. The gray rectangular region, spanning fixation times from image onset 80–160 ms, are omitted from the plots and all analyses due to insufficient sample size. B, Image-evoked response modulation functions. The y-axis depicts the relative timing between fixation onset and image onset, in milliseconds. The x-axis represents mean rectified residual voltage, in absolute millivolts. The gray rectangle represents relative fixation times at which we had insufficient samples to compute reliable statistics. C, Residual phase concentration in the alpha (8–14 Hz) band, as a function of absolute time from image onset and the relative timing of image onset and visual fixations. The horizontal axis depicts the absolute time from image onset, in milliseconds. The vertical axis depicts the relative timing between fixation onset and image onset, in milliseconds. Negative y-axis values indicate fixation preceding image onset. The vertical white solid line indicates the time of image onset. The oblique white solid line indicates the time of fixation onset. The gray dashed vertical lines enclose the time interval used to calculate the image-evoked modulation functions in D. The color bar represents phase concentration (Rayleigh's r). Cooler colors are lower residual phase concentration values, and warmer colors are high phase concentration values. The gray rectangular region, spanning fixation times from image onset 80–160 ms, are omitted from the plots and all analyses due to insufficient sample size. D, Image-evoked response modulation functions. The y-axis depicts the relative timing between fixation onset and image onset, in milliseconds. The x-axis represents phase concentration magnitude (Rayleigh's r). The gray rectangle represents relative fixation times at which we had insufficient samples to compute reliable statistics. E, As C, but for the beta (14–30 Hz) band. F, As D, but for the beta (14–30 Hz) band. Note the different x-axis phase concentration scale. G, As C, but for the gamma (30–60 Hz) band. H, As D, but for the gamma (30–60 Hz) band. Note the different x-axis phase concentration scale.

Results

Eye movement behavior

Both monkeys typically completed 360 free viewing trials per session (N = 31 session total; 27, monkey PR; 4, monkey KZ), making on average 2.3 fixations during the 500 ms image presentation epoch. Fixations were not evenly distributed throughout the duration of the epoch; rather, they occurred principally in two time windows: 0–80 and 150–500 ms following image onset (Fig. 1A). The first distribution of early fixations was continuous with a slow increase in the probability of fixations extending 100 ms before image onset (Fig. 1A). These prestimulus onset and rapid poststimulus onset fixations were most likely predictive of image onset and not in response to the image, given the regular 500 ms interval between fixation of the centrally presented spot and image onset.

Periodicity in the saccade latency distribution was also apparent in the interfixation intervals, which showed a marked peak at ∼250 ms, corresponding to a frequency of ∼4 Hz (Fig. 1A, inset). The saccade rate of 4 Hz was also seen for saccades made at the beginning of each trial, when monkeys made uninstructed saccades for up to 2 s before fixating a central spot; thus, the overall relative timing of saccades appeared to be constant across different types of saccades during the task.

Visual responsiveness and selectivity of neural activity

We recorded LFPs from 100 electrodes sites (85 sites from monkey PR; 15 sites from monkey KZ) and isolated a total of 25 single units from those electrode sites (21 from monkey PR; 4 from monkey KZ). Our LFP responses were category selective, as demonstrated by two example recordings from a session in monkey PR (Fig. 1B, top and middle trace pairs) and the population average over both monkeys (Fig. 1B, bottom trace pairs). All 100 of our recording sites showed LFP responses to visual stimuli (see Materials and Methods). Visual responsiveness and category selectivity were evident in SUA as well (Fig. 1C). Two examples of SUA are shown in the top and middle trace pairs of Figure 1C, demonstrating significant rate increases to both faces and objects, as well as category selectivity. Overall, 18 of 25 (72%) units showed an increase in mean firing rate to faces and/or objects (Fig. 1C, bottom trace pair) (see Materials and Methods).

Evoked responses to fixation onset

Neural activity in response to eye movements was calculated by taking segments of the LFP ±255 ms from the time of fixation, for all eye movements occurring 150–244 ms after image onset. This time window captured most of the image-guided eye movements while still leaving enough time following the eye movement to evaluate the time course of the neural response. One possible confound to any fixation-locked response observed during this time window was the nonuniform distribution of fixation times during stimulus presentation, which had a mode at 270 ms from image onset. This meant that any putative fixation-related activity may in fact include the LFP image onset evoked response. To account for this possibility, we performed a subtractive normalization procedure in the time domain (see Materials and Methods), which is used to factor out stimulus-locked activity when analyzing LFP data.

The mean subtraction procedure is illustrated in Figure 2A for the same electrode site shown in the top trace of Figure 1B. For each trial that contained a fixation in our window of interest, before aligning the data to that fixation time point, we subtracted the mean response to the stimulus example that was shown on that particular trial. This left us with a mean-subtracted, or residual, voltage time series. Averaging over all LFP residuals gave us the mean response to fixation onset during free viewing, having factored out the initial stimulus-locked response.

Figure 2.

Residual neural activity aligned to fixation onset. A, Fixation-locked LFP activity at one electrode site. Mean responses are shown in microvolts as a function of time from fixation onset. The black trace depicts the observed fixation-related activity. The pink trace depicts the mean of the expected (evoked) response to each image example, but aligned according to fixation times. The green trace is the mean-subtracted “residual” activity, or the mean of differences of single-trial observed activity and that expected from the within-example mean. The dashed black vertical lines denote median time of image onset for that recording session. The horizontal black solid lines indicate the nonparametric 95% confidence intervals derived from permutation testing, which accounts for multiple tests across time, for the null hypothesis that the mean-subtracted activity is equal to zero. The shaded regions indicate 95% confidence intervals, estimated by bootstrapping (10,000 samples); shown strictly for visualization purposes. The LFP activity comes from the same electrode site that was used in the top trace of Figure 1B. B, Population summary of fixation-locked LFP activity (N = 96 electrode sites). The green trace depicts the average mean-subtracted LFP activity, in microvolts, with scaling indicated on the left vertical axis. The shaded region indicates 95% confidence intervals, estimated by bootstrapping (10,000 samples), shown strictly for visualization purposes. The gray histogram depicts the percentage of electrodes with mean-subtracted voltage values significantly less than (<0) or greater than (>0) mean voltage values, with scaling indicated on the right vertical axis. The dashed black vertical lines denote median time of image onset over all sessions. C, Phase concentration spectrogram for the electrode site shown in A. The y-axis shows frequency, in hertz. The color bar indicates phase concentration, which is decomposed by frequency band (y-axis; in hertz) and as a function of time from fixation onset (x-axis; in milliseconds; using time windows of 256 and 1 ms step sizes). D, Population average phase concentration spectrogram (N = 96). The conventions are as in C, but based on the spectrogram resulting from the average mean-subtracted LFPs. E, Fixation-locked SUA activity example, as per Figure 2A, but mean firing rate is shown in hertz as a function of time from fixation onset. F, Population average of fixation-locked SUA activity, as per Figure 2A, but mean firing rate is shown in hertz as a function of time from fixation onset.

In Figure 2A, the mean-subtracted activity is shown in green, indexing fixation-related activity dissociated from stimulus onset-locked responses. Although it is not evident from the figure, the mean subtraction procedure was done in paired fashion within each trial. To assess the significance of voltage fluctuations in the residual activity, we used a paired-sample, nonparametric permutation test to generate confidence intervals that were corrected for multiple tests across time (see Materials and Methods). The 95% confidence intervals for the null distribution are shown as the solid black horizontal lines. For this example electrode, the mean-subtracted activity is relatively flat before fixation onset but shows a short-latency trough (∼40 ms) followed by a peak (∼80 ms) following fixation onset.

The prevalence of mean-subtracted fixation-locked activity is summarized in Figure 2B, for both the population average residual activity (green trace), and histograms of statistically significant peaks and troughs (gray histograms; sorted by polarity). Consistent with the response from the example electrode site, the population response shows a prominent negativity at 40 ms following fixation onset both in the average mean-subtracted fixation response, and the histogram of significant troughs. At 38 ms after fixation onset, >55% of electrodes sites showed significant fixation-locked activity.

Although the focus was on LFP activity, single units could also be influenced by fixations. Pooling all single-trial fixation-locked responses across all neurons (N = 2647 trials) revealed a significant increase in population firing rate from 16 to 32 ms after fixation (Fig. 2F), corresponding in time to the negative slope at the onset of the fixation-locked LFP. Only 1 of 25 cells (1 of the 18 visually responsive units), however, showed significant changes in activity, according to our nonparametric criterion (see Materials and Methods). One unit showing nonsignificant residual fixation-locked activity is shown in Figure 2E, exemplary of the population.

Phase concentration following fixation onset

Neural modulation by attention (Lakatos et al., 2009; Schroeder and Lakatos, 2009) and saccadic eye movements (Rajkai et al., 2008; Melloni et al., 2009; Schroeder et al., 2010) is seen in early sensory areas as an increase is the consistency of LFP phase over trials. In light of these results, we examined whether LFP phase in uSTS would show an increase in consistency across trials (i.e., phase concentration) when locked to fixation onset.

We calculated phase concentration spectrograms, which are analogous to the phase-locking value or intertrial coherence, for fixation-locked LFP activity. Unlike evoked LFP responses, this technique allows us to quantify phase consistency, independent of changes in spectral power. Figure 2C shows the phase concentration spectrogram for the same electrode site shown in Figure 2A and the top of Figure 1B. The most pronounced phase concentration occurs following fixation onset, with peaks in the alpha (∼12 Hz), beta (∼25 Hz), and gamma (≥30 Hz) bands. This pattern of phase concentration was representative of that seen across the population, as evidenced in the population average phase concentration spectrogram (Fig. 2D).

Contrasting phase concentration following image and fixation onset

Whereas the presence of fixation-locked activity is very clear, it could be a consequence of fixating another location on the image, thereby reflecting another image-evoked response. To address this issue, we examined the time course of phase concentration aligned to image onset (i.e., not saccade-locked) and aligned to fixation onset (i.e., saccade-locked). Monkeys were able to make small (<4°) SEMs during the fixation period, allowing us to examine how the image-evoked response was modulated by SEMs that occurred shortly before or after image onset.

Figure 3A–C illustrates the time courses of population-averaged phase concentration for fixation- and image onset-locked activity, for the alpha, beta, and gamma bands, respectively. Whereas the overall magnitude of phase concentration is stronger for image onset, the latency to peak population phase concentration is shorter in the fixation onset reference frame. Figure 3G–I shows the histograms of peak latency differences between the fixation and image onset phase concentration, and the median difference across the population. Postfixation peak phase concentration had a shorter latency than that seen following image onset, in all three frequency bands (paired Wilcoxon's signed-rank tests, p < 10⁻⁴ for alpha and beta; p = 0.012 for gamma). Thus, neural responses differ under these two conditions: when foveal stimulation occurs through image onset and not aligned to saccades, and when foveal stimulation results from image-guided eye movements. If the latter response is unrelated to eye movements per se and is merely a second (albeit different) visually evoked response following the initial “image onset” response, then one would predict no change in image-evoked response as a function of when the eye movement occurred.

Figure 3.

Phase concentration aligned to image onset and fixation onset. A, The black traces depict the population average phase concentration in the alpha band (8–14 Hz), aligned to image onset. The green traces depict the population-average phase concentration of the residual activity aligned to fixation onset. The x-axis shows time from event (fixation or image onset). The y-axis depicts the magnitude of average phase concentration (Rayleigh's r statistic). The shaded regions represent 95% confidence intervals for the population mean, estimated by bootstrapping (10,000 samples). B, As in A, but for the beta band (14–30 Hz). C, As in A, but for the gamma band (30–60 Hz). D, Scatterplot of latency-to-peak phase concentration, for image-locked activity and fixation-locked residual activity in the alpha (8–14 Hz) band at each electrode site. The x-axis depicts time from image onset, and the y-axis depicts time from image onset. The vertical and horizontal axes depict the time of fixation/image onset, respectively. The oblique dotted gray line represents the line of equality (equal latency for both reference frames). Each dot represents latency from a single electrode site. Electrode sites showing postevent peaks in both reference frames are shown in gray and are included in the latency comparison analysis (G–I). Electrodes showing preevent peaks in either reference frame are shown in black and are not included in the latency comparison analysis (G–I). E, As D, but for the beta (14–30 Hz) band. F, As D, but for the gamma (30–60 Hz) band. G, Histogram of differences between image-evoked and fixation-evoked peak phase concentration in the alpha (8–14 Hz) band, for electrodes showing both postfixation onset and postimage onset peak latencies (G–I, gray dots in top right quadrant). The x-axis depicts peak latency differences (image onset minus fixation onset latencies). The negative values indicate shorter latency peak responses in the image-locked reference frame, and the positive values indicate shorter latency peak responses in the fixation-locked reference frame. Histogram bin sizes are 25 ms. The y-axis depicts frequency of occurrence, in number of electrode sites. The solid black vertical line depicts no difference in peak latency. The dashed black line represents the median peak latency difference of the distribution. The insets are the number of electrodes included in the analysis (N) and the p value from a paired Wilcoxon sign-rank test on latency values. H, As G, but for the beta (14–30 Hz) band. I, As G, but for the gamma (30–60 Hz) band.

Interaction of image and fixation onset responses

After taking the residual response for each trial, trials were sorted according to the time of fixation relative to image onset. Due to insufficient sampling of fixations occurring 80–160 ms following image onset (<10 trials each over all sessions for these time bins), we have excluded this epoch from our analysis and figures. Even after subtracting out the mean image-evoked effect, we see evoked (1–300 Hz) residual responses, mediated by band-limited residual phase concentration, which is greatest when fixation and image onset coincide (Fig. 4A,C,E,G), suggesting a supra-additive effect between these two reference frames.

The timing of eye movements relative to stimulus onset has been known to both enhance and suppress visual processing; therefore, we quantified how the mean rectified evoked response and phase concentration varied as a function of the relative timing from image onset (Fig. 4B,D,F,G), specifically during the time period when the first image-evoked responses are seen, 50–100 ms after image onset (Fig. 4A,C,E,G). The interaction was clearly evident in the LFP residual fixation-evoked response, as well as within the frequency bands we previously observed residual fixation-locked phase concentration. The effect was stronger in alpha and beta bands than in the gamma band (note the difference in phase concentration scale for the gamma band plots). Despite this difference, the overall evoked response and all three frequency bands show the greatest magnitude of residual activity (1) during the image-evoked response time window and (2) on trials in which fixation and image onset times were nearly coincident, as occurs during active perception.

Relationship of oscillatory phase to spiking activity

To address whether the increased phase concentration observed in the alpha, beta, and gamma frequency bands can ultimately affect spiking behavior, we calculated LFP phase at the time of action potentials for each neuron (after correcting the overall phase distribution for any potential bias; see Materials and Methods), and then used the Rayleigh test for departures from a nonuniform circular distribution to quantify the extent to which LFP phase was correlated with the timing of action potentials (Fig. 5A–C). For each cell, the preferred phase of firing is indicated by the angle on the circular phase plot and the strength (Rayleigh's r, mean vector length) is indicated by the eccentricity from the center. Overall, phase locking increased with higher frequency bands. In the alpha band, 9 of 18 (50%) visually responsive neurons were phase locked, and 5 of 7 (71%) of nonresponsive units were as well. In the beta band, 16 of 18 (89%) of visually responsive and 5 of 7 (71%) nonresponsive single units were phased locked to the LFP. In the gamma band, 100% of neurons, visually responsive or not, were phase locked to the field potential. Across frequency bands, the mean preferred phase fell between −π and −π/2, and no cell showed a preferred phase of firing in the opposite quadrant (in the 0–π/2 range), suggesting that phase concentration across sites could also influence spike synchronization.

Discussion

Category-selective visually evoked responses

In agreement with previous electrophysiological studies (Bruce et al., 1981; Baylis et al., 1987; Perrett et al., 1992), we demonstrated that LFPs and SUA in uSTS are driven by visual stimuli and are selective for face and object categories at the single-neuron and population levels (Fig. 1). The same LFP electrode sites that show fixation-locked activity during free viewing of naturalistic stimuli (Fig. 2), and the single neurons whose firing rates they modulate (Fig. 5), are also highly selective for object category. Indeed, the region we recorded from likely plays a role in object recognition (Horel et al., 1987; Perrett et al., 1992; Horel, 1993).

Fixation-aligned residual activity

The main purpose of this study was to assess whether saccadic eye movements modulate neuronal activity in uSTS. We found a short onset latency (∼40 ms) LFP response that continued to fluctuate 100–200 ms after stimulus onset. The response was present in >55% percentage of our visually responsive electrode sites (Fig. 2A,B). This LFP activity was mediated by phase-locking of oscillations in the alpha, beta, and gamma frequency bands (Fig. 2C,D). Single-unit residual activity was generally weak on a single-neuron basis (Fig. 2E) but showed an increase in firing rate following fixation at the population level (Fig. 2F). The short latency of the firing rate changes, on the order of 20–30 ms, suggests that it may be driven by extraretinal signals such as those from oculomotor circuits.

To further demonstrate that fixation-locked activity was not a mere recapitulation of image-evoked responses, we compared the latency to peak phase concentration in our frequency bands of interest. All three frequency bands showed a shorter time to peak phase concentration when aligned to fixation onset, relative to image onset, at the population level (Fig. 3). This effect was weaker, but significant, in the gamma band. The magnitude of phase concentration also differed across conditions, with image onset evoked phase concentration exceeding the magnitude of the fixation-evoked response. Our results are consistent with recent reports on the opposing relationships of alpha band activity to sensory responsiveness in early versus late cortical areas (Bollimunta et al., 2008, 2011).

Interactions between image-aligned and fixation-aligned activity

Saccadic suppression has been described behaviorally (Latour, 1962; Burr et al., 1994), and at the neural level (Thiele et al., 2002; Bremmer et al., 2009), but few studies have focused on its corollary: the enhancement of visual processing following a saccade (for review, see Ibbotson and Krekelberg, 2011). We demonstrated that visual inputs and fixation-locked activity in uSTS show a supra-additive interaction. Even after subtracting the mean response to image onset, trials in which saccades ended around the time of image onset led to significant residual activity during the time period when the evoked response typically emerges (50–100 ms after image onset; Fig. 4). This residual activity, mediated by phase concentration of ongoing oscillations primarily in the alpha and beta ranges, was greater than residual activity observed for fixations occurring 150–245 ms after image onset. The changes in the gamma range were more modest, suggesting that the effects are not broadband, although they are short-lived (i.e., within 200 ms of fixation or approximately two alpha cycles).

Sobotka et al. (2002) reported a similar supra-additive interaction in IT and hippocampal evoked responses, when electrical microstimulation was delivered shortly following the end of a saccadic eye movement. These results are consistent with previous studies from early visual areas (Vinje and Gallant, 2000; Ibbotson et al., 2007, 2008; MacEvoy et al., 2008; Rajkai et al., 2008; Cloherty et al., 2010; Ito et al., 2011) and psychophysics (Burr et al., 1994; Diamond, 2002) that suggest visual processing is augmented following SEMs. To our knowledge, the present results are the first to show saccadic modulation in an object-selective region during viewing of faces and objects.

Modulation of spike probability by field potential phase

To understand how fixation-related LFP activity might influence spiking activity, we examined the phase of LFPs during single-unit activity. From our sample of 18 visually responsive neurons, we found that 50% of single units were significantly (p < 10⁻⁴) locked to the alpha rhythm, while 89 and 100% were locked to beta and gamma phase, respectively. Preferred phases of firing were not uniformly distributed across neurons; rather, they tended to cluster around the trough of the cycle. This analysis demonstrated that spiking outputs of single neurons in uSTS are coupled to field potential phase (Fig. 5) and are robustly present at the same frequencies at which we observed fixation-related phase concentration in the LFP (Figs. 2C,D, 3, 4C–H). Thus, by reducing the trial-to-trial variability of oscillatory phase rapidly following fixation onset, field potentials can lead to greater spike-timing reliability and consequently improve signal propagation (Azouz and Gray, 1999; Anastassiou et al., 2010, 2011; Fröhlich and McCormick, 2010; Ozen et al., 2010), as well as influence synaptic strength (Lubenov and Siapas, 2008; Masquelier et al., 2009, 2011; Deco et al., 2011). The alpha band spike-phase locking was the weakest and least frequent, allowing a greater bandwidth for putative spike-phase coding in this lower frequency range (Fell and Axmacher, 2011). The phase of firing can add additional information to rate codes (Montemurro et al., 2008) as well as population and temporal codes (Kayser et al., 2009). Moreover, recent work has modeled physiologically plausible mechanisms for reading out spike-phase codes (Masquelier et al., 2009, 2011; McLelland and Paulsen, 2009; Nadasdy, 2009, 2010; Samengo and Montemurro, 2010; Deco et al., 2011). Together, the modulation of spiking by LFP phase demonstrates how LFP phase concentration following image fixations could affect neural firing rates, suggesting that fixation-locked uSTS activity could more effectively drive downstream targets and ultimately modulate behavior.

Active vision and object recognition in natural images

Natural images have a 1/f^α power spectrum (Simoncelli and Olshausen, 2001), and independent cues for object segmentation in chromatic versus luminance defined edges (Hansen and Gegenfurtner, 2009). The selective suppression of the magnocellular pathway, combined with enhancement of the parvocellular pathway (Burr et al., 1994; Diamond, 2002; Kleiser et al., 2004) (but see Reppas et al., 2002), may be a form of adaptive filtering to emphasize the chromatic and high spatial frequency detail of natural images freshly present on the fovea following a saccade (Rucci and Desbordes, 2003), in addition to a general backwards masking of perisaccadic motion by the postfixation image (Ibbotson and Cloherty, 2009).

In this way, saccades have been posited to provide a reset or preparatory signal to the visual system (Melloni et al., 2009; Schroeder et al., 2010), acting as a tabula rasa for representation of local (high spatial frequency) image patches (Gutkin et al., 2001; Kupper et al., 2005) and removing contamination of the neural responses to preceding visual stimuli. Although this is an attractive framework for interpreting the effect of SEMs on early visual areas (but see Gawne and Woods, 2003; MacEvoy et al., 2008), it is unlikely that object recognition operates on such context-free (or history-free) representations of retinal images. On the contrary, a hysteresis, or neural “memory” between fixations may be adaptive for transsaccadic integration (Melcher, 2005) and context-dependent processing (Buonomano and Maass, 2009). Given natural image statistics, saccade planning could also involve predictions about the chromatic and high spatial frequency content of a peripheral object fragment, given the current foveal image and the peripheral view of the impending saccade target (Rao and Ballard, 1999; Lee and Mumford, 2003; Lee et al., 2009). Indeed, distinct IT neuronal responses for two objects converge following pairing of the peripheral view of one object with the foveal view of the other, “fooling” the visual system by swapping object identity during saccades (Li and DiCarlo, 2008).

The experimental convention of delaying stimulus onsets by hundreds of milliseconds relative to the end of SEMs may have led previous research to mischaracterize the functioning of the visual system during active vision, just as the experimental results presenting stimuli in isolation (i.e., stimulating only the classic receptive field) do not always generalize to stimuli embedded in contexts (Zipser et al., 1996; Vinje and Gallant, 2000, 2002; Rolls et al., 2003; MacEvoy et al., 2008). Similarly, dissociating visual inputs from their natural temporal proximity to visual fixations is another experimental simplification that comes at the cost of external validity. Coupling stimulus and fixation onset alters neural activity in visual areas, compared with passive viewing (for review, see Schroeder et al., 2010; Ibbotson and Krekelberg, 2011). Activity locked to fixation during periods of free visual exploration reveals distinct dynamics from image-evoked responses (Gallant et al., 1998; Leopold and Logothetis, 1998; Vinje and Gallant, 2000; Purpura et al., 2003; Maldonado et al., 2008; Ito et al., 2011), and eye movements facilitate learning in a face recognition task (Henderson et al., 2005). Image-evoked responses that change as a function of fixation as well as behavioral evidence for the role of eye movements in perception both suggest that our understanding of object recognition will be incomplete without studying it in the context of active vision.