Abstract
Visual event-related potentials (ERPs) produced by a stimulus are thought to reflect either an increase of synchronized activity or a phase realignment of ongoing oscillatory activity, with both mechanisms sharing the assumption that ERPs are independent of the current state of the brain at the time of stimulation. In natural viewing, however, visual inputs occur one after another at specific subject-paced intervals through unconstrained eye movements. We conjecture that during natural viewing, ERPs generated after each fixation are better explained by a superposition of ongoing oscillatory activity related to the processing of previous fixations, with new activity elicited by the visual input at the current fixation. We examined the electroencephalography (EEG) signals that occur in humans at the onset of each visual fixation, both while subjects freely viewed natural scenes and while they viewed a black or gray background. We found that the fixation ERPs show visual components that are absent when subjects move their eyes on a homogeneous gray or black screen. Single-trial EEG signals that comprise the ERP are predicted more accurately by a model of superposition than by either phase resetting or the addition of evoked responses and stimulus-independent noise. The superposition of ongoing oscillatory activity and the visually evoked response results in a modification of the ongoing oscillation phase. The results presented suggest that the observed EEG signals reflect changes occurring in a common neuronal substrate rather than a simple summation at the scalp of signals from independent sources.
Introduction
Different neuronal mechanisms are studied through the examination of event-related potentials (ERPs), which are typically evoked by controlled stimuli. ERPs have generally been considered to express changes in neuronal activity that occur time-locked to stimulation. One hypothesis, known as the additive model, explains the generation of ERPs as the summation of synchronized neuronal activity directly evoked by the stimulus, with any ongoing activity preceding the ERP, being a result of independent neuronal processes or noise (Shah et al., 2004; Mäkinen et al., 2005; Mazaheri and Jensen, 2006). A second hypothesis states that ERPs are the consequence of a mechanism that resets undefined ongoing oscillatory processes to a fixed phase (Sayers et al., 1974; Makeig et al., 2002; Hanslmayr et al., 2007). Therefore, although these hypotheses offer a different interpretation of the ongoing activity, they both share the assumption that ERPs are generated in a manner independent of the current state of the brain.
In contrast to these views, studies on local field potentials and single-unit activity have demonstrated that there is a clear dependence between the response to stimulation and the dynamics of ongoing activity (Mehta et al., 2002; Lakatos et al., 2007; Womelsdorf et al., 2007; Kayser et al., 2008; Rajkai et al., 2008). In this regard, the activity elicited after a given stimulus could not be adequately characterized by either of the above two models, because ongoing activity appears to modulate and interact with incoming sensory activity. Accordingly, a complete characterization of ERP responses depends on the dynamics of ongoing activity at the moment of stimulation. However, in typical experimental paradigms, sensory stimuli are presented in isolation and at intervals defined by the experimenters; therefore, previous ongoing activity is unrelated to perceptual processing and cannot carry successful predictive or preparatory signals. Accordingly, we conjecture that a relationship between ongoing and sensory-evoked activity should be readily apparent during natural viewing. In this case, visual stimulation occurs in a fast stream of events, because visual exploration occurs through ocular movement over the visual field (Yarbus, 1967). Here, visual inputs arrive during fixation periods when gaze remains stationary. As these fixations follow each other with very short latency, it is likely that the ongoing activity specific for visual processing will depend on the response to previous stimuli. To evaluate the presence and relationship between ongoing and evoked activity during natural conditions, we recorded electroencephalography (EEG) signals and eye movements during free viewing of natural scenes and examined the ERP signals that occurred around each visual fixation. We found that neither of the two aforementioned models by itself was sufficient to accurately describe our results. We concluded that the EEG signals observed during free viewing are best explained by a combination of both models. Single-trial EEG signals are predicted most accurately by the summation of an evoked visual response and an ongoing oscillatory activity. Furthermore, phase undergoes a slight modulation even when it is measured outside the period of evoked activation, suggesting an interaction at the source level rather a simple addition of signals at the scalp.
Materials and Methods
Participants.
Eleven students from the Faculty of Medicine of the Universidad de Chile with normal or corrected-to-normal vision participated in the study. All participants were right-handed and gave their informed consent in writing, which was approved by the institution's Ethics Committee for Human Research.
Stimuli and procedure.
A set of photographic images was presented in grayscale on a 19“ CRT monitor (Samsung SyncMaster 1100P Plus) at a refresh rate of 85 Hz and resolution of 800 × 600. The image set was composed of two different categories of natural scenes (NS): landscapes (n = 10) and construction sites (n = 10). To control for the motor contribution of the eye movements to the EEG signal, we also included full-screen black (BK) and gray (GR) images. Each natural scene was presented three times, and the black and gray images 60 times each. Each image was presented for 5 s in random order. A central fixation point appeared between images. The subjects pressed a button to trigger the next image presentation. The distance between subjects and monitor was 57 cm, subtending a visual field of 40° horizontally and 30° vertically with 20 pixels per visual degree. Subjects were instructed to freely explore each image, even when no natural image was presented (black and gray stimuli). To properly determine eye position and movements, an eye tracker drift correction procedure was performed before each image presentation.
Eye tracking and EEG recording.
Ocular movements were recorded with a head-mounted video oculographic eye-tracking system using pupil tracking at 500 Hz (Eyelink II; SR Research). Before each block of image presentation, a nine-point calibration was performed until average error fell below 0.5 degrees. Saccades were defined based on a velocity threshold of 25 deg/s and a minimum duration of 10 milliseconds. We included in our analysis fixations that lasted longer then 50 ms and before which no blinks had occurred for at least 100 ms.
EEG signals were recorded from 21 scalp sites using the standard 10–20 system montage (EasyCap). Electrodes were referenced to linked ear lobes, with all impedances kept below 5 kΩ. The EEG signal from each electrode site was recorded using a custom-made EEG amplifier built with 120 dB CMRR differential amplifiers (Burr Brown), digitized at 1 kHz with a bandpass of 0.5–250 Hz (−3 dB low- and high-frequency cutoff, respectively), and was stored for off-line processing.
Signal analysis.
To analyze EEG data, the signal was segmented in epochs of 2 s centered on fixation onsets (see Fig. 1B) and then grouped into three different condition sets: NS, BK, and GR. Rejection of major nonocular artifacts was performed through visual inspection. All data were digitally low-pass filtered below 50 Hz, applying forward and reverse filtering with an elliptic infinite impulse response filter. Because ERP and phase measurement can be altered by artifacts produced by eye movements, we performed an independent component analysis (ICA) to identify and extract artifact components from the data. ICA is a blind source decomposition algorithm that enables the separation of statistically independent sources from multichannel data, thus separating ocular movement and blink artifacts from EEG data (Jung et al., 2000a,b; Iriarte et al., 2003; Hoffmann and Falkenstein, 2008). We applied ICA Infomax algorithm (Bell and Sejnowski, 1995) as implemented in the EEGLAB toolbox for EEG analysis (Delorme and Makeig, 2004) to the complete dataset of each subject. Once blinks and eye movement components were identified through visual analysis of the components' topography and time series, they were removed from the original data. A fixation-onset ERP (fERP) was generated by averaging all epochs of each condition. Time–frequency analysis of the EEG epochs was performed using a family of Morlet wavelets (Tallon-Baudry et al., 1996) centered at integer frequency steps between 2 and 50 Hz. Each wavelet is defined by w(t, f) = A · exp(−t2/2σt2) · exp(2iπft), with σt = 1/(2πσf), A = (σt√π)−1/2, and a ratio of f/σf = 5. To find the frequency most likely involved in the generation of visual fERPs, we looked for the frequency with the actual highest power in the upper theta or alpha band. This is the frequency range more frequently associated with the first components of ERPs (Makeig et al., 2002; Gruber et al., 2005; Mazaheri and Jensen, 2006; Hanslmayr et al., 2007; Klimesch et al., 2007a; Min et al., 2007; Freunberger et al., 2008). For this purpose, the power spectrum in the period between 0 and 200 ms was calculated. This was done by taking the mean power of each frequency in the 200 ms time interval for each trial, then averaging for each condition for each subject, and finally averaging across subjects. Although this power estimate was obtained through averaging each frequency wavelet output over a window of only 200 ms, the frequency and time resolution is not determined by this window but by the σt and σf parameters of the wavelets. Therefore, the power estimate in the 200 ms interval includes some power from the periods immediately before and after the time interval, in a way depending on the σt parameter of the wavelet, which increases at lower frequencies. Once the frequency of interest for each condition was identified, instantaneous phase values in that frequency were calculated at each epoch fixation onset. All subsequent analyses that depended on the phase at fixation onset were conducted at the frequency selected for that respective condition. Trials could then be sorted into bins according to this calculated phase, and the relationship between this phase and the phase of the subsequent fixation was examined.
Correlation between image features and visual responses.
Visual ERPs are typically studied under experimental paradigms that examine signal changes after flashed visual stimuli. In our paradigm, subjects were able to view the images freely. Thus, although the image presented remained static with constant luminance, the image patch presented to the fovea changed with each saccade. To ensure that the EEG signals corresponded to a visual response, we examined the correlation of the amplitude of the signals after each fixation with a simple visual feature. It has been shown that visual responses after ocular movements depend on luminance (Green, 1957; Gaarder et al., 1964). Thus, we grouped EEG epochs in 10 groups according to the logarithm of the absolute difference in luminance between two square, 1° image patches; the first centered at the current gaze location and the second at the gaze location of the immediately preceding fixation. Then, we correlated the mean amplitude of the fERP's first positive component (P1, 40–110 ms) with the mean luminance difference of these groups.
Models of EEG signal generation.
To examine different mechanisms of generation of the activity observed in the EEG signals, we compared the observed ERPs to the predictions of three different models. This was done for all epochs between fixation onset (time 0) and the following 200 ms (Fig. 1H). In all models, predicted data were generated according to the following expression: s(t) = u(t) · a1 · cos(2πft + φ1) + v(t) · a2 · cos(2πft + φ2), where f is the frequency of interest of the corresponding experimental condition. The first term represents the ongoing oscillatory component and the second term represents an additive component. The ongoing component always started at the onset of fixation (time 0) and its instantaneous phase, φ1, and amplitude, a1, were calculated from the data. The initial time and phase value of the additive component were arranged such that the additive cosine component started at 0 μV (phase φ2 = −π/2) and peaked at the same instant as the actual ERP positive peak. Because a fourth of the period of a cosine at the identified frequency of interest of the NS condition (8 Hz) (see Fig. 3A) was ∼31 ms, whereas the ERP peak was at 88 ms (see Fig. 2A), the additive component was always initialized with an offset of 57 ms. In the first model, named ongoing activity, windowing functions u(t) and v(t) were set to 1 and 0, respectively. In this case, the ERP signal is modeled as an invariant ongoing oscillation. For the second model (phase resetting), u(t) and v(t) are step functions, with u(t) = 1 and v(t) = 0, from fixation onset to 57 ms and then set onwards to u(t) = 0 and v(t) = 1. In the third model (superposition), u(t) = 1 for the entire epoch and v(t) = 0 from fixation onset to 57 ms and then set to v(t) = 1.
For all models, we predicted the phase of each fixation onset based on the phase at the onset of the previous fixation and the time interval between consecutive fixations by the following expression (see Fig. 1I): θadv = θprev + (Δt · 2π · f/fs), with f being the frequency of interest and fs the sampling frequency in Hz. In the ongoing-activity model, phase at fixation onset is simply the corresponding phase for the elapsed time from the previous fixation. In this case, θprev = φ1prev, meaning the phase measured at the onset of the previous fixation, and Δt is the time between the onsets of the consecutive fixations. In the phase-resetting model, the phase is reset after the onset of the preceding fixation. Here, θprev = −π/2 and Δt is the time between the reset moment of the previous fixation (57 ms after the onset) and the onset of the current fixation. Finally, the phase in the superposition model is a linear combination of the ongoing and additive oscillatory components. In this case, the following expression was used: θprev = tan−1(α1sin(φ1prev + 57 · 2π · f/fs) + α2sin(−π/2))/(α1cos(φ1prev + 57 · 2π · f/fs) + α2cos(−π/2)) and Δt is the same as in the phase-resetting model.
Statistics.
Comparisons of the mean amplitude of P1 (40–110) and N1 (115–185 ms) components of fERPs were performed using a one-way ANOVA for different stimulus conditions, different fixation duration subsets, and different phase groups. To test the performance of the different models, Pearson's linear correlations were calculated between the actual data, the observed ERP, and the data predicted by each of the three models. Nonetheless, because the data predicted by the three models are not completely independent (for example, the first 60 ms of predicted data of every epoch in the superposition and ongoing-activity model are the same), we further checked the performance of the superposition model with a partial correlation with the actual data, controlled by the predictions of the other two models. Partial correlation removes the common variance associated with the control variables (ongoing-activity and phase-resetting models) from both the dependent variable (the actual data) and the independent variable (superposition model). Every r value was tested against the null hypothesis of no correlation by calculating its respective t value. For comparison of r values, confidence intervals (CIs) were computed through Fisher's Z transformation. To avoid errors due to multiple comparisons, a significance level of 0.001 was used.
All phase calculations were obtained with circular statistics (Batschelet, 1981). Rayleigh's uniformity test was used to assess the uniformity of the distribution of fixation-onset phases for each condition. To compare the phase predictions, distributions of the differences between predicted and actual phase were constructed. Once nonuniformity and the presence of only one mode were confirmed, the estimate of the distribution mean and its confidence intervals was used for model evaluation.
Results
EEG signals during visual fixations
We acquired EEG and eye movement data from 11 subjects while they explored natural scenes and control (gray and black) images. This unconstrained way of visual exploration produces eye movement sequences that are highly variable between subjects for the same image and within subject for different images. Figure 1A shows an example of an eye movement sequence for a natural scene. The fixation durations are narrowly distributed (median = 200 ms) whereas saccade amplitudes were distributed in a monotonically decreasing fashion (Fig. 1C,D). Eye movements occur preferentially along the horizontal axis (Fig. 1E).
To clean the EEG data of ocular artifacts, an ICA decomposition was performed for every subject, and ocular movement components were identified according to their time series and topography. These components were removed from the data. The result of this procedure can be observed in the grand averages for horizontal and vertical movements in the corresponding electro-oculogram (EOG) channels (Fig. 1F). The EEG data were segmented into 2 s epochs (Fig. 1B), aligned to each fixation onset, and averaged over all fixations. We analyzed the EEG data for a total of 8200 fixations in natural images and 5600 and 6000 fixations in gray and black images, respectively. The following results correspond only to occipital channels O1 and O2 of the 10/20 system, because these showed the highest visually related responses. Furthermore, these channels are less prone to artifacts produced by eye movements and thus require minimal ICA corrections for eye movement artifacts (Fig. 1G).
The grand average fERP for natural scenes—using fixation onset as the reference time event—shows clearly distinguishable P1 and N1 components (Fig. 2A). Because fixation durations varied from 50 ms through >300 ms (Fig. 1C), it is possible that fERPs differ across fixations as a function of their duration, rendering further analysis applicable only to subsets of fixations with similar duration. However, when we segregated our data according to fixation–duration quartiles, we did not find significant differences in either the P1 (F(1,10) = 0.52, p = 0.67) or the N1 (F(1,10) = 0.06, p = 0.97) components (Fig. 2B). Our results show that these fERP signals constitute a visually evoked response and not an ocular movement artifact. This is supported by the fact that ICA subtractions had little effect on the occipital signals. Also, the two most prominent peak components occur far from the end of the preceding saccade (peak P1 = 88 ms, peak N1 = 134 ms) and from the initiation of the next saccade (distance to P1: median = 130 ms, SD = 141 ms; distance to N1: median = 86 ms, SD = 141 ms). In addition, the average fERP signal for homogeneous gray or black images shows a significantly diminished amplitude for P1 (F(1,10) = 64.5, p < 0.001) and N1 (F(1,10) = 76.5, p < 0.0001), despite the occurrence of a saccade in every event (Fig. 2A). Finally, when epochs are grouped according to deciles of the absolute difference in luminance between subsequently fixated image patches, there is a strong correlation between these luminance differences and P1 mean amplitude (r = 0.90, p < 0.001) (Fig. 2C).
Dependence of visual activity on ongoing activity: effects of phase on EEG visual activity
During natural viewing, ocular movements are sequential. Therefore, we can use this sequence to examine whether fERPs are dependent upon some feature of the EEG signal that occurred in the preceding fixation. Here, we looked at the instantaneous fixation onset phase of the frequency in the upper-theta or alpha band (6–14 Hz) that contributes the most to the variance of the signal. The average power of the EEG signals for the period between 0 and 200 ms after fixation onset shows a local peak at 8 Hz for natural scenes and at 10 Hz for both black and gray images (Fig. 3A). For all subsequent analysis where phase information was required, we used the phase values at 8 Hz for NS and 10 Hz for GR and BK. Instantaneous phase values at fixation onset for the three conditions are distributed differently (Fig. 3B). For NS, the distribution of phases at 8 Hz is highly nonuniform (mean vector length, r = 0.36; Rayleigh's, z = 1095; p < 0.001). In comparison, the distribution of phases at 10 Hz is only slightly nonuniform for BK epochs (r = 0.03, z = 5.4, p = 0.004) and uniform for GR epochs (r = 0.01, z = 2.34, p = 0.09). We used this phase measure to sort and group the epochs of each condition. Stack plots generated with phase-sorted epochs (Fig. 3D) allowed us to examine modifications of the ongoing oscillation that resulted from the new visual fixation. In the GR and BK conditions, the phase progression of the ongoing oscillatory activity appears to be unmodified by the visual fixation (Fig. 3D, middle and right). Unlike the previous conditions, during NS stimuli the epochs show a phase realignment of activity after each fixation onset (Fig. 3D, left). This realignment could not be explained entirely by fixation-evoked resetting of the ongoing oscillation, because the latencies and widths of peaks and troughs of activity in the fixation period depend strongly on the phase at fixation onset. This dependence is particularly clear in the epochs with phases between −π/2 and 0, where the change in the signal following the fixation onset is not followed by another oscillation cycle, as is the case for other phases. Remarkably, this coincides with the time of the fERP P1 peak (Fig. 2A). These patterns of activity suggest that visual responses detected at the scalp in the NS condition result from a superposition of ongoing oscillatory activity and a signal associated with the change in visual stimulation.
fERPs are dependent on the ongoing oscillation phase
To examine the impact of the phase of ongoing oscillations on the shape of the fERPs, we sorted our data into seven phase groups in accordance with the phase at fixation onset of every trial, which was obtained at the respective frequency of interest of each condition. The fERP of each group in the NS condition is significantly different from any other group and significantly different from the overall average signal (P1: F(1,10) = 17.5, p < 0.001; N1: F(1,10) = 3.42, p = 0.005) (Fig. 4C, black lines). Nonetheless, the distribution of ongoing oscillation phases is highly nonuniform for natural scenes (Fig. 3B) and fERP groups have different sizes. If differences in group size account for these shape differences, then the fERPs of the preceding fixations—with the same number of observations—should exhibit similar differences. However, as shown in Figure 4B, these fERPs are indistinguishable between groups (P1: F(1,10) = 0.14, p = 0.99; N1: F(1,10) = 0.34, p = 0.91). To examine the contribution of ongoing oscillations to each sorted group, we modeled the oscillatory component for each phase group. A cosine waveform was generated with the same amplitude as the oscillatory signal previous to the fixation onset, and with a frequency equal to the frequency of interest, shifted to match the actual phase of the corresponding group at time 0 (Fig. 4C, blue lines). The modeled component matches closely the ERP immediately before fixation onset, but not subsequently. When we subtracted this modeled component from each fERP, the oscillatory component still matches the EEG signals for ∼50–60 ms, after which an additional component (Fig. 4C, red lines)—with at least one positive and one negative peak—appears necessary to fully account for the actual fERPs.
In contrast to the NS condition, phase-grouped fERPs in the BK and GR conditions show an oscillatory component at 10 Hz that is preserved after fixation onset and continues unchanged throughout the rest of the fixation (Fig. 4D,E, GR condition). Because phase distribution in these conditions is fairly uniform, with no apparent phase modifications, EEG signals cancel out in the grand average fERP (Fig. 2A). Together, these results suggest that the EEG signals during BK and GR are largely independent of new fixation events, whereas fERPs during NS do exhibit a dependence on the actual phase of the ongoing oscillations.
Dependence of fERPs on ERP visual response of preceding fixation
Evoked potentials, such as steady-state visually evoked potentials, which occur when stimuli change in rapid succession, have been explained in part by the fact that the evoked responses of the presented stimuli overlap. In this case, the evoked response to the actual stimulus is added to, or partially superimposed upon, the previous one. Given the dispersion of fixation and saccade durations in our data, it was necessary to determine whether this effect was indeed occurring. We first took the average fERP of all preceding fixations of one phase group (Fig. 4B), and then we repeatedly aligned this average signal backwards to the corresponding onset time of each actual fixation and produced a new average. We repeated this process for each phase-sorted group. The resulting signal differs markedly from the actual fERPs (Fig. 4C, green lines), demonstrating that the fERP activity around fixation onset in each phase group contains little amplitude contribution from the evoked responses of preceding fixations. Indeed, most P1 peaks occur 200 ms after the onset of the previous fixation, when the fERP has already flattened (Fig. 3C).
Fit between raw response, average fERP, and model predictions
We quantitatively estimated the fit between our data and the data predicted by each of the three models, as well as between our data and the grand-average fERP. We concatenated the first 200 ms of each epoch of the EEG signal, starting at the fixation onset, and correlated this signal with a concatenated signal composed either of repetitions of the fERP or of the model-predicted signal for that epoch. Overall, for the NS condition, Pearson's correlation values between the raw and predicted data models are higher for the superposition model (r = 0.41; CI, 0.40–0.41; p < 0.001) compared with the ERP (r = 0.36; CI, 0.36–0.37; p < 0.001) or ongoing (r = 0.32; CI, 0.32–0.33; p < 0.001) and phase-resetting (r = 0.37; CI, 0.37–0.38; p < 0.001) models. Because data predicted by the superposition model contain features present in the ongoing and phase-resetting models, we calculated partial correlation coefficients to remove the associated common variance (Table 1). These values are several times higher than the inverse comparison: the correlation between data and fERP and between data and ongoing and phase-resetting models, with the superposition data as control. This is not observed for the second-order partial correlation with the superposition model using both ongoing-activity and ERP models as controls.
Table 1.
Condition | rSup.ERP | rERP.Sup | rSup.Ong | rOng.Sup | rSup.Res | rRes.Sup | rSup.OngERP |
---|---|---|---|---|---|---|---|
NS | 0.23 | 0.12 | 0.26 | −0.03 | 0.19 | 0.06 | −0.01 |
CI | 0.22/0.23 | 0.12/0.13 | 0.26/0.27 | −0.04/−0.03 | 0.18/0.19 | 0.06/0.07 | −0.01/0.008 |
BK | 0.29 | 0.12 | −0.008 | 0.25 | 0.26 | −0.11 | 0.008 |
CI | 0.28/0.29 | 0.11/0.12 | −0.01/0.001 | 0.24/0.26 | 0.25/0.27 | −0.12/−0.11 | 0.001/0.01 |
GR | 0.24 | 0.08 | −0.001 | 0.2 | 0.20 | −0.07 | 0.008 |
CI | 0.23/0.24 | 0.07/0.08 | −0.01/0.004 | 0.19/0.2 | 0.20/0.21 | −0.08/−0.06 | 0.001/0.01 |
SUP, Superposition model; ONG, ongoing activity model; RES, phase-resetting model.
In contrast to the NS condition, Pearson's correlation coefficients for the BK and GR conditions computed with the ongoing-activity model were higher (BK: r = 0.36; CI, 0.35–0.37; p < 0.001; GR: r = 0.3; CI, 0.29–0.31; p < 0.001) than the data computed with the superposition model (BK: r = 0.27; CI, 0.26–0.27; p < 0.001; GR: r = 0.23; CI, 0.22–0.23; p < 0.001), the phase-resetting model (BK: r = 0.12; CI, 0.12–0.13; p < 0.001; GR: r = 0.13; CI, 0.12–0.13; p < 0.001) or the fERP signal (BK: r = 0.04; CI, 0.03–0.05; p < 0.001; GR: r = 0.05; CI, 0.04–0.06; p < 0.001). In these cases, the partial correlation with the superposition data decreases nearly to zero when controlled by the ongoing-activity model.
Correlations performed by phase group for NS condition show that the superposition model correlates better with actual data than the other models (Fig. 5A, left), especially in those phase groups where ongoing activity and the additive component occur in counterphase. For BK and GR conditions, correlation values computed with the ongoing-activity model are higher for every phase group (Fig. 5A, center and right, respectively).
Visual EEG responses: testing independence between ongoing and fixation-related activity
The EEG signal recorded in occipital electrodes could be the result of superposition at the scalp of the visually evoked activity with an otherwise independent ongoing activity. If these signals arise from completely independent sources, the instantaneous phase value at a given fixation onset will be entirely predictable from the phase observed at the onset of the preceding fixation, because at the time of the new fixation, the visually evoked response will have mostly subsided (Fig. 3C). However, considering differences in the phase of the signal at each fixation onset and the phase as predicted by the preceding fixation, we found that five of the seven phase groups show values significantly different from those predicted (Fig. 5B, left, blue line). Moreover, differences of actual and predicted phase computed using the phase-resetting model also differ significantly from 0 in all but the phase-aligned groups (Fig. 5B, left, red line). Nonetheless, differences between actual phases and predictions based on the superposition model (Fig. 5B, left, black line) yield values not significantly different from 0 for five phase groups and fail in only two of the phase groups, including one group in which the visually evoked component occurs exactly in counterphase with the ongoing oscillations, and thus the phase measurement is expectedly less reliable. In contrast, fixation-onset phase for GR and BK conditions are very well predicted by the ongoing, but not by the phase-resetting or superposition models (Fig. 5B, middle and right), demonstrating that the oscillatory activity at ∼10 Hz remains uninfluenced by eye movements.
Discussion
We investigated the visual responses produced during successive visual fixations as human subjects freely explore natural scenes. The fERPs show clear visual components that are otherwise absent when subjects move their eyes on a homogeneously gray or black screen. fERP amplitude is related to differences in luminance between successive fixation locations. The EEG signals immediately following each visual fixation appear to be dependent upon the phase of ongoing oscillatory activity, suggesting a superposition between ongoing oscillatory activity and a stimulus-driven, additive, evoked response. Indeed, quantitative analyses show that data predicted by a superposition model correlate better with the actual data than with the grand-average fERP, or with data predicted by a phase-resetting model or a model that considers only a stimulus-independent continuous oscillatory activity. Furthermore, the superposition of ongoing oscillatory activity and the visually evoked response results in a modification of the phase of the ongoing oscillation, suggesting that the EEG signals observed in these electrodes reflect changes occurring in a common neuronal substrate rather than a simple summation of independent signals at the scalp.
A main methodological issue in cognitive neuroscience is whether complex phenomena can be well understood by explaining their constitutive components separately. For this reason, there is increasing interest in complementing the research done in studies based on the itemization of cognitive functions with studies focusing on behavioral paradigms in more holistic settings. In this regard, the use of natural stimuli and natural behaviors such as free viewing, which includes attention and other top-down mechanisms, can provide new insights into the dynamics of visual perception (Maldonado et al., 2008). In contrast to previous studies, in which eye movements were allowed but directed to low-level visual stimuli (Evans, 1953; Green, 1957; Gaarder et al., 1964; Scott and Bickford, 1967; Scott et al., 1967, 1981; Barlow and Cigánek, 1969; Yagi, 1981; Riemslag et al., 1987; Billings, 1989; Thickbroom et al., 1991), our study was performed with ocular movements occurring in a stream of autonomous actions over a complex visual scene. Despite these methodological differences, our results share two important aspects with these previous studies. First, in our study, the clear occipital responses after fixation onset display characteristic components of visual ERP, which have been described in other visual experiments where free ocular movements were not allowed. The P1 responds linearly to differences in luminance between subsequently fixated image patches. This result is consistent with other studies, which have shown that visual responses to luminance changes occur either when the stimulus is modified at a single fixation location or when the subjects move their eyes to a new location that has different luminance (Barlow and Cigánek, 1969; Riemslag et al., 1987). Second, as in our study, eye movements over black or gray backgrounds elicit little or no EEG response (Evans, 1953; Fourment et al., 1976). Although this result reinforces the visual nature of the two most prominent components of the fERP, it does not completely exclude nonvisual responses locked to the initiation of eye moments, as neurophysiological recordings during self-initiated ocular movements in the dark have demonstrated in both monkeys and humans (Fourment et al., 1976; Skrandies and Laschke, 1997; Rajkai et al., 2008).
Our results show that EEG responses recorded at the scalp are the consequence of the superposition of an incoming stimulus-driven response with ongoing activity (Shah et al., 2004; Mäkinen et al., 2005; Mazaheri and Jensen, 2006). However, in contrast to our study, previous studies have attempted to clarify the basic mechanism underlying EEG event-related responses by using experimental paradigms in which ongoing activity is an undefined process outside of experimental control. Our approach instead exploited natural conditions of perceptual stimulation, revealing an ongoing activity resulting from the continuous processing of a sequence of visual events.
Our study shows that the experimental data under natural conditions is best explained by a model that combines a visually evoked response with a change in the phase on the ongoing oscillation. Neural mass models of hierarchical networks with backward connections, where damped oscillations arise naturally after stimulation (Jansen and Rit, 1995; David et al., 2005), have produced theoretical predictions that agree with the results shown here. These kind of models have been successful in predicting late components of real ERPs (Garrido et al., 2007). Two types of competing and mutually exclusive theories have sought to explain the nature of the observed visually evoked EEG signals. The additive model assumes that ERPs are a summation of synchronized neuronal activity directly evoked by the stimulus, with unrelated ongoing activity produced by independent neuronal sources or noise whereas the other model assumes a phase-resetting mechanism of ongoing oscillatory activity. This latter model has also been generalized to a formulation that includes not only phase reorganization but also evoked and induced oscillations as underlying sources of ERP component (Klimesch et al., 2007b). Our results show that both neuronal activity evoked by the stimulus and oscillatory phase changes occur during natural viewing. Phase changes would, however, not occur instantly and independently of the visually evoked response, but would instead appear as a result of changes in the dynamics of the neuronal activity in a common neuronal substrate. Nevertheless, it is important to point out the limitation of instantaneous phase estimation in narrow frequency bands of time series that are generated by stochastic and nonstationary processes. Wavelet analysis as used here can result in phase biases produced by an evoked response after or before the onset of a fixation (Sauseng et al., 2007). In our data, phase measurements at fixation onset are in good agreement with the previous oscillatory phase rather than the evoked component, suggesting this bias has a small impact.
Ongoing oscillatory activity is not a mixture of evoked potentials resulting from previous fixations (Fig. 4), suggesting that the power peaks in the frequency analysis are due, at least in part, to underlying oscillatory processes. Oscillatory activity in low- to medium-frequency ranges is usually associated with a mechanism for long-range coordination between different brain areas engaged in recurrent loops of activity (Engel et al., 2001; Fries, 2005; Schroeder and Lakatos, 2009). Also, a specific role for oscillations in the alpha range in the generation of visual ERP components has been suggested, with both inhibitory and long-range coordination roles (Klimesch et al., 2007a). In this sense, ongoing activity during continuous and autonomous selection of stimuli could be both quantitatively and qualitatively different to ongoing activity in the case of nonstimulation and when an unpredictable new stimulus is expected. The latter kind of experimental condition might be comparable to the BK and GR condition used in our experiment. In this case, a visual stimulus after each movement is absent or homogeneous and oscillatory power is centered at a frequency in the middle of the alpha range, typical of prestimulation periods in static paradigms. This activity could possibly reflect an inhibitory effect in posterior sensory areas related to the lack of change in visual stimulation. In contrast, in the NS condition a peak in a slightly lower frequency is revealed. Taking into account the limitations of frequency resolution in neighboring frequencies, these results are in good agreement with previous reports of change in the alpha range activity after a visual stimulus event, in which activity at the middle alpha range (∼10 Hz) is partially suppressed and activity at upper-theta/low-alpha range is increased (Gruber et al., 2005; Mazaheri and Jensen, 2006; Freunberger et al., 2008). However, in contrast to reports that explain this upper-theta/low-alpha activity as an additive component (Mazaheri and Jensen, 2006), we show that the activity at the low-alpha band is a main component of the ongoing activity when stimulation is continuous. Although we do not necessarily commit to an interpretation of the additive component we found as an evoked oscillation, in the situation of continuous stimulation, it is expected that the neuronal network underlying visual oscillatory phenomena will be uninterruptedly active (David et al., 2005) and therefore the lower-alpha activity seen after single-stimulus presentation in other studies might keep going in the case of free viewing. In this framework, the arrival of a new input, expressed as an additive component, perturbs the activity of the network, resulting in superposition patterns, maintenance of the oscillatory activity, and a change in the phase of oscillation of the network. In the NS condition, oscillatory activity is apparent during continuous visual stimulation, and its phase is changed with every new fixation, depending on both oscillatory and incoming components.
Further studies of visual responses in natural viewing experiments should consider activity statistics that result from the continuous interdependence of events. In contrast to responses to static and transient visual stimulation, neuronal activity during natural viewing may contain features intrinsic to the natural flow of goal-oriented behavior and thus would include patterns that are instrumental for building models of visual perception.
Footnotes
This study was supported in part by the Iniciativa Cientifica Milenio P04-068F, The Puelma Foundation, and Conicyt (R.M.-S.). We thank Francisco Flores, Michael Plöchl, and Robert Muil for comments on an earlier version of this manuscript.
References
- Barlow JS, Cigánek L. Lambda responses in relation to visual evoked responses in man. Electroencephalogr Clin Neurophysiol. 1969;26:183–192. doi: 10.1016/0013-4694(69)90209-0. [DOI] [PubMed] [Google Scholar]
- Batschelet E. New York: Academic; 1981. Circular statistics on biology. [Google Scholar]
- Bell AJ, Sejnowski TJ. An information-maximization approach to blind separation and blind deconvolution. Neural Comput. 1995;7:1129–1159. doi: 10.1162/neco.1995.7.6.1129. [DOI] [PubMed] [Google Scholar]
- Billings RJ. The origin of the occipital lambda wave in man. Electroencephalogr Clin Neurophysiol. 1989;72:95–113. doi: 10.1016/0013-4694(89)90170-3. [DOI] [PubMed] [Google Scholar]
- David O, Harrison L, Friston KJ. Modelling event-related responses in the brain. Neuroimage. 2005;25:756–770. doi: 10.1016/j.neuroimage.2004.12.030. [DOI] [PubMed] [Google Scholar]
- Delorme A, Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods. 2004;134:9–21. doi: 10.1016/j.jneumeth.2003.10.009. [DOI] [PubMed] [Google Scholar]
- Engel AK, Fries P, Singer W. Dynamic predictions: oscillations and synchrony in top-down processing. Nat Rev Neurosci. 2001;2:704–716. doi: 10.1038/35094565. [DOI] [PubMed] [Google Scholar]
- Evans CC. Spontaneous excitation of the visual cortex and association areas: lambda waves. Electroencephalogr Clin Neurophysiol. 1953;5:69–74. doi: 10.1016/0013-4694(53)90054-6. [DOI] [PubMed] [Google Scholar]
- Fourment A, Calvet AF, Bancaud J. Electrocorticography of waves associated with eye movements in man during wakefulness. Electroencephalogr Clin Neurophysiol. 1976;40:457–469. doi: 10.1016/0013-4694(76)90076-6. [DOI] [PubMed] [Google Scholar]
- Freunberger R, Höller Y, Griesmayr B, Gruber W, Sauseng P, Klimesch W. Functional similarities between the P1 component and alpha oscillations. Eur J Neurosci. 2008;27:2330–2340. doi: 10.1111/j.1460-9568.2008.06190.x. [DOI] [PubMed] [Google Scholar]
- Fries P. A mechanism for cognitive dynamics: neuronal communication through neuronal coherence. Trends Cogn Sci. 2005;9:474–480. doi: 10.1016/j.tics.2005.08.011. [DOI] [PubMed] [Google Scholar]
- Gaarder K, Krauskopf J, Graf V, Kropfl W, Armington JC. Averaged brain activity following saccadic eye movement. Science. 1964;146:1481–1483. doi: 10.1126/science.146.3650.1481. [DOI] [PubMed] [Google Scholar]
- Garrido MI, Kilner JM, Kiebel SJ, Friston KJ. Evoked brain responses are generated by feedback loops. Proc Natl Acad Sci U S A. 2007;104:20961–20966. doi: 10.1073/pnas.0706274105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Green J. Some observations on lambda waves and peripheral stimulation. Electroencephalogr Clin Neurophysiol. 1957;9:691–704. doi: 10.1016/0013-4694(57)90089-5. [DOI] [PubMed] [Google Scholar]
- Gruber WR, Klimesch W, Sauseng P, Doppelmayr M. Alpha phase synchronization predicts P1 and N1 latency and amplitude size. Cereb Cortex. 2005;15:371–377. doi: 10.1093/cercor/bhh139. [DOI] [PubMed] [Google Scholar]
- Hanslmayr S, Klimesch W, Sauseng P, Gruber W, Doppelmayr M, Freunberger R, Pecherstorfer T, Birbaumer N. Alpha phase reset contributes to the generation of ERPs. Cereb Cortex. 2007;17:1–8. doi: 10.1093/cercor/bhj129. [DOI] [PubMed] [Google Scholar]
- Hoffmann S, Falkenstein M. The correction of eye blink artefacts in the EEG: a comparison of two prominent methods. PLoS One. 2008;3:e3004. doi: 10.1371/journal.pone.0003004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Iriarte J, Urrestarazu E, Valencia M, Alegre M, Malanda A, Viteri C, Artieda J. Independent component analysis as a tool to eliminate artifacts in EEG: a quantitative study. J Clin Neurophysiol. 2003;20:249–257. doi: 10.1097/00004691-200307000-00004. [DOI] [PubMed] [Google Scholar]
- Jansen BH, Rit VG. Electroencephalogram and visual evoked potential generation in a mathematical model of coupled cortical columns. Biol Cybern. 1995;73:357–366. doi: 10.1007/BF00199471. [DOI] [PubMed] [Google Scholar]
- Jung TP, Makeig S, Westerfield M, Townsend J, Courchesne E, Sejnowski TJ. Removal of eye activity artifacts from visual event-related potentials in normal and clinical subjects. Clin Neurophysiol. 2000a;111:1745–1758. doi: 10.1016/s1388-2457(00)00386-2. [DOI] [PubMed] [Google Scholar]
- Jung TP, Makeig S, Humphries C, Lee TW, McKeown MJ, Iragui V, Sejnowski TJ. Removing electroencephalographic artifacts by blind source separation. Psychophysiology. 2000b;37:163–178. [PubMed] [Google Scholar]
- Kayser C, Petkov CI, Logothetis NK. Visual modulation of neurons in auditory cortex. Cereb Cortex. 2008;18:1560–1574. doi: 10.1093/cercor/bhm187. [DOI] [PubMed] [Google Scholar]
- Klimesch W, Sauseng P, Hanslmayr S. EEG alpha oscillations: the inhibition-timing hypothesis. Brain Res Rev. 2007a;53:63–88. doi: 10.1016/j.brainresrev.2006.06.003. [DOI] [PubMed] [Google Scholar]
- Klimesch W, Sauseng P, Hanslmayr S, Gruber W, Freunberger R. Event-related phase reorganization may explain evoked neural dynamics. Neurosci Biobehav Rev. 2007b;31:1003–1016. doi: 10.1016/j.neubiorev.2007.03.005. [DOI] [PubMed] [Google Scholar]
- Lakatos P, Chen CM, O'Connell MN, Mills A, Schroeder CE. Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron. 2007;53:279–292. doi: 10.1016/j.neuron.2006.12.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Makeig S, Westerfield M, Jung TP, Enghoff S, Townsend J, Courchesne E, Sejnowski TJ. Dynamic brain sources of visual evoked responses. Science. 2002;295:690–694. doi: 10.1126/science.1066168. [DOI] [PubMed] [Google Scholar]
- Mäkinen V, Tiitinen H, May P. Auditory event-related responses are generated independently of ongoing brain activity. Neuroimage. 2005;24:961–968. doi: 10.1016/j.neuroimage.2004.10.020. [DOI] [PubMed] [Google Scholar]
- Maldonado PE, Ossandon JP, Flores FJ. Attention and neurodynamical correlates of natural vision. In: Aboitiz F, Cosmelli D, editors. From attention to goal-directed behavior: neurodynamical and methodological and clinical trends. Berlin: Springer; 2008. p. 67.p. 82. [Google Scholar]
- Mazaheri A, Jensen O. Posterior alpha activity is not phase-reset by visual stimuli. Proc Natl Acad Sci U S A. 2006;103:2948–2952. doi: 10.1073/pnas.0505785103. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mehta MR, Lee AK, Wilson MA. Role of experience and oscillations in transforming a rate code into a temporal code. Nature. 2002;417:741–746. doi: 10.1038/nature00807. [DOI] [PubMed] [Google Scholar]
- Min BK, Busch NA, Debener S, Kranczioch C, Hanslmayr S, Engel AK, Herrmann CS. The best of both worlds: phase-reset of human EEG alpha activity and additive power contribute to ERP generation. Int J Psychophysiol. 2007;65:58–68. doi: 10.1016/j.ijpsycho.2007.03.002. [DOI] [PubMed] [Google Scholar]
- Rajkai C, Lakatos P, Chen CM, Pincze Z, Karmos G, Schroeder CE. Transient cortical excitation at the onset of visual fixation. Cereb Cortex. 2008;18:200–209. doi: 10.1093/cercor/bhm046. [DOI] [PubMed] [Google Scholar]
- Riemslag FC, van der Heijde GL, van Dongen MM. Are eye movement evoked potentials different from pattern reversal evoked potentials? Doc Ophthalmol. 1987;66:279–289. doi: 10.1007/BF00213656. [DOI] [PubMed] [Google Scholar]
- Sauseng P, Klimesch W, Gruber WR, Hanslmayr S, Freunberger R, Doppelmayr M. Are event-related potential components generated by phase resetting of brain oscillations?: A critical discussion. Neuroscience. 2007;146:1435–1444. doi: 10.1016/j.neuroscience.2007.03.014. [DOI] [PubMed] [Google Scholar]
- Sayers BM, Beagley HA, Henshall WR. The mechanism of auditory evoked EEG responses. Nature. 1974;247:481–483. doi: 10.1038/247481a0. [DOI] [PubMed] [Google Scholar]
- Schroeder CE, Lakatos P. Low-frequency neuronal oscillations as instruments of sensory selection. Trends Neurosci. 2009;32:9–18. doi: 10.1016/j.tins.2008.09.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Scott DF, Bickford RG. Electrophysiologic studies during scanning and passive eye movements in humans. Science. 1967;155:101–102. doi: 10.1126/science.155.3758.101. [DOI] [PubMed] [Google Scholar]
- Scott DF, Groethuysen UC, Bickford RG. Lambda responses in the human electroencephalogram. Neurology. 1967;17:770–778. doi: 10.1212/wnl.17.8.770. [DOI] [PubMed] [Google Scholar]
- Scott DF, Moffett A, Bickford RG. Comparison of two types of visual evoked potentials: pattern reversal and eye movement (lambda) Electroencephalogr Clin Neurophysiol. 1981;52:102–104. doi: 10.1016/0013-4694(81)90195-4. [DOI] [PubMed] [Google Scholar]
- Shah AS, Bressler SL, Knuth KH, Ding M, Mehta AD, Ulbert I, Schroeder CE. Neural dynamics and the fundamental mechanisms of event-related brain potentials. Cereb Cortex. 2004;14:476–483. doi: 10.1093/cercor/bhh009. [DOI] [PubMed] [Google Scholar]
- Skrandies W, Laschke K. Topography of visually evoked brain activity during eye movements: lambda waves, saccadic suppression, and discrimination performance. Int J Psychophysiol. 1997;27:15–27. doi: 10.1016/s0167-8760(97)00749-6. [DOI] [PubMed] [Google Scholar]
- Tallon-Baudry C, Bertrand O, Delpuech C, Pernier J. Stimulus specificity of phase-locked and non-phase-locked 40 Hz visual responses in human. J Neurosci. 1996;16:4240–4249. doi: 10.1523/JNEUROSCI.16-13-04240.1996. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Thickbroom GW, Knezevic W, Carroll WM, Mastaglia FL. Saccade onset and offset lambda waves: relation to pattern movement visually evoked potentials. Brain Res. 1991;551:150–156. doi: 10.1016/0006-8993(91)90927-n. [DOI] [PubMed] [Google Scholar]
- Womelsdorf T, Schoffelen JM, Oostenveld R, Singer W, Desimone R, Engel AK, Fries P. Modulation of neuronal interactions through neuronal synchronization. Science. 2007;316:1609–1612. doi: 10.1126/science.1139597. [DOI] [PubMed] [Google Scholar]
- Yagi A. Visual signal detection and lambda responses. Electroencephalogr Clin Neurophysiol. 1981;52:604–610. doi: 10.1016/0013-4694(81)91434-6. [DOI] [PubMed] [Google Scholar]
- Yarbus AL. New York: Plenum; 1967. Eye movements and vision. [Google Scholar]