Differential Roles of Frequency-following and Frequency-doubling Visual Responses Revealed by Evoked Neural Harmonics

Abstract

Frequency-following and frequency-doubling neurons are ubiquitous in both striate and extrastriate visual areas. However, responses from these two types of neural populations have not been effectively compared in humans because previous EEG studies have not successfully dissociated responses from these populations. We devised a light–dark flicker stimulus that unambiguously distinguished these responses as reflected in the first and second harmonics in the steady-state visual evoked potentials. These harmonics revealed the spatial and functional segregation of frequency-following (the first harmonic) and frequency-doubling (the second harmonic) neural populations. Spatially, the first and second harmonics in steady-state visual evoked potentials exhibited divergent posterior scalp topographies for a broad range of EEG frequencies. The scalp maximum was medial for the first harmonic and contralateral for the second harmonic, a divergence not attributable to absolute response frequency. Functionally, voluntary visual–spatial attention strongly modulated the second harmonic but had negligible effects on the simultaneously elicited first harmonic. These dissociations suggest an intriguing possibility that frequency-following and frequency-doubling neural populations may contribute complementary functions to resolve the conflicting demands of attentional enhancement and signal fidelity—the frequency-doubling population may mediate substantial top–down signal modulation for attentional selection, whereas the frequency-following population may simultaneously preserve relatively undistorted sensory qualities regardless of the observer’s cognitive state.

INTRODUCTION

Frequency-following and frequency-doubling neural responses begin as early as the primary visual cortex. When a flickered grating is presented, simple cells respond at the stimulus modulation frequency as each receptive-field subregion responds to a specific luminance polarity (lighter or darker than the surround). In contrast, complex cells respond at twice the stimulus modulation frequency as subregions that respond to opposite luminance polarities overlap in their receptive fields (e.g., Benucci, Frazor, & Carandini, 2007; De Valois, Albrecht, & Thorell, 1982; Hubel & Wiesel, 1968). These simple- and complex-cell properties are largely preserved in V2 (e.g., Foster, Gaska, Nagler, & Pollen, 1985), and the receptive-field properties of V4 neurons range from simple cell-like to complex cell-like (e.g., Hanazawa & Komatsu, 2001; Desimone & Schein, 1987). Neurons in inferotemporal cortex also exhibit varying degrees of selectivity for luminance polarity, from those that respond only to light or dark patterns to those that respond independently of luminance polarity (e.g., Ito, Fujita, Tamura, & Tanaka, 1994). Thus, neural responses along the ventral visual pathway range from polarity selective (simple cell-like) to polarity independent (complex cell-like), suggesting that both frequency-following and frequency-doubling responses are ubiquitous along the ventral visual pathway thought to mediate visual pattern perception (e.g., Fang & He, 2005; Goodale & Westwood, 2004; Mishkin, Ungerleider, & Macko, 1983). However, the potential roles of these two types of neural responses in visual pattern processing have been unclear.

To study the roles of frequency-following and frequency-doubling neural population responses in humans, we designed a light–dark flicker that effectively segregated the two types of responses into the first and second harmonics of visually evoked EEG activity. EEG responses to periodically flickered stimuli are termed steady-state visual evoked potentials (SSVEPs); the first harmonic refers to the Fourier component at the flicker frequency and the second harmonic refers to the component at twice the flicker frequency (e.g., Di Russo et al., 2007; Hermann, 2001; Regan, 1989). Previous SSVEP studies, however, did not investigate the roles of frequency-following and frequency-doubling neural populations. Moreover, because those studies used either on–off or counterphase flicker, they did not effectively segregate frequency-following and frequency-doubling neural responses into separate SSVEP harmonics (see the Methods section for Experiment 1).

Because anatomical segregation generally implies functional segregation, we first determined whether frequency-following and frequency-doubling neural population responses exhibited clear segregation in the scalp topography that might indicate different configurations of neural generators. To investigate potential functional roles of the two types of responses, we analyzed attention effects, partly because attention plays a fundamental role in signal selection (e.g., Maunsell & Treue, 2006; Kastner & Ungerleider, 2000; Desimone & Duncan, 1995) and partly because attentional modulation of neural activation imposes conflicting demands on visual processing. Whereas strong top–down modulation of neural response is desirable for stimulus selection, such modulation must occur without substantially distorting information about stimulus intensity. We examined the hypothesis that frequency-following and frequency-doubling neural populations may play distinct roles in meeting these conflicting demands; that is, one population might be strongly modulated by top–down attention—mediating attentional control—whereas the other population might be relatively immune to attentional modulations—preserving undistorted sensory qualities.

EXPERIMENT1: TOPOGRAPHIC DISTRIBUTIONS OF FREQUENCY-FOLLOWING AND FREQUENCY-DOUBLING VISUAL RESPONSES

We used a dark–light flicker that separated the frequency-following and frequency-doubling neural population responses into the first and second SSVEP harmonics to determine the topographic distributions of the frequency-following and frequency-doubling neural population activity.

Methods

Observers

Twelve observers (9 men and 3 women, ages ranging from 23 to 46 years) participated; data from two observers (1 man and 1 woman) were excluded from the analyses because of excessive blinking. All observers had normal or corrected-to-normal visual acuity, gave informed consent to participate, and were tested individually in a dimly lit room.

Stimuli

Circular gratings (1.1 cycles/degree in fundamental spatial frequency) were shown on a 19-in. CRT monitor set to a 100-Hz refresh rate. The diameter and the retinal eccentricity of each grating were 5.9° and 4.5°, respectively (Figure 1). Each grating was presented against a midgray background (64.7 cd/m²) and was flickered at a different frequency. Flicker was generated by modulating the luminance of the concentric rings symmetrically, lighter and darker, against the midgray background. This light–dark flicker prevented the creation of negative afterimages, produced no sensation of motion (unlike a counterphase flicker), and effectively separated frequency-following and frequency-doubling neural population responses into the first and second harmonics of SSVEPs (see below for details). Because visual neurons are primarily driven by luminance changes, we define the contrast, C, of the flickered gratings as, $C=\frac{L_{\text{light}}-L_{\text{dark}}}{L_{\text{light}}+L_{\text{dark}}}$, where L_light and L_dark indicate the luminance during the light and dark phases, respectively.

Figure 1.

Stimuli and a trial sequence. (A) Experiment 1. On each trial, a circular grating was presented in either the left or the right visual hemifield. The grating was flickered between a light phase and a dark phase at one of five frequencies (6.25, 8.33, 12.50, 16.67, or 25.00 Hz). Each trial was initiated by a button press, followed by a fixation screen, and then a 4.8-sec presentation of the flickered grating. (B) Experiment 2. On each trial, two circular gratings were simultaneously presented in opposite visual hemifields. Gratings were flickered (between a light and a dark phase) at different frequencies, one at 12.50 Hz and the other at 16.67 Hz. Each trial was initiated by a button press, followed by a central arrow indicating the grating to be attended, a fixation screen, and then a 4.8-sec presentation of the flickered gratings.

The luminance was square-wave modulated to produce strong SSVEP responses. Although a temporal square-wave contains odd harmonics (third, fifth, seventh, and so on), they were small in amplitude compared with the first and second harmonics, and they produced appreciable spectral peaks only for low flicker frequencies (Figure 2). We also obtained higher order even harmonics (fourth, sixth, etc.) primarily for the lowest flicker frequency (Figure 2). These higher order harmonics have been previously reported (e.g., Benucci et al., 2007; Hermann, 2001; Rager & Singer, 1998; Regan, 1989), but their exact origins are unclear; they might arise from nonlinear neural interactions (e.g., Friston, 2000) and/or from a positive-skewing distortion of stimulus waveforms that occurs for visual responses to low flicker frequencies (e.g., Rager & Singer, 1998) potentially because of rapid neural adaptation (e.g., Müller, Metha, Krauskopf, & Lennie, 1999). Regardless of the exact origins of the relatively small higher order harmonics, it is reasonable to assume that the dominant first and second harmonics primarily include a combination of frequency-following and frequency-doubling neural population responses.

Figure 2.

(A–E) Spectral plots of EEG (amplitude in µV as a function of response frequency, averaged across observers) evoked by a single circular grating presented to the left or right visual hemifield and flickered at different frequencies (data from Experiment 1). For each flicker frequency, the upper graph shows responses averaged from contralateral-posterior electrodes, and the lower graph shows responses averaged from ipsilateral-posterior electrodes (see the illustration in Figure 4B for the locations of these electrodes). (F) Spectral plots of EEG evoked by two simultaneously presented gratings (one in each visual hemifield) (data from Experiment 2, averaged across attended and ignored conditions). The spectral peaks corresponding to the first, second, third, etc., harmonics are labeled with 1fc, 1fi, 2fc, 2fi, 3fc, 3fi, etc., where “c” indicates responses from the contralateral-posterior electrodes and “i” indicates responses from the ipsilateral-posterior electrodes. (F) Harmonic labels for the 16.67-Hz flicker are italicized to distinguish them from the labels for the 12.50-Hz flicker. Note that the first harmonic (both ipsilateral and contralateral) and the contralateral second harmonic are clearly above noise in all cases.

The two commonly used methods of generating flicker, namely, a counterphase flicker and an on–off flicker, do not effectively separate frequency-following and frequency-doubling responses into distinct SSVEP harmonics. When a counterphase-flickered grating is used, frequency-doubling neurons synchronously respond at each contrast reversal regardless of polarity, generating a robust second harmonic of SSVEPs (Figure 3A, the upper trace). Because dark–light transitions occur 180° out of phase at different locations in a counterphase-flickered grating, responses of both the dark-selective and light-selective subfields of frequency-following neurons occur out of phase across space, resulting in substantial cancellation of their responses when spatially averaged over a large population of neurons as in SSVEPs (Figure 3A, the middle and lower traces; for an illustration, also see Benucci et al., 2007). Thus, when a counterphase-flickered grating is used, responses of frequency-doubling neurons are detected as the second harmonic of SSVEPs, but responses of frequency-following neurons are mostly averaged out.

Figure 3.

Responses of idealized frequency-doubling (labeled F-doubling) and frequency-following (labeled F-following) visual neurons to a counterphase flicker (A), on–off flicker (B), and light–dark flicker (C). We illustrate hypothetical membrane potentials; an average of membrane potentials over some region of neural tissue generates local field potentials (LFPs), and an aggregate of LFPs is detected as SSVEPs. (A) Counterphase flicker. (Top) A frequency-doubling neuron responds to each contrast reversal irrespective of polarity, generating a second harmonic of SSVEPs (2f). (Middle) A “dark-selective” subfield of a frequency-following neuron responds whenever a dark stimulus appears. The dark-selective subfields coinciding with the left and right parts of the stimuli respond in opposite phase, so their responses cancel out in spatially averaged SSVEPs. (Bottom) A “light-selective” subfield of a frequency-following neuron responds whenever a light stimulus appears. The light-selective subfields coinciding with the left and right parts of the stimuli respond in opposite phase, so their responses cancel out in spatially averaged SSVEPs. Thus, when a counterphase flicker is used, responses of frequency-doubling neurons contribute to the second harmonic of SSVEPs, but responses of frequency-following neurons are mostly cancelled out (even when the light and dark responses are unequal [e.g., dark responses larger than light responses in the illustration], unless the responses differ systematically across space). (B) On–off flicker. (Top) A frequency-doubling neuron primarily responds at stimulus appearance, generating a first harmonic of SSVEPs (1f), but it may also weakly respond at stimulus disappearance, generating some second harmonic. (Middle) A dark-selective subfield of a frequency-following neuron responds whenever a dark stimulus appears, generating a first harmonic of SSVEPs. (Bottom) A light-selective subfield responds little to a dark stimulus. Thus, when an on–off flicker is used, responses of both frequency-doubling and frequency-following neurons contribute to the first harmonic of SSVEPs. (C) Light–dark flicker. (Top) A frequency-doubling neuron responds to each contrast reversal irrespective of polarity, generating a second harmonic of SSVEPs. (Middle) A dark-selective subfield of a frequency-following neuron responds whenever a dark stimulus appears. (Bottom) A light-selective subfield of a frequency-following neuron responds whenever a light stimulus appears. Although the dark- and light-selective subfields respond in opposite phase because dark and light responses are asymmetric (e.g., dark responses are larger than light responses in the illustration), responses of frequency-following neurons collectively contribute to the first harmonic of SSVEPs. Thus, when a light–dark flicker is used (as in our study), responses of frequency-doubling and frequency-following neurons are segregated into the second and first harmonics of SSVEPs.

When an on–off-flickered stimulus is used, SSVEPs are commonly dominated by the first harmonic because the flicker is between stimulus presence and absence (a uniform field). Because visual neurons respond more strongly to pattern appearance than to disappearance, frequency-doubling as well as frequency-following neurons contribute to the first harmonic of SSVEPs in response to an on–off flicker (Figure 3B). Thus, when an on–off flicker is used, responses of frequency-doubling and frequency-following visual neurons are both confounded within the first harmonic of SSVEPs.

We used a light–dark flicker, a “hybrid” of a counterphase and on–off flicker, where the stimulus luminance oscillated between dark and light against a midgray background. Because frequency-doubling neurons respond synchronously at each contrast reversal regardless of polarity, they robustly produce the second harmonic of SSVEPs (Figure 3C, the upper trace). Although the light-selective and dark-selective subfields of frequency-following neurons respond in opposite phase, they still contribute to the first harmonic of SSVEPs because the light and the dark responses are not exactly equal in strength (Figure 3C, the middle and lower traces). Our light–dark flicker stimulus thus effectively separates frequency-following and frequency-doubling neural population responses into the first and second harmonics of SSVEPs.

Procedure

Each trial was initiated by the observer’s button press. A single high-contrast (0.8) grating was presented on each trial either in the left or in the right visual hemifield (Figure 1A). The hemifield (left or right) and flicker frequency of the grating (6.25, 8.33, 12.50, 16.67, or 25.00 Hz) were randomly intermixed across 300 trials, and each condition occurred with equal probability. The flickered grating was presented after a 1-sec fixation screen displaying a central bull’s-eye, and it lasted 4.8 sec. Observers maintained eye fixation at the central fixation marker and withheld eye blinks while the flickered gratings were presented. Several practice trials were given initially, and breaks were allowed when needed.

Data Recording and Analysis

EEG activity was recorded using tin electrodes embedded in an elastic cap at locations distributed relatively evenly across the scalp. For 59 EEG channels, the right mastoid served as the reference during data acquisition, and data were re-referenced to the average of the left and right mastoids prior to analyses (e.g., Luck, 2005). Four additional channels were used for monitoring vertical and horizontal eye movements to reject trials contaminated by EOG artifacts. Electrode impedances were reduced to less than 5 kΩ. Signals were amplified with a band-pass filter of 0.3 to 200 Hz and digitized at 1000 Hz.

Individual trials were rejected from further analysis on the basis of blink or other artifacts detected in vertical EOG recordings. In addition, to retain only the trials in which central eye fixation was maintained, we recursively rejected trials with the highest horizontal EOG activity until the average horizontal EOG activity for each condition (i.e., each flicker frequency presented to each visual hemifield) for each observer was less than 5 µV during the entire 4.8-sec period of grating presentation. This is a stringent criterion that approximately corresponds to central fixation within 0.5° visual angle (e.g., Müller, Picton, et al., 1998; Luck et al., 1994; we verified that the quantitative relationship between horizontal EOG activity and saccade amplitude measured with our apparatus was comparable with those reported in previous studies). After these artifact-rejection procedures, we retained a mean of 88% of the trials.

EEG waveforms from the 59 scalp electrodes were averaged separately for each condition for each observer. To exclude the initial transient response to the grating onset, we analyzed EEG waveforms recorded from 526 to 4621 msec after grating onset. This yielded 4096 (2¹²) data points per trial. Reducing the number of EEG data points from each trial to a power of 2 is optimal for a fast Fourier transform analysis. To extract SSVEP activity synchronized to the stimulus flicker, we subjected each average waveform (corresponding to a specific condition) from each scalp electrode to a fast Fourier transform. The SSVEP amplitude was then computed as the Fourier band power within the range of 0.976 Hz centered at the first and second harmonics of the stimulus flicker.

Because the absolute values of EEG signals vary from observer to observer, partly because of individual differences in scalp/skull conductivity, data were standardized prior to combining across observers. Specifically, the SSVEP amplitude from each electrode in each condition for each observer was z transformed on the basis of the observer’s overall average and standard deviation of SSVEP amplitudes across all scalp electrodes and all conditions. We normalized each harmonic separately so that we could evaluate the topographic distribution and attentional modulation of each harmonic in standardized units of signal-to-noise ratio, thus controlling for the overall differences in response amplitude and random variability between the two harmonics. Note that this normalization procedure altered neither the spatial nor the temporal pattern of SSVEPs.

The relative strengths of various harmonic peaks prior to normalization are shown in the spectral plots (EEG in µV as a function of frequency; Figure 2). These data confirm that our light–dark flicker stimuli generated strong SSVEPs at both the first and second harmonics, with amplitudes well above the background activity for all flicker frequencies. These SSVEP amplitudes are also within the range of values reported in previous studies (e.g., Di Russo, Spinelli, & Morrone, 2001; Müller, Picton, et al., 1998; Morgan, Hansen, & Hillyard, 1996). The spectral plots also show that the amplitude of the first harmonic is generally equivalent for contralateral- and ipsilateral-posterior electrodes (see Figure 4B for an illustration of the locations of these electrodes),whereas the second harmonic is stronger for contralateral than ipsilateral electrodes (except for the lowest [Figure 2A] and highest [Figure 2E] flicker frequencies). Quantitative analyses of these harmonic-specific patterns of SSVEP scalp distribution are presented in the Results section.

Figure 4.

Results of Experiment 1. (A) Topographic plots of the first (upper row) and second (lower row) harmonics of the standardized SSVEPs elicited by the flickered grating, averaged across observers and flicker frequencies. Color-scale data were interpolated on the basis of a fine Cartesian grid. Positive and negative values indicate responses above and below the mean amplitude, respectively, in z units. The left column shows SSVEP topographies when the grating was presented to the right visual hemifield, and the right column shows SSVEP topographies when the grating was presented to the left visual hemifield. (B) Contralateral SSVEPs (gray bars) and ipsilateral SSVEPs (white bars) for the first and second harmonics averaged from the 10 indicated posterior scalp electrodes from which strong SSVEPs were obtained (see part A). The graphs confirm that the first harmonic was medial (nonlateralized) whereas the second harmonic was strongly contralateral. (C) The degree of lateralization (contralateral-minus-ipsilateral standardized SSVEPs) for the first (dotted curve) and second (solid curve) harmonics as a function of flicker frequency. The numbers within the plot represent the corresponding response frequencies (the same as the flicker frequencies for the first harmonic and doubled for the second harmonic). The asterisks indicate statistically significant lateralization (i.e., significant deviations from zero at p < .05). (D) The contralateral standardized SSVEPs (solid line) and ipsilateral standardized SSVEPs (dashed line) for the first harmonic (left panel) and second harmonic (right panel) as a function of flicker frequency. Error bars represent ± 1 SEM with the variance because of the overall differences across observers removed.

Results

SSVEPs averaged across all flicker frequencies showed a clear topographic segregation on the basis of response harmonics. The first harmonic showed a medial posterior focus regardless of whether the grating was presented in the left or right visual hemifield (Figure 4A, upper row). In contrast, the second harmonic showed a contralateral posterior focus (Figure 4A, lower row), with a left posterior focus in response to a grating presented in the right visual hemifield and a right posterior focus in response to a grating presented in the left visual hemifield.

To statistically evaluate this topographic segregation, we analyzed responses from ten posterior electrodes, five over each cerebral hemisphere (illustrated in Figure 4B). These scalp locations correspond to the overall posterior maximum of the SSVEPs (Figure 4A). The degree of response lateralization was measured as the difference in SSVEP amplitudes between the contralateral and the ipsilateral sets of electrodes. Whereas the first harmonic was similar for contralateral and ipsilateral electrodes, the second harmonic was substantially stronger for contralateral than ipsilateral electrodes (Figure 4B). The contralateral versus ipsilateral difference was significant for the second harmonic, t(9) = 5.434, p < .0005, but not for the first harmonic, t(9) = 1.329, ns; the ANOVA interaction between response harmonic (first vs. second) and scalp location (contralateral vs. ipsilateral) was also significant, F(1, 9) =10.72, p < .01. Data averaged across all flicker frequencies thus demonstrate a topographic segregation of the first and second harmonics, with the first harmonic localized to a medial-posterior scalp region and the second harmonic localized to a contralateral-posterior scalp region.

We next determined whether the harmonic-based SSVEP lateralization occurred over and above any frequency dependencies of SSVEP topographies. We quantified the degree of response lateralization as the contralateral-minus-ipsilateral responses, with larger positive values indicating stronger contralateral localization and values near zero indicating no lateralization. The degree of response lateralization for the first (dotted curve) and second (solid curve) harmonics is plotted as a function of flicker frequency in Figure 4C.

The first harmonic was not lateralized for any flicker frequency. In contrast, lateralization of the second harmonic exhibited a broad and inverted-U-shaped dependence on frequency. Whereas the second harmonic was strongly lateralized for the midrange response frequencies (16.67, 25, and 33.33 Hz), the lateralization disappeared for the lowest (12.50 Hz) and highest (50.00 Hz) response frequencies. These harmonic and frequency dependencies of SSVEP lateralization cannot be simply accounted for by the frequency dependence of SSVEP amplitudes. It was not the case that lateralization disappeared when the response amplitude was weak. Specifically, the amplitude of the second harmonic monotonically decreased with increasing frequency (Figure 4D, right panel), whereas the lateralization (the difference between the solid and the dashed curve) disappeared at both the lowest and the highest frequencies. Furthermore, the amplitude of the first harmonic peaked at 8.33 Hz, but there was no lateralization of the first harmonic regardless of its amplitude (Figure 4D, left panel).

Our results thus show that the first harmonic is generally nonlateralized whereas the second harmonic is selectively lateralized at response frequencies ranging from 16.67 to 33.33 Hz (the midrange). Interestingly, this frequency range largely overlaps the range of intrinsic local-field-potential frequencies that appear to be involved in top–down attentional feedback from pFC to posterior parietal cortex (e.g., Buschman & Miller, 2007) and to visual areas (e.g., Saalmann, Pigareve, & Vidyasagar, 2007). This overlap in frequency range between the lateralization of SSVEP second harmonic and the attention-dependent intrinsic neural synchronization may be related to our second finding that the SSVEP second harmonic is selectively modulated by visual–spatial attention (see Experiment 2). Further research, however, is needed to understand why lateralization of the second harmonic occurs within this relatively broad but specific range of response frequencies.

We next confirmed that the medial versus contralateral segregation of the first and second harmonics was due to differences in harmonics rather than due to differences in absolute response frequencies. Within the range of frequencies producing lateralization for the second harmonic (Figure 4C), the first harmonic elicited by the 16.67-Hz flicker and the second harmonic elicited by the 8.33-Hz flicker had an identical response frequency of 16.67 Hz, and the first harmonic elicited by the 25.00-Hz flicker and the second harmonic elicited by the 12.50-Hz flicker had an identical response frequency of 25.00 Hz. In both cases, with matched response frequencies, the contralateral-versus-ipsilateral difference in SSVEP amplitude was significant for the second harmonic, t(9) = 3.639, p < .006 for the 16.67-Hz response and t(9) = 3.866, p < .004 for the 25.00-Hz response, but not for the first harmonic, t(9) = 0.766, ns for the 16.67-Hz response and t(9) = 1.241, ns for the 25.00-Hz response, and the ANOVA interaction between the response harmonic (first vs. second) and the scalp location (contralateral vs. ipsilateral) was significant, F(1, 9) = 25.54, p < .001 for the 16.67-Hz response and F(1, 9) = 11.47, p < .01 for the 25.00-Hz response. These results confirm the harmonic-based topographic segregation of SSVEPs into medial posterior (first harmonic) and contralateral posterior (second harmonic) scalp regions, over and above any influence of response frequency. Given the relatively coarse spatial resolution of EEG signals, the clear harmonic-based topographic segregation demonstrated here is striking, suggesting that the visual system channels frequency-following and frequency-doubling processes into well segregated neural assemblies.

In the next experiment, we investigated the possibility that this harmonic-based segregation of neural population activity might contribute to resolving conflicting demands associated with attentional modulation of visual signals. Whereas the ability to selectively enhance behaviorally relevant aspects of sensory signals is important, it is also important to preserve undistorted sensory qualities.

EXPERIMENT 2: EFFECTS OF VOLUNTARY VISUAL–SPATIAL ATTENTION ON FREQUENCY-FOLLOWING AND FREQUENCY-DOUBLING VISUAL RESPONSES

We manipulated visual–spatial attention while the observer viewed two circular gratings presented to the left and right visual hemifields (Figure 1B). The observer voluntarily attended to either the left or the right grating while we recorded the SSVEPs elicited by both gratings. The two gratings were flickered at different frequencies so that we could simultaneously monitor the SSVEPs elicited by the attended and ignored gratings on the basis of frequency tagging. The contrast of the gratings was varied to determine how the two harmonics carried information about stimulus intensity.

Methods

Observers

Eight observers (5 men and 3 women, ages ranging from 23 to 45 years) participated (the second harmonic data from this experiment were previously reported in Kim, Grabowecky, Paller, Muthu, & Suzuki, 2007). All observers had normal or corrected-to-normal visual acuity, gave informed consent to participate, and were tested individually in a dimly lit room.

Stimuli and Procedure

These were the same as in Experiment 1 except for the following. Two gratings were simultaneously presented on each trial, one in the left and the other in the right visual hemifield (Figure 1B). One grating flickered at 12.50 Hz and the other at 16.67 Hz. We varied the contrast of the gratings across eight levels (0.00625, 0.0125, 0.05, 0.1, 0.2, 0.4, and 0.8, the same for both gratings) because previous studies showed that attention effects could depend on image contrast (e.g., Ling & Carrasco, 2006; Williford & Maunsell, 2006; Reynolds & Chelazzi, 2004; Morrone, Denti, & Spinelli, 2002; Di Russo et al., 2001). By examining attention effects on the contrast response functions of the first and second harmonics, we were able to determine how attention influenced the encoding of image contrast by frequency-following and frequency-doubling neural populations.

An arrow cue presented in the initial fixation screen indicated to the observer which grating to voluntarily attend during the 4.8-sec period. The use of a central cue (or a verbal instruction) to manipulate voluntary allocations of visual–spatial attention is a commonly employed technique in attention research involving human observers (e.g., Pastukhov, Fischer, & Braun, 2009; Ling & Carrasco, 2006; Suzuki, 2001, 2003; Suzuki & Cavanagh, 1997; Cheal & Lyon, 1991; Posner, Snyder, & Davidson, 1980; Sperling & Melchner, 1978; for a control experiment that verified that our observers deployed visual–spatial attention as instructed by the central arrow cue, also see Kim et al., 2007). Two directions of attention (left or right), two assignments of flicker frequencies (12.50 Hz on the left and 16.67 Hz on the right or vice versa), and eight contrast levels were randomly intermixed across 640 trials, and each condition occurred with equal probability.

Data Recording and Analysis

The EEG data were recorded and analyzed in the same way as in Experiment 1. We retained a mean of 89% of the trials after the artifact-rejection procedures.

Results

The first and the second harmonics were medially and contralaterally localized, respectively (replicating Experiment 1), even when two gratings with different flicker frequencies were simultaneously presented (Figure 5). This confirms that the topographic segregation of the first and second harmonics generalizes to the cases where multiple Fourier components are simultaneously present in the visual signal.

Figure 5.

Results of Experiment 2. The upper half shows topographic plots of the first harmonic (left) and second harmonic (right) of the standardized SSVEPs elicited by gratings presented in the right or left visual hemifield (indicated by the square around the grating icon), averaged across observers, grating contrast, and flicker frequency. For each harmonic, the upper row shows SSVEP topographies when the grating was ignored and the lower row shows SSVEP topographies when the same grating was attended. Color-scale data were interpolated on the basis of a fine Cartesian grid. Positive and negative values indicate responses above and below the mean amplitude, respectively, in z units. The lower half shows amplitudes of the first and second harmonics averaged from posterior scalp electrodes on the basis of their characteristic topographies; the first harmonic was averaged from the five medial posterior scalp electrodes, whereas the second harmonic was averaged from the five contralateral posterior scalp electrodes (see illustration). Attention modulated the second harmonic but not the first harmonic. The contrast response functions (i.e., standardized SSVEPs as a function of stimulus contrast) elicited by the attended gratings (solid curve) and ignored gratings (dotted curve) are shown for the first harmonic (bottom left) and second harmonic (bottom right). The fits are based on the Naka–Rushton equation. Error bars represent ± 1 SEM with the variance because of the overall differences across observers removed.

Importantly, the topographic plots show that the medial first harmonic was similar in amplitude whether or not the grating was attended (Figure 5, upper left). In contrast, the contralateral second harmonic was substantially boosted by attention (Figure 5, upper right). We evaluated these attentional modulations of the two harmonics at their respective scalp foci. That is, we compared the attentional modulation of the first harmonic recorded from medial posterior electrodes with the attentional modulation of the second harmonic recorded from contralateral posterior electrodes (see illustration in Figure 5). The attention effect (attended minus ignored) was significant for the second harmonic, t(7) = 6.975, p < .0003, but not for the first harmonic, t(7) = 0.259, ns; this asymmetric effect of attention on the two harmonics was confirmed by the significant harmonic (first vs. second) by attention (attended vs. ignored) ANOVA interaction, F(1, 7) = 44.39, p < .0005 (see bar graphs in Figure 5). This pattern of results was equivalent for the two flicker frequencies; that is, there was no significant three-way interaction among flicker frequency, harmonic, and attention, F(1, 7) = 0.020, ns; furthermore, for each flicker frequency, the attention effect was significant for the second harmonic, t(7) = 3.248, p < .02 for the 12.50-Hz flicker (25-Hz response) and t(7) = 4.996, p < .002 for the 16.67-Hz flicker (33.33-Hz response), but not for the first harmonic, t(7) = 0.145, ns for the 12.50-Hz flicker and t(7) = 0.966, ns for the 16.67-Hz flicker. It is possible that a stronger allocation of attention might have modulated the first harmonic. Nevertheless, our results clearly demonstrate that visual spatial attention modulates the second harmonic substantially more strongly than the first harmonic.

Because our light–dark flicker dissociated frequency-following and frequency-doubling neural responses into the first and second SSVEP harmonics (see Figure 3C), the implications are straightforward. Responses of frequency-doubling neurons (reflected in the second harmonic) are contralateral regardless of attention and boosted when the stimulus is attended, whereas responses of frequency-following neurons (reflected in the first harmonic) are medial and little affected by attention.

We note that this result is not in conflict with prior reports of a lateralized and attention-modulated first harmonic in response to an on–off flicker (e.g., Müller, Malinowski, Gruber, & Hillyard, 2003; Belmonte, 1998; Müller, Picton, et al., 1998; Müller, Teder-Sälejärvi, & Hillyard, 1998). As explained in the Methods section for Experiment 1, an on–off flicker produces frequency-following and frequency-doubling responses that are both primarily contained in the first SSVEP harmonic (see Figure 3B). Thus, the lateralization and attentional modulation of the first harmonic in the prior studies can be attributed to a contribution from frequency-doubling neurons.

Furthermore, a seemingly puzzling finding in the SSVEP literature can be understood in light of our results. Müller, Picton, et al. (1998) reported that the first harmonic in response to an on–off flicker was medial (bilateral) when the stimulus was ignored but lateralized when it was attended (their Figure 10). By most accounts, it would be odd for a sensory-evoked steady-state response to change its scalp topography depending on attention; attention should either strengthen or weaken a stimulus-evoked response. Our results suggest that the first harmonic of SSVEPs elicited by an ignored on–off flicker would be dominated by frequency-following neural activity which is strong regardless of attention, producing a medial (bilateral) topography. When the same stimulus is attended, the attention-boosted and lateralized frequency-doubling neural activity (also contributing to the first harmonic due to the use of an on–off flicker) would induce a contralateral topography for the first harmonic. Thus, our results integrate with and extend those from prior SSVEP studies that used an on–off flicker.

DISCUSSION

We designed a light–dark-flickered stimulus to separate the responses from frequency-following and frequency-doubling populations of visual neurons into the first and second harmonics of SSVEPs (Figures 1 and 3C). We then demonstrated that frequency-following and frequency-doubling neural population responses are topographically segregated, with frequency-following responses maximal over the medial posterior scalp and frequency-doubling responses maximal over the contralateral posterior scalp. We have further shown that frequency-following responses are little affected by visual spatial attention in our experimental paradigm whereas simultaneously induced frequency-doubling responses are strongly boosted by attention.

What might be the neural sources of the topographically segregated frequency-following and frequency-doubling neural population responses recorded from the scalp? Neurophysiological research has suggested that frequency-following responses primarily originate from simple cells and frequency-doubling responses from complex cells in V1 (e.g., De Valois et al., 1982; Hubel & Wiesel, 1968). Because distributions of simple and complex cells overlap in V1 (e.g., Shapley, 2004), however, it is unlikely that the topographically segregated first and second harmonics of SSVEPs reflect the population responses from simple and complex cells in V1.

It is possible that frequency-following population responses primarily reflect low-level visual processes whereas frequency-doubling population responses primarily reflect high-level visual processes. This hypothesis is consistent with the fact that we obtained strong attentional modulation for the second SSVEP harmonic and little attentional modulation for the first SSVEP harmonic because attentional modulation of neural responses tends to be stronger in higher visual areas (for a review, see Suzuki, 2001, 2005; Pessoa, Kastner, & Ungerleider, 2003). This simple hypothesis, however, is not consistently supported by prior EEG studies that attempted to estimate anatomical sources of SSVEP harmonics.

For example, Pastor, Valencia, Artieda, Alegre, and Masdeu (2007) attempted to localize the sources of the first and second harmonics by using a combination of the LORETA algorithm and PET, assuming that the SSVEP amplitude and the CBF covary as a function of response frequency. They inferred that the first harmonic primarily originated from the pericalcarine occipital visual cortex whereas the second harmonic primarily originated from the inferior half of the parieto-occipital sulcus. Their results are consistent with ours in that the inferred source of the second harmonic is more laterally extended than the inferred source of the first harmonic. However, because they flickered the entire visual field (using a strobe lamp), they were unable to evaluate response lateralization.

Using counterphase-flickered checkerboards presented in different quadrants of the visual field, Di Russo et al. (2007) attempted to localize the sources of the second harmonic using a combination of the BESA algorithm and fMRI. They inferred that the early phase (0°–60° delay) of the second harmonic primarily originated from the medial occipital region (approximately V1) whereas the later phase (80°–120° delay) of the second harmonic primarily originated from the contralateral occipital region (approximately MT/ V5). Our result of contralaterally maximal second harmonic is broadly consistent with their results. However, we did not find a similar topographic distribution on the basis of phase delay. When we compared phase delays across the contralateral, ipsilateral, and medial sets of electrodes (see Figure 5), we found no significant differences. It is possible that we did not obtain a phase-based topographic segregation because of stimulus and frequency differences between our study and Di Russo et al. We used a large (6°) stimulus presented on the horizontal meridian, whereas they used a small (2°) stimulus presented off the vertical and horizontal meridian. We used a broad range of flicker frequencies including fast ones (6.25–25 Hz), whereas they used 6 Hz only. The phase topography that Di Russo et al. obtained might thus be specific to a small, relatively slowly flickered stimulus presented off the meridian. Note that Di Russo et al. did not analyze the first harmonic because their SSVEPs were dominated by the second harmonic because of their use of a counterphase flicker (see Figure 3A).

Using on–off-flickered random dots covering 10° of the central visual field, Srinivasan, Bibi, and Nunez (2007) attempted to localize the sources of the first harmonic using surface Laplacians and spatial spectral analyses. They inferred multiple local and distributed neural sources for the first harmonic spread across occipital, parietal, and frontal regions, depending on the flicker frequency. This result is clearly inconsistent with the simple idea that the first harmonic reflects low-level processes and second harmonic reflects high-level processes.

The divergent results from these SSVEP studies may partly reflect the use of different stimuli (whole-field flicker, counterphase flicker, on–off flicker) and partly reflect the use of different source localization methods. Given that these methods entail various assumptions about the anatomical and electrical properties of the brain that influence the inferred location of SSVEP sources, it is difficult to speculate on the exact brain regions that mediate our first and second harmonic responses.

Neurophysiological results also do not suggest any obvious origin for the broad scalp segregation of frequency-following and frequency-doubling neural population responses that we obtained. Both frequency-following and frequency-doubling responses occur as early as in V1, and they persist to some degree in higher visual areas such as V2, V4, and inferotemporal cortex (e.g., Hanazawa & Komatsu, 2001; Ito et al., 1994; Desimone & Schein, 1987; Foster et al., 1985; De Valois et al., 1982; also see the Introduction section). Future research is necessary to determine how visual neurons that produce frequency-following and frequency-doubling responses are organized in such a way that their aggregate local-field potentials (e.g., Varela, Lachaux, Rodriguez, & Martinerie, 2001) generate clear medial versus contralateral topographies measurable on the scalp. It is possible that V1 is organized so that frequency-following neurons generate stronger aggregate local-field potentials, whereas extrastriate visual areas are organized so that frequency-doubling neurons generate stronger aggregate local-field potentials, but it is also possible that the topographic segregationwe obtained derives from globally different patterns of organization or synchronization between frequency-following and frequency-doubling neural populations, spanning multiple visual areas.

Irrespective of the absolute anatomical origins of the segregated frequency-following and frequency-doubling neural population responses, our attention results suggest that these populations contribute complementary functions. On one hand, substantial attentional modulation of visual processing is desirable to selectively process behaviorally relevant signals. On the other hand, it is also necessary to preserve relatively undistorted sensory signals to correctly encode environmental information. Behavioral results indicate that these goals are generally met in the human visual system. Attention substantially modulates stimulus salience and detectability (e.g., Suzuki, 2003; Simons, 2000; Blaser, Sperling, & Lu, 1999) while at the same time only modestly modulating perceived image qualities such as contrast, color, and spatial frequency (e.g., Carrasco, Ling, & Read, 2004; Prinzmetal, Amiri, Allen, & Edwards, 1998; Prinzmetal, Nwachuku, & Bodanski, 1997).

Our results suggest that frequency-doubling neurons substantially contribute to top–down control of visual salience whereas frequency-following neurons simultaneously contribute to preservation of stimulus features regardless of the observer’s level of attention. Whereas attention substantially and multiplicatively boosted the second harmonic (see the bottom right graph in Figure 5; for quantitative confirmation of multiplicative boosting of the second harmonic, also see Kim et al., 2007), the contrast information encoded by the first harmonic remained unchanged regardless of attention (see the bottom left graph in Figure 5). It makes sense to preserve sensory quality in frequency-following neurons because their luminance-polarity selectivity allows them to encode surface features such as lightness, darkness, and shading, for which accurate encoding of magnitude is important. Note that frequency-following neurons also preserve stimulus dynamics as they reproduce temporal Fourier components of the stimulus.

The visual system takes advantage of partially segregated processing streams to allow separate, and often complementary, computations to be concurrently performed on sensory signals. Classic examples include magnocellular (extracting lower spatial and higher temporal frequencies especially for low contrast input) versus parvocellular (extracting higher spatial and lower temporal frequencies as well as spectral information) subcortical pathways (e.g., Shapley, 1995; Merigan & Maunsell, 1993; Schiller, Logothethis, & Charles, 1990) and dorsal (extracting spatial, spatiotemporal, and action-related information) versus ventral (extracting object information) cortical visual pathways (e.g., Fang & He, 2005; Goodale & Westwood, 2004; Mishkin et al., 1983; but for a modified version of the traditional view, see Konen & Kastner, 2008). Our results suggest that the ubiquitous presence of frequency-following and frequency-doubling visual neurons may add another pair of partially segregated streams, contributing to the goal of simultaneously accomplishing both substantial top–down modulation of neural signals and preservation of relatively undistorted sensory qualities.