Investigating neural correlates of conscious perception by frequency-tagged neuromagnetic responses

Abstract

In binocular rivalry, a subject views two incongruent stimuli through each eye but consciously perceives only one stimulus at a time, with a switch in perceptual dominance every few seconds. To investigate the neural correlates of perceptual dominance in humans, seven subjects were recorded with a 148-channel magnetoencephalography array while experiencing binocular rivalry. A red vertical grating flickering at one frequency was presented to one eye through a red filter and a blue horizontal grating flickering at a different frequency was presented to the other eye through a blue filter. Steady-state neuromagnetic responses at the two frequencies were used as tags for the two stimuli and analyzed with high-resolution power spectra. It was found that a large number of channels showed peaks at both frequencies, arranged in a horseshoe pattern from posterior to anterior regions, whether or not the subject was consciously perceiving the corresponding stimulus. However, the amount of power at the stimulus frequency was modulated in relation to perceptual dominance, being lower in many channels by 50–85% when the subject was not conscious of that stimulus. Such modulation by perceptual dominance, although not global, was distributed to a large subset of regions showing stimulus-related responses, including regions outside visual cortex. The results demonstrate a correlation between the conscious perception of a visual stimulus and the synchronous activity of large populations of neurons as reflected by steady-state neuromagnetic responses.

Binocular rivalry is a useful experimental paradigm for identifying aspects of neural activity that are correlated with conscious experience (1, 2). If two incongruent visual stimuli are simultaneously presented one through each eye, only one stimulus at a time is consciously perceived, and the two percepts alternate every few seconds. It was thought initially that rivalry might reflect competition between monocular neurons in primary visual cortex or at earlier stages. However, recent psychophysical studies have demonstrated that perceptual rivalry can occur even when both stimuli are presented through the same eye or when they are alternated between the eyes (2). Furthermore, single-unit recordings during binocular rivalry in monkeys indicate that, while the firing of most neurons in primary visual cortex correlates with the stimulus but not with the percept (3), the firing of cortical units in higher visual areas, such as inferior temporal cortex and superior temporal sulcus (4), is highly correlated with the visual percept.

Human subjects are the referent of choice for investigating conscious perception (5). However, brain activity associated with rivalry is difficult to study in humans with techniques such as positron emission tomography and functional MRI because of their limited temporal resolution. Unit recordings, on the other hand, while offering high temporal resolution as well as neuronal specificity, are typically performed in overtrained animals and are not practical for providing global coverage of neural responses. At the expense of spatial resolution, magnetoencephalograms (MEGs) and electroencephalograms (EEGs) offer the advantage of high temporal resolution and reflect the synchronous activity of large populations of neurons (6, 7). In this study, we made use of a 148-channel MEG array to compare whole-head, steady-state-evoked responses when subjects viewing a stimulus were consciously perceiving it and when they were not. Rivalry was produced by presenting red vertical gratings to one eye and blue horizontal gratings to the other eye. The subjects signaled which of the two stimuli was consciously perceived (perceptual dominance) by activating right- or left-hand switches. To evaluate brain responses specific to each reported percept a method of “frequency tagging” was employed. Each of the two stimuli was flickered at a different frequency in the range of 7–12 Hz, and steady-state evoked responses at the tag frequency specific to each stimulus were detected in many MEG channels. These responses allowed us to assess how stimulus-related signals were distributed over the whole head when a particular stimulus was consciously perceived and when it was not. They also allowed us to establish whether such responses were modulated in relation to conscious perception, and to determine whether such modulation was global or regionally specific.

METHODS

Seven right-handed subjects (five males and two females) of ages 25–49 participated in this study. Each had a corrected visual acuity of 20/20 and could see large-disparity random-dot stereograms. All subjects gave informed consent. Neuromagnetic data were collected using a Magnes 2500WH MEG system from Biomagnetic Technologies (San Diego). This consists of an array of 148 magnetometer coils (1-cm diameter) spaced ≈3 cm apart and providing coverage of the entire scalp. The MEG array was in a magnetically shielded room, and computer-generated stimuli were projected through a porthole and a single mirror onto a screen in front of the subjects.

In each trial, subjects viewed high-contrast (>95%) square-wave gratings of 1.7 cycles per degree in a square field subtending a visual angle of 13° at the fovea over a uniform dark gray background. The size and intensity of the stimulus were adjusted to generate complete or near-complete rivalry, with sufficiently long episodes of dominance (mean of 2 sec), and a high signal-to-noise-ratio (SNR) in the recorded signal. A vertical red grating was presented to one eye and a horizontal blue grating to the other eye by having subjects wear correspondingly colored goggles. The intensity of the red stimulus was adjusted such that, under conditions of rivalry, the subjects reported that the two stimuli were of comparable brightness. To aid convergence, subjects viewed a dim gray fixation point at the center of each grating.

In rivalry trials, one stimulus (s1) was flickered continuously at one frequency (f1) and the other stimulus (s2) was flickered at a different frequency (f2). For each f1-f2 pair, two of the following frequencies were used: 7.41 Hz (one grating-on frame every 9 video frames, at 67 frames/sec), 8.33 Hz, 9.50 Hz, or 11.12 Hz. A photodiode recorded in real time the flicker of s1 and s2 on a computer screen driven in parallel with the projector. Subjects were asked to activate one switch with their left index finger whenever the red stimulus was perceptually dominant and another switch with the right index finger whenever the blue stimulus was dominant. They were instructed to activate neither switch if neither of the two percepts was clearly dominant, i.e., when they saw a mixture of red vertical and blue horizontal gratings. The activation of the switches was recorded by an additional channel.

After a brief exposure to the stimuli, subjects had no trouble categorizing the percepts as red, blue, or mixed, and the alternation of perceptual dominance between the percepts became quite stable. If asked, the subjects reported perceiving the stimulus flicker, but they did not comment on any difference in frequency between f1 and f2. During the experiments, stimuli were presented for 30–60 sec before recording to establish a steady-state response. Neuromagnetic data were then recorded for 315 sec, resulting in a frequency resolution of 0.0032 Hz. Data were bandpass filtered at 1–50 Hz and digitized at 254 Hz. For each channel, the power spectrum of the entire recording interval was calculated by using a fast Fourier transform algorithm (MATLAB, Natick, MA). The peaks corresponding to f1 and f2 were identified in the spectrum of the photodiode signal, and the presence of peaks in the MEG data at the corresponding frequencies was verified. These peaks were contained within a single bin of 0.0032-Hz width. The recording of the behavioral response yielded two response functions r1 and r2, which were set to 1 during the intervals when the subject signaled that stimulus s1 or s2, respectively, was perceptually dominant, and to 0 otherwise. The values of r1 and r2 for episodes lasting less than 250 msec were set to 0.

To obtain the power corresponding to the periods when the subject was consciously perceiving s1 (perceptual dominance), the MEG data were multiplied by r1 prior to the Fourier transform. The power corresponding to the periods when the subject was not conscious of s1 (perceptual nondominance, defined as the periods when the subject was conscious of s2) was calculated by multiplying the MEG data by r2 before the fast Fourier transform. The power values at f1, normalized by the total duration of positive intervals in r1 and r2, were subtracted to yield the difference at f1 between perceptual dominance and perceptual nondominance. Multiplying MEG time data by the response function corresponds to convolving the respective frequency spectra and results in some smearing. Numerical simulations indicated, however, that the contamination of the signal peak was negligible compared with the size of the effects observed in this study.

To emphasize the effect of perceptual dominance/nondominance over stimulus-specific factors, stimulus-frequency pairings and stimulus-eye pairings were counterbalanced for each subject so that, for each frequency pair, each stimulus was presented at each frequency and to each eye for a total of four trials. Two different frequency pairs were used successively in the rivalry condition, with one frequency common to both pairs, yielding eight trials at that frequency. The average power difference at the common frequency across the eight rivalry trials was calculated.

To compare power differences between perceptual dominance and nondominance due to binocular rivalry with power differences due to the physical presence or absence of the stimulus, stimulus-alternation trials were used. In such trials, stimulus s1 alone was presented to one eye at frequency f1 for a random interval of time, after which stimulus s2 alone was presented to the other eye at frequency f2 for another random interval, and so on for 315 sec. The time intervals were drawn from a γ distribution (2) with a mean of 2 sec and a SD of 1 sec. Stimulus-alternation trials were performed with only one frequency pair. A total of 12 trials were performed in a session that lasted 2–3 hr.

A randomization test (8) was employed to assess the statistical dependence of the power difference between perceptual dominance and nondominance on the response functions r1 and r2. For each subject, bootstrap samples were computed by systematically permuting the pairing between the MEG data for the eight rivalry trials and their response functions. A total of 8! = 40,320 possible pairings were available, including the observed pairing. For each bootstrap sample, the power difference was computed for each trial in the same manner as for the observed data but by using the randomly assigned response functions. The sum of squared power differences across all the sensors was used as an omnibus statistic. The significance of the observed omnibus power difference was established by comparing it with the distribution of the omnibus power differences obtained from the bootstrap samples. After establishing the significance of whole-array (global) differences, power-difference values were plotted topographically to examine regional contributions.

RESULTS

The average duration of the episodes of perceptual dominance in rivalry trials was 2.1 ± 1.1 sec. In most subjects, the number and length of intervals in which the red and blue gratings were perceived were comparable (on average, 54 episodes for the red grating and 55 for the blue grating in each trial). For 5–25% of the total recording time neither stimulus was perceptually dominant.

High-resolution power spectra of steady-state evoked potentials recorded over posterior and anterior regions during a rivalry trial are shown in Fig. 1 Upper Left. Two peaks are clearly visible, one at 7.41 Hz and the other at 9.50 Hz. Note that each peak occupies just one bin (bin size = 0.0032 Hz). Note also that the magnitude of each peak is much higher than the average power in nearby frequency bins, corresponding to a SNR of 25–50. Finally, the amplitude of the peak at 9.5 Hz is higher at posterior channels (e.g., channel 103), corresponding to occipital cortex, than at anterior channels (e.g., channel 128). These narrow peaks elicited by each stimulus served as frequency tags to identify neural activity that was directly or indirectly related to each stimulus. Steady-state responses completely disappeared when the corresponding eye was occluded.

Figure 1.

(Upper Left) High-resolution power-frequency spectra for steady-state evoked potentials recorded over an anterior channel (128) and over a posterior channel (103) during rivalry trials (subject O.S.). Note the sharp peak at 7.41 Hz (f1), the flicker frequency of s1, as well as at 8.33 Hz (f2), the flicker frequency of s2. The peak is confined to 1 frequency bin (0.0032 Hz). The SNR, defined as the ratio of the power at the peak and the average power in a 0.06 Hz band (40 bins) surrounding it, is 25.0 (7.41 Hz, anterior channel), 29.7 (8.33 Hz, anterior channel), 39.2 (7.41 Hz, posterior channel), and 48.9 (8.33 Hz, posterior channel). A broad-band peak in the alpha range is visible at the posterior channel. (Upper Right) Schematic topographic representation of the 148 channels in the MEG array. For convenience, a few points designated based on the ten-twenty electrode placement system are superimposed: F, frontal; C, central; P, parietal; O, occipital; and T, temporal. The locations of channels 128 and 103 are indicated by filled blue circles. (Lower) Topographic display of signal power at the stimulus flicker frequency of 7.41 Hz. The topographic maps were generated by interpolating the amplitude values (square root of power) at 148 sensors on a best-fit sphere with a three dimensional spline. The sensor positions on the best-fit sphere are indicated by dots. The map is then projected from the sphere onto a plane. Channels meeting a SNR criterion of at least 2 are indicated by an open circle. (Left) Stimulus-alternation trials. In this and in the subsequent figures, the values represent an average of four trials. (Right) Rivalry trials. In this and the subsequent figures, the values represent an average of eight trials in which the stimulus flicker frequency 7.41 Hz was associated with either the red vertical grating or the blue horizontal grating and presented to either the right or the left eye (see Materials and Methods). Note the typical horseshoe distribution of the peak in power at 7.41 Hz, which is similar under stimulus-alternation and rivalry conditions (subject O.S.). Solid contour lines begin at 1 picotesla in steps of 0.1 picoteslas. Dashed contour lines range from 0 to 0.9 picoteslas.

Fig. 1 (Lower) shows the topographic distribution of power at the peak frequency of 7.41 Hz for stimulus-alternation and rivalry trials of one subject (O.S.). The power values were obtained by averaging all trials in which a stimulus flicker frequency of 7.41 Hz was presented. In both stimulus-alternation and rivalry trials, a horseshoe-shaped distribution of the peak at 7.41 Hz was observed, with maximum amplitude over posterior regions.

The power at 7.41 Hz (f1) when s1 was perceptually dominant and when it was perceptually nondominant (defined as when s2 was perceptually dominant) was calculated for each channel by multiplying the MEG data by the response functions r1 and r2, respectively, and subtracted to yield a power difference value (see Materials and Methods). The power difference values at 7.41 Hz were calculated with the response function offset from the neuromagnetic data by an offset time τ ranging from −2.5 to +2.5 sec in steps of 250 msec. The offsets were introduced to take into account the variable relationship between the motor output and the establishment of the steady-state response. The former depends upon the reaction time and the strategy used for perceptual decision, while the latter depends on the speed at which the steady-state response is modulated, all of which may vary across subjects.

For stimulus-alternation trials, the power differences at 7.41 Hz between the periods during which the stimulus flickering at that frequency was being presented and during which the other stimulus was being presented are shown in Fig. 2 Left) as a function of offset time τ. In the figure, the contour lines in magenta indicate a positive difference in power, while green lines indicate a negative difference in power. The figure shows a strong positive power difference for most channels that starts at τ = -250 msec and lasts until τ = +1.5 sec; the response peak is at τ = +0.25 sec. A negative difference, of reduced amplitude, is noticeable at earlier and later offsets. Such negative differences occur because on average, a 2-sec interval in which s1 is dominant is preceded and followed, at τ = ± 2 sec, by a 2-sec interval during which s2 is dominant (and hence s1 is nondominant). Correspondingly, the interval between positive and negative peaks is ≈2 sec. Note that the time course of the amplitude difference suggests that the steady-state response takes time to develop and that its peak value can occur after the onset of the behavioral response.

Figure 2.

Power difference values between perceptual dominance and nondominance for all channels at different offsets (τ) of the response function. (Left) Stimulus-alternation trials. The contour lines in magenta indicate a positive difference in power, while green lines indicate a negative difference in power. The magnitude of the power difference is indicated by the number of contour lines. Contour lines begin at 0.05 picotesla² in steps of 0.025 picotesla². (Right) Rivalry trails (subject O.S.). Note that for most channels the maximum power difference occurs at τ = 0.25 sec for stimulus-alternation trials, and at τ = 0.75 sec for rivalry trials.

In Fig. 2 Right a similar plot is shown for the rivalry condition in the same subject. The plot represents the average of eight trials in which the stimulus was presented at a flicker frequency of 7.41 Hz. In this case there is again a positive power difference in many channels, straddling τ = 0 and surrounded by negative differences. Note, however, that both the magnitude of the difference and the number of channels involved are reduced with respect to stimulus-alternation trials. Furthermore, the maximum positive difference occurs at a longer time offset than in stimulus-alternation trials (τ = 0.75 rather than τ = 0.25). As in stimulus-alternation trials, the presence of negative differences is due to a degree of periodicity manifested by the intervals of perceptual dominance (mean alternation interval = 2.9 sec). In different subjects, the peak positive differences occurred at different times, presumably reflecting differences between subjects in reaction time as well as in the strategy adopted in deciding when a percept was dominant. However, within each subject the peaks occurred at the same offset τ across trials. All subsequent analyses were performed with the reference functions offset at each subject’s characteristic offset time.

Fig. 3 shows topographic maps of the power at 7.41 Hz corresponding to the episodes of perceptual dominance, the power corresponding to the episodes of perceptual nondominance, and the difference in power between dominance and nondominance. In stimulus-alternation trials (Fig. 3 Left), the distribution of power differences was approximately coextensive with the distribution of the steady-state power during dominance. During nondominance there was no stimulus at that frequency and, as expected, there was no power contribution.

Figure 3.

Topographic display of power at 7.41 Hz corresponding to perceptual dominance (Top), to perceptual nondominance (Middle), and to the difference in power between dominance and nondominance (Bottom), at the offset for which the difference was maximal. Amplitude values (square root of power) are plotted. (Left) Stimulus-alternation trials. During nondominance, there is no amplitude contribution at the frequency of the absent stimulus. The difference in amplitude is coextensive with the distribution of stimulus-related responses. (Right) Rivalry trials. Note that the distribution of stimulus-related responses during nondominance is similar to that during dominance. Many channels show, however, amplitude values that are lower by 50–85% during nondominance. A positive difference in amplitude between perceptual dominance and nondominance is observed bilaterally at occipital, temporal, and frontal channels (subject O.S.). The omnibus significance of the map was computed as described in Materials and Methods (P < 0.05).

For rivalry trials (Fig. 3 Right), the steady-state responses at 7.41 Hz during perceptual dominance and nondominance were distributed in a similar way. However, a marked difference in power was observed according to whether the stimulus was consciously perceived or not. In many channels, the power was 50–85% lower during perceptual nondominance than during perceptual dominance, while in some channels the opposite was true. The difference in power was statistically significant (P < 0.05) using a conservative randomization test (see Materials and Methods). It is evident from the figure that the difference in power between dominance and nondominance extends to many but not all the channels showing a stimulus-related response. For the subject shown in Fig. 3 Right, a positive difference is observed bilaterally over occipital, frontal, and temporal regions. Smaller negative differences are observed in fewer channels over central and frontal regions. Several channels in which a consistent stimulus-related response was observed (SNR ≥ 2), did not show any modulation. Power difference values between perceptual dominance and nondominance for four other subjects are shown in Fig. 4. All subjects showed a marked power differences as a function of perceptual dominance at occipital, temporal, and frontal regions, although the particular set of modulated channels varied across subjects.

Figure 4.

Topographic display of power differences between perceptual dominance and nondominance in four other subjects. Amplitude values (square root of power) are plotted. The frequency tested for each subject was: R.G. (8.33 Hz), S.P. (7.41 Hz), L.G. (7.41 Hz), and M.T. (7.41 Hz). The values are based on eight runs counterbalanced across eyes and color. The omnibus significance of the maps was P < 0.005 for all subjects.

DISCUSSION

In this study, visual steady-state neuromagnetic responses to two rivalrous stimuli presented at different frequencies were simultaneously recorded over many cortical areas. These neuromagnetic responses, labeled by frequency tags, were used to determine how brain activity differs, under rivalry conditions, when a human subject is conscious of a stimulus and when the subject is not.

The present experiments resulted in several significant observations. A first observation is that neural responses to rivalrous visual stimuli occurred in a large number of cortical regions both when the subject consciously perceived the stimuli and when he did not. Moreover, such evoked responses extended to anterior areas, the activity of which has not previously been examined during binocular rivalry. The flickering stimuli used in this study activated primary visual areas directly through thalamocortical inputs. The recording of stimulus-related responses in other cortical areas, including anterior regions, is presumably due to neural circuits linking visual cortex to anterior regions through direct or indirect connections (6). It cannot be ruled out, however, that subcortical inputs may also have contributed to the signals recorded in anterior areas (9).

The second main finding of this study is that the neuromagnetic responses evoked by a stimulus over a large portion of the scalp were stronger when the subjects were conscious of it than when they were not. Interestingly, the sign of this effect was opposite in a subset of the channels. The magnitude of the modulation due to rivalry was of the order of 50–85%, to be compared with the 100% modulation due to the physical presence/absence of the stimulus in stimulus-alternation trials. The specific subset of channels showing such modulation, which varied from subject to subject, included occipital channels but was not restricted to them. Examination of single trials suggests that variations in the topography of such modulation depends on experimental variables such as frequency, eye, and color and these, as well as significant intersubject variances, deserve further investigation.

Previous EEG studies using a few occipital electrodes have reported that visual evoked potentials recorded over occipital cortex were suppressed when a rivalrous stimulus was introduced (10–12). Studies using orthogonal gratings that were modulated in counterphase (13) or tagged with different flicker frequencies (14) also reported that the amplitude of the visual evoked potential generated by the perceptually dominant stimulus was larger that that of the suppressed stimulus. In a recent study using the latter approach, the amplitude of the visual evoked potential induced by a stimulus presented to one eye was positively correlated with its perceptual dominance in real time (15). The finding of a strong modulation of evoked responses over occipital areas in the present experiments and in previous EEG studies contrasts with the results of single-unit recordings. In monkeys, only a small fraction of neurons in early visual areas showed activity that correlated with perceptual dominance, in sharp contrast with units in higher visual areas (4). This apparent discrepancy may be accounted for in part by differences in the experimental protocol, such as the use of flickering vs. nonflickering stimuli. Most importantly, the steady-state responses recorded in the present study are sensitive to the synchronous activation of a large number of synapses rather than to the firing levels of individual units. Changes in synchronization among large populations of neurons due to reentrant interactions (16, 17) are a prerequisite for the modulation of electrical and magnetic potentials recorded at the scalp (6, 17). Consistent with this interpretation, it has recently been reported that, in strabismic cats, perceptual dominance under conditions of binocular rivalry is associated with increased synchronization in early visual areas, while perceptual suppression is associated with reduced synchronization (18).

In the present study, the analysis of steady-state responses at 148 MEG sensors covering the whole head allowed us to detect stimulus-related signals with high SNR in many brain areas in addition to visual cortex. The results obtained here show that the modulation of the evoked responses by perceptual dominance extends to a large subset of the brain areas that respond to the stimulus, including lateral and anterior regions that are not part of visual cortex. A possible interpretation is that the modulation in response amplitude in early visual areas indirectly affects the responses of other areas to which visual areas are functionally connected (19). Irrespective of the specific mechanisms, these findings indicate that neural correlates of the conscious perception of visual stimuli extend to areas beyond visual cortex.

When the stimulus was consciously perceived, the corresponding frequency tag was distributed almost as widely in rivalry trials as in stimulus-alternation trials. As expected, in stimulus-alternation trials the stimulus-related response disappeared at every channel when the stimulus was not physically present. In rivalry trials, by contrast, when the stimulus was not consciously perceived the stimulus-related response was modulated at many but not all the sensors where it was detected. Thus, the present findings show that neural responses that correlate with conscious experience are not global but are distributed to a subset of brain regions. Nevertheless, the widespread modulation of neuromagnetic responses observed here implies that changes in the synchronous firing of large, distributed populations of neurons are associated with changes in perceptual dominance.

Some intrinsic limitations of the present study should be pointed out. A precise correspondence between signals recorded at different channels and neural activity in underlying cortical areas cannot be established unless further assumptions are made, for example by applying source models of neuromagnetic field generation. Previous studies achieving whole-head coverage with dense arrays of EEG electrodes have demonstrated, however, that steady-state visual evoked responses in areas other than occipital visual areas are in part due to local generators and are not far-field potentials (6). Similarly, it is likely that MEG steady-state responses recorded at anterior channels reflect significant anterior sources. Another methodological limitation is that, unlike single-unit recordings, neuromagnetic recordings cannot disentangle the responses of individual neurons that have different stimulus preferences but are spatially intermingled within a brain region. For example, the activity of neurons that fire when their preferred stimulus is not perceptually dominant could be confounded with that of neurons that fire to the competing stimulus when it is dominant. Such neurons have in fact been recorded in area MT (2). Furthermore, steady-state responses are insensitive to neural activity that is correlated with perceptual rivalry but is either sustained without being time locked to the flickering stimulus, or is present only transiently at the perceptual switch and therefore does not contribute sufficient power.

Despite these limitations, there are obvious advantages to the use of steady-state evoked responses. Frequency tagging provides the ability to sharply differentiate stimulus-related responses from background neuronal activity with high temporal resolution (20). In combination with whole-head MEG, it permits the investigation of the distribution of stimulus-related signals beyond sensory projection areas. Unlike single-unit recordings, which are not practical for global coverage of neural activity and are generally performed in overtrained animals, steady-state evoked responses permit one to sample the synchronous activity of large populations of neurons in human subjects who are not overtrained (6, 7). Frequency tagging also offers great potential for generalization, because it can be applied to stimuli in any sensory modality, provided that the frequencies used elicit widespread stimulus-related responses. As shown here, frequency tagging can be used to study neural correlates of conscious experience in human subjects who can directly report their conscious states. Further studies using the frequency tag methodology under conditions of rivalry or attentional modulation may help in delineating the cortical regions that contribute to conscious experience.

ABBREVIATIONS

MEG

magnetoencephalogram

SNR

signal-to-noise-ratio

EEG

electroencephalogram