Covariation of activity in visual and prefrontal cortex associated with subjective visual perception

Abstract

Visual areas of the occipitotemporal pathway are thought to be essential for the conscious perception of objects, but the contribution of other cortical regions and the neural mechanisms leading to the awareness of a visual stimulus remain unclear. By using functional MRI in humans exposed to bistable viewing conditions, subjective visual perception was related to covariation of activity in multiple extrastriate ventral, parietal, and prefrontal cortical areas. The coordination of activity among these regions was not linked to external sensory or motor events; rather, it reflected internal changes in perception and varied in strength with the frequency of perceptual events, suggesting that functional interactions between visual and prefrontal cortex may contribute to conscious vision. Because similar cortical systems have been implicated in short-term memory and motor planning, the results also imply that related neural processes may underlie visual awareness and the organization of voluntary behavior contingent on sensory cues.

Despite enduring interest in the neural basis of visual awareness, the exact nature of this phenomenon remains elusive. Most studies of conscious vision have emphasized the important role of visual areas of the occipitotemporal lobe. Physiological evidence for this role comes from studies in awake monkeys demonstrating pattern-selective responses in temporal areas that reflect the subjective perception of visual objects and are modulated by attention (1–5). Consistent with these observations, lesions of the temporal lobe lead to profound impairments in visual recognition and awareness (6–8). Thus, it is commonly accepted that normal visual perception depends on neural processing in the ventral visual pathway. It is less clear, though, what the contribution of other cortical regions to conscious vision may be. Several lines of evidence suggest that inferior parietal and prefrontal cortical areas also contribute to conscious vision. Visual neglect and extinction, two related deficits in visual awareness, arise after injury to regions in the inferior parietal and frontal lobes (9, 10). Classic studies in monkeys have also shown that chronic blindness follows a massive cortical ablation that spares most of the modality specific visual cortex, but not parietal and frontal areas (11–12). However, because these areas play a prominent role in the organization of coherent behavior, it is an open matter as to whether they are required for subjective visual perception or whether they simply allow its behavioral expression. Furthermore, the nature of the interactions among areas involved in conscious perception has not been established.

One way to investigate the neural sites and mechanisms associated with conscious vision is to expose the visual system to viewing conditions that dissociate subjective perception from sensory input (13–15). Such dissociation arises, for instance, when dissimilar images are shown to the two eyes. Because the images cannot be fused by the cyclopean visual system, perception alternates spontaneously between each monocular view, a phenomenon called binocular rivalry. As the alternation occurs in the absence of any changes in the stimulus itself, variation in brain activity can be related directly to subjective perception. Recordings in awake monkeys have revealed that, whereas the firing of most neurons in primary visual cortex reflects the stimulus and not the percept during rivalry, activity at higher levels in the ventral visual pathway are better correlated with perception (15). In a recent imaging study in humans, neural correlates of subjective perception during rivalry also were found in extrastriate visual areas, and in parietal and prefrontal cortices (16). However, both monkey and human studies have relied on subjects’ reports to assess their perceptual state, and so the transient activity time-locked to such reports can be attributed in principle either to perceptual processes, to the generation of reports, or to both. To distinguish between these processes and gain further insight into the biological underpinnings of conscious vision, we now examine the neural responses evoked during binocular rivalry in the absence of any motor reports. We present evidence suggesting that internally generated covariation of activity among multiple ventral, dorsal, and prefrontal cortical areas is associated specifically with subjective visual perception. The results emphasize the importance for conscious vision of neuronal interactions between widely distributed cortical centers, including regions lying outside the visual cortex; they contrast with prevailing views on visual awareness that localize its neural basis to specific visual areas or pathways.

METHODS

We measured brain activity with functional MRI (fMRI) in six human volunteers (5 males, 1 female; age 30 ± 3.7 yr) who passively viewed pairs of stimuli through nonmetallic stereoscopic glasses. The stimuli were computer-generated and projected with a liquid crystal display projector onto a small screen mounted on the radio-frequency headcoil. A keypad was used before scanning began to adjust the lateral position of the two images so that each one was seen through only one eye and so that stereoscopic superposition of the images could be attained comfortably. All subsequent stimuli presented during the scanning runs were placed at those locations.

The scanning sessions involved two different viewing conditions. In the first condition, subjects were instructed to passively look at stimuli that consisted of a red-colored drifting grating shown to one eye and a green-colored face shown to the other eye, each subtending about 3.5° of visual angle (Fig. 1A). On subsequent debriefing, subjects confirmed that they experienced ongoing perceptual alternations between face and grating when exposed to these stimuli during scanning. In post-hoc sessions, eye-movement recordings also were obtained in the scanner (turned off) while subjects reported their percepts during rivalry. These data were used to verify that significant eye movements were rare and not related to perceptual alternations. In a second viewing condition, subjects were exposed to stimuli with similar characteristics as those used during rivalry but that did not cause perceptual alternations. These stimuli consisted of pairs of monocular figures, each with half a grating or face and an equiluminant gray patch, that produce a fused percept with half a (drifting) grating and face (Fig. 1B).

Figure 1.

Viewing conditions during scanning. (A) Subjects viewed dichoptic pairs of monocular stimuli consisting of a red grating drifting upward and a green face (Upper). This stimulation produced a bistable perception, alternating every few seconds between face and grating (Lower). Stimulus presentation lasted 41 s per repetition, with ocularity of face and grating exchanged between repetitions. (B) During stable viewing, subjects were presented for periods of 41 s with complementary monocular images that produced a fused percept consisting of half of a static face and drifting grating. In this figure, the stimulus contrasts are higher than in the experimental conditions for display purpose.

Functional MRI scans were obtained with a Siemens Vision (Siemens, Erlangen, Germany) operating at two T. Image volumes with Blood Oxygenation Level-Dependent (BOLD) contrast were acquired continuously every 4.1 s, each comprising 48 contiguous 3-mm-thick slices with an in-plane resolution of 3 mm × 3 mm. Functional imaging was organized in two scanning sessions per subject. In each session, the rivalry and stable viewing conditions were repeated 6 times, with each condition lasting 41 s (10 scans) per repetition. Data analysis was carried out by using Statistical Parametric Mapping software (SPM96; http://www.fil.ion.ucl.ac.uk/spm). The imaging time series was realigned, spatially normalized to the stereotactic space of Talairach & Tournoux, and smoothed with a Gaussian kernel of 8 mm full width half maximum (17–19). The fMRI signals were then adjusted to remove low-frequency drifts in the signal, global changes in activity, and the mean responses evoked in each viewing condition. Voxels where the residual activity during either viewing condition covaried with that in selected cortical loci were identified by means of multiple regression analysis. All statistical results were based on a single-voxel Z threshold of 3.09. Resultant regions of coactivation were characterized in term of their peak heights. In assessing their statistical significance, we made a correction for multiple comparisons across the whole-brain volume examined (20).

For comparison with the experimental measure of brain activity, we also constructed a model of the physiological responses to trains of perceptual transitions. This was done by obtaining the frequency histograms of dominance times for each percept during rivalry in post-hoc behavioral sessions. The psychophysical data were well approximated by a γ distribution (mean = 4.5 s; SD = 2.1 s), that was used to generate a simulated train of transition events. These events were then convolved with an empirically derived hemodynamic impulse response function (21), and the resulting signal was sampled at the scanning rate to produce a theoretical time series.

RESULTS

In contrast to earlier imaging studies of rivalry, subjects were not instructed to, nor did they make any, motor or verbal report to indicate perceptual transitions. Nevertheless, we anticipated that the subjects’ introspective experience of perceptual alternations during rivalry would be reflected in modulation of responses in cortical areas involved in conscious visual perception. Such modulation should cause systematic correlation of activity between fMRI measurements that are close in time (because of the slow time constant of hemodynamic responses). On the other hand, such modulation would not be expected in areas whose role is confined to the sensory analysis of the steady-state visual input. We therefore first examined the activity in two preselected cortical loci. One locus in early extrastriate cortex (at the junction of Brodmann areas 18 and 19, denoted BA 18/19) has previously been shown to reflect reported perceptual transitions during rivalry, whereas the other, in striate cortex (V1), does not (16). The selected region in BA 18/19 is adjacent to the lateral occipital sulcus and appears to lie laterally with respect to the human area visual cortical V3A identified in retinotopic mapping studies (22). Successive fMRI measurements at this locus were significantly correlated (P < 0.01). This correlation in time (autocorrelation) shows that there is temporal structure to the signal evoked from BA 18/19 consistent with selective involvement of this area in the subjective experience of perceptual alternations. Indeed, the autocorrelation was comparable to that predicted by a model of the physiological responses to trains of transition events with realistic temporal dynamics (Fig. 2; and see Methods). Similar time structure was found in other extrastriate visual areas, including posterior and anterior segments of the fusiform gyrus, previously associated with reported perceptual shifts during rivalry (16). In contrast to the responses in these extrastriate areas, activity in V1 failed to exhibit such temporal structure, with successive fMRI signals in V1 being uncorrelated. These findings provide further evidence that subjective perception during rivalry is associated with evoked activity in extrastriate visual areas, but not in striate cortex (14).

Figure 2.

Autocorrelation function of fMRI time series during rivalry. (A) Data from a voxel in the calcarine sulcus (V1; x = −12, y = −96, z = −3). (B) Data from a voxel in the middle occipital gyrus, close to the lateral occipital sulcus (BA 18/19; x = 36, y = −87, z = 9). (C) Synthetic data from a model of hemodynamic responses to perceptual alternations. Autocorrelations are shown for increasing lags (expressed in number of scans) with all time series mean-corrected and normalized to unit SD. Points above or below the dashed horizontal lines exceed the 99% confidence limit for significant effects. Note the absence of significant correlation in V1 for all but the trivial zero lag. In contrast, the time series from BA 18/19 and the model both show significant correlation between successive fMRI measurements (corresponding to a time lag of 4.1 s).

It seems, therefore, that some neural processes are linked to conscious perception during rivalry, whereas others (e.g., in V1) are not. What distinguishes these two neural processes? We conjectured that such differences may reflect differential interactions with other visual or nonvisual cortical areas. To investigate this possibility, temporal responses of other brain areas were analyzed by using multiple regression with regressors specified by the activity in BA 18/19 and in V1, respectively. This analysis allowed us to identify: (i) brain areas where activity was consistently correlated with that in BA 18/19, which served as a surrogate for perceptual reports and (ii) brain areas where activity was correlated with that in V1, which served as an indicator of correlated activity unrelated to conscious perception.

All subjects showed significant modulation of activity in multiple visual cortical areas that was correlated with the responses in the selected region of BA 18/19 (P < 0.01, corrected for multiple comparisons; Fig. 3A and Table 1). This pattern of coactivation extended in both hemispheres, dorsally in middle and inferior segments of the occipital and temporal gyri and ventrally in the fusiform gyrus. In particular, responses that correlated with activity in BA 18/19 were detected in regions of the fusiform gyrus previously implicated in the processing of faces (23) and in regions of the temporal gyrus selective for visual motion (24). The presence of correlated modulation of activity in these functionally specialized visual areas is therefore consistent with their involvement in the alternating perception of a face and moving stimulus experienced by the subjects. In addition to visual areas, several regions lying outside the classical definition of visual cortex showed significant correlation with activity in BA 18/19; they were located bilaterally in the superior and inferior parietal cortex (with a right hemispheric dominance) and in the right superior frontal cortex, middle frontal gyrus (BA 6/8), inferior frontal gyrus (BA 9/44), and insula (BA 45/47; P < 0.01, corrected; Table 1). The same pattern of coactivation was obtained in regression analyses based on activity in other cortical sites previously associated with reported perceptual changes during rivalry, in the fusiform and parietal lobes (Fig. 3B). These analyses further demonstrated the coactivation of more anterior regions in mid-frontal gyrus in concert with parietal or fusiform activity (BA 46, x = 48, y = 45, z = 12, Z score = 7.6; BA 10, x = 39, y = 54, z = 18, Z score = 6.2; P < 0.01, corrected). In a previous imaging study of rivalry in which subjects reported their percepts with key presses, we showed that activity in the same visual and nonvisual cortical areas is also correlated with reported perceptual transitions (16). Taken together, these results demonstrate that coordinated activity in a distributed network of extrastriate visual, parietal, and prefrontal areas is associated specifically with subjective perception, independent of motor reports. In sharp contrast with the extended pattern of coactivation associated with BA 18/19, only a few, scattered voxels in posterior cortices were correlated with the fMRI signals measured in V1 (Fig. 3C).

Figure 3.

Coactivation maps during rivalry. (A) Areas where activity was significantly correlated with that in BA 18/19 (P < 0.01, corrected) are shown as through-projection onto a lateral representation of standard stereotactic space (Upper Left). Regional activation in selected coronal slices is overlaid onto the average Talairach normalized anatomical MRI of the six subjects. Distance from the anterior commissure is indicated below each coronal section and, through color coding, on the Upper Left lateral representation. (B) Areas where activity was significantly correlated with that in posterior fusiform gyrus (Upper, x = 42, y = −69, z = −18) and superior parietal cortex (Lower, x = 39, y = −42, z = 45) are shown as through-projection onto a lateral stereotactic representation (P < 0.01, corrected). (C) Only a few distal voxels in posterior cortices show significant correlation with V1. Markers on the see-through representations indicate the loci of activity used as regressor in the covariation analyses.

Table 1.

Cortical regions where activity is significantly correlated with that in prestriate cortex (BA 18/19) during rivalry

Cortical region	Coordinates, x, y, z	Z score
Posterior fusiform gyrus	30, −81, −15	7.0
	−36, −87, −6	7.5
Anterior fusiform gyrus	42, −54, −21	7.3
	−33, −45, −18	5.3
Temporal gyrus	48, −63, 3	6.4
	−39, −69, 3	6.7
Occipital gyrus	27, −75, 27	7.7
	−21, −78, 27	7.2
Superior parietal lobe	33, −48, 60	6.9
	−18, −60, 54	6.1
Inferior parietal lobe	63, −24, 36	7.2
	−33, −39, 24	6.3
Superior frontral/precentral sulcus	33, 0, 48	5.6
Middle frontal gyrus	57, 6, 39	7.2
Inferior frontal gyrus	60, 12, 9	5.8
Insula	42, 18, −6	5.6

Coordinates of the maximally coactivated voxels (P < 0.01, corrected) are shown. 

These results raise the possibility that cooperative interactions between extrastriate visual and nonvisual areas are important for conscious visual perception. If so, the degree of correlation between their neural activities should be related to the frequency of salient perceptual events. To test this hypothesis, we compared the correlations with the selected voxel in BA 18/19 during rivalry (a viewing condition that elicits frequent perceptual changes) and a second viewing condition that elicits a stable percept. In the second condition, subjects were exposed to monocular stimuli that are perceived together as a figure composed of half a face and grating (Fig. 1B). This stimulus was designed to produce a percept with similar visual characteristics to the rival stimulus but that does not cause perceptual alternations. Brain regions where activity was significantly more correlated with responses in BA 18/19 during rivalry compared with the stable condition were restricted to the inferior parietal and inferior frontal regions identified in the previous analysis (P < 0.001, uncorrected; Fig. 4). Interestingly, no such differences were observed in extrastriate visual areas, suggesting that functional interactions between visual and nonvisual areas, rather than within the visual cortex proper, may underlie the conscious registration of perceptual events.

Figure 4.

Areas showing greater correlation with BA 18/19 during rivalry compared with a stable viewing condition. (A) Differential coactivation during rivalry was localized in the inferior parietal (x = 63, y = −24, z = 36, Z score = 4.7; x = −33, y = −39, z = 24, Z score = 4.4), inferior frontal (x = 57, y = 12, z = 6, Z score = 4.6), and insular cortex (x = 33, y = 18, z = −3, Z score = 3.8; x = −36, y = 18, z = 3, Z score = 5.0; P < 0.001, uncorrected). This activation is shown as through-projection onto a lateral representation of standard stereotactic space and on coronal and transverse MRI sections. See Fig. 2 legend for conventions. (B) “Visual–visual” interactions. Normalized cross-correlation between activity in BA 18/19 and in a posterior region of the fusiform gyrus (x = 42, y = −69, z = −18; BA 19) are shown during rivalry (red) and a stable viewing condition (blue). (C) “Visual–frontal” interactions. Normalized cross-correlation between activity in BA 18/19 and in inferior frontal cortex (x = 60, y = 12, z = 9; BA 44) are shown during rivalry (red) and stable viewing (blue).

DISCUSSION

These results demonstrate that primary and early extrastriate visual areas behave differently both in time and with respect to other cortical regions during the passive viewing of dichoptic stimuli that elicit binocular rivalry. Activity in early extrastriate cortex, but not in V1, exhibited a correlation in time (autocorrelation) that was almost identical to that predicted by a model of the physiological responses to trains of perceptual alternations. Moreover, whereas activity in V1 was uncorrelated to that of most other cortical sites, activity in early extrastriate cortex consistently covaried with responses in areas located both at higher levels in the dorsal and ventral visual pathways and in lateral frontal cortex, indicating systematic interactions among these areas. In an earlier study, we found that activity in the same extrastriate and nonvisual areas was also correlated with key-press reports of perceptual transitions experienced under identical viewing conditions (16). The importance of the present findings is to demonstrate that the involvement of these areas in rivalry is independent of motor reports. Taken together, these results provide strong evidence in normal human subjects that nonvisual cortical areas, acting in concert with functionally specialized visual centers, are associated specifically with conscious visual perception; they support the predictions of neural theories of visual awareness that depend on interactions between visual and prefrontal cortex (25–27).

An alternative interpretation of the observed pattern of coactivation is that it merely reflects specialized neural mechanisms of binocular rivalry involved in the alternating suppression of conflicting monocular views. But the range of functions associated with the parietal and prefrontal regions delineated in the present study suggests the involvement of more general mechanisms. Previous studies in human and nonhuman primates have implicated these parietal and prefrontal areas in visual tasks involving short-term memory, selective attention, and motor planning (28–32). In particular, the location of the middle and inferior prefrontal regions identified here is remarkably consistent with that of areas recently implicated in the maintenance of facial representations in working memory (33). The regional activation observed in superior frontal cortex, located at the junction of the precentral and superior frontal sulcus, is also closely related to that associated with mnemonic processing of faces (34) and with oculomotor control (35). A similar superior frontal region, together with superior and inferior parietal areas, has also been implicated in selective visual attention (32). Neuroanatomical and electrophysiological observations in monkeys further indicate that parietal and prefrontal structures are reciprocally connected and act in concert with secondary visual areas (36–41). Consistent with these earlier findings in monkeys, our results imply a network organization that allows the human prefrontal and parietal cortex to operate with extrastriate visual areas as an integrated system. We propose that reciprocal interactions within this network may serve to selectively integrate internal representations of visual events in the broader temporal and behavioral context in which they occur, a function that appears essential for the emergence of phenomenal awareness and the apparent continuity of perceptions (42).

ABBREVIATIONS

fMRI

functional MRI

BA

Brodmann’s area

V1

striate cortex