Pattern of neuronal activity associated with conscious and unconscious processing of visual signals

Abstract

Following striate cortex damage in monkeys and humans there can be residual function mediated by parallel visual pathways. In humans this can sometimes be associated with a “feeling” that something has happened, especially with rapid movement or abrupt onset. For less transient events, discriminative performance may still be well above chance even when the subject reports no conscious awareness of the stimulus. In a previous study we examined parameters that yield good residual visual performance in the “blind” hemifield of a subject with unilateral damage to the primary visual cortex. With appropriate parameters we demonstrated good discriminative performance, both with and without conscious awareness of a visual event. These observations raise the possibility of imaging the brain activity generated in the “aware” and the “unaware” modes, with matched levels of discrimination performance, and hence of revealing patterns of brain activation associated with visual awareness. The intact hemifield also allows a comparison with normal vision. Here we report the results of a functional magnetic resonance imaging study on the same subject carried out under aware and unaware stimulus conditions. The results point to a shift in the pattern of activity from neocortex in the aware mode, to subcortical structures in the unaware mode. In the aware mode prestriate and dorsolateral prefrontal cortices (area 46) are active. In the unaware mode the superior colliculus is active, together with medial and orbital prefrontal cortical sites.

In a previous study of a “blindsight” patient, we focused on the conditions that yield good visual discrimination in the presence or absence of an accompanying experience (1). GY is a 41-year-old subject who has been investigated extensively in several studies (2–5). His left visual cortex was damaged at 8 years of age in a head injury, and as a result, his right field remains clinically blind, with the exception of a small zone (<3°) of macular sparing. Conventional structural magnetic resonance imaging reveals no intact striate cortex (V1) except for tissue at the occipital pole that would account for his macular sparing (6). When a rapidly moving target is projected into his blind hemifield, well outside the spared macular region, GY sometimes reports an “awareness,” a “knowing,” or a “feeling” that something has moved, although he denies any experience of “seeing” as such. Outside that range, he may still discriminate direction of movement very well by forced-choice guessing, but without any conscious awareness of the event. The parameters determining these two modes were studied using a “commentary key” paradigm in conjunction with the discriminative response on every trial.

The presence of both the “aware” and “unaware” modes of discrimination offers the possibility of functional brain imaging of each mode to determine the brain systems active in each, and hence of revealing distinctive brain activity associated with visual awareness. The primate retina projects not only to V1 (via a relay in the lateral geniculate nucleus), but also over parallel pathways to a variety of other targets located subcortically, and which in turn project to other cortical regions that may remain intact after a V1 lesion (7). Thus, a number of differential patterns of activity are possible. As the visual defect involves only the half-field contralateral to the lesion, the intact hemifield also provides a useful comparison.

MATERIALS AND METHODS

Because of the physical constraints of the magnetic resonance imaging (MRI) environment a portable and compact apparatus was constructed to simulate the laboratory conditions that yield well above chance aware vs. unaware discrimination. A visual stimulus could be projected onto a visible screen with minimum stray light, and the direction, speed, and contrast of the stimulus varied over the range that was found previously to be effective (1). The requirement was to replicate the original pattern of findings using the portable apparatus for subsequent use in the functional MRI (fMRI) study.

The experimental set up for the psychophysical measurements was as follows. The subject observed a wide screen (33.3° × 20.3°) from a viewing distance of 1.3 m. The screen consisted of a 0.25-mm styrene sheet that was found to have a near Lambertian transmittance function. Measurements of the stimulus luminance profile at various viewing angles showed less than 4% variation in target luminance over the visual area of interest.

The stimulus target was derived from a collimated semiconductor laser source (ACMT08/2092, 15 mW, 635 nm; Power Technology, Little Rock, AR) and the movement parameters were controlled by a servo-drive mirror scanner system (model DMC 1510; Galil Motion Control, Sunnyvale, CA).

For blind-field measurements, the subject observed a fixation point 3.5° above and 5.7° to the right of the bottom left side of the screen. The midpoint of the moving dot target was always at a point 15° to the right and 5° above the horizontal meridian. Stimulus orientation was always horizontal, moving either away or toward the vertical meridian. The luminance profile of the moving target approximated a Gaussian distribution subtending 40′ over ± 2 SD. Measurements reported here were carried out using a stimulus target of 107 cd/m² added to that of the background. The blind-field background luminances for the low, medium, and high contrast conditions tested were 85, 25.7, and 4.3 cd/m², respectively. The background luminances were achieved using two Kodak Carousel S-3AV and one Simda model 2200, 400 W (Chester, U.K.) projectors, each fitted with neutral density filters. When testing the blind field, the sighted field luminance extended 3° in the blind field.

As in our previous investigations, the method we employed to study the motion processing was based on a forced response commentary key paradigm. After each stimulus presentation, the subject (GY) was prompted by an audio beep to report, if necessary by guessing, the direction of motion of the stimulus target. In addition, after each trial, he was asked to signal on response keys whether he was aware of the stimulus during the trial. He was instructed to signal unaware when he had no sensation, feeling, or experience of the visual event. Experiments were carried out in blocks of 50 trials, and replications were given in sequences to counter order effects. A source of white noise was used to mask any possible audio cues arising from the mechanical components of the motion system. The subject was required to maintain strict fixation during the short time of the stimulus presentation. Results from two normal observers showed chance performance when either the laser path was blocked or the region of their visual field where stimuli were presented was occluded, indicating that no audio cues or possible scattered light could be used to carry out the required discrimination task.

GY’s discrimination performance and his reported awareness under three contrast conditions for two stimulus speeds are shown in Fig. 1. These results confirm the previously reported findings of well above chance performance, both with and without awareness.

Figure 1.

Reported awareness and correct discrimination of direction of horizontal movement, away or toward the vertical meridian for two speeds as a function of background luminance. Measurements were obtained using a forced response paradigm. At slow speed, subject had a little or no awareness of the stimulus presentations. The discrimination scores, however, remain well above chance for all conditions shown.

fMRI Studies.

The screen used in the psychophysical tests was mounted at the end of the scanner bore. The subject was able to view the screen through a mirror, mounted at an angle of 45° directly above the head rest. Projectors were placed some 5 m away, adjacent to the scanner room, and illuminated the screen through the observation window. The neutral density filters were adjusted to obtain the same luminance levels in the blind field as those employed in the psychophysical tests. The luminance levels on the side contralateral to that being tested, however, was kept at 80 cd/m².

Each fMRI experiment lasted 5 min and consisted of an alternating series of 30 s of no-motion followed by 30 s of stimulus motion presentation. To obtain a base-line for the measurements, each trial started with a no-motion interval. The stimulus traversed 14° of visual angle, horizontally in a random direction. The motion trajectory was 4° above fixation and terminated 3° away from the vertical meridian. To reduce the possible cortical activation due to verbal or motor response, the subject performed the task in a covert manner during the imaging procedure. Instead, the subject had to report whether he was aware of any stimulus presentation at the end of each 5-min fMRI experiment.

At the end of all blind-field presentations at slow stimulus speed, he reported no awareness of any stimulus presentation. For fast stimulus speed conditions, on the other hand, he reported that he was aware of stimulus movement during every presentation. Experimental conditions for sighted-field stimulation were mirror symmetric with respect to the center of the screen. As expected, the subject was fully aware of all stimulus presentations in the sighted hemifield.

Image Acquisition and Processing.

During each 5-min fMRI experiment, gradient-echo echoplanar magnetic resonance images were acquired using a 1.5 tesla GE Signa System (General Electric) retrofitted with Advanced NMR hardware (ANMR, Woburn MA) at the Maudsley Hospital, London. A quadrature birdcage head coil was used for radiofrequency transmission and reception. In each of 10 contiguous planes parallel to the calcarine fissure, 100 T^*₂-weighted magnetic resonance images depicting blood oxygenation level-dependent (BOLD) contrast (8) were acquired with TE = 40 ms, TR = 3 s, in-plane resolution = 3.1 mm, slice thickness = 5 mm and a 0.5 mm slice gap. Head movement was limited by foam padding within the head coil and a restraining band across the forehead. At the same session, a 43 slice, high-resolution echoplanar image of the whole brain was acquired along the intercommissural [anterior commissure–posterior commissure (AC-PC)] plane with TE = 80 ms, TI = 180 ms, TR = 16 s, in-plane resolution = 1.6 mm, slice thickness = 3 mm, and a 0.3-mm slice gap.

Small movements of the head during functional magnetic resonance image acquisition can cause changes in T₂^*-weighted signal intensity that are unrelated to alterations in cerebral blood flow. Prior to analysis for responses to visual stimulation, images were corrected for the effects of head motion in three dimensions (9). The power of the periodic signal change at the (fundamental) on–off frequency of visual stimulation (fundamental power, FP) was then estimated by iterative least squares fitting of a sinusoidal regression model to the motion-corrected time series at each voxel (10). The FP, divided by its standard error yields the fundamental power quotient or FPQ, a standardized measure of the size of the experimental effect. The significance of voxel-wise FPQs was then estimated against a null distribution constructed by re-estimation of FPQ 10 times at each voxel following random permutation of the time series (10). The data were then transformed into the standard stereotactic space of Talairach and Tournoux (11) by an automated procedure which computed the affine transform required to register each fMRI data set to a standard template. The voxel-wise significance of activation was then computed using the similarly transformed and randomized FPQ data (9). Activation maps were then produced on transverse slices derived by transforming the high-resolution echo-planar imaging (EPI) data from the subject in the study into standard space.

A conservative statistical threshold was applied to the results, based on minimal values of two nonparametric statistical indices, total FPQ and maximum FPQ (10), as well as a minimal cluster size of areas of activation. The minimal values were as follows: total FPQ of 10.0; maximum FPQ of 2.0; cluster size of 4. On conservative grounds the associated p value would be less than 0.001.

RESULTS

A summary of results, together with the Talairach coordinates, are presented in Table 1.

Table 1.

Major activation foci for aware and unaware modes and the sighted-field stimulation

Cluster size	Maximum FPQ	Total FPQ	Talairach coordinates			Side	Brodmann area	Cerebral region
			x	y	z
Aware mode
16	3.9	47.3	6	−69	−2	R	18	Prestriate
15	4	44.6	46	6	−13	R	38	Anterior temp. pole
13	6.1	48.7	40	31	20	R	46	Dorsolt. prefrontal
11	3.6	29.7	3	−69	31	R	7	Precuneus
10	3.5	29.3	−9	−42	−2	L	30	Retrosplenial
9	3.4	24.5	29	8	−18	R	28	Parahippocamp.
8	2.7	19.8	−49	3	−13	L	38	Anterior temp. pole
7	3.1	18	17	−86	15	R	18	Prestriate
7	3.2	18.4	−23	−75	−2	L	18	Prestriate
6	3.7	16.9	−20	−61	15	L	31	Post cingulate
6	2.4	13.9	55	6	26	R	6	Premotor
5	3.4	15.3	55	−33	−7	R	21	Mid. temp. G.
5	3	13.1	−35	36	−2	L	47	Ventrolat. prefrontal
5	2.7	12.7	−40	−69	15	L	39	Angular gyrus
5	3.4	13.1	14	19	15	R	—	Corpus striatum
5	3.4	14.2	−9	−83	26	L	18	Prestriate
5	3	13	49	14	−7	R	47	Vent. prefrontal
5	2.7	12.6	−9	−42	−7	L	—	Cerebellum
5	2.8	12.6	−6	−50	−7	L	—	Cerebullum
5	3.3	13.8	46	11	26	R	44	Inferior frontal G.
4	2.7	10.1	−20	−31	−18	L	—	Cerebullum
4	2.9	10	58	19	4	R	45	Inferior frontal G.
4	3	10.5	−14	−8	−18	L	34	Uncus
4	3.2	10.5	−46	−61	−7	L	37	Inf. post. temp. lobe
4	2.9	10.4	−9	−72	37	L	7	Precuneus
4	2.7	10.1	−3	−58	9	L	23	Post. cingulate G.
Unaware mode
33	4.8	98.2	9	−14	−18	R	34	Uncus
13	2.8	32.7	20	−19	−13	R	35	Retrosplenial
11	3.4	28.9	−35	−22	−2	L	—	Insula
11	3.3	29.8	3	−14	−2	R	—	Midbrain
10	2.9	25.8	3	−31	−7	R	—	Superior colliculus
10	3	25.9	−32	6	−18	L	28	Parahipp. gyrus
8	3.4	22.7	0	0	4	R	—	Thalamus
8	2.8	20.6	55	28	9	R	45	Inferior frontal G.
8	3.5	22.8	17	11	−2	R	—	Corpus striatum
8	2.8	21.3	−23	8	−7	L	—	Corpus striatum
8	2.8	20.1	−14	−44	31	L	31	Post cingulate G.
7	3	18.1	23	3	−13	R	34	Uncus
7	2.7	17.2	0	−36	−13	R	—	Brain stem
7	3.1	18.5	0	36	−2	R	24	Ant. med. cingulate
6	3.1	16	9	6	15	R	—	Corpus striatum
6	2.7	15.2	46	−25	−2	R	21	Mid. temporal G.
6	3.2	16.8	23	−36	31	R	31	Post cingulate G.
6	3.2	15.9	−12	−56	20	L	23	Post cingulate G.
5	2.8	12.6	−43	−6	20	L	4	Precentral G.
5	3.3	13.5	−9	6	26	L	24	Ant. med. cingulate
5	2.8	12.6	29	44	−13	R	11	Orbitofrontal cortex
5	3	13.9	−9	17	4	L	—	Corpus striatum
4	2.8	10.3	−38	−39	4	L	—	Hippocampus
4	2.8	10.2	52	−31	9	R	22	Superior temp. G.
4	3.3	11	23	33	−7	R	—	Medial frontal
4	2.6	10	−29	−8	4	L	—	Putamen
4	2.9	10.4	−12	3	20	L	—	Corpus striatum
4	2.9	10.4	−14	−33	20	L	—	Corpus striatum
4	2.7	10	58	0	−13	R	21	Medial temporal G.
4	3	10.3	−38	−61	31	L	19	Prestriate
Sighted-field presentations
14	3.3	37.8	58	−3	4	R	22	Superior temp. G.
12	3.3	30.3	14	25	26	R	32	Medial frontal lobe
12	3.9	35.1	−3	−6	15	L	—	Thalamus
10	2.7	24.7	−43	−69	9	L	19	Prestriate
9	3.6	25	35	−17	−18	R	36	Occip. temp. G.
x	y	z
8	2.8	20.3	58	0	31	R	4	Precentral G.
7	2.7	17.4	35	31	9	R	—	Insula
7	3.1	18.4	−26	−56	4	L	18	Prestriate
7	3.3	19.1	9	−47	26	R	23	Post. cingulate G.
7	2.8	17.8	29	31	26	R	46	Dorsolat. prefrontal
6	3.5	17.3	−6	−89	4	L	18	Prestriate
6	2.9	15	17	−11	26	R	—	Corpus striatum
6	3.4	16.2	38	36	37	R	9	Dorsolat. prefrontal
5	3.1	13.4	38	−39	−7	R	—	Hippocampus
5	2.4	11.9	55	−44	−7	R	21	Mid. temp. G.
5	3	12.9	52	−56	4	R	37	Inf. post. temp. lobe
5	2.9	12.5	14	−56	−7	R	—	Cerebellum
5	3	12.7	−26	19	20	L	—	Corpus striatum
5	2.9	12.8	26	28	−7	R	—	Corpus striatum
5	2.8	13	−29	47	15	L	10	Frontal pole
4	3	10.6	12	11	15	R	—	Corpus striatum
4	3.1	10.7	17	36	−2	R	24	Ant. med. cingulate
4	3.1	10.9	17	22	−2	R	—	Corpus striatum
4	3	10.9	−20	−64	9	L	31	Post. cingulate G.
4	3.2	11.3	20	−6	4	R	—	putamen
4	3	10.4	−20	−25	31	L	23	Post. cingulate G.
4	2.8	10.5	9	17	−13	R	25	Ant. cingulate G.
4	2.6	10.1	43	−47	15	R	22	Superior temp. G.
4	2.7	10.1	12	42	37	R	9	Dorsolat. prefrontal

FPQ, fundamental power quotient (see text); G., gyrus. 

The results can be summarized as follows for the salient features of the aware vs. unaware modes in the affected hemifield, irrespective of contrast level, as well as for the sighted hemifield. There are too few runs to allow a comparison between levels of contrast, per se. In the “aware” mode, dorsolateral prefrontal activity was observed in areas 46 (right hemisphere) (see Fig. 2 A and B), as well as area 47 (both hemispheres). There was also activity in area 18 in both hemispheres. Visual stimulation of the sighted hemifield produced stimulus dependent increases in neuronal activity in prefrontal areas 46 (right) and area 9 (right) and left frontal pole (result not shown in Table 1). In the unaware mode, and only in this mode, there was activation of the superior colliculus (Fig. 2 C and D). There was activation of the right medial and orbital frontal areas, but none in the dorsolateral region. There were two other features of note. In the unaware mode increased activity was also observed in left occipital lobe (area 19, ipsilateral to the lesion), but this was clearly not sufficient to generate phenomenal visual awareness. In addition, in the aware mode in the blind field, as mentioned above, activity was observed in area 18 contralateral to the lesion. It would appear that the intact hemisphere may have been recruited, possibly via callosal connections.

Figure 2.

(A) Sagital view showing the orientation of the images [anterior commissure–posterior commissure (AC-PC) base line] and the position of the 10 slices. In all images the left hemisphere is shown on the right-hand side of the image. (B) Slice taken at 20 mm above the AC-PC line (third slice from the top in A), showing the pattern of increased neuronal activity under the aware condition. Regions of significant increased activation are shown in black and include the right dorsolateral prefrontal area (Brodmann area 46). (C and D) The pattern of activity in two slices taken at z = −7 and z = −2 (third and fourth slice from the bottom in A), respectively, showing activity in midbrain centers, including superior colliculus in the unaware mode (see text for details).

The frontal eye field (FEF), area 8, was not activated in any condition, but the area immediately anterior to it (area 9, right hemisphere) was activated, but only during stimulation of the sighted field. Given the ambiguity of the anterior border of the FEF (12, 13) and the errors that can intrude in plotting onto an “average” brain map, area 9 may well implicate the FEF, which is also interconnected with area 46. If so, FEF would serve as a candidate for distinguishing between “awareness without seeing” (i.e., in the affected field) and normal phenomenal seeing within the intact visual system. Other possible candidates, the insula and the cingulate gyrus, did not gain support from this study design. The insula was active in both the sighted hemifield as well as in the unaware mode (albeit in opposite hemispheres), although it showed especially strong activation in the unaware mode. The cingulate was active in all modes, although there may be potential finer differences along its anterior–posterior axis.

The major outcome of the imaging results is not so much an isolated “center” for visual awareness—although area 46 is certainly of interest in this regard—but a shift in the pattern of activity between the aware mode (both in the affected and intact hemifields) and the unaware mode, from dorsolateral to medial prefrontal cortex, and from cortex to subcortical areas.

DISCUSSION

Even when using a conservative activation threshold, a large number of activated brain areas were detected by the studies reported here. The use of a less conservative, but still highly significant, activation threshold would increase the number of areas detected.

Within the pattern of results, certain findings have particular interest. First, the involvement of dorsolateral prefrontal in the aware state, and its absence in the unaware state (with good discrimination), raises the question of the importance for awareness of areas outside the classical cluster of visual cortices. The fact that prestriate cortex is activated even in the unaware condition implies that, while this might be necessary for visual processing, it is not sufficient for visual awareness. The only anatomical areas that appear to be involved differentially in visual awareness are well removed from visual association cortices. This would be consistent with a position, for example, that visual consciousness critically involves frontal sites of activity (14). It would appear that prefrontal areas may well have some special significance for visual awareness given that areas 46 and 47 were activated in both the aware mode of the blind hemifield as well as in the intact sighted field. It is of some interest that area 46 lies at an anterior point of rich convergence of many of the pathways, both ventral and dorsal, emanating from visual cortices (15).

Against such a position is the rarity of reports of frontal lobe lesions affecting visual awareness as such. However, in the monkey unilateral FEF lesions produce visual field defects that are closely similar, at least temporarily, to those of V1 itself (from which there can also be some recovery) (16). In humans it may be necessary for lesions to be both large and bilateral in the dorsolateral region for a significant effect. An activation found in an imaging study may reveal only the tip of a very large iceberg; the effects of lesioning just the tip may be relatively slight.

The possibility that posterior visual areas may be necessary but not sufficient for visual awareness is also consistent with evidence that removal or inactivation of all nonvisual cortex, leaving visual cortices intact, eliminates visual behavior (17–19). The animals behave as though they do not see, even though neural responses to visual stimuli can still be recorded in VI (20).

It is of interest that prestriate cortex was activated in both hemispheres during stimulation of the affected hemifield in the aware mode. Cross-field interactions may well be of some significance for residual visual function, as observed in visual completion phenomena across the midline when the intact as well as the affected hemifields are stimulated (ref. 21; for a review, see ref. 22). Recently, Ruddock and colleagues (23) have reported that particular arrangements of moving visual stimuli in the intact hemifield can produce an experience of phenomenal “seeing” in GY’s “blind” hemifield.

The emergence of the superior colliculus activation during the unaware mode is noteworthy. Studies with V1 lesions in the monkey clearly demonstrate its role in mediating recovery of visual function both behaviorally (24) and electrophysiologically (25). It appears especially to be pressed into action, as a parallel pathway, when the geniculo-striate pathway is damaged or blocked. Recovery of function in both human and animal studies point to the role of sustained practice in discriminating visual stimuli in the cortically blind field (26–28) and it is obvious that GY has had a considerable amount of discriminative experience associated with his affected hemifield, especially over the past year or so when he has been tested in several laboratories both in Europe and America, and his visual sensitivity is gradually improving.

In summary, although activation of certain brain regions was noteworthy, our primary observation was of change in the overall pattern of activation between the aware and unaware modes, shifting respectively from cortical to subcortical, and from dorsolateral prefrontal to medial and orbital prefrontal cortex. However, the results must be considered provisional, given the limited number of observations possible in this single subject. Further studies are needed.

ABBREVIATIONS

fMRI

functional magnetic resonance imaging

FPQ

fundamental power quotient

FEF

frontal eye field