Cortical processing of a brightness illusion

Abstract

Several brightness illusions indicate that borders can affect the perception of surfaces dramatically. In the Cornsweet illusion, two equiluminant surfaces appear to be different in brightness because of the contrast border between them. Here, we report the existence of cells in monkey visual cortex that respond to such an “illusory” brightness. We find that luminance responsive cells are located in color-activated regions (cytochrome oxidase blobs and bridges) of primary visual cortex (V1), whereas Cornsweet responsive cells are found preferentially in the color-activated regions (thin stripes) of second visual area (V2). This colocalization of brightness and color processing within V1 and V2 suggests a segregation of contour and surface processing in early visual pathways and a hierarchy of brightness information processing from V1 to V2 in monkeys.

The perception of surface brightness is influenced not only by local surface luminance but also by luminance and border contrast cues in the surrounding scene. Influence of nonlocal cues on brightness perception are illustrated by stimuli, such as simultaneous contrast stimuli (1), Mondrians (2), and scenes with 3D perceptual interpretations (3). How the brain encodes such local and global brightness cues is unknown. Only a few studies have examined neuronal response to uniform surfaces (4-9). These studies have shown that although cells modulated by luminance change are found as early as the retina and lateral geniculate nucleus (LGN), those modulated by perceived brightness change, which occur independent of actual luminance change over the receptive field (RF), can be found as early as the primary visual cortex (V1). Little is known regarding the functional organization of brightness processing in the visual system (cf. ref. 10, in cat visual cortex).

Here, we examine the functional organization of brightness processing in the first two stages of Macaque monkey visual cortex, V1 and second visual area (V2). Monkeys have organization of the early visual pathways and perception of brightness similar to those of humans (11, 12). Many single-unit recording (13-20), 2-deoxyglucose (21-23), and optical imaging studies (19, 20, 24-29, §) have demonstrated functional organization for the processing of contours and color. Such functional organization does not suggest strict segregation because each functional structure contains a mixture of neurons with varying selectivities. However, as demonstrated by optical imaging, there are clear differences in the overall population response within each structure. In V1, imaging for color, monocularity, or low spatial frequency response reveals patterns of activation that correlate well with cytochrome oxidase blobs (25, 26, 29-31, §). Domains of color activations in V1 tend to be larger than cytochrome oxidase blobs, and they occasionally span two neighboring blobs via cytochrome oxidase bridges (30). Thus, imaging for color is useful for revealing locations of cytochrome oxidase blobs and their associated bridges; regions not activated by color correspond to the cytochrome oxidase light regions (interblobs). In V2, thin stripes are clearly revealed by their preferential response to color stimuli and lack of orientation structure, whereas thick and pale stripes are characterized by arrays of orientation domains. [In this article, we use the terms blobs and thin stripes to refer to color domains in V1 and V2, respectively. We use the terms interblobs and thick and pale stripes to refer to regions outside color domains in V1 and V2, respectively.] These findings have led to the hypothesis that visual surface features, such as color and brightness, are handled preferentially by the blobs and thin stripes. We tested this hypothesis by studying the functional organization of brightness response in V1 and V2 of the Macaque monkey visual cortex.

We also examined possible hierarchical differences between V1 and V2 in their representation of brightness information. Studies on visual contour processing have demonstrated that neurons in V1 are responsive to real luminance-defined contours, whereas neurons in V2 can recognize “higher-order” illusory contours (e.g., refs. 32-35). In a similar vein, in this study, we compared V1 and V2 response with real and illusory brightness modulations. Specifically, we studied responses to perceptually matched changes in real surface luminance (Fig. 1B) and illusory (Fig. 1 A) brightness change (human psychophysics, refs. 36-38; monkey psychophysics, ref. 12) (see Movies 1-3, which are published as supporting information on the PNAS web site).

Fig. 1.

Stimuli. Real (B) and Cornsweet (A) stimuli-induced the matched perceptions of brightness modulated in time. Luminance profiles are shown below each stimulus. (A) The Cornsweet stimulus. An illusory brightness contrast is induced by a true contrast difference at an intervening border (16-30% contrast peak to peak at border). (B) The Real stimulus. A perceptually matched brightness contrast (8-15% contrast peak to peak at border; cf. refs. 37 and 38). (C) The Blank stimulus was an unmodulated isoluminant gray field of the same mean luminance as the other stimuli. Mean luminance of all stimuli are equal. Left and Right in Cornsweet and Real stimuli counterphase in brightness (0.5 Hz sinusoidal modulation over time), as shown in Movies 1-3. Imaged fields of view (dashed rectangle) and recorded RFs (square) were distant (≥2-12 RF widths) from the edge of luminance-modulated regions.

Materials and Methods

Macaque monkeys were anesthetized (i.v., thiopental sodium, 1-2 mg/kg per h), paralyzed with vercuronium bromide (i.v., 100 μg/kg per h), and artificially ventilated. Anesthetic depth was assessed continuously by means of implanted-wire electroencephalogram electrodes, end-tidal CO₂, pulse oximetry, monitoring heart rate, and regular testing for response to toe pinch. Eyes were dilated (atropine sulfate), retracted with specula, and fitted with contact lenses to focus on a computer screen. Eyes were aligned by converging the RFs of a binocular V1 cell with a Risley prism over one eye. Alignment was checked before and after each recording. Craniotomy and durotomy were performed to expose visual areas V1 and V2. All surgical and experimental procedures conformed to the guidelines of the National Institutes of Health and were approved by the Yale University (New Haven, CT) and Vanderbilt University Animal Care and Use Committees.

Stimuli. Real- and illusory-brightness stimuli (7) were created by using a custom-made computer program and presented binocularly to the animal. Each stimulus was a rectangular field divided into two half fields of uniform brightness by a stationary linear-contrast border. Stimuli were counterphased such that increase in luminance of one surface was coupled with a decrease in luminance of the other. In the real luminance stimulus (“Real” condition), brightness contrast (either 8% or 15%) between the two halves was sinusoidally modulated in time (0.5 Hz, 16 frames per modulation cycle, sign reversing around a mean luminance of 32 cd/m²; i.e., contrast incremented and then decremented in 16 luminance steps) (Fig. 1B). Thus, overall luminance remained constant throughout the modulation period.

In the illusory-brightness stimulus (“Cornsweet” condition), only the immediate border contrast (either 16% or 30%) was modulated, but it produced a percept of distant surface-brightness modulation that was very similar to that of the Real stimulus (cf. ref. 38). The Cornsweet luminance profile decayed exponentially on either side of the border with a width (from peak to surface) of 1-2° of visual angle. As with the Real stimulus, the border contrast was modulated sinusoidally over time at 0.5 Hz (sign reversing around a mean luminance of 32 cd/m², 16 frames per modulation cycle) (Fig. 1 A). Blank-stimulus conditions comprised an even gray (32 cd/m²) luminance level.

Optical Imaging. An optical chamber was attached to the skull, filled with silicone oil, and sealed with a glass window. Images of cortical reflectance change (intrinsic hemodynamic signals) were acquired by using imager 2001 (Optical Imaging, Germantown, NY) and 630-nm illumination. Signal-to-noise ratio was enhanced by trial averaging (30-100 trials per stimulus condition) and synchronization of acquisition with heart beat and respiration. Typically, 5-20 blocks of five trials per block were collected. Difference maps were obtained for pairs of stimulus conditions by subtracting summed frames acquired within 3 sec of stimulus onset. Single-condition maps were obtained by summing frames over 3 sec and subtracting the blank-condition map.

Spike Analysis. Because the stimulus was modulated at 0.5 Hz, for each peristimulus time histogram (PSTH), we fitted sinusoids (the greater of the F1 or F2 components) by using a least-squares method and calculated a modulation index (MI; 0, flat PSTH; 1.0, full modulation) from the contrast ratio of response, defined as (maximum - minimum)/(maximum + minimum) of the fitted sinusoid. To determine the confidence level (CL) of measured response contrast ratios, we compared the experimentally recorded MI with a distribution of 1,000 MIs derived from artificial bootstrapped spike trains (39). Significant response was taken at a 95% CL. Of cells with a significant response, 86% had MIs of ≥0.20 and 100% had MIs of ≥0.15. Thus, responses with an MI <0.15 were considered absent or weak, responses with 0.15-0.20 were considered moderate, and responses with >0.20 were considered robust.

Results

In these experiments, the extent of the visual field represented within the imaged field of view (Fig. 2B) was first mapped electrophysiologically (e.g., extent of visual representation in V2 shown in Fig. 2 A). We determined the location of the V1-V2 border by imaging for ocular dominance (Fig. 2C). The location of blobs in V1 (data not shown) and thin stripes in V2 were determined by imaging for response to color stimuli (Fig. 2 C and E, black arrowheads), and thick and pale stripes were mapped by imaging for response to oriented gratings (Fig. 2D, white arrowheads) (19, 20, 24-29). We then imaged response to Real (Fig. 1B), Cornsweet (Fig. 1 A), and Blank (Fig. 1C) conditions, randomly interleaved and presented for 30-100 trials each. The border of the Real and Cornsweet stimuli was placed well outside of the represented visual field (≥2° distant from the edge of Cornsweet border). Thus, in both the Cornsweet and Blank conditions, no direct luminance modulation was experienced by neurons in the field of view. In the Real condition, all cells in the field of view experienced the same amount of luminance modulation.

Fig. 2.

Optical imaging of brightness response. (A) Electrophysiological determination of visual field. RFs (green rectangles) were plotted to determine the extent of visual field representation (some in V2 shown in B). Stimuli were positioned such that width of Cornsweet border (vertical lines) was at least 2° distant from visual field extent. (B) Blood-vessel (BV) map. Green dots indicate recording penetrations. (C) Single-condition color map (red/black/green/black squarewave grating, monocular stimulation) revealing right-eye ocular dominance columns in V1, color responsive thin stripes in V2 (black arrowheads above), and the approximate V1-V2 border (short line on the left). (D) Orientation map. Horizontal minus vertical grating reveals orientation map in V1 and two thick/pale stripe regions in V2 (white arrowheads). Note expected alternation of thin and thick/pale stripes. (Scale bars in B-D, 1 mm.) (E) Higher-magnification view of map shown in C (region in rectangle in B). Locations of thin and thick/pale stripes are indicated at the top. (F-H) Same field of view shown in E. Pixels with strongest activation (top 10%) are shown in overlays at right. The gray scale given in E applies only to E, and the gray scale given in H applies to F-H. (F) Single-condition activation map in response to Real luminance stimulation. The strongest activation is shown in thin stripes with weaker activation in thick/pale region. (G) Single-condition activation map in response to Cornsweet stimulation, revealing a similar pattern. (H) Blank-stimulus map. (Scale bars in E-H, 1 mm.) Each image is the sum of 50 trials. In all single-condition maps, darker pixels indicate a larger-magnitude reflectance change. C and D were spatially filtered by using a 6 × 6-pixel moving window low-pass filter; no spatial filter was used in F-H.

Of four examined cases, we observed no detectable patterned activation in V1 by either the Real or Cornsweet stimuli (data not shown). In contrast, clear activation of V2 was observed in each of these four monkeys. We found that, similar to activation by color stimulation (Fig. 2E), Real luminance modulation (Fig. 2F) produced preferential activation of the thin stripes in V2 (black arrowheads), with weaker activation of the intervening thick/pale stripe (white arrowhead). Surprisingly, Cornsweet stimulation produced a very similar pattern of activation (Fig. 2G). This activation occurred even though, with Cornsweet stimulation, cells in the imaged field of view experienced no direct luminance modulation. Thus, the regions activated by the Cornsweet stimulus were similar to that activated by Real luminance modulation. No such activation pattern was observed in the Blank condition (Fig. 2H), indicating that this activation is not a nonspecific, baseline reflectance pattern of the cortex. Because the Blank condition is precisely what the cortex experiences in the Cornsweet condition with respect to direct luminance modulation, any differences in activation between Cornsweet and Blank must be due indirectly to the presence of the Cornsweet border. Neither is this activation a pattern characteristic of general activity, as general activity (e.g., obtained by summing all luminance and color grating stimulus conditions) produces a pattern of activation similar to that of cytochrome oxidase staining (preferential activation of thin and thick stripes but not pale stripes) (23, 26). These data suggest that the neural representation of perceived brightness is found in the thin stripes of V2.

To examine this activation further, we targeted imaged regions in V1 and V2 with microelectrodes and recorded single units from superficial layers in response to Real and Cornsweet stimuli. The locations of the V1-V2 border (Fig. 3B); orientation domains in V1 and V2 (Fig. 3C); blobs in V1; and thin, pale, and thick stripes in V2 (Fig. 3D) were determined by optical imaging. Blobs and interblobs in V1 were recorded by targeting the centers of the color domains and the centers of noncolor regions in V1, respectively. RFs of single V1 and V2 cells were characterized and the stimulus contrast border placed sufficiently distant from the Cornsweet border so that no direct luminance modulation of the classical RF (CRF) occurred (2-12 CRF widths; from edge of Cornsweet border). Brightness contrast (either Real or Cornsweet) was modulated sinusoidally in time. The edges of the CRF were determined by careful mapping, and the focus and convergence of the eyes were checked frequently throughout each recording session.

Fig. 3.

Examples of single-unit responses. (A) Blood-vessel (BV) map. Colored dots in A, C, and D indicate electrode-penetration locations. (B) Ocular dominance map reveals V1-V2 border (arrow). (Scale bars in A-D, 1 mm.) (C) Color-coded orientation map. Locations of thin (black arrowhead) and thick/pale (white arrowhead) stripes are indicated at the top in C and D. (D) Single-condition color map (red/green isoluminant grating) reveals blobs in V1 and location of a thin stripe in V2. B-D were spatially filtered by using a 6 × 6-pixel moving window low-pass filter. Each image is the sum of 30 trials. (E) Unit recorded in V1 blob [penetration 1 (P1)]. Activity is well modulated by Real (MI, 0.20; CL, 99%) but modulated poorly by Cornsweet (MI, 0.05; CL, 46%). (Maximum scale, 20 spikes per s.) (F) Unit recorded in V2 pale stripe (P5). Activity is well modulated by Real (MI, 0.34, CL, 99%) but poorly by Cornsweet (MI, 0.02; CL, 11%). (Maximum scale, 8 spikes per s.) (G) Unit recorded in V2 thin stripe (P3). Activity is moderately modulated by Cornsweet (MI, 0.16; CL, 95%) but poorly by Real (MI, 0.11; CL, 48%). (Maximum scale, 5 spikes per s.) (H-J) Examples of units recorded from V2 thin stripes from other cases. All three exhibit Cornsweet responses greater than or equal to Real responses. (H) Maximum scales were 12 (Real) and 8 (Cornsweet) spikes per s, and CLs were 98% (Real) and 99% (Cornsweet). (I) Maximum scales were 9 (Real) and 12 (Cornsweet) spikes per s, and CLs were 79% (Real) and 99% (Cornsweet). (J) Maximum scales were 4 (Real) and 3 (Cornsweet) spikes per s, and CLs were 60% (Real) and 96% (Cornsweet). The wave form shown below J is one cycle of stimulus modulation and applies to E-J.

We found that some cells in Macaque V1 and V2 are modulated by the Real and Cornsweet stimuli. Fig. 3 E-G shows the Real (Upper) and Cornsweet (Lower) responses of three cells, one in a V1 blob (Fig. 3E), one in a V2 thin stripe (Fig. 3G), and one in a V2 pale stripe (Fig. 3F). Response modulation was quantified by using an MI (see Methods). The V2 pale stripe cell (Fig. 3F) and V1 blob cell (Fig. 3E) exhibit a robust response to the Real stimulus (pale stripe MI, 0.34; blob MI, 0.20) but exhibits no modulation to the Cornsweet stimulus (pale stripe MI, 0.02; blob MI, 0.05). In contrast, the V2 thin stripe cell (Fig. 3G) responds moderately to the Cornsweet stimulus (MI, 0.16) but poorly to Real (MI, 0.11) brightness modulation. Three other examples of cells recorded from V2 thin stripes are shown in Fig. 3 H-J; each exhibits significant and comparable response to both Real and Cornsweet stimuli. In summary, we found cells in V2 that respond to the Cornsweet stimulus even though no actual luminance modulation occurs over the classical RF. Further, we found cells in monkey visual cortex that respond to both luminance modulation (Real) and to a stimulus eliciting illusory-brightness modulation in human observers (Cornsweet).

We recorded a total of 89 cells: 10 cells from the centers of V1 blobs, 13 cells from centers outside V1 color domains (interblobs), 31 cells in V2 thin stripes, 24 cells in thick stripes, and 11 cells in pale stripes (Fig. 4). To determine significance of sinusoidal modulation, a statistical bootstrap method was used (39). For each unit, significance was determined at a 95% CL. Of the cells in V1, 100% (10 of 10) of the cells in the blobs exhibited significant responses to the Real stimulus, whereas only 8% (1 of 13) of cells in interblobs achieved significance. Thus, in V1, response to Real luminance modulation was found preferentially in the color domains (i.e., cytochrome oxidase blobs and bridges); no response to the Cornsweet stimulus was found in V1 (none of 23 V1 cells showed a significant response to Cornsweet).

Fig. 4.

Response strength of all cells. (A) There were 89 units recorded in blobs (n = 10) and interblobs (n = 13) in V1 and in thin (n = 31), thick (n = 24), and pale (n = 11) stripes in V2. Each point represents response of each cell to Real (green diamonds) and Cornsweet (red diamonds) stimuli. To better reflect population response, points are ordered from low to high for each stimulus, and they do not correspond to the same cells. Response strength was measured by bootstrapped CL. Dashed line indicates 95% significance level. (B) Relative strength of responses (MI) to Real and Cornsweet stimuli. On average, responses to Real are stronger than to Cornsweet for cells recorded in blobs (black), interblobs (blue), thin stripes (red), thick stripes (purple), and pale stripes (green) (paired t test, P < 0.0001). For cells recorded in the thin stripes (red), MIs for Cornsweet response are greater on average than for other compartments (t test, P < 0.003).

Because this result suggested localization of luminance responsive cells in V1 color domains, we conducted three more imaging experiments to examine luminance response in V1 by using less subtle luminance-modulated stimuli. These stimuli included squarewave-modulated (instead of sinusoidally modulated) Real stimuli, stimuli with double the modulation contrast, full-field on/off stimuli (cf. ref. 10), large (10°) square and circular patches of luminance-modulated stimuli (cf. ref. 8), and Cornsweet gratings, which were designed to isolate either on or off response (cf. ref. 38). Despite these attempts, we did not obtain any activation consistent with maps of color domains nor any other structured pattern in V1. Neither do low-contrast (10% and 20%) luminance gratings produce any pattern of activation in V1.^§ We also imaged at higher magnifications (3-mm fields of view), suggesting the failure to observe activation is not an issue of image resolution. In contrast, color domains in V1 (blobs/bridges) are readily imaged by using low spatial frequency, isoluminant color gratings (ref. 26, §). Given that we did not obtain patterned maps in V1 in response to noncolor stimuli, these results suggest the importance of color content for strong imageable activation of V1 blobs.

Of our V2 sample, significant responses to the Real stimulus were found in all three stripe types, although most were found in the color (thin) (39%, 12/31) and pale (55%, 6/11) stripes. Interestingly, the Cornsweet stimulus produced significant activations almost exclusively in the thin stripes of V2 (23%, 7 of 31 in thin stripes; 4%, 1 of 24 in thick stripes). Thus, at least in V2, our sample population, although small, is consistent with our V2 imaging data.

Also, we examined the relative strength of responses to Real and Cornsweet stimuli. In general, responses to Real are stronger than responses to Cornsweet. As shown in Fig. 4B, for each of the blob (black), interblob (blue), thin (red), thick (purple), and pale (green) stripe populations, the MIs for response to Real luminance modulation was stronger, on average, than the response to the Cornsweet brightness modulation (paired t test: all cells, P < 0.0001; blob, P < 0.001; interblob, P < 0.001; thin, P < 0.02; thick, P < 0.05; pale, P < 0.001). Note that for cells recorded in the thin stripes (red), MIs for Cornsweet response are greater on average than for other compartments (t test, P < 0.003), indicating that response to Cornsweet is stronger in the thin stripes than in other functional compartments.

Discussion

This article reports the existence of cells responsive to brightness percepts in Macaque monkey V2. Most of the neurons responsive to Cornsweet-induced brightness modulation in V2 are also responsive to Real luminance modulation, suggesting the presence of generalized brightness response in V2. Both our electrophysiological and imaging results indicate these neurons are localized to V2 thin stripes. This colocalization of color and brightness processing suggests a preferential role for V2 thin stripes in the encoding of visual surfaces and further support at least some degree of segregation between contour and surface processing in V2. This segregation does not preclude significant interactions in processing between surface and contour processing.

We did not find cells responsive to Cornsweet in V1. Although previous studies have described luminance and brightness processing neurons in V1, the stimuli were inherently different. Some neurons in V1 respond to brightness induction (e.g., in cats, ref. 5) and encode either surface luminance or surface contrast (e.g., in monkeys, ref. 8). Such simultaneous contrast phenomena could in principle be represented primarily by surface processing pathways with limited involvement of the contour system. However, the Cornsweet stimulus is a brightness percept that is induced purely by border contrast without accompanying surface luminance contrast. This illusion suggests that neurons responsive to the Cornsweet stimulus must directly or indirectly receive inputs from oriented cells at the contrast border (cf. ref. 20, ¶). Anatomical and physiological evidence in primate V1 indicate the presence of two fairly independent horizontal networks: an interblob network, which processes primarily contour information, and a blob network, which processes primarily surface information, such as color (13, 24, 40, 41). We propose that (i) the extensive horizontal blob network in V1 makes it a plausible mediator of simultaneous contrast effects and (ii) V1 is not a likely locus for prominent interactions between contour and surface processing elements, and therefore, it is an unlikely place to find Cornsweet responses. Responses to the Cornsweet are more likely to occur in V2, where anatomical connections between different stripe types are commonly found (16, 24).

There is a discrepancy between our electrophysiological and imaging results in V1. Our electrophysiological evidence clearly suggests the presence of luminance-responsive cells in the blobs. However, despite additional attempts with various luminance stimuli containing different temporal, spatial, and contrast characteristics, we were unable to obtain evidence of blob activation with our imaging methods. This result contrasts sharply with our routine imaging of blobs with color-grating stimuli. One possible explanation is that uniform surfaces are simply a much weaker stimulus than color gratings and, coupled with the small size of V1 blobs, may be more difficult to detect than activation of the larger stripe structures in V2. Alternatively, intrinsic signal imaging of V1 blobs may require some degree of color content for robust activation.

Despite this unresolved discrepancy, our findings generally support previous studies, which associated V1 blobs and V2 thin stripes with the processing of surface properties. Blobs are reported to contain many color-selective cells that prefer low spatial frequencies and exhibit high contrast sensitivity (refs. 13, 17, 21, 22, 42, and 43; cf. ref. 44). V2 thin stripes, which are the primary recipients of V1 blob input (13, 45), contain cells and modules responsive to chromatic modulation (14, 19, 26, 27) and to luminance increment or decrement.∥ Our findings are also consistent with reports that cells with low-pass spatial frequency characteristics are located in thin stripes (16) and that thin stripes are responsive to spatially diffuse color variations (23). Furthermore, consistent with blob-derived input, imaging studies have shown that thin stripes have higher contrast gain than either thick/pale stripes,^§ which further supports the role of thin stripes in detecting brightness change. Thus, although single neurons are multidimensional (e.g., have some degree of responsiveness to both contour orientation and brightness), these findings suggest some segregation of processing with respect to brightness vs. contour response.

Also, these data suggest differential roles of V1 and V2 in the processing of brightness. Although responses of neurons in V1 blobs are modulated by full-field luminance modulation in the inducing field (Real stimulus), thus confirming their role in brightness constancy, they are activated only weakly by the Cornsweet stimulus, indicating little participation in the processing of illusory brightness (e.g. in cats, refs. 5-7). In contrast, V2 thin stripes exhibit response to perceptual brightness modulation, whether it is induced by true luminance change or by illusory brightness stimuli. For this reason, we consider these V2 cells higher-order perceptual brightness cells. These data suggest that V1 responses reflect the luminance properties conferred by local inputs, whereas V2 confers higher-order properties that result from integration of nonlocal inputs. Such a distinction finds parallel with the roles of V1 and V2 in real vs. illusory contour processing (29, 34) and underscores the fact that under certain contexts global cues (distant border contrast) can override local cues (lack of local luminance modulation).