How does the cortex construct color?

When is a color not a color? When it’s a wavelength. This may surprise those who were taught that color is carried by the dominant wavelength of the light reflected from an object. And you’d be confused in good company; Isaac Newton, for example, believed that “every body reflects its own color more copiously than the rest.” The colors we perceive, however, are determined not only by the wavelengths reflected from one surface but also by those reflected from many surrounding surfaces. This is illustrated in Fig. 1. The colored pattern is illuminated by different intensities of long, medium, and short wavelength light and different amounts of each region of the spectrum are reflected from each patch. The central green patch (Fig. 1a) reflects a greater percentage of medium (greenish) wavelengths. When the illuminants are arranged to ensure that the green patch reflects a greater absolute amount of long (reddish) than medium (greenish) wavelength light (Fig. 1b), one still perceives the patch to be green. The visual system takes account of the changes in incident illumination and what is perceived is a measure of the relative rather than absolute proportions of reflected light. This ability to perceive true color rather than be fooled by the fickle nature of light is called color constancy.

Figure 1.

Color constancy. Zeki et al. (1) illuminated an array of colors with varying amounts of long, medium, and short wavelength light. Each color patch reflects a constant proportion of the incident lights at any wavelength. (a) The green patch reflects 70% of the medium wavelength light. Normal observers report this as green when it is presented surrounded by other colors but as white when it is presented in isolation. Patient PB would report this patch as green or white depending on the intensity of the medium wavelength light. (b) How constancy was tested. Here the green patch is now illuminated by much more long wavelength light. The proportions of available light reflected from each surface remain unchanged but the absolute levels of reflected intensity are greatly changed, the long wavelengths now dominate. Normal observers still report the patch to be green when viewed in the presence of other colors. Patient PB, however, no longer perceives the patch to be green and now reports it to be red or white, depending on the intensity of the light. All units are arbitrary.

A remarkable demonstration of the segregation of color constancy from earlier processes in the color hierarchy is given by Zeki et al. in this issue of PNAS (1). The paper presents the results of experiments on a patient (PB) with severe atrophy of the occipito-parietal cortex who though described as “virtually blind” can report seeing colors. The question is does he see true colors irrespective of the illuminant or is his color perception better described as wavelength perception? The authors argue that the perception is wavelength based. When presented with a colored patch under one illuminant the patient often does not report seeing the same color as neurologically intact subjects; rather he names the color as if it were seen in isolation and therefore dominated by the wavelength information alone. The finding can be compared with studies of simultanagnosic patients whose perception of parts of objects is better than their perception of the whole. There are other accounts of patients whose color constancy mechanisms have failed (2, 3) and those with selective deficits have all suffered damage to regions of cortex that include the medial and lateral parts of the posterior fusiform gyrus that is human V4 (4). Physiological (5), behavioral (6, 7), and modeling (8) studies also suggest that V4 has a special role in color constancy. In Zeki et al.’s scheme of color processing (Fig. 2), the first stage of analysis occurs in V1 and V2 where simple wavelength information is registered; V4 occupies the second stage and is concerned with color constancy “without regard to memory, judgement or learning” (9); and the final stage centers on the inferior temporal cortex that associates color with form.

Figure 2.

Color in the cortex. Zeki et al.’s model (19) of color vision proposes three stages: wavelength processing (V1, V2), color constancy (V4), and object color (IT). Awareness of color is allocated to V4. Other possibilities are that V1 and V2 contribute to the perception of surface color (20), that V4 is important for several operations that apply to other visual attributes such as form and salience (12–16), and that the critical area for awareness of color is IT (21, 22). TEO, which stands between V4 and IT, has not been accorded any special role in the color hierarchy. V2 is much larger than is shown here, much of it is buried in the posterior bank of the lunate sulcus.

There is more to V4 than color constancy, however, and there is more to color constancy than V4. Notwithstanding the central role of V4 in color processing it is also clear that V4 is critical for the perception and learning of form (10–13), and selective attention to form and other attributes (14–16), and it also has been implicated in memory (17). An alternative view to Zeki et al.’s scheme may be to agree that V4 has a special role in constancy but to ask what is the relation between constancy and other factors such as memory and judgment. Some psychophysical studies of color constancy have shown convincingly that categorical color judgments (18) and color memory (19) are fundamental to color constancy and lesion studies support this view (6). Indeed it may not make sense to think of constancy as being separate from memory: the question “is that the same color?” need only be answered if one is matching a color with a representation of an already seen color, i.e., making comparisons over time.

There is sound electrophysiological evidence that V1 and V2 do more than register simple wavelength information; they also register combinations of hue and luminance (20). An example would be a neuron that responded to the presentation of a green color on a light background (when it is described by observers as olive) but not to the same green when presented on dark background (when it is described by observers as bright green). Such responses are examples of how V1 and V2 may contribute to processes that correct for changes in luminance as a result of changes in the chromatic content of the illuminant. Allowing a role for V1 and V2 in constancy computations is also consistent with the fact that color constancy deficits are partial phenomena, best regarded as threshold elevations, rather than total losses of the function.

To accompany the perceptual experiment, Zeki et al. (1) used functional magnetic resonance imaging to assess which brain regions may be responsible for PB’s spared chromatic vision. The results suggested that areas V1 and V2 may be sufficient for the level of chromatic analysis in this patient. It therefore would be interesting to know whether PB can name contrast colors appropriately as might be expected from Yoshioka et al.’s experiments (20). On the basis of the imaging results Zeki et al. make two further claims. The absence of any activity in area V4 in this patient is taken as an explanation of the compromised color constancy mechanisms, which seems fair. The second claim is more contentious: if the threshold for accepting activations is lowered below statistical significance, V4 is active in PB’s brain when he reports seeing colors. Zeki et al. conclude that this residual activity, though insufficient to support color constancy, is enough to support conscious color experience. The cortical site that governs the experience of color is the source of some debate. Zeki and Marini (9) have argued that damage to human visual area V4 causes achromatopsia, the inability to experience color. Others have argued that different lesions account for constancy deficits and achromatopsia (21, 22) and report that their achromatopsic patient (M.S.), although he cannot even detect the presence of a color in an array of gray patches, is able to use wavelength information to detect form. He also has some color constancy capacities. The conclusion of Heywood et al. (21, 22) is that the failure of M.S. to experience color is caused by damage anterior to V4 in the inferior temporal cortex. We thus have the position shown in Fig. 3, one patient can experience color but does not have constancy and the other has constancy but cannot experience color. The two functions are clearly separable.

Figure 3.

Zeki et al.’s (1) patient PB has awareness of color but has compromised color constancy whereas Heywood et al.’s (21, 22) patient has residual color constancy without any awareness of color. The choice on offer is between Zeki et al.’s claim that the reduced activation of V4 in patient PB is sufficient for awareness of color but not for constancy and Heywood et al.’s argument that color constancy and color awareness are dissociable.

Zeki et al.’s position described above rests on the idea that cortical area V4 mediates both experience and constancy, whereas Heywood et al.’s leans toward separate areas for each. Perhaps neither captures the complexity of cortical color processing that involves many cortical areas (23). If one were to consider constancy not as an all or none computation carried out in one area but as a graded function partially accomplished in the retina (24), enhanced in V1/V2 (20), and completed in V4 (5, 9), then constancy could be degraded by lesions that affected one of many sites in the color pathway. Similarly, there are other areas that could contribute to awareness of color. Zeki et al. (1) note that the role of V2 in awareness is unknown. Another candidate is TEO, which lies between V4 and inferior temporal cortex. Lesions to this area produce similar deficits to V4 lesions, but its role in color perception remains almost totally unexplored. It is inconceivable that TEO will not be involved in chromatic analysis in some way, whether in attention, constancy, discrimination, or learning remains to be seen, but the picture is woefully incomplete without a full study of this area.

Perhaps our present models of color perception should be considered as outlines that remain to be colored in. In the fifth century B.C., Empedocles claimed that “colors are carried by emanation to visual perception” and Ptolemy later argued that color “is only seen if light cooperates with vision” (25). I’m not sure what they meant, and sometimes I’m also not sure how much we could enlighten them.