Contribution of S opponent cells to color appearance

Abstract

We measured the regions in isoluminant color space over which observers perceive red, yellow, green, and blue and examined the extent to which the colors vary in perceived amount within these regions. We compared color scaling of various isoluminant stimuli by using large spots, which activate all cone types, to that with tiny spots in the central foveola, where S cones, and thus S opponent (S_o) cell activity, are largely or entirely absent. The addition of S_o input to that from the L and M opponent cells changes the chromatic appearance of all colors, affecting each primary color in different chromatic regions in the directions and by the amount predicted by our color model. Shifts from white to the various chromatic stimuli we used produced sinusoidal variations in cone activation as a function of color angle for each cone type and in the responses of lateral geniculate cells. However, psychophysical color-scaling functions for 2° spots were nonsinusoidal, being much more peaked. The color-scaling functions are well fit by sine waves raised to exponents between 1 and 3. The same is true for the color responses of a large subpopulation of striate cortex cells. The narrow color tuning, the discrepancies between the spectral loci of the peaks of the color-scaling curves and those of lateral geniculate cells, and the changes in color appearance produced by eliminating S_o input provide evidence for a cortical processing stage at which the color axes are rotated by a combination of the outputs of S_o cells with those of L and M opponent cells in the manner that we postulated earlier. There seems to be an expansive response nonlinearity at this stage.

The early stages in the processing of chromatic information by the visual system have been well established (1). There are three classes of cone photoreceptors (2): the S cones, the M cones, and the L cones. Each absorbs broadly across the spectrum. The outputs of different cone types are compared in three distinctly different ways in the retina (3–5) to form two spectrally opponent cell types and one spectrally nonopponent variety. One class of opponent cells (L_o and M_o) differences the outputs of the L and M cones, and the other (S_o) differences the output of the S cones from that of the combined L and M cones. The additional transformations in color information that take place within the cortex to account for color appearance are not well understood. The experiments reported herein, in conjunction with other studies of the responses of cells in monkey lateral geniculate nucleus (LGN) and visual cortex to identical stimuli (R.L.D.V., N. P. Cottaris, S. D. Elfar, and L.E.M., unpublished work), investigate the nature of these cortical transformations.

We report herein two experiments based on a color-scaling technique similar to one we and others have used previously (6–8) but with certain crucial variations. One experiment tested some fundamental predictions from our color model (1) concerning the role of S_o cells in color appearance. The other was prompted by observations we have made recently on the differences between the narrowness of color tuning in monkey LGN cells and that in certain cells in the striate cortex (R.L.D.V., N. P. Cottaris, S. D. Elfar, and L.E.M., unpublished work). Of interest is the relation between the narrowness of perceptual color tuning and the tuning properties of cells at these two processing stages.

In most color-scaling studies, including ours (8) and others (e.g., refs. 6 and 7), observers specify the hue of a stimulus by reporting the percentage of each of four hues (red, yellow, green, and blue) seen in that stimulus, with the four percentages constrained to sum to 100. With that constraint, however, the estimated shapes of the four tuning functions are not independent. We have therefore repeated the experiment by having observers judge each color separately, with no requirement that the responses add up to a certain total.

Normal observers tend to be tritanopic for very small stimuli foveally fixated (9). Recent anatomical (e.g., refs. 10–12) and psychophysical (13) studies confirm the presumed basis, namely, that S cones are essentially absent in the very center of the foveola. We (1) have suggested a special role for the S_o cells (whose activity would be absent in the absence of S cone activation) in encoding hues. We have tested this suggestion by examining color scaling with and without input from S_o cells. We compared color scaling for large spots, producing maximum S cone activation, with that for very small spots presented in the center of the foveola, activating few, if any, S cones.

Methods

Stimuli were presented on an NEC color monitor (Nippon Electric, Tokyo) controlled by a sun-taac computer graphics system (Sun Microsystems, Mountain View, CA), which also collected and analyzed the observers' responses. Stimuli were circular spots of specified diameter, with Gaussian tapered edges. The full-contrast portion of the spots subtended 2° visual angle for the large stimuli and 0.1° for the small stimuli. The viewing distance was 57 cm for the large spots and 172 cm for the small ones. Stimuli were turned on and off with temporal Gaussian ramps. The presentation duration at full contrast was 0.5 s for the 2° spots and 0.125 s for the small spots. The 0.1° spot was presented in the center of a fixation target of four line segments forming a cross with a central gap in which the stimulus appeared. The observer triggered the stimulus onset when s/he was sure that s/he was fixating the center.

The screen was white (illuminant C) at 27 cd/m². Stimuli were isoluminant shifts from this background white to various colored spots (see Fig. 1A). The chromatic stimuli were defined by vectors in a color space based on the characteristics of cones (14) and LGN cells (5), which we refer to as DKL color space. The primary axes in a horizontal (isoluminant) plane of this space are the L_o/M_o axis (0°-180°) and the S_o axis (90°–270°). Along the L_o/M_o axis, the L and M cone contrasts were 8% and 16%, respectively, and along the S_o axis, the S cone contrast was 83%. The intermediate angles were defined as combinations of these unit vectors, e.g., 45° has equal contributions from the 0° and 90° vectors. (There is no official standardization of DKL color space, and thus no accepted specification of color angles other than the L_o/M_o and S_o axes). The contrasts of the 36 chromatic stimuli were chosen such that the excursions in different color directions around the circle from 0° to 360° produced sinusoidal changes in cone activation for each cone type; see Fig. 1B. The chromaticities and contrasts of these stimuli were identical to those we used to study the responses of neurons in macaque LGN and striate cortex (R.L.D.V., N. P. Cottaris, S. D. Elfar, and L.E.M., unpublished work), a point that will be of relevance below.

Figure 1.

Stimuli used in this experiment. (A) The locations of the isoluminant white and chromatic stimuli used are shown in the CIE 1931 xyz chromaticity diagram. Each stimulus consisted of a brief shift from the white to one of the chromatic stimuli. (B) Shown are the percentages of change in L, M, and S cone activation produced by the shifts from white to the various chromatic stimuli shown in A. Note that the variations in contrast for each cone type are sinusoidal functions of the color angle. DKL, Derrington–Krauskopf–Lennie.

The observer signaled with 0 to 8 pushes of a button to indicate the amount of a particular hue, e.g., blue, s/he saw in each of the 36 chromatic stimuli. (In some cases, this judgment is as much akin to saturation scaling as to hue scaling. Because of this ambiguity, we refer to the task as color scaling, though it does not include any estimation of brightness.) After each response, pressing a second button triggered the next stimulus presentation. In the case of the 0.1° foveal stimuli, the observer could press a third button to signal not having seen the stimulus at all. In different series, each observer separately scaled each of the 36 chromatic stimuli for the amount of red, yellow, green, or blue. On a given run, each stimulus in the series was presented twice, and each run was repeated 10 times for a total of 20 judgments of each stimulus at each size pertaining to a particular hue. Runs were distributed over periods of several weeks, interspersed with other experiments, to maximize the independence of the judgments. Two of the authors and one naive observer participated. All were color-normal, as assessed by the Farnsworth–Munsell 100-hue test and the Nagel anomaloscope, had corrected-to-normal visual acuity, and gave their written consent to participate.

Results

Sharpness of the Color-Scaling Functions.

Do sinusoidal changes in cone activation produce sinusoidal changes in color, or is color appearance more sharply tuned? We find (R.L.D.V., N. P. Cottaris, S. D. Elfar, and L.E.M., unpublished work) that LGN cells, in response to the same stimuli, produce sinusoidal variations in firing rate as a function of color angle in response to stimuli that vary sinusoidally in cone contrast, whereas more than half of the striate cortex cells we have examined show more peaked response functions and thus narrower chromatic tuning.

Fig. 2 shows the results of color scaling with 2° spots. Because the results from each of our three observers were very similar, we present data averaged across all three. First, in agreement with our previous report (8), we find that the red, blue, green, and yellow color regions and peaks fail to coincide with the LGN cell peaks (which are at 0°, 90°, 180°, and 270°) but rather lie between them, as predicted by our color model (1). Secondly, the color-scaling curves are quite nonsinusoidal, with shapes that are too narrow, reflecting a nonlinearity with respect to both cone contrast and LGN cell responses.

Figure 2.

Averaged color-scaling results for three observers with a 2° foveal stimulus. Note that the color-scaling functions are all displaced from the LGN axes, and all but red are quite nonsinusoidal functions of color angle.

The shape of the cone contrast functions (c) for each of the L, M, and S cones as a function of color angle for our 36 stimuli can be specified as:

When n = 1, the curve is a sinusoid. Because LGN cells respond quite linearly, a sinusoid also precisely fits the response functions of LGN cells to these stimuli. However, if there were an expansive contrast-response nonlinearity intervening between the LGN and the presumed cortical site specifying the amount of color in a chromatic stimulus, one would expect the color-scaling data to be fit not with a sinusoid but with a sine function raised to some power greater than 1.

To examine this expectation, we fit each of the red, yellow, green, and blue color functions with an exponentiated sinusoid, solving for n. For each of the colors except red, there is evidence of an expansive nonlinearity, the color-scaling functions being better fit by sine waves raised to exponents greater than 1. The green function is well fit with a sine raised to the power of 1.7; see Fig. 3. The blue and yellow data are better fit by exponentiated sinusoids (exponents of 1.8 for blue and 2.9 for yellow) than by an untransformed sinusoid but clearly have some additional nonlinearities as is evidenced by the skewed nature of each.

Figure 3.

Color scaling functions for green, fit with a sine wave and with an exponentiated sine wave, solving for the best-fitting exponent. It can be seen that the color-scaling data are better fit with a sinusoid raised to the power of 1.7 than with a sine wave. The fact that the cone contrasts and the responses of cones and of LGN cells vary sinusoidally with color angle suggests the presence of an expansive contrast nonlinearity in the cortical processing.

Color Scaling with a Small Foveal Spot.

If there is a small foveal area devoid of S cones in the retinae of each of our observers, and if our brief, 0.1° foveal spots were fixated accurately and steadily enough to be restricted to such an S cone-free region, then color scaling with these small stimuli would give an indication of what colors look like without S cone and S_o cell input. To what extent are these assumptions justified? It can be seen in Fig. 1B that, with our stimuli at 90° and 270°, the S cone contrast would be maximal (83%), but the L and M cone contrasts would be zero. If there is an S cone-free zone, those stimuli should be invisible. Furthermore, if the stimulus with maximal S cone contrast were invisible, then it seems highly unlikely that the S cones would be making a significant contribution at other vectors at which the S cone contrast was lower.

As noted above, the observers signaled when the 0.1° foveal spot was not seen at all. If S cones were absent under our small-spot viewing conditions for a given observer (0.1° spots in the region of 90° and 270°), those of maximum S cone but zero L and M cone contrast should never be seen. We had a measure of that in the data from our regular trials, but in addition, we tested each observer for the detectability of the various small, brief, chromatic stimuli.

Fig. 4 shows the percentage of trials in which one observer reported seeing the small spots for different color vectors. The 0.1° foveal stimuli were all perfectly detectable except in the regions of 90° and 270°. (It is not clear why the null point is shifted in this observer to about 97° and 280°; the deviation from the expected 90° and 270° null points varied slightly across observers but always toward somewhat greater angles than expected from our calculations, based on Smith–Pokorny fundamentals and assumptions about preretinal absorption.) The curves for the other observers did not fall to zero at the minimum, but in each case, S cone-only stimuli were seen less than 20% of the time (and these, subjects reported, appeared extremely faint). One additional observer, from whom we had intended to collect data, was eliminated from the experiment, because she almost always reported seeing the 0.1° stimuli. She presumably either had no S cone-free foveal region or could not adequately maintain fixation.

Figure 4.

Percentage of detection of 0.1° spots of different color vectors briefly presented in the central foveola. If S cones are missing within a small area of the central foveola, very small foveal stimuli of color vectors near 90° and 270° (where the L and M cone contrasts fall to zero but S cone contrast is maximal; see Fig. 1B) should be invisible, as they are for this observer.

In theory, we could have accomplished the same goal of eliminating S cone input by examining the appearance of spots of different contrast along the 0°–180° color axis. The relative weights of the L and M cone activation vary with a shift from white to any point along this axis, with no change in activation of the S cones. We did in fact test one observer with this alternative approach, with results very comparable to those obtained with the use of a small foveal spot. However, we decided that the approach of using a small foveal spot was preferable for the main experiment. With this approach, it was not necessary to depend on calculations based on presumed cone pigment and optical filtering curves to conclude that S cones were not activated. Rather, we had verification of the absence of S cone activation in the observer's inability to see even the highest contrast S only stimulus when it was presented as a small, brief foveal spot.

Fig. 5 shows the results of color-scaling by our observers under conditions of no (or almost no) S cone input. Data points are not presented for those stimuli that were rarely seen. Any stimuli that were visible in the region from 90° to 270° were seen as almost equally green and blue, whereas those in the region from 270° to 90° (450°) appeared predominantly red with a small admixture of yellow. That there are only two different color regions, with approximately uniform appearance within each, is of course not surprising, because, with the effective absence of S cones, the observers would have only dichromatic (tritanopic) color vision. However, that the color region from 90° to 270° was seen to contain as much blue as green and the region from 270° to 90° was seen as partially yellow is not in accord with an LGN-based model that considers the 0°–180° LGN axis to be the red–green perceptual axis and the 90°–270° axis to be the blue–yellow perceptual axis. This observation is, however, in accord with our (1) multistage color model, as discussed below. The fact that our observers performed as dichromats under these conditions is additional evidence that our procedure effectively eliminated S cone input.

Figure 5.

Color scaling of 0.1° foveal stimuli, with points omitted for stimuli around 90° and 270°, which were rarely seen. For all observers, the color range from 90° to 270° is seen as almost equally blue and green, and the region from 270° to 90° was seen as red with a little yellow. Note that these color-scaling results vary sinusoidally with color axis.

Note also in Fig. 5 that, in the absence of S_o input, the color scaling as a function of color angle is not peaked, as it is for the 2° spots. This finding suggests that much of the expansive nonlinearity that produces the narrow color regions seen with larger spots is due to the S_o system. We have presented physiological evidence (15) in accord with that.

Contribution of S_o Cells to Color Appearance.

One main purpose of this study was to compare color scaling with a 2° spot, when there is maximum input from all cone types, with that seen with a 0.1° spot, when there is little or no input from S cones and thus S_o LGN cells. The difference between the 2° and the 0.1° data, then, should show the contribution to color appearance made by the S_o cells. Fig. 6A shows that difference for blue and green appearance in the color region from 90° to 270°. It can be seen that adding input from the S_o system increases the amount of green seen in the range of 180° to 270° and decreases the amount of green in the 90° to 180° range. It also increases the amount of blue seen in the 90° to 180° range and decreases the amount of blue in the 180° to 270° range.

Figure 6.

The color-scaling data for 2° spots (which activate all three cone types) minus the data for 0.1° spots (which activate L and M but not S cones) show the contribution made by S cones to color appearance. (A) The results for green and blue (centered around 180° on the x axis) are shown. Adding S cones and thus S_o cells makes the region from 90° to 180° more blue and less green and the region from 180° to 270° more green and less blue. (B) Comparable data for yellow and red (centered around 360°).

The results for red and yellow in the color range from 270° to 450° (90°) are shown in Fig. 6B (note that the x axis is now centered around 360°). Adding input from S_o cells slightly increases the amount of red seen in the range of 0°–90° (360°–450°) and slightly decreases the amount of red in the range from 270° to 360°. The changes in yellow are greater. Adding S_o input considerably increases the amount of yellow seen in the range of 270°–360° and decreases the amount of yellow from 0° to 90° (360°–450°).

Relation to De Valois and De Valois Color Model.

We earlier proposed a multistage color model (1), not to account for these or other hue-scaling data, but based on what seemed to be the most plausible anatomical organizations at the retina, LGN, and cortex, based on largely random connectivity. We suggested that the two canonical LGN axes (L/M opponent and S/LM opponent) do not constitute the red–green and yellow–blue color systems but rather that the perceptual color systems are formed at a later cortical stage at which the two LGN axes are combined. We proposed that the addition or subtraction of the S_o cell outputs from the L/M and M/L opponent cell responses rotates the color axes away from the canonical LGN axes. The only arbitrary factor included was an assumption that the S_o system was doubled in weight at the cortical level. It is of interest to see to what extent the model predicts the two sets of data in the current study, namely, color scaling with and without S_o cells. Fig. 7 shows the model's predictions of color responses with (Fig. 7A) and without (Fig. 7B) S_o input to the color systems. It can be seen that the model captures the essence of the results shown in Figs. 2 and 4. Fig. 7C shows the differences between the predicted color responses with and without S_o cells, that is, the predicted contribution that the S_o system makes to color appearance. These predictions correspond reasonably well to the color-scaling data shown in Fig. 6.

Figure 7.

The predictions of the modified De Valois and De Valois color model for the 2° and the 0.1° data. (A) The model assumes all systems are functional. It can be seen that the peaks of the model correspond closely to the peaks of the red, yellow, green, and blue color-scaling data shown in Fig. 2. (B) The predictions from the model with the inputs from S cones and S_o cells eliminated. The curves from the model now correspond closely to the color-scaling data with the 0.1° spots shown in Fig. 5. (C) The assumed contributions of S_o cells from our model. Comparing these to the data in Fig. 6 shows that our model matches the data well with respect to the directions of the changes observed and the relative magnitude of the changes for different colors.

Discussion

The various chromatic stimuli we used in this experiment were chosen such that the changes in L, M, and S cone contrasts each varied sinusoidally with different angles around the color circle. Any linear combination of two or three cone outputs in the later retinal neural processing, whether in opponent or nonopponent fashion, would produce a cell whose output should be a sinusoidal function of the color angle. In fact, we find that the responses of cells in the parvo and konio layers of the macaque monkey LGN to these stimuli do vary sinusoidally with color angle (R.L.D.V., N. P. Cottaris, S. D. Elfar, and L.E.M., unpublished work). Thus, the processing before the cortex, under these conditions of constant adaptation level, is very linear, as Derrington et al. (5) have also reported.

However, the situation in striate cortex is quite different. We find (R.L.D.V., N. P. Cottaris, S. D. Elfar, and L.E.M., unpublished work), contrary to an earlier report (16), that although some macaque striate cortex cells respond sinusoidally to color vector variations that produce sinusoidal LGN responses, the chromatic responses of about two-thirds of striate neurons are better fit by sinusoids raised to an exponent greater than 1. Across all cells in our sample, including the approximately one-third that are quite linear, the median exponent of the best-fitting exponentiated sinusoid is 1.9. This expansive nonlinearity makes striate cortex cells on average considerably more narrowly tuned for (isoluminant) color angle than are LGN cells.

Although the cone contrasts for our stimuli varied sinusoidally with color angle (Fig. 1B), our observers' color-scaling data for single colors (Fig. 2) clearly did not. The judged magnitude of each color, with the exception of red, falls off sharply from its peak. Thus, for instance, although spots over a fairly broad range of color angles are seen as having some yellow in them, only a narrow region is seen as being a highly saturated yellow. The functions for all the colors except red are better fit not with sine waves but with sinusoids raised to some exponent between 1.7 and 2.9. The magnitude of the nonlinearity for the different colors varies with the size of the S_o input in our modified color model: smallest for red, intermediate for green and blue, and largest for yellow. This result suggests, as does the more sinusoidal color scaling shown in the absence of S_o input in the 0.1° data, that the primary source of the expansive nonlinearity may be in the S_o system before it is combined with the L_o/M_o system at the cortex. This observation is also in accord with a receptive field study of cortical simple cells (15) in which we find that the major nonlinearity is in the S_o input to V1 cells.

There have been several reports of an expansive contrast-response nonlinearity in the responses of V1 cells to luminance gratings of different contrasts (17, 18). As Geisler and Albrecht (18) point out, it provides a simple way for the nervous system to increase the selectivity of neurons. For instance, an expansive response nonlinearity can increase the orientation selectivity of a cell without the need to have an extremely long receptive field or increase its spatial frequency selectivity without its having to have a large number of side bands in the receptive field. Similarly, we suggest that such an expansive nonlinearity can and does increase the color selectivity of V1 neurons without the need for a complex interaction between different chromatic units. The perceptual result may be that chromatic stimuli appear more different from each other than they would otherwise.

A second purpose of this study was to examine the contribution of the S_o system to color appearance by judging color appearance with and without S cones, by using small foveally presented spots to eliminate S cone activity. Several previous investigators have examined hue scaling with small foveal spots (or made incidental comments on color appearance) in experiments similar to ours, but these investigators used incremental colored flashes presented on a dark background rather than isoluminant color shifts from white. An incremental flash of light of any short wavelength will normally activate all three cone types in retinal regions where all are present. If an observer has no S cones in the central foveola and if s/he can fixate accurately, a very small foveal incremental flash should stimulate only L and M cones and thus be comparable to our procedure. The problem is that it is difficult to determine whether an incremental flash is actually stimulating only L and M cones for any given observer. The fact that previous experiments with small incremental spots did not reduce the observers to dichromatic performance indicates that they did not completely eliminate S cone input. Nonetheless, several of these earlier studies (19–22) reported that small short-wavelength spots were seen as blue–green, in accord with our data.

Also in accord with our data are the reports (23, 24) that unilateral tritanopic observers see the entire short-wavelength half of the spectrum as uniformly greenish-blue in their tritanopic eye. That the 90°–270° color range (equivalent to dominant wavelengths of about 400–570 nm with reference to illuminant C) was seen as a greenish-blue in the absence of S input is not in accord with an LGN-based color vision model that assumes that the S_o cells form the basis for the yellow–blue color system. Such a model would predict that this spectral region should appear green in the absence of S cones, not partially or largely blue. Assuming such a model, Mollon (25), for example, offered a convoluted, cognitive argument as to why unilateral tritanopes see blue in the absence of S cones.

The results of color scaling with the 2° and 0.1° stimuli are clearly consistent with our suggestion (1) that +S_o cells are added to +M_o−L_o cells to form blue and subtracted from +M_o−L_o cells (or, equivalently, −S_o cells added to them) to form green. Thus, adding +S_o input should increase the appearance of blue and decrease the appearance of green in the region from 0° to 180° and increase the appearance of green and decrease that of blue in the region from 180° to 360°. As Fig. 6A shows, these changes in color appearance are just what we find in the difference between the 2° and the 0.1° color scaling. Note that our model predicts, and the data show, that signals arising in the S cones make as large a contribution to the appearance of green as they do to blue. Our model also predicts, and the data show, that the +M_o−L_o cells make as much contribution to blue as to green.

In the other half of the color circle, we postulate that +S_o cell outputs are added to +L_o−M_o cells to form red and are subtracted from them to form yellow. The results shown in Fig. 6B are in accord with our modified model. We suggest that adding +S_o input should make the region from 0° to 180° slightly more red, as it does, and subtracting +S_o input should make the region from 180° to 270° much more yellow, as it does. A comparison of the data in Figs. 2 and 5 with the model predictions in Fig. 7 shows that our color model captures quite accurately the essential features of the 2° and 0.1° color-scaling results.

Although there is a large literature establishing and discussing the contribution of +S_o cells to the appearance of red at short wavelengths (e.g., refs. 26 and 27), there has been little recognition heretofore of the contribution that S_o cells make to the appearance of green. The results we report herein show that the role played by S_o cells in establishing the appearance of green is significantly greater than their role in establishing the appearance of red.

Abbreviations

LGN

lateral geniculate nucleus

S_o

S opponent

L_o

L opponent

M_o

M opponent

DKL

Derrington–Krauskopf–Lennie