The perceptual balance of color

Abstract

The cone contrasts carrying different dimensions of color vision vary greatly in magnitude, yet the perceived contrast of color and luminance in the world appears similar. We examined how this perceptual balance is adjusted by adaptation to the contrast in images. Observers set the level of L vs. M and S vs. LM contrast in 1/f noise images to match the perceived strength of a fixed level of luminance contrast. The perceptual balance of color in the images was roughly consistent with the range of contrasts characteristic of natural images. Relative perceived contrast could be strongly biased by brief prior exposure to images with lower or higher levels of chromatic contrast. Similar adaptation effects were found for luminance contrast in images of natural scenes. For both, observers reliably chose the contrast balance that appeared correct, and these choices were rapidly recalibrated by adaptation. This recalibration of the norm for contrast could reflect both changes in sensitivity and shifts in criterion. Our results are consistent with the possibility that color mechanisms adjust the range of their responses to match the range of signals in the environment, and that contrast adaptation plays an important role in these adjustments.

Introduction

Information about color is encoded by the activity in the three classes of cone receptor, and then recoded in postreceptoral channels that combine the cone signals to represent intensity (e.g. luminance or lightness), or oppose the signals to represent chromatic dimensions (hue and saturation). The magnitude of the cone signals available to luminance and chromatic channels, or to channels tuned to different chromatic axes, is very different. The cones have broad and overlapping spectral sensitivities, and as a result the difference between their signals (on which chromatic information depends) are much smaller than the range possible in their overall responses (on which luminance depends)[1, 2]. This difference is further exacerbated by the fact that natural color signals are themselves broad, smooth functions of wavelength[3]. The spectral sensitivities of the L and M cones are very similar, leading to a correlation of ^~0.99 in their responses for spectra characteristic of natural scenes[4]. Thus the cone contrasts mediating the L–M cardinal axis of postreceptoral color coding are only a small fraction of the contrasts carried by L+M or luminance mechanisms. Similarly, the cone signals available to postreceptoral mechanisms are also affected by the relative numbers of the three cone types[5, 6]. Thus chromatic differences that depend on the S cones, which make up only about 5–7% of the cone receptors[7], must draw on only a very small fraction of the receptor mosaic.

Despite the inherent and marked asymmetries imposed by these physical and physiological constraints, phenomenologically the world does not seem impoverished in color, or to be noticeably lacking in some dimensions of color space more than others. This subjective impression could potentially occur because the sensitivity of chromatic mechanisms is adjusted to match the gamut of available contrasts, so that the range of possible outputs within each channel is matched to the range of available inputs[4, 8]. In fact, sensitivity to chromatic contrast is much higher than luminance contrast when compared on the basis of cone contrasts[9–11]. Moreover, there is evidence that the signals originating from S cones are strongly amplified in visual cortex to compensate for the paucity of S cones in the photoreceptor array[12, 13]. The balance points for color appearance (e.g. the unique hues or nulls of the color-opponent channels) also depend little on the observer’s spectral sensitivity[14, 15] or on the relative numerosity of the different cone types[16, 17]. Such results thus suggest that the gains of color mechanisms are calibrated so that the experience of color is perceptually balanced.

In this study we explored how the relative gains controlling different dimensions of color appearance are maintained. An obvious signal for setting the criterion for image contrast would be to match the perceptual salience of different color dimensions to the range of color contrasts in the observer’s environment, so that what looks balanced is simply what the observer has been exposed to. To examine this we compared judgments of relative luminance and chromatic contrast to the distribution of contrasts in natural scenes. This match could reflect a number of different processes, from actual gain changes in chromatic mechanisms to learned expectations about the world. One potential mechanism for setting the balance would be if the response ranges are normalized through adaptation[18]. Adaptation to contrast induces large changes in threshold contrast sensitivity[19, 20] and suprathreshold perceived contrast[21–24], and strong contrast aftereffects occur when observers are exposed to stimuli with the characteristics of natural images[25–27]. Moreover, the adaptation is also strongly selective for different color directions, and thus could adjust the relative balance across different axes[24, 28–30]. To examine whether this adaptation can recalibrate the balance of color over short timescales, we compared the judgments after observers were exposed to images with more or less contrast.

Judgments of the balance of contrast are also of interest for understanding how different types of norms are established in perception. For many perceptual attributes, different levels along the dimension appear to be encoded relative to a norm or neutral point that itself appears neutral or perceptually balanced. For luminance or chromatic contrast this neutral point corresponds to a uniform gray (i.e. zero contrast), and a close correspondence holds between the subjective neutral point (what “looks” gray) and the neural null (the adapting level that does not bias the perceived null) [31]. However, for contrast there is a second “norm” – the level at which the world appears to have the correct or balanced level of variance. While there is some suggestion that neural mechanisms are tuned to different ranges of contrast so that perceived contrast depends on the relative responses across the cells[32], contrast coding is instead typically treated as an intensive dimension where the contrast is signaled by the size of the response, and by such accounts there is nothing unique about the neural signature of a correctly scaled image. This raises the question of whether the norm for contrast might be more dependent on a learned criterion for contrast.

Methods

Apparatus

Images were presented on a SONY 500PS monitor controlled by a VSG 2/5 graphics card, and subtended 6.25 deg. on a 19.3 by 14.5 deg. background with the same average luminance (28 Cd/m²) and chromaticity (x = 0.319 y = 0.328) as the images. Gun chromaticities and gamma functions for the monitor were measured with a PR650 spectroradiometer and intensities on the monitor were linearized through look–up tables. Observers viewed the images binocularly from a distance of 114 cm in an otherwise dark room.

Stimuli

We measured perceived contrast in two different sets of experiments using different image sets and procedures:

Filtered noise

In the first case we examined the relative perceived magnitude of luminance and chromatic contrast. The identity of specific objects (e.g. the expected color of sky or leaves) can strongly influence judgments of the natural balance of colors[33]. To avoid such cues, in the first set of experiments, observers instead judged the contrast of artificial noise images rather than actual scenes. Examples of the images are shown in Figure 1. A set of gray scale images (256 by 256 pixels) was first constructed by randomly sampling 8–bit pixel values from a normal distribution with a mean level of 128 and standard deviation of 30% of the mean. Contrast in the images was defined by this global RMS contrast[34]. The images were then filtered to a 1/f amplitude spectrum, so that they included the decline in contrast with increasing spatial frequency characteristic of natural images[2, 27, 35, 36]. Three different noise images were generated to define the red, green, and blue gun levels of the full color image. This image was then converted into separate contrast components corresponding to the L vs. M, S vs. LM and L+M+S achromatic cardinal axes, which again had pixel values with contrast values normally distributed about the mean. (Because of the random sampling the resulting images varied slightly in their absolute contrast along the three axes.) Note that the L vs. M and S vs. LM contrast at each pixel refers to the ratio of L/(L+M) or S/(L+M) for that pixel, and thus corresponds to a scaled version of the MacLeod–Boynton chromaticity. To avoid saturation of the pixel values and allow adapting and test colors to vary over a range above and below the luminance contrast, the contrast in the images was lowered to be 30% of the contrast in the original noise image. This corresponded to an RMS contrast of 10% for the achromatic component, which remained at a fixed level during adaptation (or at zero for the “pre–adapt” settings). Stimuli were then varied by scaling either the L vs. M or S vs. LM contrast.

Figure 1.

Examples of the 1/f noise images. All images had a fixed level of luminance contrast relative to the original noise image while chromatic contrasts along the L vs. M and S vs. LM axes were varied separately to lower or higher levels for adaptation.

Gray scale natural images

In the second case we focused on absolute perceived contrast, and for this restricted the experiment to measurements of luminance contrast in gray scale images of actual scenes. Five different images were chosen of natural outdoor scenes (Hill, Forest, Jungle Creek, Desert, Flowers) and were converted to gray scale. Luminance contrast was adjusted for each stimulus between 0x and 2x the original image contrast (1x representing the original, unaltered gray scale image) where 0x would be a uniform gray field matching the surround on which the stimulus was presented. The percentage of saturated pixels at the most extreme (2x) contrast averaged 3% of pixels at maximum (white) and 13% of pixels at minimum (black) pixel levels. Very little change in the percentage of saturated pixels occurred for the range of test contrasts seen by observers. The percentage of saturated pixels (0 or 255) was recorded as a function of image contrast to determine whether this saturation could predict the settings given by the observers. It could not.

Procedure

Chromatic vs. luminance contrast

In the first experiments comparing the balance of luminance to chromatic contrast, observers were instructed to adjust the chromatic contrast in the image until it appeared equal in magnitude to the fixed level of luminance contrast. The task was thus similar to previous studies of contrast matching between luminance and chromatic gratings[37] or gratings defined by arbitrary directions in color space[38], though in the present case the luminance and chromatic components were presented within the same image. Importantly, these previous results showed that observers can accurately and reliably judge perceived contrasts across different color directions, a result we also found in the present experiment.

On each run observers first adapted for 20 sec to the gray field or a set of adapting images with a chosen luminance and chromatic contrast. To avoid aftereffects from local light adaptation, during the adaptation the image was updated ^~every 300 ms with a new noise sample with the same contrast. The test image was then repeatedly shown for 500 ms interleaved with 3000 ms readaptation top–ups, with a uniform gray field shown 250 ms before and after each test. The contrast of the test was varied across successive presentations using a method of adjustment. The observer independently adjusted the L vs. M and S vs. LM contrast using separate pairs of buttons on a handheld keypad. The buttons allowed chromatic contrasts to be varied over a range from 0 to 100%, where the maximum value corresponded to the chromatic contrast present in the original color noise image. This range eliminated the possibility of pixel saturation in the stimuli. When observers were satisfied with the contrast balance in the test stimulus they pressed a button to save the setting and move on to the next image corresponding to a different noise sample. Settings were made for 5 test images during the run.

The experiment consisted of seven adapt conditions, including a pre–adapt condition where observers adapted to a uniform gray field (i.e. zero contrast). This “neutral adapt” condition was used as a baseline measure of the salience of color to luminance contrast (and as we argue below, could reflect the balance set by the observer’s long–term state of adaptation). Shifts in these baseline settings were then measured after adapting to images with a fixed luminance contrast but different levels of chromatic contrast. These settings were used to assess how the balance of chromatic and luminance contrast was altered by short–term adaptation to images that had different ratios of chromatic to luminance contrast. Relative to the base image with 30% of the original image contrast in all three dimensions, adapt stimuli had a fixed luminance contrast of 30% and either the L vs. M, S vs. LM or both contrasts reduced to 10% or increased to 90%. These are referred to in the results as Low L vs. M, High L vs. M, Low S vs. LM, High S vs. LM, Low Both and High Both. Images corresponding to the different contrast levels are illustrated in Figure 1. The seven adapt conditions were presented in random order for each observer.

In one case we measured the relative contrast in the images by showing a single test image with a fixed luminance contrast. Observers then adjusted the chromatic contrast along the L vs. M or S vs. LM axis until it appeared equal in magnitude to the luminance contrast. Note that this condition provided a measure only of the relative contrast, and not of the absolute perceived contrast. We therefore complemented these measurements with a second asymmetric matching task, where the test image had a fixed contrast, and observers instead adjusted the contrast of an image shown in a field to the side where they were under neutral adaptation. The spatial layout and temporal sequence of the stimuli in these two conditions is illustrated in Figure 3.

Figure 3.

Illustration of the spatial layout and temporal sequence of the two measurement conditions. In one case (top), the adapt and test images were alternated in the same centrally fixated field. Observers adjust the contrast of the test image. In the second case (bottom), the adapt and test were again alternated in the same field shown to one side of fixation. The contrast of the test remained fixed, and was matched by instead adjusting the contrast of a matching stimulus shown in a field on the other side of fixation.

Luminance contrast in natural images

For the grayscale scenes, presentation and adaptation sequences followed the same procedures as in the noise images, except that in this case only a single natural image was presented during adaptation, which observers were instructed to freely view. The use of actual scenes allowed observers to gauge contrast relative to an internal norm or criterion. Moreover, because the test stimuli varied along only a single dimension (luminance contrast), we modified the measurement method so that observers made a forced–choice judgment of whether the presented contrast was too low or too high. Contrast was then varied in two randomly interleaved staircases to assess the subjectively correct level, based on the mean of the last 6 reversals. Observers first made settings under neutral adaptation (to the uniform gray screen). Based on these settings, adapting images were chosen so that they were 0.5x or 1.5x the individual observer’s chosen contrast level for each scene. The same natural scene was used as the adapt and test stimulus for each run. Settings were then repeated after adaptation to the high or low contrast versions of each image.

Observers

The participants included author KM and 8 students at the University of Nevada who were unaware of the aims of the study, and who participated in different subsets of conditions. All had normal color vision as assessed by standard screening tests and normal or corrected to normal visual acuity. Participation was with informed consent and followed protocols approved by the university’s Institutional Review Board.

Results

Balance of color and luminance

In the first set of experiments we measured the chromatic contrast required to match the strength of a fixed reference level of luminance contrast, and how these matches changed after adapting to images with more or less chromatic contrast. Figure 4 plots the chromatic contrasts chosen to match the luminance contrast in the noise images for each of the different adaptation conditions. Each symbol shows the mean and standard error of the L vs. M and S vs. LM contrasts averaged across the 5 observers tested. Specifically, the level of L vs. M contrast chosen to match the reference level of luminance contrast is shown along the abscissa axis, while the level of S vs. LM contrast that appeared to match the luminance contrast is shown along the ordinate axis. Observers were able to consistently set the relative chromatic contrast in the images and thus were able to reliably judge the balance of color and luminance in the images. However, the settings under neutral adaptation to the gray field (closed circle symbol) were reliably biased by adaptation to images with altered contrast components. This was confirmed in a repeated measures analysis of variance, with the L vs. M and S vs. LM settings analyzed as separate dependent variables. The main effect of adaptation was significant for both chromatic axes (p < 0.001 for both axes). Adapting to a lower ratio of chromatic to luminance contrast reduces the ratio of color to luminance necessary for observers to perceive the stimuli as balanced. Thus the contrast level that appeared balanced under neutral adaptation now appeared to vary too strongly in color. (As we explore below, this effect may reflect adaptation to the relative levels of chromatic and luminance contrast, rather than to the absolute level of the chromatic contrast.) Conversely, adapting to higher chromatic contrasts increases the ratio of color to luminance necessary for observers to perceive the stimuli as balanced. Moreover, the adaptation effects were selective for the color direction that was altered in the adapting stimuli. That is, modulating the level of L vs. M contrast produced a 3x larger change in perceived contrast along the L vs. M axis (0.22) than the S vs. LM axis (0.072) (p = 0.006). Conversely, S vs. LM adaptation yielded a change that was 5x larger along the S vs. LM axis, though in this case the differences did not reach significance (p = 0.108).

Figure 4.

Matches for the L vs. M and S vs. LM contrast in the noise images. Settings show the mean for 5 observers ±1 SE following adaptation to the different levels of chromatic contrast, or under adaptation to the uniform gray field (gray circle).

Asymmetric matches

The results of figure 4 show that adaptation to images with higher or lower chromatic contrast altered the relative balance of chromatic to luminance contrast in the images. Again however, the preceding experiment only allowed us to assess the relative changes in contrast, and not possible absolute changes. To test for these, we repeated the settings for 3 observers but this time used an asymmetric matching task, in which the adapt and test stimuli were presented on the left side of fixation, while a comparison stimulus was shown on the right in a field that displayed a uniform gray during adaptation. This allowed observers to directly match the chromatic contrast under the two adaptation states, by adjusting the L vs. M and S vs. LM contrast of the comparison stimulus until it appeared to have the same contrast as the test stimulus. The test itself was fixed at a contrast of 0.3 along each axis, and thus remained at the same level, while the adapting images again had higher or lower contrasts. If there were no effect of adaptation, then the match should occur when the comparison and test have the same physical contrast. Alternatively, if adaptation alters the perceived contrast, then the comparison stimulus must be adjusted to have a different physical contrast in order to perceptually match the test[24].

The mean matches for these conditions are shown in Figure 5. Note in this case that the direction of the settings is reversed. In the preceding experiment high contrast adaptation reduced the perceived contrast in the test, and thus required increasing the physical contrast to null the aftereffect. In the present case the reduced perceived contrast in the test was directly matched by a lower contrast in the comparison stimulus. The relative aftereffects for the different adapting conditions remained similar to the previous conditions – adaptation to images with more color contrast reduced the apparent color contrast in the test image, and this was again selective for the chromatic axis of the contrast change in the adapting stimulus. However, under these conditions there was no evidence to suggest that low contrast adaptors actually increased the perceived contrast. Again, a physical match between the two images would occur when the comparison contrast was set to the test level (0.3, 0.3). This is very close to the match chosen for the low contrast adapting stimuli, suggesting that this adaptation did not in fact increase the absolute perceived color contrast in the images, but only their perceived contrast relative to luminance. The results are thus consistent with previous studies of short term adaptation that have found that contrast adaptation tends to have little effect on test patterns that have a higher contrast than the adapting patterns[22, 24]. On the other hand, the results of both experiments suggest that contrast adaptation exerts an important influence on perceived contrast during normal viewing, since it nevertheless selectively adjusts the balance of perceived contrast across dimensions of color coding.

Figure 5.

Asymmetric matches for the L vs. M and S vs. LM contrast in the noise images. Settings show the mean for 3 observers ±1 SE. In this case the matches correspond to the contrast in the right field image (under uniform field adaptation) that appeared equal to the contrast in the left field image (under adaptation to different levels of chromatic contrast).

The perceptual balance of color compared to color and luminance contrast in natural scenes

If contrast perception is normalized by the range of contrasts in the observer’s environment, then we should be able to predict the relative contrast matches that the observers chose from the gamut of color in natural scenes. Of course, this prediction is not straightforward because it would require knowledge of the specific visual environments of our observers and how they sampled them, as well as how this set their mean state of light and chromatic adaptation (which in turn defines the contrasts relative to this mean[39]). Nevertheless, as a qualitative test we compared the settings to the contrasts measured in natural outdoor scenes, based on the set of calibrated RGB images collected by Webster, Mizokami, and Webster[40]. The images were of natural scenes that avoided human artifacts, and were acquired in two different locations (the Western Ghats of India and the Eastern Sierras in Nevada) and at two different seasons corresponding to relatively more lush or arid periods. Details of the images are given in Webster et al.[40] To make this comparison, contrasts in the noise images were rescaled to the same metric used by Webster et al. in units that were chosen to roughly equate sensitivity and adaptation effects along each cardinal axis.

In Figure 6 the unfilled symbols plot the RMS contrast for individual scenes sampled by Webster et al. for the four different environments. That is, each point corresponds to the absolute contrast along different pairs of cardinal axes for one of their scenes. The filled circle with error bars at the lower left of each panel replots the neutral adaptation setting from the first experiment (corresponding to the “none” condition in Figure 4). It is evident from the figure that absolute contrast in the noise images was substantially lower than measured for the scenes. This was in part because we reduced the reference contrasts in the noise in order to probe adaptation to both higher and lower contrasts, but is also because the natural images included both earth and sky, giving bimodal distributions, and thus larger mean contrast than the single Gaussian distribution used to generate the noise. To visualize the relative sensitivity to contrast along the different axes, the pre–adapt matches from Figure 4 are also shown after multiplying by a factor of 6. The resulting line thus shows the ratio of sensitivity to the different axes predicted by the matches. Switkes[38] and Switkes and Crognale[37] found that contrast matches between luminance and color are linear and thus are maintained across scalar changes. (See also Kingdom Bell, Gheorgiu, and Malkoc[41]) This predicts that the relative matches we measured at low contrast should be preserved at higher absolute contrasts (though we did not directly test this for our conditions).

Figure 6.

Matches for the luminance and chromatic contrasts in the noise images compared to the range of contrasts in natural outdoor scenes from the Western Ghats in India (top row) or the Sierra Nevadas in the US (bottom row).Each panel plots the contrasts along different pairs of the L vs. M, S vs. LM, and Luminance axes. Unconnected symbols plot the RMS contrasts of individual scenes sampled during wet (green circles) or dry (red triangles) seasons, shown with the sensitivity–based contrast scaling reported in the study of the natural scenes[40]. Connected filled circles replot the pre-adapt (“none”) matches (±1 SE) of Figure 3 in the same units, at either the original contrast tested in the experiment (lower left symbol) or scaled by a factor of 6 to show the predicted settings at higher absolute contrasts similar to the natural scenes. Dashed lines (= cone contrasts) plot the matching contrasts predicted by equal pooled cone contrast[42] along each axis.

The relative contrasts along each pair of axes were compared between the observers’ noise matches and each environment with t–tests. In only one case did the relative contrasts predicted by the matches and by the natural scenes agree (ratio of luminance to S vs. LM contrast for arid Sierra scenes vs. noise settings; t(69) = 1.24, p = 0.11). Nevertheless, there is at least a rough correspondence to the contrast ratios measured for the arid season at either location. For example, these scenes had a mean ratio of S vs. LM to L vs. M contrast of 1.28 (Sierra) or 1.33 (India), compared to a ratio of 1.01 across all observers’ settings. Thus the scenes overestimated the S vs. LM contrast by only 30%. Alternatively, both of the wetter environments predicted a higher relative S vs. LM contrast (2.74 for India and 1.64 for the Sierra scenes). For all of the environments, the range of contrasts provides a reasonable approximation to the observed salience, especially when compared to the contrast matches that would be predicted from equivalent cone contrasts along the different axes. These are shown in the figures by the dashed lines representing equal pooled cone contrast[42]. In particular, sensitivity to the L vs. M contrast is approximately 6 times stronger than for the S vs. LM contrast, while ^~15 times stronger than for the pooled luminance cone contrasts. Similar differences in sensitivity to suprathreshold cone contrast were reported by Switkes and Crognale[37] (a ^~7 to 1 ratio of L vs. M to S vs. LM sensitivity), and by Kingdom et al.[41] (a ratio of 4.3), and are well established in measures of threshold sensitivity for luminance and chromatic contrast[9]. The specific sensitivity scaling depends on a variety of factors. For example, contrast salience varies with the spatial scale of the stimuli[43], and Kingdom et al.[41] showed that sensitivity to luminance contrast is suppressed when the luminance and chromatic variations are superimposed. However, all are consistent with very scaling for the cone contrasts along the different cardinal axes, and here our point is that these scalings are roughly what would be expected if the contrast coding is matched to the range of stimulus variation characteristic of the visual environment.

Adaptation to luminance contrast in images of natural scenes

In a further set of experiments, we focused on exploring the norm for contrast in images of actual scenes. In this case the measurements were restricted to grayscale images, and observers varied only the luminance contrast in the images until the contrast appeared “correct” for the image. Figure 7 shows the mean settings for the 5 scenes tested, either under neutral adaptation or after adapting to the same images with higher (1.5x) or lower (0.5x) contrast. Observers again reliably chose the subjective contrast level for the images. More importantly, these settings were again strongly biased by only brief exposures to the images at altered contrasts. As with the noise images, adaptation to higher contrast images shifted the response toward higher contrast, while adaptation to the low contrast images caused the original luminance contrast to instead appear too high, differences that in both cases were highly significant (p < 0.001).

Figure 7.

Scaling at which the grayscale images appeared to have the correct contrast relative to the original contrast. Each point plots the mean settings for 7 observers ±1 SE for a different image, under neutral adaptation to the gray field (gray circles) or after adapting to the same image displayed at 0.5x (black squares) or 1.5x (unfilled triangles) the original contrast.

Asymmetric matches

Again, the shifts in perceived contrast following the adaptation could reflect changes in either contrast sensitivity or the criterion for normal contrast or both. To examine these factors we again repeated the settings using an asymmetric matching task, in which observers first adapted to a low or high contrast image on the left and then matched the original test image on the left by adjusting the contrast of a comparison image on the right. The settings (Figure 8) were again consistent with the results we found for color – aftereffects tended to shift the settings for the stronger adapting stimulus and not the weaker one (though the difference between the mean settings was not in fact significant (p = 0.152). Thus like the color aftereffects, the shifts in perceived contrast for luminance at least trended toward losses in perceived contrast, suggesting that the shifts observed from lower contrasts might reflect a criterion change.

Figure 8.

Asymmetric matches for the grayscale images. Points plot the mean for 5 observers ±1 SE after adapting to the images at 0.5x or 1.5x the original contrast.

Discussion

In this study we explored why the world appears to vary as much in color as in lightness, even though the range of receptor signals available for conveying luminance and chromatic information are necessarily very different. Our results suggest that the perceptual balance of luminance and chromatic variation is at least qualitatively consistent with the range of variation expected from the natural visual environment, and that this balance can be rapidly and selectively readjusted when the balance in the stimulus changes. Thus what appears perceptually balanced to the observer may simply be the range of stimuli that the observer is exposed to; and at least part of this calibration may be driven by mechanisms of adaptation that adjust visual responses to match the range of contrast variation in the stimulus.

Before considering these contrast adaptation effects, it is important to emphasize that there are many additional processes regulating contrast coding. For example, adaptation typically refers to temporal adjustments to changes in the stimulus over time, and thus is a mechanism that could recalibrate sensitivity when the environment or observer changes[18]. Yet neural responses are also normalized through spatial mechanisms of contrast gain control[44–46], which probably play an important role in adjusting to the variations even within individual scenes[47, 48]. Further, the chromatic contrast adaptation we studied is thought to primarily depend on sensitivity changes at a cortical level[49, 50], and thus cannot account for gains set at retinal levels. A number of distinct mechanisms are involved in regulating contrast in retinal cells[51–55], yet it is not known how these in turn influence color appearance. For example, in primates strong contrast adaptation at precortical levels may be restricted to the magnocellular pathway which is less directly involved in perceived color and lightness[56].

While the contrast adaptation we examined is thus only one factor, our results suggest that it is an important one insofar as it could rapidly change the perceived contrast in images. These short term adaptation effects tended to reflect losses rather than enhancements in contrast sensitivity when observers were exposed to the stimulus, and were consistent with the effects on perceived contrast conventionally reported for short–term spatial or temporal contrast adaptation[21–24]. However, higher contrasts led to greater losses, and because these losses were selective for the different luminance and chromatic axes, they could change the perceived contrast balance across the axes. Consequently, the adaptation could mediate relative enhancements or losses along different color directions, and could provide one potential mechanism for maintaining the perceptual balance of color if the color environment changes. In this regard, it is important to consider that the baseline for comparing changes in contrast sensitivity should be the contrast level the observers is normally adapted to, and this is very different from the zero contrast field that is typically used to define the “pre–adaptation” sensitivity. Relative to the observer’s natural adaptation state, changes in the environment could thus modulate sensitivity to either decrease or increase responses.

Alternatively, for our conditions there were not clear absolute enhancements in sensitivity relative to the settings under adaptation to the uniform field. Recent studies have in fact found evidence for such contrast enhancements. For example, Zhang et al.[57] found that threshold contrast sensitivity for vertical patterns increased after observers spent hours viewing the world through a display that filtered in real time the energy around vertical. Kwon et al.[58] had observers view the world through contrast reducing optical filters. After 4 hours their behavioral contrast discrimination thresholds decreased while the BOLD response in V1 increased, consistent with an enhanced neural response to contrast. These enhancements occurred at longer timescales than those we measured. However, Hietanen et al.[59], testing timescales similar to those we used, measured luminance contrast matches in gratings after adapting to gratings spanning a wide range of frequencies and combinations of adapt and test contrast. They observed consistent enhancements in high contrast tests after adapting to low contrasts, but only at spatial frequencies below 1 c/deg. Our stimuli were instead broadband and it is uncertain which frequency ranges might dominate the perceived contrast in the images, or whether the enhancements they found (which they suggested might reflect the magnocellular pathway[59]) could occur for chromatic contrast. In any case, we did not observe robust increases in contrast sensitivity that could be unambiguously attributed to an absolute contrast enhancement.

The question of how and in which directions contrast can be renormalized is important for understanding how visual coding is established and maintained. Adaptation is well known to modulate neural responses[50], but it is less clear what this modulation is relative to, or specifically what mechanisms set the intrinsic response range of the neuron (e.g. so that responses are matched to the available dynamic range). Observers were very reliable at judging the contrast that looked correct in the images – even though they made their judgments in noise images with no obvious cues to possible object colors. This again illustrates that vision includes a well calibrated norm for contrast perception. Yet as noted in the introduction, this norm is distinct from the norms for many visual attributes, because it does not correspond to a clear null in the presumed neural response. One signature of a neural null is that adaptation to the norm should not alter the appearance of the stimulus, since this point should already uniquely correspond to balanced responses across the encoding mechanisms, and thus should not alter this balance[31]. A norm of this type would predict roughly equal and opposite changes in perceived contrast as observers adapted to lower or higher values, which again was not a pattern we observed. If the perceptual norm for contrast reflects a unique state of the underlying neural response, this state may thus instead depend on a unique magnitude of the response. However, it remains unclear how this state is defined.

Whatever the processes calibrating the visual response to contrast, our results suggest that they are roughly matched to the range of contrasts along different color directions in the observers’ environment. As noted in the Introduction, there is strong theoretical justification for this match, as well as empirical evidence for this correspondence in both the contrast response function[8] and in the relative sensitivity to luminance and color[4]. For example, Webster and Mollon[27] noted that the relative contrasts they found in natural images along the cardinal axes was similar when contrasts were equated by psychophysical measures of sensitivity, and this can similarly be seen in the distribution of image contrasts in Figure 6. Further, the relative sensitivity to chromatic and luminance signals, when signals are compared in terms of cone contrasts, is roughly consistent with the range of cone contrasts from natural environments[4].

While suggestive, however, the correspondence we found between the perceived contrast balance in our images and the balance in natural scenes was only approximate. In particular, observer’s matches significantly differed from the specific contrast ranges we compared them to for natural outdoor environments. There are a number of possible bases for these differences. First, as noted, one possibility is that the observers were adapted to environments with different ratios of luminance and chromatic contrasts. In fact this is likely given that most modern visual environments include carpentered indoor scenes which may have very different color properties. Second, while our 1/f noise was chosen to approximate the amplitude spectra of natural scenes, the random phase spectrum did not approximate the prevalence of edges in scenes or the natural relationships between the spatial variations in luminance and color[60–62]. Third, the specific matches we obtained could depend strongly on masking interactions between luminance and color, which might in turn depend strongly on the parameters of the stimuli. For example, luminance edges can facilitate the discrimination of color differences, while chromatic variations more strongly mask the detection of luminance variations[63–68]. Kingdom et al.[41] showed that these asymmetric masking interactions can alter the relative salience of luminance and chromatic signals even when both are suprathreshold. Moreover, in natural images, the variations in luminance contrast often include both material changes and lighting changes (e.g. shadows), while chromatic variations are more reliably tied to material changes[69]. The pattern of interactions between chromatic and luminance contrast may differ depending on the nature of these spatial variations[70]. A fourth related caveat is that the absolute contrast levels in our noise images were much lower than the natural scenes. As noted there is evidence that the relative salience of luminance and color is invariant with contrast[37, 38, 41]. Yet the extent of masking or facilitation is well known to show strong contrast dependence[66, 71]. Finally, we probed only the contrast matches defined by independent variations in contrast along the three cardinal directions. In actual environments there can be strong correlations between the luminance and chromatic variations or between the S vs. LM and L vs. M signals. For example, color in many natural scenes varies predominantly along a bluish–yellowish axis intermediate to the cardinal chromatic axes[27, 40]. In some tasks including contrast matching between luminance and chromatic gratings, the salience of intermediate chromatic directions is consistent with the independent signals along the cardinal axes[38]. However, in a number of other tasks the visual system shows a weaker sensitivity to modulations along the bluish–yellowish direction, consistent with a potential selective adaptation to the greater variance for this color direction in natural scenes[18].

Given these complications, it is notable that measures of contrast in a restricted subset of natural scenes could nevertheless roughly approximate observers’ subjective judgments of contrast salience, and in fact provided a closer prediction than a uniform color space like CIE L*a*b*. A common problem in comparing visual performance along different dimensions of color vision is that there is no established metric for scaling contrasts along the different axes, a problem which also plagues the construction and use of color spaces. Some scaling metrics are arbitrary (e.g. determined by the gamut of the display device) while others have attempted to equate the units for different axes by adjusting them for visual sensitivity (e.g. times threshold) or equivalent physiological signals (e.g. cone contrasts). As we noted at the outset, it is well established that cone contrasts do not predict performance on many tasks, presumably because contrast sensitivity is in part compensated for the range of available stimuli. This suggests that one way of scaling different color dimensions would be to base them directly on measures of the stimulus distribution. Our results suggest that this scaling would predict the relative perceived contrast along different color dimensions reasonably well under natural viewing conditions.

Figure 2.

Examples of the grayscale images for assessing the perception of luminance contrast in images. Contrasts were scaled from 0 to 2x the original contrast (1).