Adaptation and perceptual norms in color vision

Abstract

Many perceptual dimensions are thought to be represented relative to an average value or norm. Models of norm-based coding assume that the norm appears psychologically neutral because it reflects a neutral response in the underlying neural code. We tested this assumption in human color vision by asking how judgments of “white” are affected as neural responses are altered by adaptation. The adapting color was varied to determine the stimulus level that did not bias the observer’s subjective white point. This level represents a response norm at the stages at which sensitivity is regulated by the adaptation, and we show that these response norms correspond to the perceptually neutral stimulus and that they can account for how the perception of white varies both across different observers and within the same observer at different locations in the visual field. We also show that individual differences in perceived white are reduced when observers are exposed to a common white adapting stimulus, suggesting that the perceptual differences are due in part to differences in how neural responses are normalized. These results suggest a close link between the norms for appearance and coding in color vision and illustrate a general paradigm for exploring this link in other perceptual domains.

1. INTRODUCTION

Stimuli along many perceptual dimensions appear to be judged relative to a well-defined norm, which itself appears neutral or unbiased. For example, in color vision “gray” represents a unique neutral point, with hue and saturation defined by how the stimulus differs from the achromatic point [1]. Norms have been invoked to account for the perception of both simple features (e.g., orientation or motion) and high-level attributes (e.g., faces) [2]. Thus the perception of orientation might in part be normalized relative to reference axes of vertical and horizontal [3], while individual faces may be perceived according to their “identity trajectories” relative to a prototype face [4].These diverse examples suggest that norms are in fact fundamental to most perceptual judgments [5].

Models of visual coding typically assume that norms are perceived as neutral because they reflect a neutral or balanced response in the underlying neural code. That is, the norm looks special because the visual response is special. One way in which a norm might be explicitly encoded is by a response null within a channel [Fig. 1(a)]. For example, in color vision, signals from the cone receptors are combined to form color-opponent channels that respond in opposite ways to different parts of the spectrum. The null occurs when the excitatory and inhibitory inputs are balanced so that there is no net response within the channel [6]. A second way in which a norm might be represented is by an equal distribution of activity across a set of channels [Fig. 1(b)]. In color, this would correspond to the norm at the level of the cones, where “white” corresponds to balanced activity across the three cone types. In this case the norm is represented only implicitly, since it is not given directly by the response within any single mechanism.

Fig. 1.

Commonly proposed schemes in visual coding. In (a) different levels along the stimulus dimension are signaled by the degree of excitation or inhibition within a single opponent channel, with the norm coded by a response null. In (b) the dimension is represented by the population response of multiple channels each narrowly tuned to different stimulus levels. In this case the norm occurs when the responses across the channels are equal. (c) If the stimulus is narrowly tuned, then the same multiple channels result in a central-tendency code in which the level is signaled by which subset of channels respond. Such models lack a unique response norm. Upper and lower panels illustrate how the channel responses change with adaptation. In the central-tendency model, adapting to any level reduces sensitivity to that stimulus and biases appearance away from the adapting level, with no change in the perceived level of the adapter [(c) and (f)]. In the norm-based codes, adaptation to a biased stimulus renormalizes the responses so that the norm shifts toward the adapting level [(d) and (e)], while adapting to the norm itself leaves the neutral point unaltered. Thus in the norm-based models there is an asymmetry between adaptation to the norm and any other stimulus level: adapting to the norm should not bias the perceived level of other stimuli, while adapting to these stimuli does bias the norm.

An alternative to these norm-based codes is central-tendency codes, in which the stimulus is represented by the distribution of activity within a subset of narrowly tuned channels that span the dimension [Fig. 1(c)]. Such models have been used to account for dimensions like size or spatial frequency, which do not have a unique norm and for which there is no stimulus level that leads to a qualitatively unique set of responses [7]. However, note that this distinction is based as much on the stimulus as on the visual code. If the stimulus is not punctate but instead is broadband—a pattern more typical of natural stimulation—these central-tendency models again include a norm when the responses across the channels are equal. Note also that the perceptual norm for natural stimuli may often correspond to a physically biased stimulus. For example, natural images have more energy at large spatial scales, yet the sensitivity of visual channels tuned to different scales may be weighted to compensate for this bias, so that for a natural spectrum the probability of responses across the set of channels is equated [8]. Thus the norm depends on the match between the stimulus and the channel responses.

While such models are now common in vision and can account for the special subjective nature of perceptual norms, there remains little evidence that stimuli that appear neutral reflect a unique response state. In fact in color vision—where the basis for norms has been most explicitly developed—mounting evidence has pointed against this. Conventional models of color appearance assume that the unique hues (e.g., pure yellow or blue) are perceived when the response within one of the opponent channels (e.g., red versus green) is nulled [6]. However, measures of the actual spectral sensitivities of color-opponent cells in the retina or lateral geniculate nucleus do not correspond to the sensitivities predicted by the unique hues [9], and individual differences in sensitivity fail to account for differences in color naming [10,11]. Even for “gray,” which is assumed to be the null point for all opponent channels, it remains unclear how this null is represented in the visual system. Neurons combine the cone signals with different weights and thus differ in their individual response nulls [12], so that no single stimulus can silence all chromatic mechanisms. More-over, on chromatically biased backgrounds the stimulus that appears achromatic can differ from the stimulus that corresponds to the neutral point for contrast adaptation [13] or induction [14].

Discrepancies of this kind have led to the suggestion that pure or neutral colors might correspond to special states in the environment rather than in the observer [15]. Consistent with this, the axis of unique blue versus yellow falls close to the locus of daylight illuminants and thus may reflect a learned characteristic of the environment rather than a specific response pattern within the mechanisms encoding color [16]. Thus subjective norms might reflect learned criteria rather than innate coding characteristics, and the role of these criterion effects remains central in the debate over cultural versus biological determinants in color naming [17,18].

Here we explore the relationship between perceptual norms and response norms, by taking advantage of the fact that the mapping between stimuli and appearance is highly adaptable, and illustrate the implication of these adaptation effects for judgments of white under different states of chromatic adaptation. Brief exposures to a stimulus can induce large negative aftereffects [19]. Thus after viewing a red field, all colors appear greener and vice versa. By titrating between the red and green, we determined the adapting level that does not induce an after-effect and thus does not bias the observer’s white setting. This level presumably reflects the underlying neutral point for which responses are already calibrated. In the following discussion, we operationally define this stimulus as the “response norm” and ask whether this response norm coincides with the “perceptual norm”—the stimulus that appears white as reported by the observer.

Such comparisons become more powerful when used to analyze the differences in perceptual norms between observers. The stimulus that appears white or a particular hue varies widely between individuals, yet the bases for these differences remain uncertain [18]. Figure 2 illustrates how adaptation should bias achromatic settings in two observers who choose different whites (e.g., one redder than the other), because they differ either after [Fig. 2(a)] or before [Fig. 2(b)] the visual level affected by the adaptation. In the former case, the two observers have the same underlying response norm, and adaptation will therefore bias their settings in the same way. Thus adaptation will not reduce the subjective differences between them and the neutral adapting level will not predict their chosen settings. In the latter case the observers differ in their response norms. The neutral points for adaptation therefore differ and occur at their perceptual null, and adapting to any stimulus should collapse their settings toward a common value. Thus if the differences in appearance are due to differences in response norms (at the level affected by the adaptation) then perceptual nulls will be correlated with response nulls, and adaptation to a common external stimulus will reduce the differences between observers’ perceptual nulls.

Fig. 2.

(Color online) Changes in perceptual norms following adaptation. Panels show predictions for two observers (S1 and S2) who select different norms because they have (a) the same sensitivity but different criteria or (b) the same criteria but different sensitivities. Note that a criterion difference cannot be distinguished from a sensitivity difference subsequent to the site of the mechanisms affected by the adaptation. In the first case, adaptation will shift their norms in the same way and thus the criterion differences persist as the adapting level varies (solid lines). In the second case, adapting to the common stimulus will renormalize both observers to a common value so that their norms converge.

The same comparisons can also be applied to analyze how norms vary within observers. Light reaching the retina is screened by the lens and macular pigments, which selectively absorb short wavelengths. The macular pigment is concentrated in the fovea and falls rapidly with eccentricity, leading to differences in spectral sensitivity across the retina [20]. The density of lens pigment increases with age, leading to differences over time. Despite these factors, the stimulus that appears white remains similar in the fovea and periphery [21] and in young and old observers [22,23], so that color appearance is compensated for the variation in sensitivity. The basis for this compensation is unknown but could arise at several stages in the visual pathway, including gain changes in the cones [22], changes in the weights of the cone inputs to opponent sites [24–26], or changes in criterion (Fig. 3). An advantage of using adaptation to probe response norms is that adapting stimuli can be chosen to alter visual coding at different sites [19]. Here we used chromatic adaptation to uniform fields, which induces response changes that are largely (though not necessarily completely) independent within each cone class [27–32] and thus are thought to tap primarily an early retinal stage (see Discussion). However, in this study our principal aim was not to determine the sites of adaptation but rather to test whether a neutral response state as measured by adaptation can predict the stimuli that “look” normal to observers. Our results suggest that both between and within observers, the stimulus that appears white is also very close to the stimulus that leads to a neutral state of chromatic adaptation, thus implying a strong link between perceptual norms and response norms in color coding.

Fig. 3.

Norms in color vision. (a) “White” is assumed to be represented by balanced activity across the S, M, and L cone receptors and by a null response in postreceptoral opponent channels. Responses to a biased spectrum could be renormalized by compensatory adjustments (b) in the sensitivity of the receptors, (c) in the strength of inputs to the opponent channels, or (d) by changing the criterion for white.

2. METHODS

The stimulus was a 2 deg circular field displayed on a SONY 20SE monitor controlled by a Cambridge Research Systems VSG graphics card. The field had a luminance of 25 cd/m² and was shown on a black background. Observers viewed the display binocularly in a dark room and through a hood that screened extraneous light. Participants included the authors and nine naïve observers. All had normal color vision as assessed by standard tests. Testing protocols were approved by the University of Nevada Institutional Review Board, and participation was with informed consent.

White settings were made following procedures similar to those used by Delahunt et al. [24] to track achromatic loci following cataract surgery. Observers adjusted the color of the field until it appeared achromatic by using a pair of buttons to vary the chromaticity along the two axes of the CIE 1976 u′v′ uniform color space, chosen so that steps in the color were roughly equated perceptually. The settings were made while looking directly at the target or with the field shown at an eccentricity of 8 deg, for which a dim gray fixation cross was added. Each trial began with 2 min adaptation to the black background or to a 2 deg adapting field. The test field was then shown at a random starting chromaticity for 0.5 s followed by 3 s re-adaptation, with a 0.2 s dark gap between the test and adapt. This cycle continued until six successive selections of the white point were completed.

During a session, participants first made settings on the dark background to identify their dark-adapted perceptual norm (i.e., the white chosen in the absence of an external adapting color). Settings were then repeated after adapting to different chromaticities in the field to determine the adapting level that did not bias their dark-adapted settings. Adapting chromaticities were varied along the two axes of the MacLeod–Boynton color space [33], which vary signals in S cones at constant luminance (SvsLM) or in the ratio of L and M cone signals at constant luminance (LvsM). These variations correspond to the cardinal axes of early postreceptoral color vision [9]. The axes were scaled to roughly equate sensitivity to the two dimensions and were varied relative to an equal-energy white. The units for LvsM and SvsLM axes in the present study are related to the r, b axes of the MacLeod–Boynton diagram by

$$L\upsilon sM=2754*(r-0.6657)\text{and}$$

$$S\upsilon sLM=4099*(b-0.01545)$$

based on prior measurements of contrast thresholds for signals along the two axes [34].

Adapting levels along each axis were titrated over a range of ±80 relative to the equal energy white point. In a given session observers were tested on all levels of either the SvsLM or the LvsM axis for either central or peripheral viewing. For six of the observers, in subsequent sessions we obtained estimates of the differences in spectral sensitivity at the 0 and 8 deg viewing angles with a minimum-motion task [35]. In this case the stimulus was a square 2 deg field displaying a 1 cycle/deg squarewave drifted at 2.5 Hz. The chromaticities of the grating bars varied between ±80 units along the SvsLM axis, while their relative luminance was varied in a two-alternative forced-choice (2AFC) staircase to determine the motion null. Differences in the nulling luminances at the two visual loci were fitted with a template for macular pigment to estimate the density difference at the two locations [21,36].

3. RESULTS

As noted, in order to compare the perceptual norm and the response norm for white, we first measured the observer’s achromatic setting while they were dark adapted, and then remeasured the setting while they were adapted to fields with chromaticities that varied along the LvsM or SvsLM axes. Figure 4 shows an example of the white settings for one observer under different adapted states and compares these for the fovea and periphery. The stimulus that appears achromatic remains very similar at the two retinal loci, confirming previous findings that color appearance is largely compensated for spatial variations in spectral sensitivity [21,26]. For this observer the peak macular pigment density difference was estimated to be 0.46, comparable to average estimates [37]. If her settings were not corrected for the difference in macular pigment at the two locations, then a stimulus that appeared white in the fovea would look blue in the periphery, and thus her achromatic point would be shifted toward yellow. The degree of this shift is shown by the arrow in the panel and is many times larger than the difference in her actual settings.

Fig. 4.

(Color online) (a) achromatic settings for a single observer (MY) before (solid symbols) or after (open) adapting to chromaticities (crosses) that varied along the LvsM or SvsLM axes of the color space [shown in panel (b)]. Circles plot the settings for centrally fixated colors, while triangles show settings at the 8 deg eccentricity. Arrow and diamond show the shift in the 8 deg settings predicted by the difference in macular pigment at the two locations. (c) Level along the LvsM axis that appeared neutral as a function of the LvsM adapting level for colors viewed in the fovea (solid circles) or at 8 deg (triangles). Lines through the measured points are the best-fitting fourth-order polynomials. Dashed lines show the dark-adapted settings and the peripheral setting predicted by the macular pigment difference. Response norms were determined by estimating which adaptation level produced the same achromatic response as the dark-adapted perceptual norm. (d) Similar results for the SvsLM axis.

Adaptation strongly biased the white settings, but again these biases remained similar in the fovea and periphery. This would not occur if the compensation happened at sites subsequent to the stages at which chromatic adaptation altered the response. In that case the adaptation would be driven by the same effective quantal catches in the foveal and peripheral cones (with the neutral points for the adaptation again shifted by the amount shown by the arrow). That is, if adaptation in the fovea and periphery were not already compensated for the differences in spectral sensitivity, then equivalent aftereffects should have occurred for equivalent cone excitations (whether the site of the adaptation were in the cones themselves or downstream). Yet the observed results suggest that the adaptation is instead equated for the same physical stimulus.

We fitted polynomials to the changes in the white settings as a function of the adapting level to estimate the neutral adapting point that did not bias the dark-adapted settings. The fits were done separately for the LvsM and SvsLM adapting axes and are illustrated for observer MY in Fig. 4(c) and 4(d). Again these show that adaptation altered color appearance in very similar ways in the fovea and periphery and that the neutral points (or response norms) for chromatic adaptation remained very similar to her dark-adapted achromatic settings (or perceptual norms).

Figure 5 compares these response norms to the perceptual norms for white for all of the observers. The perceived achromatic points did not significantly differ between the fovea and periphery [Fig. 5(a)]. Response norms at the two loci differed along the SvsLM axis [t(10)=4.7; p=0.0008] but not along the LvsM axis. Finally, the response norms differed from the perceptual norms in the fovea [t(10)=3.33, p=0.008 for SvsLM; t(10)=3.75, p=0.0038 for LvsM] but were not significantly different for either chromatic axis at 8 deg. The significant differences suggest a possible residual influence on the white settings at sites subsequent to the adaptation. However, these differences were very small compared with the pronounced shifts expected from the differences in spectral sensitivity at the two loci. These differences are shown by the spread of connected diamonds in Fig 5(a) and Fig 5(b), which plot the range of achromatic settings predicted if there were no compensation for the measured differences in macular pigment density between the fovea and periphery. Across the set of observers tested, the density difference ranged from ~0.1 to 0.8, comparable to previous estimates [26]. The fact that both the perceptual norms and the neutral adapting levels are unaffected by this difference again suggests that most if not all of the compensation for white reflects a response norm at the stages affected by chromatic adaptation.

Fig. 5.

(a) Perceptual nulls and (b) response nulls for all observers. Circles show foveal settings, Triangles the settings at 8 deg. Diamonds show the differences between the central and peripheral settings predicted by the macular pigment densities estimated for individual observers (with peak densities noted next to the symbols). (c) Comparison of individual perceptual norms and response norms along the LvsM axis. (d) Similar results for the norm levels along the SvsLM axis.

Individual differences in the stimulus chosen for white tend to vary along a blue–yellow axis [22,38] [Fig. 5(a)]. If these appearance differences reflect differences in how chromatic responses are normalized (at the sites affected by adaptation) then they should covary with the observers’ neutral adapting levels. Consistent with this, the perceptual and response norms were significantly correlated along both the LvsM (r=0.72, p<0.00018) and SvsLM (r =0.59, p<0.0036) axes [Fig 5(c) and Fig5(d)].

As illustrated in Fig. 2, if the interobserver differences are in fact due to differences in response norms, then a second prediction is that differences between observers’ perceptual norms should be reduced in the presence of a common adapting stimulus, since this should renormalize all observers’ responses to the same level. To assess this, we compared the variance in white settings made in the dark or after adapting to the center chromaticity (equivalent to equal energy white). Interobserver variance was ~4× lower when setting white under the common adapter (Table 1). Again this suggests that much of the perceptual difference between observers resulted from differences in their underlying response norms. Notably, exposure to the more biased adapting levels (greater than ±20 units from the nominal white) did not significantly reduce the range of white settings relative to the dark-adapted settings. This could in part reflect individual differences in the strength of adaptation, and consistent with this, observers’ settings varied in the slopes relating the size of the aftereffect to adapting level. However, we cannot exclude other factors that affect this strength such as the ability to maintain fixation during the adaptation.

Table 1.

Comparisons of Interobserver Variations in White Settings Measured during Dark Adaptation (Pre) or after Adapting to a Common Stimulus with the Chromaticity of an Equal-Energy Spectrum (Post)^a

	LvsM		SvsLM
	Pre	Post	Pre	Post
Fovea
Variance	203.5	37.2	127.5	21.6
F(1,21)	3.46	—	5.90	—
p	0.00016	—	0.000087	—
Periphery
Variance	80.75	25.1	145.5	17.5
F(1,21)	3.22	—	8.31	—
p	0.0055	—	0.000005	—

^aVariances were compared separately for the settings in the fovea and periphery and for the values along the LvsM axis or SvsLM axis.

4. DISCUSSION

Models of color appearance have attempted to explain the psychological structure of color sensations directly from the functional structure of neural responses [39,40], yet the links between the phenomenology of perception and the mechanisms of visual coding remain very poorly understood. We examined a central assumption in this link by asking whether stimuli that appear psychologically neutral reflect a neutral response state in visual mechanisms, and our results suggest a clear relationship between the perceptual norm for color and the mechanisms that adapt to color. This is supported by our findings that (1) the perceptual norm for white is close to the response norm as defined by the stimulus that leaves the underlying state of adaptation in balance; (2) the similarities in perceptual norms in the fovea and periphery are mirrored by similar response norms for adaptation; (3) individual differences in the perceptual norms are correlated with observers’ response norms; and (4) these individual differences are reduced in the presence of a common external adapting stimulus near nominal white. While we examined these relationships for the specific case of color, the tests we used are general, and similar tests could be applied to many other perceptual dimensions to explore the nature of norm-based codes and the role they may play in both neural coding and visual experience. For example, norms in face perception are biased by adaptation in ways functionally similar to the case for color vision [4,41,42], and recent evidence suggests that faces closer to the norm evoke weaker neural responses consistent with a null [43,44] (though see [45]).

Adaptation has frequently been called the “psychologist’s microelectrode,” and one reason is that different adapting stimuli can be used to isolate different stages of the visual pathway. For example, varying the spatial and temporal properties of the adapting color has provided evidence for sensitivity changes from the receptors to “second-stage” postreceptoral sites to higher-order cortical mechanisms [19]. This allows the possibility of probing response changes at different stages to determine the sites at which subjective norms are ultimately defined. Our results do not themselves test the site of chromatic adaptation and were instead aimed at testing whether the perceptual norm for color is consistent with the response norm determined by adaptation (at whatever site that adaptation might occur). The extent to which cone receptors might adapt has been uncertain (e.g., [46,47]). However, several lines of evidence suggest that much of the adaptation to steady uniform fields involves sensitivity changes in the receptors. First, as noted above, the sensitivity and appearance changes resulting from adaptation to steady uniform fields occur largely independently within different classes of cones [27–32]. Second, the characteristic form of the sensitivity changes and response dynamics can be tied to specific biophysical processes within the photoreceptors [48]. Third, the spatial pooling of signals controlling the adaptation can be as small as a single cone [49,50]. And finally, recordings in primates point to strong cone-specific adaptation before summation in horizontal cells [51].

While this evidence does not preclude additional response changes from chromatic adaptation at subsequent sites—which are also well established (e.g., [52,53])—it suggests that the bulk of the adaptation effects we observed may occur very early in the visual pathway and as early as the receptors. In turn, the fact that for color the perceptual norm is consistent with the response changes produced by steady chromatic adaptation suggests that “white” is established at a very early stage in the visual system. This could occur if the intrinsic gains in the receptors are matched by long-term adaptation to the average spectral stimulus in the environment [22,54], though differences between luminance and chromatic adaptation suggest that at least part of the sensitivity adjustments may be after separate cone inputs to luminance and chromatic pathways are established (e.g., [55,56]). The stability of white across retinal location requires compensatory changes in the signals from all three cone classes. This is consistent with other studies that have found nearly complete compensation of white for differences in preretinal filtering [21,22,24], yet contrasts with hue cancellation studies that have instead pointed to compensation only along the yellow–blue axis of color appearance [26,57]. The basis for this difference remains unresolved.

Adaptation can operate over multiple time scales [58–60], and several studies have documented very long-term adaptation in color coding [24,25,61]. Our analysis implies at least two distinct time scales for chromatic adaptation: (1) a short-term response change that adjusts to the current stimulus and that is revealed by the brief color aftereffects we measured and (2) long-term changes that are best seen in the absence of an extrinsic adapting stimulus (e.g., by measuring the dark-adapted white point) and are implicated by the fact that the response norms are compensated for the differences in spectral screening between the fovea and periphery. What purpose might these long-term adjustments serve? Rapid adaptation is thought to be important for avoiding response saturation and for tying the average response to the average stimulus level in the environment [62]. However, this “average response” is likely to differ across mechanisms or retinal loci because of intrinsic morphological or physiological differences. A long-term adaptation that calibrates the intrinsic responsiveness of the neurons may be important to help equate their relative inputs into subsequent stages (e.g., for spectral or spatial opponency) or to normalize for differences across the retina. A second possible function might be to optimize the dynamics of short-term adaptation by keeping the intrinsic sensitivity of the neuron tied to the expected mean of the environment.

Adaptation is not only a tool for examining norms but may be intimately connected to them, for a primary function of the adaptation may be to establish and maintain norms in the face of changes in the environment or the observer. This renormalization could underlie perceptual constancy despite the large optical and neural changes that occur during development and aging or—as we have shown here—between different parts of the visual field. The same process may also lead to perceptual constancy between observers, discounting individual differences in visual sensitivity by normalizing them to the same properties of their environment [60]. Consistent with this, we found that adaptation to a common stimulus substantially increased the level of perceptual agreement between observers. It may generally be the case that normalization to a common visual world is what allows individuals to have shared perceptual experiences.