No Measured Effect of a Familiar Contextual Object on Color Constancy

Abstract

Some familiar objects have a typical color, such as the yellow of a banana. The presence of such objects in a scene is a potential cue to the scene illumination, since the light reflected from them should on average be consistent with their typical surface reflectance. Although there are many studies on how the identity of an object affects how its color is perceived, little is known about whether the presence of a familiar object in a scene helps the visual system stabilize the color appearance of other objects with respect to changes in illumination. We used a successive color matching procedure in three experiments designed to address this question. Across the experiments we studied a total of 6 subjects (2 in Experiment 1, 3 in Experiment 2, and 4 in Experiment 3) with partial overlap of subjects between experiments. We compared measured color constancy across conditions in which a familiar object cue to the illuminant was available with conditions in which such a cue was not present. Overall, our results do not reveal a reliable improvement in color constancy with the addition of a familiar object to a scene. An analysis of the experimental power of our data suggests that if there is such an effect, it is small: less than approximately a change of 0.09 in a constancy index where an absence of constancy corresponds to an index value of 0 and perfect constancy corresponds to an index value of 1.

1. Introduction

Some familiar objects have a typical color, for instance the yellow of a banana or the red of a stop sign. Such ‘memory colors’ can be recalled and may influence perception.¹ Indeed, memory for the color of familiar objects and its possible influence on perception has been studied extensively^2–17, with the results of interest not only for basic vision science but also for applications in color reproduction.

There is evidence that, under some circumstances, recognition of an object via its shape can have a small influence on its perceived color. Duncker, for example, found that subjects percieved a piece of paper as greener when it was cut in the shape of a leaf than when it was cut in the shape of a donkey.³ Bruner et al.’s early work also suggested that knowledge of an object’s typical color had an effect on its perceived color.⁴ On the other hand, Bolles et al. found memory color effects only when the experimental color matching procedure prohibited subjects from making a satisfactory match, and also found substantial individual differences in memory color effects.⁵

Memory color effects have been replicated and extended more recently, particularly under conditions where the variation in the illumination introduces ambiguity about the mapping between reflected light and object surface reflectance.^13,14 For example, Hansen et al. showed that the achromatic color of a stimulus whose shape is that of a familiar object can be biased in a manner that suggests that the familiar object shape makes a contribution to color appearance.¹³ As with the earlier work, the effects are delicate: Olkkonen et al. showed that the effect of familiar object color depends on the degree of naturalness of the object shapes used in the experiment.¹⁴ Granzier and Gegenfurtner¹⁷ as well as Smet et al.¹¹ include current reviews of the effect of familiar object color on judged color appearance.

The reader may find it surprising that object identity could have an effect on perceived object color, since it would naively seem that the spectrum of reflected light would uniquely determine the object’s color appearance. In fact, the spectrum of light reflected to the eye depends both on the object’s reflectance properties and on the spectrum of the illumination incident on the object. Because of this, the reflected light is ambiguous about object reflectance. For color appearance to be a useful guide to object identity, the visual system must separate the influence of the illuminant and the surface reflectance on the reflected light. The fact that the visual system does this, so as to deliver an object color percept that is relatively stable across changes of illumination is called color constancy; it is well-established that under many circumstances the visual system exhibits excellent color constancy.^18–21 Since the spectrum reflected to the eye is ambiguous about object reflectance, the visual system may take advantage of any available cues to help interpret the retinal image,^22–24 and familiar object color may be one such cue.^14–17 The presence of familiar objects in a scene is a potential cue to the illumination, since the light reflected from a familiar object should on average be consistent with their typical surface reflectance.

The question of whether familiar object color can aid in color constancy has applications in digital photography and implications for color rendering, as well as being of interest to the understanding of human color vision. For example, an estimate of the illuminant can be used to white balance images, and engineers have tried to capitalize on familiar object color to improve white balancing. Bae et al. reported that adjusting R and B gains so that the average chromaticity of faces matched that of standard faces yielded good white balanced images.²⁵

For the present study, we investigated the extent to which adding a familiar object to a scene improves human color constancy for other objects in the scene. Indeed, to separate this question from direct effects of familiarity on the appearance of an object, we studied indirect effects. In our experiments, we compared color constancy for a test patch, with and without a familiar object cue to the illuminant present in the image. To our knowledge, only two previous studies have examined this type of indirect potential effect of familiar object color. Ling reports that the presence of a familiar object in a scene can affect the appearance of other objects in the scene, but this effect did not lead to improved constancy.¹⁵ Granzier and Gegenfurtner report a small but reliable improvement in constancy with the addition of a familiar object to a scene.¹⁷ We return in the discussion to compare our study with this prior work.

2. General Methodology

2.1. Basic Task

We conducted 3 experiments. The basic task used in the experiments was successive color matching. The use of successive color matching allows us to control the adaptation level of the subject across a change in simulated illuminant more fully than is possible with simultaneous matching methods. This can be important, since chromatic adaptation can be slow relative to the duration of within-image fixations.^26,27

In our experiments, subjects adjusted the color of a square test patch on the right (or left) side of a computer controlled display so that its color appearance matched the color of a square reference patch presented on the left (or right) side of the display (Figure 1). The two patches did not appear at the same time, however. Rather, the test patch appeared after a retention interval. Across the experiments, subjects controlled the hue and chroma of the test patch using a joystick control (Figure 2). The test patch luminance was held fixed and was the same as that of the reference patch. Reducing the number of degrees of freedom in the adjustment from three to two simplified the task for the subject. Pilot data collected with the first author (EK) as subject, where test luminance could also be adjusted, indicated that any luminance effects were small.

Figure 1.

Illustration of task components for successive matches. (1) At the start of a trial, the observer adapted to the reference viewing condition. We call this the adaptation period. On some trials, a banana or a rectangle was presented during the adaptation period and continued to be present during the presentation of the reference patch. (2) After the adaptation period, the reference patch was presented on the left (or right) of the display. Here the reference patch is the green square. A thin black square enclosed the reference patch and any adjacent object/patch, and the observer was encouraged to scan all stimuli within the square. (3) After presentation of the reference patch, the retention interval began. The background could stay constant or could change at the start of the retention interval. In either case, the observer viewed the test background during the retention interval. If the banana or the rectangle had been present, it was removed at the start of the retention interval. (4) Finally, the test patch appeared and the observer adjusted it to match the reference patch.

Figure 2.

Schematic of adjustment controls. The observer initiated a series trial by pushing the start button. A joystick was used to control the CIELAB hue angle and chroma of the test patch. Two buttons allowed increase or decrease of the step size of each adjustment. When the match was complete, the observer pressed the lower right button.

2.2. Stimuli

Equipment

The experiment was conducted in a dark room using a computer-controlled 21” CRT display (Model G220fb, ViewSonic Corporation) with a Bits++ box (Cambridge Research Systems Ltd.) The host computer was an Apple Macintosh (2.66 GHz Xenon quad processor with NVIDIA GeForce GT 120 graphics card). A spectroradiometer (PR-650, Photo Research Inc.) was used to measure the spectral radiance characteristics of the three monitor primaries as well as the gamma functions for each primary. This information was used in conjunction with standard methods²⁸ to produce stimuli with desired tristimulus coordinates. The monitor had a resolution of 1024 by 768 pixels, corresponding to a display size of 372 by 279 mm, and was refreshed at 85 Hz. It was viewed from a distance of 550 mm. Subject position was stabilized with a chin/forehead rest.

CIELAB specification

It is convenient to use the CIELAB color space to specify some aspects of our stimuli, as this space provides a degree of perceptual uniformity. All conversions from CIE XYZ to CIELAB for stimulus/data specification in this paper were performed using a D65 reference white with XYZ tristimulus coordinates (82.4, 86.7, 94.4), where luminance Y is in units of cd/m². We fixed the choice of reference white so that our CIELAB stimulus/data representation would be in one-to-one correspondence with XYZ coordinates. That is, we did not incorporate any adaptation model into our data representation.

Reference and test patches

The size of the reference and test patches was 2.9° by 2.9°. The patches were centered vertically on the monitor and separated by 19° horizontally. There were six reference patches for Experiment 1 and 2 and five reference patches for Experiment 3. Details of the reference patch colors are described separately for each experiment below. On each adjustment, the test patch was initialized with the same CIELAB L* value as the reference patch (L*₀), and the CIELAB hue and chroma were selected randomly from uniform distributions. The range of the hue distribution was from 0 to 2π radians, and the range of the CIELAB chroma distribution was from C*₀/2 to C*₀, where C*₀ is the CIELAB chroma of the reference patch.

Backgrounds

In Experiments 1 and 2, the reference and test patches were both presented against backgrounds, which consisted of small colored squares (see Figure 1). The reason that checkered rather than spatially uniform backgrounds were used is that in pilot observations where a uniform background was employed, salient afterimages of the reference and test stimuli were apparent to subjects as their eyes moved across the display. The use of checkered backgrounds reduced the visual salience of these afterimages. Two backgrounds were used in each of Experiments 1 and 2. The D65 background had mean CIE xy chromaticity equal to that of CIE illuminant D65 (0.31, 0.33) and was used in both experiments. In Experiment 1, the second background had chromaticity equal to that of CIE illuminant A (0.45, 0.41); in Experiment 2, the second background had chromaticity equal to that of a 4000 K blackbody illuminant (0.38, 0.38).

In Experiment 3, the background for the reference viewing condition was black, and only the test viewing condition had a checkered background, either D65 or 4000 K.

The mean luminance of all backgrounds was 15.6 cd/m². Each small background square occupied 0.4° by 0.4° of visual angle. The lightness and chromaticity of the individual background squares were jittered around the mean values by drawing random values from a spherical normal distribution in the CIELAB color space with a standard deviation of 10 ΔE units.

For some calculations, we associated each background with the spectral power distribution of its corresponding illuminant.

Additional details about the pairing of backgrounds with the reference and test patches are provided in separately for each experiment below.

Contextual objects

The key experimental manipulation was whether a familiar object was present while the subject viewed the reference patch. We used three contextual object conditions.

In the first context condition, an image of a banana was presented below the reference patch. We call this the Banana context condition. The image of the banana was 11.9° by 6.5°. The banana’s body was centered horizontally with the reference patch and located 1.1° below the reference patch. The image of the banana was obtained from a digital photograph of an actual banana, and the color on the banana was simulated as if it were under the background’s illuminant (D65, CIE illuminant A or 4000 K illuminant). The details of the image acquisition process and its colorimetric calibration are provided below. The presence of the banana had a small effect on the spatial mean luminance and chromaticity of the backgrounds. The difference of the spatial mean luminance and chromaticity of the background between the “Absence” contextual object and “Banana” contextual object was (ΔY [cd/m²], Δx, Δy) = (0.8, 0.001, 0.002) for D65, and (0.7, −0.0002, 0.001) for illuminant A.

In the second context condition, a spatially uniform yellow rectangle was displayed adjacent to the reference patch. We call this the Rectangle context condition. The color of this rectangle was chosen to match that of the banana under each illuminant. Since the banana is spatially inhomogeneous, we determined the color of the rectangle using a matching procedure. The two authors (EK and DHB) set simultaneous color matches between a uniform rectangle and the displayed banana, under each illuminant. We then used the average of these matches as the rectangle color during the main experiment with naïve subjects. The L*a*b* coordinates (computed using D65 as the white reference for the transformation) of the rectangle were (82.5, −0.8, 51.5) for the D65 illuminant background and (83.0, 13.4, 69.7) for 4000 K illuminant background. The rectangle was placed so that its centroid was identical to that of the banana.

In the third contextual object condition, the reference patch was presented against the background without any other objects present. We call the Absence context condition.

Experiment 1 employed the Banana and Absence contextual object conditions, Experiments 2 and 3 employed all three contextual object conditions.

2.3. Successive Color Matches

The experimental procedure (again refer to Figure 1) was as follows:

1. The subject pre-adapted to the reference background (15 seconds). For Banana and Rectangle contextual object condition trials, the banana/rectangle was present during this time.

2. The reference patch appeared on the left (or right) of the display. On trials where a contextual object was present (banana/rectangle), it remained in view while the reference patch was presented; on Absence contextual object condition trials, no contextual object was present.

3. The subject memorized the reference color for 10 seconds. During this period, a flashing square outline (black line outlining a 14.8° by 14.8° square, flashing at 2 Hz) appeared to encourage the subject to look at the entire region. After 10 seconds, the reference patch and the context if it was in view disappeared.

4. The subject adapted to the test background for 15 seconds. No contextual objects were present during this period.

5. The test patch appeared on the right (or left) of the display and the subject adjusted this to match the remembered color appearance of the reference patch. No contextual objects were present during this period

The precise instructions provided to subjects can under some circumstances have a substantial influence on their asymmetric matches, although neither the nature of such instructional effects nor when they occur is well-understood.^29–44 The text of the instructions given to subjects in our experiments is provided in the supplemental material available at http://color.psych.upenn.edu/supplements/constancyfamiliar. Since we wanted to study the implicit effect of the contextual objects on color appearance, without prompting subjects to reason explicitly about how an illuminant change might affect the color appearance of a familiar object, these instructions simply asked observers to match ‘color’ and did not bias them either towards or away from explicitly judging the stimuli as illuminated surfaces.

Practice matches were set before main data collection began. These were done with three reference patch colors that differed from those used in the main experiment.

2.4. Object Image Acquisition and Calibration

To obtain banana images to use in the experiment, we used a calibrated digital camera (Nikon D70) to acquire an image of an actual banana. Image processing was then used to simulate images of the banana under different illuminants. We used a single natural daylight for image capture and simulated illumination changes, because we did not have an illumination controlled viewing booth available that could produce spectra typical of natural daylights.

The following steps were followed for image acquisition and processing.

1. We acquired an image of a banana under natural daylight. The acquired image was stored in raw (Nikon NEF) format. The raw RGB values were extracted and demosaiced using the methods described by Tkacik et al.⁴⁵ The RGB values were linear in image intensity.⁴⁵

2. The pixels corresponding to the banana were identified via hand segmentation performed using Adobe Photoshop.

3. We measured the spectrum of the acquisition illuminant. This was done by measuring the light reflected from a white reference standard using a spectroradiometer (PR-650, Photo Research Inc.)

4. We had measurements of the red, green, and blue spectral sensitivities of the camera.^45* Using these, we calculated the RGB values predicted for the 24 patches of the Macbeth color checker chart (MCC) under the acquisition illuminant, based on reflectance measurements for each of the 24 MCC patches.

5. For each rendering illuminant, we calculated the XYZ coordinates for the same 24 MCC patches. We used numerical parameter search to find the 3×3 transformation matrix that mapped acquired camera RGB values under the acquisition illuminant to XYZ coordinates under the rendering illuminant. For this parameter search, the error that was minimized was the mean CIELAB ΔE difference between desired and transformed XYZ coordinates for the 24 MCC patches. In converting between XYZ and CIELAB for purposes of optimizing the transformation matrix, we used the XYZ coordinates of the rendering illuminant as the white point. We computed these transformation matrices separately for each rendering illuminant.

6. We applied the transformation matrix to the raw RGB values of the banana image to obtain a rendered image for each rendering illuminant.

7. For display on the experimental monitor, the XYZ coordinates were converted to RGB values based on our monitor calibration measurements. A single overall factor was chosen so that the maximum of the resulting RGB values of the banana image was just inside the monitor’s gamut.

Our procedure for obtaining color transformations optimizes the transformation for the set of reflectances represented by the MCC. To verify that these were sufficiently representative of the reflectance of bananas, we measured the spectrum of the light reflected from five locations on two bananas as well as from a reflectance standard. We then computed the underlying spectral reflectance at the five locations. Using these, we calculated RGB values and XYZ coordinates for these five reflectances for both illuminant D65 and CIE illuminant A and applied the corresponding color transformation matrices to the calculated RGB values. We computed the CIELAB ΔE difference between the transformed and directly calculated XYZ coordinates and verified that these were small (average ΔE = 1.44 for D65 and ΔE = 1.04 for illuminant A).

Figure 3 shows an acquired banana image, but rendered according to our procedures under CIE illuminant D65. The figure also shows the result of the segmentation procedure.

Figure 3.

Banana image processing. The left image shows the acquired image of a banana, transformed to be rendered under illuminant D65 as described in the text. In the right image, the pixels corresponding to the banana have been extracted and inserted against a neutral background. The same insertion procedure was used to place the banana images against the experimental backgrounds.

2.5. Predictions of perfect color constancy

It is useful to predict how subject matches would shift for a color constant visual system. To make these predictions, we used the general methods described by Brainard et al.⁴⁶ Specifically, for each reference surface, we used a three-dimensional linear model for surface reflectance to convert the XYZ coordinates of the reference patch and the illuminant spectral power distribution associated with the reference background to find a surface reflectance function that yielded the reference XYZ coordinates when rendered under that illuminant. For this purpose, the three dimensional linear model was obtained via analysis of measurements of the reflectance functions of the Munsell papers.⁴⁷ Given the surface reflectance function, we then computed the XYZ coordinates that the surface would have under the illuminant associated with the test background. We transformed these computed XYZ coordinates to CIELAB, and used the difference between the CIELAB test coordinates and the CIELAB reference coordinates as our prediction of the shift in match that would be shown by a color constant subject.

3. Experiment 1

3.1. Specific Methods

Two reference backgrounds, D65 and CIE illuminant A, were used in Experiment 1. The reference patch always appeared in the left on the display, while the test patch appeared in the right. Six reference patch colors used in the Experiment 1 had CIELAB luminance L*₀ = 78 (46 cd/m²) and CIELAB chroma C*₀ = 25. The CIELAB hue angles of the reference patch colors were (in radians) π/6, 3π/6, 5π/6, 7π/6, 9π/6 and 11π/6. The CIE xy chromaticity of these stimuli were (0.37, 0.34), (0.36, 0.38), (0.30, 0.38), (0.25, 0.32), (0.26, 0.28), and (0.32, 0.29) respectively. Data were collected for both the Banana and Absence contextual object conditions.

There was a single test background, D65.

Two naïve subjects (S2 and S3, both female) participated in Experiment 1. Prior to the start of this experiment, these subjects also participated in a preliminary simultaneous matching experiment in which there was no illumination change. A third naïve female subject (S1) also participated in the simultaneous matching experiment, but was not selected to continue for the successive matching experiment because her match variability in the simultaneous matching experiment was high. All subjects had normal color vision as assessed with the Ishihara plates, and normal or corrected visual acuity of 20/40 or better.

Four replications were made for each reference color/background/contextual object, for a total of 96 (4*6*2*2) matches per subject.

3.2. Results

3.2.1. Memory Color Shift

Figure 4 shows the memory matches for each subject for the Absence contextual object condition. There are two primary effects visible in the data. First, the memory matches for the D65 reference conditions differ from the veridical reference colors. This is indicated by the deviations between the filled circles (matches) and x’s (reference colors). In this paper, we do not focus on these deviations. Second, changing the illuminant in the reference viewing condition affects the matches. This is shown by the blue arrows that extend from the filled circles (memory matches for the D65 reference condition). The individual matches for each subject and condition for this and the other experiments reported in this paper are provided in the supplemental material available at http://color.psych.upenn.edu/supplements/constancyfamiliar.

Figure 4.

Memory matching data, Experiment 1. The plots show the basic results for two observers (S2 left panel; S3 right panel) for the Absence contextual condition. The CIELAB a* b* coordinates of the reference colors are indicated by x’s. Filled circles show the mean memory matches for the D65 reference background condition. Deviations between x’s and filled circles indicate an effect of memory, with no illuminant change. Open circles show the mean memory matches for the CIE illuminant A reference background condition. Thus the blue arrows indicate the effect of changing reference illumination, once the shift introduced by memory has been accounted for. For comparison, the dashed black arrows show how the data would shift for a color constant system. These shifts were computed from the reference color coordinates, as described in Methods, and scaled to 1/5 the vector length of the shift that would be shown by a color constant system. To ease comparison with the data, the shifts are plotted with their origins at the match for the D65 background, rather than at the reference color itself. The error bars show +/− 1 SEM, taken over replications.

The effect of reference illuminant on the matches can be interpreted in terms of color constancy. Indeed, we computed how the matches would be expected to shift for a color constant visual system (see Methods). This is shown in Figure 4 by the black dashed arrows. The direction of the arrows shows the direction that the memory matches would be expected to shift for a color constant system. The vector length of the arrows indicates 1/5 of the magnitude of the expected shift under color constancy.^† Examination of Figure 4 shows that the shift in both subjects’ memory matches tends in the direction of color constancy but is smaller in magnitude than that which would be shown by a color constant system, observations that we quantify below.

3.2.2 Effect of Banana

Our primary interest was in whether adding a familiar object, the banana, to the reference viewing condition improves constancy. To examine this, we compared the shift in memory matches across reference illuminant for the Absence and Banana contextual object condition. The blue arrows in Figure 5, one for each reference color, re-plot the blue arrows from Figure 4, but with each arrow re-centered on the origin. Thus the cluster of blue arrows provides a visual sense for the overall shift in memory matches for the Absence context. The green arrows in Figure 5 show the corresponding arrows for the Banana context. Greater constancy for the Banana condition would be indicated if the green arrows tend further in the direction of constancy (black dashed arrows) than the blue arrows. For both subjects, there is an improvement in mean constancy for the Banana context compared to the Absence context. The effect for S3, however, is very small. These effects are shown more clearly in Figure 6, in which each arrow plots the average taken over the corresponding arrows from Figure 5.

Figure 5.

Constancy for individual reference colors, Absence and Banana contextual object conditions, Experiment 1. Left graph shows S2’s data and right graph shows S3’s data. Blue and green arrows indicate color constancy effect under Absence and Banana contexts respectively. Black dashed arrows indicate predictions of perfect color constancy, but with vector length scaled by 1/5.

Figure 6.

Mean constancy for Absence and Banana contextual object conditions, Experiment 1. Left graph shows S2’s data and right graph shows S3’s data. Blue and green arrows indicate the mean color constancy effect obtained by averaging the shifts shown in Figure 5 over reference color. To indicate the direction of the constancy prediction, the black dashed arrows show the corresponding average of the constancy predictions from Figure 5. The error bars show +/− 1 SEM, taken over reference colors and replications.

To quantify the effect of adding the Banana, we computed a color constancy index for each individual match set by our subjects. This was done via the equation:

$$\mathit{CCI}=\frac{\mathbf{M}·\mathbf{I}}{{\mid \mathbf{I}\mid }^2}$$ (1)

where M is the observed shift in memory matches between D65 and CIE illuminant A reference viewing conditions (shown as the blue and green arrows in Figure 5) and I is the vector representing the predicted shift in matches for a perfectly color constant observer (shown at 1/5 vector length by the black dashed arrows in Figure 5). This index takes on a value of 1 when the observed shift in memory matches agrees perfectly with the constancy shift prediction, a value of 0 when there is no shift in memory matches, and negative values when the shift in memory matches is in the opposite direction from the constancy shift prediction. We used a bootstrapping procedure to determine the mean constancy index (over reference patch colors and replications) for each context/subject as well as associated 95% and 99% confidence intervals. For each reference color, we computed a constancy index by choosing one individual at random for the D65 illuminant and second individual match at random for the CIE A illuminant. We repeated this for each of the 4 replications for each of the 6 reference colors and then took the mean over these 24 indices to obtain one resampled constancy index for the context/subject. We then repeated this over 1000 bootstrap iterations to obtain the bootstrapped mean index and bootstrapped confidence intervals for the context/subject. Table 1 provides the mean indices and confidence intervals for Experiment 1.

TABLE I.

Bootstrapped mean Color Constancy (CC) indices and 95% and 99% confidence intervals (CI) for Experiment 1

Subject	Contextual Object Condition	CC Index	95% CI	99% CI
S2	Absence	0.18	(0.12, 0.24)	(0.11, 0.25)
	Banana	0.31	(0.24, 0.38)	(0.21, 0.41)
	Banana-Absence	0.14	(0.04, 0.23)	(0.01, 0.26)
S3	Absence	0.2	(0.14, 0.26)	(0.13, 0.27)
	Banana	0.24	(0.15, 0.33)	(0.12, 0.36)
	Banana-Absence	0.04	(−0.07, 0.15)	(−0.09, 0.18)

For S2, adding the Banana produced an increase in mean constancy index from 0.18 to 0.31. There was a smaller increase in mean constancy index for S3, from 0.20 to 0.24. To determine whether these effects were significant, we ran a second bootstrap on the difference between indices in the two contexts. For each bootstrap iteration, we selected at random one of the bootstrapped indices from the Banana context and a second index at random from the Absence context, and took the difference. We repeated this 1000 times to obtain a bootstrapped distribution on the differences, and asked whether the 95% confidence intervals were strictly positive. The results of this procedure are also provided in Table 1. The increase in mean constancy with the addition of the Banana was significant for S2 but not for S3.

4. Experiment 2

The results of Experiment 1 indicated a small improvement in constancy with the addition of a familiar object to the reference background. The improvement was statistically significant for only one of our subjects, however. In addition, it was not clear whether the observed effect was dues to the familiarity of the banana per se, or whether it might be explained simply by the fact that a yellow object was near to the test square. To control for this possibility and to gain additional statistical power, we conducted a Experiment 2.

4.1. Methods

The design of Experiment 2 was the same as that of Experiment 1, with the exception of the following changes.

First, the 4000 K reference background was used rather than the illuminant A reference background. This was done because in Experiment 1, a small number of pixels on the banana were out of gamut when it was rendered under illuminant A, because proper rendering would have required negative RGB values. By changing to a 4000 K background and associated illuminant, we reduced the number of such pixels.^‡

Second, and perhaps most important, we added the Rectangle contextual object condition in Experiment 2.

Third, we added a response option to allow the subject to indicate that he/she had forgotten the reference color. This is because there were some large outliers in the data obtained in Experiment 1. We thought that perhaps these corresponded to cases where the subject had simply forgot the reference patch color. When subjects selected the “forgotten” response option, the affected trial was re-run later in the session.

Fourth, because there was some spatial variation across the monitor, we ran each trial type both with the reference patch presented on the left and test patch on the right (as in Experiment 1) and also left-right reversed (reference patch on the right, test patch on the left). This approach allowed us to average out any systematic effects of monitor inhomogeneity.

Fifth, we modified the smallest step size (expressed as ΔE) for hue adjustments to be inversely proportional to the current chroma. This provided more perceptually uniform hue steps across variation in chroma.

Finally, we increased the number of replications for each reference color/background/contextual object condition from 4 to 6 (3 each for the two horizontal arrangements), for a total of 216 (6*6*2*3) matches per subject.

We ran three subjects in Experiment 2. These were S2 from Experiment 1 plus two new naïve subjects (S4 and S5).

4.2. Results

Figure 7 summarizes the data from Experiment 2 in the same format as Figure 6, but with the addition of results from the Rectangle context. Table 2 provides the mean constancy indices and confidence intervals for each subject/context, obtained in the same manner as for Experiment 1. The key conclusion we draw from these data is that there is very little if any difference in constancy between the Absence, Rectangle, and Banana context. None of the small differences in constancy indices between the Banana and Absence and between the Banana and Rectangle contexts reached statistical significance for any of the subjects (bootstrapped 95% confidence intervals on differences include 0 in all cases, Table 2). This was true even for S2, who did show an increase of constancy with the addition of the Banana in Experiment 1. Overall, Experiment 2 offers no support for the hypothesis that adding a familiar object to the reference viewing conditions improves color constancy.

Figure 7.

Mean constancy for three contextual object conditions (Absence, Banana and Rectangle), Experiment 2. Same basic format as Figure 6 above. Upper left graph shows S2’s data, upper right graph shows S4’s data, and lower left graph shows S5’s data. Blue, green and red arrows indicate the mean shift in memory matches for the Absence, Banana, and Rectangle contextual object conditions respectively. To indicate the direction of the constancy prediction, the black dashed arrows show the corresponding average of the constancy predictions, at 1/5 the overall vector length.

TABLE II.

Bootstrapped mean Color Constancy (CC) indices and 95% and 99% confidence intervals (CI) for Experiment 2

Subject	Contextual Object Condition	CC Index	95% CI	99% CI
S2	Absence	0.31	(0.22,0.39)	(0.17,0.43)
	Rectangle	0.25	(0.17,0.34)	(0.14,0.36)
	Banana	0.3	(0.20,0.38)	(0.18,0.41)
	Banana-Absence	−0.01	(−0.08,0.11)	(−0.16,0.16)
	Banana-Rectangle	0.05	(−0.08,0.17)	(−0.12,0.21)
S4	Absence	0.39	(0.32,0.47)	(0.29,0.49)
	Rectangle	0.41	(0.33,0.49)	(0.30,0.53)
	Banana	0.41	(0.34,0.48)	(0.32,0.51)
	Banana-Absence	0.02	(−0.10,0.12)	(−0.10,0.15)
	Banana-Rectangle	0.01	(−0.10,0.11)	(−0.14,0.14)
S5	Absence	0.29	(0.13,0.47)	(0.09,0.51)
	Rectangle	0.19	(0.05,0.33)	(−0.01,0.39)
	Banana	0.28	(0.09,0.45)	(0.02,0.50)
	Banana-Absence	−0.01	(−0.15,0.22)	(−0.34,0.32)
	Banana-Rectangle	0.09	(−0.15,0.30)	(−0.24,0.37)

5. Experiment 3

Although the results of Experiment 1 indicated a statistically significant improvement in constancy with the addition of a familiar object to the reference viewing conditions for one subject, this effect was not replicated in Experiment 2 even for that subject. We wondered whether the lack of an overall effect of a familiar object in Experiments 1 and 2 might have been due to the fact that the large checkered background presented during the reference viewing conditions provided a sufficiently strong cue to the illuminant that it masked the effect of any incremental information provided by the addition of the banana. Experiment 3 was designed to probe this possibility.

5.1. Methods

The design of Experiment 3 was the same as that of Experiment 2, with the exception of the following changes.

First, the reference background was changed to a uniform black field. Thus the only cue to the illuminant for the reference patch was the banana/rectangle placed adjacent to it in the Banana and Rectangle contextual object conditions.^§

Second, we modified the test viewing conditions used while the subjects set their matches. Rather than presenting the test stimulus against a large checkered background, the immediate surround of the test stimulus was a uniform black background (15° by 15°). This change was made so that the contrast of the adjusted color would match that of the reference color. A checkered background surrounded the black background square to provide some degree of stabilization to the subject’s state of adaptation during the matching phase.

Third, to reduce any influence of the test background that might carry over from the matching phase of one trial to the memorization phase of the next trial, we used two choices of test background, D65 and 4000 K. These were paired with both the D65 and 4000 K contextual objects (banana, rectangle) that were present against the black reference background, leading to four reference/test pairs. Also, we added an interval of 10 seconds between the end of one match and the next memorization phase, during which a uniform black background was presented on the monitor.

Finally, we ran only one reference condition (Absence, D65 Banana, 4000 K Banana, D65 Rectangle, and 4000 K Rectangle) per reference color on any given day, to reduce experience of the color with the other context affecting the match. To match the number of the reference colors to the number of reference conditions, we used 5 reference colors that were equally around the hue circle (CIELAB hue angles (in radians) were π/10, 5π/10, 9π/10, 13π/10 and 17π/10), with lightness and chroma held constant (L* = 78 and C* = 25) across the reference colors.

We ran 5 subjects in Experiment 3. These were S2 and S3 from Experiment 1, S4 from Experiment 2, plus two new naïve subjects (S6 and S7). As in Experiment 2, there were 6 matches per reference color/reference condition/test background for a total of 300 (6*5*5*2) matches per subject.

5.2. Results

Figure 8 summarizes the data from Experiment 3 in the same format as Figures 6 and 7. In Figure 8, green and red arrows show the mean color constancy shifts for Banana and Rectangle context. The shifts in the matches were taken as the mean of the shifts measured for each of the two match illuminants (D65 and 4000 K). Table 3 provides the mean constancy indices and confidence intervals for each subject/context, again averaged across the two match illuminants. Constancy was generally smaller for this experiment than in Experiment 2. This is consistent with the fact that in Experiment 3 there was no background cue to the illuminant in the reference viewing conditions, and thus less information about the change in illuminant of the reference viewing condition was available in the stimulus. As with Experiment 2, there was no evidence for improvement in constancy with the addition of a familiar object. The relevant comparison is between the Banana and Rectangle context. Constancy was slightly better for the Rectangle for 2 of the 5 subjects and slightly better for the Banana for the remaining three subjects, but none of the differences reached statistical significance. Overall, Experiment 3 does not support the hypothesis that adding a familiar object to a scene provides a cue to the illuminant.

Figure 8.

Mean constancy for Banana and Rectangle contextual object conditions, Experiment 3. Same basic format as Figure 6 and Figure 7 above. Upper left graph shows S2’s data, upper right graph shows S3’s data, middle left graph shows S4’s data, middle right graph shows S6’s data, and lower left graph shows S7’s data. Green and red arrows indicate the mean color constancy shift for the Banana and Rectangle contextual object conditions respectively. To indicate the direction of the constancy prediction, the black dashed arrows show the corresponding average of the constancy predictions, at 1/5 the overall vector length.

TABLE III.

Bootstrapped mean Color Constancy (CC) indices and 95% and 99% confidence intervals (CI) for Experiment 3

Subject	Contextual Object Condition	CC Index	95% CI	99% CI
S2	Rectangle	0.07	(−0.01,0.16)	(−0.04,0.18)
	Banana	0.08	(0.01,0.16)	(−0.02,0.19)
	Banana-Rectangle	0.01	(−0.10,0.13)	(−0.13,0.19)
S3	Rectangle	0.05	(−0.04,0.14)	(−0.07,0.17)
	Banana	−0.01	(−0.06,0.05)	(−0.09,0.07)
	Banana-Rectangle	−0.06	(−0.17,0.05)	(−0.20,0.08)
S4	Rectangle	0.07	(−0.03,0.19)	(−0.06,0.22)
	Banana	0.16	(0.06,0.27)	(0.03,0.29)
	Banana-Rectangle	0.09	(−0.06,0.25)	(−0.10,0.28)
S6	Rectangle	0.07	(−0.14,0.24)	(−0.22,0.30)
	Banana	0.13	(0.02,0.25)	(−0.00,0.29)
	Banana-Rectangle	0.06	(−0.14,0.29)	(−0.19,0.33)
S7	Rectangle	0.18	(0.05,0.31)	(0.01,0.36)
	Banana	0.1	(−0.04,0.22)	(−0.08,0.26)
	Banana-Rectangle	−0.08	(−0.26,0.11)	(−0.33,0.18)

6. Discussion

The results of Experiment 1 indicated a small improvement in constancy with the addition of a familiar object to the reference viewing conditions, and this improvement reached statistical significance for one of the two subjects. The effect was not replicated in Experiments 2 and 3, however. Overall, our experiments do not provide support for the hypothesis that adding a familiar object to the viewing context leads to improved color constancy.

Since we obtained a null result, it is worth asking how much power our experiments had. That is, given our measurement variability, how large an effect could there be before we would expect to see it reliably? To get a sense for this, we then analyzed power of our measurements. We did this for the Banana/Rectangle context difference studied in Experiments 2 and 3, as this comparison is the one that best controls low-level stimulus factors across the familiar object manipulation. We used a bootstrap procedure, this time computing the mean difference in constancy between the Banana and Rectangle context, over subjects, within each experiment. In Experiment 2, this mean bootstrapped difference in index was 0.05 with a 95% confidence interval of (−0.04, 0.16). To reach statistical significance, in the sense that the bootstrapped 95% confidence interval would be expected to be strictly positive, the mean difference in index would have had to be 0.09. For Experiment 3, the corresponding values were 0.00 (−0.09, 0.10). Thus Experiment 3 would be expected to have detected a significant difference in index had this difference again been 0.09. Given these analyses, our experiments put an upper bound on the effect of a familiar object on constancy for our conditions of about 0.09.

A second consideration is whether the variability of chromaticity across the banana might have been sufficiently large so as to mask the effect of the illuminant change. To examine this possibility we computed the a*b* chromaticities of the pixels on the banana under the three illuminants used in the experiments (D65, CIE A, 4000 K). We used a common reference white for these transformations, so that the result shows the effect of the illuminant on the stimulus without confound introduced by the adaptation model implemented in the CIELAB transformation. The results are shown in Figure 9. It is clear that the distributions of banana pixel chromaticities across the three illuminants are quite distinct, suggesting that the banana images carried sufficient information to distinguish the illuminants.

Figure 9.

Color distribution of banana images used in the experiments. CIELAB a*b* chromaticity distribution of a banana image under D65 illuminant are plotted in black x’s. Those under 4000 K illuminant are plotted in red circles, and those under illuminant A are plotted in green triangles. In all cases, D65 was used as the reference white point to transform to CIELAB color space.

We did find a significant effect of the banana in Experiment 1 for one subject (S2). Although the control context (Absence) used in that experiment is not as theoretically sound a choice as the Rectangle condition of Experiments 2 and 3, the fact that the comparison was made to the Absence context seems unlikely to represent a major difference between experiments, since the Banana/Absence difference was not significant for any subjects (including S2) in Experiment 2. Nor is the fact that the illuminant change used in Experiment 1 was larger than that used in Experiment 2 seem likely to be crucial, since our critical measure was a constancy index that takes the size of the illuminant change into account. Overall, the constancy indices from Experiment 2 are slightly larger than those from Experiment 1, but not near ceiling.

As noted in the introduction, both Ling¹⁵ and Granzier and Gegenfurtner¹⁷ have reported results of experiments on the effects of familiar objects on color constancy, using experimental logic similar to ours. Ling found direct effects of object familiarity on the constancy of the object itself (Chapter 6, Experiments 1 and 2 interpreted together). But in the experiment (Chapter 6, Experiment 1) that probed the indirect effect of a familiar object in the scene (a banana) on the constancy of another familiar object (a banana) she found no effect, consistent with our results.

In contrast, Granzier and Gegenfurtner¹⁷ conclude that adding familiar objects to a scene provides a small improvement in constancy relative to control conditions consisting of colored geometrical shapes or objects that did not have characteristic colors. There are a number of possible reasons for the difference. First, their measured effects were small (e.g., 0.08 change in index in their Experiment 1, 0.11 change in index in their Experiment 3, and 0.04 change in index in their Experiment 4) and variable between subjects (e.g., in their Experiment 1, some subjects showed a considerable increase in constancy and others none at all.) The size of their reported effects is small enough that our experiment might not have detected them, given our experimental power. It is also possible that if we had run more subjects, we would have encountered subjects who showed a familiar object effect.

There are also a number of specific differences in experimental design and detail, any one of which might matter. These include the facts that Granzier and Gegenfurtner employed a simultaneous matching procedure using real three-dimensional objects while we employed computer simulations of spatially simple two-dimensional surfaces, that they had several familiar objects in each context while we had one, that they had subjects choose matches from a palette and emphasized matching “the surface” (Granzier and Gegenfurtner¹⁷, p. 198) while we had subjects adjust the less specific “color” of a displayed match, and that their familiar objects remained visible throughout the matching procedure whereas our familiar object was only present while subjects learned the appearance of the reference color.^** Finally, the specific reference colors and illuminants were different in their experiments than in ours. That this may matter is indicated by the fact that in one of their experiments (Experiment 1) the familiar object effect on color constancy was observed for only one of their two illuminant changes.

Of these differences, differences in naturalness of stimuli and instructions are perhaps the most noteworthy. In Experiments 1 and 2, our stimuli/instructions led to overall constancy indices that are quite low, but which consistent with other experiments in the literature that use similar instructions and relatively simple stimuli presented on computer displays (see Foster²⁰, Table 1.) In Experiment 3, where we removed all but the contextual object cue to the illuminant in an attempt to maximize any difference mediated by familiarity, our constancy indices were not surprisingly even lower. Granzier and Gegenfurtner’s stimuli/instructions led to higher constancy indices, again consistent with the literature.¹⁷ The factors that modulate the degree of constancy across changes in stimuli/instructions are not fully understood,^29–44 but it is certainly possible that conditions that lead to higher measured constancy do so because they encourage subjects to reason explicitly about their responses. If such is the case, it might lead to a difference in the degree to which the color of a displayed familiar object affects the data. This conjecture echoes the conclusion drawn by Bolles et al. concerning the nature of how familiarity with an object can affect its experimentally reported color.⁵ It is also consistent with speculation by Ling¹⁵ (Chapter 6, p. 119) about a possible reason her experiment may not have revealed an effect of a familiar object on the constancy of other objects in the scene. Of note, however, is that Ling’s subjects exhibited higher constancy indices than either ours or those of Granzier and Gegenfurtner.

Finally, it is possible that Granzier and Gegenfurtner’s¹⁷ positive conclusion rests on statistical fluctuations around an underlying null effect. Although their effects are significant by standard tests, it is also the case that effects that are significant by these tests in some of their experiments are not significant in others (e.g., the effect of incongruent familiar object colors relative to congruent familiar colors between their Experiments 1 and 3).

Taken together, our results as well as the work of Ling¹⁵, Granzier and Gegenfurtner¹⁷ indicate that there is at most little effect on the color appearance from the presence of a familiar object in the scene. In addition, to the extent that there are small effects, the exact factors that support them remain to be more precisely elucidated.